C# - Dynamic Loading Working Fine, Except After The Executable Is ILMerged

I have a windows application that dynamically loads DLLs using Reflection.Assembly.LoadFrom(dll_file_name_here).

It works as expected, until I ILMerge the application with another DLL.

So this scenario works fine:

MyApp.exe
MyAppComponent.dll
Plugin.dll

Once I ILMerge MyApp.exe and MyAppComponent.dll resulting in:

MyApp.exe
Plugin.dll

Calling Reflection.Assembly.LoadFrom("Plugin.dll") seems to load successfully, however once I try to do anything with it eg:

foreach ( typeAsm in Reflection.Assembly.LoadFrom("Plugin.dll"))

I get an exception "unable to load one or more of the requested types. retrieve the loader exceptions property for more informtion".

The frustrating thing is I can't really debug it, because debugging pre merging works perfectly!