AssemblyLoadContext does not provide proper isolation of types

[groogiam]

I am running into an issue where I am trying to port some logic which creates dynamic assemblies from framework to core. In this particular scenario the code that creates the dynamic assembly is in a nuget package that use CodeDomCompiler. The assembly is loaded from CompilerResult.CompiledAssembly which appears to always use the default context. This also happens with typeof comparisons inside of the library that is being dynamically loaded. What I'm seeing is that the AssemblyLoadContext seems to have some shortcomings in terms of control of how it can be used to isolate and unload dynamic code in libraries. The AppDomain concept supported these scenarios because all the types load in the domain where internally consistent. Is there any way to mimic the isolation of an AppDomain for libraries where a developer does not directly control the dynamic assembly load? If not is there any plans to perhaps support this in the future. Thanks.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke
See info in area-owners.md if you want to be subscribed.

[vitek-karas]

The problem is this line:

runtime/src/libraries/System.CodeDom/src/System/CodeDom/Compiler/CompilerResults.cs

Line 26 in 731eaf3

_compiledAssembly = Assembly.LoadFile(PathToAssembly);

Assembly.LoadFile will

• Create a new ALC for the each unique file path and load the assembly into it
• The new ALC will always fallback to Default ALC for any other assembly resolution

So the end result is that it looks almost like loading it into Default - it gets all its dependencies from Default.

In the world of ALCs this compiler API should ideally provide a way to specify which ALC to load the file into. Or at the very least react to https://docs.microsoft.com/en-us/dotnet/api/system.runtime.loader.assemblyloadcontext.currentcontextualreflectioncontext?view=netcore-3.1

As for a workaround - if you can modify the code which calls the CompilerResult.CompiledAssembly you could instead of called that use the CompilerResult.PathToAssembly and load it yourself whichever way you want. If you don't control that code in any way - I'm afraid there's no workaround - I really can't think of anything :sad:.

This is kind of related to #29842 as it's a similar type of problem (which doesn't have a workaround at least via the ContextualReflectionContext)

[groogiam]

I have modified the code to pass in an assembly load context but I still am getting type checks that fail. I have the entire default context into the new context but performing == on typeof still seems to fail. Shouldn't this be using the types from the context of the caller before the default?

[vitek-karas]

@groogiam I'm sorry but I don't think I understand the problem you're describing.
ALCs are only used during assembly resolution - which basically means assembly loading. They decide which assembly will be loaded and from where. Once assemblies are loaded ALCs are basically just buckets holding the assemblies but they're not used during execution of the code in any way.

So for example typeof(MyType) in itself has no ALC interaction. What matters is which ALC the assembly which has this code in it is loaded into - and then when the MyType is loaded and causes to load its assembly MyAssembly - how does that ALC respond to the request of loading MyAssembly.

It's pretty hard to answer what's wrong without knowing a bit more about how your ALCs are setup and the code in question.

One way to debug this is to print out which ALC the type/assembly is from. This can be done by calling AssemblyLoadContext.GetLoadContext(myAssembly) or AssemblyLoadContext.GetLoacContext(myType.Assembly). This should obviously work on any System.Type or System.Reflection.Assembly instance - so you can also use it on the result of typeof and so on.

[OlegJakushkin]

@vitek-karas: may be I have a similar issue and you could help me:

Say we want to implement the V from here:

corner right image where Json.Net is already loaded into Default ALC, yet we want to load an assembly that refrences Json.Net, yet while its referenced Json.Net version and name are same we want to use ALC to load it from another path. Currently when we load Addon Default resolves Json.Net and does not give us a chance to load it from the path we want. How would one do such thing?

[vitek-karas]

There's a guide how to write plugins/hosts here: https://learn.microsoft.com/en-us/dotnet/core/tutorials/creating-app-with-plugin-support

You have two choices on where/how to determine isolation:

• The build of the plugin - if the plugin doesn't carry a specific assembly with itself, then it can't be isolated (the plugin's version doesn't exist effectively) and so the host needs to resolve it (typically from the Default). Usually if the plugin does carry a dependency with itself, then the host should isolate it (load it into plugin's ALC). This is what the guide and associated sample above implement.
• The host has final say - even if the plugin does contain the dependency, the host may decide to not use it - this would be done in the AssemblyLoadContext.Load override. The implementation shown in the sample doesn't do that, it fully relies on AssemblyDependencyResolver which will be driven by what was included with the plugin. But it's a possibility if you have wrongly built plugins (hopefully not though). That part is shown specifically here: https://learn.microsoft.com/en-us/dotnet/core/tutorials/creating-app-with-plugin-support#reference-a-plugin-interface-from-a-nuget-package

Note that the plugin project file needs to be aware of this. It needs to use the correct properties to mark dependencies it wants from the host (and thus those won't be copied to the build output of the plugin). Typically that should be anything from the core framework and the interface assembly which defines the interface with the host.

[OlegJakushkin]

Dear @vitek-karas: Yes, I am aware of general use pattern of AssemblyLoadContext, yet now please imagine the following situation:

• We are inside an BaseApp that relies on Json.Net - it is already loaded into our AppDomain
• We want to load a Plugin with a specific Json.Net implementation (that happens to share name and version with our BaseApp Json.Net yet is in another folder and has a set of changes made to it.

The problem is AssemblyLoadContext.Load overload only asks once for the assembly (on Plugin load) and then uses ones from default AppDomain including Json.Net implementation.

One may expect that if you load the desired modified Json.Net first and than load Plugin AssemblyLoadContext would use the modified version. Yet it does not happen.

[vitek-karas]

I must admit I'm a bit confused, but let me describe how it "should" work:

• Create a new ALC for the new plugin
• Implement the override of AssemblyLoadContext.Load which will resolve Newtonsoft.Json from the plugin's directory by calling this.LoadFromAssemblyPath and returning that.
• Load the plugin into the ALC - alc.LoadFromAssemblyPath(pathToPlugin)
• Through reflection get an instance of the plugin and call it
• At some point the AssemblyLoadContext.Load override on the plugin's ALC will be called with Newtonsoft.Json assembly name. As per the implementation above, this should resolve by loading the assembly from disk.
• From this point on, all references from the plugin's assembly to Newtonsoft.Json should be resolved to the assembly loaded in the above step.

It you need to diagnose this, you can try this: https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/collect-details
It takes a bit of practice to look through the traces, but it should all of the information to figure out exactly what is happening. You can even combine it with live-debugging the project to see what happens inside the Load override.

[OlegJakushkin]

Dear @vitek-karas: created a small prototype to illustrate my point here (compilable and runable)

In it I:

1. Create a new ALC for the new plugin here
2. Implement the override of AssemblyLoadContext.Load which will resolve dependencies from the plugin's directory. here
3. Load the plugin into the ALC - alc.LoadFromAssemblyName(typeof(TargetUser).Assembly.GetName()) here
4. Through reflection get an instance of the plugin and call it here
5. At some point the AssemblyLoadContext.Load override on the plugin's ALC shall be called with dependancy assembly name. Yet it does not.

[vitek-karas]

The problem is here. This calls LoadFromAssemblyName which will call the Load overload. But in this case the overload return null, so the call falls back to the Default load context which resolves it (to the already loaded version anyway). Any future calls to types from that assembly will be done in the Default context and thus won't hit the Load override in the custom ALC.

I think the confusion is that LoadFromAssemblyName is not guaranteed to return an assembly from the target ALC - it's basically equivalent to "Resolve this assembly name in the context of this ALC". For example you could call alc.LoadFromAssemblyName(typeof(object).Assembly.Name) and that should always return the same System.Private.CoreLib no matter which alc you call it on (as CoreLib can't every be loaded more than once).

Unlike LoadFromAssemblyPath which in fact guarantees that it returns assembly from the target context (it either loads a new one, or returns an existing one).

The resolution of an assembly reference is purely based on the ALC into which the calling assembly belongs - in this case you start with assembly in Default, and thus the reference resolution happens in the Default context.

[OlegJakushkin]

Dear @vitek-karas: even using LoadFromAssemblyPath and having both

1. Main library I want to use directly (placed into a new .dll)
2. and a Dependency library (referenced from Main) (placed into a new .dll)

AssemblyLoadContext only loads Main library I want to use directly and explicitly (with Load call), yet all subsequent dependencies such as a library I want it to reference do not get to have a Load call.

Even if I modify ALC Load with so that it always loads not null and even loads from File:

    protected override Assembly Load(AssemblyName assemblyName)
    {
        string path = "";
        if (InsertionFactory.AssemblyNameToPath.TryGetValue(assemblyName.FullName, out path))
        {
            return Assembly.LoadFile(path);
        }
        var result = Assembly.LoadFile(Assembly.Load(assemblyName).Location);
        return result;
    }

Still I only get ALC call for Main and no calls for Dependency(es)... Is there any workaround that could allow force resolution of all the dependencies using ALC?

[vitek-karas]

Sorry - I missed one additional problem. Please don't call Assembly.LoadFile it will load into its own ALC. This is somewhat documented, but probably not super clear. Probably the best doc about the various assembly loading APIs is here: https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/loading-managed#when-are-managed-assemblies-loaded (in terms of their interaction with ALCs).

If you replace Assembly.LoadFile with this.LoadFromAssemblyPath it will finally start loading the assemblies into your ALC. You can verify this by calling AssemblyLoadContext.GetLoadContext(assembly) which will tell you which ALC any given assembly belongs to.

Note that the code as above will not work either - it will actually load the Influence assembly separately into the plugin's ALC as well and thus the "counting" will happen there and not where the test sees it (two copies of the Influence assembly in the process).

This is generally why I would recommend that you build the plugin into a separate location (as shown by our sample) and load it from there using the AssemblyDependencyResolver. You will avoid problems with having to handle assembly paths and so on yourself.