FileNotFoundException resource files is slowing things down ridiculously

[dgxhubbard]

In .net core 3.1, on wpf we are getting a ridiculous number of missing resource file FileNotFoundException. I have read this is a part of normal processing. If the resource file
cannot be found and it will be searched for somewhere else why throw an exception which
has a huge processing time, why not return a result that says not found. We are getting these
exceptions constantly yet we still see our resource strings. Below is an example. The worst part is the same missing resource file FileNotFoundException get thrown multiple. The delays in program startup are very noticeable. If there is something that can be done to fix this I would appreciate it.

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Configuration.ConfigurationManager.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Configuration.ConfigurationManager.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Data.SqlClient.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Private.DataContractSerialization.resources.dll
Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Private.DataContractSerialization.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.ComponentModel.Composition.resources.dll
Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.ComponentModel.Composition.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Data.SqlClient.resources.dll
Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Data.SqlClient.resources.dll

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. Please help me learn by adding exactly one area label.

[danmoseley]

@dgxhubbard can you please provide the callstack for the exception? Attach a debugger, if it's Visual Studio please swithc of "just my code" and load symbols.

[ghost]

Tagging subscribers to this area: @tarekgh, @buyaa-n, @krwq
Notify danmosemsft if you want to be subscribed.

[tarekgh]

@dgxhubbard do you have a small sample to reproduce the issue so we can take a look? also, the stack trace will help too.

[tarekgh]

CC @ericstj

[danmoseley]

I am not sure where "Failed to resolve" comes from. I cannot find it in runtime/wpf/corefx/coreclr.

[tarekgh]

I think this properly a wpf issue but wanted to get repro first to confirm. @danmosemsft does it repro with any simple wpf app?

[dgxhubbard]

I have the debugger running. I will stop on the exception and get you a stack trace

[dgxhubbard]

We are running 3.1.3 and when we load our assemblies for MEF using code below, and with "Just My Code" disabled we get a FileNotFound exception even though the path and assembly name are correct. At that point we added an assembly resolver to try and clear up the problem. This is where the errors come from. We get System.Private.DataContractSerialization.resources, Version=4.1.5.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a as the argument name and then add "dll" to it to try and resolve. Which causes an exception when LoadFrom is called. If I turn on "Enable Just My Code" and disable the assembly resolver then all problems with loading our assemblies for MEF using code below disappear. I had forgotten about the problem with creating the assembly catalog.

Stack Trace

at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)

Assembly Resolver:

    public BootstrapperBase ()
    {
        AppDomain.CurrentDomain.AssemblyResolve += CurrentDomain_AssemblyResolve;
    }

    private Assembly CurrentDomain_AssemblyResolve ( object sender, ResolveEventArgs args )
    {
        Assembly assembly = null;

        var name = args.Name;

        var pos = name.IndexOf ( "," );
        if ( pos != -1 )
        {
            name = name.Substring ( 0, pos );
        }

        name += ".dll";

        var fullExeNameAndPath = Assembly.GetEntryAssembly ().Location;
        var path = Path.GetDirectoryName ( fullExeNameAndPath );

        var filepath = Path.Combine ( path, name );

        try
        {
            assembly = Assembly.LoadFrom ( filepath );
        }
        catch ( Exception ex )
        {
            Debug.WriteLine ( "Failed to resolve " + filepath, ex );
            LogProvider.Logger.LogException ( "Failed to resolve " + filepath, ex );
        }

        return assembly;
    }

MEF Code to create assembly catalog:

`public static AssemblyCatalog GetCatalog( string assemblyName )
{
AssemblyCatalog assemblyCatalog = null;

        var assembly = System.Reflection.Assembly.GetExecutingAssembly();
        var codebase = assembly.GetName().CodeBase;

        var path = codebase.Replace( @"file:///", string.Empty );
        var basePath = System.IO.Path.GetDirectoryName( path );
        var assemblyPath = Path.Combine( basePath, assemblyName );

        try
        {
            assemblyCatalog = new AssemblyCatalog( assemblyPath );
        }
        catch ( Exception ex )
        {
            LogProvider.Logger.LogException( "Failed to load assembly catalog " + assemblyPath, ex );
        }

        return assemblyCatalog;
    }`

[ericstj]

This is the source of your exception:

        var filepath = Path.Combine ( path, name );

        try
        {
            assembly = Assembly.LoadFrom ( filepath );
        }
        catch ( Exception ex )
        {
            Debug.WriteLine ( "Failed to resolve " + filepath, ex );
            LogProvider.Logger.LogException ( "Failed to resolve " + filepath, ex );
        }

Perhaps you should do an exists check before calling Assembly.LoadFrom. Return null (fall through) if the file doesn't exist.

[dgxhubbard]

I would think the point of this is why is new AssemblyCatalog failing for something that exists. I admit that got buried because of the problem with exception.

[ghost]

Tagging subscribers to this area: @ViktorHofer
Notify danmosemsft if you want to be subscribed.

[tarekgh]

AssemblyCatelog at the end is calling Assembly.Load and Assembly.LoadFrom. Maybe the assembly trying to load is not listed in deps.json of the app which makes the loader not loading it? just a guess.

[ericstj]

Since this is not a regression it's unlikely to make it in 5.0.0 due to the bar. Let's consider for 6.0.

[cloudlene]

We are also seeing this issue in the Microsoft Test Platform. There's no such file System.Private.DataContractSerialization.resources.dll ANYWHERE on the system. Leave alone version 4.1.5.0 which also appears not to have been published on Nuget (I'm assuming resources file usually shares the version of the main assembly). This actually breaks the Test Platform (error intermittent and causes Azure DevOps agent to hang and the pipeline never completes. Is there a workaround?

BTW, we are using .Net Core 3.1.

TpTrace Error: 0 : 9328, 9, 2021/04/07, 12:34:13.303, 6021418487030, testhost.dll, LengthPrefixCommunicationChannel: MessageReceived: Exception occurred while calling handler of type Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestHandler for MessageReceivedEventArgs: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> Newtonsoft.Json.JsonSerializationException: Error setting value to 'StoreKeyValuePairs' on 'Microsoft.VisualStudio.TestPlatform.ObjectModel.TestCase'.
---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Private.DataContractSerialization.resources, Version=4.1.5.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Private.DataContractSerialization.resources, Version=4.1.5.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'
---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Private.DataContractSerialization.resources, Version=4.1.5.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Private.DataContractSerialization.resources, Version=4.1.5.0, Culture=en-US, PublicKeyToken=b03f5f7f11d50a3a'

[ericstj]

Do you have an AssemblyResolve handler involved like the original report? That was the reason for the FileNotFound exceptions. Whenever culture is set to something that isn't the "neutral" culture the runtime probes for resource assemblies. It's careful to not raise any exceptions when doing this probing

runtime/src/coreclr/System.Private.CoreLib/src/System/Resources/ManifestBasedResourceGroveler.CoreCLR.cs

Lines 12 to 17 in 2e12730

	private static Assembly? InternalGetSatelliteAssembly(Assembly mainAssembly,
	CultureInfo culture,
	Version? version)
	{
	return ((RuntimeAssembly)mainAssembly).InternalGetSatelliteAssembly(culture, version, throwOnFileNotFound: false);
	}

The same was true in 3.1. The problem comes when others hook AssemblyResolve. If their handler attempts it's own load it can trigger the FileNotFoundException. Check the stack trace for those exceptions to find the problematic AssemblyResolve handler. You should be able to do this by enabling break on first-chance-exceptions for FileNotFoundException and examining the stack trace. Let us know if you find some framework assembly at fault, or even if it's the test runner and we can help route.

[reijerh]

I'm getting this with a net6.0-windows WPF project on a CurrentCulture=nl-NL system, when calling
Application.LoadComponent(relativeUri) with a relative Uri that points to a .xaml ResourceDictionary in another assembly that was loaded at runtime using Assembly.LoadFrom(path).

Workaround is to specify <NeutralLanguage>en-US</NeutralLanguage> in a Directory.Build.props. I guess [assembly: NeutralResourcesLanguage("en-US")] would work as well.

Here is the stacktrace (@ericstj it seems the AssemblyResolve handler in this case is from the CoreLib itself: System.Reflection.Assembly.LoadFromResolveHandler. Registered as follows in Assembly.LoadFrom():
AssemblyLoadContext.AssemblyResolve += LoadFromResolveHandler!;):

[Managed to Native Transition]	
System.Private.CoreLib.dll!System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(System.IntPtr ptrNativeAssemblyLoadContext, string ilPath, string niPath, System.Runtime.CompilerServices.ObjectHandleOnStack retAssembly)	Unknown
System.Private.CoreLib.dll!System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(string assemblyPath) Line 346	C#
System.Private.CoreLib.dll!System.Reflection.Assembly.LoadFrom(string assemblyFile) Line 365	C#
System.Private.CoreLib.dll!System.Reflection.Assembly.LoadFromResolveHandler(object sender, System.ResolveEventArgs args) Line 328	C#
System.Private.CoreLib.dll!System.Runtime.Loader.AssemblyLoadContext.InvokeResolveEvent(System.ResolveEventHandler eventHandler, System.Reflection.RuntimeAssembly assembly, string name) Line 735	C#
System.Private.CoreLib.dll!System.Runtime.Loader.AssemblyLoadContext.OnAssemblyResolve(System.Reflection.RuntimeAssembly assembly, string assemblyFullName) Line 723	C#
[Native to Managed Transition]	
[Managed to Native Transition]	
System.Private.CoreLib.dll!System.Reflection.RuntimeAssembly.InternalLoad(System.Runtime.CompilerServices.ObjectHandleOnStack assemblyName, System.Runtime.CompilerServices.ObjectHandleOnStack requestingAssembly, System.Runtime.CompilerServices.StackCrawlMarkHandle stackMark, bool throwOnFileNotFound, System.Runtime.CompilerServices.ObjectHandleOnStack assemblyLoadContext, System.Runtime.CompilerServices.ObjectHandleOnStack retAssembly)	Unknown
System.Private.CoreLib.dll!System.Reflection.RuntimeAssembly.InternalLoad(System.Reflection.AssemblyName assemblyName, System.Reflection.RuntimeAssembly requestingAssembly, ref System.Threading.StackCrawlMark stackMark, bool throwOnFileNotFound, System.Runtime.Loader.AssemblyLoadContext assemblyLoadContext) Line 347	C#
System.Private.CoreLib.dll!System.Reflection.RuntimeAssembly.InternalGetSatelliteAssembly(System.Globalization.CultureInfo culture, System.Version version, bool throwOnFileNotFound) Line 598	C#
System.Private.CoreLib.dll!System.Resources.ManifestBasedResourceGroveler.InternalGetSatelliteAssembly(System.Reflection.Assembly mainAssembly, System.Globalization.CultureInfo culture, System.Version version) Line 16	C#
System.Private.CoreLib.dll!System.Resources.ManifestBasedResourceGroveler.GetSatelliteAssembly(System.Globalization.CultureInfo lookForCulture) Line 387	C#
System.Private.CoreLib.dll!System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(System.Globalization.CultureInfo culture, System.Collections.Generic.Dictionary<string, System.Resources.ResourceSet> localResourceSets, bool tryParents, bool createIfNotExists) Line 68	C#
System.Private.CoreLib.dll!System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo culture, bool createIfNotExists, bool tryParents) Line 468	C#
System.Private.CoreLib.dll!System.Resources.ResourceManager.GetObject(string name, System.Globalization.CultureInfo culture, bool wrapUnmanagedMemStream) Line 700	C#
System.Private.CoreLib.dll!System.Resources.ResourceManager.GetStream(string name, System.Globalization.CultureInfo culture) Line 739	C#
PresentationCore.dll!MS.Internal.Resources.ResourceManagerWrapper.GetStream(string name) Line 46	C#
PresentationFramework.dll!MS.Internal.AppModel.ResourcePart.EnsureResourceLocationSet() Line 147	C#
PresentationFramework.dll!MS.Internal.AppModel.ResourcePart.GetContentTypeCore() Line 105	C#
System.IO.Packaging.dll!System.IO.Packaging.PackagePart.ContentType.get() Line 165	C#
PresentationFramework.dll!System.Windows.Application.LoadComponent(System.Uri resourceLocator, bool bSkipJournaledProperties) Line 510	C#
PresentationFramework.dll!System.Windows.Application.LoadComponent(System.Uri resourceLocator) Line 487	C#

I suppose the handler is registered when I call Assembly.LoadFrom(), and sticks around because the handler is never unregistered.

Caused by:
0284966 for dotnet/coreclr#11333 for #21286

[ericstj]

I believe the fix here is for LoadFromResolveHandler to implement my suggestion from #36053 (comment)

Perhaps you should do an exists check before calling Assembly.LoadFrom. Return null (fall through) if the file doesn't exist.

So the fix here is to add an exists check here (and we might be able to delete that catch FNF 😆)

runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/Assembly.cs

Lines 323 to 339 in 9a0b8f3

	string requestedAssemblyPath = Path.Combine(Path.GetDirectoryName(requestorPath)!, requestedAssemblyName.Name + ".dll");
	#if CORECLR
	if (AssemblyLoadContext.IsTracingEnabled())
	{
	AssemblyLoadContext.TraceAssemblyLoadFromResolveHandlerInvoked(args.Name, true, requestorPath, requestedAssemblyPath);
	}
	#endif // CORECLR
	try
	{
	// Load the dependency via LoadFrom so that it goes through the same path of being in the LoadFrom list.
	return Assembly.LoadFrom(requestedAssemblyPath);
	}
	catch (FileNotFoundException)
	{
	// Catch FileNotFoundException when attempting to resolve assemblies via this handler to account for missing assemblies.
	return null;
	}

Looks like there are other exists checks in this neighborhood so it should be a reasonable change to make.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-reflection
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In .net core 3.1, on wpf we are getting a ridiculous number of missing resource file FileNotFoundException. I have read this is a part of normal processing. If the resource file
cannot be found and it will be searched for somewhere else why throw an exception which
has a huge processing time, why not return a result that says not found. We are getting these
exceptions constantly yet we still see our resource strings. Below is an example. The worst part is the same missing resource file FileNotFoundException get thrown multiple. The delays in program startup are very noticeable. If there is something that can be done to fix this I would appreciate it.

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Configuration.ConfigurationManager.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Configuration.ConfigurationManager.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Data.SqlClient.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Private.DataContractSerialization.resources.dll
Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Private.DataContractSerialization.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.ComponentModel.Composition.resources.dll
Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.ComponentModel.Composition.resources.dll

Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Data.SqlClient.resources.dll
Exception thrown: 'System.IO.FileNotFoundException' in System.Private.CoreLib.dll
Failed to resolve C:\Repository\GtNext\bin\Debug\System.Data.SqlClient.resources.dll

Author:	dgxhubbard
Assignees:	-
Labels:	area-System.Reflection, tenet-performance
Milestone:	7.0.0

[ericstj]

@dotnet/area-system-reflection this is a good candidate for 7.0.0

[steveharter]

So the fix here is to add an exists check here (and we might be able to delete that catch FNF 😆)

That seems like a reasonable approach however it is somewhat risky to do at this point in the release cycle. It should be done earlier.

Also adding a File.Exists may have to take into consideration embedded assemblies, as exists with browser\wasm scenarios.

[reijerh]

And even if you check for existence, you still need to catch the FNF exception because the file might not be there anymore.

[ericstj]

For others landing on this issue - the most common cause of this problem is an AssemblyResolve handler which doesn't do an exists check before doing a loadfrom - those get invoked as part of the resource assembly resolution process (which itself tries to avoid raising exceptions).

This issue tracks fixing this for Assembly.LoadFromResolveHandler. I see @tarekgh had some similar feedback in a related fix dotnet/coreclr#12329. @reijerh shared a great RCA in #36053 (comment). @jkotas what do you think about adding an exists check to LoadFromResolveHandler to avoid these first-chance exceptions?

[jkotas]

@jkotas what do you think about adding an exists check to LoadFromResolveHandler to avoid these first-chance exceptions?

Sounds good to me.

[fowl2]

exists check before doing a loadfrom

Exists() before Open() is an anti-pattern - it leads to TOCTOU races and wastes syscalls.

This should block on #27217