System.Reflection.MetadataLoadContext performance

[jonathanpeppers]

I have a benchmark here: https://github.com/jonathanpeppers/Benchmarks/blob/7db49fb3d272c5b07deda166dd4f5a5112258bbe/Benchmarks/Cecil.cs#L90-L111

And I am getting "not so great" results for SR.MetadataLoadContext:

// * Summary *

BenchmarkDotNet=v0.11.3, OS=Windows 10.0.18362
Intel Core i9-9900K CPU 3.60GHz, 1 CPU, 16 logical and 8 physical cores
  [Host]     : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 32bit LegacyJIT-v4.8.4010.0
  DefaultJob : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 32bit LegacyJIT-v4.8.4010.0


                                Method |        Mean |      Error |     StdDev |      Median | Gen 0/1k Op | Gen 1/1k Op | Gen 2/1k Op | Allocated Memory/Op |
-------------------------------------- |------------:|-----------:|-----------:|------------:|------------:|------------:|------------:|--------------------:|
            System.Reflection.Metadata |    32.84 ms |  0.1200 ms |  0.1123 ms |    32.85 ms |   3000.0000 |     62.5000 |           - |            15.12 MB |
                            Mono.Cecil |   351.81 ms |  7.1738 ms | 21.1522 ms |   339.38 ms |  16000.0000 |  12000.0000 |   4000.0000 |           103.57 MB |
 System.Reflection.MetadataLoadContext | 2,058.91 ms | 19.9863 ms | 18.6952 ms | 2,051.50 ms |  97000.0000 |  90000.0000 |   6000.0000 |           544.84 MB |

The performance compared to using raw SRM or Mono.Cecil is drastically worse.

I expected it to be somewhere in the middle of using SRM and Mono.Cecil. Is there something I'm doing here that would explain the poor performance?

Thanks!

[steveharter]

Moving to future due to scheduling + priority.

[steveharter]

Moving to 9; we should at least perform some traces to see where the perf hit is; MLC does have extensive caching but perhaps there are temporary allocs that are being done based on the alloc numbers above.

[jonathanpeppers]

If anyone looks into this in the future, I think it's specifically iterating over methods that is slow:

• https://github.com/jonathanpeppers/Benchmarks/blob/7db49fb3d272c5b07deda166dd4f5a5112258bbe/Benchmarks/Cecil.cs#L105-L107

Someone else did a comparison, and MLC performs somewhere in between Mono.Cecil and SRM if you remove these lines in all benchmarks.

[jpobst]

Indeed, resolving assemblies and iterating types has excellent performance, but digging any deeper into a Type tanks performance (GetNestedTypes (), GetConstructors (), GetMethods (), etc.)

Modifying @jonathanpeppers test case to simply iterate type names and nothing else shows performance very close to System.Relection.Metadata, however iterating type names and nested type names performs ~100x worse:

Method	Mean	Error	StdDev	Gen0	Gen1	Gen2	Allocated
S.R.M - Types	5.026 ms	0.0978 ms	0.2418 ms	39.0625	-	-	1.11 MB
S.R.M - Types+Nested	5.662 ms	0.1092 ms	0.1531 ms	70.3125	-	-	1.81 MB
S.R.MLC - Types	7.273 ms	0.1367 ms	0.2358 ms	140.6250	125.0000	-	3.38 MB
S.R.MLC - Types+Nested	674.997 ms	13.3861 ms	14.3229 ms	16000.0000	15000.0000	3000.0000	324.81 MB

Benchmark code:

[Benchmark (Description = "MetadataLoadContext - Types")]
public void SystemReflectionMetadataLoadContext ()
{
	var resolver = new SimpleResolver (assemblies);
	using (var context = new MetadataLoadContext (resolver)) {
		foreach (var assemblyFile in assemblies) {
			var assembly = context.LoadFromAssemblyPath (assemblyFile);
			foreach (var t in assembly.GetTypes ()) {
				var name = t.Name;
			}
		}
	}
}

[Benchmark (Description = "MetadataLoadContext - Types + Nested Types")]
public void SystemReflectionMetadataLoadContext2 ()
{
	var resolver = new SimpleResolver (assemblies);
	using (var context = new MetadataLoadContext (resolver)) {
		foreach (var assemblyFile in assemblies) {
			var assembly = context.LoadFromAssemblyPath (assemblyFile);
			foreach (var t in assembly.GetTypes ()) {
				var name = t.Name;
				foreach (var m in t.GetNestedTypes ()) {
					var mname = m.Name;
				}
			}
		}
	}
}