Test failure JIT\\jit64\\opt\\cse\\hugeexpr1\\hugeexpr1.cmd

[VincentBu]

Run: runtime-coreclr r2r 20211117.1

Failed test:

R2R windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open

- JIT\\jit64\\opt\\cse\\hugeexpr1\\hugeexpr1.cmd

Error message:

Could Not Find C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\composite-r2r.dll
Could Not Find C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
Out of memory.

Return code:      1
Raw output file:      C:\h\w\C83F0A72\w\A3BB08E0\uploads\Reports\JIT.jit64\opt\cse\hugeexpr1\hugeexpr1.output.txt
Raw output:
BEGIN EXECUTION
hugeexpr1.dll
1 file(s) copied.
Response file: C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\hugeexpr1.dll
-o:C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll
--targetarch:x86
--verify-type-and-field-layout
--method-layout:random
-r:C:\h\w\C83F0A72\p\System..dll
-r:C:\h\w\C83F0A72\p\Microsoft..dll
-r:C:\h\w\C83F0A72\p\mscorlib.dll
-r:C:\h\w\C83F0A72\p\netstandard.dll
-O
" "dotnet" "C:\h\w\C83F0A72\p\crossgen2\crossgen2.dll" @"C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp"   -r:C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2*.dll"
Emitting R2R PE file: C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll
Crossgen2 failed with exitcode - -532462766
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\C83F0A72\p
C:\h\w\C83F0A72\w\A3BB08E0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.cmd
Expected: True
Actual:   False


Stack trace
   at JIT_jit64._opt_cse_hugeexpr1_hugeexpr1_._opt_cse_hugeexpr1_hugeexpr1_cmd()

[VincentBu]

Failed again in: runtime-coreclr r2r 20211217.1

Failed test:

R2R windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open

- JIT\\jit64\\opt\\cse\\hugeexpr1\\hugeexpr1.cmd

Error message:

Could Not Find C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\composite-r2r.dll
Could Not Find C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
Out of memory.

Return code:      1
Raw output file:      C:\h\w\AA0308FA\w\B50C0954\uploads\Reports\JIT.jit64\opt\cse\hugeexpr1\hugeexpr1.output.txt
Raw output:
BEGIN EXECUTION
hugeexpr1.dll
1 file(s) copied.
Response file: C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\hugeexpr1.dll
-o:C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll
--targetarch:x86
--verify-type-and-field-layout
--method-layout:random
-r:C:\h\w\AA0308FA\p\System..dll
-r:C:\h\w\AA0308FA\p\Microsoft..dll
-r:C:\h\w\AA0308FA\p\mscorlib.dll
-r:C:\h\w\AA0308FA\p\netstandard.dll
-O
" "dotnet" "C:\h\w\AA0308FA\p\crossgen2\crossgen2.dll" @"C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp"   -r:C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2*.dll"
Emitting R2R PE file: C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll
Crossgen2 failed with exitcode - -532462766
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\AA0308FA\p
C:\h\w\AA0308FA\w\B50C0954\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.cmd
Expected: True
Actual:   False


Stack trace
   at JIT_jit64._opt_cse_hugeexpr1_hugeexpr1_._opt_cse_hugeexpr1_hugeexpr1_cmd()

[VincentBu]

Failed again in: runtime-coreclr outerloop 20220206.2

Failed test:

R2R-CG2 windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open

- JIT\\jit64\\opt\\cse\\hugeexpr1\\hugeexpr1.cmd

Error message:

Could Not Find C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\composite-r2r.dll
Could Not Find C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
Fatal error. 0xC0000005
at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr ByRef, IntPtr, IntPtr, IntPtr, Internal.JitInterface.CORINFO_METHOD_INFO ByRef, UInt32, IntPtr ByRef, UInt32 ByRef)
at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(ILCompiler.DependencyAnalysis.IMethodNode, Internal.IL.MethodIL)
at Internal.JitInterface.CorInfoImpl.CompileMethod(ILCompiler.DependencyAnalysis.ReadyToRun.MethodWithGCInfo, ILCompiler.Logger)
at ILCompiler.ReadyToRunCodegenCompilation.<ComputeDependencyNodeDependencies>b__35_0(ILCompiler.DependencyAnalysisFramework.DependencyNodeCore1<ILCompiler.DependencyAnalysis.NodeFactory>) at System.Threading.Tasks.Parallel+<>c__DisplayClass33_02[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<ForEachWorker>b__0(Int32)
at System.Threading.Tasks.Parallel+<>c__DisplayClass19_0`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<ForWorker>b__1(System.Threading.Tasks.RangeWorker ByRef, Int32, Boolean ByRef)
at System.Threading.Tasks.TaskReplicator+Replica.Execute()
at System.Threading.Tasks.TaskReplicator+Replica+<>c.<.ctor>b__4_0(System.Object)
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task+<>c.<.cctor>b__271_0(System.Object)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()

Return code:      1
Raw output file:      C:\h\w\B3BC09AF\w\A01608A0\uploads\Reports\JIT.jit64\opt\cse\hugeexpr1\hugeexpr1.output.txt
Raw output:
BEGIN EXECUTION
hugeexpr1.dll
1 file(s) copied.
Response file: C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\hugeexpr1.dll
-o:C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll
--targetarch:x86
--verify-type-and-field-layout
--method-layout:random
-r:C:\h\w\B3BC09AF\p\System..dll
-r:C:\h\w\B3BC09AF\p\Microsoft..dll
-r:C:\h\w\B3BC09AF\p\mscorlib.dll
-r:C:\h\w\B3BC09AF\p\netstandard.dll
-O
" "dotnet" "C:\h\w\B3BC09AF\p\crossgen2\crossgen2.dll" @"C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp"   -r:C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2*.dll"
Crossgen2 failed with exitcode - -1073741819
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\B3BC09AF\p
C:\h\w\B3BC09AF\w\A01608A0\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.cmd
Expected: True
Actual:   False


Stack trace
   at JIT_jit64._opt_cse_hugeexpr1_hugeexpr1_._opt_cse_hugeexpr1_hugeexpr1_cmd()

[am11]

Failed again in outerloop CoreCLR Pri1 Runtime Tests Run R2R_CG2 windows x86 checked leg: https://helix.dot.net/api/2019-06-17/jobs/a4a16d37-6da7-4595-bc2b-40431a286fa5/workitems/JIT.jit64.opt/console

cc @MichalStrehovsky, @jkotas (the first two reports above show Out of memory, but this and the previous report is I think about Access Violation (0xC0000005)?)

[MichalStrehovsky]

cc @MichalStrehovsky, @jkotas (the first two reports above show Out of memory, but this and the previous report is I think about Access Violation (0xC0000005)?)

Cc @dotnet/crossgen-contrib

[trylek]

Apparently we suppressed the test on arm32 with the following commit:

e82ae5a

I'm wondering whether they might be related.

[trylek]

Well, I guess I have poorly expressed myself. In fact I noticed this bug in one of my outerloop runs from yesterday. I think the interesting question is whether the issues observed by Sergey as described in the above commit may have expanded with overall evolution and additions to the JIT code so that something that was previously observed rarely now happens more often. Alternatively, it may be the case that we have broken something in the infra logic so that it no longer honors the property JitOptimizationSensitive, that would be truly a serious bug meriting an immediate fix.

[VincentBu]

Failed again in: runtime-coreclr outerloop 20220214.3

Failed test:

R2R-CG2 windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open

- JIT\\jit64\\opt\\cse\\hugeexpr1\\hugeexpr1.cmd

Error message:

Could Not Find C:\h\w\AF590A06\w\AA8D09A5\e\JIT\jit64\opt\cse\hugeexpr1\IL-CG2\composite-r2r.dll
Could Not Find C:\h\w\AF590A06\w\AA8D09A5\e\JIT\jit64\opt\cse\hugeexpr1\hugeexpr1.dll.rsp
Fatal error. 0xC0000005
at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr ByRef, IntPtr, IntPtr, IntPtr, Internal.JitInterface.CORINFO_METHOD_INFO ByRef, UInt32, IntPtr ByRef, UInt32 ByRef)
at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(ILCompiler.DependencyAnalysis.IMethodNode, Internal.IL.MethodIL)
at Internal.JitInterface.CorInfoImpl.CompileMethod(ILCompiler.DependencyAnalysis.ReadyToRun.MethodWithGCInfo, ILCompiler.Logger)
at ILCompiler.ReadyToRunCodegenCompilation.<ComputeDependencyNodeDependencies>b__35_0(ILCompiler.DependencyAnalysisFramework.DependencyNodeCore1<ILCompiler.DependencyAnalysis.NodeFactory>) at System.Threading.Tasks.Parallel+<>c__DisplayClass33_02[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<ForEachWorker>b__0(Int32)
at System.Threading.Tasks.Parallel+<>c__DisplayClass19_01[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<ForWorker>b__1(System.Threading.Tasks.RangeWorker ByRef, Int32, Boolean ByRef) at System.Threading.Tasks.TaskReplicator+Replica.Execute() at System.Threading.Tasks.TaskReplicator+Replica+<>c.<.ctor>b__4_0(System.Object) at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task+<>c.<.cctor>b__271_0(System.Object) at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread) at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(System.Threading.Tasks.Task, Boolean) at System.Threading.Tasks.TaskScheduler.TryRunInline(System.Threading.Tasks.Task, Boolean) at System.Threading.Tasks.Task.InternalRunSynchronously(System.Threading.Tasks.TaskScheduler, Boolean) at System.Threading.Tasks.Task.RunSynchronously(System.Threading.Tasks.TaskScheduler) at System.Threading.Tasks.TaskReplicator.Run[[System.Threading.Tasks.RangeWorker, System.Threading.Tasks.Parallel, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](ReplicatableUserAction1<System.Threading.Tasks.RangeWorker>, System.Threading.Tasks.ParallelOptions, Boolean)
at System.Threading.Tasks.Parallel.ForWorker[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Int32, Int32, System.Threading.Tasks.ParallelOptions, System.Action1<Int32>, System.Action2<Int32,System.Threading.Tasks.ParallelLoopState>, System.Func4<Int32,System.Threading.Tasks.ParallelLoopState,System.__Canon,System.__Canon>, System.Func1<System.__Canon>, System.Action1<System.__Canon>) at System.Threading.Tasks.Parallel.ForEachWorker[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IList1<System.__Canon>, System.Threading.Tasks.ParallelOptions, System.Action1<System.__Canon>, System.Action2<System.__Canon,System.Threading.Tasks.ParallelLoopState>, System.Action3<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64>, System.Func4<System.__Canon,System.Threading.Tasks.ParallelLoopState,System.__Canon,System.__Canon>, System.Func5<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64,System.__Canon,System.__Canon>, System.Func1<System.__Canon>, System.Action`1<System.__Canon>)
a


Stack trace
   at JIT_jit64._opt_cse_hugeexpr1_hugeexpr1_._opt_cse_hugeexpr1_hugeexpr1_cmd()

[mangod9]

Closing as this appears to be fixed. Please reopen if required.