ilc crash while compiling System.Text.Json.SourceGeneration.Roslyn tests

[jkotas]

/__w/1/s/artifacts/bin/coreclr/linux.arm64.Checked/build/Microsoft.NETCore.Native.targets(302,5): error MSB3073: The command ""/__w/1/s/artifacts/bin/coreclr/linux.arm64.Checked/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/System.Text.Json.SourceGeneration.Roslyn4.4.Tests/Release/net9.0/native/System.Text.Json.SourceGeneration.Roslyn4.4.Tests.ilc.rsp"" exited with code 137.

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=400925
Build error leg or test failing: Build / linux-x64 Release NativeAOT_Libs
Pull request: #91831

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "ilc.*System.Text.Json.SourceGeneration.Roslyn.*exited with code 137",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=400925
Error message validated: ilc.*System.Text.Json.SourceGeneration.Roslyn.*exited with code 137
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 9/11/2023 3:20:56 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
431002	dotnet/runtime	Build product	Log	#93176
425526	dotnet/runtime	Build product	Log	#92864
424143	dotnet/runtime	Build product	Log
422615	dotnet/runtime	Build product	Log
422196	dotnet/runtime	Build product	Log
415477	dotnet/runtime	Build product	Log	#92465
415013	dotnet/runtime	Build product	Log	#92407
414818	dotnet/runtime	Build product	Log	#92427
414812	dotnet/runtime	Build product	Log	#92385

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	9

[ghost]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

---

/__w/1/s/artifacts/bin/coreclr/linux.arm64.Checked/build/Microsoft.NETCore.Native.targets(302,5): error MSB3073: The command ""/__w/1/s/artifacts/bin/coreclr/linux.arm64.Checked/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/System.Text.Json.SourceGeneration.Roslyn4.4.Tests/Release/net9.0/native/System.Text.Json.SourceGeneration.Roslyn4.4.Tests.ilc.rsp"" exited with code 137.

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=400925
Build error leg or test failing: Build / linux-x64 Release NativeAOT_Libs
Pull request: #91831

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "ilc.*System.Text.Json.SourceGeneration.Roslyn.*exited with code 137",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Author:	jkotas
Assignees:	-
Labels:	area-Infrastructure-libraries, blocking-clean-ci, Known Build Error
Milestone:	-

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

/__w/1/s/artifacts/bin/coreclr/linux.arm64.Checked/build/Microsoft.NETCore.Native.targets(302,5): error MSB3073: The command ""/__w/1/s/artifacts/bin/coreclr/linux.arm64.Checked/x64/ilc/ilc" @"/__w/1/s/artifacts/obj/System.Text.Json.SourceGeneration.Roslyn4.4.Tests/Release/net9.0/native/System.Text.Json.SourceGeneration.Roslyn4.4.Tests.ilc.rsp"" exited with code 137.

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=400925
Build error leg or test failing: Build / linux-x64 Release NativeAOT_Libs
Pull request: #91831

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "ilc.*System.Text.Json.SourceGeneration.Roslyn.*exited with code 137",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Author:	jkotas
Assignees:	-
Labels:	area-Infrastructure-libraries, blocking-clean-ci, untriaged, area-NativeAOT-coreclr, Known Build Error
Milestone:	-

[MichalStrehovsky]

ARM64 Linux and ILC is compiled with OptimizationPreference=Speed. Could be related to #86929.

[MichalStrehovsky]

ARM64 Linux and ILC is compiled with OptimizationPreference=Speed. Could be related to #86929.

Ah, cross build to ARM64, not ARM64.

[jkotas]

Also, this looks like ILC running on CoreCLR. We need dumps for crashes during the build to make a good progress on issues like this one.

[agocke]

This looks like an OOM kill.

[jtschuster]

Reopening this as I'm seeing this in CI recently and it's blocking tests from being run.

Additional context: #87078 (comment)

[MichalStrehovsky]

Reopening this as I'm seeing this in CI recently and it's blocking tests from being run.

Where do you see this? I disabled the test on Linux in #95896

[jtschuster]

I saw it in #95808 which last ran before that change was merged. Should we keep this open as the tracking issue?

[MichalStrehovsky]

Ah, I saw it in #95808 which last ran before that change was merged. Should we keep this open as the tracking issue?

I don't think there's anything we can do about the test besides maybe completely disabling parallel build (I don't want to do that). It's a big test, and it's not platform specific, so we get enough coverage on Windows.