[Perf] Linux/x64: 3 Regressions on 5/10/2024 2:50:16 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	02e348b505ec5d95621a662835abd6d915962801
Compare	56d7e5d80ed6dd5e80ca8225ab689a60cae3cfc7
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AddMultiply_Vectors - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	738.62 ns	803.60 ns	1.09	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>*'

System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives<Single>.AddMultiply_Vectors(BufferLength: 3079)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	02e348b505ec5d95621a662835abd6d915962801
Compare	56d7e5d80ed6dd5e80ca8225ab689a60cae3cfc7
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in PerfLabTests.StackWalk

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Walk - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	39.68 ms	47.80 ms	1.20	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.StackWalk*'

PerfLabTests.StackWalk.Walk

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	02e348b505ec5d95621a662835abd6d915962801
Compare	56d7e5d80ed6dd5e80ca8225ab689a60cae3cfc7
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
BenchBitOps - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	3.27 secs	3.48 secs	1.07	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

ByteMark.BenchBitOps

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

@kg looks like dotnet/runtime#101871

[kg]

I'm confused as to why these 3 benchmarks would be benchmarking the cost of mmap/munmap, or why they would be so significantly impacted by it.

[kg]

StackWalk is real and is impacted by custom mmap, I don't have a reasonable explanation for the other two. I'm pretty confident tensors is a fake regression.