Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perf] Linux/x64: 87 Regressions on 6/15/2024 2:08:36 PM #36642

Open
performanceautofiler bot opened this issue Jun 20, 2024 · 3 comments
Open

[Perf] Linux/x64: 87 Regressions on 6/15/2024 2:08:36 PM #36642

performanceautofiler bot opened this issue Jun 20, 2024 · 3 comments

Comments

@performanceautofiler
Copy link

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 87cd2c4257bd21e84d3d6990f55465be01f63399
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
31.19 ns 94.83 ns 3.04 0.26 False
19.40 ns 36.49 ns 1.88 0.40 False
16.30 ns 38.43 ns 2.36 0.47 True
12.28 ns 34.67 ns 2.82 0.67 True
13.90 ns 24.92 ns 1.79 0.50 False
29.80 ns 63.11 ns 2.12 0.36 True
14.35 ns 43.56 ns 3.04 0.55 True
20.76 ns 37.93 ns 1.83 0.31 False
29.96 ns 99.38 ns 3.32 0.19 False
14.42 ns 43.73 ns 3.03 0.58 True
19.09 ns 35.12 ns 1.84 0.42 False
11.38 ns 27.95 ns 2.46 0.53 False
10.52 ns 32.43 ns 3.08 0.67 True
25.90 ns 78.21 ns 3.02 0.36 True
19.88 ns 36.53 ns 1.84 0.37 False
29.71 ns 62.17 ns 2.09 0.35 True
23.42 ns 74.93 ns 3.20 0.42 True
10.01 ns 31.40 ns 3.14 0.60 True
31.08 ns 110.42 ns 3.55 0.26 True
19.04 ns 35.51 ns 1.87 0.48 True
48.57 ns 124.36 ns 2.56 0.26 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MinBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MaxBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DotBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ReflectBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AbsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ClampBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Copy link
Author

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline b4a1fa2293f31aa694e5a433aca676c29b50de81
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.75 ns 37.51 ns 3.85 0.53 True
29.15 ns 67.76 ns 2.32 0.33 False
9.45 ns 43.13 ns 4.56 0.66 True
10.68 ns 53.61 ns 5.02 0.56 True
34.85 ns 119.46 ns 3.43 0.35 True
11.92 ns 44.96 ns 3.77 0.66 True
13.11 ns 36.10 ns 2.75 0.61 False
23.64 ns 60.57 ns 2.56 0.43 True
24.34 ns 117.10 ns 4.81 0.40 True
12.19 ns 53.95 ns 4.43 0.59 True
19.57 ns 80.12 ns 4.09 0.50 True
13.16 ns 36.30 ns 2.76 0.52 True
29.44 ns 81.83 ns 2.78 0.35 True
22.80 ns 61.75 ns 2.71 0.38 True
23.71 ns 74.31 ns 3.13 0.39 True
16.98 ns 34.45 ns 2.03 0.47 True
9.84 ns 39.85 ns 4.05 0.58 True
13.16 ns 32.51 ns 2.47 0.52 True
34.42 ns 93.96 ns 2.73 0.23 True
18.46 ns 35.84 ns 1.94 0.46 True
29.68 ns 74.19 ns 2.50 0.29 False
16.67 ns 32.72 ns 1.96 0.44 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

System.Numerics.Tests.Perf_Vector2.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.LengthSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DistanceSquaredBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.ClampBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.LengthBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MinBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.ReflectBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DistanceBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.SquareRootBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MaxBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.InequalityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.AbsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.DivideByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector2.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
41.50 ns 110.11 ns 2.65 0.19 True
57.63 ns 263.83 ns 4.58 0.13 True
51.71 ns 57.06 ns 1.10 0.16 False
28.01 ns 33.46 ns 1.19 0.36 False
52.08 ns 58.14 ns 1.12 0.14 False
50.87 ns 59.22 ns 1.16 0.14 False
50.97 ns 57.07 ns 1.12 0.10 False
26.01 ns 40.65 ns 1.56 0.35 False
41.70 ns 106.90 ns 2.56 0.18 True
57.74 ns 121.40 ns 2.10 0.18 False
26.21 ns 34.52 ns 1.32 0.31 False
53.73 ns 193.18 ns 3.60 0.16 True
57.35 ns 396.16 ns 6.91 0.16 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualityOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualsBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.IsIdentityBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Copy link
Author

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
37.46 ns 54.49 ns 1.45 0.40 False
118.42 ns 744.56 ns 6.29 0.06 True
348.86 ns 1.14 μs 3.26 0.12 True
36.00 ns 52.99 ns 1.47 0.41 False
45.18 ns 132.06 ns 2.92 0.28 True
48.91 ns 68.49 ns 1.40 0.25 False
46.51 ns 137.60 ns 2.96 0.27 True
58.48 ns 257.29 ns 4.40 0.09 False
36.45 ns 52.12 ns 1.43 0.38 False
135.24 ns 399.32 ns 2.95 0.03 True
146.93 ns 529.56 ns 3.60 0.14 False
190.07 ns 330.27 ns 1.74 0.10 False
158.39 ns 531.35 ns 3.35 0.13 False
128.26 ns 292.97 ns 2.28 0.12 False
145.86 ns 650.93 ns 4.46 0.06 True
48.59 ns 53.69 ns 1.11 0.30 False
331.53 ns 844.17 ns 2.55 0.08 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateReflectionBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 87cd2c4257bd21e84d3d6990f55465be01f63399
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.00 ns 57.93 ns 3.62 0.36 False
12.75 ns 31.60 ns 2.48 0.37 False
12.67 ns 31.98 ns 2.52 0.38 False
26.60 ns 48.40 ns 1.82 0.38 False
13.18 ns 34.39 ns 2.61 0.42 False
12.61 ns 33.33 ns 2.64 0.40 False
30.37 ns 73.80 ns 2.43 0.30 False
29.99 ns 98.22 ns 3.28 0.24 False
29.02 ns 98.04 ns 3.38 0.23 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

System.Numerics.Tests.Perf_Vector4.LerpBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.DivideByScalarBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 87cd2c4257bd21e84d3d6990f55465be01f63399
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.84 ns 34.17 ns 2.66 0.60 True
14.22 ns 33.92 ns 2.39 0.65 True
25.39 ns 45.15 ns 1.78 0.29 False
57.30 ns 151.07 ns 2.64 0.15 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*'

System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Plane.DotNormalBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Plane.NormalizeBenchmark

ETL Files

Histogram

JIT Disasms

System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 5ba3c05f3bf0ea789421c6e69d3983ecdc086f36
Compare 4834a507be8698d901ff36d10284ca8703743b93
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in SIMD.RayTracerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.33 secs 3.41 secs 1.46 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SIMD.RayTracerBench*'

SIMD.RayTracerBench.Bench

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@radekdoulik
Copy link
Member

looks like dotnet/runtime#103462

same issue as #36484

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant