Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 47 Regressions on 9/19/2023 3:42:14 PM #22151

Open
performanceautofiler bot opened this issue Sep 26, 2023 · 4 comments
Open

[Perf] Linux/x64: 47 Regressions on 9/19/2023 3:42:14 PM #22151

performanceautofiler bot opened this issue Sep 26, 2023 · 4 comments
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Sep 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.06 ns 12.74 ns 1.15 0.21 False
34.92 ns 44.59 ns 1.28 0.34 False
19.51 ns 23.13 ns 1.19 0.24 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.737944962576963 > 11.906927257926377.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -10.43115157409674 (T) = (0 -12.469871033189825) / Math.Sqrt((0.09634251596629442 / (20)) + (0.14970922010354398 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.09118498778088777 = (11.427824954364018 - 12.469871033189825) / 11.427824954364018 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegments

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.59065991350077 > 38.08087678055651.
IsChangePoint: Marked as a change because one of 9/15/2023 8:45:09 AM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.542505651116062 (T) = (0 -41.155766256260094) / Math.Sqrt((4.045403768407154 / (20)) + (2.402065408685345 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.09275442087464573 = (37.66241112373522 - 41.155766256260094) / 37.66241112373522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionSingleSegment

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.134266628154773 > 22.09766170071793.
IsChangePoint: Marked as a change because one of 9/19/2023 5:34:53 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.189087840123529 (T) = (0 -23.09876319322632) / Math.Sqrt((1.4708306831139029 / (19)) + (0.2378842186517002 / (30))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (30) - 2, .025) and -0.08492128958140772 = (21.290727184585393 - 23.09876319322632) / 21.290727184585393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
482.82 ns 517.77 ns 1.07 0.05 False
487.92 ns 536.21 ns 1.10 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_BitOperations.Log2_uint

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 517.77463452844 > 506.84294981808586.
IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -134.46846886175683 (T) = (0 -517.2895687738608) / Math.Sqrt((0.817851117897713 / (20)) + (0.762165716207539 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.07224036364152873 = (482.43806735371226 - 517.2895687738608) / 482.43806735371226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_BitOperations.Log2_ulong

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 536.2146087468412 > 512.3498664330951.
IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -174.69409834690535 (T) = (0 -535.5856770686918) / Math.Sqrt((0.600971765902209 / (20)) + (1.3170743059374581 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.0973329573480145 = (488.07945982327135 - 535.5856770686918) / 488.07945982327135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.XposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.12 μs 17.39 μs 1.15 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.XposMatrix*'

Payloads

Baseline
Compare

Benchstone.BenchI.XposMatrix.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.388054933582563 > 15.879922211013962.
IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -143.81018056969515 (T) = (0 -17192.699390634236) / Math.Sqrt((240.2879839976767 / (20)) + (5834.7804974655155 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.1366250237735634 = (15126.096145195672 - 17192.699390634236) / 15126.096145195672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
51.29 ns 55.89 ns 1.09 0.18 False
27.88 ns 31.12 ns 1.12 0.16 False
35.09 ns 38.34 ns 1.09 0.24 False
21.91 ns 24.41 ns 1.11 0.28 False
9.59 ns 11.04 ns 1.15 0.21 False
184.38 ns 206.17 ns 1.12 0.25 False
83.15 ns 113.34 ns 1.36 0.07 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

System.Tests.Perf_String.Concat_str_str_str_str(size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.89243736152414 > 54.26349920860425.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -12.164265058163682 (T) = (0 -55.36843962614912) / Math.Sqrt((1.2821467004764653 / (20)) + (0.31401289338772054 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.06382467706339906 = (52.046583257486716 - 55.36843962614912) / 52.046583257486716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Concat_str_str(size: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.122559327036655 > 28.993797763602622.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -10.415184265008975 (T) = (0 -30.335821449706845) / Math.Sqrt((0.6312898838035554 / (20)) + (0.15874788491824315 / (31))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (31) - 2, .025) and -0.070388068829491 = (28.340956269140957 - 30.335821449706845) / 28.340956269140957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.33842124641215 > 37.28039298623212.
IsChangePoint: Marked as a change because one of 9/18/2023 7:10:55 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.209809385064858 (T) = (0 -37.682794472054844) / Math.Sqrt((1.211325046230011 / (14)) + (0.17735569286962363 / (37))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (37) - 2, .025) and -0.052407571195417106 = (35.80627458737437 - 37.682794472054844) / 35.80627458737437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TeSt")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.409426387260982 > 23.968435580280794.
IsChangePoint: Marked as a change because one of 9/18/2023 2:28:59 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.765009885043624 (T) = (0 -24.04134210477365) / Math.Sqrt((1.4334711498767556 / (16)) + (0.10980814253670848 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.06430181847917764 = (22.588838699089383 - 24.04134210477365) / 22.588838699089383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.04226567222914 > 10.111385863597711.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -9.09260213634686 (T) = (0 -10.845261767313646) / Math.Sqrt((0.13928762444253434 / (20)) + (0.019983696043637655 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.0791516434543353 = (10.049803318279029 - 10.845261767313646) / 10.049803318279029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_Enumerable

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 206.16781265195635 > 201.9617063660344.
IsChangePoint: Marked as a change because one of 9/18/2023 2:28:59 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -5.9057971622343 (T) = (0 -205.17166015905684) / Math.Sqrt((103.50930576091129 / (16)) + (6.792743120941823 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.0803442017196016 = (189.9132330533933 - 205.17166015905684) / 189.9132330533933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.IndexerCheckPathLength

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.33978782309669 > 86.9751300530584.
IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -483.88311836904995 (T) = (0 -113.07843841409837) / Math.Sqrt((0.05114799360595992 / (20)) + (0.03754068824095912 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.36162226555283616 = (83.04684880295129 - 113.07843841409837) / 83.04684880295129 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Sep 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.49 μs 6.28 μs 1.40 0.35 False
123.51 ns 152.76 ns 1.24 0.16 True
10.57 ns 13.95 ns 1.32 0.50 True
41.50 ns 48.80 ns 1.18 0.36 False
9.38 ns 14.06 ns 1.50 0.39 True
9.10 ns 13.96 ns 1.53 0.49 True
108.74 ns 121.31 ns 1.12 0.10 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_BitArray.BitArrayGet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.283940720317078 > 4.71786526493784.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -13.004698900844856 (T) = (0 -6281.194312371646) / Math.Sqrt((213568.53258690835 / (20)) + (75.41786494415972 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.27222540116580674 = (4937.170965629093 - 6281.194312371646) / 4937.170965629093 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 152.76134161119475 > 129.40155459096067.
IsChangePoint: Marked as a change because one of 9/21/2023 3:30:33 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -16.075681063927398 (T) = (0 -152.27935661421634) / Math.Sqrt((104.03761375373251 / (30)) + (0.3036438202719415 / (20))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (20) - 2, .025) and -0.24536185348822384 = (122.27719693491984 - 152.27935661421634) / 122.27719693491984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.949089170719352 > 10.11195736491559.
IsChangePoint: Marked as a change because one of 9/21/2023 3:30:33 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -18.293165833419163 (T) = (0 -13.780108731300576) / Math.Sqrt((0.4572094625172488 / (30)) + (0.618319389684315 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.4088177626755691 = (9.781328072645788 - 13.780108731300576) / 9.781328072645788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayGet(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.79647990660787 > 43.536610728370256.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -11.831409849638883 (T) = (0 -48.36401817502863) / Math.Sqrt((12.519328273840495 / (20)) + (0.056101176622964995 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.24045973253235825 = (38.988785291961925 - 48.36401817502863) / 38.988785291961925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.057843218042835 > 9.90041357864749.
IsChangePoint: Marked as a change because one of 9/21/2023 3:30:33 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -47.882771326674316 (T) = (0 -14.078745095653284) / Math.Sqrt((0.2651720664133852 / (30)) + (0.006258601538487627 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.4829303432422017 = (9.493868110401092 - 14.078745095653284) / 9.493868110401092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.964899145221636 > 9.978234667581363.
IsChangePoint: Marked as a change because one of 9/21/2023 3:30:33 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -37.56530327263558 (T) = (0 -13.964472944719446) / Math.Sqrt((0.3889484782870404 / (30)) + (0.0007788293607919415 / (20))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (20) - 2, .025) and -0.44250228280437093 = (9.680728489088484 - 13.964472944719446) / 9.680728489088484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayCopyToBoolArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.30615011568524 > 113.9132517689517.
IsChangePoint: Marked as a change because one of 9/21/2023 3:30:33 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -40.95652930924936 (T) = (0 -121.23022579286872) / Math.Sqrt((1.293670715403571 / (30)) + (0.9066144270527217 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.11336696254877475 = (108.88613536308145 - 121.23022579286872) / 108.88613536308145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.42 μs 1.54 μs 1.08 0.45 False
1.19 μs 1.29 μs 1.09 0.29 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndClear<String>.Array(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5368642954560303 > 1.4947177032089012.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -4.409776846088237 (T) = (0 -1481.1129759070297) / Math.Sqrt((4760.636439663928 / (20)) + (3168.0338061625366 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.05874288831100459 = (1398.9354660694112 - 1481.1129759070297) / 1398.9354660694112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<String>.Span(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2899989291909049 > 1.2542774470261924.
IsChangePoint: Marked as a change because one of 9/23/2023 1:57:56 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.276395961954069 (T) = (0 -1368.5503005275611) / Math.Sqrt((4160.161707835983 / (38)) + (3763.2674561331646 / (11))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (11) - 2, .025) and -0.10798326366576676 = (1235.1723581091896 - 1368.5503005275611) / 1235.1723581091896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
54.99 μs 58.32 μs 1.06 0.02 False
54.50 μs 57.87 μs 1.06 0.06 False
19.91 μs 21.92 μs 1.10 0.05 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'

Payloads

Baseline
Compare

IfStatements.IfStatements.AndArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.31749073951142 > 57.66133186228804.
IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -74.98956899943012 (T) = (0 -58257.05589296404) / Math.Sqrt((22118.192733550706 / (20)) + (28929.689108857696 / (32))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (32) - 2, .025) and -0.0612436996936069 = (54895.07820850528 - 58257.05589296404) / 54895.07820850528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

IfStatements.IfStatements.OrArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.867702398523996 > 57.56295706817033.
IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -12.591746287671347 (T) = (0 -58328.42365481427) / Math.Sqrt((459149.1477857401 / (20)) + (1093178.0262033225 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.05553403312974452 = (55259.63334584839 - 58328.42365481427) / 55259.63334584839 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

IfStatements.IfStatements.SingleArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.91659054954728 > 20.910774296946087.
IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -132.24609092058864 (T) = (0 -21910.506582444308) / Math.Sqrt((2270.145960560157 / (20)) + (3677.8432245293784 / (32))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (32) - 2, .025) and -0.1003830269342509 = (19911.708965094283 - 21910.506582444308) / 19911.708965094283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.74 ns 19.81 ns 1.12 0.17 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Char>.SequenceCompareTo(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.812881843384204 > 18.82324828665167.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -23.807385486290993 (T) = (0 -19.740800350241127) / Math.Sqrt((0.0790385295340506 / (20)) + (0.0628968230278722 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.10350061013192749 = (17.889251867183873 - 19.740800350241127) / 17.889251867183873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.83 μs 5.71 μs 1.18 0.15 False
687.85 ns 800.33 ns 1.16 0.04 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.714582264817906 > 5.0686288237774475.
IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -82.39827308405869 (T) = (0 -5663.803612143357) / Math.Sqrt((1962.227600595406 / (20)) + (283.84825585414313 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.17770423481422795 = (4809.190155486509 - 5663.803612143357) / 4809.190155486509 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.IEnumerable(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 800.3295593783324 > 723.7804211785659.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -289.9154980832468 (T) = (0 -801.4011557074691) / Math.Sqrt((1.403306974383467 / (20)) + (2.255657079628264 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.16163858134917314 = (689.8885493082453 - 801.4011557074691) / 689.8885493082453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Sep 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.57 μs 1.69 μs 1.07 0.18 False
1.34 μs 1.45 μs 1.08 0.26 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.AddGivenSize<String>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6851179104276635 > 1.6316641822216045.
IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -9.751621714088452 (T) = (0 -1704.302881677189) / Math.Sqrt((1768.5433775948106 / (20)) + (2553.775042153891 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.08226636003570847 = (1574.753632387647 - 1704.302881677189) / 1574.753632387647 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.List(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4508186702061887 > 1.410736795739546.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -9.679311444359929 (T) = (0 -1496.0624974672069) / Math.Sqrt((1684.2290588596468 / (20)) + (3119.056283723785 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.09739691224059369 = (1363.282947837573 - 1496.0624974672069) / 1363.282947837573 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.52 ns 4.60 ns 1.31 0.13 True
3.52 ns 4.58 ns 1.30 0.07 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: True, CaptureScopes: False)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.601681076802472 > 3.6956746619601972.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -168.0088466362082 (T) = (0 -4.5996273290658385) / Math.Sqrt((0.000253010504967038 / (20)) + (0.0008350306646042205 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.30744493162427167 = (3.518027580214492 - 4.5996273290658385) / 3.518027580214492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: True, CaptureScopes: True)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.582903298501422 > 3.6962819548957206.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -99.81958518844804 (T) = (0 -4.5965252968936525) / Math.Sqrt((0.002195524631911015 / (20)) + (7.159548770219373E-05 / (31))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (31) - 2, .025) and -0.29855261945793704 = (3.539729717546917 - 4.5965252968936525) / 3.539729717546917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
109.67 μs 131.32 μs 1.20 0.03 False
1.18 ms 2.43 ms 2.05 0.08 False
161.07 μs 182.12 μs 1.13 0.28 False
109.60 μs 131.33 μs 1.20 0.04 False
2.30 ms 2.50 ms 1.08 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

PerfLabTests.LowLevelPerf.SealedClassInterfaceMethod

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.32106054921968 > 115.03998391717658.
IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -380.8985015576184 (T) = (0 -131343.32717176163) / Math.Sqrt((45555.31221815608 / (20)) + (31435.47960341856 / (31))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (31) - 2, .025) and -0.19959711719932524 = (109489.53218427717 - 131343.32717176163) / 109489.53218427717 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.LowLevelPerf.EmptyStaticFunction

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4313375514880953 > 1.2418672204296874.
IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -19.77677206634326 (T) = (0 -2440723.3208083133) / Math.Sqrt((77422783639.68121 / (20)) + (732513963.9562999 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -1.0231994551330834 = (1206368.118880185 - 2440723.3208083133) / 1206368.118880185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.LowLevelPerf.GenericClassWithSTringGenericInstanceMethod

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 182.1151090116279 > 164.74741546202242.
IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -16.328433567973075 (T) = (0 -182008.1578356058) / Math.Sqrt((57791350.18095035 / (20)) + (28397.51525211425 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.17997709832213837 = (154247.19521625567 - 182008.1578356058) / 154247.19521625567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.3283912064826 > 115.03327666386498.
IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -112.19851134375116 (T) = (0 -131402.83858027754) / Math.Sqrt((737937.1369027921 / (20)) + (21545.71907303438 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.19854580232920246 = (109635.22489079258 - 131402.83858027754) / 109635.22489079258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.495824729910714 > 2.4142015065204325.
IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -89.89043598759864 (T) = (0 -2507636.203574783) / Math.Sqrt((8625825.174346464 / (20)) + (143060927.72162446 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.09054266523030438 = (2299438.8789412584 - 2507636.203574783) / 2299438.8789412584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
25.56 ns 27.71 ns 1.08 0.29 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

System.Tests.Perf_DateTimeOffset.ToString(format: "s")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.707581971272415 > 25.35340387311906.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -8.810144448699445 (T) = (0 -27.379126573585037) / Math.Sqrt((1.784480953133729 / (20)) + (0.34832395320946197 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.1138098391220277 = (24.58150899005069 - 27.379126573585037) / 24.58150899005069 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.94 ms 2.07 ms 1.07 0.25 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0716474076502736 > 2.002466497774961.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -6.905103909439841 (T) = (0 -2048093.4074717918) / Math.Sqrt((4566497297.894623 / (20)) + (1210532955.9267588 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.058495703106900505 = (1934909.51494675 - 2048093.4074717918) / 1934909.51494675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Perf_Frozen<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
172.61 ns 250.49 ns 1.45 0.42 False
183.57 ns 226.99 ns 1.24 0.52 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;Int16&gt;*'

Payloads

Baseline
Compare

System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.48584942134096 > 181.1624399864525.
IsChangePoint: Marked as a change because one of 9/15/2023 8:45:09 AM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -9.762072805105593 (T) = (0 -233.4698683934885) / Math.Sqrt((384.6521374832989 / (20)) + (136.5025485036859 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.2572022171345258 = (185.7058993465856 - 233.4698683934885) / 185.7058993465856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 64)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 226.99395345993196 > 200.88417901637607.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -10.605096780455595 (T) = (0 -232.55007592528818) / Math.Sqrt((172.94635100016558 / (20)) + (104.23810617133914 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.1898506507263515 = (195.4447608893827 - 232.55007592528818) / 195.4447608893827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryReaderTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.53 ns 3.65 ns 1.44 0.23 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryReaderTests*'

Payloads

Baseline
Compare

System.IO.Tests.BinaryReaderTests.ReadSingle

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.648392160228528 > 2.69637560111896.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -63.62519509832795 (T) = (0 -3.646348846738466) / Math.Sqrt((0.0054444906148283685 / (20)) + (4.075405839728317E-05 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.4057528974374303 = (2.593876102539397 - 3.646348846738466) / 2.593876102539397 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Sep 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.MDBenchI.MDXposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
34.03 μs 38.43 μs 1.13 0.27 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDXposMatrix*'

Payloads

Baseline
Compare

Benchstone.MDBenchI.MDXposMatrix.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.4258549600737 > 35.64345072142171.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -12.669566073329348 (T) = (0 -38363.17276514261) / Math.Sqrt((4968984.588208863 / (20)) + (6387.487031670353 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.19715209228549657 = (32045.36250018412 - 38363.17276514261) / 32045.36250018412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.66 ns 23.39 ns 1.13 0.16 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionSingleSegment

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.39170725677192 > 22.724208608131338.
IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -5.231167412447404 (T) = (0 -23.22867315668074) / Math.Sqrt((1.8501780860979415 / (20)) + (0.059468184442307816 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.07437477196613035 = (21.620642780146202 - 23.22867315668074) / 21.620642780146202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.69 μs 4.39 μs 1.19 0.18 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndRemove<String>.Queue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.389201778413516 > 3.9592475974718693.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -11.399315706011562 (T) = (0 -4326.41408427983) / Math.Sqrt((28308.751649476075 / (20)) + (6322.841017581345 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.11918073237811593 = (3865.6974330560115 - 4326.41408427983) / 3865.6974330560115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.04 ns 8.28 ns 1.18 0.19 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.277934550572215 > 7.523401635557671.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -12.708238261795874 (T) = (0 -7.856616327779337) / Math.Sqrt((0.03825084362483581 / (20)) + (0.028064558415947103 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.09506184172135683 = (7.174586884909909 - 7.856616327779337) / 7.174586884909909 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.99 μs 18.55 μs 1.09 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.551579775755766 > 17.93687537143808.
IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM.
IsRegressionStdDev: Marked as regression because -53.423535960410476 (T) = (0 -18576.94535107189) / Math.Sqrt((14719.22215464324 / (20)) + (1265.687876692793 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.08730675877747819 = (17085.284535487503 - 18576.94535107189) / 17085.284535487503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@cincuranet
Copy link

cincuranet commented Sep 26, 2023 •
edited
Loading

Most of these regressions point to dotnet/runtime@e4df432...0eb038e. That's nth-time I saw this commit range. Could it be that i.e. dotnet/runtime#91937 confused static PGO?

cc @AndyAyersMS

This was referenced Sep 26, 2023
Closed
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cincuranet