Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x86: 32 Improvements on 4/25/2023 8:51:41 AM #17254

Closed
performanceautofiler bot opened this issue May 2, 2023 · 3 comments
Closed

[Perf] Windows/x86: 32 Improvements on 4/25/2023 8:51:41 AM #17254

performanceautofiler bot opened this issue May 2, 2023 · 3 comments
Assignees
@tannergooding
Labels
arch-x86 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 555.02 ns 489.04 ns 0.88 0.00 True 7348.794331369076 6500.343417314477 0.8845455627417821) Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 557.31 ns 478.90 ns 0.86 0.00 True 7389.991189639982 6478.740511021312 0.8766912361281093) Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 33.40 ns 24.41 ns 0.73 0.14 False 336.80914376419764 252.8914277976365 0.7508448997889663) Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 556.68 ns 488.43 ns 0.88 0.00 True 7348.675952668203 6476.20086428987 0.8812745188387918) Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 555.67 ns 477.58 ns 0.86 0.00 True 7382.039573820396 6509.515346159043 0.8818044499848434) Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 554.08 ns 485.13 ns 0.88 0.00 True Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 559.68 ns 480.77 ns 0.86 0.00 True 7353.580354232316 6511.649785192386 0.8855073952438202) Trace Trace

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 489.0431569620519 < 528.5351658079741.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 105.33920857931575 (T) = (0 -487.94776057722635) / Math.Sqrt((6.259437993461867 / (299)) + (12.216425648287919 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.12377419441762023 = (556.8744465964614 - 487.94776057722635) / 556.8744465964614 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 478.89965709189335 < 528.6424107278863.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 144.65432260038438 (T) = (0 -478.7855454440156) / Math.Sqrt((7.007307292850476 / (299)) + (8.198849758018268 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1413198378206019 = (557.583098494811 - 478.7855454440156) / 557.583098494811 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.408437325937363 < 31.62039532008684.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/25/2023 1:57:48 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 47.76063256211132 (T) = (0 -24.778170938907856) / Math.Sqrt((4.438471544830244 / (299)) + (0.31568925589257446 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.23896233826538626 = (32.558403065666425 - 24.778170938907856) / 32.558403065666425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 488.42797491772006 < 528.0584870780267.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 167.10698138768603 (T) = (0 -486.7655335502098) / Math.Sqrt((5.904478973629568 / (299)) + (4.71442845636173 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.12617087479524652 = (557.0488777610292 - 486.7655335502098) / 557.0488777610292 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 477.57992585993264 < 527.8512775929901.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 102.75459403664262 (T) = (0 -479.09661035373296) / Math.Sqrt((6.852810367679881 / (299)) + (16.77813013624218 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.14063355377013248 = (557.4997865643708 - 479.09661035373296) / 557.4997865643708 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 485.1276430881879 < 528.4874574486122.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 205.42985547266542 (T) = (0 -487.3279051331847) / Math.Sqrt((6.372076589587886 / (299)) + (2.7985722502476706 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.12488092836581423 = (556.8703973313654 - 487.3279051331847) / 556.8703973313654 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 480.7742227820103 < 530.1073887210058.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 153.58286138627668 (T) = (0 -479.2216234327005) / Math.Sqrt((6.982360218625026 / (299)) + (7.15117765226995 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.14086144291735192 = (557.7931748983301 - 479.2216234327005) / 557.7931748983301 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayCopyToBoolArray - Duration of single invocation 199.95 ns 170.67 ns 0.85 0.08 True Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayCopyToBoolArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 170.66570381068772 < 189.97442844669445.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 19.67428090752199 (T) = (0 -166.19661965271735) / Math.Sqrt((11.399118381702156 / (299)) + (91.05565358800712 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.17186295078669833 = (200.68733769440425 - 166.19661965271735) / 200.68733769440425 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 47.31 ns 36.31 ns 0.77 0.03 False 516.4062700428435 445.212855159404 0.8621368116279204) Trace Trace
ParseExactD - Duration of single invocation 45.61 ns 37.04 ns 0.81 0.01 False Trace Trace
ctor_str - Duration of single invocation 44.67 ns 36.43 ns 0.82 0.02 False 518.6155693853041 446.9167963295935 0.8617496710700521) Trace Trace

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.Parse

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.314505428837656 < 42.49691808701071.
IsChangePoint: Marked as a change because one of 4/1/2023 1:31:30 PM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 21.3224803449366 (T) = (0 -36.383955081462354) / Math.Sqrt((6.590163050184764 / (299)) + (0.7874405238302371 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11408827341445964 = (41.06950386771887 - 36.383955081462354) / 41.06950386771887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Guid.ParseExactD

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.04458494465247 < 42.95723053893363.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 14.609733858267942 (T) = (0 -36.59656537660576) / Math.Sqrt((7.326860152219454 / (299)) + (0.7735346065113036 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08216784545752263 = (39.87282990194267 - 36.59656537660576) / 39.87282990194267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Guid.ctor_str

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.43107639197295 < 42.44291225687293.
IsChangePoint: Marked as a change because one of 4/1/2023 1:31:30 PM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 23.247614332564034 (T) = (0 -35.9903229032476) / Math.Sqrt((6.217450793618348 / (299)) + (0.8698477024504913 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1259747866230177 = (41.17767125297374 - 35.9903229032476) / 41.17767125297374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 12.56 μs 11.09 μs 0.88 0.02 False 137705.80589254765 133904.5214045214 0.9723956120558024) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.087269635675419 < 11.95440755378006.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/25/2023 10:49:06 AM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 44.29429395125597 (T) = (0 -11026.240865851521) / Math.Sqrt((1183442.1481574054 / (299)) + (3812.3298559409704 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.20430055569630498 = (13857.293661302509 - 11026.240865851521) / 13857.293661302509 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Aggregate_Seed - Duration of single invocation 1.55 μs 646.49 ns 0.42 0.06 False Trace Trace
SelectToList - Duration of single invocation 565.33 ns 505.73 ns 0.89 0.02 False 3977.2655945969695 3058.7822671156005 0.769066634944091) Trace Trace
Sum - Duration of single invocation 523.36 ns 465.27 ns 0.89 0.03 False Trace Trace
SelectToList - Duration of single invocation 384.93 ns 320.66 ns 0.83 0.01 True 3210.810094324294 2154.6608235186704 0.6710645476440464) Trace Trace
Range - Duration of single invocation 400.05 ns 369.07 ns 0.92 0.09 False Trace Trace
SelectToList - Duration of single invocation 395.73 ns 289.01 ns 0.73 0.07 False 3881.315510540572 2155.2188624754845 0.555280511626151) Trace Trace
WhereSingle_LastElementMatches - Duration of single invocation 369.69 ns 337.77 ns 0.91 0.02 False Trace Trace
SelectToArray - Duration of single invocation 309.95 ns 278.81 ns 0.90 0.02 False 2358.4519718540337 2075.742749474298 0.8801293281552426) Trace Trace

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 646.4926285708249 < 707.9854340133287.
IsChangePoint: Marked as a change because one of 4/23/2023 2:59:55 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 5.927569530321035 (T) = (0 -637.1882729935899) / Math.Sqrt((15379.456698526581 / (299)) + (204.56122054900106 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06642041412976338 = (682.5216431865683 - 637.1882729935899) / 682.5216431865683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 505.72579548914706 < 537.5856194470098.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 15.64726614491507 (T) = (0 -502.52202982456595) / Math.Sqrt((43.23245121584458 / (299)) + (466.7502243539636 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11504637632274692 = (567.8512595229942 - 502.52202982456595) / 567.8512595229942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 465.2698140607221 < 495.5892019995136.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 3/6/2023 11:14:52 AM, 3/22/2023 7:51:01 PM, 4/8/2023 3:09:32 AM, 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 8.393578569604028 (T) = (0 -446.63707628607506) / Math.Sqrt((1129.5080602488595 / (299)) + (264.805103613805 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06477978991602808 = (477.57423489166496 - 446.63707628607506) / 477.57423489166496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 320.6556338174489 < 365.676047712762.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 41.618956450378064 (T) = (0 -295.621376921114) / Math.Sqrt((5.487601702149626 / (299)) + (119.16869673029399 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.22862398078909876 = (383.2390034934291 - 295.621376921114) / 383.2390034934291 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.Range

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 369.07050614919837 < 379.7829133978474.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 26.797328938877197 (T) = (0 -369.7084009796815) / Math.Sqrt((986.9215738198186 / (299)) + (0.8138196295143576 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.12247342947815675 = (421.3073579752977 - 369.7084009796815) / 421.3073579752977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 289.01045725640796 < 375.80838934160784.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 38.958412479570185 (T) = (0 -314.89070390593486) / Math.Sqrt((126.49075486223052 / (299)) + (117.00887564736212 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.21249471681040236 = (399.8585287333528 - 314.89070390593486) / 399.8585287333528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 337.7743087206354 < 351.8353572283508.
IsChangePoint: Marked as a change because one of 2/14/2023 5:47:38 PM, 2/20/2023 10:31:48 AM, 4/8/2023 3:09:32 AM, 4/10/2023 1:33:42 PM, 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 72.04234799106321 (T) = (0 -338.03419834850234) / Math.Sqrt((61.308545330049256 / (299)) + (0.13508523734561836 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.10012918677359355 = (375.6474744819325 - 338.03419834850234) / 375.6474744819325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 278.8089316533624 < 294.3850989554753.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/14/2023 5:47:38 PM, 2/20/2023 10:31:48 AM, 2/23/2023 8:54:44 PM, 4/8/2023 3:09:32 AM, 4/9/2023 11:04:55 PM, 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 58.23016474987383 (T) = (0 -279.15768735535363) / Math.Sqrt((40.374842743941315 / (299)) + (0.24326107934282126 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.09557673162101273 = (308.6582323956487 - 279.15768735535363) / 308.6582323956487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 60.14 μs 56.07 μs 0.93 0.00 True 640693.9040207523 611481.1435523115 0.95440449755318) Trace Trace
ToStringWithFormat - Duration of single invocation 60.15 μs 56.09 μs 0.93 0.00 True 640588.2352941176 611401.098901099 0.95443697716425) Trace Trace

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "F50")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 56.069696428571426 < 57.127383304908456.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 73.77262620713815 (T) = (0 -56272.5947093354) / Math.Sqrt((24675.898899793992 / (299)) + (70006.73746805219 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06053873954789186 = (59898.792082448046 - 56272.5947093354) / 59898.792082448046 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "F50")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 56.09093261131167 < 57.06646208492977.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 70.18060953727026 (T) = (0 -56301.251352116684) / Math.Sqrt((16419.96343232025 / (299)) + (77768.71653382285 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06026933810276063 = (59912.114859005516 - 56301.251352116684) / 59912.114859005516 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64DecodeDestinationTooSmall - Duration of single invocation 103.01 ns 96.30 ns 0.93 0.18 False 1137.6072555388682 1035.8790796878795 0.9105770683549294) Trace Trace
Base64Decode - Duration of single invocation 109.63 ns 92.05 ns 0.84 0.17 False 1138.007537077559 1035.675415778981 0.9100778176202855) Trace Trace
ConvertToBase64CharArray - Duration of single invocation 161.94 ns 147.11 ns 0.91 0.55 False Trace Trace

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.Base64DecodeDestinationTooSmall(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 96.29803357956436 < 99.04830468898759.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 27.74630887634405 (T) = (0 -93.82227970563595) / Math.Sqrt((8.762553996175777 / (299)) + (3.0801595187562834 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09701337793189804 = (103.90218128675666 - 93.82227970563595) / 103.90218128675666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 92.05075294929757 < 100.57489622284517.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 22.523359382657485 (T) = (0 -94.25856000439214) / Math.Sqrt((8.287339833985117 / (299)) + (4.373891825321881 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09052517436266962 = (103.64064770934017 - 94.25856000439214) / 103.64064770934017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Buffers.Text.Tests.Base64Tests.ConvertToBase64CharArray(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 147.11349607182942 < 153.28599223408776.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 12.975335211624703 (T) = (0 -150.84243676335646) / Math.Sqrt((125.41186091335344 / (299)) + (29.879925919235188 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09283782848434902 = (166.2794608281944 - 150.84243676335646) / 166.2794608281944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateGetPositionSingleSegment - Duration of single invocation 67.00 ns 60.62 ns 0.90 0.00 True 526.7102319289747 510.7676257562262 0.9697317325422713) Trace Trace
IterateGetPositionMemory - Duration of single invocation 83.72 ns 77.15 ns 0.92 0.11 False 690.8383919646992 658.8444081992203 0.9536881792650664) Trace Trace
IterateGetPositionArray - Duration of single invocation 62.13 ns 54.89 ns 0.88 0.00 True 471.61370727004027 452.44562374053515 0.9593563901260196) Trace Trace

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionSingleSegment

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 60.624534287220705 < 63.84466960406709.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 133.52670881010607 (T) = (0 -60.078343923534824) / Math.Sqrt((0.2062434883766898 / (299)) + (0.0690673565261329 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1083940841889001 = (67.3821728390857 - 60.078343923534824) / 67.3821728390857 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionMemory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 77.15240702334384 < 79.78996365673277.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 30.015063688100312 (T) = (0 -76.86043151432588) / Math.Sqrt((2.927579399811908 / (299)) + (1.846857385848484 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09446053546279126 = (84.87805835564185 - 76.86043151432588) / 84.87805835564185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionArray

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 54.892206036019424 < 58.40759004413047.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 113.75992950220483 (T) = (0 -53.7393262589734) / Math.Sqrt((0.1692948424289785 / (299)) + (0.1439244848817464 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.13422510994953563 = (62.07078407626379 - 53.7393262589734) / 62.07078407626379 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 77.98 ns 72.25 ns 0.93 0.12 False 862.951605127262 832.7196742994906 0.9649668293700977)
EncodeUtf8 - Duration of single invocation 74.22 ns 66.12 ns 0.89 0.08 False 865.1675280278864 832.2776063021146 0.9619843317504726) Trace Trace
EncodeUtf16 - Duration of single invocation 92.55 ns 85.46 ns 0.92 0.26 False 1003.8151117826739 976.1686997572262 0.9724586612604881)

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 72.24522884602438 < 73.03367036938617.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 8.313573314706854 (T) = (0 -71.25487588112534) / Math.Sqrt((7.329724378369084 / (299)) + (7.267138546741408 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05683485954858592 = (75.54867416645786 - 71.25487588112534) / 75.54867416645786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 66.11863661513829 < 72.25591510363525.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 20.838357352413905 (T) = (0 -67.0596216516231) / Math.Sqrt((7.873366667802977 / (299)) + (4.4278792132140525 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11472678831162274 = (75.75019865757396 - 67.0596216516231) / 75.75019865757396 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 85.45628834046371 < 87.62440246015193.
IsChangePoint: Marked as a change because one of 4/28/2023 4:38:49 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 5.282537947877136 (T) = (0 -86.22089090069375) / Math.Sqrt((11.870865014802883 / (299)) + (9.982097042757072 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.05410265910686519 = (91.15248259317686 - 86.22089090069375) / 91.15248259317686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 620b0db3a987bab2694a0043cacb3e2a86673df5
Compare 2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToBase64String - Duration of single invocation 300.31 ns 280.05 ns 0.93 0.27 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToBase64String(formattingOptions: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 280.04903815686447 < 283.66164792209946.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 30.445039612927992 (T) = (0 -281.3251412886547) / Math.Sqrt((108.10183030126576 / (299)) + (9.042155838630093 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08097922933572305 = (306.1140186040738 - 281.3251412886547) / 306.1140186040738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@cincuranet cincuranet mentioned this issue May 4, 2023
Closed
@kunalspathak
Copy link
Member

possibly dotnet/runtime#85275?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tannergooding tannergooding

Labels
arch-x86 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tannergooding @kunalspathak