Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 14 Improvements on 8/16/2023 3:30:19 PM #20687

Closed
performanceautofiler bot opened this issue Aug 22, 2023 · 2 comments
Closed

[Perf] Linux/x64: 14 Improvements on 8/16/2023 3:30:19 PM #20687

performanceautofiler bot opened this issue Aug 22, 2023 · 2 comments
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Aug 22, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
284.10 μs 263.39 μs 0.93 0.04 True
262.74 μs 197.30 μs 0.75 0.04 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

PerfLabTests.CastingPerf2.CastingPerf.FooObjCastIfIsa

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 9.494813040091731 (T) = (0 -263292.68133247213) / Math.Sqrt((37983993.93114604 / (12)) + (157494.67324158226 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.06039134534189695 = (280215.25773224904 - 263292.68133247213) / 280215.25773224904 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 925.0650528613221 (T) = (0 -197284.71624611624) / Math.Sqrt((48148.56048974083 / (12)) + (13930.72248670721 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.24914079980954146 = (262745.28726034675 - 197284.71624611624) / 262745.28726034675 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
142.13 ns 106.48 ns 0.75 0.04 True
120.91 ns 99.65 ns 0.82 0.04 True
159.35 ns 138.06 ns 0.87 0.04 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 75.7009147103962 (T) = (0 -106.89806289493576) / Math.Sqrt((1.739466336191545 / (12)) + (1.1858806376126108 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.2533810790993385 = (143.17620395419723 - 106.89806289493576) / 143.17620395419723 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 44.388262705935716 (T) = (0 -101.71821652646386) / Math.Sqrt((1.0556236641214536 / (12)) + (1.582356725562636 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.16362892930307377 = (121.61852566433787 - 101.71821652646386) / 121.61852566433787 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 54.559133713635724 (T) = (0 -138.4091012823689) / Math.Sqrt((1.3455071778623504 / (12)) + (0.6116076204468842 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.13464691705547968 = (159.94523392856811 - 138.4091012823689) / 159.94523392856811 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
157.95 ns 123.41 ns 0.78 0.25 True
28.54 ns 24.52 ns 0.86 0.14 True
22.47 ns 19.10 ns 0.85 0.21 True
20.60 ns 17.98 ns 0.87 0.14 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 6.014983560580014 (T) = (0 -128.19914603351714) / Math.Sqrt((101.54829333457236 / (12)) + (168.0138870611611 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.17508431353573928 = (155.40878678523146 - 128.19914603351714) / 155.40878678523146 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 6.276982990039618 (T) = (0 -23.856979244812518) / Math.Sqrt((5.366606212959045 / (12)) + (0.1305410794945401 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.1509428588462258 = (28.098202215687788 - 23.856979244812518) / 28.098202215687788 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 7.901123872415249 (T) = (0 -19.285460762198692) / Math.Sqrt((1.2134373732907595 / (12)) + (0.17561570436383034 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.12136129229541678 = (21.94925012190889 - 19.285460762198692) / 21.94925012190889 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 7.112611568208002 (T) = (0 -18.315414631699998) / Math.Sqrt((0.9421052810494398 / (12)) + (0.1342598700414755 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.10335207337868169 = (20.42653988027918 - 18.315414631699998) / 20.42653988027918 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.44 μs 1.36 μs 0.94 0.04 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.AddGivenSize<String>.List(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 46.739509744546524 (T) = (0 -1346.8668945236454) / Math.Sqrt((13.387186643725055 / (12)) + (42.00009385481513 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.06577032768604833 = (1441.6871294482128 - 1346.8668945236454) / 1441.6871294482128 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.63 μs 1.47 μs 0.90 0.11 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.IndexerSetReverse<String>.IList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 4.392962314145936 (T) = (0 -1468.7321758928558) / Math.Sqrt((13967.605190909264 / (12)) + (112.65845370867737 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.09288462276642113 = (1619.1238873846835 - 1468.7321758928558) / 1619.1238873846835 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
182.08 ms 169.79 ms 0.93 0.32 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'

Payloads

Baseline
Compare

PerfLabTests.DelegatePerf.MulticastDelegateCombineInvoke

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 7.913382005031206 (T) = (0 -161895246.05491892) / Math.Sqrt((22674177261221.266 / (12)) + (29983713259305.64 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.08936922519382426 = (177783631.44972527 - 161895246.05491892) / 177783631.44972527 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.99 μs 2.61 μs 0.87 0.36 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

System.Text.Json.Tests.Perf_Get.GetDateTime

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 5.8997892803552 (T) = (0 -2725.9438347077967) / Math.Sqrt((3170.354807182759 / (12)) + (31137.387225590537 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.09744189657756022 = (3020.2419371907476 - 2725.9438347077967) / 3020.2419371907476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 22, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 2470610ea493b650177a5303684f87ee9fcec966
Compare 374b1116d14a2912f8b36ffbd9523001cceb8316
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollectionNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.37 μs 2.24 μs 0.95 0.04 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.CtorFromCollectionNonGeneric<String>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsImprovementStdDev: Marked as improvement because 11.81170991719676 (T) = (0 -2194.381062217904) / Math.Sqrt((1797.7126137567145 / (12)) + (257.1973720259285 / (14))) is greater than 2.063898561620592 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (14) - 2, .975) and 0.06525051476091227 = (2347.5605997864027 - 2194.381062217904) / 2347.5605997864027 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo
Copy link
Member

EgorBo commented Aug 22, 2023

dotnet/runtime#90412

@EgorBo EgorBo closed this as completed Aug 22, 2023
@EgorBo EgorBo mentioned this issue Aug 24, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo