[Perf] Linux/x64: 23 Improvements on 2/9/2023 4:10:34 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	8.85 μs	6.70 μs	0.76	0.42	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.695488891847749 < 8.424729018831327.
IsChangePoint: Marked as a change because one of 1/17/2023 5:52:57 PM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 16.87007661340385 (T) = (0 -7113.668794532318) / Math.Sqrt((429300.8657158672 / (299)) + (235248.69442729262 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2038990417949693 = (8935.636518478148 - 7113.668794532318) / 8935.636518478148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AddThenRemove_NoExpiration - Duration of single invocation	85.42 μs	71.44 μs	0.84	0.26	False
AddThenRemove_ExpirationTokens - Duration of single invocation	115.57 μs	98.75 μs	0.85	0.19	False
AddThenRemove_SlidingExpiration - Duration of single invocation	89.40 μs	77.08 μs	0.86	0.24	False
AddThenRemove_RelativeExpiration - Duration of single invocation	92.01 μs	76.74 μs	0.83	0.23	False
AddThenRemove_AbsoluteExpiration - Duration of single invocation	91.59 μs	84.91 μs	0.93	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_NoExpiration

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 71.4353974537037 < 80.91591113549575.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 22.013964262122858 (T) = (0 -73771.01654998092) / Math.Sqrt((23582391.9454046 / (299)) + (7481041.139237767 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1594543089490907 = (87765.62337467606 - 73771.01654998092) / 87765.62337467606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.74694809561481 < 108.1990874942555.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 32.92592685195982 (T) = (0 -101042.46114396666) / Math.Sqrt((22401157.23905777 / (299)) + (3490836.9862547694 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13431264032403903 = (116719.34447765102 - 101042.46114396666) / 116719.34447765102 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_SlidingExpiration

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.08379817961165 < 84.83972717499405.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 19.251635670842855 (T) = (0 -77242.43329244948) / Math.Sqrt((23300109.637943782 / (299)) + (9285001.634872157 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.14982755484884577 = (90855.01857063406 - 77242.43329244948) / 90855.01857063406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_RelativeExpiration

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.74484379464285 < 85.28761728243022.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 28.047211166859892 (T) = (0 -76564.90709485898) / Math.Sqrt((29687225.44682067 / (299)) + (3715245.9129676814 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15759791805354625 = (90888.79139276122 - 76564.90709485898) / 90888.79139276122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_AbsoluteExpiration

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.9089671244177 < 86.51513938538206.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 15.99871418471292 (T) = (0 -79801.03452741304) / Math.Sqrt((15312911.7805698 / (299)) + (11915409.309352476 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1313900358900544 = (91872.11501676039 - 79801.03452741304) / 91872.11501676039 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	251.64 μs	147.66 μs	0.59	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.66373993055555 < 237.42486660221402.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 65.55569452756656 (T) = (0 -149447.063655062) / Math.Sqrt((246876051.55087999 / (299)) + (11697352.459561871 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3362933565205263 = (225170.3597113141 - 149447.063655062) / 225170.3597113141 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EmptyStaticFunction - Duration of single invocation	2.08 ms	1.85 ms	0.89	0.17	False
IntegerFormatting - Duration of single invocation	15.83 ms	6.52 ms	0.41	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.EmptyStaticFunction

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8543731472222222 < 1.9749580215159972.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 7.432341588152203 (T) = (0 -1871950.6998785078) / Math.Sqrt((32442083518.633114 / (299)) + (1600191847.7717698 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.05031670043593053 = (1971131.534836702 - 1871950.6998785078) / 1971131.534836702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.IntegerFormatting

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.524692720486111 < 14.973042752420634.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/12/2023 11:53:55 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 19.394117789487204 (T) = (0 -6507706.108978175) / Math.Sqrt((17421041881621.164 / (299)) + (10891082742.83801 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.4228589493395197 = (11275763.700278735 - 6507706.108978175) / 11275763.700278735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	27.59 μs	24.23 μs	0.88	0.30	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.23490981728858 < 26.096619244717026.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 17.184250947561814 (T) = (0 -24543.315284837605) / Math.Sqrt((3483236.369107018 / (299)) + (923731.0812190642 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13746403951956077 = (28454.83134542795 - 24543.315284837605) / 28454.83134542795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	249.10 μs	133.33 μs	0.54	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 133.3347222934473 < 246.31892651145102.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 69.10498032766829 (T) = (0 -127411.50950235543) / Math.Sqrt((451949283.63005376 / (299)) + (12330915.131497653 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4369755643693724 = (226298.36546906785 - 127411.50950235543) / 226298.36546906785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	250.61 μs	121.52 μs	0.48	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 121.52323705128205 < 237.94465072260627.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 88.09625084862516 (T) = (0 -125058.49324093097) / Math.Sqrt((250382437.01263618 / (299)) + (11456063.225965975 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.44875487526144014 = (226865.48620315274 - 125058.49324093097) / 226865.48620315274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	178.03 μs	121.19 μs	0.68	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 121.19085779151405 < 171.82332595943205.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 49.07824880789299 (T) = (0 -121953.31790946392) / Math.Sqrt((93157046.44554016 / (299)) + (14094228.320761628 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.27897489323599783 = (169138.79525887343 - 121953.31790946392) / 169138.79525887343 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.TryAddDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	247.07 μs	136.35 μs	0.55	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<String>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.3474686883629 < 243.6645270392813.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 71.719286132957 (T) = (0 -131241.30237985819) / Math.Sqrt((474360225.314664 / (299)) + (5027226.718716004 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4233637584728071 = (227598.08164723052 - 131241.30237985819) / 227598.08164723052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	270.96 μs	98.71 μs	0.36	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.70557996651786 < 248.79306968112766.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 83.13807609846766 (T) = (0 -102682.6281246911) / Math.Sqrt((438842276.4487376 / (299)) + (20489800.325369176 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5542569305869175 = (230362.81474862873 - 102682.6281246911) / 230362.81474862873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.TryAddDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	256.79 μs	131.58 μs	0.51	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<Int32>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.5808145222833 < 240.25557888019117.
IsChangePoint: Marked as a change because one of 12/9/2022 5:51:37 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 87.52480775391658 (T) = (0 -127964.18538502771) / Math.Sqrt((273806949.56565493 / (299)) + (8861923.29611834 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4382555348865307 = (227797.85709001982 - 127964.18538502771) / 227797.85709001982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.TryAddGiventSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	173.33 μs	114.13 μs	0.66	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<Int32>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.13480852450469 < 165.57314120198708.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 32.32430934458255 (T) = (0 -120113.76008470503) / Math.Sqrt((78846537.08865099 / (299)) + (29601481.200836957 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.25822353231101997 = (161927.16447169313 - 120113.76008470503) / 161927.16447169313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.Concurrent.Count<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	2.06 μs	226.58 ns	0.11	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 226.57922812585625 < 1.8499467239705014.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 466.1200634244242 (T) = (0 -230.1722167246815) / Math.Sqrt((3192.77959137436 / (299)) + (71.45045074998842 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8826115148465792 = (1960.7733792957458 - 230.1722167246815) / 1960.7733792957458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	187.26 μs	140.42 μs	0.75	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 140.41663147522524 < 178.75163972279458.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 59.01156074374134 (T) = (0 -134760.50976898425) / Math.Sqrt((177885032.07553056 / (299)) + (16451296.618770162 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.3366146271108192 = (203140.6106861752 - 134760.50976898425) / 203140.6106861752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.Concurrent.Count<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	1.73 μs	85.62 ns	0.05	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<String>.Dictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.61621870764877 < 1.658280748262585.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 213.50807309734247 (T) = (0 -87.3364895722301) / Math.Sqrt((18645.24243704238 / (299)) + (29.467019631995697 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9512246953763487 = (1790.5882955753025 - 87.3364895722301) / 1790.5882955753025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToArray - Duration of single invocation	39.97 ns	33.81 ns	0.85	0.37	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.ToArray(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.8080014969563 < 34.96765723615885.
IsChangePoint: Marked as a change because one of 12/9/2022 9:29:01 AM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 8.76123049532951 (T) = (0 -34.385753965243836) / Math.Sqrt((4.576252937552744 / (299)) + (3.3527019489013283 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.0927719884175146 = (37.90199765245837 - 34.385753965243836) / 37.90199765245837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	183.06 μs	118.58 μs	0.65	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 118.5822849025974 < 173.73136617050204.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 10:41:19 PM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 47.472361609528214 (T) = (0 -119879.06669094959) / Math.Sqrt((86135433.39797015 / (299)) + (16537928.623972148 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.2911310795271985 = (169113.1649713076 - 119879.06669094959) / 169113.1649713076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	3ff80e90e828bac0370c1930c9950c9650ae61b9
Compare	2b701237cf3169b63d6f61efd2e611c34d2622e2
Diff	Diff

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentDictionary - Duration of single invocation	174.48 μs	114.04 μs	0.65	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.03955979535974 < 165.77102610880002.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/9/2023 12:40:40 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 43.84766690142461 (T) = (0 -119849.22653785907) / Math.Sqrt((79304628.40098572 / (299)) + (16009104.200571079 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2640046054663945 = (162839.6419705949 - 119849.22653785907) / 162839.6419705949 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

looks like dotnet/runtime#81557