[Perf] Linux/x64: 14 Improvements on 2/2/2023 2:14:14 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	4354abd606119bd9fb0fdefa9f3c0f58f414bfa9
Compare	ddba2aafec61e780d795775d2bfc0e545964c208
Diff	Diff

Improvements in Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AddThenRemove_RelativeExpiration - Duration of single invocation	59.38 μs	38.27 μs	0.64	0.50	False
AddThenRemove_ExpirationTokens - Duration of single invocation	64.44 μs	53.61 μs	0.83	0.48	False
SetOverride - Duration of single invocation	243.08 ns	208.66 ns	0.86	0.04	False
AddThenRemove_AbsoluteExpiration - Duration of single invocation	62.41 μs	39.08 μs	0.63	0.45	False
AddThenRemove_SlidingExpiration - Duration of single invocation	58.83 μs	37.79 μs	0.64	0.53	False
AddThenRemove_NoExpiration - Duration of single invocation	58.50 μs	36.68 μs	0.63	0.53	False
TryGetValueHit - Duration of single invocation	85.76 ns	63.43 ns	0.74	0.14	False
GetHit - Duration of single invocation	90.83 ns	65.90 ns	0.73	0.21	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_RelativeExpiration

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.272749987043944 < 51.85225152320906.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 55.04520639740798 (T) = (0 -38004.69954581673) / Math.Sqrt((28192450.915191524 / (299)) + (42943.47084925982 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.3097640121882505 = (55060.443409076324 - 38004.69954581673) / 55060.443409076324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.60703483560091 < 62.87107051583837.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 53.332394931481 (T) = (0 -53364.40877814733) / Math.Sqrt((29160679.27294821 / (299)) + (112492.30863918459 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.24197925511062068 = (70399.66800108484 - 53364.40877814733) / 70399.66800108484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.SetOverride

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.6554596108031 < 232.5005766718238.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 32.1749908266241 (T) = (0 -209.30524323864287) / Math.Sqrt((9.554394726525823 / (299)) + (31.21918517300462 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.14818697106081588 = (245.71735360669905 - 209.30524323864287) / 245.71735360669905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_AbsoluteExpiration

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.076046310116084 < 57.756381907577214.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 52.79425304786823 (T) = (0 -39289.72719542231) / Math.Sqrt((28636845.14770942 / (299)) + (50810.263969097665 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.29589222281858024 = (55800.728906448305 - 39289.72719542231) / 55800.728906448305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_SlidingExpiration

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.786696281199674 < 51.809575913202046.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 50.911754930691714 (T) = (0 -38048.02681436718) / Math.Sqrt((29699356.017753076 / (299)) + (164431.4435479146 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.30335840806985653 = (54616.35833276875 - 38048.02681436718) / 54616.35833276875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_NoExpiration

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.67549963885879 < 51.16219872107226.
IsChangePoint: Marked as a change because one of 2/2/2023 5:46:53 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 52.57500762142119 (T) = (0 -36663.09287434654) / Math.Sqrt((28990422.684002776 / (299)) + (114575.20478791492 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.31344873977833015 = (53401.828819757706 - 36663.09287434654) / 53401.828819757706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.TryGetValueHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 63.434864319780026 < 80.97376407457098.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 106.4403394115516 (T) = (0 -64.40144331336306) / Math.Sqrt((4.547250579661217 / (299)) + (0.6644505429874968 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2525334636885627 = (86.15963415722699 - 64.40144331336306) / 86.15963415722699 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.GetHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.8962969345844 < 86.33976638911906.
IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 73.49325765824314 (T) = (0 -67.2064617204041) / Math.Sqrt((7.932408020668414 / (299)) + (1.7567330626495234 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2538592184129769 = (90.0720927992404 - 67.2064617204041) / 90.0720927992404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	4354abd606119bd9fb0fdefa9f3c0f58f414bfa9
Compare	ddba2aafec61e780d795775d2bfc0e545964c208
Diff	Diff

Improvements in System.Numerics.Tests.Perf_BitOperations

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LeadingZeroCount_ulong - Duration of single invocation	1.10 μs	746.80 ns	0.68	0.02	False
LeadingZeroCount_uint - Duration of single invocation	1.10 μs	667.93 ns	0.61	0.02	False
Log2_ulong - Duration of single invocation	1.32 μs	895.20 ns	0.68	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 746.8004559327833 < 1.0476533012312217.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/18/2023 11:30:09 AM, 1/27/2023 11:43:56 PM, 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 11.884490396114638 (T) = (0 -745.8643573195286) / Math.Sqrt((27910.61528319709 / (299)) + (0.43166989860404303 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.13341939024435173 = (860.6981842460584 - 745.8643573195286) / 860.6981842460584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 667.925187289137 < 1.047981203799868.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 11.944714637037519 (T) = (0 -667.946933892995) / Math.Sqrt((41357.63277983202 / (299)) + (0.2735867379307138 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.17377641309172825 = (808.4336304080257 - 667.946933892995) / 808.4336304080257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.Log2_ulong

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 895.1969885107435 < 1.253684400877916.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 11.985963356262094 (T) = (0 -895.279434519972) / Math.Sqrt((39622.216765955534 / (299)) + (0.4227885120430693 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1335434594854074 = (1033.2652506590364 - 895.279434519972) / 1033.2652506590364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	4354abd606119bd9fb0fdefa9f3c0f58f414bfa9
Compare	ddba2aafec61e780d795775d2bfc0e545964c208
Diff	Diff

Improvements in System.Collections.IndexerSet<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	1.58 μs	1.28 μs	0.81	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.Array(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2759436464877212 < 1.4965632207072441.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 1/11/2023 4:46:06 PM, 1/27/2023 11:43:56 PM, 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 43.93676732591497 (T) = (0 -1284.6157731700423) / Math.Sqrt((13555.560165624469 / (299)) + (514.6470958748006 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.21734164119422408 = (1641.3493304156123 - 1284.6157731700423) / 1641.3493304156123 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	4354abd606119bd9fb0fdefa9f3c0f58f414bfa9
Compare	ddba2aafec61e780d795775d2bfc0e545964c208
Diff	Diff

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexerCheckLengthHoisting - Duration of single invocation	90.34 ns	65.98 ns	0.73	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.IndexerCheckLengthHoisting

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.98101334247443 < 85.85209182158064.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 19.395276866876 (T) = (0 -66.17487509025203) / Math.Sqrt((185.09726096484044 / (299)) + (0.08046504963039612 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1877862062552652 = (81.47469988800718 - 66.17487509025203) / 81.47469988800718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	4354abd606119bd9fb0fdefa9f3c0f58f414bfa9
Compare	ddba2aafec61e780d795775d2bfc0e545964c208
Diff	Diff

Improvements in System.Collections.IterateFor<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IList - Duration of single invocation	3.36 μs	2.94 μs	0.88	0.31	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.IList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.942323408949174 < 3.1950694472724277.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/11/2023 4:46:06 PM, 2/2/2023 8:18:56 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsImprovementStdDev: Marked as improvement because 7.547948961682004 (T) = (0 -3165.6652359277673) / Math.Sqrt((82503.71538771075 / (299)) + (49465.51178201551 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10169886481724548 = (3524.057926614698 - 3165.6652359277673) / 3524.057926614698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository