[Perf] Windows/x64: 4 Improvements on 7/12/2023 7:15:00 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare	82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ICollection - Duration of single invocation 📝 - Benchmark Source	20.54 μs	18.85 μs	0.92	0.08	False
ImmutableList - Duration of single invocation 📝 - Benchmark Source	1.10 ms	869.29 μs	0.79	0.11	False
Stack - Duration of single invocation 📝 - Benchmark Source	23.96 μs	16.34 μs	0.68	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.846548382173385 < 19.51214253449408.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/2/2023 6:53:36 PM, 6/8/2023 8:20:44 PM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM.
IsImprovementStdDev: Marked as improvement because 5.144166141236766 (T) = (0 -17581.63591125001) / Math.Sqrt((2011000.4928781653 / (299)) + (550163.3337876791 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.06777464766231941 = (18859.856007092803 - 17581.63591125001) / 18859.856007092803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 869.2882060185184 < 1.0467333937499999.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM.
IsImprovementStdDev: Marked as improvement because 70.85391983089009 (T) = (0 -861598.652436212) / Math.Sqrt((3838963442.659013 / (299)) + (25753487.5389589 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.24786171068260357 = (1145532.2308057956 - 861598.652436212) / 1145532.2308057956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.339378057302586 < 22.55298928461792.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 7/5/2023 7:07:19 PM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM.
IsImprovementStdDev: Marked as improvement because 16.398701803166635 (T) = (0 -16537.089217502267) / Math.Sqrt((4823449.515724427 / (299)) + (9966.115817124439 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.11487260706186675 = (18683.287117132688 - 16537.089217502267) / 18683.287117132688 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare	82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Ackermann

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source	3.38 μs	2.90 μs	0.86	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Ackermann*'

Payloads

Baseline
Compare

Benchstone.BenchI.Ackermann.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.8996398133116887 < 3.2116435598733584.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM.
IsImprovementStdDev: Marked as improvement because 5.252670489960784 (T) = (0 -2961.3281476270595) / Math.Sqrt((20674.8104631033 / (299)) + (22729.702385258963 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.07905537641119224 = (3215.533346714299 - 2961.3281476270595) / 3215.533346714299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository