Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 3 Regressions in LINQ #96852

Closed
performanceautofiler bot opened this issue Jan 11, 2024 · 4 comments · Fixed by #97458
Closed

[Perf] Linux/arm64: 3 Regressions in LINQ #96852

performanceautofiler bot opened this issue Jan 11, 2024 · 4 comments · Fixed by #97458
Labels
arch-arm64 area-System.Linq os-linux Linux OS (any supported distro) PGO runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
9.0.0

Comments

@performanceautofiler
Copy link

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 21b4a8585362c1bc12d545b63e62a0d9dd4e8673
Compare b4ec422386bd358602c7693ff4ab28c96a806ec0
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.00 μs 2.10 μs 1.05 0.01 False
1.99 μs 2.11 μs 1.06 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 21b4a8585362c1bc12d545b63e62a0d9dd4e8673
Compare b4ec422386bd358602c7693ff4ab28c96a806ec0
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.81 μs 6.22 μs 1.29 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added arch-arm64 os-linux Linux OS (any supported distro) PGO runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels Jan 11, 2024
@performanceautofiler performanceautofiler bot mentioned this issue Jan 11, 2024
Closed
@EgorBo EgorBo changed the title [Perf] Linux/arm64: 3 Regressions on 1/8/2024 10:15:44 PM [Perf] Linux/arm64: 3 Regressions in LINQ Jan 11, 2024
@EgorBo EgorBo transferred this issue from dotnet/perf-autofiling-issues Jan 11, 2024
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jan 11, 2024
@EgorBo
Copy link
Member

EgorBo commented Jan 11, 2024

Commit range abffa8f...17335f4

We suspect it's #96570 cc @stephentoub

@EgorBo EgorBo added area-System.Linq tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Jan 11, 2024
@ghost
Copy link

ghost commented Jan 11, 2024

Tagging subscribers to this area: @dotnet/area-system-linq
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 21b4a8585362c1bc12d545b63e62a0d9dd4e8673
Compare b4ec422386bd358602c7693ff4ab28c96a806ec0
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.00 μs 2.10 μs 1.05 0.01 False
1.99 μs 2.11 μs 1.06 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 21b4a8585362c1bc12d545b63e62a0d9dd4e8673
Compare b4ec422386bd358602c7693ff4ab28c96a806ec0
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.81 μs 6.22 μs 1.29 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: -
Labels:

arch-arm64, area-System.Linq, os-linux, tenet-performance, tenet-performance-benchmarks, untriaged, runtime-coreclr, PGO
Milestone: -

@stephentoub
Copy link
Member

Which test has a regression?

@EgorBo
Copy link
Member

EgorBo commented Jan 11, 2024

Which test has a regression?

FirstWithPredicate_LastElementMatches

https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu%2020.04/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20IOrderedEnumerable).html

basically, first two, the last one must be noise.

@eiriktsarpalis eiriktsarpalis removed the untriaged New issue has not been triaged by the area owner label Jan 12, 2024
@eiriktsarpalis eiriktsarpalis added this to the 9.0.0 milestone Jan 12, 2024
This was referenced Jan 18, 2024
Closed
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jan 24, 2024
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jan 27, 2024
@cincuranet cincuranet mentioned this issue Jan 30, 2024
Closed
@github-actions github-actions bot locked and limited conversation to collaborators Feb 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-arm64 area-System.Linq os-linux Linux OS (any supported distro) PGO runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
9.0.0
Development

Successfully merging a pull request may close this issue.

Tweak Enumerable.ToArray to reduce some overheads stephentoub/runtime
3 participants
@EgorBo @stephentoub @eiriktsarpalis