Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regression in QuickSortSpan / Perf_Integer.ToByteArray benchmark #59414

Closed
Tracked by #64603
performanceautofiler bot opened this issue Sep 21, 2021 · 6 comments
Closed
Tracked by #64603

Regression in QuickSortSpan / Perf_Integer.ToByteArray benchmark #59414

performanceautofiler bot opened this issue Sep 21, 2021 · 6 comments
Labels
arch-x64 area-System.Runtime os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
7.0.0

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Sep 21, 2021 •
edited by DrewScoggins
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
QuickSortSpan - Duration of single invocation 7.94 μs 8.91 μs 1.12 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Histogram

Span.Sorting.QuickSortSpan(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToByteArray - Duration of single invocation 50.23 ns 67.39 ns 1.34 0.01 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf16 - Duration of single invocation 609.53 ns 649.28 ns 1.07 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 366.22 ns 599.00 ns 1.64 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_NoDecodingRequired - Duration of single invocation 57.48 ns 61.60 ns 1.07 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InterfaceInterfaceMethodLongHierarchy - Duration of single invocation 266.95 μs 306.76 μs 1.15 0.03 False
GenericClassGenericStaticField - Duration of single invocation 45.59 μs 59.65 μs 1.31 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy

PerfLabTests.LowLevelPerf.GenericClassGenericStaticField

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@kunalspathak kunalspathak changed the title [Perf] Changes at 9/16/2021 5:40:24 PM Regression in QuickSortSpan / Perf_Integer.ToByteArray benchmark Sep 21, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Sep 21, 2021
@kunalspathak kunalspathak transferred this issue from dotnet/perf-autofiling-issues Sep 21, 2021
@kunalspathak
Copy link
Member

Mostly because of #57970

@kunalspathak kunalspathak added arch-x64 os-linux Linux OS (any supported distro) labels Sep 21, 2021
@DrewScoggins DrewScoggins added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark labels Sep 21, 2021
@ghost
Copy link

ghost commented Sep 22, 2021

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
QuickSortSpan - Duration of single invocation 7.94 μs 8.91 μs 1.12 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Histogram

Span.Sorting.QuickSortSpan(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToByteArray - Duration of single invocation 50.23 ns 67.39 ns 1.34 0.01 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf16 - Duration of single invocation 609.53 ns 649.28 ns 1.07 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 366.22 ns 599.00 ns 1.64 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_NoDecodingRequired - Duration of single invocation 57.48 ns 61.60 ns 1.07 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 909880b5a7a737f86d388f263e0bfa8c2ddb186c
Compare fe50d91a99dc17f8b3cbbb18103a7e8dca051075
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InterfaceInterfaceMethodLongHierarchy - Duration of single invocation 266.95 μs 306.76 μs 1.15 0.03 False
GenericClassGenericStaticField - Duration of single invocation 45.59 μs 59.65 μs 1.31 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy

PerfLabTests.LowLevelPerf.GenericClassGenericStaticField

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: -
Labels:

area-System.Runtime, os-linux, tenet-performance, tenet-performance-benchmarks, arch-x64, untriaged
Milestone: -

@jeffhandley jeffhandley added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration and removed untriaged New issue has not been triaged by the area owner labels Sep 30, 2021
@jeffhandley jeffhandley added this to the 6.0.0 milestone Sep 30, 2021
@AndyAyersMS
Copy link
Member

@jeffhandley this is an issue in main, so post 6.0, I believe.

@jeffhandley jeffhandley modified the milestones: 6.0.0, 7.0.0 Oct 5, 2021
@jeffhandley
Copy link
Member

Thanks. Moved to 7.0.0.

@jeffhandley jeffhandley mentioned this issue Feb 1, 2022
Closed
28 tasks
@dakersnar
Copy link
Contributor

The BigInteger one should be fixed by the eventual refactor. The first one seems to have gone down and is no longer significant enough to warrant investigation.

@ghost ghost locked as resolved and limited conversation to collaborators Sep 8, 2022
@tannergooding tannergooding removed the needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration label Jun 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-x64 area-System.Runtime os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
7.0.0
Development

No branches or pull requests
7 participants
@jeffhandley @DrewScoggins @jeffschwMSFT @AndyAyersMS @tannergooding @kunalspathak @dakersnar