Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 11/8/2021 4:08:09 PM #2173

Closed
performanceautofiler bot opened this issue Nov 9, 2021 · 2 comments
Closed

[Perf] Changes at 11/8/2021 4:08:09 PM #2173

performanceautofiler bot opened this issue Nov 9, 2021 · 2 comments
Assignees
@naricc
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 9, 2021 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db042594240709d61e6e9fc44c0d5b88ccb7a579
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 156.06 ns 1.21 μs 7.76 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db042594240709d61e6e9fc44c0d5b88ccb7a579
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAny - Duration of single invocation 113.25 ns 435.40 ns 3.84 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.IndexOfAny

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5fc05889a8f7afe2af98bc13edf053786b34aa3e
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSpanEmptyLoop - Duration of single invocation 2.50 μs 4.27 μs 1.71 0.01 False
ReadReturnBytes - Duration of single invocation 7.84 μs 13.38 μs 1.71 0.00 True
ReadReturnBytes - Duration of single invocation 3.82 μs 5.75 μs 1.51 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 4.25 μs 5.64 μs 1.33 0.00 True
ReadReturnBytes - Duration of single invocation 5.94 μs 7.19 μs 1.21 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: DeepTree)

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfStrings)

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfNumbers)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dac1290d4659a60e44cde2db38d07fb982f9a6e7
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Average - Duration of single invocation 491.04 ns 638.43 ns 1.30 0.45 False
AnyWithPredicate_LastElementMatches - Duration of single invocation 649.12 ns 727.42 ns 1.12 0.34 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable)

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dac1290d4659a60e44cde2db38d07fb982f9a6e7
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in BenchmarksGame.RegexRedux_5

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 97.67 ms 122.05 ms 1.25 0.55 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.RegexRedux_5*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.RegexRedux_5.RunBench(options: None)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db042594240709d61e6e9fc44c0d5b88ccb7a579
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 81.09 ns 605.79 ns 7.47 0.01 False
IndexOfAnyTwoValues - Duration of single invocation 324.91 ns 1.48 μs 4.55 0.01 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dac1290d4659a60e44cde2db38d07fb982f9a6e7
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf - Duration of single invocation 12.66 ns 13.89 ns 1.10 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db042594240709d61e6e9fc44c0d5b88ccb7a579
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 127.34 μs 140.63 μs 1.10 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dac1290d4659a60e44cde2db38d07fb982f9a6e7
Compare 704f023147b49692cec842427b116e9ee7e64305
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 57.36 μs 60.48 μs 1.05 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.SortedDictionary(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added LLVM=false PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Nov 9, 2021
@naricc
Copy link

naricc commented Nov 13, 2021

Something in this span of changes: dotnet/runtime@a50e1e6...ff135fe

Again seeing older dates; why?

@naricc naricc self-assigned this Nov 13, 2021
@naricc naricc mentioned this issue Nov 15, 2021
Closed
@fanyang-mono
Copy link
Member

Some of these regressions happened between Sept 6th - Nov 8th. Some of these regressions happened exactly on Sept 30th. Some of these regressions happened between Oct 14th - Nov 8th. And some of them doesn't look like regressions. It would be helpful for further investigating, if we could split them into sub-categories.

@naricc naricc closed this as completed Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@naricc naricc

Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@naricc @fanyang-mono