Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix expensive typo in JsonNode #78130

Merged
merged 1 commit into from
Nov 10, 2022

Conversation

stephentoub
Copy link
Member

Fixes #78089 (but assuming this is the right fix, we should backport this to release/7.0)

@ghost
Copy link

ghost commented Nov 9, 2022

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #78089 (but assuming this is the right fix, we should backport this to release/7.0)

Author: stephentoub
Assignees: -
Labels:

area-System.Text.Json

Milestone: -

@stephentoub
Copy link
Member Author

Using the repro benchmark from the linked issue:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Text.Json.Nodes;

[MemoryDiagnoser]
public partial class Program
{
    static void Main(string[] args) => BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);

    private JsonObject[] _jsonObjs = new JsonObject[100_000];

    [Benchmark]
    public void Sum()
    {
        for (int i = 0; i < 100_000; i++)
        {
            _jsonObjs[i] = new JsonObject()
            {
                ["aaa"] = 1,
                ["bbbb"] = 2,
                ["cccc"] = new JsonArray()
                    {
                        new JsonObject()
                        {
                            ["aaaa"] = "aaaaa",
                            ["ffffff"] = "ffffffffffffff"
                        }
                    }
            };
        }
    }
}
Method Runtime Mean Error StdDev Ratio Allocated
Sum .NET 6.0 46.32 ms 0.926 ms 1.067 ms 1.10 127.41 MB
Sum .NET 7.0 292.87 ms 5.241 ms 7.174 ms 6.89 313.13 MB
Sum pr 42.42 ms 0.617 ms 0.547 ms 1.00 128.94 MB

@teo-tsirpanis teo-tsirpanis added this to the 8.0.0 milestone Nov 9, 2022
@EgorBo
Copy link
Member

EgorBo commented Nov 9, 2022

I assume this scenario is not covered in the dotnet/performance suite or we somehow missed this.

Copy link
Member

@eiriktsarpalis eiriktsarpalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, fix is spot on. We should backport this.

@eiriktsarpalis
Copy link
Member

Test failures are unrelated, mergin away.

@eiriktsarpalis eiriktsarpalis merged commit b00a4ee into dotnet:main Nov 10, 2022
@eiriktsarpalis
Copy link
Member

/backport to release/7.0

@github-actions
Copy link
Contributor

Started backporting to release/7.0: https://github.com/dotnet/runtime/actions/runs/3435456807

@adamsitnik
Copy link
Member

@stephentoub we have added the "Allocation ratio" column to BDN based on your request: dotnet/BenchmarkDotNet#722 and I can see that you have removed it here while I would expect this particular issue to be perfect use case for it. May I ask why? Does the implementation do not meet your requirements?

@stephentoub
Copy link
Member Author

I can see that you have removed it here while I would expect this particular issue to be perfect use case for it. May I ask why?

Because in this case I was focusing on the throughput impact rather than on the allocation impact, and removed extra columns to try to focus attention (e.g. RatioSD, Gen0, and Alloc Ratio). The 6x improvement in throughput was the primary thing to show, rather than the 2x reduction in allocation.

@stephentoub stephentoub deleted the fixstaticoptions branch November 29, 2022 15:09
@dakersnar
Copy link
Contributor

I can confirm that the newly added unit tests (dotnet/performance#2714) are shown as heavily regressing in the .NET 7 vs 6 GA performance report.

Is there anything more that needs to be done to confirm we have addressed the problem with this backport, or are we confident this is a complete fix?

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 0.31 +12534 Windows 11 Arm64 Microsoft SQ1 3.0 GHz bimodal
Slower 0.43 +12970 macOS 13.0 Arm64 Apple M1
Slower 0.34 +12660 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.26 +12442 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower 0.30 +12568 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) several?
Slower 0.38 +12563 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 0.33 +12660 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.28 +7782 raspbian 11 Arm ARMv7 Processor rev 3 (v7l)
Slower 0.27 +12513 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings)

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 0.36 +12469 Windows 11 Arm64 Microsoft SQ1 3.0 GHz several?
Slower 0.46 +12995 macOS 13.0 Arm64 Apple M1
Slower 0.37 +12691 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz bimodal
Slower 0.29 +12533 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower 0.33 +12578 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 0.42 +12552 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 0.39 +12652 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.29 +7724 raspbian 11 Arm ARMv7 Processor rev 3 (v7l)
Slower 0.30 +12551 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB)

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 0.43 +2891666 Windows 11 Arm64 Microsoft SQ1 3.0 GHz bimodal
Slower 0.56 +2887897 macOS 13.0 Arm64 Apple M1
Slower 0.49 +2913026 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz bimodal
Slower 0.45 +2964348 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower 0.39 +2765957 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 0.58 +2912668 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 0.48 +2937050 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.55 +1675483 raspbian 11 Arm ARMv7 Processor rev 3 (v7l) several?
Slower 0.42 +2938707 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 0.51 +10563 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 0.61 +10136 macOS 13.0 Arm64 Apple M1
Slower 0.55 +10666 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.46 +10542 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower 0.51 +10581 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 0.58 +10596 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 0.50 +10711 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.44 +6202 raspbian 11 Arm ARMv7 Processor rev 3 (v7l)
Slower 0.45 +10558 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

(etc for all other benchmarks)

@eiriktsarpalis
Copy link
Member

Yes, we're fairly confident that the regression has been addressed. I've verified that the particular benchmark has parity between .NET 6 and the patched .NET 7 branch.

@ghost ghost locked as resolved and limited conversation to collaborators Jan 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

.net 7 cycles Performance slow
6 participants