Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Stream.CopyTo{Async} in NonCryptographicHashAlgorithm.Append{Async} #103669

Merged
merged 1 commit into from
Jun 19, 2024

Conversation

stephentoub
Copy link
Member

@stephentoub stephentoub commented Jun 18, 2024

Rather than using a manual loop to copy data from a stream to a buffer, use the Stream.CopyTo{Async} method. Downside is there's an extra 32 byte allocation for the temporary destination passthrough Stream object. Upside is it allows the source Stream's to optimize via its CopyTo{Async} override, e.g. MemoryStream.CopyTo just makes a single Write call with its buffer. The base Stream.CopyTo also uses a larger rented buffer size than these APIs were using.

Related to #103539

Before:

Method Kind Mean Allocated
Hash MemoryStream 79.76 us 568 B
Hash FileStream 837.08 us 568 B

After:

Method Kind Mean Allocated
Hash MemoryStream 59.76 us 600 B
Hash FileStream 131.51 us 600 B
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.IO.Hashing;
using System.Security.Cryptography;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[MemoryDiagnoser]
[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
public class Tests
{
    private Stream _stream;
    private byte[] _bytes = RandomNumberGenerator.GetBytes(1024 * 1024);

    [Params("MemoryStream", "FileStream")]
    public string Kind { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        switch (Kind)
        {
            case "MemoryStream":
                _stream = new MemoryStream(_bytes);
                break;

            case "FileStream":
                string path = Path.GetRandomFileName();
                File.WriteAllBytes(path, _bytes);
                _stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 0, FileOptions.DeleteOnClose);
                break;
        }
    }

    [GlobalCleanup]
    public void Cleanup() => _stream.Dispose();

    [Benchmark]
    public ulong Hash()
    {
        _stream.Position = 0;
        var hash = new XxHash3();
        hash.Append(_stream);
        return hash.GetCurrentHashAsUInt64();
    }
}

@stephentoub stephentoub merged commit f8e575c into dotnet:main Jun 19, 2024
80 of 83 checks passed
@stephentoub stephentoub deleted the hashcopyto branch June 19, 2024 23:46
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io-hashing, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants