[API Proposal]: ConcurrentDictionary.Clear(bool noResize)

[benaadams]

Background and motivation

ConcurrentDictionary<TKey, TValue> allows setting the initial size; however when you call .Clear() it resets it to size 31

This is a problem if your expected size is 1 million; and it now goes through very many increasingly expensive resizes getting back to its previous size

API Proposal

namespace System.Collections.Generic;

public class ConcurrentDictionary<TKey, TValue>
{
    public void Clear(bool noResize);
}

If noResize is true it should use the current length of _tables._buckets.Length rather than HashHelpers.GetPrime(DefaultCapacity) when recreating the arrays

runtime/src/libraries/System.Collections.Concurrent/src/System/Collections/Concurrent/ConcurrentDictionary.cs

Lines 723 to 726 in 9a31a5b

	Tables tables = _tables;
	var newTables = new Tables(new VolatileNode[HashHelpers.GetPrime(DefaultCapacity)], tables._locks, new int[tables._countPerLock.Length], tables._comparer);
	_tables = newTables;
	_budget = Math.Max(1, newTables._buckets.Length / newTables._locks.Length);

API Usage

// Fancy the value
var c = new ConcurrentDictionary<long, long>(concurrencyLevel: Environment.ProcessorCount * 16, capacity: 4_000_000);


c.Clear(noResize: true);

Alternative Designs

No response

Risks

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[eiriktsarpalis]

Could we entertain the possibility of making this the default behaviour for Clear? Dictionary.Clear doesn't resize either and we use TrimExcess if there's an explicit need to downsize.

[benaadams]

Could we entertain the possibility of making this the default behaviour for Clear? Dictionary.Clear doesn't resize either and we use TrimExcess if there's an explicit need to downsize.

I'd be happy with that

[eiriktsarpalis]

We would probably need to add corresponding ConcurrentDictionary.TrimExcess(...) methods to unblock people relying on the current behavior. Paging @stephentoub @jkotas in case there are any objections.

[karakasa]

Will the breaking change actually break anything other than GC timing?

[stephentoub]

it now goes through very many increasingly expensive resizes getting back to its previous size

I believe it'd be ~14 to get up to 1,000,000.

Could we entertain the possibility of making this the default behaviour for Clear? Dictionary.Clear doesn't resize either and we use TrimExcess if there's an explicit need to downsize.

ConcurrentDictionary is different. It has to allocate a new array as part of Clear, which is not the case for Dictionary, and why it prefers to allocate one of the default size rather than the current size, since it has no information to suggest that the capacity after clearing will get to be anywhere close to where it was. For the same reason, I'm not a fan of "noResize"; that would imply it's somehow able to keep the current capacity, but it's not.

I'd be ok considering storing the initial user provided capacity if one was provided and having Clear prefer to use that. But I don't think we should just have Clear always allocate a new array of length equal to the current array's length.

[eiriktsarpalis]

I'd be ok considering storing the initial user provided capacity if one was provided and having Clear prefer to use that.

@benaadams would this address your use case?

[benaadams]

I'd be ok considering storing the initial user provided capacity if one was provided and having Clear prefer to use that.

@benaadams would this address your use case?

Yes