Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pool collections during tagging. #73708

Merged
merged 11 commits into from
May 25, 2024

Conversation

CyrusNajmabadi
Copy link
Member

@CyrusNajmabadi CyrusNajmabadi commented May 25, 2024

Followup to #73703.

Drops memory allocations from:

image

to just

image

A dropping from 98mb of allocs in the ide tagger to just 20 (an 80% decrease).

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-IDE untriaged Issues and PRs which have not yet been triaged by a lead labels May 25, 2024
@@ -171,6 +172,7 @@ private sealed partial class TagSource
_dataSource = dataSource;
_asyncListener = asyncListener;
_nonFrozenComputationCancellationSeries = new(_disposalTokenSource.Token);
_tagSpanSetPool = new ObjectPool<HashSet<ITagSpan<TTag>>>(() => new HashSet<ITagSpan<TTag>>(this), trimOnFree: false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs to be an instance member as it creates hashsets that use 'this' as the special IEqualityComparer to compare tag-spans.

=> x != null && y != null && x.Span == y.Span && _dataSource.TagEquals(x.Tag, y.Tag);
{
if (x == y)
return true;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this clause was added as a fast path. The rest was just broken into separate clauses.

var buffersToTag = spansToTag.Select(dss => dss.SnapshotSpan.Snapshot.TextBuffer).ToSet();
var newTagsByBuffer =
context.TagSpans.Where(ts => buffersToTag.Contains(ts.Span.Snapshot.TextBuffer))
.ToLookup(t => t.Span.Snapshot.TextBuffer);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lots of wasteful linq/enumerator code here. it all got changes to simple code using pools.

@CyrusNajmabadi CyrusNajmabadi changed the title WIP: pool collections during tagging. Pool collections during tagging. May 25, 2024
@CyrusNajmabadi
Copy link
Member Author

@ToddGrun this is ready for review.

@@ -24,13 +24,14 @@ namespace Microsoft.CodeAnalysis.Editor.Shared.Tagging;
internal sealed partial class TagSpanIntervalTree<TTag>(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: my plan fo rthis type is to get rid of it, inlining its underlying IntervalTree into the tagger directly. The helpers we have here would then be static helper methods in the tagger as they're all tagger specific helpers (most of which can be improved).

@CyrusNajmabadi
Copy link
Member Author

@ToddGrun this isready for review.

@CyrusNajmabadi CyrusNajmabadi marked this pull request as ready for review May 25, 2024 16:11
@CyrusNajmabadi CyrusNajmabadi requested a review from a team as a code owner May 25, 2024 16:11
@@ -152,7 +152,7 @@ public static void ClearAndFree<T>(this ObjectPool<HashSet<T>> pool, HashSet<T>
var count = set.Count;
set.Clear();

if (count > Threshold)
if (count > Threshold && pool.TrimOnFree)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (count > Threshold && pool.TrimOnFree)

Should the other ClearAndFree methods be doing this too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably :) i can try to address in followup pr.

if (noSpansToInvalidate)
{
// If we have no spans to invalidate, then we can just keep the old tags and add the new tags.
var oldTagsToKeep = oldTagTree.GetSpans(newTags.First().Span.Snapshot);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.First()

nit: [0]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ArrayBuilder.First() is fine. it's not linq :)

Copy link
Contributor

@ToddGrun ToddGrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@CyrusNajmabadi CyrusNajmabadi merged commit 386b920 into dotnet:main May 25, 2024
25 checks passed
@CyrusNajmabadi CyrusNajmabadi deleted the treeEfficiency branch May 25, 2024 17:26
@dotnet-policy-service dotnet-policy-service bot added this to the Next milestone May 25, 2024
@Cosifne Cosifne removed this from the Next milestone May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-IDE untriaged Issues and PRs which have not yet been triaged by a lead
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants