Epic: GC Regions Support

[mangod9]

AB#1244342
We are switching segments to regions for managing memory on the GC heap. Segments have their own benefits, eg, checking if an address is in ephemeral requires comparing with 2 values (in Workstation case) but regions enable many optimizations that we've been wanting to have for a while. For example -

• Currently there’s no way to exchange memory between different generations without a compacting GC so if BGC builds up a lot of free space in gen2/LOH and gen2 has a lot more free spaces than LOH, there’s not a way to repurpose memory in gen2 for LOH. Another example is we often observe a lot of free spaces in gen0 due to pinning and we cannot repurpose it for other generations. With regions we can repurpose empty regions to any generation which will solve both problems. We could also choose to decommit these free spaces but that would require its own bookkeeping and we might as well just switch to regions.
• This will also help with container perf - because we repurpose free spaces to other generations, we can regulate the memory usage much more easily without incurring full compacting GCs.
• One of the evolutions I’d like to make in GC is to decouple the GC threads and their respective heaps and this will allow that to happen a lot more naturally.
• This will build a solid foundation for new flavors of GCs like incremental/concurrent compacting GCs.

Functionality

• Enable support wks
• Enable support srv
• Enable Regions by default in Standalone GC (clrgc.dll)

Testing

• P0/P1 Tests passing
• CI/CD InnerLoop Testing enabled with Regions
• CI/CD OuterLoop Testing enabled with Regions
• CI/CD Stress testing enabled

Stability

• Basic tests passing rate > 99%
• GCStress tests passing rate > 99%
• All tests passing rate > 99.9%

Performance

• TE Performance is comparable to segments
• Micro benchmarks dont show any major performance regressions.

Release

• CLRGC available within installers/gzips.

[ghost]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[En3Tho]

@mangod9 been wondering about this for a while. Any info you might share?)

[mangod9]

Hey @En3Tho, we are just getting started and will add more details as we make progress. If there are any specific details you are interested in please let us know. Thx!

[En3Tho]

@mangod9 Thank you for an answer. For now, I guess it would be really cool if you could at least briefly describe what this is about (a new under-the-hood mechanism to make GC even faster or a new kinda "give me a memory region, let me so my stuff and have it back", api, sorta like arraypool/malloc, but more general version.
For now we can only wonder.

[Maoni0]

it's my fault - I was supposed to write the description. I'll edit the 1st comment on the thread with the description.

[BreyerW]

Hoh last bullet sounds particularly interesting of course others sound nice too. The only thing sour is new flavors of gc are multiyears efforts on their own i suspect never mind switch to regions itself. Maybe im too pessimistic?

[tannergooding]

@Maoni0, do you think regions would make it easier to support custom alignments (such as 16 or 32-byte alignments) for objects/arrays?

Or would that still be relegated to the POH and extending the GC.AllocateArray method to support specifying alignments?

[Maoni0]

@tannergooding those are orthogonal features. the reason we haven't enabled alignment is just because we haven't had time due to other more urgent work. it's something I want to enable because it's clearly useful for some (very specific) scenarios. I know you care about it a lot 😃 it's in the queue... just haven't got there yet.

[mjsabby]

Will regions be helpful in supporting safe memory mapped files as byte arrays?

[Maoni0]

@mjsabby what kind of issues do you hit currently doing that? are you talking about ro segments or something else?

[mjsabby]

It’s not safe. It can be made safe if memory mapped files are backed by GC regions.

There was a discussion on GitHub with @GrabYourPitchforks and @jkotas

[Maoni0]

regions make no difference for that particular issue - I presume you are talking about #37227. I don't think GC is actually a big part in that issue - essentially from GC's POV, if it gets more ro segs it's not a big deal - we don't reclaim ro segs anyway (they can only be removed by the user code). all GC cares about is if an object points to something on the ro segs it needs to be a legit object. I saw that @jkotas already said the special treatment of Dispose was too expensive.

[mjsabby]

Today we look at the stack for by refs, couldn't we also look at another special location for memory mapped files? I thought GC regions could accomplish that in a more natural way, maybe not, and the discussion can then stay in that thread.

[jkotas]

@mjsabby Let's keep this discussion in #37227. Regions do not help with this at all.

[sirinath]

Regarding GC can a truly pauseless GC be introduced. E.g. Azul has C4: The Continuously Concurrent Compacting Collector

[InCerryGit]

Regarding GC can a truly pauseless GC be introduced. E.g. Azul has C4: The Continuously Concurrent Compacting Collector

i think so

[mangod9]

Regions are now enabled by default in main. Standalone GC has been switched to use segments to help with any backcompat issues. It can be configured using the following config (though clrgc.dll is not shipped with the sdk, and will need to be manually built if required):

set COMPLUS_GCName=clrgc.dll

[datduyng]

Does enabling "GC Regions" only support for .net7 preview3? is there a way to opt in to this in .net6?

[mangod9]

@datduyng, you could get it to work with .net 6 but will have to build runtime repo manually and use the built clrgc.dll with dotnet_gcname=clrgc.dll

[mangod9]

Closing this since the regions enablement is now complete.

[ivanjx]

@mangod9 i noticed lower memory usage but higher cpu usage with net 7 preview 6 asp net core compared with net 6 (observed with docker stats though). is this to be expected? thanks.

[mangod9]

hi @realivanjx, is there a repro you can share? With preview6 in fact we had a working set regression as noted here: #72557 which is fixed in preview7, but we havent observed higher cpu for our asp.net based benchmarks. Thanks.

[ivanjx]

@mangod9 not at the moment since it is the production program im using. im running it inside a docker with max memory of 300 MB so it is really a memory constrained environment (raspberry pi 4). i will try to create an empty asp net core project to see if it still happen. thanks

[mangod9]

Ok, could you also quantify how much increase in cpu utilization are you noticing? It could be specific to your scenario.

[ivanjx]

sorry i was completely blundered. the memory was actually increased. here is the top result (typical):
net 6

net 7

the nuget packages i use if that matters:

• MongoDB.Driver
• Serilog.AspNetCore
• Serilog.Sinks.Seq

[mangod9]

yeah that memory regression should hopefully be fixed in preview7. We will have to get more info to investigate the cpu regression.