-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add --cpus context #267
Merged
Merged
Add --cpus context #267
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
bc4a845
Add --cpus context
richlander 172b30d
Add comment about .NET Framework
richlander 1558f19
Clarify CPU affinity
richlander e7527c9
Update accepted/2019/support-for-memory-limits.md
richlander 8aced4a
Update accepted/2019/support-for-memory-limits.md
richlander d6b79ac
Add more context
richlander 593dfa6
Add GCHeapCount
richlander 99192e0
Update accepted/2019/support-for-memory-limits.md
richlander 9f0970c
MiB -> MB
richlander 50cd55b
Update per feedback
richlander edd1bb9
Update index.md
richlander 92c93b1
Add runtime config doc link
richlander e900593
Update per feedbac
richlander File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,58 +1,125 @@ | ||
# .NET Core GC Support for Docker Limits | ||
# .NET GC Support for Container Limits | ||
|
||
**Owner** [Rich Lander](https://github.com/richlander) | ||
|
||
.NET Core has support for [control groups](https://en.wikipedia.org/wiki/Cgroups) (cgroups), which is the basis of [Docker limits](https://docs.docker.com/config/containers/resource_constraints/). We found that the algorithm we use to honor cgroups works well for larger memory size limits (for example, >500MB), but that it is not possible to configure a .NET Core application to run indefinitely at lower memory levels. This document proposes an approach to support low memory size limits, <100MB. | ||
.NET has support for [control groups (cgroups)](https://en.wikipedia.org/wiki/Cgroups) , which is the basis of [Docker resource limits](https://docs.docker.com/config/containers/resource_constraints/). .NET has supported cgroups since .NET Core 2.1. | ||
|
||
Note: Windows has a concept similar to cgroups called [job objects](https://docs.microsoft.com/windows/desktop/ProcThread/job-objects). .NET Core should honor job objects in the same way as cgroups, as appropriate. This document will focus on cgroups throughout. | ||
Windows has a concept similar to cgroups called [job objects](https://docs.microsoft.com/windows/desktop/ProcThread/job-objects). .NET 6+ correctly honors job objects in the same way as cgroups. This document will focus on cgroups throughout. | ||
|
||
It is critical to provide effective and well defined experiences when .NET Core applications are run within memory-limited cgroups. An application should run indefinitely given a sensible configuration for that application. We considered relying on orchestators to manage failing applications (that can no longer satisfy the configuration of a cgroup), but believe this to be antithetical as a primary solution for building reliable systems. We also expect that there are scenarios where orchestrators will be unavailable or primitive or hardware will be constrainted, and therefore not tolerant of frequently failing applications. As a result, we need a better tuned algorithm for cgroup support to the end of running reliable software within constrained environments. | ||
It is critical to provide effective and well-defined capabilities for .NET applications within memory-limited cgroups. An application should run indefinitely given a sensible configuration for that application. It is important that .NET developers have good controls to optimize their container hosted applications. Our goal is that certain classes of .NET applications can be run with <100 MB memory constraints. | ||
|
||
See [implementing hard limit for GC heap dotnet/coreclr #22180](https://github.com/dotnet/coreclr/pull/22180). | ||
Related: | ||
|
||
- [implementing hard limit for GC heap dotnet/coreclr #22180](https://github.com/dotnet/coreclr/pull/22180). | ||
- [Validate container improvements with .NET 6](https://github.com/dotnet/runtime/issues/53149). | ||
- [Runtime configuration options for garbage collection](https://docs.microsoft.com/dotnet/core/runtime-config/garbage-collector) | ||
|
||
## Cgroup constraints | ||
|
||
cgroups control two main resources: memory and cores. Both are relevant to the .NET GC. | ||
|
||
Memory constraints defines the maximum memory available to the cgroup. This memory is used by the guest operating system, the .NET runtime, the GC heap, and potentially other users. If a cgroup has `100 MB` available, the app will have less than that. The cgroup will be terminated (AKA `OOMKilled`) when the memory limit is reached. | ||
|
||
Core constraints determine how many GC heaps should be created, at maximum. The maximum heap value matches `Environment.ProcessorCount`. There are three primary ways that this value can be set (using the `docker` CLI to demonstrate): | ||
|
||
- Not specified -- `Environment.ProcessorCount` will match the total number of machine cores. | ||
- Via `--cpus` -- `Environment.ProcessorCount` uses that (decimal) value (rounded up to the next integer). | ||
- Via `--cpu-sets` -- `Environment.ProcessorCount` matches the count of specified CPUs. | ||
- `DOTNET_PROCESSOR_COUNT` -- `Environment.ProcessorCount` uses this value. If other values are also specified, they are ignored. | ||
|
||
In the general case, there will be one heap per core. If the GC creates too many heaps, that can over-eagerly use up the memory limit, at least in part. There are controls to avoid that. | ||
|
||
## GC Heap Hard Limit | ||
|
||
The following configuration knobs will be exposed to enable developers to configure their applications: | ||
The *GC Heap Hard Limit* is the maximum managed heap size. It only applies when running within a cgroup. By default, it is lower than the cgroup memory constraint (AKA "the cgroup hard limit"). | ||
|
||
* `GCHeapHardLimit` - specifies a hard limit for the GC heap as an absolute value | ||
* `GCHeapHardLimitPercent` - specifies a hard limit for the GC heap as a percentage of physical memory that the process is allowed to use | ||
The following configuration knobs are exposed to configure applications: | ||
|
||
* `GCHeapHardLimit` - specifies a hard limit for the GC heap as an absolute value, in bytes (hex value). | ||
* `GCHeapHardLimitPercent` - specifies a hard limit for the GC heap as a percentage of the cgroup hard limit (hex value). | ||
|
||
If both are specified, `GCHeapHardLimit` is used. | ||
|
||
The `GCHeapHardLimit` will be calculated using the following formular if it is not specified and the process is running inside a container (or cgroup or job object) with a memory limit specified: | ||
By default, the `GCHeapHardLimit` will be calculated using the following formula: | ||
|
||
```console | ||
max (20mb, 75% of the memory limit on the container) | ||
max (20 MB, 75% of the memory limit on the container) | ||
``` | ||
|
||
The GC will more aggressive perform GCs as the GC heap grows closer to the `GCHeapHardLimit` with the goal of making more memory available so that the application can continue to safely function. The GC will avoid continuously performing full blocking GCs if they are not considered productive. | ||
|
||
The GC will throw an `OutOfMemoryException` for allocations that would cause the committed heap size to exceed the `GCHeapHardLimit` memory size, even after a full compacting GC. | ||
|
||
## GC Heap Heap Minimum Size | ||
## GC Heap Count | ||
|
||
Using Server GC, there are multiple GC heaps created, up to one per core. This model doesn't scale well when a small memory limit is set on a machine with many cores. | ||
|
||
The minimum _reserved_ segment size per heap: 16mb | ||
The heap count can be set two ways: | ||
|
||
- Manually via `DOTNET_GCHeapCount`. | ||
- Automatically by the GC, relying on: | ||
- Number of observed or configured cores. | ||
- A minimum _reserved_ memory size per heap of `16 MB`. | ||
|
||
If [`DOTNET_PROCESSOR_COUNT`](https://github.com/dotnet/runtime/issues/48094) is set, including if it differs from `--cpus`, then the GC will use the ENV value for determining the maximum number of heaps to create. | ||
|
||
Note: [.NET Framework 4.8 and 4.8.1](https://github.com/microsoft/dotnet-framework-docker/discussions/935) have the same behavior but `COMPlus_RUNNING_IN_CONTAINER` must be set. Also processor count is affected (in the same way) by `COMPlus_PROCESSOR_COUNT`. | ||
|
||
Note: The next section talks about how many cores can be used by an application. It isn't defined in this doc, but is assumed to be per the [container runtime policy](https://docs.docker.com/config/containers/resource_constraints/#cpu). | ||
|
||
Let's look at some examples. They are also demonstrated in [Testing GC Heap Counts with Containers](https://github.com/dotnet/runtime/issues/71413). | ||
|
||
### Memory constrained; CPU unconstrained | ||
|
||
Example: | ||
```bash | ||
docker run --rm -m 256mb mcr.microsoft.com/dotnet/samples | ||
``` | ||
|
||
* 48 core machine | ||
* cgroup has a 200MB memory limit | ||
* cgroup has a 256 MB memory limit | ||
* cgroup has no CPU/core limit | ||
* 160MB `GCHeapHardLimit` | ||
* Server GC will create 10 GC heaps | ||
* 192 MB `GCHeapHardLimit` | ||
* Server GC will create 12 GC heaps, with 16 MB reserved memory | ||
* All 48 cores can be used by the application | ||
|
||
Example: | ||
`heaps = (256 * .75) / 16` | ||
`heaps = 12` | ||
|
||
### Memory and CPU constrained | ||
|
||
```bash | ||
docker run --rm -m 256mb --cpus 2 mcr.microsoft.com/dotnet/samples | ||
``` | ||
|
||
* 48 core machine | ||
* cgroup has a 200MB memory limit | ||
* cgroup has 4 CPU/core limit | ||
* 160MB `GCHeapHardLimit` | ||
* Server GC will create 4 GC heaps | ||
* Only 4 cores can be used by the application | ||
* cgroup has a 256 MB memory limit | ||
* cgroup has 2 CPU/core limit | ||
* 192 MB `GCHeapHardLimit` | ||
* Server GC will create 2 GC heaps | ||
* 2 cores can be used by the application | ||
|
||
### Memory and CPU constrained (with CPU affinity): | ||
|
||
```bash | ||
docker run --rm -m 256mb --cpuset-cpus 0,2,3 mcr.microsoft.com/dotnet/samples | ||
``` | ||
|
||
* 48 core machine | ||
* cgroup has a 256 MB memory limit | ||
* cgroup has 3 CPU/core limit | ||
* 192 MB `GCHeapHardLimit` | ||
* Server GC will create 3 GC heaps | ||
* 3 cores can be used by the application | ||
|
||
## Previous behavior | ||
### Memory and CPU constrained (overriden by `DOTNET_PROCESSOR_COUNT`): | ||
|
||
Previously, the **maximum GC heap size** matched the cgroup limit. | ||
```bash | ||
docker run --rm -m 256mb --cpus 2 -e DOTNET_PROCESSOR_COUNT=4 mcr.microsoft.com/dotnet/samples | ||
``` | ||
|
||
* 48 core machine | ||
* cgroup has a 256 MB memory limit | ||
* cgroup has 2 CPU/core limit | ||
* 192 MB `GCHeapHardLimit` | ||
* Server GC will create 4 GC heaps | ||
* 2 cores can be used by the application |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it make sense to add an example of what a
GCHeapHardLimit
hex value looks like? Does it have a prefix like0xf00d
orf00d
?For
GCHeapHardLimitPercent
, does it range from 0 to 100 (or hex0x0
to0x64
)?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have the full documentation with examples and other details at https://docs.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector
Should we move some of these edits to the official docs instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I think just having links to the full docs will be great!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added the link to the doc.