Simplify libp2p ResourceMgr computed defaults (no System.StreamsInbound limits and allow more System.ConnsInbound per MB)

core/node/libp2p/rcmgr_defaults.go

@@ -44,7 +44,7 @@ var noLimitIncrease = rcmgr.BaseLimitIncrease{
 // This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled
 
 // createDefaultLimitConfig creates LimitConfig to pass to libp2p's resource manager.
-// The defaults follow the documentation in docs/config.md.
+// The defaults follow the documentation in docs/libp2p-resource-management.md.
 // Any changes in the logic here should be reflected there.
 func createDefaultLimitConfig(cfg config.SwarmConfig) (rcmgr.LimitConfig, error) {
 	maxMemoryDefaultString := humanize.Bytes(uint64(memory.TotalMemory()) / 4)
@@ -72,33 +72,28 @@ Run 'ipfs swarm limit all' to see the resulting limits.
 			Memory: int64(maxMemory),
 			FD:     int(numFD),
 
-			// By default, we just limit connections on the inbound side.
+			// At least as of 2023-01-25, it's possible to open a connection that 
+			// doesn't for any memory usage with the libp2p Resource Manager/Accountant
+			// (see https://github.com/libp2p/go-libp2p/issues/2010#issuecomment-1404280736).
+			// As a result, we can't curretly rely on Memory limits to full protect us.
+			// Until https://github.com/libp2p/go-libp2p/issues/2010 is addressed, 
+			// we take a proxy now of restricting to 1 inbound connection per MB.
+			// Note: this is more generous than go-libp2p's default autoscaled limits which do
+			// 64 connections per 1GB
+			// (see https://github.com/libp2p/go-libp2p/blob/master/p2p/host/resource-manager/limit_defaults.go#L357 ).
+			ConnsInbound:  maxMemory >> 20, // Steve: I assume this isn't valid Go code.
+
 			Conns:         bigEnough,
-			ConnsInbound:  rcmgr.DefaultLimits.SystemBaseLimit.ConnsInbound, // same as libp2p default
 			ConnsOutbound: bigEnough,
-
-			// We limit streams since they not only take up memory and CPU.
-			// The Memory limit protects us on the memory side,
-			// but a StreamsInbound limit helps protect against unbound CPU consumption from stream processing.
 			Streams:         bigEnough,
-			StreamsInbound:  rcmgr.DefaultLimits.SystemBaseLimit.StreamsInbound,
+			StreamsInbound:  bigEnough,
 			StreamsOutbound: bigEnough,
 		},
-		// Most limits don't see an increase because they're already infinite/bigEnough or at their max value.
-		// The values that should scale based on the amount of memory allocated to libp2p need to increase accordingly.
-		SystemLimitIncrease: rcmgr.BaseLimitIncrease{
-			Memory:     0,
-			FDFraction: 0,
-
-			Conns:         0,
-			ConnsInbound:  rcmgr.DefaultLimits.SystemLimitIncrease.ConnsInbound,
-			ConnsOutbound: 0,
-
-			Streams:         0,
-			StreamsInbound:  rcmgr.DefaultLimits.SystemLimitIncrease.StreamsInbound,
-			StreamsOutbound: 0,
-		},
+		SystemLimitIncrease: noLimitIncrease,
 
+		// Transient connections won't cause any memory to accounted for by the resource manager.
+		// Only established connections do.
+		// As a result, we can't rely on System.Memory to protect us from a bunch of transient connection being opened.
 		TransientBaseLimit: rcmgr.BaseLimit{
 			Memory: rcmgr.DefaultLimits.TransientBaseLimit.Memory,
 			FD:     rcmgr.DefaultLimits.TransientBaseLimit.FD,
@@ -111,7 +106,6 @@ Run 'ipfs swarm limit all' to see the resulting limits.
 			StreamsInbound:  rcmgr.DefaultLimits.TransientBaseLimit.StreamsInbound,
 			StreamsOutbound: bigEnough,
 		},
-
 		TransientLimitIncrease: rcmgr.BaseLimitIncrease{
 			Memory:     rcmgr.DefaultLimits.TransientLimitIncrease.Memory,
 			FDFraction: rcmgr.DefaultLimits.TransientLimitIncrease.FDFraction,
@@ -152,6 +146,7 @@ Run 'ipfs swarm limit all' to see the resulting limits.
 		StreamBaseLimit:     infiniteBaseLimit,
 		StreamLimitIncrease: noLimitIncrease,
 
+		// Steve 2023-01-24: I don't see any reason to change this.
 		// Limit the resources consumed by a peer.
 		// This doesn't protect us against intentional DoS attacks since an attacker can easily spin up multiple peers.
 		// We specify this limit against unintentional DoS attacks (e.g., a peer has a bug and is sending too much traffic intentionally).
@@ -181,6 +176,7 @@ Run 'ipfs swarm limit all' to see the resulting limits.
 		},
 	}
 
+	// Steve 2023-01-24: I don't see any reasons to change this.
 	// Whatever limits libp2p has specifically tuned for its protocols/services we'll apply.
 	libp2p.SetDefaultServiceLimits(&scalingLimitConfig)
 

docs/changelogs/v0.18.1

@@ -0,0 +1,36 @@
+# Kubo changelog v0.18
+
+## v0.18.1
+
+### Overview
+
+Below is an outline of all that is in this release, so you get a sense of all that's included.
+
+<!-- TOC depthfrom:3 -->
+
+- [Overview](#overview)
+- [🔦 Highlights](#-highlights)
+    - [Improving libp2p resource management integration](#improving-libp2p-resource-management-integration)
+- [📝 Changelog](#-changelog)
+- [👨‍👩‍👧‍👦 Contributors](#-contributors)
+
+<!-- /TOC -->
+
+### 🔦 Highlights
+
+#### Improving libp2p resource management integration
+This builds on the default protection nodes get against DoS (resource exhaustion) and eclipse attacks
+with the [go-libp2p Network Resource Manager/Accountant](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md)
+that was fine-tuned in [Kubo 0.18](https://github.com/ipfs/kubo/blob/biglep/resource-manager-example-of-what-want/docs/changelogs/v0.18.md#improving-libp2p-resource-management-integration).
+
+Adding default hard-limits from the Resource Manager/Accountant after the fact is tricky,
+and some additional improvements have been made to improve the [computed defaults](https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md#computed-default-limits).
+As much as possible, the aim is for a user to only think about how much memory they want to bound libp2p to, 
+and not need to think about translating that to hard numbers for connections, streams, etc.
+More updates are likely in future Kubo releases, but with this release: 
+1.``System.StreamsInbound`` is no longer bounded directly
+2. ``System.ConnsInbound`` has higher default computed values.
+
+### 📝 Changelog
+
+### 👨‍👩‍👧‍👦 Contributors

docs/libp2p-resource-management.md

@@ -70,8 +70,7 @@ The reason these scopes are chosen is because:
 
 Within these scopes, limits are just set on
 [memory](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#memory),
-[file descriptors (FD)](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#file-descriptors), [*inbound* connections](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#connections),
-and [*inbound* streams](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#streams).
+[file descriptors (FD)](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#file-descriptors), and [*inbound* connections](https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#connections).
 Limits are set based on the `Swarm.ResourceMgr.MaxMemory` and `Swarm.ResourceMgr.MaxFileDescriptors` inputs above.
 
 There are also some special cases where minimum values are enforced.
@@ -89,7 +88,6 @@ These become the [active limits](#how-does-one-see-the-active-limits).
 
 While `Swarm.ResourceMgr.Limits` can be edited directly, it is also possible to use `ipfs swarm limit` command to inspect and tweak specific limits at runtime.
 
-
 To see all resources that are close to hitting their respective limit:
 
 ```console