Connectd throttle #7366
Merged
Connectd throttle #7366
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cdecker
approved these changes
Jun 23, 2024
rustyrussell
force-pushed
the
connectd-throttle
branch
from
8a52e9e
to
3a19205
Compare
|
Simple rebase now prerequisites are merged! |
rustyrussell
force-pushed
the
connectd-throttle
branch
from
3a19205
to
572ed95
Compare
We currently stream gossip as fast as we can, even if they start at timestamp 0. Instead, use a simple token bucket filter and only let them have 1MB per second (500 bytes per second for testing). Signed-off-by: Rusty Russell <[email protected]> Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
gossipd no longer connects directly to per-peer daemons, so remove old file. Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
This is more efficient in a few ways: 1. It's trivial to get to the end of the gossip_store, we don't have to iterate. 2. It tends to be mmaped so we don't have to call pread(). Signed-off-by: Rusty Russell <[email protected]>
This was removed from the spec on Apr 25, 2022. We stopped ever sending them in 0.12.0 (2022-08-23). Now we remove receive support. Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses. Signed-off-by: Rusty Russell <[email protected]>
This basically means moving the code from gossipd to connectd to handle these queries. This will get connectd have finer control over ratelimiting them. Signed-off-by: Rusty Russell <[email protected]>
A bit tricky, since we get more than one message at a time. However, this just means we go over quota for a bit, and will get caught when those are sent (we do this for a single message already, so it's not that much worse). Note: this not only limits sending, but it limits the actuall query processing, which is nice. Signed-off-by: Rusty Russell <[email protected]>
For some reason CI now hits a race where it tries to analyze a still-being-generated file:
```
tools/headerversions.c:52:30: error: syntax error [syntaxError]
new = tal_fmt(NULL, template,
^
make: *** [Makefile:552: check-cppcheck] Error 123
```
Restrict it to real source files instead.
Signed-off-by: Rusty Russell <[email protected]>
rustyrussell
force-pushed
the
connectd-throttle
branch
from
6dc9a5d
to
360d2e5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Builds on #7365 and #7364
This makes connectd serve gossip and gossip query responses with a ratelimit of 1MB/second. This at least puts some limit on how fast they stream gossip from us. It also includes a few optimizations and removes support for decoding compressed scids (dropped from spec and implementations in 2022).