Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connectd throttle #7366

Merged
merged 8 commits into from
Jul 10, 2024
Merged

Connectd throttle #7366

merged 8 commits into from
Jul 10, 2024

Commits on Jul 9, 2024

  1. connectd: throttle streaming gossip for peers. 

    We currently stream gossip as fast as we can, even if they start at
timestamp 0.  Instead, use a simple token bucket filter and only let
them have 1MB per second (500 bytes per second for testing).

Signed-off-by: Rusty Russell <[email protected]>
Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    753c53e View commit details
    Browse the repository at this point in the history

  2. gossipd: delete empty gossipd_peerd_wire.csv 

    gossipd no longer connects directly to per-peer daemons, so remove old file.

Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    7f830ec View commit details
    Browse the repository at this point in the history

  3. gossmap: add linear streaming interface. 

    Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    3b0d2d3 View commit details
    Browse the repository at this point in the history

  4. connectd: use gossmap streaming interface. 

    This is more efficient in a few ways:
1. It's trivial to get to the end of the gossip_store, we don't have
   to iterate.
2. It tends to be mmaped so we don't have to call pread().

Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    cda48e8 View commit details
    Browse the repository at this point in the history

  5. gossipd: drop zlib support. 

    This was removed from the spec on Apr 25, 2022.  We stopped ever sending them
in 0.12.0 (2022-08-23).  Now we remove receive support.

Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses.
Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    223f4b0 View commit details
    Browse the repository at this point in the history

  6. connectd: do response to gossip queries, don't hand them to gossipd. 

    This basically means moving the code from gossipd to connectd to handle
these queries.

This will get connectd have finer control over ratelimiting them.

Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    a275e6e View commit details
    Browse the repository at this point in the history

  7. connectd: tie gossip query responses into ratelimiting code. 

    A bit tricky, since we get more than one message at a time.  However,
this just means we go over quota for a bit, and will get caught when
those are sent (we do this for a single message already, so it's not
that much worse).

Note: this not only limits sending, but it limits the actuall query
processing, which is nice.

Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    68eb7c1 View commit details
    Browse the repository at this point in the history

  8. Makefile: only run cppcheck on non-generated files. 

    For some reason CI now hits a race where it tries to analyze a still-being-generated file:

```
tools/headerversions.c:52:30: error: syntax error [syntaxError]
 new = tal_fmt(NULL, template,
                             ^
make: *** [Makefile:552: check-cppcheck] Error 123
```

Restrict it to real source files instead.

Signed-off-by: Rusty Russell <[email protected]>
    @rustyrussell
    rustyrussell committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    360d2e5 View commit details
    Browse the repository at this point in the history