Skip to content

Commit

Permalink
common/gossip_store: optimize case where entries are filtered out.
Browse files Browse the repository at this point in the history
@whitslack complained of large CPU usage by connectd at startup;
I ran perf record on connectd on my machine (which sees a little spike, only)
and I see the cost of reading and discarding the entries:

```
-   95.52%     5.24%  lightning_conne  lightning_connectd  [.] gossip_store_next
   - 90.28% gossip_store_next
      + 40.27% tal_alloc_arr_
      + 22.78% tal_free
      + 11.74% crc32c
      + 9.32% fromwire_peektype
      + 4.10% __libc_pread64 (inlined)
        1.70% be32_to_cpu
```

Much of this is caused by the search for our own gossip: keeping this separately
would be even better, but this fix is minimal.

Signed-off-by: Rusty Russell <[email protected]>
  • Loading branch information
rustyrussell committed Jun 20, 2022
1 parent cff8593 commit cf1bc8a
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 7 deletions.
14 changes: 9 additions & 5 deletions common/gossip_store.c
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,16 @@ u8 *gossip_store_next(const tal_t *ctx,
continue;
}

checksum = be32_to_cpu(hdr.crc);
/* Skip any timestamp filtered */
timestamp = be32_to_cpu(hdr.timestamp);
if (!push &&
!timestamp_filter(timestamp_min, timestamp_max,
timestamp)) {
*off += r + msglen;
continue;
}

checksum = be32_to_cpu(hdr.crc);
msg = tal_arr(ctx, u8, msglen);
r = pread(*gossip_store_fd, msg, msglen, *off + r);
if (r != msglen)
Expand All @@ -105,10 +113,6 @@ u8 *gossip_store_next(const tal_t *ctx,
&& type != WIRE_CHANNEL_UPDATE
&& type != WIRE_NODE_ANNOUNCEMENT) {
msg = tal_free(msg);
} else if (!push &&
!timestamp_filter(timestamp_min, timestamp_max,
timestamp)) {
msg = tal_free(msg);
} else if (!push && push_only) {
msg = tal_free(msg);
}
Expand Down
4 changes: 2 additions & 2 deletions gossipd/gossip_store.c
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ static u32 gossip_store_compact_offline(struct routing_state *rstate)
oldlen = lseek(old_fd, SEEK_END, 0);
newlen = lseek(new_fd, SEEK_END, 0);
append_msg(old_fd, towire_gossip_store_ended(tmpctx, newlen),
0, false, &oldlen);
0, true, &oldlen);
close(old_fd);
status_debug("gossip_store_compact_offline: %zu deleted, %zu copied",
deleted, count);
Expand Down Expand Up @@ -565,7 +565,7 @@ bool gossip_store_compact(struct gossip_store *gs)

/* Write end marker now new one is ready */
append_msg(gs->fd, towire_gossip_store_ended(tmpctx, len),
0, false, &gs->len);
0, true, &gs->len);

gs->count = count;
gs->deleted = 0;
Expand Down

0 comments on commit cf1bc8a

Please sign in to comment.