Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: store SLRU blocks on shard 0 only #9423

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jcsp
Copy link
Collaborator

@jcsp jcsp commented Oct 16, 2024

Problem

SLRU blocks, which can add up to several gigabytes, are currently ingested by all shards, multiplying their capacity cost by the shard count and slowing down ingest.

We do this because all shards need the SLRU pages to do timestamp->LSN lookup for GC.

Summary of changes

  • On non-zero shards, learn the GC offset from shard 0's index instead of calculating it.
  • Do not ingest SLRU pages on non-zero shards

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

Copy link

5202 tests run: 4995 passed, 0 failed, 207 skipped (full report)


Flaky tests (1)

Postgres 17

Code coverage* (full report)

  • functions: 31.3% (7557 of 24116 functions)
  • lines: 49.1% (60446 of 123025 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
d86b440 at 2024-10-16T17:51:10.100Z :recycle:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant