[EIP-4844] Blobs pruning #6628
Merged
[EIP-4844] Blobs pruning #6628
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tbenr
force-pushed
the
block_and_blob_pruner
branch
from
a30db8d
to
9feb99f
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -110,6 +124,11 @@ | |||
| __ -> | |||
| blockPruner | |||
| .map(BlockPruner::start) | |||
| .orElseGet(() -> SafeFuture.completedFuture(null))) | |||
| .thenCompose( | |||
| __ -> | |||
Check notice
Code scanning / CodeQL
Useless parameter
prune blobs via key stream
ajsutton
approved these changes
Dec 20, 2022
|
It is currently tested via |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Blocks and Blobs pruning work slightly different.
Blocks jobs run every 1h and always prune all "prunable" blocks (that's why the first time you switch MINIMAL on, takes a while to complete).
Blobs pruning are designed to run frequently and with an hard cap on the
BlobsSidecars that can be pruned. Currently configured to run every minute and prune maximum of 32 blobs. Under finalization, with mainnet config, we are supposed to create 5 BlobsSidecar each minute, so 32 is able to catch up relatively quickly.The idea is to avoid those pruning jobs to overlap, so we limit the CPU\Disk pressure while node is doing its main work that's why I configured
storagePrunerAsyncRunnerto be single-threaded.I also lowered down the thread priority.
I added the unconfirmed blobs pruning, which make sure that unconfirmed blobs (blobs its block has been received but never imported (remaining in FutureItems or in PendingPool)) are removed from DB once we finalize those slots. It shares the same prune limit (32)
Improved pruning by adding the ability to stream keys only from the KV store, so value doesn't need to be read\deserialized.
related to #6629
Documentation
doc-change-requiredlabel to this PR if updates are required.Changelog