Skip to content
This repository has been archived by the owner on Sep 28, 2021. It is now read-only.

Optimize backFill range finder query #40

Open
i-norden opened this issue Sep 14, 2020 · 1 comment
Open

Optimize backFill range finder query #40

i-norden opened this issue Sep 14, 2020 · 1 comment
Assignees

Comments

@i-norden
Copy link
Collaborator

i-norden commented Sep 14, 2020

Current method for finding missing headers takes too long once the database grows large. Rather than finding entire ranges of blocks that are missing and then fetching chunks in batches, have multiple workers concurrently SELECTing single block heights where a header is missing and fetching statediff payloads at those heights.

@i-norden
Copy link
Collaborator Author

The separate, ongoing backfill process is now deprecated. We now do a lazy backfill approach, only backfilling when there is a cache miss on an incoming request.

This will cause some problems with certain operations, namely state operations where it is unknown what block contains the diffs that contain the missing nodes required for the operation.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants