Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize archiver initialization during node restart #1852

Closed
nazar-pc opened this issue Aug 20, 2023 · 1 comment · Fixed by #1980
Closed

Optimize archiver initialization during node restart #1852

nazar-pc opened this issue Aug 20, 2023 · 1 comment · Fixed by #1980
Assignees
Labels
improvement it is already working, but can be better node Node (service library/node app) performance Related to performance measurement or improvement

Comments

@nazar-pc
Copy link
Member

Reading blocks in Substrate is slow, like very slow. When we have thousands of blocks to read it could take seconds.

We read blocks from the tip backwards and then forward again when restarting node to restore internal archiver state.

We should be able to accelerate it in two ways:

  • use segment headers store to find interested block faster rather than going from tip towards genesis (find_last_archived_block function)
  • try reading blocks in parallel, this should help since currently search is CPU-bound and SSDs can easily do more parallel work
@nazar-pc nazar-pc added improvement it is already working, but can be better node Node (service library/node app) performance Related to performance measurement or improvement labels Aug 20, 2023
@nazar-pc nazar-pc added this to the Protocol UX Improvements milestone Aug 20, 2023
@nazar-pc nazar-pc self-assigned this Aug 20, 2023
@mathematicw
Copy link

use segment headers store to find interested block faster rather than going from tip towards genesis (find_last_archived_block function) should be faster and more resource economic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement it is already working, but can be better node Node (service library/node app) performance Related to performance measurement or improvement
Projects
Development

Successfully merging a pull request may close this issue.

2 participants