Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rewind_single_block slow for large rewinds #3245

Closed
antiochp opened this issue Feb 25, 2020 · 2 comments · Fixed by #3246
Closed

rewind_single_block slow for large rewinds #3245

antiochp opened this issue Feb 25, 2020 · 2 comments · Fixed by #3246
Labels
Milestone

Comments

@antiochp
Copy link
Member

Related #3236

The introduction of rewind_single_block() makes the output_pos handling more robust but naively involved a call to apply_to_bitmap_accumulator() for each block being rewound.
This is fine for relatively short rewinds such as during a fork or potential reorg scenario.

We have one use case for large rewinds though - this occurs when we are taking a "snapshot" of the txhashset at the horizon to provide to a peer during fast sync. Here we rewind approx 2 days of blocks.

We need to build a vec of "affected pos" across the full set of rewound blocks and then call apply_to_bitmap_accumulator() once with this aggregate set of pos.
It is too costly to do this specific rewind step for each block sequentially.

@antiochp antiochp added the bug label Feb 25, 2020
@antiochp antiochp added this to the 3.1.0 milestone Feb 25, 2020
@antiochp
Copy link
Member Author

This was not identified during testing for #3236 as the performance regression is on the node being sync'd from, not the node doing the syncing.
We should have considered this use case when thinking through potential impact.

@antiochp
Copy link
Member Author

Fix coming shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant