Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

pallet-mmr: fix offchain db for sync from zero #12498

Merged

Conversation

acatangiu
Copy link
Contributor

MMR nodes are saved in off-chain db under fork-resistant keys based on frame_system::block_hash(), as such only nodes added by latest frame_system::BlockHashCount can be accessed that way.
Nodes added by blocks older than frame_system::BlockHashCount are moved by offchain_worker to "canon" (not fork resistant) location - thus being made available regardless of age.
See #11594 for more details.

Issue is that offchain_worker doesn't run for blocks imported during initial sync. So for initial syncs > frame_system::BlockHashCount (practically any node syncing from scratch), MMR nodes aren't moved to canon position and access to them is lost.

Solution proposed by this PR: runtime block execution saves node to both fork-resistant and canon locations:

  • initial sync nodes directly end up in the right location,
  • for the rest, we do a superfluous offchain_index::set() because it will be overwritten by offchain_worker with canon values anyway.

@acatangiu acatangiu added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit. labels Oct 14, 2022
@acatangiu acatangiu self-assigned this Oct 14, 2022
@acatangiu
Copy link
Contributor Author

@andresilva @svyatonik @serban300 @Lederstrumpf please prioritize reviewing this PR as it fixes a not-yet released pallet-mmr. If we're lucky to catch this fix in the same release as the PR introducing the issue we'll avoid needing offchain db migration.

Copy link
Contributor

@serban300 serban300 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch ! The solution looks good. I just left 2 nits.

frame/merkle-mountain-range/src/lib.rs Outdated Show resolved Hide resolved
frame/merkle-mountain-range/src/mmr/storage.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@svyatonik svyatonik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

…nt_hash)`

Do this so that both canon and fork-resitant keys have the same
`(prefix, pos).encode()` prefix. Might be useful in the future if we'd
be able to to "get" offchain db entries using key prefixes as well.

Signed-off-by: acatangiu <[email protected]>
@acatangiu
Copy link
Contributor Author

bot merge

@paritytech-processbot
Copy link

Waiting for commit status.

@paritytech-processbot paritytech-processbot bot merged commit 273e264 into paritytech:master Oct 17, 2022
@acatangiu acatangiu deleted the mmr-offchain-db-hotfix branch December 13, 2022 08:39
ark0f pushed a commit to gear-tech/substrate that referenced this pull request Feb 27, 2023
* pallet-mmr: cosmetic improvements

* pallet-mmr: fix offchain storage for initial sync

* address review comments

* pallet-mmr: change offchain fork-resistant key to `(prefix, pos, parent_hash)`

Do this so that both canon and fork-resitant keys have the same
`(prefix, pos).encode()` prefix. Might be useful in the future if we'd
be able to to "get" offchain db entries using key prefixes as well.

Signed-off-by: acatangiu <[email protected]>

Signed-off-by: acatangiu <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants