Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat(standalone): Bootstrap storage from relayer and state snapshots #379

Merged

Conversation

birchmd
Copy link
Member

@birchmd birchmd commented Nov 29, 2021

The purpose of this feature is to allow filling the standalone engine database from historical data, specifically data we already hold in the relayer and have extracted from NEAR RPC nodes (the JSON state snapshots).

I believe the code here is complete for now, however more work is needed on the relayer side for it to reliably rebuild the same state as is present on the blockchain.

Known issues:

  • The relayer does not index all transactions which change the engine state. (Make sure to index all transactions affecting the state aurora.js#23) In particular, when attempting the storage bootstrap, many transactions fail due to insufficient funds. This is likely because state changes caused by the bridge are not indexed.
  • With the new random precompile, we now need to index the random seed which comes from each NEAR block to accurately replay transactions using that precompile. For now this is not a problem as that change to the engine has not yet been deployed.

This means the bootstrapping process will not work for many transactions as of today, however I still think we should merge this PR and we will need to coordinate with @artob @JonathanLogan @mfornet on re-indexing as that work is done on the infrastructure side.
For now these issues can be mitigated through the use of JSON snapshots. These can be taken from NEAR RPC nodes at heights where there is known to be an issue with the relayer indexing and sent to the standalone storage to ensure it has the correct state at that height.

@birchmd birchmd added the A-standalone Area: the standalone engine EVM label Nov 29, 2021
Copy link
Contributor

@joshuajbouw joshuajbouw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few notes, and small requests. I think that I am fairly surprised that there isn't a lot of constructors for typical postgres calls, and other "Rust" friendly means?

engine-sdk/src/promise.rs Outdated Show resolved Hide resolved
engine-standalone-storage/src/json_snapshot/types.rs Outdated Show resolved Hide resolved
engine-standalone-storage/src/relayer_db/mod.rs Outdated Show resolved Hide resolved
engine-standalone-storage/src/relayer_db/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@joshuajbouw joshuajbouw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolute legend.

@birchmd birchmd force-pushed the bootstrap-standalone-storage branch from bf2b6d8 to 2cca9b1 Compare December 6, 2021 19:22
@birchmd birchmd merged commit 2cca9b1 into aurora-is-near:develop Dec 6, 2021
@birchmd birchmd deleted the bootstrap-standalone-storage branch December 6, 2021 19:30
artob pushed a commit that referenced this pull request Dec 10, 2021
* Feat(engine): London hard fork support (#244)
* Fix(exit precompile): Address to refund in case of error is an argument (#311)
* Feat(engine): Make engine parametric in storage access (#314)
* Test verifying the EVM log returns the correct address (#341)
* Remove sdk::current_account_id usage from engine-precompiles (#346)
* Remove Default trait bound from engine IO (#342)
* Remove some sdk usage from core logic (#347)
* Factor out blockchain environment variable access as a trait (#349)
* Factor out NEAR promise host functions into a trait (#353)
* Borsh deserialized value field for call args (#351)
* Refactor(eth-connector): Use Result return values instead of panicking (#355)
* Gate all NEAR host functions behind the contract feature (#356)
* Bump @openzeppelin/contracts from 4.3.2 to 4.3.3 in /etc/eth-contracts
* Chore: Newtypes for gas (#344)
* Feat(standalone): Standalone (#345)
* Minor fixes to sdk refactor (#359)
* Refactor(engine): Move submit logic into engine module (#366)
* Feat(standalone): Storage backend (#375)
* NEAR random numbers from solidity contract (#368)
* Feat(standalone): EVM tracing via SputnikVM (#383)
* Feat(standalone): Bootstrap storage from relayer and state snapshots (#379)
* Feat(standalone): Structures and logic for keeping storage in sync with the blockchain (#382)
* Feat(standalone): Capture geth-like tracing from SputnikVM events (#384)
* Connector cleanup (#374)
* Remove betanet
* Fix(engine): original_storage bug fix; more tracing tests (#390)
* Increase NEAR Gas for ft_on_transfer (#389)

Co-authored-by: Andrew Bednoff <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Evgeny Ukhanov <[email protected]>
Co-authored-by: Marcelo Fornet <[email protected]>
Co-authored-by: Michael Birch <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-standalone Area: the standalone engine EVM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants