Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downloader: add ProhibitNewDownloads() #8939

Merged
merged 37 commits into from
Dec 12, 2023
Merged

Downloader: add ProhibitNewDownloads() #8939

merged 37 commits into from
Dec 12, 2023

Conversation

AskAlexSharov
Copy link
Collaborator

@AskAlexSharov AskAlexSharov commented Dec 9, 2023

"whitelisting" mechanism (list of files - stored in DB) - which protecting us from downloading new files after upgrade/downgrade was broken. And seems it became over-complicated with time.
I replacing it by 1 persistent flag inside downloader: "prohibit_new_downloads.lock"
Erigon will turn downloader into this mode after downloading/verification of first snapshots.

//Corner cases:
	// - Erigon generated file X with hash H1. User upgraded Erigon. New version has preverified file X with hash H2. Must ignore H2 (don't send to Downloader)
	// - Erigon "download once": means restart/upgrade/downgrade must not download files (and will be fast)
	// - After "download once" - Erigon will produce and seed new files

downloader --seedbox is never "prohibit new downloads"

Copy link
Contributor

@Giulio2002 Giulio2002 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but maybe we should actually look into whether the files with differing hash are even generated correctly. if that was done it is a pass for me

@AskAlexSharov
Copy link
Collaborator Author

@Giulio2002 this ticket is not related to "differrent file hash" problem. it's about:

  • we releasing good files for new block range: synced nodes must not download them (synced nodes can generate and seed new files by self)
  • we releasing new type of snapshots (like caplin / bor): existing synced nodes must not download them.

@AskAlexSharov AskAlexSharov merged commit d41d523 into devel Dec 12, 2023
6 checks passed
@AskAlexSharov AskAlexSharov deleted the d_no_d branch December 12, 2023 09:05
@AskAlexSharov
Copy link
Collaborator Author

Actually there is invariant: "after first sync cycle - where it downloading all snapshots for first time - erigon must not see uncomplete / partially-downloaded files" - because we download and read them in same and erigon/rpcdaemon just assume - everything it see is valid/fully-downloaded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants