Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upgrade] Go-Ethereum release v1.9.20 #1149

Merged
merged 34 commits into from
Mar 15, 2021

Conversation

ricardolyn
Copy link
Contributor

@ricardolyn ricardolyn commented Mar 10, 2021

TODO

Plan & Analyse

  • Review the Release Notes
  • Review PRs in the section below

As you review, list extra changes and/or tests to be implemented to ensure compatibility with GoQuorum specific features.

Build & Test

  • Pull and checkout PR branch locally, then merge GoQuorum master into this branch
  • Resolve conflicts, taking into account the prior analysis
  • Implement required changes until lint passes
  • Implement required changes until all unit tests pass
  • Implement required changes until acceptance tests pass
  • Implement extra changes and/or tests
  • Verify any left TODOs in the code

Extra Changes & Tests

Go-Ethereum Release: Paragade (v1.9.20)

  • Version: v1.9.20
  • Published: 2020-08-25T14:23:55Z

Release notes

Geth v1.9.20 is another maintenance release containing bug fixes and security enhancements. This update is recommended for all users.

Please note that reverting to Geth v1.9.19 or prior after upgrading to v1.9.20 is not possible without a resync because the blockchain database layout has changed.

New Features:

  • You can now query the Chain ID using GraphQL (#21451)
  • The evm command can now appends the TX hash to state transition output files (#21406)

Bug Fixes & Optimizations:

  • Blockchain rewinding and chain repair, i.e. using the SetHead operation, now behave
    correctly in all cases. An extensive test suite for chain repair has been added. (#21409, #21409)
  • Discovery DHT bootstrapping now works properly in very small private networks. (#21396)
  • Contract code is now stored separately from state tree data in LevelDB. This change will
    help with future database upgrades. (#21080)
  • Two additional metrics counters for blockchain reorgs have been added. (#21420)
  • Metrics collection is now lock-free, causing less lock contention among peer connections (#21446, #21470)
  • The GetNodeData operation in the eth peer-to-peer protocol now uses the fast-sync bloom filter
    to speed up request processing. (#21445)
  • Importing block data during fast-sync now performs significantly fewer database reads. (#21468)

Build Changes:

  • This release is built with the new and shiny Go 1.15 compiler version (#21466)
  • The Ubuntu PPA now contains builds for Ubuntu 20.10 Groovy Gorilla. Ubuntu 19.04 Disco
    Dingo is no longer supported. (#21461)

For a full rundown of the changes please consult the Geth 1.9.20 release milestone


As with all our previous releases, you can find the:

Codebase changes assessment

Legend

File Stats: (A) Added, (M) Modified and (R) Removed

Line Stats: (A) Added and (R) Removed

Assessment:

  • ✅ No conflict expected
  • ⚠ Review required to assess changes
  • ‼️ Conflicts expected and review required

18 Pull Requests

🔍 Link Title File Stats
M/A/R
Packages changed
(files changed)
Line Stats
A/R
Top 5 Changed Files
(lines changed)
⚠️ #21410 cmd/geth/tests: try to fix spurious travis failure in les tests 1/0/0
cmd/geth (1)
7/6
cmd/geth/les_test.go (26)
#21446 metrics: make meter updates lock-free 3/0/0
metrics (3)
37/38
metrics/meter.go (130)
metrics/ewma.go (10)
metrics/meter_test.go (10)
‼️ #21450 Fixed some comments 3/0/0
core/state (3)
14/15
core/state/statedb.go (20)
core/state/statedb_test.go (20)
core/state/state_object.go (18)
⚠️ #21461 build: drop disco, enable groovy on Ubuntu PPAs 1/0/0
build (1)
2/1
build/ci.go (6)
#21406 cmd/evm: statet8n output folder + tx hashes on trace filenames 6/0/0
cmd/evm (3)
cmd/evm/internal/t8ntool (3)
46/26
cmd/evm/internal/t8ntool/transition.go (80)
cmd/evm/README.md (36)
cmd/evm/internal/t8ntool/flags.go (10)
cmd/evm/internal/t8ntool/execution.go (8)
cmd/evm/transition-test.sh (8)
⚠️ #21420 core: better metering for reorgs 2/0/0
core (2)
8/2
core/blockchain.go (16)
core/tx_pool.go (4)
#21409 core: define and test chain reparation cornercases 11/2/0
core (5)
core/rawdb (4)
eth/downloader (2)
eth (1)
trie (1)
3952/214
core/blockchain_sethead_test.go (3898)
core/blockchain_repair_test.go (3306)
core/blockchain.go (406)
core/rawdb/freezer.go (158)
eth/downloader/downloader.go (122)
#21466 travis, dockerfile, appveyor, build: bump to Go 1.15 5/0/0
.travis.yml (1)
Dockerfile (1)
Dockerfile.alltools (1)
appveyor.yml (1)
build (1)
26/16
.travis.yml (64)
appveyor.yml (8)
Dockerfile (4)
Dockerfile.alltools (4)
build/checksums.txt (4)
#21470 metrics: zero temp variable in updateMeter 2/0/0
metrics (2)
22/2
metrics/meter_test.go (32)
metrics/meter.go (16)
⚠️ #21472 eth/downloader: fix rollback issue on short chains 1/0/0
eth/downloader (1)
2/0
eth/downloader/downloader.go (4)
⚠️ #21080 core, eth, les, trie: add a prefix to contract code 29/1/0
core/rawdb (7)
core (5)
core/state (5)
core/types (3)
eth/downloader (3)
eth/fetcher (2)
eth (1)
consensus/clique (1)
consensus/ethash (1)
cmd/evm/internal/t8ntool (1)
cmd/geth (1)
374/95
core/rawdb/accessors_state.go (192)
core/state/sync_test.go (190)
core/state/database.go (70)
core/rawdb/accessors_indexes_test.go (56)
core/types/block_test.go (56)
‼️ #21468 core: do less lookups when writing fast-sync block bodies 1/0/0
core (1)
12/3
core/blockchain.go (30)
⚠️ #21445 eth/downloader, eth/handler: utilize sync bloom for getNodeData 2/0/0
eth (1)
eth/downloader (1)
13/0
eth/downloader/downloader.go (18)
eth/handler.go (8)
#21477 core/state/snapshot: reduce disk layer depth during generation 3/0/0
core/state/snapshot (3)
18/9
core/state/snapshot/generate.go (36)
core/state/snapshot/snapshot.go (14)
core/state/snapshot/journal.go (4)
#21396 p2p/discover: avoid dropping unverified nodes when table is almost empty 5/0/0
p2p/discover (5)
140/25
p2p/discover/v4_udp_test.go (176)
p2p/discover/table.go (86)
p2p/discover/v4_udp.go (34)
p2p/discover/lookup.go (26)
p2p/discover/table_test.go (8)
#21483 core/rawdb: only complain loudly if truncating many items 1/0/0
core/rawdb (1)
7/2
core/rawdb/freezer_table.go (18)
⚠️ #21451 graphql: add query for chainID 2/0/0
graphql (2)
6/0
graphql/graphql.go (8)
graphql/schema.go (4)
⚠️ #21486 params: update CHTs for v1.9.20 release 1/0/0
params (1)
16/16
params/config.go (64)

80 Changed files

🔍 File Lines Changed Linked PR
core/blockchain_sethead_test.go 3898 #21409
core/blockchain_repair_test.go 3306 #21409
‼️ core/blockchain.go 502 #21468
#21409
#21420
#21080
trie/sync.go 386 #21409
⚠️ trie/database.go 258
core/rawdb/accessors_state.go 192 #21080
core/state/sync_test.go 190 #21080
p2p/discover/v4_udp_test.go 176 #21396
‼️ core/rawdb/freezer.go 158 #21409
⚠️ eth/downloader/downloader.go 148 #21472
#21445
#21409
#21080
metrics/meter.go 138 #21446
#21470
⚠️ eth/downloader/downloader_test.go 106 #21409
core/headerchain.go 104 #21409
⚠️ core/blockchain_test.go 102 #21409
#21080
p2p/discover/table.go 86 #21396
cmd/evm/internal/t8ntool/transition.go 80 #21406
trie/sync_test.go 72
‼️ core/state/database.go 70 #21080
.travis.yml 64 #21466
⚠️ params/config.go 64 #21486
⚠️ core/state/statedb.go 62 #21450
#21080
core/types/block_test.go 56 #21080
core/rawdb/accessors_indexes_test.go 56 #21080
core/rawdb/accessors_chain.go 52 #21409
metrics/meter_test.go 42 #21446
#21470
core/rawdb/database.go 42 #21409
#21080
core/rawdb/schema.go 38 #21409
#21080
trie/sync_bloom.go 38
⚠️ eth/sync.go 36 #21409
cmd/evm/README.md 36 #21406
core/state/snapshot/generate.go 36 #21477
p2p/discover/v4_udp.go 34 #21396
core/rawdb/accessors_metadata.go 34 #21080
core/types/derive_sha.go 34 #21080
⚠️ eth/handler.go 30 #21445
#21080
p2p/discover/lookup.go 26 #21396
⚠️ cmd/geth/les_test.go 26 #21410
‼️ core/state/statedb_test.go 20 #21450
core/rawdb/freezer_table.go 18 #21483
⚠️ core/state/state_object.go 18 #21450
⚠️ cmd/evm/internal/t8ntool/execution.go 18 #21406
#21080
eth/downloader/statesync.go 16 #21080
⚠️ core/types/block.go 16 #21080
core/state/snapshot/snapshot.go 14 #21477
core/state/iterator_test.go 12 #21080
trie/trie.go 12
eth/fetcher/block_fetcher.go 10 #21080
metrics/ewma.go 10 #21446
eth/downloader/queue.go 10 #21080
⚠️ core/block_validator.go 10 #21080
cmd/evm/internal/t8ntool/flags.go 10 #21406
core/rawdb/accessors_chain_test.go 8 #21080
core/rawdb/chain_iterator_test.go 8 #21080
cmd/evm/transition-test.sh 8 #21406
⚠️ light/trie.go 8
les/odr_requests.go 8
appveyor.yml 8 #21466
p2p/discover/table_test.go 8 #21396
⚠️ graphql/graphql.go 8 #21451
eth/fetcher/block_fetcher_test.go 6 #21080
⚠️ consensus/clique/clique.go 6 #21080
⚠️ build/ci.go 6 #21461
⚠️ core/genesis.go 6 #21080
⚠️ core/tx_pool_test.go 6 #21080
⚠️ consensus/ethash/consensus.go 6 #21080
trie/secure_trie.go 6
build/checksums.txt 4 #21466
⚠️ graphql/schema.go 4 #21451
‼️ Dockerfile 4 #21466
⚠️ light/odr_test.go 4
⚠️ cmd/geth/retesteth.go 4 #21080
core/state/sync.go 4 #21080
⚠️ params/version.go 4
⚠️ core/tx_pool.go 4 #21420
core/state/snapshot/journal.go 4 #21477
Dockerfile.alltools 4 #21466
light/odr.go 4
⚠️ miner/worker.go 4
⚠️ les/server_handler.go 4
cmd/evm/main.go 2 #21406

karalabe and others added 26 commits August 11, 2020 14:11
* cmd/geth/tests: try to fix spurious travis failure in les tests

* cmd/geth: les_test - remove extraneous option during boot
build: drop disco, enable groovy on Ubuntu PPAs
* t8ntool: add output basedir

* t8ntool: add txhash to trace filename

* t8ntool: don't default to '.' basedir, allow absolute paths
* core: define and test chain reparation cornercases

* core: write up a variety of set-head tests

* core, eth: unify chain rollbacks, handle all the cases

* core: make linter smile

* core: remove commented out legacy code

* core, eth/downloader: fix review comments

* core: revert a removed recovery mechanism
travis, dockerfile, appveyor, build: bump to Go 1.15
* metrics: zero temp variable in  updateMeter

Previously the temp variable was not updated properly after summing it to count.
This meant we had astronomically high metrics, now we zero out the temp whenever we
sum it onto the snapshot count

* metrics: move temp variable to be aligned, unit tests

Moves the temp variable in MeterSnapshot to be 64-bit aligned because of the atomic bug.
Adds a unit test, that catches the previous bug.
eth/downloader: fix rollback issue on short chains
* eth/downloader, eth/handler: utilize sync bloom for getNodeData

* trie: handle if bloom is nil

* trie, downloader: check bloom nilness externally
core/state/snapshot: reduce disk layer depth during generation
…pty (#21396)

This change improves discovery behavior in small networks. Very small
networks would often fail to bootstrap because all member nodes were
dropping table content due to findnode failure. The check is now changed
to avoid dropping nodes on findnode failure when their bucket is almost
empty. It also relaxes the liveness check requirement for FINDNODE/v4
response nodes, returning unverified nodes as results when there aren't
any verified nodes yet.

The "findnode failed" log now reports whether the node was dropped
instead of the number of results. The value of the "results" was
always zero by definition.

Co-authored-by: Felix Lange <[email protected]>
core/rawdb: only complain loudly if truncating many items
params: update CHTs for v1.9.20 release
@CLAassistant
Copy link

CLAassistant commented Mar 10, 2021

CLA assistant check
All committers have signed the CLA.

@ricardolyn ricardolyn marked this pull request as ready for review March 10, 2021 16:21
Copy link
Contributor

@nmvalera nmvalera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ricardolyn ricardolyn merged commit 2c38911 into master Mar 15, 2021
@ricardolyn ricardolyn deleted the upgrade/go-ethereum/v1.9.20-2021310134659 branch March 15, 2021 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Chain rewind and failure to sync with err=missing parent after non-graceful restart of nodes