Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

geth 1.10.8 stop sync #23546

Closed
FantomJoker opened this issue Sep 8, 2021 · 10 comments
Closed

geth 1.10.8 stop sync #23546

FantomJoker opened this issue Sep 8, 2021 · 10 comments
Labels
type:bug

Comments

@FantomJoker
Copy link

System information

Geth version: geth-linux-amd64-1.10.8-26675454
OS & Version: Linux, mem:15g(7.8g left), disk:1.4T(900g left)

--maxpeers 70 --cache 1024 --txpool.globalslots 1024 --syncmode fast -rpcapi debug,txpool,admin,net,personal,eth,web3 --rpc

Actual behaviour

> eth.syncing
{
  currentBlock: 13178787,
  highestBlock: 13182182,
  knownStates: 862497244,
  pulledStates: 862497244,
  startingBlock: 13178786
}

Backtrace

ERROR[09-08|09:43:50.097] 
########## BAD BLOCK #########
Chain config: {ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7280000 Petersburg: 7280000 Istanbul: 9069000, Muir Glacier: 9200000, Berlin: 12244000, London: 12965000, Engine: ethash}

Number: 13178788
Hash: 0x508eeced0dea430210f378913ca1cfe616024a22c08f05e211df92bd0dd8c6be


Error: could not apply tx 364 [0x1d065e10d9a2da515cc1f5fdc5a7bac2d359143be446fffa67147eefcb416d8a]: gas limit reached
##############################
 
WARN [09-08|09:43:50.137] Synchronisation failed, dropping peer    peer=7d0bd2d3f7ac769460a7bd55d83ed66d2bd10b5980dc68fe7c04a48be605277e err="retrieved hash chain is invalid: could not apply tx 364 [0x1d065e10d9a2da515cc1f5fdc5a7bac2d359143be446fffa67147eefcb416d8a]: gas limit reached"
WARN [09-08|09:43:50.138] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
INFO [09-08|09:43:51.883] Generating state snapshot                root=2eb66e..89d2fc at=c718c6..4cf5f3 accounts=112,739,094 slots=395,334,488 storage=33.41GiB   elapsed=105h14m58.528s eta=30h4m51.785s
INFO [09-08|09:43:59.883] Generating state snapshot                root=2eb66e..89d2fc in=c71a83..5ec5e9 at=11d855..bd8506 accounts=112,743,105 slots=395,341,671 storage=33.41GiB   elapsed=105h15m6.528s  eta=30h4m37.447s
WARN [09-08|09:44:07.114] Synchronisation failed, dropping peer    peer=97d418327f8627fb801abeed609c0c9464cf95ed76670d027c884eb697685f86 err=timeout
WARN [09-08|09:44:07.114] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
INFO [09-08|09:44:07.883] Generating state snapshot                root=2eb66e..89d2fc in=c71a83..5ec5e9 at=31bd8b..80a1c9 accounts=112,743,105 slots=395,352,612 storage=33.41GiB   elapsed=105h15m14.529s eta=30h4m39.734s
INFO [09-08|09:44:15.884] Generating state snapshot                root=2eb66e..89d2fc in=c71a83..5ec5e9 at=519b0d..11a2da accounts=112,743,105 slots=395,363,393 storage=33.41GiB   elapsed=105h15m22.529s eta=30h4m42.02s
INFO [09-08|09:44:23.885] Generating state snapshot                root=2eb66e..89d2fc in=c71a83..5ec5e9 at=722e01..e97b43 accounts=112,743,105 slots=395,374,280 storage=33.42GiB   elapsed=105h15m30.530s eta=30h4m44.306s
WARN [09-08|09:44:27.902] Synchronisation failed, dropping peer    peer=fc125e693e8df9d630d4c94e7d73f74089d5a524789afee8d15a5c1342f0d43d err=timeout
WARN [09-08|09:44:27.902] Synchronisation failed, retrying         err="peer is unknown or unhealthy"
@0xcircuitbreaker
Copy link

I am getting the same issue

@0xcircuitbreaker
Copy link 

########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: 4230000 Petersburg: 4939394 Istanbul: 6485846, Muir Glacier: 7117117, Berlin: 9812189, London: 10499401, Engine: ethash}

Number: 10920274
Hash: 0x2fd63eb95c8059a46b5afedb903ef1639b9414e9d38f70b3dfb0f733b7ced14f
         0: cumulative: 47261 gas: 47261 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x95080369054e617280b3fa77a8c389c6e832ef6075c8574830733eb8599b09ba logs: [0xc2c68ad290] bloom: 00000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000004000100000000200000000000000000000000000000012000000000000000000000000000000000000000000000000000000000800000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000020000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000010000000000000000000000000000000000000000000000000000000000000 state:
         1: cumulative: 246246 gas: 198985 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x1e69365f032f87f1dda449fcc1360318590cb9db4981839d7b8eac5fc798d353 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         2: cumulative: 445231 gas: 198985 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x998febc5d866ec03d5e9a0ff8fb478aebb21ec4ca2bd7f3720d3f77bbb0ff7a0 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         3: cumulative: 491390 gas: 46159 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x5b88b08f2b0ae0bc84673c082c5ee4c1d31a5cd8642b181f2761bb9f1b4943fe logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         4: cumulative: 537658 gas: 46268 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x098f383f532592bc45567ced73522f207ff7a633830014b9902c6887314c3717 logs: [0xc2c68ad340] bloom: 00000000000000000000000000000000000000000000000000001000000000000000000100000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000004000000000000000000000000000000000000000000000200000000000000000000000000080000000000000020002000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000000000000000000000000002000010000000000000000000000000000000000000000000000000000000000000 state:
         5: cumulative: 593803 gas: 56145 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xb82786bc71eeacb6d4808bcaaffdb397565d15340bedf2e491e5dfae7a6da48d logs: [0xc2c68ad3f0 0xc2c68ad4a0] bloom: 00000000000000000000000000000000000000000000000020000000000000000000000000000800000000000000000000000000000000000000000000000001000000000000000000000040000000000000000000400000000000000000000018000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         6: cumulative: 660136 gas: 66333 contract: 0x4a4511ADDABA6ee5A81448dBa122163CeC3b3cDF status: 1 tx: 0x6ea35435fbef2fc1cc56e252474628662b135fdf821a2ae6903a94fccabbf202 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         7: cumulative: 706283 gas: 46147 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x23de793b440cdbbd2be9c1896764ff954975aedc8c31c52b0ddcfafb019b7161 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         8: cumulative: 752442 gas: 46159 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xec349372328e2582f27f1e4bb45c6ba56b210b974093506186f8fb05bb3a2044 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         9: cumulative: 922780 gas: 170338 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x26b2634cd5d765715d13c667f1ae79f42fb2c175cc5c0f102343b29b13d2e749 logs: [0xc2ccfddb80 0xc2ccfddc30 0xc2ccfddce0 0xc2ccfddd90 0xc2ccfdde40] bloom: 00000000000000020000000000000000000000000000000000000001000004000000000000000000000000201000000000000000200000000000000000000000002000000000000000000008000000040008000010000000000410000000000000000000020000000000000000000800000000000000000020000010000000000000010000020000000000000008000000000000000000000000000100000000000000000000000000000800000000000000000000000000000000000000000000000002000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000020010000000000000000000000000 state:
         10: cumulative: 1056898 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xc4cf32b724efbcc75de35a8f07afb15f71fd56e17b1ed95d7047c31b13693772 logs: [0xc2ccfddef0 0xc2d0056000] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         11: cumulative: 1191016 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8460a92b5ff1c06c40df6b128a94805a081f007585ff7409cb7d32697fae98c8 logs: [0xc2d00560b0 0xc2d0056160] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         12: cumulative: 1325134 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x40f5ca48896dab3005ed3f2ac3f927f83fb21f7e72f0c819144efb4d1e55fcee logs: [0xc2d0056210 0xc2d00562c0] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         13: cumulative: 1459252 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x6ffa9139d207505acacd29ef2786d1ca6d10c770eefb921b092e2546da4bab90 logs: [0xc2d0056370 0xc2d0056420] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         14: cumulative: 1728059 gas: 268807 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x540cf48abff64a05514abf0d5bbf4cddac8bb9bc6e0d482f736f04927345555d logs: [0xc2d00564d0 0xc2d0056580 0xc2d0056630 0xc2d00566e0 0xc2d0056790] bloom: 80000000040100000000004000000000000000000040001000000000800000080040000000000080000000000000000000010000000040000000000000000000000000000000000000000008000000000000000000000000080200000000000000004100020000000000000000000800000000000000020000008010000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000002000000000000000000040000000000000000000000000000000020000000000000000000000000000000000000000100000000000000010000000000 state:


Error: invalid merkle root (remote: b5658d9dca913332ab88be9504437fb4205c281d9cd18de4e098689e33873ed5 local: 5bb4d949229f3b2c0501729d20cc4435fab35faa448c53d2a64a6489abb8cd11)
##############################

WARN [09-08|17:03:52.138] Synchronisation failed, dropping peer    peer=cb2bf0beb664e1dad1af77fe86d9eeb33c36f148d8dc9f786c24442ad42483ad err="retrieved hash chain is invalid: invalid merkle root (remote: b5658d9dca913332ab88be9504437fb4205c281d9cd18de4e098689e33873ed5 local: 5bb4d949229f3b2c0501729d20cc4435fab35faa448c53d2a64a6489abb8cd11)"

@ligi
Copy link
Member

ligi commented Sep 8, 2021

It's 2 different things:

@circuitbreaker88 for you it's the ropsten split - please read: ethereum/pm#354 (comment)

@FantomJoker did you have an unclean shutdown?

@FantomJoker
Copy link
Author

@ligi I haven clean all my chaindata, what else should I clean?

@0xcircuitbreaker
Copy link

I used the whitelist command to block, yet I still hit error on the same block.

geth --ropsten --maxpeers 70 --cache 1024 --txpool.globalslots 1024 --syncmode "full" -rpcapi debug,txpool,admin,net,personal,eth,web3 --rpc --whitelist 10920274=0x2fd63eb95c8059a46b5afedb903ef1639b9414e9d38f70b3dfb0f733b7ced14f

Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: 4230000 Petersburg: 4939394 Istanbul: 6485846, Muir Glacier: 7117117, Berlin: 9812189, London: 10499401, Engine: ethash}

Number: 10920274
Hash: 0x2fd63eb95c8059a46b5afedb903ef1639b9414e9d38f70b3dfb0f733b7ced14f
         0: cumulative: 47261 gas: 47261 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x95080369054e617280b3fa77a8c389c6e832ef6075c8574830733eb8599b09ba logs: [0xc2cc7bbce0] bloom: 00000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000004000100000000200000000000000000000000000000012000000000000000000000000000000000000000000000000000000000800000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000020000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000010000000000000000000000000000000000000000000000000000000000000 state:
         1: cumulative: 246246 gas: 198985 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x1e69365f032f87f1dda449fcc1360318590cb9db4981839d7b8eac5fc798d353 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         2: cumulative: 445231 gas: 198985 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x998febc5d866ec03d5e9a0ff8fb478aebb21ec4ca2bd7f3720d3f77bbb0ff7a0 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         3: cumulative: 491390 gas: 46159 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x5b88b08f2b0ae0bc84673c082c5ee4c1d31a5cd8642b181f2761bb9f1b4943fe logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         4: cumulative: 537658 gas: 46268 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x098f383f532592bc45567ced73522f207ff7a633830014b9902c6887314c3717 logs: [0xc2cc7bbd90] bloom: 00000000000000000000000000000000000000000000000000001000000000000000000100000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000004000000000000000000000000000000000000000000000200000000000000000000000000080000000000000020002000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000000000000000000000000002000010000000000000000000000000000000000000000000000000000000000000 state:
         5: cumulative: 593803 gas: 56145 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xb82786bc71eeacb6d4808bcaaffdb397565d15340bedf2e491e5dfae7a6da48d logs: [0xc2cc7bbe40 0xc2cc7bbef0] bloom: 00000000000000000000000000000000000000000000000020000000000000000000000000000800000000000000000000000000000000000000000000000001000000000000000000000040000000000000000000400000000000000000000018000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         6: cumulative: 660136 gas: 66333 contract: 0x4a4511ADDABA6ee5A81448dBa122163CeC3b3cDF status: 1 tx: 0x6ea35435fbef2fc1cc56e252474628662b135fdf821a2ae6903a94fccabbf202 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         7: cumulative: 706283 gas: 46147 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x23de793b440cdbbd2be9c1896764ff954975aedc8c31c52b0ddcfafb019b7161 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         8: cumulative: 752442 gas: 46159 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xec349372328e2582f27f1e4bb45c6ba56b210b974093506186f8fb05bb3a2044 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state:
         9: cumulative: 922780 gas: 170338 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x26b2634cd5d765715d13c667f1ae79f42fb2c175cc5c0f102343b29b13d2e749 logs: [0xc2a1c5cdc0 0xc2a1c5ce70 0xc2a1c5cf20 0xc2a1c5cfd0 0xc2a1c5d080] bloom: 00000000000000020000000000000000000000000000000000000001000004000000000000000000000000201000000000000000200000000000000000000000002000000000000000000008000000040008000010000000000410000000000000000000020000000000000000000800000000000000000020000010000000000000010000020000000000000008000000000000000000000000000100000000000000000000000000000800000000000000000000000000000000000000000000000002000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000020010000000000000000000000000 state:
         10: cumulative: 1056898 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xc4cf32b724efbcc75de35a8f07afb15f71fd56e17b1ed95d7047c31b13693772 logs: [0xc2a1c5d130 0xc2a1c5d1e0] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         11: cumulative: 1191016 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8460a92b5ff1c06c40df6b128a94805a081f007585ff7409cb7d32697fae98c8 logs: [0xc2a1c5d290 0xc2a1c5d340] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         12: cumulative: 1325134 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x40f5ca48896dab3005ed3f2ac3f927f83fb21f7e72f0c819144efb4d1e55fcee logs: [0xc2a1c5d3f0 0xc2a1c5d4a0] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         13: cumulative: 1459252 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x6ffa9139d207505acacd29ef2786d1ca6d10c770eefb921b092e2546da4bab90 logs: [0xc2a1c5d550 0xc2a1c5d600] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state:
         14: cumulative: 1728059 gas: 268807 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x540cf48abff64a05514abf0d5bbf4cddac8bb9bc6e0d482f736f04927345555d logs: [0xc2a1c5d6b0 0xc2a1c5d760 0xc2a1c5d810 0xc2a1c5d8c0 0xc2a1c5d970] bloom: 80000000040100000000004000000000000000000040001000000000800000080040000000000080000000000000000000010000000040000000000000000000000000000000000000000008000000000000000000000000080200000000000000004100020000000000000000000800000000000000020000008010000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000002000000000000000000040000000000000000000000000000000020000000000000000000000000000000000000000100000000000000010000000000 state:


Error: invalid merkle root (remote: b5658d9dca913332ab88be9504437fb4205c281d9cd18de4e098689e33873ed5 local: 5bb4d949229f3b2c0501729d20cc4435fab35faa448c53d2a64a6489abb8cd11)
##############################```

@jsolman jsolman mentioned this issue Sep 10, 2021
Closed
@kuzdogan
Copy link
Member

kuzdogan commented Sep 14, 2021 •
edited
Loading

Geth version 1.10.6 We also had a bad block at 10920274 in logs but now it seems to have converged to one single wrong chain, current block 10920274 is different.

Logs:

########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: 4230000 Petersburg: 4939394 Istanbul: 6485846, Muir Glacier: 7117117, Berlin: 9812189, London: 10499401, Engine: ethash}

Number: 10920274
Hash: 0xfd652086d220d506ae5b7cb80fde97d2f3f7028d346cc7d9d384a83d3d638532
	 0: cumulative: 195152 gas: 195152 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x436e5530b3fc76858d6dcfa10868533d7bbd2307cc06eef10137f8ffdbdf3724 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
	 1: cumulative: 394137 gas: 198985 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x1e69365f032f87f1dda449fcc1360318590cb9db4981839d7b8eac5fc798d353 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
	 2: cumulative: 440296 gas: 46159 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x5b88b08f2b0ae0bc84673c082c5ee4c1d31a5cd8642b181f2761bb9f1b4943fe logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
	 3: cumulative: 486564 gas: 46268 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x098f383f532592bc45567ced73522f207ff7a633830014b9902c6887314c3717 logs: [0xc043b37c30] bloom: 00000000000000000000000000000000000000000000000000001000000000000000000100000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000004000000000000000000000000000000000000000000000200000000000000000000000000080000000000000020002000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000000000000000000000000002000010000000000000000000000000000000000000000000000000000000000000 state: 
	 4: cumulative: 542709 gas: 56145 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xb82786bc71eeacb6d4808bcaaffdb397565d15340bedf2e491e5dfae7a6da48d logs: [0xc043b37ce0 0xc043b37d90] bloom: 00000000000000000000000000000000000000000000000020000000000000000000000000000800000000000000000000000000000000000000000000000001000000000000000000000040000000000000000000400000000000000000000018000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
	 5: cumulative: 609042 gas: 66333 contract: 0x4a4511ADDABA6ee5A81448dBa122163CeC3b3cDF status: 1 tx: 0x6ea35435fbef2fc1cc56e252474628662b135fdf821a2ae6903a94fccabbf202 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
	 6: cumulative: 655189 gas: 46147 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x23de793b440cdbbd2be9c1896764ff954975aedc8c31c52b0ddcfafb019b7161 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
	 7: cumulative: 825527 gas: 170338 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x26b2634cd5d765715d13c667f1ae79f42fb2c175cc5c0f102343b29b13d2e749 logs: [0xc043b37e40 0xc043b37ef0 0xc045ef8000 0xc045ef80b0 0xc045ef8160] bloom: 00000000000000020000000000000000000000000000000000000001000004000000000000000000000000201000000000000000200000000000000000000000002000000000000000000008000000040008000010000000000410000000000000000000020000000000000000000800000000000000000020000010000000000000010000020000000000000008000000000000000000000000000100000000000000000000000000000800000000000000000000000000000000000000000000000002000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000020010000000000000000000000000 state: 
	 8: cumulative: 959645 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xc4cf32b724efbcc75de35a8f07afb15f71fd56e17b1ed95d7047c31b13693772 logs: [0xc045ef8210 0xc045ef82c0] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state: 
	 9: cumulative: 1093763 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8460a92b5ff1c06c40df6b128a94805a081f007585ff7409cb7d32697fae98c8 logs: [0xc045ef8370 0xc045ef8420] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state: 
	 10: cumulative: 1227881 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x40f5ca48896dab3005ed3f2ac3f927f83fb21f7e72f0c819144efb4d1e55fcee logs: [0xc045ef84d0 0xc045ef8580] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state: 
	 11: cumulative: 1361999 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x6ffa9139d207505acacd29ef2786d1ca6d10c770eefb921b092e2546da4bab90 logs: [0xc045ef8630 0xc045ef86e0] bloom: 00000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000040000000000000000000010000000000000000000020000000000000000000800000000000000000020000000000000000000010000020000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000000020000000200000000000000008000000000000000000000000000000000000000000 state: 
	 12: cumulative: 1630806 gas: 268807 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x540cf48abff64a05514abf0d5bbf4cddac8bb9bc6e0d482f736f04927345555d logs: [0xc045ef8790 0xc045ef8840 0xc045ef88f0 0xc045ef89a0 0xc045ef8a50] bloom: 80000000040100000000004000000000000000000040001000000000800000080040000000000080000000000000000000010000000040000000000000000000000000000000000000000008000000000000000000000000080200000000000000004100020000000000000000000800000000000000020000008010000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000002000000000000000000040000000000000000000000000000000020000000000000000000000000000000000000000100000000000000010000000000 state: 


Error: invalid merkle root (remote: 085cf200399752c0bcb8cd472740f98c6584f389b81bd3da41e1421328d9b8bc local: 330d2257da6a28e770a724cb4bbbb46bdb5dde521acabe40c55ed9d3b8bfb961)
##############################

RPC call getBlockByNumber on block 10920273 returns the correct block with hash 0x1a481511d4b8e7a2ab55e5e03eabc81be7ccda47763da9ab18414074ae678355 etherscan

But on block 10920274 it returns the wrong block etherscan:

{
    "jsonrpc": "2.0",
    "id": 1,
    "result": {
        "baseFeePerGas": "0x7",
        "difficulty": "0x54937ae3",
        "extraData": "0x627472632d3031",
        "gasLimit": "0x7a4f0e",
        "gasUsed": "0x1a5e3b",
        "hash": "0x2fd63eb95c8059a46b5afedb903ef1639b9414e9d38f70b3dfb0f733b7ced14f",
        "logsBloom": "0x80000000040100020000004001000000000000000040001020001001800004080040000100000880000000201000000000010000200044000100000000200001002000000000000000000048012000040008000010400000080610000000000018004100020000000000800000000802000080004000020020008010000000020000010000020000000000200008000000004000000000000080000100000000020002000000000000000801008000000000000400000000000000000000000000000002000000000008000000040000000000005000001000000000000022000010200000000000000008000000000000020110000000000000010000000000",
        "miner": "0xca3f57dffbcf67c074b8cf54e4c873138facfc7f",
        "mixHash": "0x5cb4c52e2db39991b41d7b186f46ad2a59c06f7dabc46ee88039a4909e4cf85f",
        "nonce": "0xfdee7d24494c67b9",
        "number": "0xa6a152",
        "parentHash": "0x1a481511d4b8e7a2ab55e5e03eabc81be7ccda47763da9ab18414074ae678355",
        "receiptsRoot": "0xa1d98d854d3156c2fd1dc7d64c15b3eb70e6daed349d92dd3f00ef0d57961584",
        "sha3Uncles": "0xd717359d939712d5b101b2db4d305f9db75493d361dc8bdd6435ea922ec2a40e",
        "size": "0x1149",
        "stateRoot": "0xb5658d9dca913332ab88be9504437fb4205c281d9cd18de4e098689e33873ed5",
        "timestamp": "0x6128f786",
        "totalDifficulty": "0x7b25ea1bd78242",
        "transactions": [...]
        "transactionsRoot": "0x35e20dce2b3250b4271031e6269f8f804194a5985a09d7664d7b35c6dbc38bdd",
        "uncles": [
            "0x5de7d69d9701675e9c8ad09cda865a042cd0c73c01f086a2399356fb6b655786"
        ]
    }
}

Edit:

After only --whitelisting it failed to sync. I both used debug.setHead and --whitelisted. In my case the node was in sync at block 10920273 but forked at 10920274. So did debug.setHead(10920273) via RPC and and added the flag --whitelist 109020274=0xfd652086d220d506ae5b7cb80fde97d2f3f7028d346cc7d9d384a83d3d638532. I also got an invalid merkle root (with correct hash?). What I noticed is the transactions are in reverse order in comparison to etherscan 🤔, and maybe hence the wrong Merkle root.

geth-ropsten | INFO [09-14|12:08:38.479] Skip duplicated bad block                number=10,920,274 hash=fd6520..638532
geth-ropsten | ERROR[09-14|12:08:38.484] 
geth-ropsten | ########## BAD BLOCK #########
geth-ropsten | Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: 4230000 Petersburg: 4939394 Istanbul: 6485846, Muir Glacier: 7117117, Berlin: 9812189, London: 10499401, Engine: ethash}
geth-ropsten | 
geth-ropsten | Number: 10920274
geth-ropsten | Hash: 0xfd652086d220d506ae5b7cb80fde97d2f3f7028d346cc7d9d384a83d3d638532
geth-ropsten | 	 0: cumulative: 195152 gas: 195152 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x436e5530b3fc76858d6dcfa10868533d7bbd2307cc06eef10137f8ffdbdf3724 logs: [] bloom: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 state: 
geth-ropsten | 	 1: cumulative: 394137 gas: 198985 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x1e69365f032f87f1dda449fcc1360318590cb9db4981839d7b8eac5fc798d353 logs:...
geth-ropsten | 	 2: cumulative: 440296 gas: 46159 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x5b88b08f2b0ae0bc84673c082c5ee4c1d31a5cd8642b181f2761bb9f1b4943fe logs:...
geth-ropsten | 	 3: cumulative: 486564 gas: 46268 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x098f383f532592bc45567ced73522f207ff7a633830014b9902c6887314c3717 logs:...
geth-ropsten | 	 4: cumulative: 542709 gas: 56145 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xb82786bc71eeacb6d4808bcaaffdb397565d15340bedf2e491e5dfae7a6da48d logs:...
geth-ropsten | 	 5: cumulative: 609042 gas: 66333 contract: 0x4a4511ADDABA6ee5A81448dBa122163CeC3b3cDF status: 1 tx: 0x6ea35435fbef2fc1cc56e252474628662b135fdf821a2ae6903a94fccabbf202 logs:...
geth-ropsten | 	 6: cumulative: 655189 gas: 46147 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x23de793b440cdbbd2be9c1896764ff954975aedc8c31c52b0ddcfafb019b7161 logs:...
geth-ropsten | 	 7: cumulative: 825527 gas: 170338 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x26b2634cd5d765715d13c667f1ae79f42fb2c175cc5c0f102343b29b13d2e749 logs:...
geth-ropsten | 	 8: cumulative: 959645 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0xc4cf32b724efbcc75de35a8f07afb15f71fd56e17b1ed95d7047c31b13693772 logs:...
geth-ropsten | 	 9: cumulative: 1093763 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x8460a92b5ff1c06c40df6b128a94805a081f007585ff7409cb7d32697fae98c8 logs:...
geth-ropsten | 	 10: cumulative: 1227881 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x40f5ca48896dab3005ed3f2ac3f927f83fb21f7e72f0c819144efb4d1e55fcee logs:...
geth-ropsten | 	 11: cumulative: 1361999 gas: 134118 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x6ffa9139d207505acacd29ef2786d1ca6d10c770eefb921b092e2546da4bab90 logs:...
geth-ropsten | 	 12: cumulative: 1630806 gas: 268807 contract: 0x0000000000000000000000000000000000000000 status: 1 tx: 0x540cf48abff64a05514abf0d5bbf4cddac8bb9bc6e0d482f736f04927345555d logs:...
geth-ropsten | 
geth-ropsten | 
geth-ropsten | Error: invalid merkle root (remote: 085cf200399752c0bcb8cd472740f98c6584f389b81bd3da41e1421328d9b8bc local: 330d2257da6a28e770a724cb4bbbb46bdb5dde521acabe40c55ed9d3b8bfb961)
geth-ropsten | ##############################
geth-ropsten |  
geth-ropsten | WARN [09-14|12:08:38.487] Synchronisation failed, dropping peer    peer=cbcec50a9234c6ce33a9d57db13d11d3fb69a35ea67414fea40b2d33ef159a2e err="retrieved hash chain is invalid: invalid merkle root (remote: 085cf200399752c0bcb8cd472740f98c6584f389b81bd3da41e1421328d9b8bc local: 330d2257da6a28e770a724cb4bbbb46bdb5dde521acabe40c55ed9d3b8bfb961)"
geth-ropsten | INFO [09-14|12:08:40.885] Whitelist mismatch, dropping peer        id=cb2bf0beb664e1da conn=dyndial            number=10,920,274 hash=2fd63e..ced14f want=fd6520..638532

@kuzdogan
Copy link
Member

I also noticed you are providing the incorrect hash for the whitelist @circuitbreaker88 . Assuming the Etherscan is correct you should pass --whitelist 10920274=0xfd652086d220d506ae5b7cb80fde97d2f3f7028d346cc7d9d384a83d3d638532. Interesting that we both fork at the same block but not at 10679538 where issue came up.

@kuzdogan
Copy link
Member

Other issues reporting split at the same block 10920274

#23503

@0xcircuitbreaker
Copy link

Got it, trying sync once more will report results

@holiman
Copy link
Contributor

holiman commented Oct 20, 2021

trying sync once more will report results

It's been a month, I'll assume this was resolved

@holiman holiman closed this as completed Oct 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type:bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ligi @holiman @kuzdogan @0xcircuitbreaker @FantomJoker