Synced onto a fork #1744
Comments
nazar-pc
added
the
open-problem
nazar-pc
added
bug
|
@rahulksnv I believe we have some latency issue with block relay. Here is my node: Here is Jeremy's node: If you open them side by side you can see that everything was fine until block 2368, Jeremy's node produced it and my node got it roughly a second later. What happens next is interesting because my node produced block 2369 was produced on both nodes at xx:18:52, fair enough, neither knows what the future chain will look like and sticks to their block for now. Block 2370 was produced by my node at xx:19:00, but Jeremy's node has not seen it and produced own block at xx:19:04. From there chains diverged further and further until they stopped being able to import each other's blocks at all saying "block has an unknown parent". This looks suspiciously potentially related to block relay we have enabled in this devnet. UPD: I was running two nodes locally, both got stuck exactly the same way and continue running in sync with each other. Do you think it is possible for new block relay to introduce multiple seconds of latency under some circumstances? |
|
Can we enable more logs on the block relay side? (couple of logs are trace right now, we can bump it up) |
|
Potentially yes. Looks like all nodes in the US remained with Jeremy, my nodes were further away and forked. Will notify everyone on Slack. |
|
#1748 waiting to go in. The issue may potentially be what we discussed before: sending 1 block at at time, which may slow down the sync. I will try to make a change for this issue meanwhile |
|
There must be something else. We were sending just one block at a time because both nodes were synced, also I have not seen anything special about those blocks like segment headers produced around that time or similar, just a few votes and that is is, not a big block at all. |
|
There is paritytech/polkadot-sdk#493 upstream, for compact blocks side of things I believe we have fixed it and DSN sync was resolved in #1803. Closing for now, will reopen if it keeps happening with compact blocks enabled by default. |
Sync without DSN ended up on a fork, might be related to new block relay protocol or might be not. Needs reliable reproduction.
Basically my node managed to sync 128 blocks, I stopped it, then started again and got this:
As you can see it was going successfully, but then it got block 1991 from a fork, failed to import 1992 and got stuck there 😕
Tried to reproduce, didn't work, so probably not very common issue, but still concerning.
None of the blocks are finalized, so I don't understand why it didn't converge to canonical chain.
The text was updated successfully, but these errors were encountered: