Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sc-consensus-beefy: improve gossip logic #1852

Merged
merged 5 commits into from
Oct 13, 2023
Merged

sc-consensus-beefy: improve gossip logic #1852

merged 5 commits into from
Oct 13, 2023

Conversation

acatangiu
Copy link
Contributor

  • Remove cached messages used for deduplication in GossipValidator since they're already deduplicated in upper layer NetworkGossip.
  • Add cache for "justified rounds" to quickly discard any further (even if potentially different) justifications at the gossip level, once a valid one (for a respective round) is submitted to the worker.
  • Add short-circuit in worker finalize() method to not attempt to finalize same block multiple times (for example when we get justifications for same block from multiple components like block-import, gossip or on-demand).
  • Change a test which had A LOT of latency in syncing blocks for some weird reason and would only run after ~150seconds. It now runs instantly.

Fixes #1728

@acatangiu
sc-consensus-beefy: fix laggy test
db157c5
@acatangiu
sc-consensus-beefy: do not attempt to finalize blocks already finalized
f2ed623
@acatangiu
sc-consensus-beefy: remove votes cache from GossipValidator
08be4f4 
Duplicated messages are already filtered one layer above in network-gossip
state_machine. No need for this extra complexity and locking here.

Instead add a "justified rounds" cache to quickly discard any further (even
if potentially different) justifications at the gossip level, once a valid
one is submitted to the worker.
@acatangiu acatangiu added the T0-node This PR/Issue is related to the topic “node”. label Oct 11, 2023
@acatangiu acatangiu self-assigned this Oct 11, 2023
@acatangiu acatangiu mentioned this pull request Oct 11, 2023
Closed
@acatangiu acatangiu enabled auto-merge (squash) October 13, 2023 05:52
@acatangiu acatangiu merged commit 8320600 into paritytech:master Oct 13, 2023
108 of 109 checks passed
@acatangiu acatangiu deleted the improve-beefy-gossip branch October 13, 2023 08:38
ordian added a commit that referenced this pull request Oct 16, 2023
@ordian
Merge branch 'master' into tsv-disabling-backing
c44093d 
* master: (54 commits)
  Publish `xcm-emulator` crate (#1881)
  Adding migrations to clean Rococo Gov 1 storage & reserved funds (#1849)
  Arkworks Elliptic Curve utils overhaul (#1870)
  Fix typos (#1878)
  fix: GoAhead signal only set when runtime upgrade is enacted from parachain side (#1176)
  Refactor staking ledger (#1484)
  Paired-key Crypto Scheme (#1705)
  Include polkadot version in artifact path (#1828)
  add link to rfc-0001 in broker README (#1862)
  Discard `Executor` (#1855)
  Macros to use path instead of ident (#1474)
  Remove clippy clone-double-ref lint noise (#1860)
  Refactor alliance benchmarks to v2 (#1868)
  Check executor params coherence (#1774)
  frame: use derive-impl for beefy and mmr pallets (#1867)
  sc-consensus-beefy: improve gossip logic (#1852)
  Adds instance support for composite enums (#1857)
  Fix links to implementers' guide (#1865)
  Disabled validators runtime API (#1257)
  Adding `try_state` hook for `Treasury` pallet (#1820)
  ...
ordian added a commit that referenced this pull request Oct 16, 2023
@ordian
Merge branch 'tsv-disabling-backing' into ao-disabling-statement-dist…
b67c19f 
…ribution

* tsv-disabling-backing: (54 commits)
  Publish `xcm-emulator` crate (#1881)
  Adding migrations to clean Rococo Gov 1 storage & reserved funds (#1849)
  Arkworks Elliptic Curve utils overhaul (#1870)
  Fix typos (#1878)
  fix: GoAhead signal only set when runtime upgrade is enacted from parachain side (#1176)
  Refactor staking ledger (#1484)
  Paired-key Crypto Scheme (#1705)
  Include polkadot version in artifact path (#1828)
  add link to rfc-0001 in broker README (#1862)
  Discard `Executor` (#1855)
  Macros to use path instead of ident (#1474)
  Remove clippy clone-double-ref lint noise (#1860)
  Refactor alliance benchmarks to v2 (#1868)
  Check executor params coherence (#1774)
  frame: use derive-impl for beefy and mmr pallets (#1867)
  sc-consensus-beefy: improve gossip logic (#1852)
  Adds instance support for composite enums (#1857)
  Fix links to implementers' guide (#1865)
  Disabled validators runtime API (#1257)
  Adding `try_state` hook for `Treasury` pallet (#1820)
  ...
@kiltbot kiltbot mentioned this pull request Nov 7, 2023
Closed
@ahmadkaouk ahmadkaouk mentioned this pull request Nov 16, 2023
Closed
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
@acatangiu
sc-consensus-beefy: improve gossip logic (paritytech#1852)
0e01998 
- Remove cached messages used for deduplication in `GossipValidator`
since they're already deduplicated in upper layer `NetworkGossip`.
- Add cache for "justified rounds" to quickly discard any further (even
if potentially different) justifications at the gossip level, once a
valid one (for a respective round) is submitted to the worker.
- Add short-circuit in worker `finalize()` method to not attempt to
finalize same block multiple times (for example when we get
justifications for same block from multiple components like
block-import, gossip or on-demand).
- Change a test which had A LOT of latency in syncing blocks for some
weird reason and would only run after ~150seconds. It now runs
instantly.

Fixes paritytech#1728
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Mar 26, 2024
@svyatonik
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
72e64a3 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Mar 27, 2024
@svyatonik
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
6a1c1c1 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@svyatonik
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
65bcd2b 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
8a43b69 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
283264b 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
3a3f33c 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
45778c9 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
abcaa3f 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
41bc149 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
fa197f6 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
d4fe018 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
ddb2da7 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
acbd191 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 10, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
31d2c49 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 10, 2024
@svyatonik @serban300
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
b578d08 
…00_000 to 2_048 (paritytech#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
bkchr pushed a commit that referenced this pull request Apr 10, 2024
@svyatonik @bkchr
Decrease number of GRANDPA authorities in Polkadot-like chains from 1…
411150b 
…00_000 to 2_048 (#1852)

* decrease number of GRANDPA authorities in Polkadot-like chains from 100_000 to 2048

* spelling
This was referenced Jun 5, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@svyatonik svyatonik svyatonik approved these changes

@serban300 serban300 serban300 approved these changes

@andresilva andresilva Awaiting requested review from andresilva

Assignees

@acatangiu acatangiu

Labels
T0-node This PR/Issue is related to the topic “node”.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Rococo beefy error: BadJustification("Duplicate consensus engine ID") on appending justification
3 participants
@acatangiu @svyatonik @serban300