Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to @chainsafe/discv5 v3.0.0 #5073

Merged
merged 5 commits into from
Feb 1, 2023
Merged

Update to @chainsafe/discv5 v3.0.0 #5073

merged 5 commits into from
Feb 1, 2023

Conversation

wemeetagain
Copy link
Member

Motivation

Several things:

  • Our ENR api was error-prone, and was refactored in 3.0.0
  • We had to re-encode ENRs across the worker boundary (which costs us a signature verification in the main thread)
  • We frequently had errors decoding ENRs (that we had successfully decoded and re-encoded within the worker)

Description

  • Update to @chainsafe/discv5 v3.0.0
  • Use new toObject / fromObject to pass ENRs as objects across the thread boundary (no encoding/decoding necessary, no decoding errors possible)

@wemeetagain wemeetagain requested a review from a team as a code owner January 28, 2023 19:29
@github-actions
Copy link
Contributor

github-actions bot commented Jan 28, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 35f9f59 Previous: b679987 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 832.94 us/op 751.60 us/op 1.11
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 44.507 us/op 44.454 us/op 1.00
BLS verify - blst-native 1.1967 ms/op 1.1857 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.4318 ms/op 2.4051 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 5.2237 ms/op 5.1754 ms/op 1.01
BLS verifyMultipleSignatures 32 - blst-native 18.874 ms/op 18.772 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 25.206 us/op 25.138 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 99.398 us/op 97.892 us/op 1.02
getAttestationsForBlock 52.475 ms/op 50.448 ms/op 1.04
isKnown best case - 1 super set check 263.00 ns/op 262.00 ns/op 1.00
isKnown normal case - 2 super set checks 243.00 ns/op 253.00 ns/op 0.96
isKnown worse case - 16 super set checks 244.00 ns/op 251.00 ns/op 0.97
CheckpointStateCache - add get delete 5.0430 us/op 5.0940 us/op 0.99
validate gossip signedAggregateAndProof - struct 2.7285 ms/op 2.6903 ms/op 1.01
validate gossip attestation - struct 1.3101 ms/op 1.3135 ms/op 1.00
pickEth1Vote - no votes 1.3056 ms/op 1.2251 ms/op 1.07
pickEth1Vote - max votes 11.491 ms/op 10.070 ms/op 1.14
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.1376 ms/op 8.7433 ms/op 0.93
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.138 ms/op 13.958 ms/op 1.01
pickEth1Vote - Eth1Data fastSerialize value x2048 673.18 us/op 620.86 us/op 1.08
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.6017 ms/op 7.8640 ms/op 0.97
bytes32 toHexString 477.00 ns/op 476.00 ns/op 1.00
bytes32 Buffer.toString(hex) 330.00 ns/op 333.00 ns/op 0.99
bytes32 Buffer.toString(hex) from Uint8Array 527.00 ns/op 552.00 ns/op 0.95
bytes32 Buffer.toString(hex) + 0x 326.00 ns/op 333.00 ns/op 0.98
Object access 1 prop 0.15800 ns/op 0.16500 ns/op 0.96
Map access 1 prop 0.15400 ns/op 0.16400 ns/op 0.94
Object get x1000 6.2630 ns/op 6.3900 ns/op 0.98
Map get x1000 0.52700 ns/op 0.59100 ns/op 0.89
Object set x1000 50.395 ns/op 51.139 ns/op 0.99
Map set x1000 42.050 ns/op 43.050 ns/op 0.98
Return object 10000 times 0.23030 ns/op 0.23300 ns/op 0.99
Throw Error 10000 times 4.0163 us/op 4.0898 us/op 0.98
fastMsgIdFn sha256 / 200 bytes 3.3520 us/op 3.3880 us/op 0.99
fastMsgIdFn h32 xxhash / 200 bytes 272.00 ns/op 277.00 ns/op 0.98
fastMsgIdFn h64 xxhash / 200 bytes 370.00 ns/op 379.00 ns/op 0.98
fastMsgIdFn sha256 / 1000 bytes 11.304 us/op 11.573 us/op 0.98
fastMsgIdFn h32 xxhash / 1000 bytes 396.00 ns/op 395.00 ns/op 1.00
fastMsgIdFn h64 xxhash / 1000 bytes 439.00 ns/op 464.00 ns/op 0.95
fastMsgIdFn sha256 / 10000 bytes 101.78 us/op 104.59 us/op 0.97
fastMsgIdFn h32 xxhash / 10000 bytes 1.8600 us/op 2.2960 us/op 0.81
fastMsgIdFn h64 xxhash / 10000 bytes 1.3160 us/op 1.6750 us/op 0.79
enrSubnets - fastDeserialize 64 bits 1.2310 us/op 1.3920 us/op 0.88
enrSubnets - ssz BitVector 64 bits 466.00 ns/op 511.00 ns/op 0.91
enrSubnets - fastDeserialize 4 bits 163.00 ns/op 170.00 ns/op 0.96
enrSubnets - ssz BitVector 4 bits 457.00 ns/op 494.00 ns/op 0.93
prioritizePeers score -10:0 att 32-0.1 sync 2-0 97.414 us/op 98.372 us/op 0.99
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 126.41 us/op 126.15 us/op 1.00
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 159.27 us/op 172.06 us/op 0.93
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 292.68 us/op 303.48 us/op 0.96
prioritizePeers score 0:0 att 64-1 sync 4-1 346.64 us/op 354.35 us/op 0.98
array of 16000 items push then shift 1.6040 us/op 1.6674 us/op 0.96
LinkedList of 16000 items push then shift 8.7210 ns/op 8.9230 ns/op 0.98
array of 16000 items push then pop 79.488 ns/op 99.166 ns/op 0.80
LinkedList of 16000 items push then pop 8.2790 ns/op 8.6470 ns/op 0.96
array of 24000 items push then shift 2.2753 us/op 2.3436 us/op 0.97
LinkedList of 24000 items push then shift 8.6590 ns/op 8.7860 ns/op 0.99
array of 24000 items push then pop 75.373 ns/op 76.534 ns/op 0.98
LinkedList of 24000 items push then pop 8.3830 ns/op 8.5510 ns/op 0.98
intersect bitArray bitLen 8 13.213 ns/op 13.424 ns/op 0.98
intersect array and set length 8 76.078 ns/op 78.915 ns/op 0.96
intersect bitArray bitLen 128 43.909 ns/op 44.130 ns/op 0.99
intersect array and set length 128 1.0345 us/op 1.0573 us/op 0.98
Buffer.concat 32 items 2.6670 us/op 2.6960 us/op 0.99
Uint8Array.set 32 items 2.8400 us/op 2.1010 us/op 1.35
pass gossip attestations to forkchoice per slot 3.8651 ms/op 2.3233 ms/op 1.66
computeDeltas 3.4169 ms/op 3.2063 ms/op 1.07
computeProposerBoostScoreFromBalances 2.1744 ms/op 1.8096 ms/op 1.20
altair processAttestation - 250000 vs - 7PWei normalcase 2.0612 ms/op 2.4965 ms/op 0.83
altair processAttestation - 250000 vs - 7PWei worstcase 3.2439 ms/op 5.0760 ms/op 0.64
altair processAttestation - setStatus - 1/6 committees join 138.74 us/op 148.40 us/op 0.93
altair processAttestation - setStatus - 1/3 committees join 272.12 us/op 295.39 us/op 0.92
altair processAttestation - setStatus - 1/2 committees join 363.89 us/op 390.31 us/op 0.93
altair processAttestation - setStatus - 2/3 committees join 461.75 us/op 490.10 us/op 0.94
altair processAttestation - setStatus - 4/5 committees join 644.23 us/op 684.32 us/op 0.94
altair processAttestation - setStatus - 100% committees join 743.76 us/op 808.88 us/op 0.92
altair processBlock - 250000 vs - 7PWei normalcase 18.196 ms/op 18.723 ms/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase hashState 29.217 ms/op 26.794 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase 47.086 ms/op 56.250 ms/op 0.84
altair processBlock - 250000 vs - 7PWei worstcase hashState 66.397 ms/op 79.960 ms/op 0.83
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9515 ms/op 2.3489 ms/op 0.83
phase0 processBlock - 250000 vs - 7PWei worstcase 26.993 ms/op 32.574 ms/op 0.83
altair processEth1Data - 250000 vs - 7PWei normalcase 471.82 us/op 562.16 us/op 0.84
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8500 us/op 9.5660 us/op 0.72
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.520 us/op 32.973 us/op 0.59
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.2780 us/op 13.008 us/op 0.64
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.4300 us/op 9.9790 us/op 0.64
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 74.866 us/op 119.03 us/op 0.63
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 607.02 us/op 716.85 us/op 0.85
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 908.41 us/op 925.27 us/op 0.98
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 857.42 us/op 920.27 us/op 0.93
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.2561 ms/op 2.9871 ms/op 0.76
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.4710 ms/op 1.7664 ms/op 0.83
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.6934 ms/op 4.4269 ms/op 0.83
Tree 40 250000 create 298.88 ms/op 470.98 ms/op 0.63
Tree 40 250000 get(125000) 177.63 ns/op 202.51 ns/op 0.88
Tree 40 250000 set(125000) 884.72 ns/op 1.3354 us/op 0.66
Tree 40 250000 toArray() 16.328 ms/op 23.140 ms/op 0.71
Tree 40 250000 iterate all - toArray() + loop 16.368 ms/op 23.339 ms/op 0.70
Tree 40 250000 iterate all - get(i) 66.757 ms/op 79.158 ms/op 0.84
MutableVector 250000 create 10.351 ms/op 12.693 ms/op 0.82
MutableVector 250000 get(125000) 6.4110 ns/op 6.6580 ns/op 0.96
MutableVector 250000 set(125000) 255.30 ns/op 287.59 ns/op 0.89
MutableVector 250000 toArray() 2.7976 ms/op 3.5765 ms/op 0.78
MutableVector 250000 iterate all - toArray() + loop 2.8472 ms/op 3.6904 ms/op 0.77
MutableVector 250000 iterate all - get(i) 1.5037 ms/op 1.6133 ms/op 0.93
Array 250000 create 2.5179 ms/op 3.4469 ms/op 0.73
Array 250000 clone - spread 1.2466 ms/op 1.1368 ms/op 1.10
Array 250000 get(125000) 0.59600 ns/op 0.63200 ns/op 0.94
Array 250000 set(125000) 0.68700 ns/op 0.71600 ns/op 0.96
Array 250000 iterate all - loop 81.675 us/op 92.048 us/op 0.89
effectiveBalanceIncrements clone Uint8Array 300000 27.497 us/op 50.981 us/op 0.54
effectiveBalanceIncrements clone MutableVector 300000 415.00 ns/op 431.00 ns/op 0.96
effectiveBalanceIncrements rw all Uint8Array 300000 166.35 us/op 174.33 us/op 0.95
effectiveBalanceIncrements rw all MutableVector 300000 82.965 ms/op 119.00 ms/op 0.70
phase0 afterProcessEpoch - 250000 vs - 7PWei 116.06 ms/op 120.53 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 35.991 ms/op 49.104 ms/op 0.73
altair processEpoch - mainnet_e81889 324.05 ms/op 323.59 ms/op 1.00
mainnet_e81889 - altair beforeProcessEpoch 60.004 ms/op 54.248 ms/op 1.11
mainnet_e81889 - altair processJustificationAndFinalization 19.230 us/op 27.985 us/op 0.69
mainnet_e81889 - altair processInactivityUpdates 5.7124 ms/op 7.9694 ms/op 0.72
mainnet_e81889 - altair processRewardsAndPenalties 51.519 ms/op 56.759 ms/op 0.91
mainnet_e81889 - altair processRegistryUpdates 2.8720 us/op 5.3640 us/op 0.54
mainnet_e81889 - altair processSlashings 458.00 ns/op 1.3870 us/op 0.33
mainnet_e81889 - altair processEth1DataReset 549.00 ns/op 1.1120 us/op 0.49
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2004 ms/op 1.4606 ms/op 0.82
mainnet_e81889 - altair processSlashingsReset 5.1250 us/op 6.2260 us/op 0.82
mainnet_e81889 - altair processRandaoMixesReset 4.5400 us/op 6.2620 us/op 0.73
mainnet_e81889 - altair processHistoricalRootsUpdate 617.00 ns/op 1.0640 us/op 0.58
mainnet_e81889 - altair processParticipationFlagUpdates 2.3920 us/op 3.2930 us/op 0.73
mainnet_e81889 - altair processSyncCommitteeUpdates 525.00 ns/op 872.00 ns/op 0.60
mainnet_e81889 - altair afterProcessEpoch 116.23 ms/op 133.51 ms/op 0.87
phase0 processEpoch - mainnet_e58758 308.92 ms/op 391.51 ms/op 0.79
mainnet_e58758 - phase0 beforeProcessEpoch 122.00 ms/op 157.86 ms/op 0.77
mainnet_e58758 - phase0 processJustificationAndFinalization 15.415 us/op 18.734 us/op 0.82
mainnet_e58758 - phase0 processRewardsAndPenalties 55.677 ms/op 69.536 ms/op 0.80
mainnet_e58758 - phase0 processRegistryUpdates 7.6770 us/op 10.771 us/op 0.71
mainnet_e58758 - phase0 processSlashings 515.00 ns/op 733.00 ns/op 0.70
mainnet_e58758 - phase0 processEth1DataReset 540.00 ns/op 671.00 ns/op 0.80
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 942.45 us/op 1.0466 ms/op 0.90
mainnet_e58758 - phase0 processSlashingsReset 2.8240 us/op 3.7450 us/op 0.75
mainnet_e58758 - phase0 processRandaoMixesReset 4.6570 us/op 4.7790 us/op 0.97
mainnet_e58758 - phase0 processHistoricalRootsUpdate 598.00 ns/op 618.00 ns/op 0.97
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.0140 us/op 4.6330 us/op 0.87
mainnet_e58758 - phase0 afterProcessEpoch 96.071 ms/op 102.41 ms/op 0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2113 ms/op 1.2998 ms/op 0.93
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4228 ms/op 1.5475 ms/op 0.92
altair processInactivityUpdates - 250000 normalcase 20.096 ms/op 26.710 ms/op 0.75
altair processInactivityUpdates - 250000 worstcase 23.193 ms/op 27.181 ms/op 0.85
phase0 processRegistryUpdates - 250000 normalcase 6.5090 us/op 6.8910 us/op 0.94
phase0 processRegistryUpdates - 250000 badcase_full_deposits 232.53 us/op 269.05 us/op 0.86
phase0 processRegistryUpdates - 250000 worstcase 0.5 113.56 ms/op 122.27 ms/op 0.93
altair processRewardsAndPenalties - 250000 normalcase 56.762 ms/op 68.204 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 52.328 ms/op 68.387 ms/op 0.77
phase0 getAttestationDeltas - 250000 normalcase 6.3346 ms/op 6.7418 ms/op 0.94
phase0 getAttestationDeltas - 250000 worstcase 6.5775 ms/op 6.7355 ms/op 0.98
phase0 processSlashings - 250000 worstcase 3.3442 ms/op 3.5412 ms/op 0.94
altair processSyncCommitteeUpdates - 250000 172.61 ms/op 170.72 ms/op 1.01
BeaconState.hashTreeRoot - No change 252.00 ns/op 315.00 ns/op 0.80
BeaconState.hashTreeRoot - 1 full validator 49.320 us/op 49.408 us/op 1.00
BeaconState.hashTreeRoot - 32 full validator 484.47 us/op 527.38 us/op 0.92
BeaconState.hashTreeRoot - 512 full validator 5.5395 ms/op 5.4656 ms/op 1.01
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 60.584 us/op 66.996 us/op 0.90
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 840.83 us/op 975.00 us/op 0.86
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.022 ms/op 13.203 ms/op 0.83
BeaconState.hashTreeRoot - 1 balances 47.387 us/op 49.932 us/op 0.95
BeaconState.hashTreeRoot - 32 balances 427.47 us/op 486.66 us/op 0.88
BeaconState.hashTreeRoot - 512 balances 4.2391 ms/op 4.6073 ms/op 0.92
BeaconState.hashTreeRoot - 250000 balances 72.147 ms/op 72.742 ms/op 0.99
aggregationBits - 2048 els - zipIndexesInBitList 16.307 us/op 23.569 us/op 0.69
regular array get 100000 times 32.444 us/op 35.944 us/op 0.90
wrappedArray get 100000 times 32.363 us/op 35.769 us/op 0.90
arrayWithProxy get 100000 times 15.809 ms/op 15.533 ms/op 1.02
ssz.Root.equals 549.00 ns/op 648.00 ns/op 0.85
byteArrayEquals 545.00 ns/op 603.00 ns/op 0.90
shuffle list - 16384 els 6.6182 ms/op 7.2277 ms/op 0.92
shuffle list - 250000 els 97.018 ms/op 106.36 ms/op 0.91
processSlot - 1 slots 8.4570 us/op 9.8240 us/op 0.86
processSlot - 32 slots 1.3094 ms/op 1.4705 ms/op 0.89
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 186.40 us/op 221.78 us/op 0.84
getCommitteeAssignments - req 1 vs - 250000 vc 2.8693 ms/op 3.1004 ms/op 0.93
getCommitteeAssignments - req 100 vs - 250000 vc 4.0998 ms/op 4.4042 ms/op 0.93
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4001 ms/op 4.7693 ms/op 0.92
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3700 ns/op 5.2300 ns/op 0.84
state getBlockRootAtSlot - 250000 vs - 7PWei 930.62 ns/op 669.78 ns/op 1.39
computeProposers - vc 250000 10.512 ms/op 10.883 ms/op 0.97
computeEpochShuffling - vc 250000 98.110 ms/op 106.71 ms/op 0.92
getNextSyncCommittee - vc 250000 168.56 ms/op 182.11 ms/op 0.93

by benchmarkbot/action

@wemeetagain
Copy link
Member Author

deployed this branch to feat3

@wemeetagain wemeetagain mentioned this pull request Jan 31, 2023
Closed
@philknows philknows added this to the v1.4.1 milestone Jan 31, 2023
@wemeetagain
Copy link
Member Author

From my profiling, #5084 is no longer visible in this branch

@twoeths
Copy link
Contributor

twoeths commented Feb 1, 2023

From my profiling, #5084 is no longer visible in this branch

it does not happen with goerli nodes or prod mainnet node, just test mainnet node subscribing to all subnets

twoeths
twoeths approved these changes Feb 1, 2023
View reviewed changes
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@twoeths twoeths merged commit e8db281 into unstable Feb 1, 2023
@twoeths twoeths deleted the cayman/update-discv5 branch February 1, 2023 06:08
wemeetagain added a commit that referenced this pull request Feb 1, 2023
@wemeetagain @twoeths
Update to @chainsafe/discv5 v3.0.0 (#5073)
b033491 
* Update to @chainsafe/discv5 v3.0.0

* Fix createNodeJsLibp2p

* Fix linter error

* Fix linter errors

* Minor refactor enrStr to enrObj

---------

Co-authored-by: Tuyen Nguyen <[email protected]>
This was referenced Feb 5, 2023
Closed
Merged
@wemeetagain wemeetagain mentioned this pull request Feb 13, 2023
Closed
3 tasks
@wemeetagain
Copy link
Member Author

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.4.1
Development

Successfully merging this pull request may close these issues.
3 participants
@wemeetagain @twoeths @philknows