Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix random subnet subscription #4930

Merged
merged 7 commits into from
Dec 22, 2022
Merged

Fix random subnet subscription #4930

merged 7 commits into from
Dec 22, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Dec 22, 2022

Motivation

  • Random subscription starts at a random slot in epoch and unsubscribed per onEpoch check so there are some inactive slots. This is unexpected because for a node with >= 64 validators we want to always subscribe to all random subnets
  • During the inactive slots, a committee subscription may be expired and we unsubscribe the subnet topic
  • Due to this change in v1.3.0 Do not unsubscribe subnets if not less than 64 validators #4829 we don't have a chance to rebalance random subscriptions

Description

  • Check the random subscription per onSlot instead of onEpoch, do it before committee subscription
  • Bring back attnetsService unit test, reproduce the issue in a test case. Since ESM support, there is ES Modules cannot be stubbed error so I have to put in randBetweenFn, shuffleFn to do the unit test
  • Add metrics for future investigation if any issues happen

Closes #4929

@github-actions
Copy link
Contributor

github-actions bot commented Dec 22, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: d7f6e1d Previous: 176b57e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0026 ms/op 1.5131 ms/op 0.66
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 60.014 us/op 100.06 us/op 0.60
BLS verify - blst-native 2.1772 ms/op 2.1759 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 4.4899 ms/op 4.4237 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 9.6953 ms/op 9.4559 ms/op 1.03
BLS verifyMultipleSignatures 32 - blst-native 35.227 ms/op 34.716 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 46.393 us/op 47.475 us/op 0.98
BLS aggregatePubkeys 128 - blst-native 182.04 us/op 180.29 us/op 1.01
getAttestationsForBlock 78.562 ms/op 110.86 ms/op 0.71
isKnown best case - 1 super set check 439.00 ns/op 471.00 ns/op 0.93
isKnown normal case - 2 super set checks 431.00 ns/op 482.00 ns/op 0.89
isKnown worse case - 16 super set checks 430.00 ns/op 493.00 ns/op 0.87
CheckpointStateCache - add get delete 8.6130 us/op 10.738 us/op 0.80
validate gossip signedAggregateAndProof - struct 5.0299 ms/op 5.2279 ms/op 0.96
validate gossip attestation - struct 2.3667 ms/op 2.4174 ms/op 0.98
pickEth1Vote - no votes 2.1948 ms/op 2.5087 ms/op 0.87
pickEth1Vote - max votes 16.060 ms/op 19.637 ms/op 0.82
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.965 ms/op 15.271 ms/op 0.85
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.901 ms/op 24.661 ms/op 0.85
pickEth1Vote - Eth1Data fastSerialize value x2048 1.0852 ms/op 1.4398 ms/op 0.75
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.7052 ms/op 13.750 ms/op 0.71
bytes32 toHexString 863.00 ns/op 1.3170 us/op 0.66
bytes32 Buffer.toString(hex) 656.00 ns/op 705.00 ns/op 0.93
bytes32 Buffer.toString(hex) from Uint8Array 994.00 ns/op 1.0700 us/op 0.93
bytes32 Buffer.toString(hex) + 0x 676.00 ns/op 662.00 ns/op 1.02
Object access 1 prop 0.28700 ns/op 0.35100 ns/op 0.82
Map access 1 prop 0.26500 ns/op 0.30400 ns/op 0.87
Object get x1000 10.971 ns/op 16.080 ns/op 0.68
Map get x1000 0.94000 ns/op 0.90500 ns/op 1.04
Object set x1000 84.201 ns/op 104.03 ns/op 0.81
Map set x1000 51.257 ns/op 73.661 ns/op 0.70
Return object 10000 times 0.43690 ns/op 0.41140 ns/op 1.06
Throw Error 10000 times 6.7749 us/op 7.7431 us/op 0.87
fastMsgIdFn sha256 / 200 bytes 4.8510 us/op 5.1180 us/op 0.95
fastMsgIdFn h32 xxhash / 200 bytes 476.00 ns/op 646.00 ns/op 0.74
fastMsgIdFn h64 xxhash / 200 bytes 651.00 ns/op 776.00 ns/op 0.84
fastMsgIdFn sha256 / 1000 bytes 15.670 us/op 16.492 us/op 0.95
fastMsgIdFn h32 xxhash / 1000 bytes 650.00 ns/op 798.00 ns/op 0.81
fastMsgIdFn h64 xxhash / 1000 bytes 732.00 ns/op 823.00 ns/op 0.89
fastMsgIdFn sha256 / 10000 bytes 136.27 us/op 135.20 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 2.5350 us/op 2.6670 us/op 0.95
fastMsgIdFn h64 xxhash / 10000 bytes 1.7450 us/op 1.9300 us/op 0.90
enrSubnets - fastDeserialize 64 bits 2.5440 us/op 2.9500 us/op 0.86
enrSubnets - ssz BitVector 64 bits 844.00 ns/op 974.00 ns/op 0.87
enrSubnets - fastDeserialize 4 bits 304.00 ns/op 366.00 ns/op 0.83
enrSubnets - ssz BitVector 4 bits 840.00 ns/op 1.0230 us/op 0.82
prioritizePeers score -10:0 att 32-0.1 sync 2-0 81.866 us/op 109.29 us/op 0.75
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 112.87 us/op 145.26 us/op 0.78
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 210.93 us/op 285.44 us/op 0.74
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 321.20 us/op 448.56 us/op 0.72
prioritizePeers score 0:0 att 64-1 sync 4-1 411.89 us/op 586.91 us/op 0.70
array of 16000 items push then shift 51.639 us/op 5.1988 us/op 9.93
LinkedList of 16000 items push then shift 12.902 ns/op 20.557 ns/op 0.63
array of 16000 items push then pop 186.07 ns/op 262.02 ns/op 0.71
LinkedList of 16000 items push then pop 12.393 ns/op 18.891 ns/op 0.66
array of 24000 items push then shift 77.450 us/op 7.8854 us/op 9.82
LinkedList of 24000 items push then shift 12.890 ns/op 20.503 ns/op 0.63
array of 24000 items push then pop 192.69 ns/op 233.17 ns/op 0.83
LinkedList of 24000 items push then pop 12.178 ns/op 18.508 ns/op 0.66
intersect bitArray bitLen 8 21.676 ns/op 26.646 ns/op 0.81
intersect array and set length 8 123.93 ns/op 188.73 ns/op 0.66
intersect bitArray bitLen 128 71.131 ns/op 92.392 ns/op 0.77
intersect array and set length 128 1.6946 us/op 2.2028 us/op 0.77
Buffer.concat 32 items 4.5690 us/op 5.7610 us/op 0.79
Uint8Array.set 32 items 3.6920 us/op 5.0240 us/op 0.73
pass gossip attestations to forkchoice per slot 3.7757 ms/op 6.9083 ms/op 0.55
computeDeltas 5.0593 ms/op 5.3366 ms/op 0.95
computeProposerBoostScoreFromBalances 2.4872 ms/op 2.8035 ms/op 0.89
altair processAttestation - 250000 vs - 7PWei normalcase 3.5476 ms/op 5.1799 ms/op 0.68
altair processAttestation - 250000 vs - 7PWei worstcase 5.3117 ms/op 9.7875 ms/op 0.54
altair processAttestation - setStatus - 1/6 committees join 176.91 us/op 251.92 us/op 0.70
altair processAttestation - setStatus - 1/3 committees join 353.44 us/op 482.15 us/op 0.73
altair processAttestation - setStatus - 1/2 committees join 517.30 us/op 675.10 us/op 0.77
altair processAttestation - setStatus - 2/3 committees join 674.33 us/op 887.46 us/op 0.76
altair processAttestation - setStatus - 4/5 committees join 941.26 us/op 1.2740 ms/op 0.74
altair processAttestation - setStatus - 100% committees join 1.1316 ms/op 1.4943 ms/op 0.76
altair processBlock - 250000 vs - 7PWei normalcase 23.501 ms/op 32.034 ms/op 0.73
altair processBlock - 250000 vs - 7PWei normalcase hashState 31.435 ms/op 42.995 ms/op 0.73
altair processBlock - 250000 vs - 7PWei worstcase 81.209 ms/op 105.86 ms/op 0.77
altair processBlock - 250000 vs - 7PWei worstcase hashState 111.49 ms/op 117.97 ms/op 0.95
phase0 processBlock - 250000 vs - 7PWei normalcase 3.5611 ms/op 4.7747 ms/op 0.75
phase0 processBlock - 250000 vs - 7PWei worstcase 52.026 ms/op 54.082 ms/op 0.96
altair processEth1Data - 250000 vs - 7PWei normalcase 652.85 us/op 980.76 us/op 0.67
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 7.7860 us/op 22.345 us/op 0.35
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 22.534 us/op 50.059 us/op 0.45
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 10.255 us/op 28.881 us/op 0.36
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 7.8720 us/op 22.442 us/op 0.35
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 90.423 us/op 149.70 us/op 0.60
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 876.64 us/op 1.2886 ms/op 0.68
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.1694 ms/op 1.6297 ms/op 0.72
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.1303 ms/op 1.6378 ms/op 0.69
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 3.9982 ms/op 5.7397 ms/op 0.70
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 2.4607 ms/op 3.3923 ms/op 0.73
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 6.8761 ms/op 9.0685 ms/op 0.76
Tree 40 250000 create 606.11 ms/op 922.35 ms/op 0.66
Tree 40 250000 get(125000) 230.26 ns/op 300.36 ns/op 0.77
Tree 40 250000 set(125000) 1.9900 us/op 2.5853 us/op 0.77
Tree 40 250000 toArray() 26.446 ms/op 35.294 ms/op 0.75
Tree 40 250000 iterate all - toArray() + loop 27.121 ms/op 35.277 ms/op 0.77
Tree 40 250000 iterate all - get(i) 109.02 ms/op 124.06 ms/op 0.88
MutableVector 250000 create 13.858 ms/op 32.402 ms/op 0.43
MutableVector 250000 get(125000) 10.563 ns/op 13.587 ns/op 0.78
MutableVector 250000 set(125000) 740.13 ns/op 683.89 ns/op 1.08
MutableVector 250000 toArray() 6.7164 ms/op 6.8662 ms/op 0.98
MutableVector 250000 iterate all - toArray() + loop 6.0638 ms/op 7.3722 ms/op 0.82
MutableVector 250000 iterate all - get(i) 2.8320 ms/op 3.3033 ms/op 0.86
Array 250000 create 5.8607 ms/op 7.0460 ms/op 0.83
Array 250000 clone - spread 2.8405 ms/op 2.3063 ms/op 1.23
Array 250000 get(125000) 1.1460 ns/op 0.92200 ns/op 1.24
Array 250000 set(125000) 1.0560 ns/op 0.99400 ns/op 1.06
Array 250000 iterate all - loop 150.91 us/op 135.24 us/op 1.12
effectiveBalanceIncrements clone Uint8Array 300000 36.573 us/op 76.211 us/op 0.48
effectiveBalanceIncrements clone MutableVector 300000 546.00 ns/op 610.00 ns/op 0.90
effectiveBalanceIncrements rw all Uint8Array 300000 243.79 us/op 280.08 us/op 0.87
effectiveBalanceIncrements rw all MutableVector 300000 136.60 ms/op 206.29 ms/op 0.66
phase0 afterProcessEpoch - 250000 vs - 7PWei 187.62 ms/op 195.37 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.107 ms/op 85.767 ms/op 0.68
altair processEpoch - mainnet_e81889 529.95 ms/op 557.58 ms/op 0.95
mainnet_e81889 - altair beforeProcessEpoch 128.04 ms/op 157.07 ms/op 0.82
mainnet_e81889 - altair processJustificationAndFinalization 16.693 us/op 75.259 us/op 0.22
mainnet_e81889 - altair processInactivityUpdates 9.2152 ms/op 10.934 ms/op 0.84
mainnet_e81889 - altair processRewardsAndPenalties 76.972 ms/op 103.78 ms/op 0.74
mainnet_e81889 - altair processRegistryUpdates 2.1680 us/op 15.547 us/op 0.14
mainnet_e81889 - altair processSlashings 507.00 ns/op 3.1080 us/op 0.16
mainnet_e81889 - altair processEth1DataReset 561.00 ns/op 3.4660 us/op 0.16
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.2767 ms/op 2.3169 ms/op 0.98
mainnet_e81889 - altair processSlashingsReset 4.0210 us/op 24.760 us/op 0.16
mainnet_e81889 - altair processRandaoMixesReset 3.9620 us/op 23.782 us/op 0.17
mainnet_e81889 - altair processHistoricalRootsUpdate 646.00 ns/op 4.3030 us/op 0.15
mainnet_e81889 - altair processParticipationFlagUpdates 2.1110 us/op 14.952 us/op 0.14
mainnet_e81889 - altair processSyncCommitteeUpdates 532.00 ns/op 3.2890 us/op 0.16
mainnet_e81889 - altair afterProcessEpoch 197.07 ms/op 201.87 ms/op 0.98
phase0 processEpoch - mainnet_e58758 497.06 ms/op 626.49 ms/op 0.79
mainnet_e58758 - phase0 beforeProcessEpoch 205.36 ms/op 286.28 ms/op 0.72
mainnet_e58758 - phase0 processJustificationAndFinalization 16.055 us/op 66.813 us/op 0.24
mainnet_e58758 - phase0 processRewardsAndPenalties 65.734 ms/op 173.40 ms/op 0.38
mainnet_e58758 - phase0 processRegistryUpdates 7.6280 us/op 34.378 us/op 0.22
mainnet_e58758 - phase0 processSlashings 584.00 ns/op 3.0400 us/op 0.19
mainnet_e58758 - phase0 processEth1DataReset 553.00 ns/op 3.3550 us/op 0.16
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6946 ms/op 2.1563 ms/op 0.79
mainnet_e58758 - phase0 processSlashingsReset 3.3240 us/op 18.255 us/op 0.18
mainnet_e58758 - phase0 processRandaoMixesReset 4.0480 us/op 26.779 us/op 0.15
mainnet_e58758 - phase0 processHistoricalRootsUpdate 662.00 ns/op 3.9000 us/op 0.17
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.6530 us/op 24.730 us/op 0.15
mainnet_e58758 - phase0 afterProcessEpoch 163.28 ms/op 173.59 ms/op 0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.0206 ms/op 2.3272 ms/op 0.87
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.3033 ms/op 2.8223 ms/op 0.82
altair processInactivityUpdates - 250000 normalcase 33.135 ms/op 50.571 ms/op 0.66
altair processInactivityUpdates - 250000 worstcase 41.038 ms/op 60.010 ms/op 0.68
phase0 processRegistryUpdates - 250000 normalcase 6.2980 us/op 29.453 us/op 0.21
phase0 processRegistryUpdates - 250000 badcase_full_deposits 379.34 us/op 534.52 us/op 0.71
phase0 processRegistryUpdates - 250000 worstcase 0.5 192.36 ms/op 253.95 ms/op 0.76
altair processRewardsAndPenalties - 250000 normalcase 106.67 ms/op 143.31 ms/op 0.74
altair processRewardsAndPenalties - 250000 worstcase 82.480 ms/op 97.365 ms/op 0.85
phase0 getAttestationDeltas - 250000 normalcase 11.765 ms/op 13.128 ms/op 0.90
phase0 getAttestationDeltas - 250000 worstcase 11.801 ms/op 13.090 ms/op 0.90
phase0 processSlashings - 250000 worstcase 5.4412 ms/op 6.4061 ms/op 0.85
altair processSyncCommitteeUpdates - 250000 288.93 ms/op 316.55 ms/op 0.91
BeaconState.hashTreeRoot - No change 453.00 ns/op 511.00 ns/op 0.89
BeaconState.hashTreeRoot - 1 full validator 66.352 us/op 74.142 us/op 0.89
BeaconState.hashTreeRoot - 32 full validator 743.63 us/op 764.48 us/op 0.97
BeaconState.hashTreeRoot - 512 full validator 7.0916 ms/op 7.9267 ms/op 0.89
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 96.209 us/op 98.035 us/op 0.98
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2465 ms/op 1.4795 ms/op 0.84
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.634 ms/op 19.462 ms/op 0.85
BeaconState.hashTreeRoot - 1 balances 69.370 us/op 74.926 us/op 0.93
BeaconState.hashTreeRoot - 32 balances 616.42 us/op 667.59 us/op 0.92
BeaconState.hashTreeRoot - 512 balances 6.2383 ms/op 8.2701 ms/op 0.75
BeaconState.hashTreeRoot - 250000 balances 97.224 ms/op 108.77 ms/op 0.89
aggregationBits - 2048 els - zipIndexesInBitList 26.450 us/op 28.390 us/op 0.93
regular array get 100000 times 60.526 us/op 57.870 us/op 1.05
wrappedArray get 100000 times 60.539 us/op 56.107 us/op 1.08
arrayWithProxy get 100000 times 26.403 ms/op 32.264 ms/op 0.82
ssz.Root.equals 927.00 ns/op 1.1600 us/op 0.80
byteArrayEquals 934.00 ns/op 1.1520 us/op 0.81
shuffle list - 16384 els 11.455 ms/op 11.111 ms/op 1.03
shuffle list - 250000 els 168.27 ms/op 167.36 ms/op 1.01
processSlot - 1 slots 13.189 us/op 16.439 us/op 0.80
processSlot - 32 slots 1.9832 ms/op 2.2052 ms/op 0.90
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 322.87 us/op 387.08 us/op 0.83
getCommitteeAssignments - req 1 vs - 250000 vc 5.5447 ms/op 5.4791 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 8.0994 ms/op 7.3665 ms/op 1.10
getCommitteeAssignments - req 1000 vs - 250000 vc 8.6196 ms/op 8.1210 ms/op 1.06
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.5400 ns/op 9.3300 ns/op 0.81
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0976 us/op 1.4484 us/op 0.76
computeProposers - vc 250000 17.446 ms/op 19.223 ms/op 0.91
computeEpochShuffling - vc 250000 170.38 ms/op 178.08 ms/op 0.96
getNextSyncCommittee - vc 250000 291.43 ms/op 323.93 ms/op 0.90

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review December 22, 2022 01:35
@twoeths twoeths requested a review from a team as a code owner December 22, 2022 01:35
dapplion
dapplion reviewed Dec 22, 2022
View reviewed changes
packages/beacon-node/test/unit/network/attestationService.test.ts Outdated
@@ -156,10 +187,10 @@ describe.skip("AttnetsService", function () {
});

it.skip("handle committee subnet the same to random subnet", () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still skipped

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added back

dapplion
dapplion previously approved these changes Dec 22, 2022
View reviewed changes
@dapplion dapplion merged commit 3166e0e into unstable Dec 22, 2022
@dapplion dapplion deleted the tuyen/fix-random-subnet-subscription branch December 22, 2022 04:18
twoeths added a commit that referenced this pull request Dec 22, 2022
@twoeths
Fix random subnet subscription (#4930)
004406a 
* Add randBetweenFn option to AttnetsService for unit test

* Add shuffleFn option

* Add back AttestationService unit test

* Reproduce and fix #4929

* Add metrics to attnetsService syncnetsService

* Address PR comments

* Unskip the last unit test in attestationService.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

v1.3.0-rc.0: Not enough subnet mesh peers
2 participants
@twoeths @dapplion