Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(slashing-protection): resolve minEpoch max call stack issue #5461

Merged
merged 1 commit into from
May 5, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented May 4, 2023

Motivation

Closes #5392

Description

Epochs array might have more than ~10^5 elements, cannot use Math.min as it would throw RangeError: Maximum call stack size exceeded if epochs.length >= 125626.

This change makes the solution to determine min epoch more robust and adds few unit tests.

@nflaig nflaig requested a review from a team as a code owner May 4, 2023 11:13
@@ -28,7 +28,7 @@ export function numToString(num: number): string {
}

export function minEpoch(epochs: Epoch[]): Epoch | null {
return epochs.length > 0 ? Math.min(...epochs) : null;
return epochs.length > 0 ? epochs.reduce((minEpoch, epoch) => (minEpoch < epoch ? minEpoch : epoch)) : null;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternative implementation

export function minEpoch(epochs: Epoch[]): Epoch | null {
  if (epochs.length === 0) {
    return null;
  }

  let minEpoch = epochs[0];
  for (let i = 1; i < epochs.length; i++) {
    if (epochs[i] < minEpoch) {
      minEpoch = epochs[i];
    }
  }

  return minEpoch;
}

but there is no noticeable difference in performance

@github-actions
Copy link
Contributor

github-actions bot commented May 4, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 48d0a51 Previous: a24ada9 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0237 ms/op 812.82 us/op 1.26
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 61.041 us/op 51.210 us/op 1.19
BLS verify - blst-native 1.3613 ms/op 1.2408 ms/op 1.10
BLS verifyMultipleSignatures 3 - blst-native 2.5788 ms/op 2.5205 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 5.5688 ms/op 5.4257 ms/op 1.03
BLS verifyMultipleSignatures 32 - blst-native 20.086 ms/op 19.835 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 28.857 us/op 26.628 us/op 1.08
BLS aggregatePubkeys 128 - blst-native 109.26 us/op 103.52 us/op 1.06
getAttestationsForBlock 68.406 ms/op 55.745 ms/op 1.23
isKnown best case - 1 super set check 279.00 ns/op 247.00 ns/op 1.13
isKnown normal case - 2 super set checks 276.00 ns/op 249.00 ns/op 1.11
isKnown worse case - 16 super set checks 277.00 ns/op 254.00 ns/op 1.09
CheckpointStateCache - add get delete 5.9250 us/op 5.0170 us/op 1.18
validate gossip signedAggregateAndProof - struct 2.8907 ms/op 2.7501 ms/op 1.05
validate gossip attestation - struct 1.3883 ms/op 1.3123 ms/op 1.06
pickEth1Vote - no votes 1.4347 ms/op 1.2240 ms/op 1.17
pickEth1Vote - max votes 13.423 ms/op 9.4101 ms/op 1.43
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.296 ms/op 8.3577 ms/op 1.23
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.090 ms/op 15.388 ms/op 1.18
pickEth1Vote - Eth1Data fastSerialize value x2048 793.07 us/op 722.25 us/op 1.10
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.0375 ms/op 7.8064 ms/op 1.03
bytes32 toHexString 731.00 ns/op 499.00 ns/op 1.46
bytes32 Buffer.toString(hex) 434.00 ns/op 347.00 ns/op 1.25
bytes32 Buffer.toString(hex) from Uint8Array 644.00 ns/op 542.00 ns/op 1.19
bytes32 Buffer.toString(hex) + 0x 425.00 ns/op 351.00 ns/op 1.21
Object access 1 prop 0.20000 ns/op 0.15900 ns/op 1.26
Map access 1 prop 0.17000 ns/op 0.14400 ns/op 1.18
Object get x1000 7.1470 ns/op 6.3780 ns/op 1.12
Map get x1000 0.69800 ns/op 0.52500 ns/op 1.33
Object set x1000 66.776 ns/op 48.846 ns/op 1.37
Map set x1000 56.730 ns/op 41.205 ns/op 1.38
Return object 10000 times 0.25640 ns/op 0.22370 ns/op 1.15
Throw Error 10000 times 4.4647 us/op 4.0053 us/op 1.11
fastMsgIdFn sha256 / 200 bytes 3.6410 us/op 3.2400 us/op 1.12
fastMsgIdFn h32 xxhash / 200 bytes 319.00 ns/op 263.00 ns/op 1.21
fastMsgIdFn h64 xxhash / 200 bytes 476.00 ns/op 374.00 ns/op 1.27
fastMsgIdFn sha256 / 1000 bytes 12.049 us/op 11.022 us/op 1.09
fastMsgIdFn h32 xxhash / 1000 bytes 455.00 ns/op 387.00 ns/op 1.18
fastMsgIdFn h64 xxhash / 1000 bytes 546.00 ns/op 447.00 ns/op 1.22
fastMsgIdFn sha256 / 10000 bytes 107.33 us/op 98.111 us/op 1.09
fastMsgIdFn h32 xxhash / 10000 bytes 2.0490 us/op 1.8180 us/op 1.13
fastMsgIdFn h64 xxhash / 10000 bytes 1.4850 us/op 1.2890 us/op 1.15
enrSubnets - fastDeserialize 64 bits 1.5980 us/op 1.2050 us/op 1.33
enrSubnets - ssz BitVector 64 bits 617.00 ns/op 456.00 ns/op 1.35
enrSubnets - fastDeserialize 4 bits 211.00 ns/op 158.00 ns/op 1.34
enrSubnets - ssz BitVector 4 bits 626.00 ns/op 461.00 ns/op 1.36
prioritizePeers score -10:0 att 32-0.1 sync 2-0 119.11 us/op 100.23 us/op 1.19
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 155.81 us/op 125.45 us/op 1.24
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 191.44 us/op 160.63 us/op 1.19
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 370.43 us/op 280.82 us/op 1.32
prioritizePeers score 0:0 att 64-1 sync 4-1 435.44 us/op 338.26 us/op 1.29
array of 16000 items push then shift 1.7346 us/op 1.5360 us/op 1.13
LinkedList of 16000 items push then shift 9.3950 ns/op 8.3710 ns/op 1.12
array of 16000 items push then pop 116.27 ns/op 79.064 ns/op 1.47
LinkedList of 16000 items push then pop 9.2440 ns/op 8.4930 ns/op 1.09
array of 24000 items push then shift 2.4301 us/op 2.3361 us/op 1.04
LinkedList of 24000 items push then shift 9.6250 ns/op 8.8050 ns/op 1.09
array of 24000 items push then pop 89.781 ns/op 79.605 ns/op 1.13
LinkedList of 24000 items push then pop 9.3210 ns/op 8.0630 ns/op 1.16
intersect bitArray bitLen 8 14.030 ns/op 12.632 ns/op 1.11
intersect array and set length 8 106.01 ns/op 70.619 ns/op 1.50
intersect bitArray bitLen 128 45.704 ns/op 41.672 ns/op 1.10
intersect array and set length 128 1.2808 us/op 980.01 ns/op 1.31
Buffer.concat 32 items 3.1890 us/op 2.5240 us/op 1.26
Uint8Array.set 32 items 3.1660 us/op 2.8370 us/op 1.12
pass gossip attestations to forkchoice per slot 3.2119 ms/op 2.6728 ms/op 1.20
computeDeltas 3.0526 ms/op 2.8119 ms/op 1.09
computeProposerBoostScoreFromBalances 1.8155 ms/op 1.7826 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei normalcase 2.2414 ms/op 2.3633 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei worstcase 3.5419 ms/op 3.4205 ms/op 1.04
altair processAttestation - setStatus - 1/6 committees join 147.22 us/op 144.53 us/op 1.02
altair processAttestation - setStatus - 1/3 committees join 291.36 us/op 279.62 us/op 1.04
altair processAttestation - setStatus - 1/2 committees join 392.68 us/op 377.93 us/op 1.04
altair processAttestation - setStatus - 2/3 committees join 477.95 us/op 483.49 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 672.04 us/op 707.35 us/op 0.95
altair processAttestation - setStatus - 100% committees join 811.31 us/op 776.44 us/op 1.04
altair processBlock - 250000 vs - 7PWei normalcase 18.320 ms/op 19.004 ms/op 0.96
altair processBlock - 250000 vs - 7PWei normalcase hashState 29.264 ms/op 25.041 ms/op 1.17
altair processBlock - 250000 vs - 7PWei worstcase 48.460 ms/op 53.380 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase hashState 70.495 ms/op 65.727 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei normalcase 2.4453 ms/op 2.2050 ms/op 1.11
phase0 processBlock - 250000 vs - 7PWei worstcase 28.836 ms/op 31.327 ms/op 0.92
altair processEth1Data - 250000 vs - 7PWei normalcase 496.23 us/op 483.51 us/op 1.03
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 7.7710 us/op 8.9090 us/op 0.87
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 25.257 us/op 25.460 us/op 0.99
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.7910 us/op 13.017 us/op 0.68
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.7680 us/op 10.924 us/op 0.62
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 94.367 us/op 124.88 us/op 0.76
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 645.77 us/op 732.18 us/op 0.88
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 897.01 us/op 925.63 us/op 0.97
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 899.25 us/op 910.09 us/op 0.99
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.3195 ms/op 2.7347 ms/op 0.85
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.7346 ms/op 1.6738 ms/op 1.04
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.9911 ms/op 4.1563 ms/op 0.96
Tree 40 250000 create 310.80 ms/op 376.13 ms/op 0.83
Tree 40 250000 get(125000) 193.60 ns/op 201.52 ns/op 0.96
Tree 40 250000 set(125000) 1.0255 us/op 998.72 ns/op 1.03
Tree 40 250000 toArray() 20.406 ms/op 22.264 ms/op 0.92
Tree 40 250000 iterate all - toArray() + loop 19.743 ms/op 22.701 ms/op 0.87
Tree 40 250000 iterate all - get(i) 71.053 ms/op 77.749 ms/op 0.91
MutableVector 250000 create 10.755 ms/op 12.099 ms/op 0.89
MutableVector 250000 get(125000) 6.4780 ns/op 6.5880 ns/op 0.98
MutableVector 250000 set(125000) 266.22 ns/op 321.58 ns/op 0.83
MutableVector 250000 toArray() 2.9250 ms/op 4.1429 ms/op 0.71
MutableVector 250000 iterate all - toArray() + loop 3.3173 ms/op 4.3315 ms/op 0.77
MutableVector 250000 iterate all - get(i) 1.5212 ms/op 1.6103 ms/op 0.94
Array 250000 create 2.6983 ms/op 4.1108 ms/op 0.66
Array 250000 clone - spread 1.1483 ms/op 1.1696 ms/op 0.98
Array 250000 get(125000) 0.56800 ns/op 0.58300 ns/op 0.97
Array 250000 set(125000) 0.63700 ns/op 0.67500 ns/op 0.94
Array 250000 iterate all - loop 87.653 us/op 91.349 us/op 0.96
effectiveBalanceIncrements clone Uint8Array 300000 26.632 us/op 43.159 us/op 0.62
effectiveBalanceIncrements clone MutableVector 300000 337.00 ns/op 367.00 ns/op 0.92
effectiveBalanceIncrements rw all Uint8Array 300000 172.84 us/op 187.34 us/op 0.92
effectiveBalanceIncrements rw all MutableVector 300000 79.036 ms/op 88.947 ms/op 0.89
phase0 afterProcessEpoch - 250000 vs - 7PWei 115.31 ms/op 119.16 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 43.235 ms/op 44.517 ms/op 0.97
altair processEpoch - mainnet_e81889 319.76 ms/op 362.57 ms/op 0.88
mainnet_e81889 - altair beforeProcessEpoch 51.026 ms/op 51.475 ms/op 0.99
mainnet_e81889 - altair processJustificationAndFinalization 17.152 us/op 18.967 us/op 0.90
mainnet_e81889 - altair processInactivityUpdates 5.5021 ms/op 6.1837 ms/op 0.89
mainnet_e81889 - altair processRewardsAndPenalties 63.810 ms/op 69.058 ms/op 0.92
mainnet_e81889 - altair processRegistryUpdates 2.4470 us/op 2.6950 us/op 0.91
mainnet_e81889 - altair processSlashings 611.00 ns/op 491.00 ns/op 1.24
mainnet_e81889 - altair processEth1DataReset 599.00 ns/op 553.00 ns/op 1.08
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2411 ms/op 1.2691 ms/op 0.98
mainnet_e81889 - altair processSlashingsReset 4.7120 us/op 3.9880 us/op 1.18
mainnet_e81889 - altair processRandaoMixesReset 5.5700 us/op 5.6540 us/op 0.99
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1020 us/op 924.00 ns/op 1.19
mainnet_e81889 - altair processParticipationFlagUpdates 3.0770 us/op 2.9800 us/op 1.03
mainnet_e81889 - altair processSyncCommitteeUpdates 759.00 ns/op 619.00 ns/op 1.23
mainnet_e81889 - altair afterProcessEpoch 126.26 ms/op 128.62 ms/op 0.98
phase0 processEpoch - mainnet_e58758 365.23 ms/op 353.39 ms/op 1.03
mainnet_e58758 - phase0 beforeProcessEpoch 141.66 ms/op 131.68 ms/op 1.08
mainnet_e58758 - phase0 processJustificationAndFinalization 18.269 us/op 17.441 us/op 1.05
mainnet_e58758 - phase0 processRewardsAndPenalties 63.169 ms/op 64.524 ms/op 0.98
mainnet_e58758 - phase0 processRegistryUpdates 8.2210 us/op 8.1390 us/op 1.01
mainnet_e58758 - phase0 processSlashings 534.00 ns/op 525.00 ns/op 1.02
mainnet_e58758 - phase0 processEth1DataReset 642.00 ns/op 721.00 ns/op 0.89
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0185 ms/op 1.4524 ms/op 0.70
mainnet_e58758 - phase0 processSlashingsReset 4.3260 us/op 3.6180 us/op 1.20
mainnet_e58758 - phase0 processRandaoMixesReset 5.0340 us/op 4.5520 us/op 1.11
mainnet_e58758 - phase0 processHistoricalRootsUpdate 665.00 ns/op 719.00 ns/op 0.92
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.1940 us/op 4.0350 us/op 1.04
mainnet_e58758 - phase0 afterProcessEpoch 97.506 ms/op 97.697 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2875 ms/op 1.2280 ms/op 1.05
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6213 ms/op 1.5045 ms/op 1.08
altair processInactivityUpdates - 250000 normalcase 25.740 ms/op 25.949 ms/op 0.99
altair processInactivityUpdates - 250000 worstcase 28.262 ms/op 26.215 ms/op 1.08
phase0 processRegistryUpdates - 250000 normalcase 6.5550 us/op 6.7790 us/op 0.97
phase0 processRegistryUpdates - 250000 badcase_full_deposits 241.49 us/op 253.02 us/op 0.95
phase0 processRegistryUpdates - 250000 worstcase 0.5 110.76 ms/op 105.09 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 64.756 ms/op 69.839 ms/op 0.93
altair processRewardsAndPenalties - 250000 worstcase 69.195 ms/op 73.917 ms/op 0.94
phase0 getAttestationDeltas - 250000 normalcase 6.5098 ms/op 6.9788 ms/op 0.93
phase0 getAttestationDeltas - 250000 worstcase 6.5368 ms/op 6.9877 ms/op 0.94
phase0 processSlashings - 250000 worstcase 3.5368 ms/op 3.6666 ms/op 0.96
altair processSyncCommitteeUpdates - 250000 173.18 ms/op 185.09 ms/op 0.94
BeaconState.hashTreeRoot - No change 255.00 ns/op 268.00 ns/op 0.95
BeaconState.hashTreeRoot - 1 full validator 51.215 us/op 54.043 us/op 0.95
BeaconState.hashTreeRoot - 32 full validator 486.97 us/op 519.36 us/op 0.94
BeaconState.hashTreeRoot - 512 full validator 5.3315 ms/op 5.7377 ms/op 0.93
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 60.958 us/op 66.341 us/op 0.92
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 881.90 us/op 971.42 us/op 0.91
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 10.967 ms/op 12.058 ms/op 0.91
BeaconState.hashTreeRoot - 1 balances 45.842 us/op 51.537 us/op 0.89
BeaconState.hashTreeRoot - 32 balances 433.47 us/op 496.98 us/op 0.87
BeaconState.hashTreeRoot - 512 balances 4.2174 ms/op 4.6157 ms/op 0.91
BeaconState.hashTreeRoot - 250000 balances 73.327 ms/op 79.514 ms/op 0.92
aggregationBits - 2048 els - zipIndexesInBitList 15.244 us/op 16.964 us/op 0.90
regular array get 100000 times 32.401 us/op 33.888 us/op 0.96
wrappedArray get 100000 times 41.394 us/op 33.823 us/op 1.22
arrayWithProxy get 100000 times 16.935 ms/op 17.494 ms/op 0.97
ssz.Root.equals 532.00 ns/op 566.00 ns/op 0.94
byteArrayEquals 530.00 ns/op 556.00 ns/op 0.95
shuffle list - 16384 els 6.6874 ms/op 7.1460 ms/op 0.94
shuffle list - 250000 els 96.664 ms/op 105.77 ms/op 0.91
processSlot - 1 slots 8.5580 us/op 9.4340 us/op 0.91
processSlot - 32 slots 1.2929 ms/op 1.4169 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 38.030 ms/op 39.315 ms/op 0.97
getCommitteeAssignments - req 1 vs - 250000 vc 2.8565 ms/op 3.0276 ms/op 0.94
getCommitteeAssignments - req 100 vs - 250000 vc 4.1023 ms/op 4.3820 ms/op 0.94
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4298 ms/op 4.7117 ms/op 0.94
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3800 ns/op 5.2800 ns/op 0.83
state getBlockRootAtSlot - 250000 vs - 7PWei 980.76 ns/op 775.01 ns/op 1.27
computeProposers - vc 250000 10.097 ms/op 11.738 ms/op 0.86
computeEpochShuffling - vc 250000 98.566 ms/op 110.06 ms/op 0.90
getNextSyncCommittee - vc 250000 174.75 ms/op 189.04 ms/op 0.92
computeSigningRoot for AttestationData 13.173 us/op 13.706 us/op 0.96
hash AttestationData serialized data then Buffer.toString(base64) 2.3386 us/op 2.5687 us/op 0.91
toHexString serialized data 1.0247 us/op 1.2000 us/op 0.85
Buffer.toString(base64) 311.84 ns/op 348.38 ns/op 0.90

by benchmarkbot/action

@nflaig nflaig force-pushed the nflaig/fix-minEpoch-rangeError branch 3 times, most recently from b532921 to e693095 Compare May 4, 2023 12:54
Epochs array might have more than ~10^5 elements, cannot use `Math.min`
as it would throw `RangeError: Maximum call stack size exceeded`.
@nflaig nflaig force-pushed the nflaig/fix-minEpoch-rangeError branch from e693095 to cd6513e Compare May 4, 2023 21:15
@nflaig nflaig changed the title fix(slashing-protection): more robust solution to determine min epoch fix(slashing-protection): resolve minEpoch max call stack issue May 4, 2023
@nflaig nflaig merged commit 81b9998 into unstable May 5, 2023
@nflaig nflaig deleted the nflaig/fix-minEpoch-rangeError branch May 5, 2023 10:30
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot import big slashing protection
3 participants