Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce afterEach time in network e2e tests #5206

Merged
merged 2 commits into from
Feb 27, 2023
Merged

Reduce afterEach time in network e2e tests #5206

merged 2 commits into from
Feb 27, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Feb 27, 2023 •
edited
Loading

Motivation

  • Fix network e2e timeout issue

Description

  • Execute callbacks in parallel
  • Do not configure discv5

Closes #5180 #5138

@github-actions
Copy link
Contributor

github-actions bot commented Feb 27, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 8753058 Previous: c8d244f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 920.28 us/op 876.84 us/op 1.05
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 44.359 us/op 46.886 us/op 0.95
BLS verify - blst-native 1.1699 ms/op 1.1982 ms/op 0.98
BLS verifyMultipleSignatures 3 - blst-native 2.3772 ms/op 2.4410 ms/op 0.97
BLS verifyMultipleSignatures 8 - blst-native 5.0992 ms/op 5.2972 ms/op 0.96
BLS verifyMultipleSignatures 32 - blst-native 18.501 ms/op 19.168 ms/op 0.97
BLS aggregatePubkeys 32 - blst-native 24.569 us/op 25.821 us/op 0.95
BLS aggregatePubkeys 128 - blst-native 96.323 us/op 100.01 us/op 0.96
getAttestationsForBlock 53.777 ms/op 56.899 ms/op 0.95
isKnown best case - 1 super set check 265.00 ns/op 277.00 ns/op 0.96
isKnown normal case - 2 super set checks 266.00 ns/op 278.00 ns/op 0.96
isKnown worse case - 16 super set checks 258.00 ns/op 270.00 ns/op 0.96
CheckpointStateCache - add get delete 5.0940 us/op 5.2850 us/op 0.96
validate gossip signedAggregateAndProof - struct 2.7333 ms/op 2.8278 ms/op 0.97
validate gossip attestation - struct 1.3066 ms/op 1.3386 ms/op 0.98
pickEth1Vote - no votes 1.2248 ms/op 1.2736 ms/op 0.96
pickEth1Vote - max votes 10.344 ms/op 9.9037 ms/op 1.04
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.0598 ms/op 8.4759 ms/op 0.95
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.893 ms/op 14.520 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize value x2048 633.41 us/op 657.98 us/op 0.96
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.5827 ms/op 7.1517 ms/op 1.06
bytes32 toHexString 479.00 ns/op 510.00 ns/op 0.94
bytes32 Buffer.toString(hex) 336.00 ns/op 357.00 ns/op 0.94
bytes32 Buffer.toString(hex) from Uint8Array 554.00 ns/op 594.00 ns/op 0.93
bytes32 Buffer.toString(hex) + 0x 338.00 ns/op 382.00 ns/op 0.88
Object access 1 prop 0.17000 ns/op 0.17500 ns/op 0.97
Map access 1 prop 0.16200 ns/op 0.16000 ns/op 1.01
Object get x1000 6.5310 ns/op 6.6880 ns/op 0.98
Map get x1000 0.60600 ns/op 0.60700 ns/op 1.00
Object set x1000 50.305 ns/op 52.403 ns/op 0.96
Map set x1000 42.466 ns/op 44.013 ns/op 0.96
Return object 10000 times 0.23140 ns/op 0.23640 ns/op 0.98
Throw Error 10000 times 4.0289 us/op 4.1216 us/op 0.98
fastMsgIdFn sha256 / 200 bytes 3.3960 us/op 3.4520 us/op 0.98
fastMsgIdFn h32 xxhash / 200 bytes 274.00 ns/op 276.00 ns/op 0.99
fastMsgIdFn h64 xxhash / 200 bytes 376.00 ns/op 381.00 ns/op 0.99
fastMsgIdFn sha256 / 1000 bytes 11.462 us/op 11.730 us/op 0.98
fastMsgIdFn h32 xxhash / 1000 bytes 400.00 ns/op 415.00 ns/op 0.96
fastMsgIdFn h64 xxhash / 1000 bytes 456.00 ns/op 464.00 ns/op 0.98
fastMsgIdFn sha256 / 10000 bytes 102.06 us/op 105.00 us/op 0.97
fastMsgIdFn h32 xxhash / 10000 bytes 1.8820 us/op 1.8900 us/op 1.00
fastMsgIdFn h64 xxhash / 10000 bytes 1.3420 us/op 1.3420 us/op 1.00
enrSubnets - fastDeserialize 64 bits 1.2640 us/op 1.2910 us/op 0.98
enrSubnets - ssz BitVector 64 bits 480.00 ns/op 502.00 ns/op 0.96
enrSubnets - fastDeserialize 4 bits 164.00 ns/op 175.00 ns/op 0.94
enrSubnets - ssz BitVector 4 bits 478.00 ns/op 485.00 ns/op 0.99
prioritizePeers score -10:0 att 32-0.1 sync 2-0 91.514 us/op 94.143 us/op 0.97
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 119.53 us/op 134.27 us/op 0.89
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 164.52 us/op 164.06 us/op 1.00
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 285.64 us/op 297.94 us/op 0.96
prioritizePeers score 0:0 att 64-1 sync 4-1 342.26 us/op 361.63 us/op 0.95
array of 16000 items push then shift 1.5345 us/op 1.6254 us/op 0.94
LinkedList of 16000 items push then shift 8.3570 ns/op 8.7980 ns/op 0.95
array of 16000 items push then pop 76.202 ns/op 88.575 ns/op 0.86
LinkedList of 16000 items push then pop 8.4250 ns/op 8.6280 ns/op 0.98
array of 24000 items push then shift 2.3336 us/op 2.4231 us/op 0.96
LinkedList of 24000 items push then shift 8.6790 ns/op 9.0590 ns/op 0.96
array of 24000 items push then pop 73.552 ns/op 74.057 ns/op 0.99
LinkedList of 24000 items push then pop 8.3460 ns/op 8.6310 ns/op 0.97
intersect bitArray bitLen 8 13.100 ns/op 14.083 ns/op 0.93
intersect array and set length 8 75.535 ns/op 75.968 ns/op 0.99
intersect bitArray bitLen 128 43.400 ns/op 44.569 ns/op 0.97
intersect array and set length 128 1.0405 us/op 1.0463 us/op 0.99
Buffer.concat 32 items 2.7440 us/op 2.6430 us/op 1.04
Uint8Array.set 32 items 2.7830 us/op 2.0810 us/op 1.34
pass gossip attestations to forkchoice per slot 3.2693 ms/op 2.5736 ms/op 1.27
computeDeltas 2.8449 ms/op 2.9013 ms/op 0.98
computeProposerBoostScoreFromBalances 1.7732 ms/op 1.8114 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 2.0224 ms/op 2.1789 ms/op 0.93
altair processAttestation - 250000 vs - 7PWei worstcase 3.1980 ms/op 3.4673 ms/op 0.92
altair processAttestation - setStatus - 1/6 committees join 137.64 us/op 145.75 us/op 0.94
altair processAttestation - setStatus - 1/3 committees join 273.13 us/op 288.21 us/op 0.95
altair processAttestation - setStatus - 1/2 committees join 356.76 us/op 384.09 us/op 0.93
altair processAttestation - setStatus - 2/3 committees join 453.34 us/op 478.67 us/op 0.95
altair processAttestation - setStatus - 4/5 committees join 629.49 us/op 671.18 us/op 0.94
altair processAttestation - setStatus - 100% committees join 749.29 us/op 784.50 us/op 0.96
altair processBlock - 250000 vs - 7PWei normalcase 19.522 ms/op 15.649 ms/op 1.25
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.403 ms/op 26.076 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase 51.651 ms/op 48.587 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase hashState 65.919 ms/op 68.338 ms/op 0.96
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9946 ms/op 1.9373 ms/op 1.03
phase0 processBlock - 250000 vs - 7PWei worstcase 26.947 ms/op 28.828 ms/op 0.93
altair processEth1Data - 250000 vs - 7PWei normalcase 445.32 us/op 480.72 us/op 0.93
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8250 us/op 7.5650 us/op 0.90
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.558 us/op 22.912 us/op 0.85
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.8110 us/op 9.3990 us/op 0.94
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.2980 us/op 8.2830 us/op 0.76
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 73.933 us/op 83.696 us/op 0.88
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 587.32 us/op 650.86 us/op 0.90
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 876.41 us/op 899.53 us/op 0.97
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 842.23 us/op 881.68 us/op 0.96
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.1513 ms/op 2.3048 ms/op 0.93
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5108 ms/op 1.5011 ms/op 1.01
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.6332 ms/op 3.9256 ms/op 0.93
Tree 40 250000 create 309.62 ms/op 308.17 ms/op 1.00
Tree 40 250000 get(125000) 177.26 ns/op 181.37 ns/op 0.98
Tree 40 250000 set(125000) 951.30 ns/op 913.60 ns/op 1.04
Tree 40 250000 toArray() 17.171 ms/op 18.110 ms/op 0.95
Tree 40 250000 iterate all - toArray() + loop 15.835 ms/op 18.317 ms/op 0.86
Tree 40 250000 iterate all - get(i) 63.364 ms/op 69.629 ms/op 0.91
MutableVector 250000 create 9.4377 ms/op 10.969 ms/op 0.86
MutableVector 250000 get(125000) 6.2150 ns/op 6.3460 ns/op 0.98
MutableVector 250000 set(125000) 244.99 ns/op 261.32 ns/op 0.94
MutableVector 250000 toArray() 2.5916 ms/op 2.8113 ms/op 0.92
MutableVector 250000 iterate all - toArray() + loop 2.6866 ms/op 2.9070 ms/op 0.92
MutableVector 250000 iterate all - get(i) 1.4315 ms/op 1.5251 ms/op 0.94
Array 250000 create 2.4260 ms/op 2.5914 ms/op 0.94
Array 250000 clone - spread 1.1278 ms/op 1.2577 ms/op 0.90
Array 250000 get(125000) 0.53500 ns/op 0.60600 ns/op 0.88
Array 250000 set(125000) 0.62100 ns/op 0.69000 ns/op 0.90
Array 250000 iterate all - loop 77.220 us/op 83.277 us/op 0.93
effectiveBalanceIncrements clone Uint8Array 300000 24.456 us/op 31.263 us/op 0.78
effectiveBalanceIncrements clone MutableVector 300000 359.00 ns/op 404.00 ns/op 0.89
effectiveBalanceIncrements rw all Uint8Array 300000 158.86 us/op 167.54 us/op 0.95
effectiveBalanceIncrements rw all MutableVector 300000 77.356 ms/op 85.542 ms/op 0.90
phase0 afterProcessEpoch - 250000 vs - 7PWei 109.61 ms/op 115.54 ms/op 0.95
phase0 beforeProcessEpoch - 250000 vs - 7PWei 34.537 ms/op 33.432 ms/op 1.03
altair processEpoch - mainnet_e81889 322.28 ms/op 350.04 ms/op 0.92
mainnet_e81889 - altair beforeProcessEpoch 63.702 ms/op 66.914 ms/op 0.95
mainnet_e81889 - altair processJustificationAndFinalization 16.550 us/op 19.036 us/op 0.87
mainnet_e81889 - altair processInactivityUpdates 5.4331 ms/op 5.8206 ms/op 0.93
mainnet_e81889 - altair processRewardsAndPenalties 51.107 ms/op 64.117 ms/op 0.80
mainnet_e81889 - altair processRegistryUpdates 3.0280 us/op 3.1900 us/op 0.95
mainnet_e81889 - altair processSlashings 467.00 ns/op 412.00 ns/op 1.13
mainnet_e81889 - altair processEth1DataReset 547.00 ns/op 477.00 ns/op 1.15
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.1914 ms/op 1.2705 ms/op 0.94
mainnet_e81889 - altair processSlashingsReset 4.5660 us/op 4.6340 us/op 0.99
mainnet_e81889 - altair processRandaoMixesReset 4.5990 us/op 4.5640 us/op 1.01
mainnet_e81889 - altair processHistoricalRootsUpdate 628.00 ns/op 588.00 ns/op 1.07
mainnet_e81889 - altair processParticipationFlagUpdates 2.4650 us/op 2.5750 us/op 0.96
mainnet_e81889 - altair processSyncCommitteeUpdates 439.00 ns/op 434.00 ns/op 1.01
mainnet_e81889 - altair afterProcessEpoch 126.07 ms/op 121.81 ms/op 1.03
phase0 processEpoch - mainnet_e58758 336.51 ms/op 323.89 ms/op 1.04
mainnet_e58758 - phase0 beforeProcessEpoch 129.66 ms/op 123.59 ms/op 1.05
mainnet_e58758 - phase0 processJustificationAndFinalization 15.405 us/op 15.959 us/op 0.97
mainnet_e58758 - phase0 processRewardsAndPenalties 54.970 ms/op 57.634 ms/op 0.95
mainnet_e58758 - phase0 processRegistryUpdates 7.7790 us/op 9.3880 us/op 0.83
mainnet_e58758 - phase0 processSlashings 531.00 ns/op 441.00 ns/op 1.20
mainnet_e58758 - phase0 processEth1DataReset 493.00 ns/op 479.00 ns/op 1.03
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 939.01 us/op 1.0087 ms/op 0.93
mainnet_e58758 - phase0 processSlashingsReset 3.2590 us/op 2.7100 us/op 1.20
mainnet_e58758 - phase0 processRandaoMixesReset 4.3340 us/op 4.1790 us/op 1.04
mainnet_e58758 - phase0 processHistoricalRootsUpdate 569.00 ns/op 569.00 ns/op 1.00
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.1210 us/op 5.5070 us/op 0.75
mainnet_e58758 - phase0 afterProcessEpoch 92.098 ms/op 97.149 ms/op 0.95
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2080 ms/op 1.2604 ms/op 0.96
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.3810 ms/op 1.4762 ms/op 0.94
altair processInactivityUpdates - 250000 normalcase 22.387 ms/op 25.930 ms/op 0.86
altair processInactivityUpdates - 250000 worstcase 19.852 ms/op 27.376 ms/op 0.73
phase0 processRegistryUpdates - 250000 normalcase 6.5590 us/op 6.9230 us/op 0.95
phase0 processRegistryUpdates - 250000 badcase_full_deposits 240.69 us/op 268.60 us/op 0.90
phase0 processRegistryUpdates - 250000 worstcase 0.5 124.33 ms/op 125.80 ms/op 0.99
altair processRewardsAndPenalties - 250000 normalcase 48.067 ms/op 46.225 ms/op 1.04
altair processRewardsAndPenalties - 250000 worstcase 57.233 ms/op 68.457 ms/op 0.84
phase0 getAttestationDeltas - 250000 normalcase 6.3334 ms/op 6.6880 ms/op 0.95
phase0 getAttestationDeltas - 250000 worstcase 6.4148 ms/op 6.5208 ms/op 0.98
phase0 processSlashings - 250000 worstcase 3.4294 ms/op 3.2543 ms/op 1.05
altair processSyncCommitteeUpdates - 250000 164.19 ms/op 176.78 ms/op 0.93
BeaconState.hashTreeRoot - No change 245.00 ns/op 276.00 ns/op 0.89
BeaconState.hashTreeRoot - 1 full validator 49.399 us/op 52.990 us/op 0.93
BeaconState.hashTreeRoot - 32 full validator 468.99 us/op 533.94 us/op 0.88
BeaconState.hashTreeRoot - 512 full validator 4.8377 ms/op 5.0599 ms/op 0.96
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 60.041 us/op 62.864 us/op 0.96
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 850.42 us/op 877.72 us/op 0.97
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.139 ms/op 11.110 ms/op 1.00
BeaconState.hashTreeRoot - 1 balances 48.787 us/op 48.442 us/op 1.01
BeaconState.hashTreeRoot - 32 balances 448.39 us/op 464.89 us/op 0.96
BeaconState.hashTreeRoot - 512 balances 4.2236 ms/op 4.4293 ms/op 0.95
BeaconState.hashTreeRoot - 250000 balances 72.923 ms/op 76.659 ms/op 0.95
aggregationBits - 2048 els - zipIndexesInBitList 15.714 us/op 14.807 us/op 1.06
regular array get 100000 times 31.853 us/op 40.156 us/op 0.79
wrappedArray get 100000 times 40.405 us/op 42.398 us/op 0.95
arrayWithProxy get 100000 times 15.001 ms/op 15.105 ms/op 0.99
ssz.Root.equals 534.00 ns/op 527.00 ns/op 1.01
byteArrayEquals 521.00 ns/op 522.00 ns/op 1.00
shuffle list - 16384 els 6.6272 ms/op 6.7669 ms/op 0.98
shuffle list - 250000 els 97.262 ms/op 99.348 ms/op 0.98
processSlot - 1 slots 8.8400 us/op 8.1600 us/op 1.08
processSlot - 32 slots 1.3311 ms/op 1.3452 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 191.97 us/op 196.13 us/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 2.8930 ms/op 2.8609 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 4.1383 ms/op 4.0937 ms/op 1.01
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4348 ms/op 4.4117 ms/op 1.01
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4800 ns/op 4.6600 ns/op 0.96
state getBlockRootAtSlot - 250000 vs - 7PWei 880.35 ns/op 953.02 ns/op 0.92
computeProposers - vc 250000 10.217 ms/op 10.391 ms/op 0.98
computeEpochShuffling - vc 250000 100.85 ms/op 102.42 ms/op 0.98
getNextSyncCommittee - vc 250000 176.86 ms/op 171.78 ms/op 1.03

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review February 27, 2023 06:57
@twoeths twoeths requested a review from a team as a code owner February 27, 2023 06:57
@dapplion dapplion merged commit fc33339 into unstable Feb 27, 2023
@dapplion dapplion deleted the tuyen/fix_network_e2e_timeout branch February 27, 2023 10:49
@twoeths twoeths mentioned this pull request Mar 6, 2023
Closed
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

@twoeths twoeths mentioned this pull request Apr 27, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Failed e2e network tests in CI
3 participants
@twoeths @wemeetagain @dapplion