Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add stateClonedCount metric #4128

Merged
merged 2 commits into from
Jul 11, 2022
Merged

Add stateClonedCount metric #4128

merged 2 commits into from
Jul 11, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Jun 7, 2022 •
edited
Loading

Motivation

Description

Add stateClonedCount metric

Closes #3680

TODO

  • Pick bucket sizes that make sense

@dapplion dapplion requested a review from a team as a code owner June 7, 2022 19:33
@dapplion dapplion marked this pull request as draft June 7, 2022 19:33
@github-actions
Copy link
Contributor

github-actions bot commented Jun 7, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 5c30fbd Previous: 7bf6764 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0753 ms/op 1.9863 ms/op 1.04
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 83.676 us/op 70.611 us/op 1.19
BLS verify - blst-native 2.1748 ms/op 1.8575 ms/op 1.17
BLS verifyMultipleSignatures 3 - blst-native 4.4976 ms/op 3.8009 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 9.5094 ms/op 8.1791 ms/op 1.16
BLS verifyMultipleSignatures 32 - blst-native 34.499 ms/op 29.678 ms/op 1.16
BLS aggregatePubkeys 32 - blst-native 46.166 us/op 39.195 us/op 1.18
BLS aggregatePubkeys 128 - blst-native 177.28 us/op 153.41 us/op 1.16
getAttestationsForBlock 50.224 ms/op 42.545 ms/op 1.18
isKnown best case - 1 super set check 494.00 ns/op 427.00 ns/op 1.16
isKnown normal case - 2 super set checks 476.00 ns/op 412.00 ns/op 1.16
isKnown worse case - 16 super set checks 477.00 ns/op 413.00 ns/op 1.15
CheckpointStateCache - add get delete 11.663 us/op 9.1990 us/op 1.27
validate gossip signedAggregateAndProof - struct 4.9980 ms/op 3.7726 ms/op 1.32
validate gossip attestation - struct 2.3648 ms/op 1.7934 ms/op 1.32
altair verifyImport mainnet_s3766816:31 7.3736 s/op 5.6788 s/op 1.30
pickEth1Vote - no votes 2.3854 ms/op 1.8527 ms/op 1.29
pickEth1Vote - max votes 25.595 ms/op 24.042 ms/op 1.06
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.161 ms/op 10.911 ms/op 1.21
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.776 ms/op 20.779 ms/op 1.14
pickEth1Vote - Eth1Data fastSerialize value x2048 1.7330 ms/op 1.3506 ms/op 1.28
pickEth1Vote - Eth1Data fastSerialize tree x2048 19.475 ms/op 12.204 ms/op 1.60
bytes32 toHexString 1.1860 us/op 914.00 ns/op 1.30
bytes32 Buffer.toString(hex) 780.00 ns/op 589.00 ns/op 1.32
bytes32 Buffer.toString(hex) from Uint8Array 1.0430 us/op 784.00 ns/op 1.33
bytes32 Buffer.toString(hex) + 0x 800.00 ns/op 584.00 ns/op 1.37
Object access 1 prop 0.39900 ns/op 0.29400 ns/op 1.36
Map access 1 prop 0.34900 ns/op 0.25200 ns/op 1.38
Object get x1000 21.011 ns/op 15.700 ns/op 1.34
Map get x1000 1.3000 ns/op 0.85800 ns/op 1.52
Object set x1000 126.51 ns/op 102.32 ns/op 1.24
Map set x1000 78.245 ns/op 59.710 ns/op 1.31
Return object 10000 times 0.44140 ns/op 0.33110 ns/op 1.33
Throw Error 10000 times 6.9166 us/op 5.2971 us/op 1.31
enrSubnets - fastDeserialize 64 bits 3.0640 us/op 2.2050 us/op 1.39
enrSubnets - ssz BitVector 64 bits 878.00 ns/op 633.00 ns/op 1.39
enrSubnets - fastDeserialize 4 bits 446.00 ns/op 326.00 ns/op 1.37
enrSubnets - ssz BitVector 4 bits 821.00 ns/op 657.00 ns/op 1.25
prioritizePeers score -10:0 att 32-0.1 sync 2-0 104.78 us/op 79.227 us/op 1.32
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 142.26 us/op 117.52 us/op 1.21
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 233.74 us/op 185.90 us/op 1.26
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 496.66 us/op 344.71 us/op 1.44
prioritizePeers score 0:0 att 64-1 sync 4-1 536.67 us/op 407.21 us/op 1.32
RateTracker 1000000 limit, 1 obj count per request 214.27 ns/op 159.94 ns/op 1.34
RateTracker 1000000 limit, 2 obj count per request 161.25 ns/op 120.47 ns/op 1.34
RateTracker 1000000 limit, 4 obj count per request 130.06 ns/op 98.378 ns/op 1.32
RateTracker 1000000 limit, 8 obj count per request 118.22 ns/op 88.870 ns/op 1.33
RateTracker with prune 4.7620 us/op 3.5790 us/op 1.33
array of 16000 items push then shift 3.7192 us/op 2.7925 us/op 1.33
LinkedList of 16000 items push then shift 26.639 ns/op 22.940 ns/op 1.16
array of 16000 items push then pop 263.30 ns/op 207.19 ns/op 1.27
LinkedList of 16000 items push then pop 23.898 ns/op 19.615 ns/op 1.22
array of 24000 items push then shift 5.3886 us/op 4.0407 us/op 1.33
LinkedList of 24000 items push then shift 27.821 ns/op 24.155 ns/op 1.15
array of 24000 items push then pop 214.59 ns/op 174.41 ns/op 1.23
LinkedList of 24000 items push then pop 24.103 ns/op 19.820 ns/op 1.22
intersect bitArray bitLen 8 13.914 ns/op 10.116 ns/op 1.38
intersect array and set length 8 181.25 ns/op 137.06 ns/op 1.32
intersect bitArray bitLen 128 72.393 ns/op 63.617 ns/op 1.14
intersect array and set length 128 2.5312 us/op 1.9581 us/op 1.29
pass gossip attestations to forkchoice per slot 4.3241 ms/op 3.1017 ms/op 1.39
computeDeltas 3.4465 ms/op 3.7253 ms/op 0.93
computeProposerBoostScoreFromBalances 1.0794 ms/op 813.07 us/op 1.33
altair processAttestation - 250000 vs - 7PWei normalcase 4.3014 ms/op 4.0438 ms/op 1.06
altair processAttestation - 250000 vs - 7PWei worstcase 6.3891 ms/op 5.6920 ms/op 1.12
altair processAttestation - setStatus - 1/6 committees join 234.33 us/op 189.29 us/op 1.24
altair processAttestation - setStatus - 1/3 committees join 464.42 us/op 359.07 us/op 1.29
altair processAttestation - setStatus - 1/2 committees join 645.46 us/op 498.37 us/op 1.30
altair processAttestation - setStatus - 2/3 committees join 833.49 us/op 639.34 us/op 1.30
altair processAttestation - setStatus - 4/5 committees join 1.1652 ms/op 883.95 us/op 1.32
altair processAttestation - setStatus - 100% committees join 1.3314 ms/op 1.0519 ms/op 1.27
altair processBlock - 250000 vs - 7PWei normalcase 29.818 ms/op 25.802 ms/op 1.16
altair processBlock - 250000 vs - 7PWei normalcase hashState 40.992 ms/op 40.855 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 80.435 ms/op 70.887 ms/op 1.13
altair processBlock - 250000 vs - 7PWei worstcase hashState 111.48 ms/op 91.267 ms/op 1.22
phase0 processBlock - 250000 vs - 7PWei normalcase 3.8367 ms/op 3.2410 ms/op 1.18
phase0 processBlock - 250000 vs - 7PWei worstcase 52.976 ms/op 41.516 ms/op 1.28
altair processEth1Data - 250000 vs - 7PWei normalcase 916.61 us/op 679.90 us/op 1.35
Tree 40 250000 create 822.73 ms/op 743.85 ms/op 1.11
Tree 40 250000 get(125000) 332.48 ns/op 289.10 ns/op 1.15
Tree 40 250000 set(125000) 2.6419 us/op 2.2918 us/op 1.15
Tree 40 250000 toArray() 35.409 ms/op 31.506 ms/op 1.12
Tree 40 250000 iterate all - toArray() + loop 35.484 ms/op 31.563 ms/op 1.12
Tree 40 250000 iterate all - get(i) 142.24 ms/op 99.579 ms/op 1.43
MutableVector 250000 create 19.269 ms/op 14.047 ms/op 1.37
MutableVector 250000 get(125000) 17.324 ns/op 11.584 ns/op 1.50
MutableVector 250000 set(125000) 651.76 ns/op 580.49 ns/op 1.12
MutableVector 250000 toArray() 7.3472 ms/op 6.8233 ms/op 1.08
MutableVector 250000 iterate all - toArray() + loop 7.7078 ms/op 6.8573 ms/op 1.12
MutableVector 250000 iterate all - get(i) 4.0343 ms/op 2.9015 ms/op 1.39
Array 250000 create 6.4845 ms/op 6.4732 ms/op 1.00
Array 250000 clone - spread 3.8050 ms/op 3.3200 ms/op 1.15
Array 250000 get(125000) 1.7300 ns/op 1.3880 ns/op 1.25
Array 250000 set(125000) 1.6980 ns/op 1.4300 ns/op 1.19
Array 250000 iterate all - loop 197.45 us/op 148.09 us/op 1.33
effectiveBalanceIncrements clone Uint8Array 300000 81.424 us/op 90.189 us/op 0.90
effectiveBalanceIncrements clone MutableVector 300000 1.2830 us/op 594.00 ns/op 2.16
effectiveBalanceIncrements rw all Uint8Array 300000 295.79 us/op 223.01 us/op 1.33
effectiveBalanceIncrements rw all MutableVector 300000 217.77 ms/op 150.96 ms/op 1.44
phase0 afterProcessEpoch - 250000 vs - 7PWei 214.00 ms/op 170.08 ms/op 1.26
phase0 beforeProcessEpoch - 250000 vs - 7PWei 102.22 ms/op 67.467 ms/op 1.52
altair processEpoch - mainnet_e81889 654.71 ms/op 561.52 ms/op 1.17
mainnet_e81889 - altair beforeProcessEpoch 185.21 ms/op 127.80 ms/op 1.45
mainnet_e81889 - altair processJustificationAndFinalization 42.924 us/op 21.143 us/op 2.03
mainnet_e81889 - altair processInactivityUpdates 11.697 ms/op 11.118 ms/op 1.05
mainnet_e81889 - altair processRewardsAndPenalties 103.66 ms/op 89.157 ms/op 1.16
mainnet_e81889 - altair processRegistryUpdates 8.6900 us/op 3.5970 us/op 2.42
mainnet_e81889 - altair processSlashings 2.3080 us/op 798.00 ns/op 2.89
mainnet_e81889 - altair processEth1DataReset 2.4540 us/op 818.00 ns/op 3.00
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7583 ms/op 2.4118 ms/op 1.14
mainnet_e81889 - altair processSlashingsReset 16.735 us/op 6.2330 us/op 2.68
mainnet_e81889 - altair processRandaoMixesReset 17.356 us/op 6.0530 us/op 2.87
mainnet_e81889 - altair processHistoricalRootsUpdate 2.6500 us/op 863.00 ns/op 3.07
mainnet_e81889 - altair processParticipationFlagUpdates 7.2560 us/op 4.6350 us/op 1.57
mainnet_e81889 - altair processSyncCommitteeUpdates 2.2380 us/op 2.2020 us/op 1.02
mainnet_e81889 - altair afterProcessEpoch 224.44 ms/op 194.05 ms/op 1.16
phase0 processEpoch - mainnet_e58758 596.29 ms/op 504.79 ms/op 1.18
mainnet_e58758 - phase0 beforeProcessEpoch 277.96 ms/op 245.21 ms/op 1.13
mainnet_e58758 - phase0 processJustificationAndFinalization 35.424 us/op 29.352 us/op 1.21
mainnet_e58758 - phase0 processRewardsAndPenalties 89.338 ms/op 74.686 ms/op 1.20
mainnet_e58758 - phase0 processRegistryUpdates 22.163 us/op 10.696 us/op 2.07
mainnet_e58758 - phase0 processSlashings 2.3670 us/op 887.00 ns/op 2.67
mainnet_e58758 - phase0 processEth1DataReset 2.3450 us/op 1.1100 us/op 2.11
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.4181 ms/op 1.9893 ms/op 1.22
mainnet_e58758 - phase0 processSlashingsReset 9.7100 us/op 6.3640 us/op 1.53
mainnet_e58758 - phase0 processRandaoMixesReset 16.365 us/op 6.6090 us/op 2.48
mainnet_e58758 - phase0 processHistoricalRootsUpdate 2.5590 us/op 1.1320 us/op 2.26
mainnet_e58758 - phase0 processParticipationRecordUpdates 11.334 us/op 4.9010 us/op 2.31
mainnet_e58758 - phase0 afterProcessEpoch 189.12 ms/op 148.88 ms/op 1.27
phase0 processEffectiveBalanceUpdates - 250000 normalcase 3.4652 ms/op 2.4912 ms/op 1.39
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.0937 ms/op 2.2286 ms/op 1.39
altair processInactivityUpdates - 250000 normalcase 44.591 ms/op 42.157 ms/op 1.06
altair processInactivityUpdates - 250000 worstcase 45.734 ms/op 37.213 ms/op 1.23
phase0 processRegistryUpdates - 250000 normalcase 15.432 us/op 12.167 us/op 1.27
phase0 processRegistryUpdates - 250000 badcase_full_deposits 487.06 us/op 374.61 us/op 1.30
phase0 processRegistryUpdates - 250000 worstcase 0.5 268.78 ms/op 191.97 ms/op 1.40
altair processRewardsAndPenalties - 250000 normalcase 142.95 ms/op 111.18 ms/op 1.29
altair processRewardsAndPenalties - 250000 worstcase 100.37 ms/op 76.171 ms/op 1.32
phase0 getAttestationDeltas - 250000 normalcase 14.875 ms/op 11.693 ms/op 1.27
phase0 getAttestationDeltas - 250000 worstcase 15.214 ms/op 11.766 ms/op 1.29
phase0 processSlashings - 250000 worstcase 6.6965 ms/op 4.7806 ms/op 1.40
altair processSyncCommitteeUpdates - 250000 323.04 ms/op 260.08 ms/op 1.24
BeaconState.hashTreeRoot - No change 593.00 ns/op 401.00 ns/op 1.48
BeaconState.hashTreeRoot - 1 full validator 67.329 us/op 48.857 us/op 1.38
BeaconState.hashTreeRoot - 32 full validator 648.02 us/op 478.13 us/op 1.36
BeaconState.hashTreeRoot - 512 full validator 7.4022 ms/op 5.5478 ms/op 1.33
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 81.188 us/op 60.683 us/op 1.34
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2000 ms/op 908.32 us/op 1.32
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.384 ms/op 12.735 ms/op 1.29
BeaconState.hashTreeRoot - 1 balances 64.507 us/op 47.085 us/op 1.37
BeaconState.hashTreeRoot - 32 balances 575.19 us/op 433.46 us/op 1.33
BeaconState.hashTreeRoot - 512 balances 5.3748 ms/op 4.0167 ms/op 1.34
BeaconState.hashTreeRoot - 250000 balances 103.31 ms/op 84.566 ms/op 1.22
aggregationBits - 2048 els - zipIndexesInBitList 33.121 us/op 26.966 us/op 1.23
regular array get 100000 times 80.899 us/op 59.404 us/op 1.36
wrappedArray get 100000 times 80.917 us/op 59.549 us/op 1.36
arrayWithProxy get 100000 times 35.678 ms/op 27.397 ms/op 1.30
ssz.Root.equals 579.00 ns/op 428.00 ns/op 1.35
byteArrayEquals 589.00 ns/op 417.00 ns/op 1.41
shuffle list - 16384 els 13.231 ms/op 10.350 ms/op 1.28
shuffle list - 250000 els 194.42 ms/op 152.03 ms/op 1.28
processSlot - 1 slots 14.357 us/op 10.163 us/op 1.41
processSlot - 32 slots 1.9903 ms/op 1.4964 ms/op 1.33
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 445.75 us/op 356.90 us/op 1.25
getCommitteeAssignments - req 1 vs - 250000 vc 6.3137 ms/op 4.7192 ms/op 1.34
getCommitteeAssignments - req 100 vs - 250000 vc 8.7476 ms/op 6.4794 ms/op 1.35
getCommitteeAssignments - req 1000 vs - 250000 vc 9.5107 ms/op 6.8758 ms/op 1.38
computeProposers - vc 250000 19.163 ms/op 15.177 ms/op 1.26
computeEpochShuffling - vc 250000 218.40 ms/op 155.09 ms/op 1.41
getNextSyncCommittee - vc 250000 312.37 ms/op 250.47 ms/op 1.25

by benchmarkbot/action

@wemeetagain
Copy link
Member

How do we ensure that this data capture makes sense?

@dapplion
Copy link
Contributor Author

How do we ensure that this data capture makes sense?

Deploy to a node and check the captured data

@dapplion dapplion marked this pull request as ready for review July 11, 2022 08:44
@dapplion dapplion force-pushed the dapplion/metric-state-clone branch from 863a0a6 to 9e96a9c Compare July 11, 2022 08:49
@wemeetagain wemeetagain merged commit 2b16141 into unstable Jul 11, 2022
@wemeetagain wemeetagain deleted the dapplion/metric-state-clone branch July 11, 2022 14:44
@dapplion dapplion mentioned this pull request Jul 11, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add state cloneCount metric
2 participants
@dapplion @wemeetagain