Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add proposerBalanceDiff metric #4124

Merged
merged 2 commits into from
Jul 11, 2022
Merged

Add proposerBalanceDiff metric #4124

merged 2 commits into from
Jul 11, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Jun 7, 2022 •
edited
Loading

Motivation

Description

Track proposer balance diff for known and all validators

  • Should run first to inform what are sane values for the buckets

TODO

  • Pick bucket sizes that make sense

@dapplion dapplion force-pushed the dapplion/metric-proposer-balance-diff branch from 5ea6c39 to d5c9015 Compare June 7, 2022 19:21
@dapplion dapplion mentioned this pull request Jun 7, 2022
Open
7 tasks
@wemeetagain
Copy link
Member

has this been run? do we know good bucket values now?

@dapplion dapplion force-pushed the dapplion/metric-proposer-balance-diff branch from d5c9015 to b0941dd Compare July 11, 2022 08:37
@dapplion dapplion marked this pull request as ready for review July 11, 2022 08:46
@dapplion dapplion requested a review from a team as a code owner July 11, 2022 08:46
@github-actions
Copy link
Contributor

github-actions bot commented Jul 11, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4d3fa8f Previous: 7bf6764 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.7678 ms/op 1.9863 ms/op 0.89
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 62.767 us/op 70.611 us/op 0.89
BLS verify - blst-native 2.1646 ms/op 1.8575 ms/op 1.17
BLS verifyMultipleSignatures 3 - blst-native 4.4759 ms/op 3.8009 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 9.6737 ms/op 8.1791 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 35.155 ms/op 29.678 ms/op 1.18
BLS aggregatePubkeys 32 - blst-native 47.102 us/op 39.195 us/op 1.20
BLS aggregatePubkeys 128 - blst-native 184.19 us/op 153.41 us/op 1.20
getAttestationsForBlock 42.259 ms/op 42.545 ms/op 0.99
isKnown best case - 1 super set check 491.00 ns/op 427.00 ns/op 1.15
isKnown normal case - 2 super set checks 489.00 ns/op 412.00 ns/op 1.19
isKnown worse case - 16 super set checks 484.00 ns/op 413.00 ns/op 1.17
CheckpointStateCache - add get delete 10.952 us/op 9.1990 us/op 1.19
validate gossip signedAggregateAndProof - struct 5.0217 ms/op 3.7726 ms/op 1.33
validate gossip attestation - struct 2.3800 ms/op 1.7934 ms/op 1.33
altair verifyImport mainnet_s3766816:31 6.9845 s/op 5.6788 s/op 1.23
pickEth1Vote - no votes 2.1112 ms/op 1.8527 ms/op 1.14
pickEth1Vote - max votes 23.711 ms/op 24.042 ms/op 0.99
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.336 ms/op 10.911 ms/op 1.22
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.435 ms/op 20.779 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4349 ms/op 1.3506 ms/op 1.06
pickEth1Vote - Eth1Data fastSerialize tree x2048 17.512 ms/op 12.204 ms/op 1.43
bytes32 toHexString 963.00 ns/op 914.00 ns/op 1.05
bytes32 Buffer.toString(hex) 769.00 ns/op 589.00 ns/op 1.31
bytes32 Buffer.toString(hex) from Uint8Array 1.0080 us/op 784.00 ns/op 1.29
bytes32 Buffer.toString(hex) + 0x 745.00 ns/op 584.00 ns/op 1.28
Object access 1 prop 0.35800 ns/op 0.29400 ns/op 1.22
Map access 1 prop 0.30200 ns/op 0.25200 ns/op 1.20
Object get x1000 11.672 ns/op 15.700 ns/op 0.74
Map get x1000 0.97800 ns/op 0.85800 ns/op 1.14
Object set x1000 71.141 ns/op 102.32 ns/op 0.70
Map set x1000 46.353 ns/op 59.710 ns/op 0.78
Return object 10000 times 0.44450 ns/op 0.33110 ns/op 1.34
Throw Error 10000 times 5.9351 us/op 5.2971 us/op 1.12
enrSubnets - fastDeserialize 64 bits 2.7100 us/op 2.2050 us/op 1.23
enrSubnets - ssz BitVector 64 bits 866.00 ns/op 633.00 ns/op 1.37
enrSubnets - fastDeserialize 4 bits 427.00 ns/op 326.00 ns/op 1.31
enrSubnets - ssz BitVector 4 bits 826.00 ns/op 657.00 ns/op 1.26
prioritizePeers score -10:0 att 32-0.1 sync 2-0 83.525 us/op 79.227 us/op 1.05
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 124.99 us/op 117.52 us/op 1.06
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 212.54 us/op 185.90 us/op 1.14
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 340.85 us/op 344.71 us/op 0.99
prioritizePeers score 0:0 att 64-1 sync 4-1 404.46 us/op 407.21 us/op 0.99
RateTracker 1000000 limit, 1 obj count per request 178.71 ns/op 159.94 ns/op 1.12
RateTracker 1000000 limit, 2 obj count per request 132.19 ns/op 120.47 ns/op 1.10
RateTracker 1000000 limit, 4 obj count per request 107.45 ns/op 98.378 ns/op 1.09
RateTracker 1000000 limit, 8 obj count per request 95.800 ns/op 88.870 ns/op 1.08
RateTracker with prune 3.9440 us/op 3.5790 us/op 1.10
array of 16000 items push then shift 51.583 us/op 2.7925 us/op 18.47
LinkedList of 16000 items push then shift 15.296 ns/op 22.940 ns/op 0.67
array of 16000 items push then pop 213.27 ns/op 207.19 ns/op 1.03
LinkedList of 16000 items push then pop 14.043 ns/op 19.615 ns/op 0.72
array of 24000 items push then shift 77.339 us/op 4.0407 us/op 19.14
LinkedList of 24000 items push then shift 17.644 ns/op 24.155 ns/op 0.73
array of 24000 items push then pop 189.62 ns/op 174.41 ns/op 1.09
LinkedList of 24000 items push then pop 14.940 ns/op 19.820 ns/op 0.75
intersect bitArray bitLen 8 11.167 ns/op 10.116 ns/op 1.10
intersect array and set length 8 136.98 ns/op 137.06 ns/op 1.00
intersect bitArray bitLen 128 55.379 ns/op 63.617 ns/op 0.87
intersect array and set length 128 1.7388 us/op 1.9581 us/op 0.89
pass gossip attestations to forkchoice per slot 2.8337 ms/op 3.1017 ms/op 0.91
computeDeltas 3.1340 ms/op 3.7253 ms/op 0.84
computeProposerBoostScoreFromBalances 803.84 us/op 813.07 us/op 0.99
altair processAttestation - 250000 vs - 7PWei normalcase 3.4814 ms/op 4.0438 ms/op 0.86
altair processAttestation - 250000 vs - 7PWei worstcase 5.3248 ms/op 5.6920 ms/op 0.94
altair processAttestation - setStatus - 1/6 committees join 175.84 us/op 189.29 us/op 0.93
altair processAttestation - setStatus - 1/3 committees join 347.07 us/op 359.07 us/op 0.97
altair processAttestation - setStatus - 1/2 committees join 496.73 us/op 498.37 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 668.38 us/op 639.34 us/op 1.05
altair processAttestation - setStatus - 4/5 committees join 910.25 us/op 883.95 us/op 1.03
altair processAttestation - setStatus - 100% committees join 1.0938 ms/op 1.0519 ms/op 1.04
altair processBlock - 250000 vs - 7PWei normalcase 26.152 ms/op 25.802 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.704 ms/op 40.855 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase 83.136 ms/op 70.887 ms/op 1.17
altair processBlock - 250000 vs - 7PWei worstcase hashState 97.389 ms/op 91.267 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2052 ms/op 3.2410 ms/op 0.99
phase0 processBlock - 250000 vs - 7PWei worstcase 50.518 ms/op 41.516 ms/op 1.22
altair processEth1Data - 250000 vs - 7PWei normalcase 688.72 us/op 679.90 us/op 1.01
Tree 40 250000 create 703.99 ms/op 743.85 ms/op 0.95
Tree 40 250000 get(125000) 225.86 ns/op 289.10 ns/op 0.78
Tree 40 250000 set(125000) 1.9981 us/op 2.2918 us/op 0.87
Tree 40 250000 toArray() 27.270 ms/op 31.506 ms/op 0.87
Tree 40 250000 iterate all - toArray() + loop 27.379 ms/op 31.563 ms/op 0.87
Tree 40 250000 iterate all - get(i) 111.66 ms/op 99.579 ms/op 1.12
MutableVector 250000 create 13.148 ms/op 14.047 ms/op 0.94
MutableVector 250000 get(125000) 10.701 ns/op 11.584 ns/op 0.92
MutableVector 250000 set(125000) 549.55 ns/op 580.49 ns/op 0.95
MutableVector 250000 toArray() 5.9398 ms/op 6.8233 ms/op 0.87
MutableVector 250000 iterate all - toArray() + loop 5.3107 ms/op 6.8573 ms/op 0.77
MutableVector 250000 iterate all - get(i) 2.6749 ms/op 2.9015 ms/op 0.92
Array 250000 create 5.1160 ms/op 6.4732 ms/op 0.79
Array 250000 clone - spread 2.2811 ms/op 3.3200 ms/op 0.69
Array 250000 get(125000) 1.2350 ns/op 1.3880 ns/op 0.89
Array 250000 set(125000) 1.2430 ns/op 1.4300 ns/op 0.87
Array 250000 iterate all - loop 153.72 us/op 148.09 us/op 1.04
effectiveBalanceIncrements clone Uint8Array 300000 63.811 us/op 90.189 us/op 0.71
effectiveBalanceIncrements clone MutableVector 300000 692.00 ns/op 594.00 ns/op 1.16
effectiveBalanceIncrements rw all Uint8Array 300000 247.34 us/op 223.01 us/op 1.11
effectiveBalanceIncrements rw all MutableVector 300000 127.38 ms/op 150.96 ms/op 0.84
phase0 afterProcessEpoch - 250000 vs - 7PWei 187.14 ms/op 170.08 ms/op 1.10
phase0 beforeProcessEpoch - 250000 vs - 7PWei 56.052 ms/op 67.467 ms/op 0.83
altair processEpoch - mainnet_e81889 541.02 ms/op 561.52 ms/op 0.96
mainnet_e81889 - altair beforeProcessEpoch 113.56 ms/op 127.80 ms/op 0.89
mainnet_e81889 - altair processJustificationAndFinalization 15.948 us/op 21.143 us/op 0.75
mainnet_e81889 - altair processInactivityUpdates 8.7000 ms/op 11.118 ms/op 0.78
mainnet_e81889 - altair processRewardsAndPenalties 77.894 ms/op 89.157 ms/op 0.87
mainnet_e81889 - altair processRegistryUpdates 2.5280 us/op 3.5970 us/op 0.70
mainnet_e81889 - altair processSlashings 528.00 ns/op 798.00 ns/op 0.66
mainnet_e81889 - altair processEth1DataReset 571.00 ns/op 818.00 ns/op 0.70
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.0986 ms/op 2.4118 ms/op 0.87
mainnet_e81889 - altair processSlashingsReset 4.3840 us/op 6.2330 us/op 0.70
mainnet_e81889 - altair processRandaoMixesReset 3.7260 us/op 6.0530 us/op 0.62
mainnet_e81889 - altair processHistoricalRootsUpdate 581.00 ns/op 863.00 ns/op 0.67
mainnet_e81889 - altair processParticipationFlagUpdates 1.7320 us/op 4.6350 us/op 0.37
mainnet_e81889 - altair processSyncCommitteeUpdates 522.00 ns/op 2.2020 us/op 0.24
mainnet_e81889 - altair afterProcessEpoch 198.46 ms/op 194.05 ms/op 1.02
phase0 processEpoch - mainnet_e58758 483.74 ms/op 504.79 ms/op 0.96
mainnet_e58758 - phase0 beforeProcessEpoch 174.73 ms/op 245.21 ms/op 0.71
mainnet_e58758 - phase0 processJustificationAndFinalization 16.803 us/op 29.352 us/op 0.57
mainnet_e58758 - phase0 processRewardsAndPenalties 73.588 ms/op 74.686 ms/op 0.99
mainnet_e58758 - phase0 processRegistryUpdates 7.7460 us/op 10.696 us/op 0.72
mainnet_e58758 - phase0 processSlashings 564.00 ns/op 887.00 ns/op 0.64
mainnet_e58758 - phase0 processEth1DataReset 581.00 ns/op 1.1100 us/op 0.52
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6263 ms/op 1.9893 ms/op 0.82
mainnet_e58758 - phase0 processSlashingsReset 3.6740 us/op 6.3640 us/op 0.58
mainnet_e58758 - phase0 processRandaoMixesReset 3.9560 us/op 6.6090 us/op 0.60
mainnet_e58758 - phase0 processHistoricalRootsUpdate 676.00 ns/op 1.1320 us/op 0.60
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.1510 us/op 4.9010 us/op 0.64
mainnet_e58758 - phase0 afterProcessEpoch 164.75 ms/op 148.88 ms/op 1.11
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.1296 ms/op 2.4912 ms/op 0.85
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.4316 ms/op 2.2286 ms/op 1.09
altair processInactivityUpdates - 250000 normalcase 33.470 ms/op 42.157 ms/op 0.79
altair processInactivityUpdates - 250000 worstcase 40.332 ms/op 37.213 ms/op 1.08
phase0 processRegistryUpdates - 250000 normalcase 6.0900 us/op 12.167 us/op 0.50
phase0 processRegistryUpdates - 250000 badcase_full_deposits 366.98 us/op 374.61 us/op 0.98
phase0 processRegistryUpdates - 250000 worstcase 0.5 181.21 ms/op 191.97 ms/op 0.94
altair processRewardsAndPenalties - 250000 normalcase 107.59 ms/op 111.18 ms/op 0.97
altair processRewardsAndPenalties - 250000 worstcase 74.259 ms/op 76.171 ms/op 0.97
phase0 getAttestationDeltas - 250000 normalcase 12.287 ms/op 11.693 ms/op 1.05
phase0 getAttestationDeltas - 250000 worstcase 12.500 ms/op 11.766 ms/op 1.06
phase0 processSlashings - 250000 worstcase 5.1408 ms/op 4.7806 ms/op 1.08
altair processSyncCommitteeUpdates - 250000 299.96 ms/op 260.08 ms/op 1.15
BeaconState.hashTreeRoot - No change 588.00 ns/op 401.00 ns/op 1.47
BeaconState.hashTreeRoot - 1 full validator 66.992 us/op 48.857 us/op 1.37
BeaconState.hashTreeRoot - 32 full validator 650.38 us/op 478.13 us/op 1.36
BeaconState.hashTreeRoot - 512 full validator 7.0488 ms/op 5.5478 ms/op 1.27
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 81.881 us/op 60.683 us/op 1.35
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1860 ms/op 908.32 us/op 1.31
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.698 ms/op 12.735 ms/op 1.15
BeaconState.hashTreeRoot - 1 balances 63.855 us/op 47.085 us/op 1.36
BeaconState.hashTreeRoot - 32 balances 586.73 us/op 433.46 us/op 1.35
BeaconState.hashTreeRoot - 512 balances 5.3119 ms/op 4.0167 ms/op 1.32
BeaconState.hashTreeRoot - 250000 balances 107.45 ms/op 84.566 ms/op 1.27
aggregationBits - 2048 els - zipIndexesInBitList 25.572 us/op 26.966 us/op 0.95
regular array get 100000 times 62.876 us/op 59.404 us/op 1.06
wrappedArray get 100000 times 64.920 us/op 59.549 us/op 1.09
arrayWithProxy get 100000 times 28.171 ms/op 27.397 ms/op 1.03
ssz.Root.equals 539.00 ns/op 428.00 ns/op 1.26
byteArrayEquals 538.00 ns/op 417.00 ns/op 1.29
shuffle list - 16384 els 11.366 ms/op 10.350 ms/op 1.10
shuffle list - 250000 els 168.07 ms/op 152.03 ms/op 1.11
processSlot - 1 slots 12.788 us/op 10.163 us/op 1.26
processSlot - 32 slots 1.9189 ms/op 1.4964 ms/op 1.28
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 416.76 us/op 356.90 us/op 1.17
getCommitteeAssignments - req 1 vs - 250000 vc 5.4552 ms/op 4.7192 ms/op 1.16
getCommitteeAssignments - req 100 vs - 250000 vc 7.9776 ms/op 6.4794 ms/op 1.23
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5915 ms/op 6.8758 ms/op 1.25
computeProposers - vc 250000 18.662 ms/op 15.177 ms/op 1.23
computeEpochShuffling - vc 250000 171.47 ms/op 155.09 ms/op 1.11
getNextSyncCommittee - vc 250000 289.00 ms/op 250.47 ms/op 1.15

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 143603e into unstable Jul 11, 2022
@wemeetagain wemeetagain deleted the dapplion/metric-proposer-balance-diff branch July 11, 2022 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dapplion @wemeetagain