Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add JSON RPC client metrics #4127

Merged
merged 3 commits into from
Jun 7, 2022
Merged

Add JSON RPC client metrics #4127

merged 3 commits into from
Jun 7, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Jun 7, 2022

Motivation

Description

Add JSON RPC client metrics

Apply to both execution engine client, and "previous" eth1 client

@dapplion dapplion requested a review from a team as a code owner June 7, 2022 19:31
@dapplion dapplion mentioned this pull request Jun 7, 2022
Open
7 tasks
wemeetagain
wemeetagain previously approved these changes Jun 7, 2022
View reviewed changes
@wemeetagain
Copy link
Member

https://github.com/ChainSafe/lodestar/runs/6781253178?check_suite_focus=true#step:7:25

@github-actions
Copy link
Contributor

github-actions bot commented Jun 7, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9a7fed1 Previous: fe5c98e Ratio
altair processAttestation - 250000 vs - 7PWei normalcase 4.0294 ms/op 4.2532 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei worstcase 6.3759 ms/op 7.1125 ms/op 0.90
altair processAttestation - setStatus - 1/6 committees join 204.71 us/op 244.42 us/op 0.84
altair processAttestation - setStatus - 1/3 committees join 406.25 us/op 466.06 us/op 0.87
altair processAttestation - setStatus - 1/2 committees join 568.44 us/op 658.40 us/op 0.86
altair processAttestation - setStatus - 2/3 committees join 743.02 us/op 832.57 us/op 0.89
altair processAttestation - setStatus - 4/5 committees join 1.1379 ms/op 1.1577 ms/op 0.98
altair processAttestation - setStatus - 100% committees join 1.2033 ms/op 1.4989 ms/op 0.80
altair processBlock - 250000 vs - 7PWei normalcase 29.123 ms/op 37.455 ms/op 0.78
altair processBlock - 250000 vs - 7PWei normalcase hashState 38.479 ms/op 39.085 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase 97.568 ms/op 87.269 ms/op 1.12
altair processBlock - 250000 vs - 7PWei worstcase hashState 116.38 ms/op 114.69 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei normalcase 3.5745 ms/op 3.9335 ms/op 0.91
phase0 processBlock - 250000 vs - 7PWei worstcase 58.755 ms/op 56.672 ms/op 1.04
altair processEth1Data - 250000 vs - 7PWei normalcase 795.47 us/op 791.53 us/op 1.00
Tree 40 250000 create 916.58 ms/op 877.28 ms/op 1.04
Tree 40 250000 get(125000) 302.24 ns/op 368.65 ns/op 0.82
Tree 40 250000 set(125000) 3.5098 us/op 2.9955 us/op 1.17
Tree 40 250000 toArray() 36.251 ms/op 37.378 ms/op 0.97
Tree 40 250000 iterate all - toArray() + loop 35.830 ms/op 37.781 ms/op 0.95
Tree 40 250000 iterate all - get(i) 143.64 ms/op 130.26 ms/op 1.10
MutableVector 250000 create 15.619 ms/op 19.064 ms/op 0.82
MutableVector 250000 get(125000) 13.860 ns/op 18.942 ns/op 0.73
MutableVector 250000 set(125000) 707.30 ns/op 780.59 ns/op 0.91
MutableVector 250000 toArray() 6.1752 ms/op 8.5731 ms/op 0.72
MutableVector 250000 iterate all - toArray() + loop 7.0114 ms/op 8.6508 ms/op 0.81
MutableVector 250000 iterate all - get(i) 3.2112 ms/op 3.9903 ms/op 0.80
Array 250000 create 5.0347 ms/op 8.3192 ms/op 0.61
Array 250000 clone - spread 2.4209 ms/op 3.2465 ms/op 0.75
Array 250000 get(125000) 1.0790 ns/op 1.3280 ns/op 0.81
Array 250000 set(125000) 1.1010 ns/op 1.2500 ns/op 0.88
Array 250000 iterate all - loop 134.04 us/op 194.02 us/op 0.69
effectiveBalanceIncrements clone Uint8Array 300000 87.658 us/op 84.898 us/op 1.03
effectiveBalanceIncrements clone MutableVector 300000 678.00 ns/op 860.00 ns/op 0.79
effectiveBalanceIncrements rw all Uint8Array 300000 273.64 us/op 288.67 us/op 0.95
effectiveBalanceIncrements rw all MutableVector 300000 200.74 ms/op 193.32 ms/op 1.04
phase0 afterProcessEpoch - 250000 vs - 7PWei 184.63 ms/op 233.26 ms/op 0.79
phase0 beforeProcessEpoch - 250000 vs - 7PWei 73.169 ms/op 77.428 ms/op 0.94
altair processEpoch - mainnet_e81889 603.39 ms/op 658.04 ms/op 0.92
mainnet_e81889 - altair beforeProcessEpoch 167.95 ms/op 153.23 ms/op 1.10
mainnet_e81889 - altair processJustificationAndFinalization 80.803 us/op 35.021 us/op 2.31
mainnet_e81889 - altair processInactivityUpdates 11.450 ms/op 13.525 ms/op 0.85
mainnet_e81889 - altair processRewardsAndPenalties 93.150 ms/op 161.07 ms/op 0.58
mainnet_e81889 - altair processRegistryUpdates 19.530 us/op 8.7040 us/op 2.24
mainnet_e81889 - altair processSlashings 5.1720 us/op 2.0300 us/op 2.55
mainnet_e81889 - altair processEth1DataReset 4.6240 us/op 2.0680 us/op 2.24
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4732 ms/op 2.9057 ms/op 0.85
mainnet_e81889 - altair processSlashingsReset 23.229 us/op 12.266 us/op 1.89
mainnet_e81889 - altair processRandaoMixesReset 30.987 us/op 13.214 us/op 2.35
mainnet_e81889 - altair processHistoricalRootsUpdate 6.7720 us/op 2.3160 us/op 2.92
mainnet_e81889 - altair processParticipationFlagUpdates 18.278 us/op 6.9480 us/op 2.63
mainnet_e81889 - altair processSyncCommitteeUpdates 4.5940 us/op 2.0350 us/op 2.26
mainnet_e81889 - altair afterProcessEpoch 221.72 ms/op 238.42 ms/op 0.93
phase0 processEpoch - mainnet_e58758 588.41 ms/op 667.07 ms/op 0.88
mainnet_e58758 - phase0 beforeProcessEpoch 269.01 ms/op 219.97 ms/op 1.22
mainnet_e58758 - phase0 processJustificationAndFinalization 66.565 us/op 29.213 us/op 2.28
mainnet_e58758 - phase0 processRewardsAndPenalties 121.62 ms/op 85.764 ms/op 1.42
mainnet_e58758 - phase0 processRegistryUpdates 34.785 us/op 18.951 us/op 1.84
mainnet_e58758 - phase0 processSlashings 3.1810 us/op 2.2310 us/op 1.43
mainnet_e58758 - phase0 processEth1DataReset 4.1370 us/op 1.7860 us/op 2.32
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.9983 ms/op 2.3437 ms/op 0.85
mainnet_e58758 - phase0 processSlashingsReset 27.146 us/op 10.571 us/op 2.57
mainnet_e58758 - phase0 processRandaoMixesReset 31.152 us/op 14.669 us/op 2.12
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.8130 us/op 2.1640 us/op 2.22
mainnet_e58758 - phase0 processParticipationRecordUpdates 24.212 us/op 13.094 us/op 1.85
mainnet_e58758 - phase0 afterProcessEpoch 181.64 ms/op 190.95 ms/op 0.95
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.4664 ms/op 3.4786 ms/op 0.71
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.6134 ms/op 4.3407 ms/op 0.60
altair processInactivityUpdates - 250000 normalcase 33.177 ms/op 37.380 ms/op 0.89
altair processInactivityUpdates - 250000 worstcase 32.673 ms/op 38.025 ms/op 0.86
phase0 processRegistryUpdates - 250000 normalcase 32.814 us/op 14.914 us/op 2.20
phase0 processRegistryUpdates - 250000 badcase_full_deposits 518.95 us/op 514.87 us/op 1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5 214.86 ms/op 243.77 ms/op 0.88
altair processRewardsAndPenalties - 250000 normalcase 90.098 ms/op 94.623 ms/op 0.95
altair processRewardsAndPenalties - 250000 worstcase 138.43 ms/op 147.20 ms/op 0.94
phase0 getAttestationDeltas - 250000 normalcase 12.889 ms/op 14.519 ms/op 0.89
phase0 getAttestationDeltas - 250000 worstcase 13.233 ms/op 14.873 ms/op 0.89
phase0 processSlashings - 250000 worstcase 5.9958 ms/op 6.2311 ms/op 0.96
altair processSyncCommitteeUpdates - 250000 326.64 ms/op 328.87 ms/op 0.99
BeaconState.hashTreeRoot - No change 666.00 ns/op 669.00 ns/op 1.00
BeaconState.hashTreeRoot - 1 full validator 82.190 us/op 72.223 us/op 1.14
BeaconState.hashTreeRoot - 32 full validator 859.06 us/op 702.44 us/op 1.22
BeaconState.hashTreeRoot - 512 full validator 8.3388 ms/op 7.9878 ms/op 1.04
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 105.42 us/op 97.692 us/op 1.08
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.4924 ms/op 1.3758 ms/op 1.08
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.926 ms/op 18.669 ms/op 1.07
BeaconState.hashTreeRoot - 1 balances 83.195 us/op 83.282 us/op 1.00
BeaconState.hashTreeRoot - 32 balances 720.42 us/op 559.57 us/op 1.29
BeaconState.hashTreeRoot - 512 balances 7.3137 ms/op 6.0020 ms/op 1.22
BeaconState.hashTreeRoot - 250000 balances 102.81 ms/op 99.552 ms/op 1.03
aggregationBits - 2048 els - zipIndexesInBitList 31.394 us/op 40.944 us/op 0.77
regular array get 100000 times 54.269 us/op 79.541 us/op 0.68
wrappedArray get 100000 times 53.852 us/op 78.662 us/op 0.68
arrayWithProxy get 100000 times 31.979 ms/op 34.433 ms/op 0.93
ssz.Root.equals 550.00 ns/op 600.00 ns/op 0.92
byteArrayEquals 547.00 ns/op 597.00 ns/op 0.92
shuffle list - 16384 els 11.224 ms/op 13.531 ms/op 0.83
shuffle list - 250000 els 161.77 ms/op 192.73 ms/op 0.84
processSlot - 1 slots 15.653 us/op 15.030 us/op 1.04
processSlot - 32 slots 2.2173 ms/op 2.2033 ms/op 1.01
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 953.32 us/op 464.24 us/op 2.05
getCommitteeAssignments - req 1 vs - 250000 vc 4.9839 ms/op 6.2901 ms/op 0.79
getCommitteeAssignments - req 100 vs - 250000 vc 7.1157 ms/op 8.6212 ms/op 0.83
getCommitteeAssignments - req 1000 vs - 250000 vc 7.8944 ms/op 9.3607 ms/op 0.84
computeProposers - vc 250000 20.789 ms/op 21.965 ms/op 0.95
computeEpochShuffling - vc 250000 166.13 ms/op 190.77 ms/op 0.87
getNextSyncCommittee - vc 250000 308.62 ms/op 319.84 ms/op 0.96
pass gossip attestations to forkchoice per slot 3.5347 ms/op 3.7587 ms/op 0.94
computeDeltas 3.8344 ms/op 3.8167 ms/op 1.00
computeProposerBoostScoreFromBalances 856.69 us/op 1.0540 ms/op 0.81
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.2084 ms/op 2.3684 ms/op 0.93
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 89.686 us/op 83.964 us/op 1.07
BLS verify - blst-native 2.5021 ms/op 2.2210 ms/op 1.13
BLS verifyMultipleSignatures 3 - blst-native 5.5772 ms/op 4.3908 ms/op 1.27
BLS verifyMultipleSignatures 8 - blst-native 12.118 ms/op 9.5043 ms/op 1.27
BLS verifyMultipleSignatures 32 - blst-native 40.038 ms/op 34.271 ms/op 1.17
BLS aggregatePubkeys 32 - blst-native 56.795 us/op 45.155 us/op 1.26
BLS aggregatePubkeys 128 - blst-native 215.39 us/op 180.74 us/op 1.19
getAttestationsForBlock 64.396 ms/op 72.374 ms/op 0.89
isKnown best case - 1 super set check 525.00 ns/op 512.00 ns/op 1.03
isKnown normal case - 2 super set checks 525.00 ns/op 498.00 ns/op 1.05
isKnown worse case - 16 super set checks 505.00 ns/op 478.00 ns/op 1.06
CheckpointStateCache - add get delete 12.738 us/op 12.525 us/op 1.02
validate gossip signedAggregateAndProof - struct 5.9015 ms/op 4.8674 ms/op 1.21
validate gossip attestation - struct 2.7483 ms/op 2.3562 ms/op 1.17
altair verifyImport mainnet_s3766816:31 7.8664 s/op 7.3233 s/op 1.07
pickEth1Vote - no votes 2.2927 ms/op 2.5509 ms/op 0.90
pickEth1Vote - max votes 25.319 ms/op 31.434 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.203 ms/op 14.564 ms/op 0.98
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.500 ms/op 24.151 ms/op 1.01
pickEth1Vote - Eth1Data fastSerialize value x2048 1.8386 ms/op 1.8461 ms/op 1.00
pickEth1Vote - Eth1Data fastSerialize tree x2048 20.224 ms/op 21.053 ms/op 0.96
bytes32 toHexString 1.2450 us/op 1.3450 us/op 0.93
bytes32 Buffer.toString(hex) 839.00 ns/op 851.00 ns/op 0.99
bytes32 Buffer.toString(hex) from Uint8Array 1.0690 us/op 1.1440 us/op 0.93
bytes32 Buffer.toString(hex) + 0x 798.00 ns/op 851.00 ns/op 0.94
Object access 1 prop 0.41200 ns/op 0.44700 ns/op 0.92
Map access 1 prop 0.34400 ns/op 0.34500 ns/op 1.00
Object get x1000 15.718 ns/op 20.574 ns/op 0.76
Map get x1000 1.0560 ns/op 1.1700 ns/op 0.90
Object set x1000 104.07 ns/op 142.08 ns/op 0.73
Map set x1000 72.375 ns/op 85.586 ns/op 0.85
Return object 10000 times 0.39620 ns/op 0.42840 ns/op 0.92
Throw Error 10000 times 6.3300 us/op 6.8597 us/op 0.92
enrSubnets - fastDeserialize 64 bits 2.8670 us/op 3.3620 us/op 0.85
enrSubnets - ssz BitVector 64 bits 831.00 ns/op 905.00 ns/op 0.92
enrSubnets - fastDeserialize 4 bits 416.00 ns/op 501.00 ns/op 0.83
enrSubnets - ssz BitVector 4 bits 858.00 ns/op 889.00 ns/op 0.97
prioritizePeers score -10:0 att 32-0.1 sync 2-0 104.25 us/op 114.25 us/op 0.91
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 141.11 us/op 160.76 us/op 0.88
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 251.63 us/op 267.66 us/op 0.94
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 571.53 us/op 468.63 us/op 1.22
prioritizePeers score 0:0 att 64-1 sync 4-1 591.74 us/op 531.97 us/op 1.11
RateTracker 1000000 limit, 1 obj count per request 183.35 ns/op 224.79 ns/op 0.82
RateTracker 1000000 limit, 2 obj count per request 137.76 ns/op 168.90 ns/op 0.82
RateTracker 1000000 limit, 4 obj count per request 121.24 ns/op 139.84 ns/op 0.87
RateTracker 1000000 limit, 8 obj count per request 111.28 ns/op 126.59 ns/op 0.88
RateTracker with prune 5.0760 us/op 5.0990 us/op 1.00
array of 16000 items push then shift 5.3229 us/op 3.7792 us/op 1.41
LinkedList of 16000 items push then shift 28.439 ns/op 29.212 ns/op 0.97
array of 16000 items push then pop 258.45 ns/op 275.78 ns/op 0.94
LinkedList of 16000 items push then pop 23.236 ns/op 24.357 ns/op 0.95
array of 24000 items push then shift 7.7145 us/op 5.2526 us/op 1.47
LinkedList of 24000 items push then shift 25.335 ns/op 29.023 ns/op 0.87
array of 24000 items push then pop 213.18 ns/op 232.81 ns/op 0.92
LinkedList of 24000 items push then pop 21.552 ns/op 23.869 ns/op 0.90
intersect bitArray bitLen 8 11.399 ns/op 13.221 ns/op 0.86
intersect array and set length 8 172.56 ns/op 192.20 ns/op 0.90
intersect bitArray bitLen 128 64.938 ns/op 83.692 ns/op 0.78
intersect array and set length 128 2.1987 us/op 2.6482 us/op 0.83

by benchmarkbot/action

@wemeetagain wemeetagain merged commit d716b07 into unstable Jun 7, 2022
@wemeetagain wemeetagain deleted the dapplion/metrics-json-rpc-client branch June 7, 2022 20:55
This was referenced Jun 13, 2022
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dapplion @wemeetagain