Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: migrate state-transition unit tests to vitest #6173

Merged
merged 7 commits into from
Dec 18, 2023
Merged

test: migrate state-transition unit tests to vitest #6173

merged 7 commits into from
Dec 18, 2023

Conversation

nazarhussain
Copy link
Contributor

Motivation

Consolidate the testing frameworks and migrate to vitest.

Description

Migrate state-transition unit tests to vitest.

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner December 8, 2023 15:17
@nazarhussain nazarhussain self-assigned this Dec 8, 2023
@codecov Codecov
Copy link

codecov bot commented Dec 8, 2023 •
edited
Loading

Codecov Report

Merging #6173 (83101fa) into unstable (8bd19f4) will increase coverage by 10.33%.
Report is 16 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files 
@@              Coverage Diff              @@
##           unstable    #6173       +/-   ##
=============================================
+ Coverage     80.02%   90.35%   +10.33%     
=============================================
  Files            19       78       +59     
  Lines          1717     8087     +6370     
  Branches        155      490      +335     
=============================================
+ Hits           1374     7307     +5933     
- Misses          341      772      +431     
- Partials          2        8        +6

wemeetagain
wemeetagain previously approved these changes Dec 12, 2023
View reviewed changes
@nazarhussain nazarhussain marked this pull request as draft December 14, 2023 16:54
@nazarhussain
Copy link
Contributor Author

Converting to draft to shortly fix the perf test issue.

wemeetagain
wemeetagain previously approved these changes Dec 14, 2023
View reviewed changes
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, straightforward conversion with no ???s

@nazarhussain nazarhussain marked this pull request as ready for review December 15, 2023 13:26
@nazarhussain
Copy link
Contributor Author

lgtm, straightforward conversion with no ???s

Yes for this package it was very straight forward transition. Expectations was translated with tool and mocking was done manually.

@github-actions GitHub Actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: d10023f Previous: 4658348 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 628.10 us/op 882.70 us/op 0.71
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 84.444 us/op 120.35 us/op 0.70
BLS verify - blst-native 1.3107 ms/op 1.4129 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 2.7728 ms/op 3.0208 ms/op 0.92
BLS verifyMultipleSignatures 8 - blst-native 6.1511 ms/op 6.7764 ms/op 0.91
BLS verifyMultipleSignatures 32 - blst-native 22.433 ms/op 25.414 ms/op 0.88
BLS verifyMultipleSignatures 64 - blst-native 44.295 ms/op 46.629 ms/op 0.95
BLS verifyMultipleSignatures 128 - blst-native 88.011 ms/op 95.842 ms/op 0.92
BLS deserializing 10000 signatures 952.60 ms/op 1.0122 s/op 0.94
BLS deserializing 100000 signatures 9.7051 s/op 9.7136 s/op 1.00
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3716 ms/op 1.4406 ms/op 0.95
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5430 ms/op 1.6743 ms/op 0.92
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.3770 ms/op 2.5283 ms/op 0.94
BLS verifyMultipleSignatures - same message - 64 - blst-native 4.4940 ms/op 3.9022 ms/op 1.15
BLS verifyMultipleSignatures - same message - 128 - blst-native 7.9523 ms/op 6.2711 ms/op 1.27
BLS aggregatePubkeys 32 - blst-native 26.361 us/op 28.518 us/op 0.92
BLS aggregatePubkeys 128 - blst-native 102.81 us/op 111.93 us/op 0.92
getAttestationsForBlock 53.167 ms/op 71.687 ms/op 0.74
getSlashingsAndExits - default max 204.18 us/op 213.84 us/op 0.95
getSlashingsAndExits - 2k 425.59 us/op 629.77 us/op 0.68
proposeBlockBody type=full, size=empty 5.5540 ms/op 6.4094 ms/op 0.87
isKnown best case - 1 super set check 622.00 ns/op 392.00 ns/op 1.59
isKnown normal case - 2 super set checks 554.00 ns/op 442.00 ns/op 1.25
isKnown worse case - 16 super set checks 546.00 ns/op 431.00 ns/op 1.27
CheckpointStateCache - add get delete 6.6070 us/op 6.2830 us/op 1.05
validate api signedAggregateAndProof - struct 2.9366 ms/op 2.9800 ms/op 0.99
validate gossip signedAggregateAndProof - struct 3.0424 ms/op 2.9898 ms/op 1.02
validate gossip attestation - vc 640000 1.4698 ms/op 1.4403 ms/op 1.02
batch validate gossip attestation - vc 640000 - chunk 32 192.61 us/op 172.52 us/op 1.12
batch validate gossip attestation - vc 640000 - chunk 64 176.56 us/op 149.08 us/op 1.18
batch validate gossip attestation - vc 640000 - chunk 128 170.87 us/op 141.79 us/op 1.21
batch validate gossip attestation - vc 640000 - chunk 256 154.02 us/op 139.49 us/op 1.10
pickEth1Vote - no votes 1.4188 ms/op 1.4380 ms/op 0.99
pickEth1Vote - max votes 14.030 ms/op 13.693 ms/op 1.02
pickEth1Vote - Eth1Data hashTreeRoot value x2048 26.275 ms/op 19.386 ms/op 1.36
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 28.818 ms/op 34.656 ms/op 0.83
pickEth1Vote - Eth1Data fastSerialize value x2048 739.54 us/op 727.01 us/op 1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.1359 ms/op 8.4885 ms/op 0.72
bytes32 toHexString 882.00 ns/op 552.00 ns/op 1.60
bytes32 Buffer.toString(hex) 341.00 ns/op 303.00 ns/op 1.13
bytes32 Buffer.toString(hex) from Uint8Array 585.00 ns/op 455.00 ns/op 1.29
bytes32 Buffer.toString(hex) + 0x 316.00 ns/op 309.00 ns/op 1.02
Object access 1 prop 0.25000 ns/op 0.17300 ns/op 1.45
Map access 1 prop 0.15800 ns/op 0.15400 ns/op 1.03
Object get x1000 7.8570 ns/op 7.5740 ns/op 1.04
Map get x1000 0.90700 ns/op 0.85500 ns/op 1.06
Object set x1000 76.986 ns/op 60.155 ns/op 1.28
Map set x1000 59.734 ns/op 49.719 ns/op 1.20
Return object 10000 times 0.27660 ns/op 0.25420 ns/op 1.09
Throw Error 10000 times 4.1712 us/op 3.9618 us/op 1.05
fastMsgIdFn sha256 / 200 bytes 3.6320 us/op 3.3960 us/op 1.07
fastMsgIdFn h32 xxhash / 200 bytes 386.00 ns/op 334.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 200 bytes 420.00 ns/op 411.00 ns/op 1.02
fastMsgIdFn sha256 / 1000 bytes 12.173 us/op 12.253 us/op 0.99
fastMsgIdFn h32 xxhash / 1000 bytes 540.00 ns/op 501.00 ns/op 1.08
fastMsgIdFn h64 xxhash / 1000 bytes 504.00 ns/op 479.00 ns/op 1.05
fastMsgIdFn sha256 / 10000 bytes 110.25 us/op 112.67 us/op 0.98
fastMsgIdFn h32 xxhash / 10000 bytes 2.1110 us/op 2.0950 us/op 1.01
fastMsgIdFn h64 xxhash / 10000 bytes 1.4740 us/op 1.4890 us/op 0.99
send data - 1000 256B messages 23.541 ms/op 22.716 ms/op 1.04
send data - 1000 512B messages 31.627 ms/op 29.809 ms/op 1.06
send data - 1000 1024B messages 46.293 ms/op 46.404 ms/op 1.00
send data - 1000 1200B messages 41.331 ms/op 46.073 ms/op 0.90
send data - 1000 2048B messages 52.658 ms/op 50.023 ms/op 1.05
send data - 1000 4096B messages 43.436 ms/op 48.695 ms/op 0.89
send data - 1000 16384B messages 111.26 ms/op 121.81 ms/op 0.91
send data - 1000 65536B messages 494.41 ms/op 555.25 ms/op 0.89
enrSubnets - fastDeserialize 64 bits 1.3450 us/op 1.8390 us/op 0.73
enrSubnets - ssz BitVector 64 bits 438.00 ns/op 551.00 ns/op 0.79
enrSubnets - fastDeserialize 4 bits 218.00 ns/op 255.00 ns/op 0.85
enrSubnets - ssz BitVector 4 bits 520.00 ns/op 560.00 ns/op 0.93
prioritizePeers score -10:0 att 32-0.1 sync 2-0 107.32 us/op 120.19 us/op 0.89
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 147.49 us/op 152.32 us/op 0.97
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 197.74 us/op 227.55 us/op 0.87
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 345.44 us/op 358.07 us/op 0.96
prioritizePeers score 0:0 att 64-1 sync 4-1 374.49 us/op 389.00 us/op 0.96
array of 16000 items push then shift 1.7247 us/op 1.7831 us/op 0.97
LinkedList of 16000 items push then shift 9.4930 ns/op 9.9810 ns/op 0.95
array of 16000 items push then pop 107.88 ns/op 109.65 ns/op 0.98
LinkedList of 16000 items push then pop 9.0110 ns/op 9.7930 ns/op 0.92
array of 24000 items push then shift 2.6070 us/op 2.6875 us/op 0.97
LinkedList of 24000 items push then shift 9.1870 ns/op 10.010 ns/op 0.92
array of 24000 items push then pop 142.55 ns/op 150.97 ns/op 0.94
LinkedList of 24000 items push then pop 9.1280 ns/op 9.3180 ns/op 0.98
intersect bitArray bitLen 8 6.8080 ns/op 6.8220 ns/op 1.00
intersect array and set length 8 73.140 ns/op 77.427 ns/op 0.94
intersect bitArray bitLen 128 34.640 ns/op 36.184 ns/op 0.96
intersect array and set length 128 1.0049 us/op 1.1185 us/op 0.90
bitArray.getTrueBitIndexes() bitLen 128 1.8270 us/op 1.8200 us/op 1.00
bitArray.getTrueBitIndexes() bitLen 248 2.8170 us/op 3.0830 us/op 0.91
bitArray.getTrueBitIndexes() bitLen 512 6.4020 us/op 5.8150 us/op 1.10
Buffer.concat 32 items 1.0460 us/op 1.0460 us/op 1.00
Uint8Array.set 32 items 2.0070 us/op 2.0830 us/op 0.96
Set add up to 64 items then delete first 5.1942 us/op 4.8910 us/op 1.06
OrderedSet add up to 64 items then delete first 6.9806 us/op 6.5355 us/op 1.07
Set add up to 64 items then delete last 5.5546 us/op 5.1979 us/op 1.07
OrderedSet add up to 64 items then delete last 7.1876 us/op 6.6742 us/op 1.08
Set add up to 64 items then delete middle 5.3996 us/op 5.1421 us/op 1.05
OrderedSet add up to 64 items then delete middle 8.6564 us/op 8.4158 us/op 1.03
Set add up to 128 items then delete first 10.659 us/op 10.801 us/op 0.99
OrderedSet add up to 128 items then delete first 15.583 us/op 15.508 us/op 1.00
Set add up to 128 items then delete last 10.933 us/op 11.019 us/op 0.99
OrderedSet add up to 128 items then delete last 14.443 us/op 13.310 us/op 1.09
Set add up to 128 items then delete middle 10.305 us/op 10.886 us/op 0.95
OrderedSet add up to 128 items then delete middle 19.140 us/op 19.622 us/op 0.98
Set add up to 256 items then delete first 21.730 us/op 21.072 us/op 1.03
OrderedSet add up to 256 items then delete first 28.947 us/op 27.862 us/op 1.04
Set add up to 256 items then delete last 21.486 us/op 21.466 us/op 1.00
OrderedSet add up to 256 items then delete last 27.598 us/op 25.540 us/op 1.08
Set add up to 256 items then delete middle 21.646 us/op 19.239 us/op 1.13
OrderedSet add up to 256 items then delete middle 51.717 us/op 51.681 us/op 1.00
transfer serialized Status (84 B) 1.9780 us/op 1.8330 us/op 1.08
copy serialized Status (84 B) 1.7440 us/op 1.6010 us/op 1.09
transfer serialized SignedVoluntaryExit (112 B) 2.2710 us/op 1.9120 us/op 1.19
copy serialized SignedVoluntaryExit (112 B) 1.8180 us/op 1.6830 us/op 1.08
transfer serialized ProposerSlashing (416 B) 3.3220 us/op 2.1880 us/op 1.52
copy serialized ProposerSlashing (416 B) 2.6140 us/op 2.1440 us/op 1.22
transfer serialized Attestation (485 B) 2.2230 us/op 2.2670 us/op 0.98
copy serialized Attestation (485 B) 2.2050 us/op 3.1110 us/op 0.71
transfer serialized AttesterSlashing (33232 B) 2.2380 us/op 3.1600 us/op 0.71
copy serialized AttesterSlashing (33232 B) 8.7970 us/op 6.8690 us/op 1.28
transfer serialized Small SignedBeaconBlock (128000 B) 3.8490 us/op 3.2410 us/op 1.19
copy serialized Small SignedBeaconBlock (128000 B) 20.500 us/op 20.516 us/op 1.00
transfer serialized Avg SignedBeaconBlock (200000 B) 4.2010 us/op 3.6510 us/op 1.15
copy serialized Avg SignedBeaconBlock (200000 B) 26.091 us/op 30.211 us/op 0.86
transfer serialized BlobsSidecar (524380 B) 3.9530 us/op 3.7160 us/op 1.06
copy serialized BlobsSidecar (524380 B) 84.172 us/op 146.42 us/op 0.57
transfer serialized Big SignedBeaconBlock (1000000 B) 4.6270 us/op 3.9060 us/op 1.18
copy serialized Big SignedBeaconBlock (1000000 B) 188.19 us/op 440.02 us/op 0.43
pass gossip attestations to forkchoice per slot 4.4249 ms/op 4.2543 ms/op 1.04
forkChoice updateHead vc 100000 bc 64 eq 0 705.19 us/op 822.73 us/op 0.86
forkChoice updateHead vc 600000 bc 64 eq 0 5.1792 ms/op 4.4371 ms/op 1.17
forkChoice updateHead vc 1000000 bc 64 eq 0 8.8486 ms/op 7.6048 ms/op 1.16
forkChoice updateHead vc 600000 bc 320 eq 0 4.9868 ms/op 4.8353 ms/op 1.03
forkChoice updateHead vc 600000 bc 1200 eq 0 4.6985 ms/op 4.9266 ms/op 0.95
forkChoice updateHead vc 600000 bc 7200 eq 0 5.5727 ms/op 5.8589 ms/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 1000 11.693 ms/op 12.515 ms/op 0.93
forkChoice updateHead vc 600000 bc 64 eq 10000 12.484 ms/op 12.687 ms/op 0.98
forkChoice updateHead vc 600000 bc 64 eq 300000 27.234 ms/op 20.370 ms/op 1.34
computeDeltas 500000 validators 300 proto nodes 7.3265 ms/op 7.3850 ms/op 0.99
computeDeltas 500000 validators 1200 proto nodes 6.9936 ms/op 7.5132 ms/op 0.93
computeDeltas 500000 validators 7200 proto nodes 6.6059 ms/op 7.1197 ms/op 0.93
computeDeltas 750000 validators 300 proto nodes 9.9355 ms/op 10.983 ms/op 0.90
computeDeltas 750000 validators 1200 proto nodes 10.606 ms/op 10.807 ms/op 0.98
computeDeltas 750000 validators 7200 proto nodes 10.473 ms/op 10.959 ms/op 0.96
computeDeltas 1400000 validators 300 proto nodes 19.399 ms/op 21.770 ms/op 0.89
computeDeltas 1400000 validators 1200 proto nodes 18.879 ms/op 20.724 ms/op 0.91
computeDeltas 1400000 validators 7200 proto nodes 18.938 ms/op 20.340 ms/op 0.93
computeDeltas 2100000 validators 300 proto nodes 27.883 ms/op 30.408 ms/op 0.92
computeDeltas 2100000 validators 1200 proto nodes 28.495 ms/op 30.126 ms/op 0.95
computeDeltas 2100000 validators 7200 proto nodes 27.871 ms/op 30.063 ms/op 0.93
computeProposerBoostScoreFromBalances 500000 validators 3.7261 ms/op 3.9348 ms/op 0.95
computeProposerBoostScoreFromBalances 750000 validators 3.7130 ms/op 3.9708 ms/op 0.94
computeProposerBoostScoreFromBalances 1400000 validators 3.7804 ms/op 4.0003 ms/op 0.95
computeProposerBoostScoreFromBalances 2100000 validators 3.8081 ms/op 3.9232 ms/op 0.97
altair processAttestation - 250000 vs - 7PWei normalcase 3.1484 ms/op 2.3746 ms/op 1.33
altair processAttestation - 250000 vs - 7PWei worstcase 4.5425 ms/op 3.8248 ms/op 1.19
altair processAttestation - setStatus - 1/6 committees join 194.10 us/op 155.76 us/op 1.25
altair processAttestation - setStatus - 1/3 committees join 359.74 us/op 295.18 us/op 1.22
altair processAttestation - setStatus - 1/2 committees join 476.51 us/op 396.45 us/op 1.20
altair processAttestation - setStatus - 2/3 committees join 620.26 us/op 488.79 us/op 1.27
altair processAttestation - setStatus - 4/5 committees join 855.28 us/op 678.30 us/op 1.26
altair processAttestation - setStatus - 100% committees join 1.0102 ms/op 817.90 us/op 1.24
altair processBlock - 250000 vs - 7PWei normalcase 11.384 ms/op 11.305 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 38.793 ms/op 40.306 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase 43.817 ms/op 40.115 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase hashState 93.417 ms/op 102.61 ms/op 0.91
phase0 processBlock - 250000 vs - 7PWei normalcase 3.5991 ms/op 2.6923 ms/op 1.34
phase0 processBlock - 250000 vs - 7PWei worstcase 38.484 ms/op 33.188 ms/op 1.16
altair processEth1Data - 250000 vs - 7PWei normalcase 858.98 us/op 692.00 us/op 1.24
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 20.609 us/op 9.6230 us/op 2.14
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 68.715 us/op 73.695 us/op 0.93
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 35.408 us/op 26.418 us/op 1.34
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 23.351 us/op 15.115 us/op 1.54
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 204.60 us/op 195.04 us/op 1.05
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.5065 ms/op 1.2987 ms/op 1.16
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.1299 ms/op 1.8292 ms/op 1.16
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.6057 ms/op 1.5619 ms/op 1.67
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.9654 ms/op 3.5488 ms/op 1.40
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.7794 ms/op 2.7979 ms/op 0.99
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 7.7276 ms/op 6.3707 ms/op 1.21
Tree 40 250000 create 841.93 ms/op 364.58 ms/op 2.31
Tree 40 250000 get(125000) 230.41 ns/op 205.96 ns/op 1.12
Tree 40 250000 set(125000) 2.7467 us/op 1.0673 us/op 2.57
Tree 40 250000 toArray() 28.060 ms/op 20.403 ms/op 1.38
Tree 40 250000 iterate all - toArray() + loop 26.061 ms/op 20.892 ms/op 1.25
Tree 40 250000 iterate all - get(i) 79.587 ms/op 70.407 ms/op 1.13
MutableVector 250000 create 18.599 ms/op 15.209 ms/op 1.22
MutableVector 250000 get(125000) 6.9290 ns/op 6.6350 ns/op 1.04
MutableVector 250000 set(125000) 626.43 ns/op 293.36 ns/op 2.14
MutableVector 250000 toArray() 5.0616 ms/op 3.3986 ms/op 1.49
MutableVector 250000 iterate all - toArray() + loop 5.1969 ms/op 5.8806 ms/op 0.88
MutableVector 250000 iterate all - get(i) 1.6309 ms/op 1.5632 ms/op 1.04
Array 250000 create 6.8856 ms/op 3.0244 ms/op 2.28
Array 250000 clone - spread 1.5400 ms/op 1.2683 ms/op 1.21
Array 250000 get(125000) 1.5920 ns/op 1.0620 ns/op 1.50
Array 250000 set(125000) 5.5940 ns/op 4.2250 ns/op 1.32
Array 250000 iterate all - loop 190.90 us/op 169.60 us/op 1.13
effectiveBalanceIncrements clone Uint8Array 300000 61.442 us/op 25.863 us/op 2.38
effectiveBalanceIncrements clone MutableVector 300000 395.00 ns/op 365.00 ns/op 1.08
effectiveBalanceIncrements rw all Uint8Array 300000 216.49 us/op 211.60 us/op 1.02
effectiveBalanceIncrements rw all MutableVector 300000 106.42 ms/op 84.749 ms/op 1.26
phase0 afterProcessEpoch - 250000 vs - 7PWei 123.63 ms/op 117.50 ms/op 1.05
phase0 beforeProcessEpoch - 250000 vs - 7PWei 60.125 ms/op 54.913 ms/op 1.09
altair processEpoch - mainnet_e81889 525.90 ms/op 491.14 ms/op 1.07
mainnet_e81889 - altair beforeProcessEpoch 85.417 ms/op 88.570 ms/op 0.96
mainnet_e81889 - altair processJustificationAndFinalization 15.229 us/op 16.587 us/op 0.92
mainnet_e81889 - altair processInactivityUpdates 6.1219 ms/op 6.8458 ms/op 0.89
mainnet_e81889 - altair processRewardsAndPenalties 63.296 ms/op 66.975 ms/op 0.95
mainnet_e81889 - altair processRegistryUpdates 2.4220 us/op 2.6990 us/op 0.90
mainnet_e81889 - altair processSlashings 428.00 ns/op 381.00 ns/op 1.12
mainnet_e81889 - altair processEth1DataReset 440.00 ns/op 440.00 ns/op 1.00
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4430 ms/op 1.4510 ms/op 0.99
mainnet_e81889 - altair processSlashingsReset 3.0150 us/op 3.3780 us/op 0.89
mainnet_e81889 - altair processRandaoMixesReset 3.9550 us/op 5.2960 us/op 0.75
mainnet_e81889 - altair processHistoricalRootsUpdate 969.00 ns/op 656.00 ns/op 1.48
mainnet_e81889 - altair processParticipationFlagUpdates 1.4290 us/op 2.3550 us/op 0.61
mainnet_e81889 - altair processSyncCommitteeUpdates 584.00 ns/op 676.00 ns/op 0.86
mainnet_e81889 - altair afterProcessEpoch 120.27 ms/op 125.40 ms/op 0.96
capella processEpoch - mainnet_e217614 2.6717 s/op 2.2830 s/op 1.17
mainnet_e217614 - capella beforeProcessEpoch 531.41 ms/op 507.27 ms/op 1.05
mainnet_e217614 - capella processJustificationAndFinalization 33.369 us/op 15.519 us/op 2.15
mainnet_e217614 - capella processInactivityUpdates 28.832 ms/op 21.027 ms/op 1.37
mainnet_e217614 - capella processRewardsAndPenalties 491.74 ms/op 424.59 ms/op 1.16
mainnet_e217614 - capella processRegistryUpdates 48.529 us/op 22.098 us/op 2.20
mainnet_e217614 - capella processSlashings 1.6420 us/op 888.00 ns/op 1.85
mainnet_e217614 - capella processEth1DataReset 1.3790 us/op 398.00 ns/op 3.46
mainnet_e217614 - capella processEffectiveBalanceUpdates 8.5844 ms/op 6.1410 ms/op 1.40
mainnet_e217614 - capella processSlashingsReset 9.4520 us/op 3.6900 us/op 2.56
mainnet_e217614 - capella processRandaoMixesReset 11.737 us/op 4.6470 us/op 2.53
mainnet_e217614 - capella processHistoricalRootsUpdate 1.2040 us/op 472.00 ns/op 2.55
mainnet_e217614 - capella processParticipationFlagUpdates 4.2560 us/op 1.4450 us/op 2.95
mainnet_e217614 - capella afterProcessEpoch 397.60 ms/op 317.27 ms/op 1.25
phase0 processEpoch - mainnet_e58758 602.21 ms/op 463.71 ms/op 1.30
mainnet_e58758 - phase0 beforeProcessEpoch 187.12 ms/op 131.53 ms/op 1.42
mainnet_e58758 - phase0 processJustificationAndFinalization 30.291 us/op 16.907 us/op 1.79
mainnet_e58758 - phase0 processRewardsAndPenalties 62.923 ms/op 53.273 ms/op 1.18
mainnet_e58758 - phase0 processRegistryUpdates 23.937 us/op 23.246 us/op 1.03
mainnet_e58758 - phase0 processSlashings 1.1090 us/op 718.00 ns/op 1.54
mainnet_e58758 - phase0 processEth1DataReset 1.1600 us/op 1.1730 us/op 0.99
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1735 ms/op 1.1838 ms/op 1.84
mainnet_e58758 - phase0 processSlashingsReset 6.9890 us/op 3.2500 us/op 2.15
mainnet_e58758 - phase0 processRandaoMixesReset 9.0640 us/op 5.3970 us/op 1.68
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4840 us/op 490.00 ns/op 3.03
mainnet_e58758 - phase0 processParticipationRecordUpdates 9.6060 us/op 5.5510 us/op 1.73
mainnet_e58758 - phase0 afterProcessEpoch 127.31 ms/op 101.45 ms/op 1.25
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.3918 ms/op 1.4237 ms/op 1.68
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.5794 ms/op 1.4887 ms/op 1.73
altair processInactivityUpdates - 250000 normalcase 41.362 ms/op 26.112 ms/op 1.58
altair processInactivityUpdates - 250000 worstcase 36.606 ms/op 27.046 ms/op 1.35
phase0 processRegistryUpdates - 250000 normalcase 19.568 us/op 9.7150 us/op 2.01
phase0 processRegistryUpdates - 250000 badcase_full_deposits 588.79 us/op 390.29 us/op 1.51
phase0 processRegistryUpdates - 250000 worstcase 0.5 226.82 ms/op 120.32 ms/op 1.89
altair processRewardsAndPenalties - 250000 normalcase 78.350 ms/op 58.650 ms/op 1.34
altair processRewardsAndPenalties - 250000 worstcase 81.419 ms/op 57.520 ms/op 1.42
phase0 getAttestationDeltas - 250000 normalcase 13.978 ms/op 10.101 ms/op 1.38
phase0 getAttestationDeltas - 250000 worstcase 13.549 ms/op 10.058 ms/op 1.35
phase0 processSlashings - 250000 worstcase 141.81 us/op 95.215 us/op 1.49
altair processSyncCommitteeUpdates - 250000 188.55 ms/op 175.48 ms/op 1.07
BeaconState.hashTreeRoot - No change 368.00 ns/op 272.00 ns/op 1.35
BeaconState.hashTreeRoot - 1 full validator 219.17 us/op 129.98 us/op 1.69
BeaconState.hashTreeRoot - 32 full validator 2.3332 ms/op 1.4092 ms/op 1.66
BeaconState.hashTreeRoot - 512 full validator 20.070 ms/op 13.575 ms/op 1.48
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 248.72 us/op 152.07 us/op 1.64
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 3.2255 ms/op 2.1238 ms/op 1.52
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 41.335 ms/op 28.175 ms/op 1.47
BeaconState.hashTreeRoot - 1 balances 180.99 us/op 143.13 us/op 1.26
BeaconState.hashTreeRoot - 32 balances 1.6240 ms/op 1.2551 ms/op 1.29
BeaconState.hashTreeRoot - 512 balances 15.866 ms/op 12.253 ms/op 1.29
BeaconState.hashTreeRoot - 250000 balances 263.03 ms/op 235.73 ms/op 1.12
aggregationBits - 2048 els - zipIndexesInBitList 38.904 us/op 16.370 us/op 2.38
byteArrayEquals 32 110.63 ns/op 74.860 ns/op 1.48
Buffer.compare 32 73.910 ns/op 56.012 ns/op 1.32
byteArrayEquals 1024 3.8141 us/op 2.0292 us/op 1.88
Buffer.compare 1024 97.565 ns/op 71.329 ns/op 1.37
byteArrayEquals 16384 54.486 us/op 32.293 us/op 1.69
Buffer.compare 16384 380.01 ns/op 261.50 ns/op 1.45
byteArrayEquals 123687377 353.67 ms/op 240.13 ms/op 1.47
Buffer.compare 123687377 12.309 ms/op 6.0344 ms/op 2.04
byteArrayEquals 32 - diff last byte 107.53 ns/op 70.432 ns/op 1.53
Buffer.compare 32 - diff last byte 74.035 ns/op 55.944 ns/op 1.32
byteArrayEquals 1024 - diff last byte 3.0624 us/op 1.9874 us/op 1.54
Buffer.compare 1024 - diff last byte 89.310 ns/op 70.364 ns/op 1.27
byteArrayEquals 16384 - diff last byte 47.549 us/op 31.684 us/op 1.50
Buffer.compare 16384 - diff last byte 357.93 ns/op 270.69 ns/op 1.32
byteArrayEquals 123687377 - diff last byte 364.32 ms/op 249.41 ms/op 1.46
Buffer.compare 123687377 - diff last byte 11.687 ms/op 6.1991 ms/op 1.89
byteArrayEquals 32 - random bytes 8.2970 ns/op 5.2710 ns/op 1.57
Buffer.compare 32 - random bytes 69.416 ns/op 59.580 ns/op 1.17
byteArrayEquals 1024 - random bytes 8.0230 ns/op 5.1280 ns/op 1.56
Buffer.compare 1024 - random bytes 69.418 ns/op 58.982 ns/op 1.18
byteArrayEquals 16384 - random bytes 7.8230 ns/op 5.0890 ns/op 1.54
Buffer.compare 16384 - random bytes 67.798 ns/op 58.769 ns/op 1.15
byteArrayEquals 123687377 - random bytes 19.180 ns/op 8.2000 ns/op 2.34
Buffer.compare 123687377 - random bytes 96.580 ns/op 62.110 ns/op 1.55
regular array get 100000 times 51.833 us/op 43.527 us/op 1.19
wrappedArray get 100000 times 54.281 us/op 43.518 us/op 1.25
arrayWithProxy get 100000 times 19.248 ms/op 13.787 ms/op 1.40
ssz.Root.equals 68.953 ns/op 53.000 ns/op 1.30
byteArrayEquals 68.081 ns/op 52.130 ns/op 1.31
Buffer.compare 17.501 ns/op 10.666 ns/op 1.64
shuffle list - 16384 els 8.5517 ms/op 6.9635 ms/op 1.23
shuffle list - 250000 els 125.31 ms/op 102.05 ms/op 1.23
processSlot - 1 slots 22.201 us/op 17.018 us/op 1.30
processSlot - 32 slots 4.6623 ms/op 3.5627 ms/op 1.31
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 71.483 ms/op 58.614 ms/op 1.22
getCommitteeAssignments - req 1 vs - 250000 vc 3.0152 ms/op 2.4704 ms/op 1.22
getCommitteeAssignments - req 100 vs - 250000 vc 4.5665 ms/op 3.6362 ms/op 1.26
getCommitteeAssignments - req 1000 vs - 250000 vc 4.9449 ms/op 4.0091 ms/op 1.23
findModifiedValidators - 10000 modified validators 786.61 ms/op 533.52 ms/op 1.47
findModifiedValidators - 1000 modified validators 662.53 ms/op 440.96 ms/op 1.50
findModifiedValidators - 100 modified validators 653.48 ms/op 409.15 ms/op 1.60
findModifiedValidators - 10 modified validators 569.66 ms/op 413.39 ms/op 1.38
findModifiedValidators - 1 modified validators 645.61 ms/op 389.95 ms/op 1.66
findModifiedValidators - no difference 593.76 ms/op 414.72 ms/op 1.43
compare ViewDUs 6.1613 s/op 4.3646 s/op 1.41
compare each validator Uint8Array 2.0530 s/op 1.8015 s/op 1.14
compare ViewDU to Uint8Array 1.6856 s/op 1.1058 s/op 1.52
migrate state 1000000 validators, 24 modified, 0 new 1.0802 s/op 793.59 ms/op 1.36
migrate state 1000000 validators, 1700 modified, 1000 new 1.5097 s/op 1.0999 s/op 1.37
migrate state 1000000 validators, 3400 modified, 2000 new 2.0780 s/op 1.3564 s/op 1.53
migrate state 1500000 validators, 24 modified, 0 new 1.1555 s/op 766.56 ms/op 1.51
migrate state 1500000 validators, 1700 modified, 1000 new 1.6154 s/op 1.0926 s/op 1.48
migrate state 1500000 validators, 3400 modified, 2000 new 1.9529 s/op 1.3374 s/op 1.46
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.6400 ns/op 4.1600 ns/op 1.60
state getBlockRootAtSlot - 250000 vs - 7PWei 892.16 ns/op 790.40 ns/op 1.13
computeProposers - vc 250000 13.499 ms/op 9.2034 ms/op 1.47
computeEpochShuffling - vc 250000 146.81 ms/op 104.71 ms/op 1.40
getNextSyncCommittee - vc 250000 192.82 ms/op 158.41 ms/op 1.22
computeSigningRoot for AttestationData 30.615 us/op 26.799 us/op 1.14
hash AttestationData serialized data then Buffer.toString(base64) 2.5195 us/op 2.3085 us/op 1.09
toHexString serialized data 1.1959 us/op 1.0741 us/op 1.11
Buffer.toString(base64) 241.72 ns/op 215.07 ns/op 1.12

by benchmarkbot/action

@nazarhussain nazarhussain mentioned this pull request Dec 15, 2023
Open
g11tech
g11tech approved these changes Dec 18, 2023
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@g11tech g11tech merged commit e09702d into unstable Dec 18, 2023
15 checks passed
@g11tech g11tech deleted the nh/test-state-transition branch December 18, 2023 09:22
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.13.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@g11tech g11tech g11tech approved these changes

@wemeetagain wemeetagain Awaiting requested review from wemeetagain

@nflaig nflaig Awaiting requested review from nflaig

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nazarhussain @wemeetagain @nflaig @g11tech