Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: stabilize sim tests #6312

Merged
merged 16 commits into from
Jan 17, 2024
Merged

test: stabilize sim tests #6312

merged 16 commits into from
Jan 17, 2024

Conversation

nazarhussain
Copy link
Contributor

Motivation

Stabilize sim tests.

Description

  • Fix genesis delay issue for github runner by increasing the genesis delay for CI
  • Fix genesis delay calculation
  • Fix unknown block publication during sim tests

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner January 16, 2024 19:03
@nazarhussain nazarhussain self-assigned this Jan 16, 2024
@codecov Codecov
Copy link

codecov bot commented Jan 16, 2024 •
edited
Loading

Codecov Report

Merging #6312 (4ff14a3) into unstable (9eb9cce) will increase coverage by 0.00%.
Report is 1 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files 
@@            Coverage Diff            @@
##           unstable    #6312   +/-   ##
=========================================
  Coverage     76.52%   76.53%           
=========================================
  Files           248      248           
  Lines         25935    25943    +8     
  Branches       1449     1449           
=========================================
+ Hits          19847    19855    +8     
  Misses         6058     6058           
  Partials         30       30

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 16, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: e7a4968 Previous: 9eb9cce Ratio
forkChoice updateHead vc 600000 bc 64 eq 300000 18.987 ms/op 62.507 ms/op 0.30
Full benchmark results
Benchmark suite Current: e7a4968 Previous: 9eb9cce Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 603.00 us/op 791.59 us/op 0.76
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 119.96 us/op 103.01 us/op 1.16
BLS verify - blst-native 1.3306 ms/op 1.3680 ms/op 0.97
BLS verifyMultipleSignatures 3 - blst-native 2.8194 ms/op 2.8373 ms/op 0.99
BLS verifyMultipleSignatures 8 - blst-native 6.1769 ms/op 6.3555 ms/op 0.97
BLS verifyMultipleSignatures 32 - blst-native 22.611 ms/op 23.450 ms/op 0.96
BLS verifyMultipleSignatures 64 - blst-native 44.895 ms/op 46.093 ms/op 0.97
BLS verifyMultipleSignatures 128 - blst-native 89.043 ms/op 91.733 ms/op 0.97
BLS deserializing 10000 signatures 933.74 ms/op 958.74 ms/op 0.97
BLS deserializing 100000 signatures 9.6149 s/op 9.7453 s/op 0.99
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3552 ms/op 1.5295 ms/op 0.89
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.8226 ms/op 1.6159 ms/op 1.13
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.3682 ms/op 2.4445 ms/op 0.97
BLS verifyMultipleSignatures - same message - 64 - blst-native 4.8650 ms/op 3.9249 ms/op 1.24
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.8616 ms/op 6.1466 ms/op 0.95
BLS aggregatePubkeys 32 - blst-native 28.240 us/op 28.447 us/op 0.99
BLS aggregatePubkeys 128 - blst-native 105.58 us/op 110.03 us/op 0.96
getAttestationsForBlock 56.815 ms/op 52.539 ms/op 1.08
getSlashingsAndExits - default max 168.88 us/op 168.34 us/op 1.00
getSlashingsAndExits - 2k 487.79 us/op 500.70 us/op 0.97
proposeBlockBody type=full, size=empty 5.5716 ms/op 6.3014 ms/op 0.88
isKnown best case - 1 super set check 610.00 ns/op 375.00 ns/op 1.63
isKnown normal case - 2 super set checks 644.00 ns/op 400.00 ns/op 1.61
isKnown worse case - 16 super set checks 627.00 ns/op 436.00 ns/op 1.44
CheckpointStateCache - add get delete 6.2780 us/op 6.3900 us/op 0.98
validate api signedAggregateAndProof - struct 2.8418 ms/op 3.0515 ms/op 0.93
validate gossip signedAggregateAndProof - struct 2.8852 ms/op 3.0198 ms/op 0.96
validate gossip attestation - vc 640000 1.4079 ms/op 1.4890 ms/op 0.95
batch validate gossip attestation - vc 640000 - chunk 32 177.51 us/op 181.00 us/op 0.98
batch validate gossip attestation - vc 640000 - chunk 64 155.90 us/op 184.28 us/op 0.85
batch validate gossip attestation - vc 640000 - chunk 128 145.89 us/op 157.37 us/op 0.93
batch validate gossip attestation - vc 640000 - chunk 256 134.52 us/op 138.16 us/op 0.97
pickEth1Vote - no votes 1.2932 ms/op 1.3260 ms/op 0.98
pickEth1Vote - max votes 9.8835 ms/op 13.394 ms/op 0.74
pickEth1Vote - Eth1Data hashTreeRoot value x2048 18.061 ms/op 21.799 ms/op 0.83
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 26.888 ms/op 30.195 ms/op 0.89
pickEth1Vote - Eth1Data fastSerialize value x2048 721.54 us/op 648.50 us/op 1.11
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.6065 ms/op 7.0413 ms/op 1.08
bytes32 toHexString 660.00 ns/op 547.00 ns/op 1.21
bytes32 Buffer.toString(hex) 307.00 ns/op 293.00 ns/op 1.05
bytes32 Buffer.toString(hex) from Uint8Array 504.00 ns/op 438.00 ns/op 1.15
bytes32 Buffer.toString(hex) + 0x 292.00 ns/op 290.00 ns/op 1.01
Object access 1 prop 0.20800 ns/op 0.16600 ns/op 1.25
Map access 1 prop 0.14800 ns/op 0.15200 ns/op 0.97
Object get x1000 7.8030 ns/op 8.0580 ns/op 0.97
Map get x1000 0.87300 ns/op 0.79600 ns/op 1.10
Object set x1000 61.571 ns/op 53.724 ns/op 1.15
Map set x1000 45.548 ns/op 42.656 ns/op 1.07
Return object 10000 times 0.24890 ns/op 0.25040 ns/op 0.99
Throw Error 10000 times 3.9903 us/op 3.9591 us/op 1.01
fastMsgIdFn sha256 / 200 bytes 3.4140 us/op 3.4080 us/op 1.00
fastMsgIdFn h32 xxhash / 200 bytes 327.00 ns/op 294.00 ns/op 1.11
fastMsgIdFn h64 xxhash / 200 bytes 380.00 ns/op 346.00 ns/op 1.10
fastMsgIdFn sha256 / 1000 bytes 11.595 us/op 12.100 us/op 0.96
fastMsgIdFn h32 xxhash / 1000 bytes 456.00 ns/op 459.00 ns/op 0.99
fastMsgIdFn h64 xxhash / 1000 bytes 452.00 ns/op 445.00 ns/op 1.02
fastMsgIdFn sha256 / 10000 bytes 107.16 us/op 105.99 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 2.0070 us/op 2.0250 us/op 0.99
fastMsgIdFn h64 xxhash / 10000 bytes 1.3570 us/op 1.4000 us/op 0.97
send data - 1000 256B messages 20.367 ms/op 20.806 ms/op 0.98
send data - 1000 512B messages 29.037 ms/op 27.238 ms/op 1.07
send data - 1000 1024B messages 41.038 ms/op 42.930 ms/op 0.96
send data - 1000 1200B messages 41.508 ms/op 38.623 ms/op 1.07
send data - 1000 2048B messages 52.513 ms/op 55.818 ms/op 0.94
send data - 1000 4096B messages 46.291 ms/op 49.729 ms/op 0.93
send data - 1000 16384B messages 119.39 ms/op 118.16 ms/op 1.01
send data - 1000 65536B messages 493.38 ms/op 556.71 ms/op 0.89
enrSubnets - fastDeserialize 64 bits 1.3540 us/op 1.7200 us/op 0.79
enrSubnets - ssz BitVector 64 bits 490.00 ns/op 620.00 ns/op 0.79
enrSubnets - fastDeserialize 4 bits 211.00 ns/op 278.00 ns/op 0.76
enrSubnets - ssz BitVector 4 bits 532.00 ns/op 771.00 ns/op 0.69
prioritizePeers score -10:0 att 32-0.1 sync 2-0 120.81 us/op 158.24 us/op 0.76
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 138.10 us/op 174.28 us/op 0.79
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 191.45 us/op 246.50 us/op 0.78
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 355.73 us/op 425.84 us/op 0.84
prioritizePeers score 0:0 att 64-1 sync 4-1 383.99 us/op 431.71 us/op 0.89
array of 16000 items push then shift 1.7206 us/op 2.0011 us/op 0.86
LinkedList of 16000 items push then shift 9.5440 ns/op 11.875 ns/op 0.80
array of 16000 items push then pop 115.86 ns/op 131.59 ns/op 0.88
LinkedList of 16000 items push then pop 9.2540 ns/op 12.246 ns/op 0.76
array of 24000 items push then shift 2.6779 us/op 2.8680 us/op 0.93
LinkedList of 24000 items push then shift 9.3970 ns/op 10.988 ns/op 0.86
array of 24000 items push then pop 165.67 ns/op 198.24 ns/op 0.84
LinkedList of 24000 items push then pop 9.2560 ns/op 11.729 ns/op 0.79
intersect bitArray bitLen 8 6.8330 ns/op 7.9110 ns/op 0.86
intersect array and set length 8 89.279 ns/op 109.27 ns/op 0.82
intersect bitArray bitLen 128 37.066 ns/op 42.096 ns/op 0.88
intersect array and set length 128 1.0263 us/op 1.3022 us/op 0.79
bitArray.getTrueBitIndexes() bitLen 128 1.7610 us/op 2.1540 us/op 0.82
bitArray.getTrueBitIndexes() bitLen 248 3.1870 us/op 3.5600 us/op 0.90
bitArray.getTrueBitIndexes() bitLen 512 5.4560 us/op 8.6730 us/op 0.63
Buffer.concat 32 items 1.1100 us/op 1.4250 us/op 0.78
Uint8Array.set 32 items 2.7290 us/op 2.6590 us/op 1.03
Set add up to 64 items then delete first 4.8456 us/op 6.0124 us/op 0.81
OrderedSet add up to 64 items then delete first 6.3438 us/op 8.1531 us/op 0.78
Set add up to 64 items then delete last 5.5185 us/op 6.4095 us/op 0.86
OrderedSet add up to 64 items then delete last 7.1218 us/op 9.0354 us/op 0.79
Set add up to 64 items then delete middle 5.4684 us/op 6.8116 us/op 0.80
OrderedSet add up to 64 items then delete middle 9.0451 us/op 10.682 us/op 0.85
Set add up to 128 items then delete first 12.911 us/op 12.387 us/op 1.04
OrderedSet add up to 128 items then delete first 16.092 us/op 16.667 us/op 0.97
Set add up to 128 items then delete last 11.440 us/op 11.479 us/op 1.00
OrderedSet add up to 128 items then delete last 15.754 us/op 19.986 us/op 0.79
Set add up to 128 items then delete middle 11.088 us/op 12.675 us/op 0.87
OrderedSet add up to 128 items then delete middle 20.790 us/op 27.400 us/op 0.76
Set add up to 256 items then delete first 21.863 us/op 25.894 us/op 0.84
OrderedSet add up to 256 items then delete first 28.620 us/op 34.277 us/op 0.83
Set add up to 256 items then delete last 20.572 us/op 25.513 us/op 0.81
OrderedSet add up to 256 items then delete last 28.730 us/op 42.812 us/op 0.67
Set add up to 256 items then delete middle 21.417 us/op 24.615 us/op 0.87
OrderedSet add up to 256 items then delete middle 52.760 us/op 66.267 us/op 0.80
transfer serialized Status (84 B) 1.8570 us/op 2.5780 us/op 0.72
copy serialized Status (84 B) 1.7100 us/op 2.1110 us/op 0.81
transfer serialized SignedVoluntaryExit (112 B) 1.9990 us/op 2.5620 us/op 0.78
copy serialized SignedVoluntaryExit (112 B) 1.7230 us/op 2.3970 us/op 0.72
transfer serialized ProposerSlashing (416 B) 2.3100 us/op 3.2320 us/op 0.71
copy serialized ProposerSlashing (416 B) 2.3430 us/op 3.3670 us/op 0.70
transfer serialized Attestation (485 B) 2.3330 us/op 3.8990 us/op 0.60
copy serialized Attestation (485 B) 2.3700 us/op 3.7870 us/op 0.63
transfer serialized AttesterSlashing (33232 B) 3.0750 us/op 5.5450 us/op 0.55
copy serialized AttesterSlashing (33232 B) 10.255 us/op 12.540 us/op 0.82
transfer serialized Small SignedBeaconBlock (128000 B) 3.8370 us/op 4.6560 us/op 0.82
copy serialized Small SignedBeaconBlock (128000 B) 28.783 us/op 40.181 us/op 0.72
transfer serialized Avg SignedBeaconBlock (200000 B) 4.3090 us/op 5.7870 us/op 0.74
copy serialized Avg SignedBeaconBlock (200000 B) 46.254 us/op 53.403 us/op 0.87
transfer serialized BlobsSidecar (524380 B) 5.4800 us/op 6.2430 us/op 0.88
copy serialized BlobsSidecar (524380 B) 133.33 us/op 336.52 us/op 0.40
transfer serialized Big SignedBeaconBlock (1000000 B) 4.4980 us/op 14.067 us/op 0.32
copy serialized Big SignedBeaconBlock (1000000 B) 231.40 us/op 492.89 us/op 0.47
pass gossip attestations to forkchoice per slot 4.8492 ms/op 5.1405 ms/op 0.94
forkChoice updateHead vc 100000 bc 64 eq 0 706.46 us/op 1.0162 ms/op 0.70
forkChoice updateHead vc 600000 bc 64 eq 0 6.5358 ms/op 8.6248 ms/op 0.76
forkChoice updateHead vc 1000000 bc 64 eq 0 8.0609 ms/op 11.635 ms/op 0.69
forkChoice updateHead vc 600000 bc 320 eq 0 4.5243 ms/op 5.4436 ms/op 0.83
forkChoice updateHead vc 600000 bc 1200 eq 0 4.4831 ms/op 7.2623 ms/op 0.62
forkChoice updateHead vc 600000 bc 7200 eq 0 6.3524 ms/op 7.7341 ms/op 0.82
forkChoice updateHead vc 600000 bc 64 eq 1000 11.508 ms/op 15.114 ms/op 0.76
forkChoice updateHead vc 600000 bc 64 eq 10000 12.248 ms/op 16.043 ms/op 0.76
forkChoice updateHead vc 600000 bc 64 eq 300000 18.987 ms/op 62.507 ms/op 0.30
computeDeltas 500000 validators 300 proto nodes 7.0610 ms/op 8.3379 ms/op 0.85
computeDeltas 500000 validators 1200 proto nodes 6.8132 ms/op 8.4451 ms/op 0.81
computeDeltas 500000 validators 7200 proto nodes 6.7614 ms/op 8.1976 ms/op 0.82
computeDeltas 750000 validators 300 proto nodes 10.174 ms/op 12.277 ms/op 0.83
computeDeltas 750000 validators 1200 proto nodes 10.807 ms/op 12.609 ms/op 0.86
computeDeltas 750000 validators 7200 proto nodes 10.885 ms/op 11.994 ms/op 0.91
computeDeltas 1400000 validators 300 proto nodes 21.083 ms/op 23.369 ms/op 0.90
computeDeltas 1400000 validators 1200 proto nodes 18.759 ms/op 24.034 ms/op 0.78
computeDeltas 1400000 validators 7200 proto nodes 20.316 ms/op 24.347 ms/op 0.83
computeDeltas 2100000 validators 300 proto nodes 28.933 ms/op 32.664 ms/op 0.89
computeDeltas 2100000 validators 1200 proto nodes 28.900 ms/op 32.968 ms/op 0.88
computeDeltas 2100000 validators 7200 proto nodes 28.847 ms/op 34.412 ms/op 0.84
computeProposerBoostScoreFromBalances 500000 validators 3.7593 ms/op 4.5806 ms/op 0.82
computeProposerBoostScoreFromBalances 750000 validators 3.7357 ms/op 4.6154 ms/op 0.81
computeProposerBoostScoreFromBalances 1400000 validators 3.7527 ms/op 4.7396 ms/op 0.79
computeProposerBoostScoreFromBalances 2100000 validators 3.7042 ms/op 4.5402 ms/op 0.82
altair processAttestation - 250000 vs - 7PWei normalcase 2.1798 ms/op 5.0228 ms/op 0.43
altair processAttestation - 250000 vs - 7PWei worstcase 3.4855 ms/op 6.1000 ms/op 0.57
altair processAttestation - setStatus - 1/6 committees join 137.00 us/op 281.64 us/op 0.49
altair processAttestation - setStatus - 1/3 committees join 266.65 us/op 459.03 us/op 0.58
altair processAttestation - setStatus - 1/2 committees join 353.86 us/op 624.70 us/op 0.57
altair processAttestation - setStatus - 2/3 committees join 448.56 us/op 873.25 us/op 0.51
altair processAttestation - setStatus - 4/5 committees join 625.28 us/op 1.1522 ms/op 0.54
altair processAttestation - setStatus - 100% committees join 752.34 us/op 1.1969 ms/op 0.63
altair processBlock - 250000 vs - 7PWei normalcase 9.8635 ms/op 17.274 ms/op 0.57
altair processBlock - 250000 vs - 7PWei normalcase hashState 38.489 ms/op 57.265 ms/op 0.67
altair processBlock - 250000 vs - 7PWei worstcase 41.024 ms/op 58.490 ms/op 0.70
altair processBlock - 250000 vs - 7PWei worstcase hashState 92.515 ms/op 139.63 ms/op 0.66
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2317 ms/op 5.4960 ms/op 0.59
phase0 processBlock - 250000 vs - 7PWei worstcase 30.492 ms/op 40.786 ms/op 0.75
altair processEth1Data - 250000 vs - 7PWei normalcase 517.33 us/op 1.0833 ms/op 0.48
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 17.974 us/op 26.646 us/op 0.67
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 70.286 us/op 126.28 us/op 0.56
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 13.315 us/op 31.306 us/op 0.43
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 12.342 us/op 27.636 us/op 0.45
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 199.45 us/op 235.91 us/op 0.85
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.3483 ms/op 2.9163 ms/op 0.46
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.8802 ms/op 3.2881 ms/op 0.57
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.0935 ms/op 2.8855 ms/op 0.73
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.3577 ms/op 5.7392 ms/op 0.59
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.7433 ms/op 3.4394 ms/op 0.80
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.7833 ms/op 10.660 ms/op 0.54
Tree 40 250000 create 400.19 ms/op 855.15 ms/op 0.47
Tree 40 250000 get(125000) 208.39 ns/op 230.35 ns/op 0.90
Tree 40 250000 set(125000) 1.1192 us/op 1.3870 us/op 0.81
Tree 40 250000 toArray() 23.945 ms/op 24.882 ms/op 0.96
Tree 40 250000 iterate all - toArray() + loop 21.700 ms/op 28.085 ms/op 0.77
Tree 40 250000 iterate all - get(i) 72.296 ms/op 82.785 ms/op 0.87
MutableVector 250000 create 16.904 ms/op 22.605 ms/op 0.75
MutableVector 250000 get(125000) 6.5330 ns/op 7.0330 ns/op 0.93
MutableVector 250000 set(125000) 290.60 ns/op 438.14 ns/op 0.66
MutableVector 250000 toArray() 3.8934 ms/op 5.6022 ms/op 0.69
MutableVector 250000 iterate all - toArray() + loop 4.2502 ms/op 5.6824 ms/op 0.75
MutableVector 250000 iterate all - get(i) 1.5400 ms/op 1.7159 ms/op 0.90
Array 250000 create 3.4377 ms/op 6.0469 ms/op 0.57
Array 250000 clone - spread 1.7060 ms/op 2.4923 ms/op 0.68
Array 250000 get(125000) 1.0720 ns/op 2.4120 ns/op 0.44
Array 250000 set(125000) 4.1950 ns/op 6.1790 ns/op 0.68
Array 250000 iterate all - loop 165.72 us/op 198.24 us/op 0.84
effectiveBalanceIncrements clone Uint8Array 300000 31.935 us/op 83.049 us/op 0.38
effectiveBalanceIncrements clone MutableVector 300000 351.00 ns/op 538.00 ns/op 0.65
effectiveBalanceIncrements rw all Uint8Array 300000 200.32 us/op 209.70 us/op 0.96
effectiveBalanceIncrements rw all MutableVector 300000 83.857 ms/op 140.29 ms/op 0.60
phase0 afterProcessEpoch - 250000 vs - 7PWei 120.94 ms/op 126.02 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 50.692 ms/op 61.049 ms/op 0.83
altair processEpoch - mainnet_e81889 499.91 ms/op 616.40 ms/op 0.81
mainnet_e81889 - altair beforeProcessEpoch 83.493 ms/op 123.82 ms/op 0.67
mainnet_e81889 - altair processJustificationAndFinalization 13.602 us/op 26.018 us/op 0.52
mainnet_e81889 - altair processInactivityUpdates 5.8921 ms/op 8.1916 ms/op 0.72
mainnet_e81889 - altair processRewardsAndPenalties 63.813 ms/op 74.802 ms/op 0.85
mainnet_e81889 - altair processRegistryUpdates 2.6490 us/op 3.7960 us/op 0.70
mainnet_e81889 - altair processSlashings 451.00 ns/op 727.00 ns/op 0.62
mainnet_e81889 - altair processEth1DataReset 433.00 ns/op 935.00 ns/op 0.46
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4487 ms/op 1.9928 ms/op 0.73
mainnet_e81889 - altair processSlashingsReset 3.4650 us/op 9.6330 us/op 0.36
mainnet_e81889 - altair processRandaoMixesReset 4.7880 us/op 8.3430 us/op 0.57
mainnet_e81889 - altair processHistoricalRootsUpdate 830.00 ns/op 882.00 ns/op 0.94
mainnet_e81889 - altair processParticipationFlagUpdates 3.3440 us/op 2.6210 us/op 1.28
mainnet_e81889 - altair processSyncCommitteeUpdates 589.00 ns/op 1.0270 us/op 0.57
mainnet_e81889 - altair afterProcessEpoch 119.02 ms/op 127.13 ms/op 0.94
capella processEpoch - mainnet_e217614 2.6733 s/op 2.5535 s/op 1.05
mainnet_e217614 - capella beforeProcessEpoch 556.91 ms/op 540.00 ms/op 1.03
mainnet_e217614 - capella processJustificationAndFinalization 44.801 us/op 30.922 us/op 1.45
mainnet_e217614 - capella processInactivityUpdates 41.646 ms/op 28.395 ms/op 1.47
mainnet_e217614 - capella processRewardsAndPenalties 509.30 ms/op 463.85 ms/op 1.10
mainnet_e217614 - capella processRegistryUpdates 29.443 us/op 30.967 us/op 0.95
mainnet_e217614 - capella processSlashings 466.00 ns/op 814.00 ns/op 0.57
mainnet_e217614 - capella processEth1DataReset 433.00 ns/op 714.00 ns/op 0.61
mainnet_e217614 - capella processEffectiveBalanceUpdates 5.3711 ms/op 6.3268 ms/op 0.85
mainnet_e217614 - capella processSlashingsReset 4.2020 us/op 6.4720 us/op 0.65
mainnet_e217614 - capella processRandaoMixesReset 5.0870 us/op 7.5620 us/op 0.67
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0260 us/op 1.1990 us/op 0.86
mainnet_e217614 - capella processParticipationFlagUpdates 1.6350 us/op 4.3360 us/op 0.38
mainnet_e217614 - capella afterProcessEpoch 327.78 ms/op 371.76 ms/op 0.88
phase0 processEpoch - mainnet_e58758 441.22 ms/op 739.45 ms/op 0.60
mainnet_e58758 - phase0 beforeProcessEpoch 145.31 ms/op 203.35 ms/op 0.71
mainnet_e58758 - phase0 processJustificationAndFinalization 19.404 us/op 32.653 us/op 0.59
mainnet_e58758 - phase0 processRewardsAndPenalties 58.767 ms/op 64.884 ms/op 0.91
mainnet_e58758 - phase0 processRegistryUpdates 12.235 us/op 25.976 us/op 0.47
mainnet_e58758 - phase0 processSlashings 614.00 ns/op 1.0570 us/op 0.58
mainnet_e58758 - phase0 processEth1DataReset 441.00 ns/op 1.1170 us/op 0.39
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.1545 ms/op 1.6747 ms/op 0.69
mainnet_e58758 - phase0 processSlashingsReset 4.8780 us/op 7.5670 us/op 0.64
mainnet_e58758 - phase0 processRandaoMixesReset 4.9370 us/op 9.4380 us/op 0.52
mainnet_e58758 - phase0 processHistoricalRootsUpdate 426.00 ns/op 1.1370 us/op 0.37
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.8220 us/op 9.3040 us/op 0.41
mainnet_e58758 - phase0 afterProcessEpoch 95.143 ms/op 105.97 ms/op 0.90
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4400 ms/op 2.1864 ms/op 0.66
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4771 ms/op 2.5729 ms/op 0.57
altair processInactivityUpdates - 250000 normalcase 30.138 ms/op 39.608 ms/op 0.76
altair processInactivityUpdates - 250000 worstcase 26.934 ms/op 50.352 ms/op 0.53
phase0 processRegistryUpdates - 250000 normalcase 8.6520 us/op 20.662 us/op 0.42
phase0 processRegistryUpdates - 250000 badcase_full_deposits 402.67 us/op 580.12 us/op 0.69
phase0 processRegistryUpdates - 250000 worstcase 0.5 134.92 ms/op 183.57 ms/op 0.73
altair processRewardsAndPenalties - 250000 normalcase 45.540 ms/op 68.748 ms/op 0.66
altair processRewardsAndPenalties - 250000 worstcase 42.922 ms/op 84.115 ms/op 0.51
phase0 getAttestationDeltas - 250000 normalcase 8.8430 ms/op 14.683 ms/op 0.60
phase0 getAttestationDeltas - 250000 worstcase 8.8634 ms/op 13.037 ms/op 0.68
phase0 processSlashings - 250000 worstcase 101.28 us/op 126.56 us/op 0.80
altair processSyncCommitteeUpdates - 250000 165.56 ms/op 213.77 ms/op 0.77
BeaconState.hashTreeRoot - No change 299.00 ns/op 303.00 ns/op 0.99
BeaconState.hashTreeRoot - 1 full validator 158.02 us/op 178.64 us/op 0.88
BeaconState.hashTreeRoot - 32 full validator 1.3551 ms/op 1.9040 ms/op 0.71
BeaconState.hashTreeRoot - 512 full validator 13.646 ms/op 22.272 ms/op 0.61
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 164.45 us/op 245.27 us/op 0.67
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.0361 ms/op 3.0218 ms/op 0.67
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.543 ms/op 37.984 ms/op 0.75
BeaconState.hashTreeRoot - 1 balances 140.07 us/op 166.06 us/op 0.84
BeaconState.hashTreeRoot - 32 balances 1.2581 ms/op 1.5801 ms/op 0.80
BeaconState.hashTreeRoot - 512 balances 11.874 ms/op 15.919 ms/op 0.75
BeaconState.hashTreeRoot - 250000 balances 220.37 ms/op 291.74 ms/op 0.76
aggregationBits - 2048 els - zipIndexesInBitList 19.361 us/op 34.288 us/op 0.56
byteArrayEquals 32 74.867 ns/op 90.774 ns/op 0.82
Buffer.compare 32 56.051 ns/op 58.936 ns/op 0.95
byteArrayEquals 1024 2.0438 us/op 2.1640 us/op 0.94
Buffer.compare 1024 70.492 ns/op 81.118 ns/op 0.87
byteArrayEquals 16384 32.657 us/op 37.212 us/op 0.88
Buffer.compare 16384 272.55 ns/op 317.16 ns/op 0.86
byteArrayEquals 123687377 246.80 ms/op 285.15 ms/op 0.87
Buffer.compare 123687377 7.8242 ms/op 8.4471 ms/op 0.93
byteArrayEquals 32 - diff last byte 72.201 ns/op 83.331 ns/op 0.87
Buffer.compare 32 - diff last byte 56.460 ns/op 60.469 ns/op 0.93
byteArrayEquals 1024 - diff last byte 2.0417 us/op 2.1362 us/op 0.96
Buffer.compare 1024 - diff last byte 71.200 ns/op 76.499 ns/op 0.93
byteArrayEquals 16384 - diff last byte 33.091 us/op 36.493 us/op 0.91
Buffer.compare 16384 - diff last byte 261.52 ns/op 318.87 ns/op 0.82
byteArrayEquals 123687377 - diff last byte 244.95 ms/op 296.69 ms/op 0.83
Buffer.compare 123687377 - diff last byte 7.6045 ms/op 9.1159 ms/op 0.83
byteArrayEquals 32 - random bytes 5.4540 ns/op 7.1290 ns/op 0.77
Buffer.compare 32 - random bytes 61.573 ns/op 70.094 ns/op 0.88
byteArrayEquals 1024 - random bytes 5.5580 ns/op 6.1880 ns/op 0.90
Buffer.compare 1024 - random bytes 60.747 ns/op 66.605 ns/op 0.91
byteArrayEquals 16384 - random bytes 5.6540 ns/op 6.3000 ns/op 0.90
Buffer.compare 16384 - random bytes 63.050 ns/op 69.270 ns/op 0.91
byteArrayEquals 123687377 - random bytes 8.8000 ns/op 9.2700 ns/op 0.95
Buffer.compare 123687377 - random bytes 70.470 ns/op 74.540 ns/op 0.95
regular array get 100000 times 45.289 us/op 49.394 us/op 0.92
wrappedArray get 100000 times 45.208 us/op 48.543 us/op 0.93
arrayWithProxy get 100000 times 14.005 ms/op 15.502 ms/op 0.90
ssz.Root.equals 54.232 ns/op 59.572 ns/op 0.91
byteArrayEquals 53.940 ns/op 58.565 ns/op 0.92
Buffer.compare 11.864 ns/op 12.540 ns/op 0.95
shuffle list - 16384 els 7.0163 ms/op 7.3903 ms/op 0.95
shuffle list - 250000 els 102.39 ms/op 108.71 ms/op 0.94
processSlot - 1 slots 17.242 us/op 19.208 us/op 0.90
processSlot - 32 slots 3.2918 ms/op 4.5259 ms/op 0.73
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 58.915 ms/op 60.510 ms/op 0.97
getCommitteeAssignments - req 1 vs - 250000 vc 2.4603 ms/op 2.7159 ms/op 0.91
getCommitteeAssignments - req 100 vs - 250000 vc 3.6770 ms/op 4.0714 ms/op 0.90
getCommitteeAssignments - req 1000 vs - 250000 vc 4.0504 ms/op 4.3140 ms/op 0.94
findModifiedValidators - 10000 modified validators 529.02 ms/op 573.92 ms/op 0.92
findModifiedValidators - 1000 modified validators 429.67 ms/op 447.02 ms/op 0.96
findModifiedValidators - 100 modified validators 417.14 ms/op 458.15 ms/op 0.91
findModifiedValidators - 10 modified validators 403.86 ms/op 463.46 ms/op 0.87
findModifiedValidators - 1 modified validators 401.87 ms/op 485.65 ms/op 0.83
findModifiedValidators - no difference 399.74 ms/op 455.52 ms/op 0.88
compare ViewDUs 4.5709 s/op 5.1169 s/op 0.89
compare each validator Uint8Array 1.3049 s/op 1.9278 s/op 0.68
compare ViewDU to Uint8Array 1.1539 s/op 1.4412 s/op 0.80
migrate state 1000000 validators, 24 modified, 0 new 744.87 ms/op 903.31 ms/op 0.82
migrate state 1000000 validators, 1700 modified, 1000 new 1.0388 s/op 1.2271 s/op 0.85
migrate state 1000000 validators, 3400 modified, 2000 new 1.2742 s/op 1.5579 s/op 0.82
migrate state 1500000 validators, 24 modified, 0 new 798.58 ms/op 1.0198 s/op 0.78
migrate state 1500000 validators, 1700 modified, 1000 new 1.0440 s/op 1.4855 s/op 0.70
migrate state 1500000 validators, 3400 modified, 2000 new 1.2951 s/op 1.9378 s/op 0.67
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.5000 ns/op 6.4800 ns/op 0.69
state getBlockRootAtSlot - 250000 vs - 7PWei 880.25 ns/op 819.92 ns/op 1.07
computeProposers - vc 250000 9.6619 ms/op 11.624 ms/op 0.83
computeEpochShuffling - vc 250000 104.70 ms/op 119.66 ms/op 0.87
getNextSyncCommittee - vc 250000 165.50 ms/op 225.91 ms/op 0.73
computeSigningRoot for AttestationData 24.760 us/op 33.525 us/op 0.74
hash AttestationData serialized data then Buffer.toString(base64) 2.3366 us/op 2.7060 us/op 0.86
toHexString serialized data 1.1171 us/op 1.5569 us/op 0.72
Buffer.toString(base64) 234.98 ns/op 255.00 ns/op 0.92

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 912ec09 into unstable Jan 17, 2024
14 of 15 checks passed
@wemeetagain wemeetagain deleted the nh/fix-sim-tests branch January 17, 2024 18:48
@wemeetagain wemeetagain mentioned this pull request Jan 19, 2024
Closed
ensi321 pushed a commit to ensi321/lodestar that referenced this pull request Jan 22, 2024
@nazarhussain @ensi321
test: stabilize sim tests (ChainSafe#6312)
0727d53 
* Fix sim test genesis delay

* Update the defaults for the all sim tests

* Increase the limit for slotImportTolerance for unknown block sync

* Update the additional slots for TTD

* Adjust the ttd time

* Adjust the ttd

* Adjust the ttd

* Update the ttd calculation

* Add timestamp to report

* Update the slot to check the merge transition

* Update ttd calculations

* Increase the genesis delay

* Update the shangai ttd calculation

* Increase the slot tolerance

* Disable broadcast validation for unknown block tests

* Fix lint errors
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.15.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nazarhussain @wemeetagain