Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement blob sidecars with inclusion proof instead of signtaures #6089

Merged
merged 1 commit into from
Dec 25, 2023
Merged

feat: implement blob sidecars with inclusion proof instead of signtaures #6089

merged 1 commit into from
Dec 25, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Nov 6, 2023

@g11tech g11tech requested a review from a team as a code owner November 6, 2023 11:22
@g11tech g11tech mentioned this pull request Nov 6, 2023
Closed
40 tasks
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 20, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9e2d8cd Previous: 61cf1a8 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 715.96 us/op 742.51 us/op 0.96
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 82.255 us/op 83.202 us/op 0.99
BLS verify - blst-native 1.2310 ms/op 1.2916 ms/op 0.95
BLS verifyMultipleSignatures 3 - blst-native 2.5843 ms/op 2.7098 ms/op 0.95
BLS verifyMultipleSignatures 8 - blst-native 5.6742 ms/op 5.9470 ms/op 0.95
BLS verifyMultipleSignatures 32 - blst-native 21.439 ms/op 21.868 ms/op 0.98
BLS verifyMultipleSignatures 64 - blst-native 42.471 ms/op 43.092 ms/op 0.99
BLS verifyMultipleSignatures 128 - blst-native 84.320 ms/op 85.271 ms/op 0.99
BLS deserializing 10000 signatures 895.74 ms/op 926.05 ms/op 0.97
BLS deserializing 100000 signatures 9.0765 s/op 9.2364 s/op 0.98
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.2538 ms/op 1.2979 ms/op 0.97
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.4154 ms/op 1.5787 ms/op 0.90
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.1857 ms/op 2.8037 ms/op 0.78
BLS verifyMultipleSignatures - same message - 64 - blst-native 4.3605 ms/op 4.1494 ms/op 1.05
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.6516 ms/op 7.0970 ms/op 0.80
BLS aggregatePubkeys 32 - blst-native 25.669 us/op 26.028 us/op 0.99
BLS aggregatePubkeys 128 - blst-native 99.296 us/op 97.411 us/op 1.02
getAttestationsForBlock 43.263 ms/op 42.102 ms/op 1.03
getSlashingsAndExits - default max 149.24 us/op 137.60 us/op 1.08
getSlashingsAndExits - 2k 388.16 us/op 387.29 us/op 1.00
proposeBlockBody type=full, size=empty 4.9895 ms/op 5.1376 ms/op 0.97
isKnown best case - 1 super set check 297.00 ns/op 300.00 ns/op 0.99
isKnown normal case - 2 super set checks 286.00 ns/op 298.00 ns/op 0.96
isKnown worse case - 16 super set checks 281.00 ns/op 295.00 ns/op 0.95
CheckpointStateCache - add get delete 4.8810 us/op 4.9950 us/op 0.98
validate api signedAggregateAndProof - struct 2.6764 ms/op 2.7412 ms/op 0.98
validate gossip signedAggregateAndProof - struct 2.6718 ms/op 2.7427 ms/op 0.97
validate gossip attestation - vc 640000 1.3132 ms/op 1.3377 ms/op 0.98
batch validate gossip attestation - vc 640000 - chunk 32 156.85 us/op 158.14 us/op 0.99
batch validate gossip attestation - vc 640000 - chunk 64 136.14 us/op 140.19 us/op 0.97
batch validate gossip attestation - vc 640000 - chunk 128 128.26 us/op 129.45 us/op 0.99
batch validate gossip attestation - vc 640000 - chunk 256 127.00 us/op 126.10 us/op 1.01
pickEth1Vote - no votes 1.1185 ms/op 1.1660 ms/op 0.96
pickEth1Vote - max votes 9.7141 ms/op 9.8745 ms/op 0.98
pickEth1Vote - Eth1Data hashTreeRoot value x2048 18.227 ms/op 20.884 ms/op 0.87
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 26.543 ms/op 28.761 ms/op 0.92
pickEth1Vote - Eth1Data fastSerialize value x2048 543.71 us/op 664.99 us/op 0.82
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.1798 ms/op 6.9981 ms/op 1.03
bytes32 toHexString 529.00 ns/op 508.00 ns/op 1.04
bytes32 Buffer.toString(hex) 284.00 ns/op 285.00 ns/op 1.00
bytes32 Buffer.toString(hex) from Uint8Array 418.00 ns/op 417.00 ns/op 1.00
bytes32 Buffer.toString(hex) + 0x 278.00 ns/op 292.00 ns/op 0.95
Object access 1 prop 0.15600 ns/op 0.16400 ns/op 0.95
Map access 1 prop 0.13600 ns/op 0.14600 ns/op 0.93
Object get x1000 7.0750 ns/op 7.1930 ns/op 0.98
Map get x1000 0.75400 ns/op 0.75800 ns/op 0.99
Object set x1000 51.545 ns/op 53.193 ns/op 0.97
Map set x1000 38.844 ns/op 39.913 ns/op 0.97
Return object 10000 times 0.23470 ns/op 0.23670 ns/op 0.99
Throw Error 10000 times 3.8552 us/op 3.8538 us/op 1.00
fastMsgIdFn sha256 / 200 bytes 3.2360 us/op 3.2380 us/op 1.00
fastMsgIdFn h32 xxhash / 200 bytes 273.00 ns/op 282.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 200 bytes 337.00 ns/op 455.00 ns/op 0.74
fastMsgIdFn sha256 / 1000 bytes 11.323 us/op 11.333 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 412.00 ns/op 470.00 ns/op 0.88
fastMsgIdFn h64 xxhash / 1000 bytes 417.00 ns/op 454.00 ns/op 0.92
fastMsgIdFn sha256 / 10000 bytes 102.62 us/op 101.96 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 1.9500 us/op 1.9450 us/op 1.00
fastMsgIdFn h64 xxhash / 10000 bytes 1.3500 us/op 1.3280 us/op 1.02
send data - 1000 256B messages 18.315 ms/op 17.737 ms/op 1.03
send data - 1000 512B messages 25.854 ms/op 25.776 ms/op 1.00
send data - 1000 1024B messages 41.035 ms/op 39.561 ms/op 1.04
send data - 1000 1200B messages 22.406 ms/op 38.307 ms/op 0.58
send data - 1000 2048B messages 42.865 ms/op 44.034 ms/op 0.97
send data - 1000 4096B messages 42.631 ms/op 40.539 ms/op 1.05
send data - 1000 16384B messages 115.79 ms/op 116.22 ms/op 1.00
send data - 1000 65536B messages 442.78 ms/op 468.96 ms/op 0.94
enrSubnets - fastDeserialize 64 bits 1.2380 us/op 1.2340 us/op 1.00
enrSubnets - ssz BitVector 64 bits 404.00 ns/op 412.00 ns/op 0.98
enrSubnets - fastDeserialize 4 bits 166.00 ns/op 164.00 ns/op 1.01
enrSubnets - ssz BitVector 4 bits 409.00 ns/op 413.00 ns/op 0.99
prioritizePeers score -10:0 att 32-0.1 sync 2-0 96.769 us/op 101.22 us/op 0.96
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 121.34 us/op 123.14 us/op 0.99
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 150.34 us/op 157.73 us/op 0.95
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 274.34 us/op 283.74 us/op 0.97
prioritizePeers score 0:0 att 64-1 sync 4-1 318.35 us/op 335.56 us/op 0.95
array of 16000 items push then shift 1.5173 us/op 1.6047 us/op 0.95
LinkedList of 16000 items push then shift 8.4450 ns/op 8.8490 ns/op 0.95
array of 16000 items push then pop 66.411 ns/op 66.981 ns/op 0.99
LinkedList of 16000 items push then pop 8.2620 ns/op 8.6580 ns/op 0.95
array of 24000 items push then shift 2.2387 us/op 2.3616 us/op 0.95
LinkedList of 24000 items push then shift 8.4490 ns/op 8.8000 ns/op 0.96
array of 24000 items push then pop 95.170 ns/op 101.59 ns/op 0.94
LinkedList of 24000 items push then pop 8.4870 ns/op 8.5600 ns/op 0.99
intersect bitArray bitLen 8 6.3270 ns/op 6.3680 ns/op 0.99
intersect array and set length 8 61.352 ns/op 61.273 ns/op 1.00
intersect bitArray bitLen 128 33.216 ns/op 33.768 ns/op 0.98
intersect array and set length 128 843.29 ns/op 844.35 ns/op 1.00
bitArray.getTrueBitIndexes() bitLen 128 1.4060 us/op 1.3590 us/op 1.03
bitArray.getTrueBitIndexes() bitLen 248 2.3490 us/op 2.3820 us/op 0.99
bitArray.getTrueBitIndexes() bitLen 512 4.4630 us/op 4.7020 us/op 0.95
Buffer.concat 32 items 928.00 ns/op 972.00 ns/op 0.95
Uint8Array.set 32 items 1.7790 us/op 1.6850 us/op 1.06
Set add up to 64 items then delete first 4.0411 us/op 4.2453 us/op 0.95
OrderedSet add up to 64 items then delete first 5.1110 us/op 5.3630 us/op 0.95
Set add up to 64 items then delete last 4.5820 us/op 4.5401 us/op 1.01
OrderedSet add up to 64 items then delete last 5.7321 us/op 5.6927 us/op 1.01
Set add up to 64 items then delete middle 4.5702 us/op 4.5136 us/op 1.01
OrderedSet add up to 64 items then delete middle 6.9615 us/op 6.9056 us/op 1.01
Set add up to 128 items then delete first 8.8941 us/op 9.1905 us/op 0.97
OrderedSet add up to 128 items then delete first 11.569 us/op 12.177 us/op 0.95
Set add up to 128 items then delete last 8.5570 us/op 9.0209 us/op 0.95
OrderedSet add up to 128 items then delete last 10.963 us/op 11.431 us/op 0.96
Set add up to 128 items then delete middle 8.5440 us/op 8.9683 us/op 0.95
OrderedSet add up to 128 items then delete middle 15.911 us/op 16.654 us/op 0.96
Set add up to 256 items then delete first 17.705 us/op 18.520 us/op 0.96
OrderedSet add up to 256 items then delete first 23.729 us/op 24.915 us/op 0.95
Set add up to 256 items then delete last 17.109 us/op 17.943 us/op 0.95
OrderedSet add up to 256 items then delete last 22.031 us/op 23.189 us/op 0.95
Set add up to 256 items then delete middle 17.052 us/op 17.976 us/op 0.95
OrderedSet add up to 256 items then delete middle 42.399 us/op 45.657 us/op 0.93
transfer serialized Status (84 B) 1.7010 us/op 1.7480 us/op 0.97
copy serialized Status (84 B) 1.4140 us/op 1.4380 us/op 0.98
transfer serialized SignedVoluntaryExit (112 B) 1.9230 us/op 1.8830 us/op 1.02
copy serialized SignedVoluntaryExit (112 B) 1.4900 us/op 1.4840 us/op 1.00
transfer serialized ProposerSlashing (416 B) 2.3260 us/op 2.3630 us/op 0.98
copy serialized ProposerSlashing (416 B) 2.1430 us/op 2.0530 us/op 1.04
transfer serialized Attestation (485 B) 2.3160 us/op 2.3660 us/op 0.98
copy serialized Attestation (485 B) 2.1670 us/op 2.0690 us/op 1.05
transfer serialized AttesterSlashing (33232 B) 2.5520 us/op 2.4470 us/op 1.04
copy serialized AttesterSlashing (33232 B) 5.8000 us/op 5.3730 us/op 1.08
transfer serialized Small SignedBeaconBlock (128000 B) 2.8270 us/op 2.6890 us/op 1.05
copy serialized Small SignedBeaconBlock (128000 B) 16.159 us/op 13.655 us/op 1.18
transfer serialized Avg SignedBeaconBlock (200000 B) 3.4270 us/op 3.1200 us/op 1.10
copy serialized Avg SignedBeaconBlock (200000 B) 21.418 us/op 20.003 us/op 1.07
transfer serialized BlobsSidecar (524380 B) 3.3700 us/op 3.0490 us/op 1.11
copy serialized BlobsSidecar (524380 B) 82.024 us/op 90.258 us/op 0.91
transfer serialized Big SignedBeaconBlock (1000000 B) 3.4170 us/op 3.1020 us/op 1.10
copy serialized Big SignedBeaconBlock (1000000 B) 175.80 us/op 164.41 us/op 1.07
pass gossip attestations to forkchoice per slot 4.0933 ms/op 3.8611 ms/op 1.06
forkChoice updateHead vc 100000 bc 64 eq 0 658.35 us/op 680.17 us/op 0.97
forkChoice updateHead vc 600000 bc 64 eq 0 4.5286 ms/op 4.7760 ms/op 0.95
forkChoice updateHead vc 1000000 bc 64 eq 0 6.7460 ms/op 7.1404 ms/op 0.94
forkChoice updateHead vc 600000 bc 320 eq 0 4.0005 ms/op 4.1705 ms/op 0.96
forkChoice updateHead vc 600000 bc 1200 eq 0 4.2912 ms/op 4.2588 ms/op 1.01
forkChoice updateHead vc 600000 bc 7200 eq 0 5.0200 ms/op 5.2611 ms/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 1000 11.063 ms/op 10.907 ms/op 1.01
forkChoice updateHead vc 600000 bc 64 eq 10000 11.840 ms/op 11.830 ms/op 1.00
forkChoice updateHead vc 600000 bc 64 eq 300000 16.124 ms/op 15.667 ms/op 1.03
computeDeltas 500000 validators 300 proto nodes 6.5122 ms/op 6.5118 ms/op 1.00
computeDeltas 500000 validators 1200 proto nodes 6.3865 ms/op 6.5308 ms/op 0.98
computeDeltas 500000 validators 7200 proto nodes 6.2764 ms/op 6.3836 ms/op 0.98
computeDeltas 750000 validators 300 proto nodes 9.5735 ms/op 9.6910 ms/op 0.99
computeDeltas 750000 validators 1200 proto nodes 9.4459 ms/op 9.5638 ms/op 0.99
computeDeltas 750000 validators 7200 proto nodes 9.4524 ms/op 9.3827 ms/op 1.01
computeDeltas 1400000 validators 300 proto nodes 18.409 ms/op 18.789 ms/op 0.98
computeDeltas 1400000 validators 1200 proto nodes 18.250 ms/op 18.481 ms/op 0.99
computeDeltas 1400000 validators 7200 proto nodes 18.858 ms/op 18.356 ms/op 1.03
computeDeltas 2100000 validators 300 proto nodes 28.082 ms/op 28.105 ms/op 1.00
computeDeltas 2100000 validators 1200 proto nodes 27.549 ms/op 28.680 ms/op 0.96
computeDeltas 2100000 validators 7200 proto nodes 27.359 ms/op 28.460 ms/op 0.96
computeProposerBoostScoreFromBalances 500000 validators 3.5740 ms/op 3.6585 ms/op 0.98
computeProposerBoostScoreFromBalances 750000 validators 3.5241 ms/op 3.6459 ms/op 0.97
computeProposerBoostScoreFromBalances 1400000 validators 3.6095 ms/op 3.6861 ms/op 0.98
computeProposerBoostScoreFromBalances 2100000 validators 3.6766 ms/op 3.6803 ms/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 2.2530 ms/op 2.1281 ms/op 1.06
altair processAttestation - 250000 vs - 7PWei worstcase 3.3210 ms/op 3.1053 ms/op 1.07
altair processAttestation - setStatus - 1/6 committees join 175.06 us/op 178.28 us/op 0.98
altair processAttestation - setStatus - 1/3 committees join 341.03 us/op 343.09 us/op 0.99
altair processAttestation - setStatus - 1/2 committees join 439.96 us/op 465.62 us/op 0.94
altair processAttestation - setStatus - 2/3 committees join 557.23 us/op 582.40 us/op 0.96
altair processAttestation - setStatus - 4/5 committees join 786.03 us/op 769.99 us/op 1.02
altair processAttestation - setStatus - 100% committees join 925.39 us/op 917.22 us/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 10.692 ms/op 9.4017 ms/op 1.14
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.679 ms/op 43.499 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase 36.774 ms/op 36.897 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase hashState 93.016 ms/op 97.205 ms/op 0.96
phase0 processBlock - 250000 vs - 7PWei normalcase 2.5336 ms/op 2.4242 ms/op 1.05
phase0 processBlock - 250000 vs - 7PWei worstcase 30.246 ms/op 29.729 ms/op 1.02
altair processEth1Data - 250000 vs - 7PWei normalcase 520.85 us/op 496.20 us/op 1.05
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 10.109 us/op 10.237 us/op 0.99
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 66.091 us/op 58.444 us/op 1.13
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 17.714 us/op 10.492 us/op 1.69
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 13.910 us/op 12.222 us/op 1.14
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 162.19 us/op 167.56 us/op 0.97
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.4882 ms/op 1.1131 ms/op 1.34
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.6737 ms/op 1.5782 ms/op 1.06
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.5911 ms/op 1.6159 ms/op 0.98
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.9488 ms/op 3.8290 ms/op 1.03
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.4642 ms/op 2.2321 ms/op 1.10
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.4475 ms/op 5.4921 ms/op 0.99
Tree 40 250000 create 364.01 ms/op 370.11 ms/op 0.98
Tree 40 250000 get(125000) 216.44 ns/op 189.57 ns/op 1.14
Tree 40 250000 set(125000) 1.0923 us/op 987.81 ns/op 1.11
Tree 40 250000 toArray() 20.891 ms/op 17.907 ms/op 1.17
Tree 40 250000 iterate all - toArray() + loop 20.500 ms/op 18.103 ms/op 1.13
Tree 40 250000 iterate all - get(i) 70.258 ms/op 64.635 ms/op 1.09
MutableVector 250000 create 12.199 ms/op 14.049 ms/op 0.87
MutableVector 250000 get(125000) 7.1030 ns/op 6.4700 ns/op 1.10
MutableVector 250000 set(125000) 270.14 ns/op 274.42 ns/op 0.98
MutableVector 250000 toArray() 3.3352 ms/op 3.6164 ms/op 0.92
MutableVector 250000 iterate all - toArray() + loop 3.3063 ms/op 3.2608 ms/op 1.01
MutableVector 250000 iterate all - get(i) 1.5243 ms/op 1.5168 ms/op 1.00
Array 250000 create 2.9065 ms/op 2.8586 ms/op 1.02
Array 250000 clone - spread 1.2142 ms/op 1.2228 ms/op 0.99
Array 250000 get(125000) 1.0250 ns/op 1.0290 ns/op 1.00
Array 250000 set(125000) 4.1070 ns/op 4.1320 ns/op 0.99
Array 250000 iterate all - loop 166.29 us/op 164.47 us/op 1.01
effectiveBalanceIncrements clone Uint8Array 300000 27.575 us/op 27.797 us/op 0.99
effectiveBalanceIncrements clone MutableVector 300000 354.00 ns/op 354.00 ns/op 1.00
effectiveBalanceIncrements rw all Uint8Array 300000 201.02 us/op 199.81 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 80.729 ms/op 80.992 ms/op 1.00
phase0 afterProcessEpoch - 250000 vs - 7PWei 108.42 ms/op 112.34 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 51.966 ms/op 50.121 ms/op 1.04
altair processEpoch - mainnet_e81889 483.23 ms/op 489.33 ms/op 0.99
mainnet_e81889 - altair beforeProcessEpoch 82.464 ms/op 82.699 ms/op 1.00
mainnet_e81889 - altair processJustificationAndFinalization 14.935 us/op 14.688 us/op 1.02
mainnet_e81889 - altair processInactivityUpdates 5.6492 ms/op 6.0283 ms/op 0.94
mainnet_e81889 - altair processRewardsAndPenalties 65.219 ms/op 42.627 ms/op 1.53
mainnet_e81889 - altair processRegistryUpdates 2.4450 us/op 2.4020 us/op 1.02
mainnet_e81889 - altair processSlashings 417.00 ns/op 505.00 ns/op 0.83
mainnet_e81889 - altair processEth1DataReset 491.00 ns/op 519.00 ns/op 0.95
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4274 ms/op 1.3793 ms/op 1.03
mainnet_e81889 - altair processSlashingsReset 2.8570 us/op 4.0150 us/op 0.71
mainnet_e81889 - altair processRandaoMixesReset 3.9930 us/op 3.8210 us/op 1.05
mainnet_e81889 - altair processHistoricalRootsUpdate 761.00 ns/op 720.00 ns/op 1.06
mainnet_e81889 - altair processParticipationFlagUpdates 2.6000 us/op 2.3010 us/op 1.13
mainnet_e81889 - altair processSyncCommitteeUpdates 594.00 ns/op 542.00 ns/op 1.10
mainnet_e81889 - altair afterProcessEpoch 116.09 ms/op 114.51 ms/op 1.01
capella processEpoch - mainnet_e217614 2.0943 s/op 2.0786 s/op 1.01
mainnet_e217614 - capella beforeProcessEpoch 482.38 ms/op 487.08 ms/op 0.99
mainnet_e217614 - capella processJustificationAndFinalization 18.784 us/op 14.770 us/op 1.27
mainnet_e217614 - capella processInactivityUpdates 19.843 ms/op 19.419 ms/op 1.02
mainnet_e217614 - capella processRewardsAndPenalties 393.06 ms/op 410.56 ms/op 0.96
mainnet_e217614 - capella processRegistryUpdates 19.715 us/op 15.140 us/op 1.30
mainnet_e217614 - capella processSlashings 558.00 ns/op 524.00 ns/op 1.06
mainnet_e217614 - capella processEth1DataReset 432.00 ns/op 398.00 ns/op 1.09
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.6965 ms/op 4.3140 ms/op 1.09
mainnet_e217614 - capella processSlashingsReset 3.3600 us/op 2.8440 us/op 1.18
mainnet_e217614 - capella processRandaoMixesReset 4.4230 us/op 3.7900 us/op 1.17
mainnet_e217614 - capella processHistoricalRootsUpdate 489.00 ns/op 600.00 ns/op 0.81
mainnet_e217614 - capella processParticipationFlagUpdates 1.9860 us/op 2.1830 us/op 0.91
mainnet_e217614 - capella afterProcessEpoch 297.53 ms/op 321.52 ms/op 0.93
phase0 processEpoch - mainnet_e58758 421.62 ms/op 467.60 ms/op 0.90
mainnet_e58758 - phase0 beforeProcessEpoch 140.90 ms/op 137.00 ms/op 1.03
mainnet_e58758 - phase0 processJustificationAndFinalization 14.904 us/op 16.434 us/op 0.91
mainnet_e58758 - phase0 processRewardsAndPenalties 34.488 ms/op 36.457 ms/op 0.95
mainnet_e58758 - phase0 processRegistryUpdates 8.6650 us/op 8.5600 us/op 1.01
mainnet_e58758 - phase0 processSlashings 532.00 ns/op 507.00 ns/op 1.05
mainnet_e58758 - phase0 processEth1DataReset 406.00 ns/op 388.00 ns/op 1.05
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.1107 ms/op 1.0903 ms/op 1.02
mainnet_e58758 - phase0 processSlashingsReset 2.2740 us/op 2.5030 us/op 0.91
mainnet_e58758 - phase0 processRandaoMixesReset 3.7680 us/op 3.9200 us/op 0.96
mainnet_e58758 - phase0 processHistoricalRootsUpdate 533.00 ns/op 630.00 ns/op 0.85
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.8480 us/op 3.8630 us/op 1.25
mainnet_e58758 - phase0 afterProcessEpoch 97.710 ms/op 97.529 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4407 ms/op 1.3574 ms/op 1.06
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4466 ms/op 1.9426 ms/op 0.74
altair processInactivityUpdates - 250000 normalcase 22.288 ms/op 29.908 ms/op 0.75
altair processInactivityUpdates - 250000 worstcase 27.086 ms/op 27.441 ms/op 0.99
phase0 processRegistryUpdates - 250000 normalcase 10.044 us/op 7.4180 us/op 1.35
phase0 processRegistryUpdates - 250000 badcase_full_deposits 328.52 us/op 336.72 us/op 0.98
phase0 processRegistryUpdates - 250000 worstcase 0.5 120.25 ms/op 125.17 ms/op 0.96
altair processRewardsAndPenalties - 250000 normalcase 58.739 ms/op 58.226 ms/op 1.01
altair processRewardsAndPenalties - 250000 worstcase 57.549 ms/op 57.595 ms/op 1.00
phase0 getAttestationDeltas - 250000 normalcase 8.7059 ms/op 7.7168 ms/op 1.13
phase0 getAttestationDeltas - 250000 worstcase 8.8331 ms/op 8.0777 ms/op 1.09
phase0 processSlashings - 250000 worstcase 86.208 us/op 81.527 us/op 1.06
altair processSyncCommitteeUpdates - 250000 158.13 ms/op 150.88 ms/op 1.05
BeaconState.hashTreeRoot - No change 267.00 ns/op 243.00 ns/op 1.10
BeaconState.hashTreeRoot - 1 full validator 117.98 us/op 141.16 us/op 0.84
BeaconState.hashTreeRoot - 32 full validator 1.2068 ms/op 1.5932 ms/op 0.76
BeaconState.hashTreeRoot - 512 full validator 14.361 ms/op 19.003 ms/op 0.76
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 151.82 us/op 181.62 us/op 0.84
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.9578 ms/op 2.3335 ms/op 0.84
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 24.067 ms/op 26.680 ms/op 0.90
BeaconState.hashTreeRoot - 1 balances 114.82 us/op 146.18 us/op 0.79
BeaconState.hashTreeRoot - 32 balances 1.0358 ms/op 1.2754 ms/op 0.81
BeaconState.hashTreeRoot - 512 balances 9.8971 ms/op 11.689 ms/op 0.85
BeaconState.hashTreeRoot - 250000 balances 211.59 ms/op 203.38 ms/op 1.04
aggregationBits - 2048 els - zipIndexesInBitList 16.074 us/op 16.885 us/op 0.95
byteArrayEquals 32 72.423 ns/op 75.733 ns/op 0.96
Buffer.compare 32 54.137 ns/op 56.509 ns/op 0.96
byteArrayEquals 1024 2.0829 us/op 2.0717 us/op 1.01
Buffer.compare 1024 72.628 ns/op 71.225 ns/op 1.02
byteArrayEquals 16384 32.762 us/op 33.016 us/op 0.99
Buffer.compare 16384 277.76 ns/op 286.08 ns/op 0.97
byteArrayEquals 123687377 245.98 ms/op 245.99 ms/op 1.00
Buffer.compare 123687377 6.4917 ms/op 6.2051 ms/op 1.05
byteArrayEquals 32 - diff last byte 71.876 ns/op 71.473 ns/op 1.01
Buffer.compare 32 - diff last byte 56.377 ns/op 55.266 ns/op 1.02
byteArrayEquals 1024 - diff last byte 2.0404 us/op 2.0244 us/op 1.01
Buffer.compare 1024 - diff last byte 72.247 ns/op 72.063 ns/op 1.00
byteArrayEquals 16384 - diff last byte 32.598 us/op 32.266 us/op 1.01
Buffer.compare 16384 - diff last byte 263.55 ns/op 273.78 ns/op 0.96
byteArrayEquals 123687377 - diff last byte 247.88 ms/op 243.11 ms/op 1.02
Buffer.compare 123687377 - diff last byte 6.7022 ms/op 6.1118 ms/op 1.10
byteArrayEquals 32 - random bytes 5.5250 ns/op 5.2130 ns/op 1.06
Buffer.compare 32 - random bytes 61.585 ns/op 59.864 ns/op 1.03
byteArrayEquals 1024 - random bytes 5.3220 ns/op 5.1260 ns/op 1.04
Buffer.compare 1024 - random bytes 61.135 ns/op 59.311 ns/op 1.03
byteArrayEquals 16384 - random bytes 5.2960 ns/op 5.2220 ns/op 1.01
Buffer.compare 16384 - random bytes 60.597 ns/op 59.402 ns/op 1.02
byteArrayEquals 123687377 - random bytes 8.7600 ns/op 8.4500 ns/op 1.04
Buffer.compare 123687377 - random bytes 64.630 ns/op 62.850 ns/op 1.03
regular array get 100000 times 45.113 us/op 43.610 us/op 1.03
wrappedArray get 100000 times 44.766 us/op 43.573 us/op 1.03
arrayWithProxy get 100000 times 14.301 ms/op 13.913 ms/op 1.03
ssz.Root.equals 54.002 ns/op 53.193 ns/op 1.02
byteArrayEquals 52.928 ns/op 52.457 ns/op 1.01
Buffer.compare 10.778 ns/op 10.774 ns/op 1.00
shuffle list - 16384 els 6.9527 ms/op 7.3159 ms/op 0.95
shuffle list - 250000 els 101.65 ms/op 101.54 ms/op 1.00
processSlot - 1 slots 17.215 us/op 16.518 us/op 1.04
processSlot - 32 slots 4.1338 ms/op 3.3980 ms/op 1.22
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 54.379 ms/op 61.460 ms/op 0.88
getCommitteeAssignments - req 1 vs - 250000 vc 2.4978 ms/op 2.4802 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 3.7012 ms/op 3.6986 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 4.0793 ms/op 4.0062 ms/op 1.02
findModifiedValidators - 10000 modified validators 542.56 ms/op 558.72 ms/op 0.97
findModifiedValidators - 1000 modified validators 424.28 ms/op 412.82 ms/op 1.03
findModifiedValidators - 100 modified validators 394.94 ms/op 403.66 ms/op 0.98
findModifiedValidators - 10 modified validators 422.60 ms/op 412.65 ms/op 1.02
findModifiedValidators - 1 modified validators 406.86 ms/op 384.54 ms/op 1.06
findModifiedValidators - no difference 431.88 ms/op 402.66 ms/op 1.07
compare ViewDUs 4.4670 s/op 4.2669 s/op 1.05
compare each validator Uint8Array 1.6739 s/op 1.7537 s/op 0.95
compare ViewDU to Uint8Array 1.1273 s/op 1.1311 s/op 1.00
migrate state 1000000 validators, 24 modified, 0 new 715.11 ms/op 790.44 ms/op 0.90
migrate state 1000000 validators, 1700 modified, 1000 new 1.0891 s/op 1.0998 s/op 0.99
migrate state 1000000 validators, 3400 modified, 2000 new 1.3019 s/op 1.3276 s/op 0.98
migrate state 1500000 validators, 24 modified, 0 new 786.78 ms/op 793.75 ms/op 0.99
migrate state 1500000 validators, 1700 modified, 1000 new 1.0332 s/op 1.1007 s/op 0.94
migrate state 1500000 validators, 3400 modified, 2000 new 1.2678 s/op 1.3339 s/op 0.95
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.0800 ns/op 4.3100 ns/op 0.95
state getBlockRootAtSlot - 250000 vs - 7PWei 737.68 ns/op 728.58 ns/op 1.01
computeProposers - vc 250000 8.6912 ms/op 9.2821 ms/op 0.94
computeEpochShuffling - vc 250000 97.408 ms/op 102.40 ms/op 0.95
getNextSyncCommittee - vc 250000 149.06 ms/op 157.80 ms/op 0.94
computeSigningRoot for AttestationData 25.860 us/op 29.525 us/op 0.88
hash AttestationData serialized data then Buffer.toString(base64) 2.2356 us/op 2.2632 us/op 0.99
toHexString serialized data 1.0152 us/op 1.0628 us/op 0.96
Buffer.toString(base64) 215.22 ns/op 221.11 ns/op 0.97

by benchmarkbot/action

@dapplion
Copy link
Contributor

@g11tech g11tech changed the base branch from unstable to optimistic-verify November 21, 2023 11:07
@g11tech g11tech marked this pull request as draft November 21, 2023 11:11
@g11tech
Copy link
Contributor Author

g11tech commented Nov 21, 2023

yes it did, i changed base to just reflect this diff

@g11tech g11tech force-pushed the blobs-inclproof branch 5 times, most recently from 860784d to 17154a8 Compare November 21, 2023 14:09
@g11tech g11tech force-pushed the optimistic-verify branch 2 times, most recently from 0e793db to f43984e Compare December 19, 2023 10:28
Base automatically changed from optimistic-verify to unstable December 19, 2023 12:34
@g11tech g11tech marked this pull request as ready for review December 19, 2023 16:00
@codecov Codecov
Copy link

codecov bot commented Dec 22, 2023 •
edited
Loading

Codecov Report

Merging #6089 (3ceb6bf) into unstable (61cf1a8) will increase coverage by 0.11%.
The diff coverage is n/a.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6089      +/-   ##
============================================
+ Coverage     80.83%   80.95%   +0.11%     
============================================
  Files           185      185              
  Lines         17986    17935      -51     
  Branches       1082     1078       -4     
============================================
- Hits          14539    14519      -20     
+ Misses         3421     3389      -32     
- Partials         26       27       +1

@g11tech
feat: implement blob sidecars with inclusion proof instead of signtaures
3ceb6bf 
modify the api

fix validator handling

fix val build

refactor the beacon node impl for the new blobs architecture

get the build working with no blobs

fix the blob sidecars transmisssion

relocate compute blob sidecars

relocate verify merkle branch

verify inclusion proof

fix tests and types

fix unit test

update the spec test versions

skip newly required merkle proof runner

change the minimal/mainnet preset based constant strategy

add other constants to tests

apply feedback

apply feedback

relocate utils
twoeths
twoeths reviewed Dec 25, 2023
View reviewed changes
packages/beacon-node/src/chain/validation/blobSidecar.ts
@@ -207,3 +215,13 @@ function validateBlobsAndProofs(
throw Error("Invalid verifyBlobKzgProofBatch");
}
}

function validateInclusionProof(config: ChainForkConfig, blobSidecar: deneb.BlobSidecar): boolean {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused config

twoeths
twoeths reviewed Dec 25, 2023
View reviewed changes
packages/beacon-node/src/chain/errors/blobSidecarError.ts
@@ -37,6 +38,7 @@ export type BlobSidecarErrorType =
| {code: BlobSidecarErrorCode.PARENT_UNKNOWN; parentRoot: RootHex}
| {code: BlobSidecarErrorCode.NOT_LATER_THAN_PARENT; parentSlot: Slot; slot: Slot}
| {code: BlobSidecarErrorCode.PROPOSAL_SIGNATURE_INVALID}
| {code: BlobSidecarErrorCode.INCLUSION_PROOF_INVALID}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to include slot, blob index here

twoeths
twoeths approved these changes Dec 25, 2023
View reviewed changes
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me overall, there are 2 minor points, approving in case @g11tech wants to merge and address them in another PR

@g11tech g11tech merged commit 6f4a9d6 into unstable Dec 25, 2023
14 of 15 checks passed
@g11tech g11tech deleted the blobs-inclproof branch December 25, 2023 09:24
@g11tech
Copy link
Contributor Author

g11tech commented Dec 25, 2023

thanks @tuyennhv , yes let me address then in a followup

@g11tech g11tech mentioned this pull request Dec 26, 2023
Merged
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.14.0 🎉

ensi321 pushed a commit to ensi321/lodestar that referenced this pull request Jan 22, 2024
@g11tech @ensi321
feat: implement blob sidecars with inclusion proof instead of signtau…
428c110 
…res (ChainSafe#6089)

modify the api

fix validator handling

fix val build

refactor the beacon node impl for the new blobs architecture

get the build working with no blobs

fix the blob sidecars transmisssion

relocate compute blob sidecars

relocate verify merkle branch

verify inclusion proof

fix tests and types

fix unit test

update the spec test versions

skip newly required merkle proof runner

change the minimal/mainnet preset based constant strategy

add other constants to tests

apply feedback

apply feedback

relocate utils
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion left review comments

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@g11tech @dapplion @wemeetagain @twoeths