Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validator option not to submit attestation early #3944

Merged
merged 4 commits into from
May 6, 2022

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Apr 22, 2022

Motivation

As monitored in hetzner-c0, there are 2 missed attestations with:

  • correct head
  • sent peers is 2 or 8
  • delaySec is around -1.5

It's possible that the missed attestations were because we submit them too early, and peers ignore them (maybe due to unknown block root?). Note that lighthouse does not submit attestations early at this time

Description

Add an option to submit attestations right at 1/3 of slot so that we can test in mainnet nodes

part of #3943

@twoeths twoeths requested a review from a team as a code owner April 22, 2022 04:20
@codecov
Copy link

codecov bot commented Apr 22, 2022

Codecov Report

Merging #3944 (38b3705) into master (97f6012) will increase coverage by 0.44%.
The diff coverage is n/a.

❗ Current head 38b3705 differs from pull request most recent head a082d75. Consider uploading reports for the commit a082d75 to get more accurate results

@@            Coverage Diff             @@
##           master    #3944      +/-   ##
==========================================
+ Coverage   36.33%   36.77%   +0.44%     
==========================================
  Files         324      325       +1     
  Lines        9063     9223     +160     
  Branches     1455     1498      +43     
==========================================
+ Hits         3293     3392      +99     
- Misses       5597     5640      +43     
- Partials      173      191      +18     

@github-actions
Copy link
Contributor

github-actions bot commented Apr 22, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 38b3705 Previous: efdad3f Ratio
BeaconState.hashTreeRoot - No change 431.00 ns/op 507.00 ns/op 0.85
BeaconState.hashTreeRoot - 1 full validator 55.947 us/op 54.724 us/op 1.02
BeaconState.hashTreeRoot - 32 full validator 540.85 us/op 539.29 us/op 1.00
BeaconState.hashTreeRoot - 512 full validator 6.1952 ms/op 6.0110 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 69.009 us/op 68.558 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 954.90 us/op 947.83 us/op 1.01
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 13.727 ms/op 13.117 ms/op 1.05
BeaconState.hashTreeRoot - 1 balances 53.524 us/op 52.702 us/op 1.02
BeaconState.hashTreeRoot - 32 balances 463.44 us/op 474.61 us/op 0.98
BeaconState.hashTreeRoot - 512 balances 4.5760 ms/op 4.5800 ms/op 1.00
BeaconState.hashTreeRoot - 250000 balances 96.713 ms/op 95.917 ms/op 1.01
processSlot - 1 slots 9.4510 us/op 9.6560 us/op 0.98
processSlot - 32 slots 1.5727 ms/op 1.5634 ms/op 1.01
getCommitteeAssignments - req 1 vs - 250000 vc 5.2295 ms/op 5.2723 ms/op 0.99
getCommitteeAssignments - req 100 vs - 250000 vc 7.2970 ms/op 7.3524 ms/op 0.99
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7917 ms/op 7.8458 ms/op 0.99
computeProposers - vc 250000 16.160 ms/op 16.367 ms/op 0.99
computeEpochShuffling - vc 250000 156.77 ms/op 160.26 ms/op 0.98
getNextSyncCommittee - vc 250000 270.26 ms/op 272.81 ms/op 0.99
altair processAttestation - 250000 vs - 7PWei normalcase 4.1765 ms/op 3.9881 ms/op 1.05
altair processAttestation - 250000 vs - 7PWei worstcase 5.6946 ms/op 5.8581 ms/op 0.97
altair processAttestation - setStatus - 1/6 committees join 196.61 us/op 196.37 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 376.93 us/op 374.77 us/op 1.01
altair processAttestation - setStatus - 1/2 committees join 525.63 us/op 526.42 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 678.37 us/op 674.30 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 947.81 us/op 938.32 us/op 1.01
altair processAttestation - setStatus - 100% committees join 1.1340 ms/op 1.1082 ms/op 1.02
altair processBlock - 250000 vs - 7PWei normalcase 27.800 ms/op 25.076 ms/op 1.11
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.222 ms/op 35.518 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase 71.926 ms/op 72.913 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 107.01 ms/op 111.62 ms/op 0.96
altair processEth1Data - 250000 vs - 7PWei normalcase 812.10 us/op 793.51 us/op 1.02
altair processEpoch - mainnet_e81889 546.83 ms/op 579.90 ms/op 0.94
mainnet_e81889 - altair beforeProcessEpoch 135.02 ms/op 99.756 ms/op 1.35
mainnet_e81889 - altair processJustificationAndFinalization 22.327 us/op 24.056 us/op 0.93
mainnet_e81889 - altair processInactivityUpdates 10.604 ms/op 12.451 ms/op 0.85
mainnet_e81889 - altair processRewardsAndPenalties 145.72 ms/op 141.55 ms/op 1.03
mainnet_e81889 - altair processRegistryUpdates 3.1980 us/op 3.6580 us/op 0.87
mainnet_e81889 - altair processSlashings 1.1620 us/op 1.1380 us/op 1.02
mainnet_e81889 - altair processEth1DataReset 969.00 ns/op 1.0460 us/op 0.93
mainnet_e81889 - altair processEffectiveBalanceUpdates 7.8919 ms/op 7.5597 ms/op 1.04
mainnet_e81889 - altair processSlashingsReset 5.0430 us/op 4.7790 us/op 1.06
mainnet_e81889 - altair processRandaoMixesReset 6.3960 us/op 6.6310 us/op 0.96
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1990 us/op 1.4340 us/op 0.84
mainnet_e81889 - altair processParticipationFlagUpdates 2.8770 us/op 3.2350 us/op 0.89
mainnet_e81889 - altair processSyncCommitteeUpdates 863.00 ns/op 933.00 ns/op 0.92
mainnet_e81889 - altair afterProcessEpoch 181.96 ms/op 183.68 ms/op 0.99
altair processInactivityUpdates - 250000 normalcase 31.216 ms/op 32.523 ms/op 0.96
altair processInactivityUpdates - 250000 worstcase 25.645 ms/op 27.676 ms/op 0.93
altair processRewardsAndPenalties - 250000 normalcase 117.03 ms/op 123.73 ms/op 0.95
altair processRewardsAndPenalties - 250000 worstcase 105.04 ms/op 78.862 ms/op 1.33
altair processSyncCommitteeUpdates - 250000 286.54 ms/op 284.98 ms/op 1.01
Tree 40 250000 create 719.29 ms/op 712.15 ms/op 1.01
Tree 40 250000 get(125000) 281.38 ns/op 282.29 ns/op 1.00
Tree 40 250000 set(125000) 2.2561 us/op 2.2319 us/op 1.01
Tree 40 250000 toArray() 31.137 ms/op 32.987 ms/op 0.94
Tree 40 250000 iterate all - toArray() + loop 31.415 ms/op 33.181 ms/op 0.95
Tree 40 250000 iterate all - get(i) 109.00 ms/op 110.10 ms/op 0.99
MutableVector 250000 create 14.626 ms/op 15.175 ms/op 0.96
MutableVector 250000 get(125000) 14.810 ns/op 14.757 ns/op 1.00
MutableVector 250000 set(125000) 577.82 ns/op 587.98 ns/op 0.98
MutableVector 250000 toArray() 6.7902 ms/op 6.6816 ms/op 1.02
MutableVector 250000 iterate all - toArray() + loop 6.8417 ms/op 7.0611 ms/op 0.97
MutableVector 250000 iterate all - get(i) 3.4449 ms/op 3.2826 ms/op 1.05
Array 250000 create 5.9899 ms/op 6.1593 ms/op 0.97
Array 250000 clone - spread 2.6690 ms/op 2.6268 ms/op 1.02
Array 250000 get(125000) 1.1590 ns/op 1.1780 ns/op 0.98
Array 250000 set(125000) 1.1720 ns/op 1.2080 ns/op 0.97
Array 250000 iterate all - loop 167.92 us/op 167.83 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 92.786 us/op 85.174 us/op 1.09
effectiveBalanceIncrements clone MutableVector 300000 561.00 ns/op 776.00 ns/op 0.72
effectiveBalanceIncrements rw all Uint8Array 300000 301.98 us/op 301.86 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 179.34 ms/op 192.09 ms/op 0.93
aggregationBits - 2048 els - zipIndexesInBitList 27.768 us/op 31.718 us/op 0.88
regular array get 100000 times 67.454 us/op 67.400 us/op 1.00
wrappedArray get 100000 times 67.431 us/op 67.416 us/op 1.00
arrayWithProxy get 100000 times 30.049 ms/op 29.947 ms/op 1.00
ssz.Root.equals 461.00 ns/op 460.00 ns/op 1.00
byteArrayEquals 439.00 ns/op 450.00 ns/op 0.98
phase0 processBlock - 250000 vs - 7PWei normalcase 3.6427 ms/op 3.5731 ms/op 1.02
phase0 processBlock - 250000 vs - 7PWei worstcase 45.924 ms/op 46.888 ms/op 0.98
phase0 afterProcessEpoch - 250000 vs - 7PWei 173.22 ms/op 199.54 ms/op 0.87
phase0 beforeProcessEpoch - 250000 vs - 7PWei 75.675 ms/op 76.630 ms/op 0.99
phase0 processEpoch - mainnet_e58758 508.40 ms/op 533.78 ms/op 0.95
mainnet_e58758 - phase0 beforeProcessEpoch 226.81 ms/op 263.23 ms/op 0.86
mainnet_e58758 - phase0 processJustificationAndFinalization 21.819 us/op 23.771 us/op 0.92
mainnet_e58758 - phase0 processRewardsAndPenalties 128.19 ms/op 80.194 ms/op 1.60
mainnet_e58758 - phase0 processRegistryUpdates 11.070 us/op 10.341 us/op 1.07
mainnet_e58758 - phase0 processSlashings 1.1490 us/op 1.2920 us/op 0.89
mainnet_e58758 - phase0 processEth1DataReset 1.0450 us/op 974.00 ns/op 1.07
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 6.4148 ms/op 5.5904 ms/op 1.15
mainnet_e58758 - phase0 processSlashingsReset 5.2490 us/op 5.4380 us/op 0.97
mainnet_e58758 - phase0 processRandaoMixesReset 5.6170 us/op 6.2790 us/op 0.89
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2190 us/op 1.2030 us/op 1.01
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.9060 us/op 5.2050 us/op 0.94
mainnet_e58758 - phase0 afterProcessEpoch 151.52 ms/op 176.58 ms/op 0.86
phase0 processEffectiveBalanceUpdates - 250000 normalcase 6.3740 ms/op 6.3990 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 6.8666 ms/op 6.8927 ms/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 8.2290 us/op 9.5890 us/op 0.86
phase0 processRegistryUpdates - 250000 badcase_full_deposits 411.86 us/op 419.35 us/op 0.98
phase0 processRegistryUpdates - 250000 worstcase 0.5 216.79 ms/op 215.80 ms/op 1.00
phase0 getAttestationDeltas - 250000 normalcase 16.046 ms/op 15.107 ms/op 1.06
phase0 getAttestationDeltas - 250000 worstcase 15.946 ms/op 15.336 ms/op 1.04
phase0 processSlashings - 250000 worstcase 6.1253 ms/op 5.4529 ms/op 1.12
shuffle list - 16384 els 10.574 ms/op 10.686 ms/op 0.99
shuffle list - 250000 els 152.57 ms/op 154.33 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 363.11 us/op 361.78 us/op 1.00
pass gossip attestations to forkchoice per slot 3.4838 ms/op 3.4401 ms/op 1.01
computeDeltas 3.2848 ms/op 4.1471 ms/op 0.79
computeProposerBoostScoreFromBalances 503.20 us/op 503.04 us/op 1.00
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.1649 ms/op 2.0581 ms/op 1.05
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 84.101 us/op 80.806 us/op 1.04
BLS verify - blst-native 1.8597 ms/op 1.8571 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 3.8050 ms/op 3.8054 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 8.1923 ms/op 8.2046 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 29.678 ms/op 29.733 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 39.645 us/op 39.642 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 153.94 us/op 153.94 us/op 1.00
getAttestationsForBlock 61.295 ms/op 63.282 ms/op 0.97
CheckpointStateCache - add get delete 10.666 us/op 11.207 us/op 0.95
validate gossip signedAggregateAndProof - struct 4.2705 ms/op 4.2661 ms/op 1.00
validate gossip attestation - struct 2.0277 ms/op 2.0285 ms/op 1.00
pickEth1Vote - no votes 2.1516 ms/op 2.1919 ms/op 0.98
pickEth1Vote - max votes 20.102 ms/op 20.313 ms/op 0.99
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.308 ms/op 11.207 ms/op 1.01
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.886 ms/op 21.397 ms/op 0.98
pickEth1Vote - Eth1Data fastSerialize value x2048 1.7754 ms/op 1.6139 ms/op 1.10
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.083 ms/op 13.309 ms/op 1.06
bytes32 toHexString 1.0190 us/op 993.00 ns/op 1.03
bytes32 Buffer.toString(hex) 679.00 ns/op 669.00 ns/op 1.01
bytes32 Buffer.toString(hex) from Uint8Array 854.00 ns/op 915.00 ns/op 0.93
bytes32 Buffer.toString(hex) + 0x 691.00 ns/op 679.00 ns/op 1.02
Object access 1 prop 0.36000 ns/op 0.34400 ns/op 1.05
Map access 1 prop 0.30200 ns/op 0.29300 ns/op 1.03
Object get x1000 17.066 ns/op 18.249 ns/op 0.94
Map get x1000 1.0950 ns/op 1.0870 ns/op 1.01
Object set x1000 109.41 ns/op 110.82 ns/op 0.99
Map set x1000 68.570 ns/op 66.844 ns/op 1.03
Return object 10000 times 0.36990 ns/op 0.36820 ns/op 1.00
Throw Error 10000 times 5.8284 us/op 5.8561 us/op 1.00
enrSubnets - fastDeserialize 64 bits 2.8190 us/op 2.7370 us/op 1.03
enrSubnets - ssz BitVector 64 bits 749.00 ns/op 781.00 ns/op 0.96
enrSubnets - fastDeserialize 4 bits 422.00 ns/op 396.00 ns/op 1.07
enrSubnets - ssz BitVector 4 bits 776.00 ns/op 782.00 ns/op 0.99
prioritizePeers score -10:0 att 32-0.1 sync 2-0 90.178 us/op 91.985 us/op 0.98
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 131.14 us/op 125.09 us/op 1.05
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 208.91 us/op 206.48 us/op 1.01
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 389.85 us/op 437.52 us/op 0.89
prioritizePeers score 0:0 att 64-1 sync 4-1 461.09 us/op 459.84 us/op 1.00
RateTracker 1000000 limit, 1 obj count per request 193.20 ns/op 181.33 ns/op 1.07
RateTracker 1000000 limit, 2 obj count per request 148.52 ns/op 137.02 ns/op 1.08
RateTracker 1000000 limit, 4 obj count per request 126.39 ns/op 113.99 ns/op 1.11
RateTracker 1000000 limit, 8 obj count per request 114.74 ns/op 103.13 ns/op 1.11
RateTracker with prune 4.1970 us/op 4.3790 us/op 0.96
array of 16000 items push then shift 3.1500 us/op 3.0935 us/op 1.02
LinkedList of 16000 items push then shift 17.577 ns/op 18.333 ns/op 0.96
array of 16000 items push then pop 199.75 ns/op 211.93 ns/op 0.94
LinkedList of 16000 items push then pop 15.253 ns/op 17.243 ns/op 0.88
array of 24000 items push then shift 4.5573 us/op 4.5401 us/op 1.00
LinkedList of 24000 items push then shift 20.781 ns/op 23.279 ns/op 0.89
array of 24000 items push then pop 206.51 ns/op 209.59 ns/op 0.99
LinkedList of 24000 items push then pop 17.273 ns/op 19.885 ns/op 0.87

by benchmarkbot/action

@@ -61,6 +62,12 @@ export const validatorOptions: ICliCommandOptions<IValidatorCliArgs> = {
type: "string",
},

dontSubmitAttestationEarly: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we hidden:true this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the option to earlyAttestationDelayMs and use hidden:true for it

@dapplion
Copy link
Contributor

dapplion commented Apr 26, 2022

Checkout this PR by Nimbus, they do a happy medium, wait at least 2 seconds but still get ahead status-im/nimbus-eth2#3518

// for early attestations, wait until now + 2000ms, or 1/3 of slot, whichever comes first
await sleep(Math.min(msToOneThirdSlot, this.earlyAttestationDelayMs));
}
this.metrics?.attesterStepCallPublishAttestation.observe(this.clock.secFromSlot(slot + 1 / 3));
Copy link
Contributor

@dadepo dadepo Apr 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it is this.earlyAttestationDelayMs that comes first, won't this metric be wrong?
edit: No. Metric tracks now (whenever that might be) up till slot + 1/3

@dadepo
Copy link
Contributor

dadepo commented Apr 28, 2022

Looking at the Nimbus PR here, initially the wait was 250ms, then it was changed to 1000ms and now 2000ms due to the latency introduced as the network grows.

Won't a more robust approach be to have beacon clients cache early attestations instead? Not sure if doing this won't conflict with other parts of the spec/implementations

@twoeths
Copy link
Contributor Author

twoeths commented Apr 29, 2022

Looking at the Nimbus PR here, initially the wait was 250ms, then it was changed to 1000ms and now 2000ms due to the latency introduced as the network grows.

Won't a more robust approach be to have beacon clients cache early attestations instead? Not sure if doing this won't conflict with other parts of the spec/implementations

This is from the spec

A validator should create and broadcast the attestation to the associated attestation subnet when either (a) the validator has received a valid block from the expected block proposer for the assigned slot or (b) 1 / INTERVALS_PER_SLOT of the slot has transpired (SECONDS_PER_SLOT / INTERVALS_PER_SLOT seconds after the start of slot) -- whichever comes first.

so we still follow the spec. I don't think we want to introduce another cache at beacon node side unnecessarily

@dapplion dapplion merged commit d7a3211 into master May 6, 2022
@dapplion dapplion deleted the tuyen/validator-option-to-wait-for-1/3-of-slot branch May 6, 2022 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants