Flat state transition across forks #4120

dapplion · 2022-06-07T17:50:55Z

Motivation

Beacon state transition code is currently split between forks to "mimic" the spec definition style in some cases. I think that's not the best for clarity and is the cause for a lot of boilerplate downstream.

Split style

function processNumberPhase0(n: number): number {
  return n + 1;
}

function processNumberAltair(n: number): number {
  return n + 2;
}

Flat style

function processNumber(fork: ForkSeq, n: number): number {
  if (fork === ForkSeq.phase0) {
    return n + 1;
  else {
    return n + 2;
  }
}

Description

Flat state transition across forks. Re-arrange code to not be in fork directories anymore:

src
├── block
│   ├── index.ts
│   ├── processAttestations.ts
│   └── ...
├── epoch
│   ├── index.ts
│   ├── processEffectiveBalanceUpdates.ts
│   └── ...
├── signatureSets
│   └── ..
├── slot
│   ├── index.ts
│   ├── upgradeStateToAltair.ts
│   └── upgradeStateToBellatrix.ts
└── util
    ├── altair.ts
    ├── bellatrix.ts
    └── ...

It also removes the by-fork exports of the beacon-state-transition library. Code like:

allForks.stateTransition()

becomes

stateTransition()

github-actions · 2022-06-07T18:13:01Z

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: `48d45f1`	Previous: `f74f9ef`	Ratio
altair processAttestation - 250000 vs - 7PWei normalcase	4.5997 ms/op	3.5326 ms/op	1.30
altair processAttestation - 250000 vs - 7PWei worstcase	7.6639 ms/op	5.1985 ms/op	1.47
altair processAttestation - setStatus - 1/6 committees join	235.66 us/op	190.14 us/op	1.24
altair processAttestation - setStatus - 1/3 committees join	458.46 us/op	369.19 us/op	1.24
altair processAttestation - setStatus - 1/2 committees join	645.91 us/op	518.02 us/op	1.25
altair processAttestation - setStatus - 2/3 committees join	828.73 us/op	671.44 us/op	1.23
altair processAttestation - setStatus - 4/5 committees join	1.1901 ms/op	907.26 us/op	1.31
altair processAttestation - setStatus - 100% committees join	1.3545 ms/op	1.0816 ms/op	1.25
altair processBlock - 250000 vs - 7PWei normalcase	31.225 ms/op	23.409 ms/op	1.33
altair processBlock - 250000 vs - 7PWei normalcase hashState	36.894 ms/op	35.767 ms/op	1.03
altair processBlock - 250000 vs - 7PWei worstcase	94.053 ms/op	76.414 ms/op	1.23
altair processBlock - 250000 vs - 7PWei worstcase hashState	117.95 ms/op	95.214 ms/op	1.24
phase0 processBlock - 250000 vs - 7PWei normalcase	4.2078 ms/op	3.3958 ms/op	1.24
phase0 processBlock - 250000 vs - 7PWei worstcase	55.076 ms/op	44.400 ms/op	1.24
altair processEth1Data - 250000 vs - 7PWei normalcase	825.20 us/op	770.46 us/op	1.07
Tree 40 250000 create	910.40 ms/op	738.91 ms/op	1.23
Tree 40 250000 get(125000)	350.88 ns/op	282.25 ns/op	1.24
Tree 40 250000 set(125000)	2.9489 us/op	2.2475 us/op	1.31
Tree 40 250000 toArray()	37.690 ms/op	30.272 ms/op	1.25
Tree 40 250000 iterate all - toArray() + loop	38.050 ms/op	30.265 ms/op	1.26
Tree 40 250000 iterate all - get(i)	134.19 ms/op	106.57 ms/op	1.26
MutableVector 250000 create	19.427 ms/op	14.535 ms/op	1.34
MutableVector 250000 get(125000)	15.993 ns/op	13.134 ns/op	1.22
MutableVector 250000 set(125000)	924.04 ns/op	660.02 ns/op	1.40
MutableVector 250000 toArray()	8.7244 ms/op	6.1312 ms/op	1.42
MutableVector 250000 iterate all - toArray() + loop	9.1657 ms/op	6.5430 ms/op	1.40
MutableVector 250000 iterate all - get(i)	3.9370 ms/op	3.2823 ms/op	1.20
Array 250000 create	8.1572 ms/op	5.5670 ms/op	1.47
Array 250000 clone - spread	5.0090 ms/op	3.4666 ms/op	1.44
Array 250000 get(125000)	2.2010 ns/op	1.5250 ns/op	1.44
Array 250000 set(125000)	2.2440 ns/op	1.5110 ns/op	1.49
Array 250000 iterate all - loop	201.50 us/op	167.85 us/op	1.20
effectiveBalanceIncrements clone Uint8Array 300000	115.92 us/op	97.281 us/op	1.19
effectiveBalanceIncrements clone MutableVector 300000	1.6010 us/op	672.00 ns/op	2.38
effectiveBalanceIncrements rw all Uint8Array 300000	303.17 us/op	252.90 us/op	1.20
effectiveBalanceIncrements rw all MutableVector 300000	287.28 ms/op	155.51 ms/op	1.85
phase0 afterProcessEpoch - 250000 vs - 7PWei	236.84 ms/op	181.72 ms/op	1.30
phase0 beforeProcessEpoch - 250000 vs - 7PWei	76.843 ms/op	69.306 ms/op	1.11
altair processEpoch - mainnet_e81889	673.31 ms/op	531.48 ms/op	1.27
mainnet_e81889 - altair beforeProcessEpoch	189.30 ms/op	96.528 ms/op	1.96
mainnet_e81889 - altair processJustificationAndFinalization	39.938 us/op	36.732 us/op	1.09
mainnet_e81889 - altair processInactivityUpdates	14.020 ms/op	10.373 ms/op	1.35
mainnet_e81889 - altair processRewardsAndPenalties	160.78 ms/op	83.015 ms/op	1.94
mainnet_e81889 - altair processRegistryUpdates	7.9420 us/op	5.7750 us/op	1.38
mainnet_e81889 - altair processSlashings	1.8350 us/op	1.6960 us/op	1.08
mainnet_e81889 - altair processEth1DataReset	2.2230 us/op	1.3530 us/op	1.64
mainnet_e81889 - altair processEffectiveBalanceUpdates	4.1440 ms/op	2.4549 ms/op	1.69
mainnet_e81889 - altair processSlashingsReset	13.367 us/op	5.9270 us/op	2.26
mainnet_e81889 - altair processRandaoMixesReset	15.406 us/op	7.7820 us/op	1.98
mainnet_e81889 - altair processHistoricalRootsUpdate	2.9880 us/op	1.1080 us/op	2.70
mainnet_e81889 - altair processParticipationFlagUpdates	6.6550 us/op	4.3020 us/op	1.55
mainnet_e81889 - altair processSyncCommitteeUpdates	2.0240 us/op	1.4250 us/op	1.42
mainnet_e81889 - altair afterProcessEpoch	232.42 ms/op	192.83 ms/op	1.21
phase0 processEpoch - mainnet_e58758	617.23 ms/op	500.66 ms/op	1.23
mainnet_e58758 - phase0 beforeProcessEpoch	264.80 ms/op	212.50 ms/op	1.25
mainnet_e58758 - phase0 processJustificationAndFinalization	31.727 us/op	30.910 us/op	1.03
mainnet_e58758 - phase0 processRewardsAndPenalties	144.69 ms/op	127.92 ms/op	1.13
mainnet_e58758 - phase0 processRegistryUpdates	16.373 us/op	14.375 us/op	1.14
mainnet_e58758 - phase0 processSlashings	1.7080 us/op	1.1380 us/op	1.50
mainnet_e58758 - phase0 processEth1DataReset	1.7870 us/op	1.4870 us/op	1.20
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	2.6654 ms/op	2.2134 ms/op	1.20
mainnet_e58758 - phase0 processSlashingsReset	11.895 us/op	8.3770 us/op	1.42
mainnet_e58758 - phase0 processRandaoMixesReset	13.415 us/op	10.352 us/op	1.30
mainnet_e58758 - phase0 processHistoricalRootsUpdate	2.2130 us/op	1.7030 us/op	1.30
mainnet_e58758 - phase0 processParticipationRecordUpdates	11.659 us/op	9.9980 us/op	1.17
mainnet_e58758 - phase0 afterProcessEpoch	190.59 ms/op	158.37 ms/op	1.20
phase0 processEffectiveBalanceUpdates - 250000 normalcase	3.5534 ms/op	3.1584 ms/op	1.13
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	3.6322 ms/op	3.0133 ms/op	1.21
altair processInactivityUpdates - 250000 normalcase	31.538 ms/op	33.522 ms/op	0.94
altair processInactivityUpdates - 250000 worstcase	31.989 ms/op	27.810 ms/op	1.15
phase0 processRegistryUpdates - 250000 normalcase	16.510 us/op	14.698 us/op	1.12
phase0 processRegistryUpdates - 250000 badcase_full_deposits	613.00 us/op	609.62 us/op	1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5	263.76 ms/op	212.09 ms/op	1.24
altair processRewardsAndPenalties - 250000 normalcase	99.262 ms/op	111.94 ms/op	0.89
altair processRewardsAndPenalties - 250000 worstcase	141.77 ms/op	77.964 ms/op	1.82
phase0 getAttestationDeltas - 250000 normalcase	15.528 ms/op	13.135 ms/op	1.18
phase0 getAttestationDeltas - 250000 worstcase	15.400 ms/op	13.470 ms/op	1.14
phase0 processSlashings - 250000 worstcase	6.4650 ms/op	5.9979 ms/op	1.08
altair processSyncCommitteeUpdates - 250000	345.48 ms/op	261.90 ms/op	1.32
BeaconState.hashTreeRoot - No change	700.00 ns/op	433.00 ns/op	1.62
BeaconState.hashTreeRoot - 1 full validator	73.985 us/op	56.004 us/op	1.32
BeaconState.hashTreeRoot - 32 full validator	738.57 us/op	548.04 us/op	1.35
BeaconState.hashTreeRoot - 512 full validator	7.7589 ms/op	6.0415 ms/op	1.28
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	95.985 us/op	70.118 us/op	1.37
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	1.3135 ms/op	956.20 us/op	1.37
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	19.458 ms/op	12.978 ms/op	1.50
BeaconState.hashTreeRoot - 1 balances	77.068 us/op	53.911 us/op	1.43
BeaconState.hashTreeRoot - 32 balances	734.75 us/op	468.90 us/op	1.57
BeaconState.hashTreeRoot - 512 balances	6.2642 ms/op	4.6197 ms/op	1.36
BeaconState.hashTreeRoot - 250000 balances	102.60 ms/op	102.28 ms/op	1.00
aggregationBits - 2048 els - zipIndexesInBitList	43.995 us/op	24.424 us/op	1.80
regular array get 100000 times	80.998 us/op	67.397 us/op	1.20
wrappedArray get 100000 times	81.189 us/op	67.416 us/op	1.20
arrayWithProxy get 100000 times	35.478 ms/op	33.074 ms/op	1.07
ssz.Root.equals	652.00 ns/op	454.00 ns/op	1.44
byteArrayEquals	647.00 ns/op	443.00 ns/op	1.46
shuffle list - 16384 els	13.285 ms/op	10.956 ms/op	1.21
shuffle list - 250000 els	195.25 ms/op	162.83 ms/op	1.20
processSlot - 1 slots	16.631 us/op	10.304 us/op	1.61
processSlot - 32 slots	2.4508 ms/op	1.5612 ms/op	1.57
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	457.12 us/op	357.08 us/op	1.28
getCommitteeAssignments - req 1 vs - 250000 vc	6.3919 ms/op	5.2716 ms/op	1.21
getCommitteeAssignments - req 100 vs - 250000 vc	8.8620 ms/op	7.2795 ms/op	1.22
getCommitteeAssignments - req 1000 vs - 250000 vc	9.4349 ms/op	7.7877 ms/op	1.21
computeProposers - vc 250000	22.687 ms/op	16.138 ms/op	1.41
computeEpochShuffling - vc 250000	195.70 ms/op	164.80 ms/op	1.19
getNextSyncCommittee - vc 250000	338.03 ms/op	266.76 ms/op	1.27
pass gossip attestations to forkchoice per slot	7.0642 ms/op	3.2032 ms/op	2.21
computeDeltas	3.9485 ms/op	3.1904 ms/op	1.24
computeProposerBoostScoreFromBalances	1.1059 ms/op	907.62 us/op	1.22
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	2.3217 ms/op	1.9486 ms/op	1.19
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	96.225 us/op	71.691 us/op	1.34
BLS verify - blst-native	2.2288 ms/op	1.8639 ms/op	1.20
BLS verifyMultipleSignatures 3 - blst-native	4.5749 ms/op	3.8058 ms/op	1.20
BLS verifyMultipleSignatures 8 - blst-native	9.8329 ms/op	8.1812 ms/op	1.20
BLS verifyMultipleSignatures 32 - blst-native	35.685 ms/op	29.640 ms/op	1.20
BLS aggregatePubkeys 32 - blst-native	47.954 us/op	39.260 us/op	1.22
BLS aggregatePubkeys 128 - blst-native	184.38 us/op	153.52 us/op	1.20
getAttestationsForBlock	76.548 ms/op	61.371 ms/op	1.25
isKnown best case - 1 super set check	536.00 ns/op	440.00 ns/op	1.22
isKnown normal case - 2 super set checks	518.00 ns/op	433.00 ns/op	1.20
isKnown worse case - 16 super set checks	521.00 ns/op	427.00 ns/op	1.22
CheckpointStateCache - add get delete	14.416 us/op	10.333 us/op	1.40
validate gossip signedAggregateAndProof - struct	5.1854 ms/op	4.2624 ms/op	1.22
validate gossip attestation - struct	2.4338 ms/op	2.0284 ms/op	1.20
altair verifyImport mainnet_s3766816:31	7.5174 s/op	6.0918 s/op	1.23
pickEth1Vote - no votes	2.5459 ms/op	2.0544 ms/op	1.24
pickEth1Vote - max votes	30.842 ms/op	24.059 ms/op	1.28
pickEth1Vote - Eth1Data hashTreeRoot value x2048	14.980 ms/op	11.325 ms/op	1.32
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	25.485 ms/op	24.953 ms/op	1.02
pickEth1Vote - Eth1Data fastSerialize value x2048	1.9413 ms/op	1.6018 ms/op	1.21
pickEth1Vote - Eth1Data fastSerialize tree x2048	22.315 ms/op	12.672 ms/op	1.76
bytes32 toHexString	1.4000 us/op	972.00 ns/op	1.44
bytes32 Buffer.toString(hex)	880.00 ns/op	652.00 ns/op	1.35
bytes32 Buffer.toString(hex) from Uint8Array	1.1660 us/op	874.00 ns/op	1.33
bytes32 Buffer.toString(hex) + 0x	905.00 ns/op	650.00 ns/op	1.39
Object access 1 prop	0.46700 ns/op	0.32300 ns/op	1.45
Map access 1 prop	0.34400 ns/op	0.28600 ns/op	1.20
Object get x1000	21.932 ns/op	17.327 ns/op	1.27
Map get x1000	1.2150 ns/op	0.97800 ns/op	1.24
Object set x1000	142.96 ns/op	106.11 ns/op	1.35
Map set x1000	89.704 ns/op	64.688 ns/op	1.39
Return object 10000 times	0.45030 ns/op	0.36650 ns/op	1.23
Throw Error 10000 times	6.9745 us/op	5.9072 us/op	1.18
enrSubnets - fastDeserialize 64 bits	3.4400 us/op	2.3910 us/op	1.44
enrSubnets - ssz BitVector 64 bits	945.00 ns/op	706.00 ns/op	1.34
enrSubnets - fastDeserialize 4 bits	506.00 ns/op	352.00 ns/op	1.44
enrSubnets - ssz BitVector 4 bits	945.00 ns/op	681.00 ns/op	1.39
prioritizePeers score -10:0 att 32-0.1 sync 2-0	118.10 us/op	86.349 us/op	1.37
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	169.70 us/op	130.77 us/op	1.30
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	279.29 us/op	200.31 us/op	1.39
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	475.32 us/op	386.59 us/op	1.23
prioritizePeers score 0:0 att 64-1 sync 4-1	557.42 us/op	455.77 us/op	1.22
RateTracker 1000000 limit, 1 obj count per request	234.39 ns/op	178.32 ns/op	1.31
RateTracker 1000000 limit, 2 obj count per request	177.91 ns/op	133.09 ns/op	1.34
RateTracker 1000000 limit, 4 obj count per request	150.75 ns/op	109.12 ns/op	1.38
RateTracker 1000000 limit, 8 obj count per request	137.57 ns/op	97.839 ns/op	1.41
RateTracker with prune	5.8260 us/op	3.7700 us/op	1.55
array of 16000 items push then shift	3.8110 us/op	3.1619 us/op	1.21
LinkedList of 16000 items push then shift	29.159 ns/op	22.868 ns/op	1.28
array of 16000 items push then pop	278.57 ns/op	215.38 ns/op	1.29
LinkedList of 16000 items push then pop	23.837 ns/op	20.183 ns/op	1.18
array of 24000 items push then shift	5.5431 us/op	4.5605 us/op	1.22
LinkedList of 24000 items push then shift	29.608 ns/op	23.423 ns/op	1.26
array of 24000 items push then pop	257.31 ns/op	188.84 ns/op	1.36
LinkedList of 24000 items push then pop	23.887 ns/op	20.326 ns/op	1.18
intersect bitArray bitLen 8	13.938 ns/op	11.643 ns/op	1.20
intersect array and set length 8	221.76 ns/op	151.74 ns/op	1.46
intersect bitArray bitLen 128	86.650 ns/op	71.966 ns/op	1.20
intersect array and set length 128	2.7875 us/op	2.1033 us/op	1.33

by benchmarkbot/action

wemeetagain

lgtm

dapplion added 6 commits June 7, 2022 19:42

Update src

d315068

Update tests

2872a90

Update spec tests

f610571

Fix Lodestar src

459e8ea

Fix type issues

07316b2

Fix perf test types

8e08d4d

dapplion added 4 commits June 7, 2022 20:22

Review PR

f31edb7

Remove un-used import

5d74971

Don't check state root in computeNewStateRoot

e08ddaa

Rename slashValidator

cf0f995

dapplion marked this pull request as ready for review June 7, 2022 18:58

dapplion requested a review from a team as a code owner June 7, 2022 18:58

wemeetagain approved these changes Jun 7, 2022

View reviewed changes

wemeetagain merged commit fe5c98e into unstable Jun 7, 2022

wemeetagain deleted the dapplion/state-transition-flat branch June 7, 2022 19:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flat state transition across forks #4120

Flat state transition across forks #4120

dapplion commented Jun 7, 2022

github-actions bot commented Jun 7, 2022 •

edited

Loading

wemeetagain left a comment

Flat state transition across forks #4120

Flat state transition across forks #4120

Conversation

dapplion commented Jun 7, 2022

github-actions bot commented Jun 7, 2022 • edited Loading

Performance Report

wemeetagain left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 7, 2022 •

edited

Loading