Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove SSZ from params #3651

Merged
merged 4 commits into from
Jan 22, 2022
Merged

Remove SSZ from params #3651

merged 4 commits into from
Jan 22, 2022

Conversation

dapplion
Copy link
Contributor

Motivation

Currently config and params use SSZ to define the type of its values. This approach is sub-optimal because:

  • Config API is too strict, some node may return not all values and SSZ will throw an Error, breaking the route
  • CLI doesn't do enough verification, we may want to validate that any additional params are not ignored
  • SSZ is overkill, since only the toJson fromJson methods are used, not merkleization bytes serialization etc

Description

  • Remove SSZ from params

@codeclimate
Copy link

codeclimate bot commented Jan 21, 2022

Code Climate has analyzed commit 7180c6f and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 1

View more on Code Climate.

@codecov
Copy link

codecov bot commented Jan 21, 2022

Codecov Report

Merging #3651 (7180c6f) into master (f9cb593) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3651      +/-   ##
==========================================
- Coverage   37.16%   37.16%   -0.01%     
==========================================
  Files         321      321              
  Lines        8696     8689       -7     
  Branches     1348     1346       -2     
==========================================
- Hits         3232     3229       -3     
+ Misses       5319     5317       -2     
+ Partials      145      143       -2     

@github-actions
Copy link
Contributor

github-actions bot commented Jan 21, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 2534af3 Previous: f9cb593 Ratio
BeaconState.hashTreeRoot - No change 521.00 ns/op 609.00 ns/op 0.86
BeaconState.hashTreeRoot - 1 full validator 137.26 us/op 159.44 us/op 0.86
BeaconState.hashTreeRoot - 32 full validator 2.0902 ms/op 2.3704 ms/op 0.88
BeaconState.hashTreeRoot - 512 full validator 27.173 ms/op 31.651 ms/op 0.86
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 141.69 us/op 175.12 us/op 0.81
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.1972 ms/op 2.4974 ms/op 0.88
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 29.268 ms/op 33.576 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 98.690 us/op 108.47 us/op 0.91
BeaconState.hashTreeRoot - 32 balances 793.55 us/op 986.89 us/op 0.80
BeaconState.hashTreeRoot - 512 balances 7.5200 ms/op 8.9634 ms/op 0.84
BeaconState.hashTreeRoot - 250000 balances 147.94 ms/op 173.64 ms/op 0.85
processSlot - 1 slots 59.805 us/op 63.504 us/op 0.94
processSlot - 32 slots 3.3948 ms/op 3.8824 ms/op 0.87
getCommitteeAssignments - req 1 vs - 250000 vc 5.1252 ms/op 4.9995 ms/op 1.03
getCommitteeAssignments - req 100 vs - 250000 vc 6.6218 ms/op 7.1395 ms/op 0.93
getCommitteeAssignments - req 1000 vs - 250000 vc 7.0956 ms/op 7.6830 ms/op 0.92
computeProposers - vc 250000 21.990 ms/op 23.805 ms/op 0.92
computeEpochShuffling - vc 250000 175.69 ms/op 192.56 ms/op 0.91
getNextSyncCommittee - vc 250000 371.94 ms/op 393.42 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei normalcase 55.669 ms/op 48.600 ms/op 1.15
altair processAttestation - 250000 vs - 7PWei worstcase 46.647 ms/op 47.806 ms/op 0.98
altair processAttestation - setStatus - 1/6 committees join 11.227 ms/op 13.250 ms/op 0.85
altair processAttestation - setStatus - 1/3 committees join 24.331 ms/op 28.716 ms/op 0.85
altair processAttestation - setStatus - 1/2 committees join 37.370 ms/op 43.438 ms/op 0.86
altair processAttestation - setStatus - 2/3 committees join 50.065 ms/op 59.659 ms/op 0.84
altair processAttestation - setStatus - 4/5 committees join 59.312 ms/op 69.598 ms/op 0.85
altair processAttestation - setStatus - 100% committees join 82.808 ms/op 90.314 ms/op 0.92
altair processAttestation - updateEpochParticipants - 1/6 committees join 16.646 ms/op 14.648 ms/op 1.14
altair processAttestation - updateEpochParticipants - 1/3 committees join 24.842 ms/op 30.095 ms/op 0.83
altair processAttestation - updateEpochParticipants - 1/2 committees join 31.296 ms/op 33.499 ms/op 0.93
altair processAttestation - updateEpochParticipants - 2/3 committees join 28.460 ms/op 28.366 ms/op 1.00
altair processAttestation - updateEpochParticipants - 4/5 committees join 27.676 ms/op 29.527 ms/op 0.94
altair processAttestation - updateEpochParticipants - 100% committees join 29.772 ms/op 30.929 ms/op 0.96
altair processAttestation - updateAllStatus 21.111 ms/op 22.470 ms/op 0.94
altair processBlock - 250000 vs - 7PWei normalcase 53.188 ms/op 52.853 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase 124.39 ms/op 145.40 ms/op 0.86
altair processEpoch - mainnet_e81889 1.0561 s/op 1.1122 s/op 0.95
mainnet_e81889 - altair beforeProcessEpoch 281.00 ms/op 293.17 ms/op 0.96
mainnet_e81889 - altair processJustificationAndFinalization 118.89 us/op 127.37 us/op 0.93
mainnet_e81889 - altair processInactivityUpdates 16.228 ms/op 18.736 ms/op 0.87
mainnet_e81889 - altair processRewardsAndPenalties 183.90 ms/op 194.97 ms/op 0.94
mainnet_e81889 - altair processRegistryUpdates 21.682 us/op 23.212 us/op 0.93
mainnet_e81889 - altair processSlashings 6.4420 us/op 5.7900 us/op 1.11
mainnet_e81889 - altair processEth1DataReset 6.6270 us/op 6.4380 us/op 1.03
mainnet_e81889 - altair processEffectiveBalanceUpdates 11.423 ms/op 12.646 ms/op 0.90
mainnet_e81889 - altair processSlashingsReset 37.939 us/op 40.261 us/op 0.94
mainnet_e81889 - altair processRandaoMixesReset 39.272 us/op 49.903 us/op 0.79
mainnet_e81889 - altair processHistoricalRootsUpdate 9.6120 us/op 11.999 us/op 0.80
mainnet_e81889 - altair processParticipationFlagUpdates 106.18 ms/op 103.20 ms/op 1.03
mainnet_e81889 - altair processSyncCommitteeUpdates 5.6490 us/op 6.0910 us/op 0.93
mainnet_e81889 - altair afterProcessEpoch 213.09 ms/op 231.37 ms/op 0.92
altair processInactivityUpdates - 250000 normalcase 75.025 ms/op 112.81 ms/op 0.67
altair processInactivityUpdates - 250000 worstcase 73.936 ms/op 120.61 ms/op 0.61
altair processParticipationFlagUpdates - 250000 anycase 120.62 ms/op 108.71 ms/op 1.11
altair processRewardsAndPenalties - 250000 normalcase 128.52 ms/op 154.48 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 125.09 ms/op 148.87 ms/op 0.84
altair processSyncCommitteeUpdates - 250000 419.28 ms/op 410.66 ms/op 1.02
Tree 40 250000 create 877.86 ms/op 1.0024 s/op 0.88
Tree 40 250000 get(125000) 287.44 ns/op 340.44 ns/op 0.84
Tree 40 250000 set(125000) 1.8808 us/op 2.2930 us/op 0.82
Tree 40 250000 toArray() 46.326 ms/op 41.475 ms/op 1.12
Tree 40 250000 iterate all - toArray() + loop 47.837 ms/op 48.289 ms/op 0.99
Tree 40 250000 iterate all - get(i) 113.14 ms/op 126.88 ms/op 0.89
MutableVector 250000 create 18.908 ms/op 27.334 ms/op 0.69
MutableVector 250000 get(125000) 12.735 ns/op 14.675 ns/op 0.87
MutableVector 250000 set(125000) 483.36 ns/op 695.34 ns/op 0.70
MutableVector 250000 toArray() 7.9463 ms/op 8.8668 ms/op 0.90
MutableVector 250000 iterate all - toArray() + loop 7.5118 ms/op 8.9496 ms/op 0.84
MutableVector 250000 iterate all - get(i) 2.9845 ms/op 3.6344 ms/op 0.82
Array 250000 create 5.7489 ms/op 5.1774 ms/op 1.11
Array 250000 clone - spread 1.6083 ms/op 2.2036 ms/op 0.73
Array 250000 get(125000) 0.84000 ns/op 1.1150 ns/op 0.75
Array 250000 set(125000) 0.85500 ns/op 1.1330 ns/op 0.75
Array 250000 iterate all - loop 120.40 us/op 132.94 us/op 0.91
aggregationBits - 2048 els - readonlyValues 194.21 us/op 237.80 us/op 0.82
aggregationBits - 2048 els - zipIndexesInBitList 32.011 us/op 40.075 us/op 0.80
regular array get 100000 times 48.055 us/op 52.234 us/op 0.92
wrappedArray get 100000 times 49.822 us/op 54.583 us/op 0.91
arrayWithProxy get 100000 times 31.940 ms/op 33.350 ms/op 0.96
ssz.Root.equals 1.0360 us/op 1.1170 us/op 0.93
ssz.Root.equals with valueOf() 1.1950 us/op 1.4920 us/op 0.80
byteArrayEquals with valueOf() 1.1450 us/op 1.4010 us/op 0.82
phase0 processBlock - 250000 vs - 7PWei normalcase 12.037 ms/op 12.966 ms/op 0.93
phase0 processBlock - 250000 vs - 7PWei worstcase 85.321 ms/op 104.18 ms/op 0.82
phase0 afterProcessEpoch - 250000 vs - 7PWei 190.11 ms/op 215.66 ms/op 0.88
phase0 beforeProcessEpoch - 250000 vs - 7PWei 624.76 ms/op 750.79 ms/op 0.83
phase0 processEpoch - mainnet_e58758 848.67 ms/op 935.25 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 565.52 ms/op 525.64 ms/op 1.08
mainnet_e58758 - phase0 processJustificationAndFinalization 116.42 us/op 120.67 us/op 0.96
mainnet_e58758 - phase0 processRewardsAndPenalties 101.21 ms/op 127.90 ms/op 0.79
mainnet_e58758 - phase0 processRegistryUpdates 87.394 us/op 137.21 us/op 0.64
mainnet_e58758 - phase0 processSlashings 7.6810 us/op 8.2050 us/op 0.94
mainnet_e58758 - phase0 processEth1DataReset 6.8100 us/op 7.0950 us/op 0.96
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 10.354 ms/op 10.974 ms/op 0.94
mainnet_e58758 - phase0 processSlashingsReset 30.761 us/op 33.054 us/op 0.93
mainnet_e58758 - phase0 processRandaoMixesReset 41.870 us/op 45.157 us/op 0.93
mainnet_e58758 - phase0 processHistoricalRootsUpdate 7.3880 us/op 8.8080 us/op 0.84
mainnet_e58758 - phase0 processParticipationRecordUpdates 27.731 us/op 25.242 us/op 1.10
mainnet_e58758 - phase0 afterProcessEpoch 174.38 ms/op 189.61 ms/op 0.92
phase0 processEffectiveBalanceUpdates - 250000 normalcase 10.814 ms/op 12.409 ms/op 0.87
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6080 s/op 1.5396 s/op 1.04
phase0 processRegistryUpdates - 250000 normalcase 91.446 us/op 92.552 us/op 0.99
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.4371 ms/op 3.8679 ms/op 0.89
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.1063 s/op 2.3237 s/op 0.91
phase0 getAttestationDeltas - 250000 normalcase 35.703 ms/op 39.007 ms/op 0.92
phase0 getAttestationDeltas - 250000 worstcase 34.013 ms/op 36.754 ms/op 0.93
phase0 processSlashings - 250000 worstcase 38.131 ms/op 43.583 ms/op 0.87
shuffle list - 16384 els 12.200 ms/op 13.468 ms/op 0.91
shuffle list - 250000 els 169.76 ms/op 190.67 ms/op 0.89
getEffectiveBalances - 250000 vs - 7PWei 10.391 ms/op 11.390 ms/op 0.91
pass gossip attestations to forkchoice per slot 17.204 ms/op 17.853 ms/op 0.96
computeDeltas 2.9961 ms/op 3.7698 ms/op 0.79
computeProposerBoostScoreFromBalances 246.19 us/op 272.89 us/op 0.90
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.2832 ms/op 2.3276 ms/op 0.98
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 645.04 us/op 712.70 us/op 0.91
BLS verify - blst-native 1.8462 ms/op 2.6380 ms/op 0.70
BLS verifyMultipleSignatures 3 - blst-native 4.1918 ms/op 5.2770 ms/op 0.79
BLS verifyMultipleSignatures 8 - blst-native 8.4594 ms/op 10.713 ms/op 0.79
BLS verifyMultipleSignatures 32 - blst-native 30.890 ms/op 39.395 ms/op 0.78
BLS aggregatePubkeys 32 - blst-native 40.335 us/op 52.204 us/op 0.77
BLS aggregatePubkeys 128 - blst-native 157.39 us/op 206.41 us/op 0.76
getAttestationsForBlock 83.356 ms/op 99.368 ms/op 0.84
CheckpointStateCache - add get delete 23.702 us/op 21.242 us/op 1.12
validate gossip signedAggregateAndProof - struct 4.6465 ms/op 5.9653 ms/op 0.78
validate gossip signedAggregateAndProof - treeBacked 4.6213 ms/op 5.9978 ms/op 0.77
validate gossip attestation - struct 2.2011 ms/op 2.9106 ms/op 0.76
validate gossip attestation - treeBacked 2.3375 ms/op 2.8667 ms/op 0.82
bytes32 toHexString 1.4480 us/op 2.0270 us/op 0.71
bytes32 Buffer.toString(hex) 650.00 ns/op 710.00 ns/op 0.92
bytes32 Buffer.toString(hex) from Uint8Array 892.00 ns/op 1.0020 us/op 0.89
bytes32 Buffer.toString(hex) + 0x 683.00 ns/op 749.00 ns/op 0.91
Object access 1 prop 0.29300 ns/op 0.36800 ns/op 0.80
Map access 1 prop 0.27800 ns/op 0.31600 ns/op 0.88
Object get x1000 14.068 ns/op 15.975 ns/op 0.88
Map get x1000 0.80000 ns/op 0.94900 ns/op 0.84
Object set x1000 81.090 ns/op 103.90 ns/op 0.78
Map set x1000 62.713 ns/op 68.768 ns/op 0.91
Return object 10000 times 0.36820 ns/op 0.38690 ns/op 0.95
Throw Error 10000 times 6.1753 us/op 6.4811 us/op 0.95
enrSubnets - fastDeserialize 64 bits 1.2170 us/op 1.3970 us/op 0.87
enrSubnets - ssz BitVector 64 bits 15.982 us/op 17.354 us/op 0.92
enrSubnets - fastDeserialize 4 bits 438.00 ns/op 487.00 ns/op 0.90
enrSubnets - ssz BitVector 4 bits 3.1350 us/op 3.2940 us/op 0.95
RateTracker 1000000 limit, 1 obj count per request 172.83 ns/op 186.06 ns/op 0.93
RateTracker 1000000 limit, 2 obj count per request 117.24 ns/op 138.24 ns/op 0.85
RateTracker 1000000 limit, 4 obj count per request 97.380 ns/op 112.53 ns/op 0.87
RateTracker 1000000 limit, 8 obj count per request 85.458 ns/op 107.44 ns/op 0.80
RateTracker with prune 3.5750 us/op 4.4640 us/op 0.80

by benchmarkbot/action

Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structure lgtm

/**
* Run-time chain configuration
*/
export interface IChainConfig {
export type IChainConfig = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change to type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise Typescript doesnt allow this to be used as a Record<string,unknown> argument in functions

};

export const chainConfigTypes: SpecTypes<IChainConfig> = {
PRESET_BASE: "string",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these values be enum? or overkill?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overkill? This whole approach will be review soon anyway to make parsing im the validator client and cli more resilient

@@ -23,7 +25,7 @@ presetStatus.frozen = true;
*
* The active preset can be manually overridden with `setActivePreset`
*/
export const ACTIVE_PRESET: PresetName =
export const ACTIVE_PRESET =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this type now inferred to be PresetName?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently it's not needed, I can't think of a reason why it would be necessary. Not sure if it's a legacy forgotten though or an old uncommented reason

@@ -16,7 +15,7 @@ export interface IPhase0Preset {
SAFE_SLOTS_TO_UPDATE_JUSTIFIED: number;

// Gwei Values
MIN_DEPOSIT_AMOUNT: bigint;
MIN_DEPOSIT_AMOUNT: number;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't used anywhere?! I guess its hard to argue against changing it to a number

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, not used anywhere 🤷


export const bellatrix: IBellatrixPreset = {
/* eslint-disable @typescript-eslint/naming-convention */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

? is this needed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes to allow having all caps

import {PresetName} from "@chainsafe/lodestar-params";

/* eslint-disable @typescript-eslint/naming-convention */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes to allow having all caps

g11tech
g11tech previously approved these changes Jan 22, 2022
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ❤️ , as discussed on discord, can/will do a followup PR to fill config for beacon <> validator interop and to match preset more deeply 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants