Benchmark memory

[dapplion]

Motivation

Get data on the memory cost of various data structures relevant to SSZ and Lodestar.

This benchmarks have been run in a Linux laptop, Max OS laptop and Linux VPS getting almost exact results. So the method testRunnerMemory() appears to be very reliably for not huge objects. Once we want to measure the whole beacon state, it will need a different strategy.

Description

Benchmark memory of various SSZ and Lodestar data structures

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: bdcf6d7	Previous: 041c8dd	Ratio
bitlist bytes to struct (120,90)	40.013 us/op	30.979 us/op	1.29
bitlist bytes to tree (120,90)	3.1180 us/op	2.2790 us/op	1.37
bitlist bytes to struct (2048,2048)	672.73 us/op	521.99 us/op	1.29
bitlist bytes to tree (2048,2048)	9.3950 us/op	6.9410 us/op	1.35
ByteListType - deserialize	4.8433 ms/op	3.8651 ms/op	1.25
BasicListType - deserialize	181.74 ms/op	132.41 ms/op	1.37
ByteListType - serialize	59.102 ms/op	44.075 ms/op	1.34
BasicListType - serialize	59.197 ms/op	44.055 ms/op	1.34
ByteListType - tree_convertToStruct	195.55 ms/op	147.89 ms/op	1.32
BasicListType - tree_convertToStruct	861.30 ms/op	581.48 ms/op	1.48
deserialize Attestation - struct	54.051 us/op	38.361 us/op	1.41
deserialize Attestation - tree	22.622 us/op	12.966 us/op	1.74
deserialize SignedAggregateAndProof - struct	67.431 us/op	48.926 us/op	1.38
deserialize SignedAggregateAndProof - tree	31.399 us/op	18.401 us/op	1.71
deserialize SignedContributionAndProof - struct	52.174 us/op	38.639 us/op	1.35
epochParticipation len 250000 rws 7813	28.000 ms/op	23.926 ms/op	1.17
Array - for of	10.086 us/op	10.542 us/op	0.96
Array - for(;;)	9.8640 us/op	7.1800 us/op	1.37
basicListValue.readonlyValues()	233.99 ms/op	201.30 ms/op	1.16
basicListValue.readonlyValuesArray()	205.95 ms/op	175.17 ms/op	1.18
basicListValue.readonlyValues() + iterate all	222.93 ms/op	187.11 ms/op	1.19
basicListValue.readonlyValuesArray() + loop all	206.58 ms/op	175.00 ms/op	1.18
compositeListValue.readonlyValues()	296.79 ms/op	216.01 ms/op	1.37
compositeListValue.readonlyValuesArray()	302.38 ms/op	226.25 ms/op	1.34
compositeListValue.readonlyValues() + iterate all	203.03 ms/op	138.94 ms/op	1.46
compositeListValue.readonlyValuesArray() + loop all	277.82 ms/op	210.45 ms/op	1.32
NumberUintType - get balances list	891.39 ms/op	696.70 ms/op	1.28
Number64UintType - get balances list	539.78 ms/op	442.20 ms/op	1.22
NumberUintType - set balances list	1.3837 s/op	1.0305 s/op	1.34
Number64UintType - set balances list	1.2265 s/op	805.94 ms/op	1.52
Number64UintType - get and increase 10 then set	1.6709 s/op	1.2304 s/op	1.36
Number64UintType - increase 10 using applyDelta	576.03 ms/op	486.02 ms/op	1.19
Number64UintType - increase 10 using applyDeltaInBatch	588.79 ms/op	506.18 ms/op	1.16
tree_newTreeFromUint64Deltas	61.219 ms/op	53.666 ms/op	1.14
unsafeUint8ArrayToTree	58.476 ms/op	49.303 ms/op	1.19
Container {a,b,vec} - as struct x100000	73.384 us/op	59.763 us/op	1.23
Container {a,b,vec} - as tree x100000	106.01 ms/op	81.071 ms/op	1.31
Container {a,vec,b} - as struct x100000	111.75 us/op	89.009 us/op	1.26
Container {a,vec,b} - as tree x100000	116.16 ms/op	81.683 ms/op	1.42
Simple object binary -> struct	2.5480 us/op	1.9020 us/op	1.34
Simple object binary -> tree_backed	3.0310 us/op	2.2210 us/op	1.36
Simple object struct -> tree_backed	2.3310 us/op	1.6140 us/op	1.44
Simple object tree_backed -> struct	2.2550 us/op	1.5560 us/op	1.45
Simple object struct -> binary	2.6610 us/op	2.0540 us/op	1.30
Simple object tree_backed -> binary	4.1910 us/op	3.0600 us/op	1.37
aggregationBits binary -> struct	44.877 us/op	29.496 us/op	1.52
aggregationBits binary -> tree_backed	2.1660 us/op	1.6240 us/op	1.33
aggregationBits struct -> tree_backed	6.5080 us/op	4.4610 us/op	1.46
aggregationBits tree_backed -> struct	47.583 us/op	31.588 us/op	1.51
aggregationBits struct -> binary	4.8130 us/op	3.4350 us/op	1.40
aggregationBits tree_backed -> binary	5.8780 us/op	4.5940 us/op	1.28
List(BigInt) 1000 binary -> struct	465.06 us/op	330.42 us/op	1.41
List(BigInt) 1000 binary -> tree_backed	320.01 us/op	215.28 us/op	1.49
List(BigInt) 1000 struct -> tree_backed	427.30 us/op	256.96 us/op	1.66
List(BigInt) 1000 tree_backed -> struct	1.1015 ms/op	790.91 us/op	1.39
List(BigInt) 1000 struct -> binary	242.64 us/op	164.00 us/op	1.48
List(BigInt) 1000 tree_backed -> binary	60.060 us/op	38.912 us/op	1.54
List(BigInt) 10000 binary -> struct	4.6794 ms/op	3.3598 ms/op	1.39
List(BigInt) 10000 binary -> tree_backed	3.4909 ms/op	2.7368 ms/op	1.28
List(BigInt) 10000 struct -> tree_backed	4.0241 ms/op	2.6747 ms/op	1.50
List(BigInt) 10000 tree_backed -> struct	12.594 ms/op	9.3951 ms/op	1.34
List(BigInt) 10000 struct -> binary	2.3179 ms/op	1.6132 ms/op	1.44
List(BigInt) 10000 tree_backed -> binary	456.70 us/op	367.59 us/op	1.24
List(BigInt) 100000 binary -> struct	56.856 ms/op	44.425 ms/op	1.28
List(BigInt) 100000 binary -> tree_backed	52.185 ms/op	43.700 ms/op	1.19
List(BigInt) 100000 struct -> tree_backed	50.032 ms/op	35.948 ms/op	1.39
List(BigInt) 100000 tree_backed -> struct	158.02 ms/op	133.00 ms/op	1.19
List(BigInt) 100000 struct -> binary	22.885 ms/op	18.092 ms/op	1.26
List(BigInt) 100000 tree_backed -> binary	5.1248 ms/op	4.1192 ms/op	1.24
struct - increase slot to 1000000	3.1343 ms/op	2.6764 ms/op	1.17
NumberUintType - increase slot to 1000000	1.3567 s/op	1.1050 s/op	1.23
Number64UintType - increase slot to 1000000	529.80 ms/op	322.20 ms/op	1.64
get_exitEpoch - ContainerType	942.00 ns/op	705.00 ns/op	1.34
get_exitEpoch - ContainerLeafNodeStructType	705.00 ns/op	523.00 ns/op	1.35
set_exitEpoch - ContainerType	918.00 ns/op	763.00 ns/op	1.20
set_exitEpoch - ContainerLeafNodeStructType	1.2040 us/op	842.00 ns/op	1.43
get_pubkey - ContainerType	2.5580 us/op	1.6660 us/op	1.54
get_pubkey - ContainerLeafNodeStructType	686.00 ns/op	503.00 ns/op	1.36
hashTreeRoot - ContainerType	859.00 ns/op	637.00 ns/op	1.35
hashTreeRoot - ContainerLeafNodeStructType	923.00 ns/op	656.00 ns/op	1.41
createProof - ContainerType	9.9050 us/op	6.0420 us/op	1.64
createProof - ContainerLeafNodeStructType	41.951 us/op	22.398 us/op	1.87
serialize - ContainerType	7.2240 us/op	5.0530 us/op	1.43
serialize - ContainerLeafNodeStructType	4.3910 us/op	2.9650 us/op	1.48
set_exitEpoch_and_hashTreeRoot - ContainerType	11.896 us/op	6.3700 us/op	1.87
set_exitEpoch_and_hashTreeRoot - ContainerLeafNodeStructType	42.492 us/op	22.640 us/op	1.88

by benchmarkbot/action