Update approval-voting-regression-bench #5504
Conversation
The accepted divergence rate of 1/1000 is execesive and leads to false positives especially after #4772 and #5042, so let's increase it to 1/100 since we do have some randomness in the system and there is no point in being that strict. Signed-off-by: Alexandru Gheorghe <[email protected]>
|
0.001 was set deliberately, because we always sure that we send same amount of data every time. Current standard deviation for values after last change is 0.0005 with average 63999.3651. |
That's not really the case for approval-voting benchmarks because when the messages are generated they are not always getting the same seed, so you deterministic approach here some small deviation is acceptable and 0.001 seems to strict too me. |
I'm ok with 0.01, but I think we should change the base because it's not a base anymore. The base is more like 63999.3651. |
Signed-off-by: Alexandru Gheorghe <[email protected]>
Signed-off-by: Alexandru Gheorghe <[email protected]>
* master: (39 commits) short-term fix for para inherent weight overestimation (#5082) CI: Add backporting bot (#4795) Fix benchmark failures when using `insecure_zero_ed` flag (#5354) Command bot GHA v2 - /cmd <cmd> (#5457) Remove pallet::getter usage from treasury (#4962) Bump blake2b_simd from 1.0.1 to 1.0.2 (#5404) Bump rustversion from 1.0.14 to 1.0.17 (#5405) Bridge zombienet tests: remove old command (#5434) polkadot-parachain: Add omni-node variant with u64 block number (#5269) Refactor verbose test (#5506) Use umbrella crate for minimal template (#5155) IBP Coretime Polkadot bootnodes (#5499) rpc server: listen to `ipv6 socket` if available and `--experimental-rpc-endpoint` CLI option (#4792) Update approval-voting-regression-bench (#5504) change try-runtime rpc domains (#5443) polkadot-parachain-bin: Remove contracts parachain (#5471) Add feature to allow Aura collator to use full PoV size (#5393) Adding stkd bootnodes (#5470) Make `PendingConfigs` storage item public (#5467) frame-omni-bencher maintenance (#5466) ...
The accepted divergence rate of 1/1000 is excessive and leads to false positives especially after #4772 and #5042, so let's increase it to 1/100 since we do have some randomness in the system and there is no point in being that strict.
Fixes: #5463