test: add performance benchmarks for the phase0 block attributes #6145
Conversation
nazarhussain
force-pushed
the
nh/5793-phase0-performance
branch
from
cd9c65e
to
68611d2
Compare
| executionBlockProductionTimeSteps: register.histogram<"step">({ | ||
| name: "beacon_block_production_execution_steps_seconds", | ||
| help: "Detailed steps runtime of execution block production", | ||
| buckets: [0.1, 1, 2, 4, 10], |
There was a problem hiding this comment.
buckets value seems to big for each step. If a step takes more than 1s then it's already an issue so I suggest removing 2, 4, 10 and track more < 1 buckets
There was a problem hiding this comment.
I removed the buckets completely, as these are not intended in the metric graph we need.
|
As the failed perf tests are not related to the code changes in this PR, so why merging it. |
| name: "beacon_block_production_execution_steps_seconds", | ||
| help: "Detailed steps runtime of execution block production", | ||
| buckets: [0.01, 0.1, 0.2, 0.5, 1], | ||
| /** |
There was a problem hiding this comment.
I feel like commenting those values here is not the right approach. We should use an enum in such cases as we did for block production source.
Might be worth to look into how we could make label values type safe.
There was a problem hiding this comment.
We have to change the types from pom-client for that. That's a bigger task to do. Feel free to create a separate issue to track in future.
@nazarhussain are you sure about that? it failed with same errors again on unstable benchmark run https://github.com/ChainSafe/lodestar/actions/runs/7103589338/job/19336631592 while before it was consistently passing. |
Some of 'state-transition' perf tests were failing earlier. And this PR didn't changed any code for that package. Will look on unstable branch tests in some time. |
Are you sure? Please investigate |
* Add opPool benchmark * Add benchmark for empty block body * Optimize a skipping condition * Fix the identifier for a graph * Add detailed steps progress for block production * Update the usage of new metric * Add block produciton test code * Add dashboard panels for the block processing * Revert test run code * Fix a type in perf test * Fix dashboard * Fix the code feedback * Convert multi-label metric to single label * Remove the buckets * Add special bucket values
|
🎉 This PR is included in v1.13.0 🎉 |
Motivation
Observe the performance regressions.
Description
Add a base performance benchmark for phase0 block attributes.
Added new metric and a graph, which will helps us to identify any issue with the individual processing steps.
part of #5793
Steps to test or reproduce
Run all tests.