Tuning and combined improvements for perf and observability #1493
Conversation
AhmedSoliman
changed the title
There was a problem hiding this comment.
Thanks a lot for creating this PR @AhmedSoliman. The changes look good to me. I think the stall detector will be really useful :-) Also the additional performance metrics are super helpful.
My main question is how much the different RocksDB users are already respecting the calculated write buffer sizes in RocksDbOptions and whether there are planned follow-ups. Apart from this, +1 for merging.
| let common_opts = CommonOptions::default(); | ||
| let all_memtables = common_opts.rocksdb_total_memtables_size(); | ||
| let num_partitions = WorkerOptions::default().bootstrap_num_partitions(); |
There was a problem hiding this comment.
I guess these numbers should not be taken from the defaults but from the actually configured values, right?
There was a problem hiding this comment.
Those values calculate the default value. I'll add a comment explaining that, but in a possible future, those will be calculated dynamically based on the actual system values (i.e. the current number of configured partitions). We cannot rely on bootstrap_num_partitions() because the system can diverge from this value after initial bootstrap.
Tuning and combined improvements for perf and observability
Stack created with Sapling. Best reviewed with ReviewStack.