[BUG] Caching avg total bytes and avg free bytes inside ClusterInfo #15113
Labels
bug
Something isn't working
Indexing:Replication
Issues and PRs related to core replication framework eg segrep
untriaged
Describe the bug
Description
In case disk usage of a particular node is not populated inside ClusterInfo (due to an underlying issue), we use the average usage for all nodes as the usage for this node inside DiskThresholdDecider. For calculating this value, right now we are iterating through all the diskUsages of every node inside DiskThresholdDecider to calculate total sum and divide by the total number of nodes. Since this computation is performed for each node during shard allocation, it becomes computationally expensive.
Related component
Indexing:Replication
To Reproduce
Create 500k shards on a setup with 1000 data nodes and 3 master nodes.
Expected behavior
Do an offline calculation of Average total bytes and Average free bytes and cache it inside
ClusterInfo
.Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: