Vulture does not evenly distribute search space #1760
Labels
good first issue
Good for newcomers
Comments
rfratto
added a commit
to rfratto/tempo
that referenced
this issue
Sep 27, 2022
grafana#1760 noted that creating a new rand.Rand every time a random number is generated causes a poor distrubution of the search space, where numbers appear 7x more frequently than they do when using a shared rand.Rand instance. This is demonstrated using the following Go playground link: https://go.dev/play/p/EkhINRfOO5p This commit changes Vulture to use a shared rand.Rand instance instead of creating a new one each time a search or read is performed.
joe-elliott
pushed a commit
that referenced
this issue
Oct 5, 2022
…1763) * cmd/tempo-vulture: share rand.Rand instance across multiple searches #1760 noted that creating a new rand.Rand every time a random number is generated causes a poor distrubution of the search space, where numbers appear 7x more frequently than they do when using a shared rand.Rand instance. This is demonstrated using the following Go playground link: https://go.dev/play/p/EkhINRfOO5p This commit changes Vulture to use a shared rand.Rand instance instead of creating a new one each time a search or read is performed. * update CHANGELOG
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Vulture currently creates a new random number generator every time it performs a search. The random number generator is then used to generate a random number between two values (a min timestamp and a max timestamp).
However, until Vulture has been alive for two weeks (specified by the
--tempo-retention-durationfield), the seed used for the random number generator will only change once. The seed will initially be set to the process start time, and then will change to the process start time rounded to the nearest--tempo-read-backoff-duration. After this point, it will stay on that value until the retention duration period has elapsed.Using the same seed leads to search distribution issues. This Go playground link demonstrates that always creating a new RNG with the same seed will repeat numbers seven times more often over sharing a single RNG for all searches.
The text was updated successfully, but these errors were encountered: