What is the proper way to setup autoscaling for receiver? #5546
Unanswered
cybervedaa
asked this question in
Questions & Answers
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have deployed Thanos receiver in a router-ingestor setup. I have also deployed thanos-receive-controller to automatically add/remove endpoints to hashrings.json.
I have setup a horizontal pod autoscaler to scale up the ingestor hashring when memory utilization exceeds 6Gib. All components work as excepted, till a scale out event or a scale in event happens.
To help with 1. I even tried adding a component to automatically restart the router deployment as soon as the hashrings configmap was updated by thanos-receiver-controller. This is to ensure all router pods have the latest hashrings.conf. But I still the http 500 errors.
I am looking for guidance on how to avoid these two problems.
Beta Was this translation helpful? Give feedback.
All reactions