Thanos ruler: Alertmanager v0.27.0 /api/v1 endpoints deprecated

[federicopires]

Thanos, Prometheus and Golang version used:

v0.34.1

Object Storage Provider:
AWS S3

What happened:
Thanos ruler is randomly failing to send alerts to Alertmanager v0.27.0 with bad response status 410 Gone

What you expected to happen:
No errors while sending alerts

How to reproduce it (as minimally and precisely as possible):
Install latest Alertmanager v0.27.0 (I'm using latest kube-prometheus-stack version with their implementation of thanos ruler) and Thanos ruler 0.34.1 setup to send alerts to it

Full logs to relevant components:
Thanos ruler:

err="bad response status 410 Gone from \"http://prometheus-kube-prometheus-alertmanager.monitoring.svc.cluster.local:9093/api/v1/alerts

Alertmanager:
ts=2024-03-13T14:32:22.436Z caller=v1_deprecation_router.go:51 level=warn component=api version=v1 msg="v1 API received a request on a removed endpoint" path=/api/v1/alerts method=POST

Anything else we need to know:
Related to prometheus/alertmanager#2970

[jacobbaungard]

v2 will be default by the next release #7123 . Until then you can change the API version in the alertmanager config: https://thanos.io/v0.34/components/rule.md/#configuration

[MichaHoffmann]

This is fixed by upgrading to a recent thanos version, Ill close this; please reopen if I missed something!