Connection failed when Redis master down

[SandyXSD]

What happened:
Connection to Redis failed when master went down.

redis: 2022/03/17 05:37:39 sentinel.go:536: sentinel: GetMasterAddrByName name="redis" failed: dial tcp {old-master}:26379: i/o timeout
2022/03/17 05:37:39.428142 juicefs[7] <ERROR>: error: redis: connection pool timeout

What you expected to happen:
Automatically connect to the new master.

How to reproduce it (as minimally and precisely as possible):

• Redis sentinel mode
• Power down the Redis master node

Anything else we need to know?

Environment:

• JuiceFS version (use juicefs --version) or Hadoop Java SDK version: 1.0.0-beta1
• Metadata engine info (version, cloud provider managed or self maintained): Redis server v=6.2.6

[davies]

@SandyXSD we should update the docs on how to adjust these connection parameters.

[SandyXSD]

I'm afraid it's a too advanced topic.

[Nick-0314]

I added a proxy (k8S service or any other load balancer with health check) in front of Sentinel, which effectively solved the sentinel Timeout problem