Support LoadBalance mode in providers

[zhenlan]

Background

App Configuration providers can failover to replicas if current connected endpoint experiences failures, but it will stay with the current endpoint if everything is running well. In that case, applications are not taking advantage of the replicas for higher volume/scalability.

Proposal

Add LoadBalance mode to providers in addition to the FailOver mode. In the LoadBalance mode, the provider will distribute traffic evenly to all replicas (including auto-discovered ones if not disabled).

• The FailOver mode remains the default, but users can opt-in to the LoadBalance mode.
• The FailOver still occurs even under LoadBalance mode.

Tasks

Beta Give feedback

1. .NET provider
2. Java Spring provider
3. Python provider
4. JavaScript provider
5. Kubernetes provider

cc: @drago-draganov @jimmyca15

[RichardChen820]

Do we treat all replicas (including user-provided and auto-discovered) evenly? Just round-robin between them when LoadBalance is enabled.

[zhenlan]

Do we treat all replicas (including user-provided and auto-discovered) evenly?

Yes, but users can disable replica auto-discovery.

Just round-robin between them when LoadBalance is enabled.

There could be other strategies. I'll leave this to later discussions.