The correct behavior of load balancer algorithms depend on optional stats

[brucedlg]

Issue Template

Title: One line description
The correct behavior of load balancer algorithms, in particular, unweighted round-robin and unweighted least-request (P2C), depend on optional stats value

Description:
When testing istio default configuration, load-balancer algorithms for unweighted round-robin and unweighted least-request behave like weighted round-robin with all weight 1.

Troubleshooting shows the root cause is due to two stats are not enabled:

• max_host_weight in EdfLoadBalancerBase::chooseHostOnce()
  https://github.com/envoyproxy/envoy/blob/master/source/common/upstream/load_balancer_impl.cc#L692

• rq_active in LeastRequestLoadBalancer::hostWeight()
  https://github.com/envoyproxy/envoy/blob/master/source/common/upstream/load_balancer_impl.h#L443

Since these stats are service/cluster specific, it's unreasonable asking control plane installation (e.g. istio) to turn them on by default, unless blindly turning on all cluster.outbound stats.

Ideally it's nice for Envoy to remove the dependency of call logic to the optional stats, say, saving needed info in some other instance variables in addition to stats. Then the call logic is no longer depending on the enabling or disabling of certain stats.

If too hard to do, it's suggested to add clarification in LB documentation. Current doc does not describe dependency of LB to these stats.

[optional Relevant Links:]

[mattklein123]

Marking help wanted. I agree we can/should do better here.

[antoniovicente]

I think the current issue is related to max_host_weight so #7877 should fix the issue.

About rq_active, I think there is currently no way to disable host stats. In the future we may want to add a way to disable or otherwise reduce the memory burden of those stats.

[mattklein123]

Fixed by #7877