HA Egress for AWS EKS - Include functionality to assign Egress IPs as secondary IP addresses to the appropriate node instances

[lukasmrtvy]

Describe the problem/challenge you have

HA Egress does not work for AWS EKS

Describe the solution you'd like

Include functionality to assign Egress IPs as secondary IP addresses to the appropriate node instances

Anything else you would like to add?

[antoninbas]

@lukasmrtvy based on discussions we had on Slack, it seems that you are not really asking for Egress HA support across multiple AZs (see #4385), but you are first asking for Egress (with ExternalIPPool) to work out-of-the box for a single AZ?

cc @tnqn

[lukasmrtvy]

yes, HA = for multiple nodes, not a single one, and yes, for a single AZ

[robbo10]

@antoninbas - Are there any plans to have an Egress assigned to more than one node for H/A purposes?

Current limitation is that each time we rotate workers, or a node crashes there will be an intermittent blip whilst the controller assigns the Egress to a healthy node.

[antoninbas]

@robbo10 sorry for the delay, I just came back from vacation today

I think a "blip" is unavoidable. Even if we had an "active-active" implementation, in case of an Egress Node failure, we would still need to fail over connections to the remaining Node. Additionally, it's not clear how we would handle return traffic, if the Egress IP (which is the destination IP for return traffic) is "assigned" to multiple Egress Nodes.

While some earlier Antrea versions (most notably, v1.11.0 and v1.12.0) had a bug causing longer than normal failover times for Egress IPs, the bug was patched in Antrea v1.13, v1.12.1 and v1.11.3 (look for the following in release notes: Ensure the Egress IP is always correctly advertised to the network, including when the userspace ARP responder is not running or when the Egress IP is temporarily claimed by multiple Nodes.). In the absence of the bug, my experience is that failover is very fast (under 1s).

Adding @tnqn in case he has further comments.

[robbo10]

@antoninbas - no problem at all this makes sense. I had a conversation with @tnqn who confirmed the same.

We were able to verify that the 1.11.3 release greatly improves Egress failover performance :)

Thanks for all the work in releasing this.

[antoninbas]

@tnqn has created a documentation PR to describe this limitation

[tnqn]

@tnqn has created a documentation PR to describe this limitation

https://github.com/antrea-io/antrea/blob/main/docs/egress.md#egress-on-cloud describes how Egress works on cloud platform today and what's missing to make it work on AWS.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days