Considerations of traefik-proxy, z2jh, and traefik v2

[consideRatio]

A k8s native way to configure Traefik

Traefik v2 can discover its own configuration of where it should route etc by looking for configuration snippets in various custom k8s resource. We would then create/edit/delete these k8s resources (IngressRoute).

This way, we would not have to run a key value store like etcd/consul ourselves to keep traefiks configuration, as we would rely on the k8s api-server that is backed by its own etcd already. We could have many traefik server pods running in parallel thanks to this.

A limitation with Let's encrypt integration

There is a caveat of using Traefik with Let's encrypt and having more than traefik server running though. They don't support resolving the ACME challenges Let's encrypt will challenge us with then sadly, unless the Enterprise Edition of traefik is used.

This only relates to the acquisition of certificates though. The certificates can still be used to do the TLS termination. It made me sad... It would have all come together so nicely.

[consideRatio]

Ping @yuvipanda as you have worked on Treafik-matters recently and may be interested.

[yuvipanda]

I think automatic let's encrypt is a must, and unfortunately it looks like we can't use the highly available mode with Let's Encrypt. So we'll always need a mode that gives people Let's Encrypt without needing an external service (like cert-manager). We can provide the highly available mode as an alternative?

The other issue is where an IngressRoute can route to. Currently we route directly to pods, without a Service (and Endpoint) objects in between. There is code in KubeSpawner that does this, but idk if I like it!