Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KubeIngressProxy] Add a long docstring to KubeIngressProxy class #568

Merged
merged 4 commits into from
Mar 11, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions kubespawner/proxy.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,91 @@ def endpoints(self):


class KubeIngressProxy(Proxy):
"""
DISCLAIMER:

This JupyterHub Proxy class is not maintained thoroughly with tests and
documentation, or actively used in any official distribution of
JupyterHub.

KubeIngressProxy was originally developed by @yuvipanda in preparation
to support a JupyterHub that could end up with ~20k simultaneous users.
The single ConfigurableHTTPProxy server that is controlled by the
default JupyterHub Proxy class could bottleneck for such heavy load,
even though it has been found to be good enough for ~2k simultaneous
users. By instead using this JupyterHub Proxy class that would create
k8s resources (Ingress, Service, Endpoint) that in turn controlled how a
flexible set of proxy servers would proxy traffic, the bottleneck could
be removed.

KubeIngressProxy's efficiency relates greatly to the performance of the
k8s api-server and the k8s controller that routes traffic based on
changes to Ingress resources registered by the k8s api-server. This
means users of KubeIngressProxy may have greatly varying experiences of
using it depending on the performance of their k8s cluster setup.

Use of KubeIngressProxy as a JupyterHub Proxy class, is entirely
independent of use of KubeSpawner as a JupyterHub Spawner class. For
reasons related to sharing code with KubeSpawner in reflectors.py, it
has been made and remained part of the jupyterhub/kubespawner project.

Related issues:
- Need for tests: https://github.com/jupyterhub/kubespawner/issues/496
- Need for docs: https://github.com/jupyterhub/kubespawner/issues/163

---

KubeIngressProxy is an implementation of the JupyterHub Proxy class that
JupyterHub can be configured to rely on:

c.JupyterHub.proxy_class = "kubespawner:KubeIngressProxy"

Like all JupyterHub Proxy implementations, KubeIngressProxy will know
how to respond to hub requests like `get_all_routes`, `add_route`, and
`delete_route` in a way that will ensure traffic will get routed to the user
pods or JupyterHub registered external services. For reference, the official
documentation on writing a custom Proxy class like this is documented here:
https://jupyterhub.readthedocs.io/en/stable/reference/proxy.html.

KubeIngressProxy communicates with the k8s api-server in order to
create/delete Ingress resources according to the hub's `add_route`/`delete_route`
requests. It doesn't route traffic by itself, but instead relies on the k8s cluster's
ability to route traffic according to these Ingress resources.

Because KubeIngressProxy interacts with a k8s api-server that manages
Ingress resources, it must have permissions to do so as well. These
permissions should be granted to a k8s service account for where the
JupyterHub is running, as that is also where the KubeIngressProxy class
instance will run its code.

FIXME: Verify what k8s RBAC permissions are required for KubeIngressProxy
to function.

- The IngressReflector, ServiceReflector, and EndpointsReflector
require permission to read/list/watch those resources.
- `add_route` and `delete_route` requires permission to
create/update/patch/delete Ingress, Service, and Endpoints
resources.

These permissions are needed on a k8s Role resource bound to the k8s
ServiceAccount (via a k8s RoleBinding) used on the k8s Pod where
JupyterHub runs:

```yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kube-ingress-proxy
rules:
- apiGroups: [""]
resources: ["endpoints", "services"]
consideRatio marked this conversation as resolved.
Show resolved Hide resolved
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
```
"""

namespace = Unicode(
config=True,
help="""
Expand Down