Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gatewayapi: add support for session affinity #1507

Merged
merged 3 commits into from
Sep 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion artifacts/examples/appmesh-abtest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion artifacts/examples/appmesh-canary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion artifacts/examples/istio-abtest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion artifacts/examples/istio-canary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion artifacts/examples/linkerd-canary-steps.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion artifacts/examples/linkerd-canary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
4 changes: 2 additions & 2 deletions charts/podinfo/templates/canary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ spec:
kind: Deployment
name: {{ template "podinfo.fullname" . }}
autoscalerRef:
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: {{ template "podinfo.fullname" . }}
service:
Expand Down Expand Up @@ -57,4 +57,4 @@ spec:
metadata:
cmd: "hey -z 1m -q 5 -c 2 http://{{ template "podinfo.fullname" . }}.{{ .Release.Namespace }}:{{ .Values.service.port }}"
{{- end }}
{{- end }}
{{- end }}
2 changes: 1 addition & 1 deletion charts/podinfo/templates/hpa.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{- if .Values.hpa.enabled -}}
apiVersion: autoscaling/v2beta1
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ template "podinfo.fullname" . }}
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/appmesh-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/contour-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ spec:
name: podinfo
# HPA reference
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
119 changes: 115 additions & 4 deletions docs/gitbook/tutorials/gatewayapi-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Save the above resource as metric-templates.yaml and then apply it:
kubectl apply -f metric-templates.yaml
```

Create a canary custom resource \(replace "loaclproject.contour.io" with your own domain\):
Create a canary custom resource \(replace "localproject.contour.io" with your own domain\):

```yaml
apiVersion: flagger.app/v1beta1
Expand All @@ -156,7 +156,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down Expand Up @@ -382,13 +382,124 @@ Events:
Warning Synced 1m flagger Canary failed! Scaling down podinfo.test
```

## Session Affinity

While Flagger can perform weighted routing and A/B testing individually, with Gateway API it can combine the two leading to a Canary
release with session affinity.
For more information you can read the [deployment strategies docs](../usage/deployment-strategies.md#canary-release-with-session-affinity).

> **Note:** The implementation must have support for the [`ResponseHeaderModifier`](https://github.com/kubernetes-sigs/gateway-api/blob/3d22aa5a08413222cb79e6b2e245870360434614/apis/v1beta1/httproute_types.go#L651) API.

Create a canary custom resource \(replace localproject.contour.io with your own domain\):

```yaml
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
name: podinfo
namespace: test
spec:
# deployment reference
targetRef:
apiVersion: apps/v1
kind: Deployment
name: podinfo
# the maximum time in seconds for the canary deployment
# to make progress before it is rollback (default 600s)
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
# service port number
port: 9898
# container port number or name (optional)
targetPort: 9898
# Gateway API HTTPRoute host names
hosts:
- localproject.contour.io
# Reference to the Gateway that the generated HTTPRoute would attach to.
gatewayRefs:
- name: contour
namespace: projectcontour
analysis:
# schedule interval (default 60s)
interval: 1m
# max number of failed metric checks before rollback
threshold: 5
# max traffic percentage routed to canary
# percentage (0-100)
maxWeight: 50
# canary increment step
# percentage (0-100)
stepWeight: 10
# session affinity config
sessionAffinity:
# name of the cookie used
cookieName: flagger-cookie
# max age of the cookie (in seconds)
# optional; defaults to 86400
maxAge: 21600
metrics:
- name: error-rate
# max error rate (5xx responses)
# percentage (0-100)
templateRef:
name: error-rate
namespace: flagger-system
thresholdRange:
max: 1
interval: 1m
- name: latency
templateRef:
name: latency
namespace: flagger-system
# seconds
thresholdRange:
max: 0.5
interval: 30s
# testing (optional)
webhooks:
- name: smoke-test
type: pre-rollout
url: http://flagger-loadtester.test/
timeout: 15s
metadata:
type: bash
cmd: "curl -sd 'anon' http://podinfo-canary.test:9898/token | grep token"
- name: load-test
url: http://flagger-loadtester.test/
timeout: 5s
metadata:
cmd: "hey -z 2m -q 10 -c 2 -host localproject.contour.io http://envoy.projectcontour/"
```

Save the above resource as podinfo-canary-session-affinity.yaml and then apply it:

```bash
kubectl apply -f ./podinfo-canary-session-affinity.yaml
```

Trigger a canary deployment by updating the container image:

```bash
kubectl -n test set image deployment/podinfo \
podinfod=ghcr.io/stefanprodan/podinfo:6.0.1
```

You can load `localproject.contour.io` in your browser and refresh it until you see the requests being served by `podinfo:6.0.1`.
All subsequent requests after that will be served by `podinfo:6.0.1` and not `podinfo:6.0.0` because of the session affinity
configured by Flagger in the HTTPRoute object.

# A/B Testing

Besides weighted routing, Flagger can be configured to route traffic to the canary based on HTTP match conditions. In an A/B testing scenario, you'll be using HTTP headers or cookies to target a certain segment of your users. This is particularly useful for frontend applications that require session affinity.

![Flagger A/B Testing Stages](https://raw.githubusercontent.com/fluxcd/flagger/main/docs/diagrams/flagger-abtest-steps.png)

Create a canary custom resource \(replace "loaclproject.contour.io" with your own domain\):
Create a canary custom resource \(replace "localproject.contour.io" with your own domain\):

```yaml
apiVersion: flagger.app/v1beta1
Expand All @@ -407,7 +518,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/gloo-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ spec:
name: podinfo
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/istio-ab-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
4 changes: 2 additions & 2 deletions docs/gitbook/tutorials/istio-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down Expand Up @@ -316,7 +316,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/kubernetes-blue-green.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ spec:
progressDeadlineSeconds: 60
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
4 changes: 2 additions & 2 deletions docs/gitbook/tutorials/linkerd-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ spec:
name: podinfo
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
# the maximum time in seconds for the canary deployment
Expand Down Expand Up @@ -426,7 +426,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
service:
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/nginx-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ spec:
name: podinfo
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
# the maximum time in seconds for the canary deployment
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/osm-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ spec:
name: podinfo
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
# the maximum time in seconds for the canary deployment
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/skipper-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ spec:
name: podinfo
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
# the maximum time in seconds for the canary deployment
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/traefik-progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ spec:
name: podinfo
# HPA reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
# the maximum time in seconds for the canary deployment
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/tutorials/zero-downtime-deployments.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ Note that without resource requests the horizontal pod autoscaler can't determin
A production environment should be able to handle traffic bursts without impacting the quality of service. This can be achieved with Kubernetes autoscaling capabilities. Autoscaling in Kubernetes has two dimensions: the Cluster Autoscaler that deals with node scaling operations and the Horizontal Pod Autoscaler that automatically scales the number of pods in a deployment.

```yaml
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
scaleTargetRef:
Expand Down
4 changes: 2 additions & 2 deletions docs/gitbook/usage/deployment-strategies.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Flagger can run automated application analysis, promotion and rollback for the f
* **Blue/Green Mirroring** \(traffic shadowing\)
* Istio
* **Canary Release with Session Affinity** \(progressive traffic shifting combined with cookie based routing\)
* Istio
* Istio, Gateway API

For Canary releases and A/B testing you'll need a Layer 7 traffic management solution like
a service mesh or an ingress controller. For Blue/Green deployments no service mesh or ingress controller is required.
Expand Down Expand Up @@ -408,7 +408,7 @@ cookie based routing with regular weight based routing. This means once a user i
version of our application (based on the traffic weights), they're always routed to that version, i.e.
they're never routed back to the old version of our application.

You can enable this, by specifying `.spec.analsyis.sessionAffinity` in the Canary (only Istio is supported):
You can enable this, by specifying `.spec.analsyis.sessionAffinity` in the Canary:

```yaml
analysis:
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/usage/how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ spec:
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
name: podinfo
primaryScalerReplicas:
Expand Down
Loading
Loading