Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix session affinity for canaries #7169

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
b381330
More prereqs
wasker May 24, 2021
ab70af2
Clone session affinity settings from pri to alt up
wasker May 25, 2021
3d1a5ee
New annotation.
wasker May 25, 2021
8dbd630
High-level implementation of is_affinitized()
wasker May 25, 2021
ec36aab
Affinitization works.
wasker May 25, 2021
d13ba21
Make existing lua tests pass.
wasker May 25, 2021
28af836
Tests for split.
wasker May 26, 2021
ddeeaa4
Reformat lua code.
wasker May 26, 2021
b21add3
Reshuffle alt balancer tests.
wasker May 26, 2021
52d6ec8
Added test for chashsubset impl.
wasker May 26, 2021
4cb1e91
Always reset balancer after ngx mocked.
wasker May 26, 2021
60e0642
Exposed get_balancer_by_upstream_name()
wasker May 26, 2021
f6621a6
UTs for affinitized requests.
wasker May 26, 2021
c1a006e
Follow same patterns as balancer UTs.
wasker May 26, 2021
1e6bc30
UTs for cookies
wasker May 26, 2021
e5b9bcd
Use same style across the board.
wasker May 26, 2021
c89c060
Documentation updates.
wasker May 26, 2021
006c136
Make existing UTs pass.
wasker May 26, 2021
2f3e54d
Added UTs for "sticky" canary behavior.
wasker May 27, 2021
131b509
Tests for annotations parser.
wasker May 27, 2021
dcfafba
Fixed lua linter errors.
wasker Jun 14, 2021
bea4768
Revert unnecessary changes.
wasker Jun 21, 2021
c15b7c5
Drop v1beta1 from ingress nginx (#7156)
rikatz Jun 23, 2021
f2b73ce
add docs for syslog feature (#7219)
tianhaopx Jun 27, 2021
60afa7d
Fix link to e2e-tests.md in developer-guide (#7201)
rctay Jun 27, 2021
cfde9df
Use ENV expansion for namespace in args (#7146)
cassid4 Jun 27, 2021
9898bae
chart: using Helm builtin capabilities check (#7190)
tao12345666333 Jun 27, 2021
4168ab9
Update proper default value for HTTP2MaxConcurrentStreams in Docs (#6…
kutysam Jun 27, 2021
62609f2
Fix MaxWorkerOpenFiles calculation on high cores nodes (#7107)
nanorobocop Jun 29, 2021
39e1d31
Use v1 API contract.
wasker Jul 1, 2021
67ec22a
Fixed setup for SSL UTs.
wasker Jul 1, 2021
55ea378
ingress/tcp: add additional error logging on failed (#7208)
Matt1360 Jun 29, 2021
8ca9c12
replacing master with main for k/k8s.io links (#7307)
RinkiyaKeDad Jul 1, 2021
2adde86
Add file containing stable release (#7313)
rikatz Jul 1, 2021
a26d8f8
Handle named (non-numeric) ports correctly (#7311)
cpanato Jul 4, 2021
bfc5a8a
Updated v1beta1 to v1 as its deprecated (#7308)
kundan2707 Jul 4, 2021
f152efe
use listen to ensure the port is free (#6990)
boxjan Jul 4, 2021
b5507fb
remove mercurial from build (#7031)
kd7lxl Jul 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,11 @@ jobs:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Go 1.15
- name: Set up Go 1.16
id: go
uses: actions/setup-go@v2
with:
go-version: 1.15
go-version: 1.16

- name: Set up Docker Buildx
id: buildx
Expand Down
8 changes: 5 additions & 3 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The sha is available in output from [cloud build](https://console.cloud.google.c

2a. Promote images:

* Open pull request to promote [staging image](https://github.com/kubernetes/k8s.io/blob/master/k8s.gcr.io/images/k8s-staging-ingress-nginx/images.yaml)
* Open pull request to promote [staging image](https://github.com/kubernetes/k8s.io/blob/main/k8s.gcr.io/images/k8s-staging-ingress-nginx/images.yaml)
* e2e-test-runner
* e2e-test-cfssl
* e2e-test-echo
Expand All @@ -53,7 +53,7 @@ The sha is available in output from [cloud build](https://console.cloud.google.c

4a. Promote images:

* Open pull request to promote [staging image](https://github.com/kubernetes/k8s.io/blob/master/k8s.gcr.io/images/k8s-staging-ingress-nginx/images.yaml)
* Open pull request to promote [staging image](https://github.com/kubernetes/k8s.io/blob/main/k8s.gcr.io/images/k8s-staging-ingress-nginx/images.yaml)
* controller

5. Release helm chart:
Expand All @@ -74,5 +74,7 @@ The sha is available in output from [cloud build](https://console.cloud.google.c
* Update the version in [docs/deploy/index.md](docs/deploy/index.md)
* Update Supported versions in the Support Versions table in the README.md
* Merge

* Update the stable.txt file to reflect the release to be created

7. Github release
7. Github release
3 changes: 3 additions & 0 deletions build/run-in-docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ set -o pipefail
# temporal directory for the /etc/ingress-controller directory
INGRESS_VOLUME=$(mktemp -d)

# make sure directory for SSL cert storage exists under ingress volume
mkdir "${INGRESS_VOLUME}/ssl"

if [[ "$OSTYPE" == darwin* ]]; then
INGRESS_VOLUME=/private$INGRESS_VOLUME
fi
Expand Down
2 changes: 1 addition & 1 deletion charts/ingress-nginx/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Get the application URL by running these commands:

An example Ingress that makes use of the controller:

apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ webhooks:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1beta1
- v1
operations:
- CREATE
- UPDATE
Expand All @@ -28,12 +28,11 @@ webhooks:
sideEffects: None
admissionReviewVersions:
- v1
- v1beta1
clientConfig:
service:
namespace: {{ .Release.Namespace | quote }}
name: {{ include "ingress-nginx.controller.fullname" . }}-admission
path: /networking/v1beta1/ingresses
path: /networking/v1/ingresses
{{- if .Values.controller.admissionWebhooks.timeoutSeconds }}
timeoutSeconds: {{ .Values.controller.admissionWebhooks.timeoutSeconds }}
{{- end }}
Expand Down
10 changes: 5 additions & 5 deletions charts/ingress-nginx/templates/controller-daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,22 +74,22 @@ spec:
args:
- /nginx-ingress-controller
{{- if .Values.defaultBackend.enabled }}
- --default-backend-service={{ .Release.Namespace }}/{{ include "ingress-nginx.defaultBackend.fullname" . }}
- --default-backend-service=$(POD_NAMESPACE)/{{ include "ingress-nginx.defaultBackend.fullname" . }}
{{- end }}
{{- if .Values.controller.publishService.enabled }}
- --publish-service={{ template "ingress-nginx.controller.publishServicePath" . }}
{{- end }}
- --election-id={{ .Values.controller.electionID }}
- --ingress-class={{ .Values.controller.ingressClass }}
- --configmap={{ .Release.Namespace }}/{{ include "ingress-nginx.controller.fullname" . }}
- --configmap=$(POD_NAMESPACE)/{{ include "ingress-nginx.controller.fullname" . }}
{{- if .Values.tcp }}
- --tcp-services-configmap={{ .Release.Namespace }}/{{ include "ingress-nginx.fullname" . }}-tcp
- --tcp-services-configmap=$(POD_NAMESPACE)/{{ include "ingress-nginx.fullname" . }}-tcp
{{- end }}
{{- if .Values.udp }}
- --udp-services-configmap={{ .Release.Namespace }}/{{ include "ingress-nginx.fullname" . }}-udp
- --udp-services-configmap=$(POD_NAMESPACE)/{{ include "ingress-nginx.fullname" . }}-udp
{{- end }}
{{- if .Values.controller.scope.enabled }}
- --watch-namespace={{ default .Release.Namespace .Values.controller.scope.namespace }}
- --watch-namespace={{ default "$(POD_NAMESPACE)" .Values.controller.scope.namespace }}
{{- end }}
{{- if and .Values.controller.reportNodeInternalIp .Values.controller.hostNetwork }}
- --report-node-internal-ip-address={{ .Values.controller.reportNodeInternalIp }}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and .Values.controller.metrics.enabled .Values.controller.metrics.serviceMonitor.enabled -}}
{{- if and ( .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" ) .Values.controller.metrics.enabled .Values.controller.metrics.serviceMonitor.enabled -}}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
Expand Down
41 changes: 17 additions & 24 deletions cmd/nginx/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,36 +104,29 @@ func main() {
conf.FakeCertificate = ssl.GetFakeSSLCert()
klog.InfoS("SSL fake certificate created", "file", conf.FakeCertificate.PemFileName)

var isNetworkingIngressAvailable bool

isNetworkingIngressAvailable, k8s.IsIngressV1Beta1Ready, _ = k8s.NetworkingIngressAvailable(kubeClient)
if !isNetworkingIngressAvailable {
klog.Fatalf("ingress-nginx requires Kubernetes v1.14.0 or higher")
if !k8s.NetworkingIngressAvailable(kubeClient) {
klog.Fatalf("ingress-nginx requires Kubernetes v1.19.0 or higher")
}

if k8s.IsIngressV1Beta1Ready {
klog.InfoS("Enabling new Ingress features available since Kubernetes v1.18")
k8s.IngressClass, err = kubeClient.NetworkingV1beta1().IngressClasses().
Get(context.TODO(), class.IngressClass, metav1.GetOptions{})
if err != nil {
if !errors.IsNotFound(err) {
if !errors.IsUnauthorized(err) && !errors.IsForbidden(err) {
klog.Fatalf("Error searching IngressClass: %v", err)
}

klog.ErrorS(err, "Searching IngressClass", "class", class.IngressClass)
k8s.IngressClass, err = kubeClient.NetworkingV1().IngressClasses().
Get(context.TODO(), class.IngressClass, metav1.GetOptions{})
if err != nil {
if !errors.IsNotFound(err) {
if !errors.IsUnauthorized(err) && !errors.IsForbidden(err) {
klog.Fatalf("Error searching IngressClass: %v", err)
}
klog.ErrorS(err, "Searching IngressClass", "class", class.IngressClass)
}

klog.Warningf("No IngressClass resource with name %v found. Only annotation will be used.", class.IngressClass)
klog.Warningf("No IngressClass resource with name %v found. Only annotation will be used.", class.IngressClass)
// TODO: remove once this is fixed in client-go
k8s.IngressClass = nil

// TODO: remove once this is fixed in client-go
k8s.IngressClass = nil
}
}

if k8s.IngressClass != nil && k8s.IngressClass.Spec.Controller != k8s.IngressNGINXController {
klog.Errorf(`Invalid IngressClass (Spec.Controller) value "%v". Should be "%v"`, k8s.IngressClass.Spec.Controller, k8s.IngressNGINXController)
klog.Fatalf("IngressClass with name %v is not valid for ingress-nginx (invalid Spec.Controller)", class.IngressClass)
}
if k8s.IngressClass != nil && k8s.IngressClass.Spec.Controller != k8s.IngressNGINXController {
klog.Errorf(`Invalid IngressClass (Spec.Controller) value "%v". Should be "%v"`, k8s.IngressClass.Spec.Controller, k8s.IngressNGINXController)
klog.Fatalf("IngressClass with name %v is not valid for ingress-nginx (invalid Spec.Controller)", class.IngressClass)
}

conf.Client = kubeClient
Expand Down
29 changes: 23 additions & 6 deletions cmd/plugin/commands/ingresses/ingresses.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ import (
"text/tabwriter"

"github.com/spf13/cobra"
networking "k8s.io/api/networking/v1beta1"
networking "k8s.io/api/networking/v1"
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/cli-runtime/pkg/genericclioptions"

"k8s.io/ingress-nginx/cmd/plugin/request"
Expand Down Expand Up @@ -157,9 +158,10 @@ func getIngressRows(ingresses *[]networking.Ingress) []ingressRow {

defaultBackendService := ""
defaultBackendPort := ""
if ing.Spec.Backend != nil {
defaultBackendService = ing.Spec.Backend.ServiceName
defaultBackendPort = ing.Spec.Backend.ServicePort.String()
if ing.Spec.DefaultBackend != nil {
name, port := serviceToNameAndPort(ing.Spec.DefaultBackend.Service)
defaultBackendService = name
defaultBackendPort = port.String()
}

// Handle catch-all ingress
Expand Down Expand Up @@ -197,14 +199,15 @@ func getIngressRows(ingresses *[]networking.Ingress) []ingressRow {
}

for _, path := range rule.HTTP.Paths {
svcName, svcPort := serviceToNameAndPort(path.Backend.Service)
row := ingressRow{
Namespace: ing.Namespace,
IngressName: ing.Name,
Host: rule.Host,
Path: path.Path,
TLS: hasTLS,
ServiceName: path.Backend.ServiceName,
ServicePort: path.Backend.ServicePort.String(),
ServiceName: svcName,
ServicePort: svcPort.String(),
Address: address,
}

Expand All @@ -215,3 +218,17 @@ func getIngressRows(ingresses *[]networking.Ingress) []ingressRow {

return rows
}

func serviceToNameAndPort(svc *networking.IngressServiceBackend) (string, intstr.IntOrString) {
var svcName string
if svc != nil {
svcName = svc.Name
if svc.Port.Number > 0 {
return svcName, intstr.FromInt(int(svc.Port.Number))
}
if svc.Port.Name != "" {
return svcName, intstr.FromString(svc.Port.Name)
}
}
return "", intstr.IntOrString{}
}
75 changes: 75 additions & 0 deletions cmd/plugin/commands/ingresses/ingresses_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
Copyright 2021 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package ingresses

import (
"testing"

networking "k8s.io/api/networking/v1"
"k8s.io/apimachinery/pkg/util/intstr"
)

func TestGetIngressInformation(t *testing.T) {

testcases := map[string]struct {
ServiceBackend *networking.IngressServiceBackend
wantName string
wantPort intstr.IntOrString
}{
"empty ingressServiceBackend": {
ServiceBackend: &networking.IngressServiceBackend{},
wantName: "",
wantPort: intstr.IntOrString{},
},
"ingressServiceBackend with port 8080": {
ServiceBackend: &networking.IngressServiceBackend{
Name: "test",
Port: networking.ServiceBackendPort{
Number: 8080,
},
},
wantName: "test",
wantPort: intstr.IntOrString{
Type: intstr.Int,
IntVal: 8080,
},
},
"ingressServiceBackend with port name a-svc": {
ServiceBackend: &networking.IngressServiceBackend{
Name: "test",
Port: networking.ServiceBackendPort{
Name: "a-svc",
},
},
wantName: "test",
wantPort: intstr.IntOrString{
Type: intstr.String,
StrVal: "a-svc",
},
},
}

for title, testCase := range testcases {
gotName, gotPort := serviceToNameAndPort(testCase.ServiceBackend)
if gotName != testCase.wantName {
t.Fatalf("%s: expected '%v' but returned %v", title, testCase.wantName, gotName)
}
if gotPort != testCase.wantPort {
t.Fatalf("%s: expected '%v' but returned %v", title, testCase.wantPort, gotPort)
}
}
}
2 changes: 1 addition & 1 deletion cmd/plugin/commands/lint/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import (
"github.com/spf13/cobra"

appsv1 "k8s.io/api/apps/v1"
networking "k8s.io/api/networking/v1beta1"
networking "k8s.io/api/networking/v1"
kmeta "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/cli-runtime/pkg/genericclioptions"

Expand Down
2 changes: 1 addition & 1 deletion cmd/plugin/lints/ingress.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"fmt"
"strings"

networking "k8s.io/api/networking/v1beta1"
networking "k8s.io/api/networking/v1"
kmeta "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/ingress-nginx/cmd/plugin/util"
)
Expand Down
4 changes: 2 additions & 2 deletions cmd/plugin/request/request.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ import (

appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
networking "k8s.io/api/networking/v1beta1"
networking "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/cli-runtime/pkg/genericclioptions"
appsv1client "k8s.io/client-go/kubernetes/typed/apps/v1"
corev1 "k8s.io/client-go/kubernetes/typed/core/v1"
typednetworking "k8s.io/client-go/kubernetes/typed/networking/v1beta1"
typednetworking "k8s.io/client-go/kubernetes/typed/networking/v1"

"k8s.io/ingress-nginx/cmd/plugin/util"
)
Expand Down
5 changes: 2 additions & 3 deletions deploy/static/provider/aws/deploy-tls-termination.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,7 @@ webhooks:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1beta1
- v1
operations:
- CREATE
- UPDATE
Expand All @@ -450,12 +450,11 @@ webhooks:
sideEffects: None
admissionReviewVersions:
- v1
- v1beta1
clientConfig:
service:
namespace: ingress-nginx
name: ingress-nginx-controller-admission
path: /networking/v1beta1/ingresses
path: /networking/v1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
Expand Down
5 changes: 2 additions & 3 deletions deploy/static/provider/aws/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ webhooks:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1beta1
- v1
operations:
- CREATE
- UPDATE
Expand All @@ -437,12 +437,11 @@ webhooks:
sideEffects: None
admissionReviewVersions:
- v1
- v1beta1
clientConfig:
service:
namespace: ingress-nginx
name: ingress-nginx-controller-admission
path: /networking/v1beta1/ingresses
path: /networking/v1/ingresses
---
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
apiVersion: v1
Expand Down
Loading