TCP service need to recieve source ip

[jp250595]

Hi,
We have exposed tcp service using config map and passed the argument for the config map in the argument section of the deployment. All seems to be working and we are able to hit the backend tcp service.
Only issues is that when traffic is received at the backend pod, the source ip is of the nginx ingress controller pod instead of client machines which are iniating the connections.

Can we preserve the client IP address for the TCP services?

[longwuyuan]

can you show the related state with kubectl commands like ;

• k get svc -A =o wide | grep -i load
• k -n get deploy -o wide
• k -n get svc -o wide
• k -n describe po
• k -n logs
• k -n describe svc
• k -n describe po
• k -n describe cm
• k -n logs
• Your curl command and output
• All other related info like kubectl commands for the LB involved etc
• Any other related logs & information

[coryallegory]

https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip

try using externalTrafficPolicy: Local in your Service and see if that does the trick for you

[jp250595]

@longwuyuan @coryallegory Please find below details:
Ingress service details:
Name: ingress-nginx-controller
Namespace: ingress-nginx
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/version=0.44.0
helm.sh/chart=ingress-nginx-3.23.0
Annotations: cloud.google.com/neg: {"ingress":true}
meta.helm.sh/release-name: cxth-ingress-nginx
meta.helm.sh/release-namespace: cxth-application
Selector: app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type: LoadBalancer
IP Families:
IP: 10.88.0.56
IPs:
LoadBalancer Ingress: XXXXX
Port: http 80/TCP
TargetPort: http/TCP
NodePort: http 32091/TCP
Endpoints: 10.84.1.51:80
Port: https 443/TCP
TargetPort: https/TCP
NodePort: https 30264/TCP
Endpoints: 10.84.1.51:443
Port: tcp 6000/TCP
TargetPort: 6000/TCP
NodePort: tcp 31179/TCP
Endpoints: 10.84.1.51:6000
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 32544

Ingress Pod details:
Name: ingress-nginx-controller-7949c885b-dnxnq
Namespace: ingress-nginx
Priority: 0
Node: XXXXXXXXXXXXXX
Start Time: Tue, 06 Apr 2021 11:10:53 +0000
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/name=ingress-nginx
pod-template-hash=7949c885b
Annotations:
Status: Running
IP: 10.84.1.51
IPs:
IP: 10.84.1.51
Controlled By: ReplicaSet/ingress-nginx-controller-7949c885b
Containers:
controller:
Container ID: containerd://843065ad2f42e9f2d21d187a942ef279884ce3bebb7d8b3177a2b71b6e01cb2e
Image: k8s.gcr.io/ingress-nginx/controller:v0.44.0@sha256:3dd0fac48073beaca2d67a78c746c7593f9c575168a17139a9955a82c63c4b9a
Image ID: k8s.gcr.io/ingress-nginx/controller@sha256:3dd0fac48073beaca2d67a78c746c7593f9c575168a17139a9955a82c63c4b9a
Ports: 80/TCP, 443/TCP, 8443/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Args:
/nginx-ingress-controller
--publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
--election-id=ingress-controller-leader
--ingress-class=nginx
--configmap=$(POD_NAMESPACE)/ingress-nginx-controller
--tcp-services-configmap=cxth-application/tcp-services
--validating-webhook=:8443
--validating-webhook-certificate=/usr/local/certificates/cert
--validating-webhook-key=/usr/local/certificates/key
State: Running
Started: Tue, 06 Apr 2021 11:10:54 +0000
Ready: True
Restart Count: 0
Requests:
cpu: 100m
memory: 90Mi
Liveness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
Readiness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
Environment:
POD_NAME: ingress-nginx-controller-7949c885b-dnxnq (v1:metadata.name)
POD_NAMESPACE: ingress-nginx (v1:metadata.namespace)
LD_PRELOAD: /usr/local/lib/libmimalloc.so
Mounts:
/usr/local/certificates/ from webhook-cert (ro)
/var/run/secrets/kubernetes.io/serviceaccount from ingress-nginx-token-4hctr (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
webhook-cert:
Type: Secret (a volume populated by a Secret)
SecretName: ingress-nginx-admission
Optional: false
ingress-nginx-token-4hctr:
Type: Secret (a volume populated by a Secret)
SecretName: ingress-nginx-token-4hctr
Optional: false
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Events:

Name: tcp-services
Namespace: XXXXXXXXXXX
Labels: app.kubernetes.io/managed-by=Helm
Annotations: meta.helm.sh/release-name: cxth-common
meta.helm.sh/release-namespace: cxth-application

Data

6000:

namespace/ndc:6000
Events:

Also to mention when we receive the traffic on port 6000 & service ndc, the source ip is of the ingress pod. (Required is of external source ip)

[jp250595]

https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip

try using externalTrafficPolicy: Local in your Service and see if that does the trick for you

---Not working :(

[longwuyuan]

Even after actually typing the commands that will show information on this, you don't care to help provide enough information to understand the problem. So expect delay in solving the problem or even maybe no solution.

If you want to help, then read the commands suggested and provide some data that can be analyzed. Setting the "externalTrafficPolicy" to "Local" is the normal solution but you are also saying that is not working but not providing any other information for someone to analyze. So try to help with info and hopefully this will be solved.

[longwuyuan]

@jp250595 Actually I suggest close this issue and chat in kubernetes.slack.com for more activity on this

[Azbesciak]

@jp250595 did you find solution for that issue? I have the same one I thing

• service exposed via tcp config map
• controller receives client IP
• in my app logs I see controller internal eth0 IP