NetworkServiceController: Add hairpin-mode support

[bzub]

Tested on my local cluster with quay.io/bzub/kube-router:hairpin-mode

Fixes #9

Test examples:

kubectl -n default run alpine --command --image alpine --replicas 2 --port 80 --expose sleep 300d

IPs:

$kubectl -n default get pods,services,endpoints -o wide
NAME                         READY     STATUS    RESTARTS   AGE       IP         NODE
po/alpine-4025682865-7r4tw   1/1       Running   0          1h        10.2.0.2   node3-test.zbrbdl
po/alpine-4025682865-pqdfr   1/1       Running   0          1h        10.2.1.8   node2-test.zbrbdl

NAME             CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE       SELECTOR
svc/alpine       10.3.0.64    <none>        80/TCP    1h        run=alpine
svc/kubernetes   10.3.0.1     <none>        443/TCP   1h        <none>

NAME            ENDPOINTS                 AGE
ep/alpine       10.2.0.2:80,10.2.1.8:80   1h
ep/kubernetes   10.10.3.3:443             1h

iptables on nodes:

$ for i in node2-test node3-test
$ do
$   echo ${i}
$   ssh ${i} 'sudo iptables -t nat -S|grep -F KUBE-ROUTER'
$   echo
$ done
node2-test
-N KUBE-ROUTER-HAIRPIN
-A POSTROUTING -m ipvs --vdir ORIGINAL -j KUBE-ROUTER-HAIRPIN
-A KUBE-ROUTER-HAIRPIN -s 10.2.0.2/32 -d 10.2.0.2/32 -m ipvs --vaddr 10.3.0.64 -j SNAT --to-source 10.3.0.64
-A KUBE-ROUTER-HAIRPIN -s 10.2.1.8/32 -d 10.2.1.8/32 -m ipvs --vaddr 10.3.0.64 -j SNAT --to-source 10.3.0.64

node3-test
  -N KUBE-ROUTER-HAIRPIN
  -A POSTROUTING -m ipvs --vdir ORIGINAL -j KUBE-ROUTER-HAIRPIN
  -A KUBE-ROUTER-HAIRPIN -s 10.2.0.2/32 -d 10.2.0.2/32 -m ipvs --vaddr 10.3.0.64 -j SNAT --to-source 10.3.0.64
-A KUBE-ROUTER-HAIRPIN -s 10.2.1.8/32 -d 10.2.1.8/32 -m ipvs --vaddr 10.3.0.64 -j SNAT --to-source 10.3.0.64

Log into both pods:

for i in $(kd get pods -l run=alpine --output name)
  do
  kd exec -it $(basename ${i}) sh
done

Run httpd server and access it via Service IP:

/ # hostname
alpine-4025682865-7r4tw
alpine-4025682865-pqdfr

/ # nohup httpd -fv &

/ # netstat -lnp                                     
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 :::80                   :::*                    LISTEN      9/httpd

/ # while true; do wget alpine.default; done
Connecting to alpine.default (10.3.0.64:80)
wget: server returned error: HTTP/1.0 404 Not Found
Connecting to alpine.default (10.3.0.64:80)
wget: server returned error: HTTP/1.0 404 Not Found
Connecting to alpine.default (10.3.0.64:80)
wget: server returned error: HTTP/1.0 404 Not Found
Connecting to alpine.default (10.3.0.64:80)
wget: server returned error: HTTP/1.0 404 Not Found
Connecting to alpine.default (10.3.0.64:80)

/ # tail nohup.out (pod1)
[::ffff:10.2.0.2]:58120: response:404
[::ffff:10.3.0.64]:34658: response:404
[::ffff:10.2.0.2]:58124: response:404
[::ffff:10.3.0.64]:34662: response:404
[::ffff:10.2.0.2]:58128: response:404
[::ffff:10.3.0.64]:34666: response:404
[::ffff:10.2.0.2]:58132: response:404
[::ffff:10.3.0.64]:34670: response:404
[::ffff:10.3.0.64]:34674: response:404
[::ffff:10.2.0.2]:58136: response:404

/ # tail nohup.out (pod2)
[::ffff:10.3.0.64]:58118: response:404
[::ffff:10.2.1.8]:34656: response:404
[::ffff:10.3.0.64]:58122: response:404
[::ffff:10.2.1.8]:34660: response:404
[::ffff:10.3.0.64]:58126: response:404
[::ffff:10.2.1.8]:34664: response:404
[::ffff:10.3.0.64]:58130: response:404
[::ffff:10.2.1.8]:34668: response:404
[::ffff:10.3.0.64]:58134: response:404
[::ffff:10.2.1.8]:34672: response:404

[murali-reddy]

Excellent @bzub

Can you please ensure syncHairpinIptablesRules does not create any haripin chains by default. i.e) if if nsc.globalHairpin || svcInfo.hairpin condition not met, then return in beginning of the method before adding rueles.

Also, for node-port service type accesing the node service on which pod is running, can we add in the documentation its not supported.

[bzub]

@murali-reddy thanks for the review.

I've added support for NodePort and ensured no rules would be created if no hairpin options are set. I added to my manual tests for NodePort and deleting rules on Service/options change. Everything should be resolved but check it over please. Once I get your LGTM I'll squash commits and merge.

[murali-reddy]

Great Work! LGTM. Please go ahead with the merge.