dklb supports exposing TCP services only.
In particular, exposing UDP or SCTP services is NOT supported.
To expose a TCP application running on MKE to either inside or outside the DC/OS cluster, a Kubernetes Service resource of type LoadBalancer must be created.
dklb will react to this event by provisioning an EdgeLB pool (henceforth referred to as the target EdgeLB pool) for the Service resource.
This EdgeLB pool is provisioned using sane default values for its name, its CPU, memory and size requirements, and other options.
After provisioning said EdgeLB pool, dklb will periodically query EdgeLB in order to obtain the list of hostnames and IPs at which the service can be reached.
These will eventually be reported on the .status field of the Service resource.
It should be noted that, due to the way EdgeLB pool scheduling and metadata reporting works, it may take from a few seconds to several minutes for these hostnames and IPs to be reported.
As mentioned before, dklb uses sane defaults when provisioning EdgeLB pools for Service resources of type LoadBalancer.
It is, however, possible to customize the target EdgeLB pool for a given Service resource by using the kubernetes.dcos.io/dklb-config annotation.
This annotation accepts a YAML object (henceforth called the configuration object) with the following structure:
kubernetes.dcos.io/dklb-config: | # NOTE: The "|" character is mandatory.
name: "dklb-pool-0"
role: "*"
network: "dcos"
size: 3
cpus: 0.2
memory: 512
frontends:
- port: 18080
servicePort: 8080
- port: 19090
servicePort: 9090The subsections below provide more insight on each of the fields on the configuration object.
|
ℹ️
|
The |
|
|
The kubernetes.dcos.io/dklb-config cannot be removed after the Service resource is created.
|
By default, dklb uses the MKE cluster’s name and a randomly-generated five-character suffix to compute the name of the target EdgeLB pool.
To specify a custom name for said EdgeLB pool, one may use the .name field of the configuration object:
kubernetes.dcos.io/dklb-config: |
name: "<edgelb-pool-name>"Depending on whether the <edgelb-pool-name> EdgeLB pool exists or not, dklb will create or update it in order to expose all ports defined in the Service resource.
|
❗
|
This field cannot be changed or removed after the Service resource is created.
|
By default, dklb exposes services to outside the DC/OS cluster by requesting for the target EdgeLB pool to be scheduled onto a public DC/OS agent.
However, and in order to accommodate all possible needs, dklb supports explicitly specifying a Mesos role for the target EdgeLB pool using the .role field of the configuration object:
kubernetes.dcos.io/dklb-config: |
role: "<edgelb-pool-role>"In particular, to expose a service to inside DC/OS only, * should be used as the value of <edgelb-pool-role>.
Providing said value will cause dklb to request for the target EdgeLB pool to be scheduled onto a private DC/OS agent.
|
❗
|
This field cannot be changed or removed after the Service resource is created.
|
dklb provisions the target EdgeLB pool by looking at the service’s ports and creating an EdgeLB backend and an EdgeLB frontend per port.
By default, dklb uses the port number that is defined on the Service resource as the frontend’s bind port.
In some situations, however, using a different port number as the frontend’s bind port may be required.[1]
In order to accommodate more advanced use cases, dklb supports defining custom port mappings via the .frontends field of the configuration object:
kubernetes.dcos.io/dklb-config: |
frontends:
- port: <frontend-bind-port>
servicePort: <service-port>When such an item is provided on the .frontends field, dklb will use <frontend-bind-port> instead of <service-port> as the actual frontend bind port.
|
|
Changing the value of a One should plan port mappings ahead whenever possible in order to prevent changes from being required in the first place. |
dklb supports customizing CPU, memory and size requests for the target EdgeLB pool.
Custom values for these requests can be specified using the .cpus, .memory and .size fields, respectively:
kubernetes.dcos.io/dklb-config: |
cpus: <edgelb-pool-cpus>
memory: <edgelb-pool-memory>
size: <edgelb-pool-size>In the above representation, <edgelb-pool-cpus> is a floating-point number (e.g. 0.2), and <edgelb-pool-memory> and <edgelb-pool-size> are integers (e.g. 512 and 3, respectively).
dklb supports customizing load balancer instance placement for the target EdgeLB Pool.
By default, no constraint is specified. A custom value can be specified using the constraints field.
|
|
Take special care to escape strings in the placement constraint. |
kubernetes.dcos.io/dklb-config: |
contraints: "<Marathon style constraints for load balancer instance placement>"By design, pools exposing Kubernetes services to outside the DC/OS cluster (i.e. pools using the slave_public role) will run atop the DC/OS agents' host network.
Also by design, pools exposing Kubernetes services to inside the DC/OS cluster will run atop the dcos virtual network.
It is, however, possible to override this and pick any https://docs.mesosphere.com/1.12/networking/SDN/usage/ [custom DC/OS virtual network] for these pools by using the .network field of the configuration object:
kubernetes.dcos.io/dklb-config: |
network: "<edgelb-pool-network>"|
❗
|
This field cannot be changed or removed after the Service resource is created.
|
In certain scenarios, it may be desirable to use a pre-existing EdgeLB pool to expose a Kubernetes service (instead of having dklb creating one).
This can easily be achieved by providing the name of the pre-existing EdgeLB pool as the value of the .name field of the configuration object.
To share an EdgeLB pool between two or more Kubernetes services, it is enough to provide the name of said pool as the value of the .name field of the configuration object in all of the corresponding Service resources.
When an EdgeLB pool is shared between two or more Kubernetes services, the following aspects should be taken into consideration:
-
The
.role,.network,.cpus,.memoryand.sizefields must have the exact same value across allServiceresources sharing an EdgeLB pool. -
Sharing an EdgeLB pool between services in different MKE clusters is allowed, but should be avoided whenever possible.
-
Changing or deleting one of the
Serviceresources exposed on a shared EdgeLB pool may cause disruption in all applications exposed on said EdgeLB pool.
This example illustrates how to expose a Redis instance to outside the DC/OS cluster.
To start with, a simple redis pod will be created:
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
labels:
app: redis
name: redis
spec:
containers:
- name: redis
image: redis:5.0.3
ports:
- name: redis
containerPort: 6379
protocol: TCP
EOF
pod/redis created$ kubectl get pod --selector "app=redis"
NAME READY STATUS RESTARTS AGE
redis 1/1 Running 0 100sThen, a Service resource of type LoadBalancer targeting the specified pod will also be created:
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Service
metadata:
annotations:
kubernetes.dcos.io/dklb-config: |
name: dklb-redis
frontends:
- port: 16379
servicePort: 6379
labels:
app: redis
name: redis
spec:
type: LoadBalancer
selector:
app: redis
ports:
- protocol: TCP
port: 6379
targetPort: 6379
EOF
service/redis created$ kubectl get svc --selector "app=redis"
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis LoadBalancer 10.100.80.96 <pending> 6379:32213/TCP 2m42sThe kubernetes.dcos.io/dklb-config defined on this Service resource will cause dklb to expose the service using an EdgeLB pool called dklb-redis, mapping the service’s 6379 port to the EdgeLB pool’s 16379 port.
At this point, querying the EdgeLB API should confirm the existence of a pool called dklb-redis exposing said port:
$ dcos edgelb list
NAME APIVERSION COUNT ROLE PORTS
dklb-redis V2 1 slave_public 9090, 16379This means that dklb has successfully created and provisioned the target EdgeLB pool based on the spec of the redis service.
To test connectivity, it is necessary to determine the public IP at which the target EdgeLB pool can be reached.
This IP will eventually be reported in the .status field of the Service resource:
$ kubectl get svc --selector "app=redis"
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis LoadBalancer 10.100.80.96 <public-dcos-agent-ip> 6379:32213/TCP 2m42stelnet may then be used to confirm that the Redis instance is correctly exposed to outside the DC/OS cluster:
$ telnet <public-dcos-agent-ip> 16379
Trying <public-dcos-agent-ip>...
Connected to <public-dcos-agent-ip>.
Escape character is '^]'.|
|
Depending on the firewall rules in place for the DC/OS cluster, it may be necessary to manually allow traffic to port 16379 in order to allow connectivity.
|
This means that the Redis instance is indeed reachable at <public-dcos-agent-ip>:16379 (i.e., from outside the DC/OS cluster).
Additional commands may be run inside telnet in order to verify that everything is working as expected:
$ telnet <public-dcos-agent-ip> 16379
(...)
SET foo bar
+OK
GET foo
$3
bar
QUIT
+OK
Connection closed by foreign host.