diff --git a/docs/cookbooks/ids/README.md b/docs/cookbooks/ids/README.md new file mode 100644 index 00000000000..52427bbc3fc --- /dev/null +++ b/docs/cookbooks/ids/README.md @@ -0,0 +1,175 @@ +# Using Antrea with IDS + +This guide will describe how to use Project Antrea with threat detection +engines, in order to provide network-based intrusion detection service to your +Pods. In this scenario, Antrea is used for the default Pod network. For the sake +of this guide, we will use [Suricata](https://suricata.io/) as the threat +detection engine, but similar steps should apply for other engines as well. + +The solution works by configuring a TrafficControl resource applying to specific +Pods. Traffic originating from the Pods or destined for the Pods is mirrored, +and then inspected by Suricata to provide threat detection. Suricata is +configured with IDS mode in this example, but it can also be configured with +IPS/inline mode to proactively drop the traffic determined to be malicious. + + +- [Prerequisites](#prerequisites) +- [Practical steps](#practical-steps) + - [Step 1: Deploy Antrea](#step-1-deploy-antrea) + - [Step 2: Configure TrafficControl resource](#step-2-configure-trafficcontrol-resource) + - [Step 3: Deploy Suricata as a DaemonSet](#step-3-deploy-suricata-as-a-daemonset) +- [Testing](#testing) + + +## Prerequisites + +The general prerequisites are: + +* a K8s cluster running a K8s version supported by Antrea. +* [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/) + +The [TrafficControl](../../traffic-control.md) capability was added in Antrea +version 1.7. Therefore, an Antrea version >= v1.7.0 should be used to configure +Pod traffic mirroring. + +All the required software will be deployed using YAML manifests, and the +corresponding container images will be downloaded from public registries. + +## Practical steps + +### Step 1: Deploy Antrea + +For detailed information on the Antrea requirements and instructions on how to +deploy Antrea, please refer to [getting-started.md](../../getting-started.md). +As of now, the `TrafficControl` feature gate is disabled by default, you will +need to enable it like the following command. + +To deploy the latest version of Antrea, use: + +```bash +curl -s https://raw.githubusercontent.com/antrea-io/antrea/main/build/yamls/antrea.yml | \ + sed "s/.*TrafficControl:.*/ TrafficControl: true/" | \ + kubectl apply -f - +``` + +You may also choose a [released Antrea +version](https://github.com/antrea-io/antrea/releases). + +### Step 2: Configure TrafficControl resource + +To replicate Pod traffic to Suricata for analysis, create a TrafficControl with +the `Mirror` action, and set the `targetPort` to an OVS internal port that +Suricata will capture traffic from. This cookbook uses `tap0` as the port name +and performs intrusion detection for Pods with the `app=web` label: + +```bash +cat <. + +As the TrafficControl resource configured in the second step mirrors traffic to +`tap0`, we run Suricata in the host network and specify the network interface to +`tap0`. + +```yaml +spec: + hostNetwork: true + containers: + - name: suricata + image: jasonish/suricata:latest + command: + - /usr/bin/suricata + - -i + - tap0 +``` + +Suricata uses Signatures (rules) to trigger alerts. We use the default ruleset +installed at `/var/lib/suricata/rules` of the image `jasonish/suricata`. + +The directory `/var/log/suricata` contains alert events. We mount the directory +as a `hostPath` volume to expose and persist them on the host: + +```yaml +spec: + containers: + - name: suricata + volumeMounts: + - name: host-var-log-suricata + mountPath: /var/log/suricata + volumes: + - name: host-var-log-suricata + hostPath: + path: /var/log/suricata + type: DirectoryOrCreate +``` + +To deploy Suricata, run: + +```bash +kubectl apply -f docs/cookbooks/ids/resources/suricata.yml +``` + +## Testing + +To test the IDS functionality, you can create a Pod with the `app=web` label, +using the following command: + +```bash +kubectl create deploy web --image nginx:1.21.6 +``` + +Let's log into the Node that the test Pod runs on and start `tail` to see +updates to the alert log `/var/log/suricata/fast.log`: + +```bash +tail -f /var/log/suricata/fast.log +``` + +You can then generate malicious requests to trigger alerts. For ingress traffic, +you can fake a web application attack against the Pod with the following command +(assuming that the Pod IP is 10.10.2.3): + +```bash +curl http://10.10.2.3/dlink/hwiz.html +``` + +The following output should now be seen in the log: + +```text +05/17/2022-04:29:51.717452 [**] [1:2008942:8] ET POLICY Dlink Soho Router Config Page Access Attempt [**] [Classification: Attempted Administrator Privilege Gain] [Priority: 1] {TCP} 10.10.2.1:48600 -> 10.10.2.3:80 +``` + +For egress traffic, you can `kubectl exec` into the Pods and generate malicious +requests against external web server with the following command: + +```bash +kubectl exec deploy/web -- curl -s http://testmynids.org/uid/index.html +``` + +The following output should now be seen in the log: + +```text +05/17/2022-04:36:46.706373 [**] [1:2013028:6] ET POLICY curl User-Agent Outbound [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 10.10.2.3:55132 -> 65.8.161.92:80 +05/17/2022-04:36:46.708833 [**] [1:2100498:7] GPL ATTACK_RESPONSE id check returned root [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 65.8.161.92:80 -> 10.10.2.3:55132 +``` diff --git a/docs/cookbooks/ids/resources/suricata.yml b/docs/cookbooks/ids/resources/suricata.yml new file mode 100644 index 00000000000..279d3c82d49 --- /dev/null +++ b/docs/cookbooks/ids/resources/suricata.yml @@ -0,0 +1,37 @@ +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: suricata +spec: + selector: + matchLabels: + app: suricata + template: + metadata: + labels: + app: suricata + name: suricata + spec: + hostNetwork: true + containers: + - name: suricata + image: jasonish/suricata:latest + imagePullPolicy: IfNotPresent + command: + - /usr/bin/suricata + - -i + - tap0 + securityContext: + capabilities: + add: + - NET_ADMIN + - NET_RAW + - SYS_NICE + volumeMounts: + - name: host-var-log-suricata + mountPath: /var/log/suricata + volumes: + - name: host-var-log-suricata + hostPath: + path: /var/log/suricata + type: DirectoryOrCreate diff --git a/docs/feature-gates.md b/docs/feature-gates.md index 023251f56de..e9678bbd884 100644 --- a/docs/feature-gates.md +++ b/docs/feature-gates.md @@ -48,6 +48,7 @@ example, to enable `AntreaProxy` on Linux, edit the Agent configuration in the | `Multicast` | Agent | `false` | Alpha | v1.5 | N/A | N/A | Yes | | | `SecondaryNetwork` | Agent | `false` | Alpha | v1.5 | N/A | N/A | Yes | | | `ServiceExternalIP` | Agent + Controller | `false` | Alpha | v1.5 | N/A | N/A | Yes | | +| `TrafficControl` | Agent | `false` | Alpha | v1.7 | N/A | N/A | No | | ## Description and Requirements of Features @@ -323,3 +324,13 @@ Refer to this [document](service-loadbalancer.md) for more information. #### Requirements for this Feature This feature is currently only supported for Nodes running Linux. + +### TrafficControl + +`TrafficControl` enables a CRD API for Antrea that controls and manipulates the +transmission of Pod traffic. It allows users to mirror or redirect traffic +originating from specific Pods or destined for specific Pods to a local network +device or a remote destination via a tunnel of various types. It enables a +monitoring solution to get full visibility into network traffic, including both +north-south and east-west traffic. Refer to this [document](traffic-control.md) +for more information. diff --git a/docs/traffic-control.md b/docs/traffic-control.md new file mode 100644 index 00000000000..5998192c966 --- /dev/null +++ b/docs/traffic-control.md @@ -0,0 +1,278 @@ +# Traffic Control With Antrea + +## Table of Contents + + +- [What is TrafficControl?](#what-is-trafficcontrol) +- [Prerequisites](#prerequisites) +- [The TrafficControl resource](#the-trafficcontrol-resource) + - [AppliedTo](#appliedto) + - [Direction](#direction) + - [Action](#action) + - [TargetPort](#targetport) + - [ReturnPort](#returnport) +- [Examples](#examples) + - [Mirroring all traffic to remote analyzer](#mirroring-all-traffic-to-remote-analyzer) + - [Redirecting specific traffic to local receiver](#redirecting-specific-traffic-to-local-receiver) +- [What's next](#whats-next) + + +## What is TrafficControl? + +`TrafficControl` is a CRD API that manages and manipulates the transmission of +Pod traffic. It allows users to mirror or redirect specific traffic originating +from specific Pods or destined for specific Pods to a local network device or a +remote destination via a tunnel of various types. It provides full visibility +into network traffic, including both north-south and east-west traffic. + +You may be interested in using this capability if any of the following apply: + +- You want to monitor network traffic passing in or out of a set of Pods for + purposes such as troubleshooting, intrusion detection, and so on. + +- You want to redirect network traffic passing in or out of a set of Pods to + applications that enforce policies, and reject traffic to prevent intrusion. + +This guide demonstrates how to configure `TrafficControl` to achieve the above +goals. + +## Prerequisites + +TrafficControl was introduced in v1.7 as an alpha feature. A feature gate, +`TrafficControl` must be enabled on the antrea-agent in the `antrea-config` +ConfigMap for the feature to work, like the following: + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: antrea-config + namespace: kube-system +data: + antrea-agent.conf: | + featureGates: + TrafficControl: true +``` + +## The TrafficControl resource + +A TrafficControl in Kubernetes is a REST object. Like all the REST objects, you +can POST a TrafficControl definition to the API server to create a new instance. +For example, supposing you have a set of Pods which contain a label `app=web`, +the following specification creates a new TrafficControl object named +"mirror-web-app", which mirrors all traffic from or to any Pod with the +`app=web` label and send them to a receiver running on "10.0.10.2" encapsulated +within a VXLAN tunnel: + +```yaml +apiVersion: crd.antrea.io/v1alpha2 +kind: TrafficControl +metadata: + name: mirror-web-app +spec: + appliedTo: + podSelector: + matchLabels: + app: web + direction: Both + action: Mirror + targetPort: + vxlan: + remoteIP: 10.0.10.2 +``` + +### AppliedTo + +The `appliedTo` field specifies the grouping criteria of Pods to which the +TrafficControl applies to. Pods can be selected cluster-wide using +`podSelector`. If set with a `namespaceSelector`, all Pods from Namespaces +selected by the `namespaceSelector` will be selected. Specific Pods from +specific Namespaces can be selected by providing both a `podSelector` and a +`namespaceSelector`. Empty `appliedTo` selects nothing. The field is mandatory. + +### Direction + +The `direction` field specifies the direction of traffic that should be matched. +It can be `Ingress`, `Egress`, or `Both`. + +### Action + +The `action` field specifies which action should be taken for the traffic. It +can be `Mirror` or `Redirect`. For the `Mirror` action, `targetPort` must be +set to the port to which the traffic will be mirrored. For the `Redirect` +action, both `targetPort` and `returnPort` need to be specified, the latter of +which represents the port from which the traffic could be sent back to OVS and +be forwarded to its original destination. Once redirected, a packet should be +either dropped or sent back to OVS without modification, otherwise it would lead +to undefined behavior. + +### TargetPort + +The `targetPort` field specifies the port to which the traffic should be +redirected or mirrored. There are five kinds of ports that can be used to +receive mirrored traffic: + +**ovsInternal**: This specifies an OVS internal port on all Nodes. A Pod's +traffic will be redirected or mirrored to the OVS internal port on the same Node +that hosts the Pod. The port doesn't need to exist in advance, Antrea will +create the port if it doesn't exist. To use an OVS internal port, the `name` of +the port must be provided: + +```yaml +ovsInternal: + name: tap0 +``` + +**device**: This specifies a network device on all Nodes. A Pod's traffic will +be redirected or mirrored to the network device on the same Node that hosts the +Pod. The network device must exist on all Nodes and Antrea will attach it to the +OVS bridge if not already attached. To use a network device, the `name` of the +device must be provided: + +```yaml +device: + name: eno2 +``` + +**geneve**: This specifies a remote destination for a GENEVE tunnel. All +selected Pods' traffic will be redirected or mirrored to the destination via +a GENEVE tunnel. The `remoteIP` field must be provided to specify the IP address +of the destination. Optionally, the `destinationPort` field could be used to +specify the UDP destination port of the tunnel, or 6081 will be used by default. +If Virtual Network Identifier (VNI) is desired, the `vni` field can be specified +to an integer in the range 0-16,777,215: + +```yaml +geneve: + remoteIP: 10.0.10.2 + destinationPort: 6081 + vni: 1 +``` + +**vxlan**: This specifies a remote destination for a VXLAN tunnel. All +selected Pods' traffic will be redirected or mirrored to the destination via +a VXLAN tunnel. The `remoteIP` field must be provided to specify the IP address +of the destination. Optionally, the `destinationPort` field could be used to +specify the UDP destination port of the tunnel, or 4789 will be used by default. +If Virtual Network Identifier (VNI) is desired, the `vni` field can be specified +to an integer in the range 0-16,777,215: + +```yaml +vxlan: + remoteIP: 10.0.10.2 + destinationPort: 4789 + vni: 1 +``` + +**gre**: This specifies a remote destination for a GRE tunnel. All selected +Pods' traffic will be redirected or mirrored to the destination via a GRE +tunnel. The `remoteIP` field must be provided to specify the IP address of the +destination. If GRE key is desired, the `key` field can be specified to an +integer in the range 0-4,294,967,295: + +```yaml +gre: + remoteIP: 10.0.10.2 + key: 1 +``` + +**erspan**: This specifies a remote destination for an ERSPAN tunnel. All +selected Pods' traffic will be mirrored to the destination via an ERSPAN tunnel. +The `remoteIP` field must be provided to specify the IP address of the +destination. If ERSPAN session ID is desired, the `sessionID` field can be +specified to an integer in the range 0-1,023. The `version` field must be +provided to specify the ERSPAN version: 1 for version 1 (type II), or 2 for +version 2 (type III). + +For version 1, the `index` field can be specified to associate with the ERSPAN +traffic's source port and direction. An example of version 1 might look like +this: + +```yaml +erspan: + remoteIP: 10.0.10.2 + sessionID: 1 + version: 1 + index: 1 +``` + +For version 2, the `dir` field can be specified to indicate the mirrored +traffic's direction: 0 for ingress traffic, 1 for egress traffic. The +`hardwareID` field can be specified as an unique identifier of an ERSPAN v2 +engine. An example of version 2 might look like this: + +```yaml +erspan: + remoteIP: 10.0.10.2 + sessionID: 1 + version: 2 + dir: 0 + hardwareID: 4 +``` + +### ReturnPort + +The `returnPort` field should only be set when the `action` is `Redirect`. It is +similar to the `targetPort` field, but meant for specifying the port from which +the traffic will be sent back to OVS and be forwarded to its original +destination. + +## Examples + +### Mirroring all traffic to remote analyzer + +In this example, we will mirror all Pods' traffic and send them to a remote +destination via a GENEVE tunnel: + +```yaml +apiVersion: crd.antrea.io/v1alpha2 +kind: TrafficControl +metadata: + name: mirror-all-to-remote +spec: + appliedTo: + podSelector: {} + direction: Both + action: Mirror + targetPort: + geneve: + remoteIP: 10.0.10.2 +``` + +### Redirecting specific traffic to local receiver + +In this example, we will redirect traffic of all Pods in the Namespace `prod` to +OVS internal ports named `tap0` configured on Nodes that these Pods run on. +The `returnPort` configuration means, if the traffic is sent back to OVS from +OVS internal ports named `tap1`, it will be forwarded to its original +destination. Therefore, if an intrusion prevention system or a network firewall +is configured to capture and forward traffic between `tap0` and `tap1`, it can +actively scan forwarded network traffic for malicious activities and known +attack patterns, and drop the traffic determined to be malicious. + +```yaml +apiVersion: crd.antrea.io/v1alpha2 +kind: TrafficControl +metadata: + name: redirect-prod-to-local +spec: + appliedTo: + namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: prod + direction: Both + action: Mirror + targetPort: + ovsInternal: + name: tap0 + returnPort: + ovsInternal: + name: tap1 +``` + +## What's next + +With the `TrafficControl` capability, Antrea can be used with threat detection +engines to provide network-based IDS/IPS to Pods. We provide a reference +cookbook on how to implement IDS using Suricata. For more information, refer to +the [cookbook](cookbooks/ids).