Skip to content

Commit

Permalink
update protocol detection doc
Browse files Browse the repository at this point in the history
- Add NATS to the list of protocols that need opaque ports (not sure how this
  was dropped)
- Instruct that the resource should always be set on the Service object.

Signed-off-by: William Morgan <[email protected]>
  • Loading branch information
wmorgan committed Sep 29, 2024
1 parent c08a91b commit cc98d98
Showing 1 changed file with 32 additions and 33 deletions.
65 changes: 32 additions & 33 deletions linkerd.io/content/2.16/features/protocol-detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ If Linkerd *cannot* determine that a connection is using HTTP or HTTP/2,
Linkerd will proxy the connection as a plain TCP connection, applying
[mTLS](../automatic-mtls/) and providing byte-level metrics as usual.

(Note that HTTPS calls to or from meshed pods are treated as TCP, not as HTTP.
Because the client initiates the TLS connection, Linkerd is not be able to
decrypt the connection to observe the HTTP transactions.)
(Note that HTTPS calls pods are treated as TCP, not as HTTP. Because the client
initiates the TLS connection, Linkerd is not be able to decrypt the connection
to observe the HTTP traffic.)

## Configuring protocol detection

Expand All @@ -34,11 +34,13 @@ This section will help you understand how to fix this.
{{< /note >}}

In some cases, Linkerd's protocol detection will time out because it doesn't see
any bytes from the client. This situation is commonly encountered when using
protocols where the server sends data before the client does (such as SMTP) or
protocols that proactively establish connections without sending data (such as
Memcache). In this case, the connection will proceed as a TCP connection after a
10-second protocol detection delay.
any bytes from the client within the first 10 seconds of connection
establishment. This is commonly encountered when using protocols where the
server sends data before the client does (such as SMTP) or protocols that
proactively establish connections without sending data (such as Memcache). In
this case, Linkerd will wait for 10 seconds before establishing a connection to
the server. After this delay, the connection will be esablished and treated by
Linkerd as a TCP connection.

To avoid this delay, you will need to provide some configuration for Linkerd.
There are two basic mechanisms for configuring protocol detection: _opaque
Expand All @@ -49,29 +51,28 @@ ports_ and _skip ports_:
* Skip ports bypass the proxy entirely.

Opaque ports are generally preferred as they allow Linkerd to provide mTLS,
TCP-level metrics, policy, etc. Skip ports circumvent Linkerd's ability to
provide security features.
TCP-level metrics, policy, etc. Skip ports, by contrast, create networking rules
that avoid the proxy entirely, circumventing Linkerd's ability to provide
security features.

Linkerd maintains a default list of opaque ports that corresponds to the
standard ports used by protocols that interact poorly with protocol detection.
As of the 2.12 release, that list is: **25** (SMTP), **587** (SMTP), **3306**
(MySQL), **4444** (Galera), **5432** (Postgres), **6379** (Redis), **9300**
(ElasticSearch), and **11211** (Memcache).

## Protocols that may require configuration

The following table contains common protocols that may require additional
configuration.

| Protocol | Standard port(s) | In default list? | Notes |
| Protocol | Standard ports | In default list? | Notes |
|-----------------|------------------|------------------|-------|
| SMTP | 25, 587 | Yes | |
| MySQL | 3306 | Yes | |
| MySQL with Galera | 3306, 4444, 4567, 4568 | Partially | Ports 4567 and 4568 are not in Linkerd's default set of opaque ports |
| MySQL with Galera | 3306, 4444, 4567, 4568 | Partially | Ports 4567 and 4568 are not in Linkerd's default list of opaque ports |
| PostgreSQL | 5432 | Yes | |
| Redis | 6379 | Yes | |
| ElasticSearch | 9300 | Yes | |
| Memcache | 11211 | Yes | |
| NATS | 4222, 6222, 8222 | No | |

If you are using one of those protocols, follow this decision tree to determine
which configuration you need to apply.
Expand All @@ -81,27 +82,31 @@ which configuration you need to apply.
## Marking ports as opaque

You can use the `config.linkerd.io/opaque-ports` annotation to mark a port as
opaque. Note that this annotation should be set on the _destination_, not on the
source, of the traffic.
opaque. Linkerd will skip protocol detection on opaque ports, and treat
connections to them as TCP streams.

This annotation can be set in a variety of ways:
This annotation should be set on the _destination_, not on the source, of the
traffic. This is true even if the destination is unmeshed, as it controls the
behavior of meshed clients.

1. On the workload itself, e.g. on the Deployment's Pod spec receiving the traffic.
This annotation *must* be set in two places:
1. On the Service receiving the traffic.
1. On a namespace (in which it will apply to all workloads in the namespace).
1. In an [authorization policy](../server-policy/) `Server` object's
`proxyProtocol` field, in which case it will apply to all pods targeted by that
`Server`.

When set, Linkerd will skip protocol detection both on the client side and on
the server side. Note that since this annotation informs the behavior of meshed
_clients_, it can be applied to unmeshed workloads as well as meshed ones.
2. On the workload itself (e.g. on the Deployment's Pod spec receiving the
traffic), or on enclosing namespace, in which it will apply to all workloads in
the namespace.

{{< note >}}
Multiple ports can be provided as a comma-delimited string. The values you
provide will _replace_, not augment, the default list of opaque ports.
{{< /note >}}

{{< note >}}
If you are using [authorization policies](../server-policy/), the `Server`'s
`proxyProtocol` field which can be used to control protocol detection behavior
and can be used instead of a Service annotation. Regardless, we suggest
annotating the Service object for clarity.
{{< /note >}}

## Marking ports as skip ports

Sometimes it is necessary to bypass the proxy altogether. In this case, you can
Expand All @@ -128,12 +133,6 @@ Note that the default set of opaque ports can be configured at install
time, e.g. by using `--set proxy.opaquePorts`. This may be helpful in
conjunction with `enable-external-profiles`.

{{< note >}}
There was a bug in Linkerd 2.11.0 and 2.11.1 that prevented the opaque ports
behavior of `enable-external-profiles` from working. This was fixed in Linkerd
2.11.2.
{{< /note >}}

## Using `NetworkPolicy` resources with opaque ports

When a service has a port marked as opaque, any `NetworkPolicy` resources that
Expand Down

0 comments on commit cc98d98

Please sign in to comment.