Skip to content

Commit

Permalink
Document the new xff_num_trusted_hops feature (#479)
Browse files Browse the repository at this point in the history 
Envoy issue #2503
Corresponding code PR #2559

Signed-off-by: Brian Pane <[email protected]>
  • Loading branch information
@brian-pane @htuch
brian-pane authored and htuch committed Feb 13, 2018
1 parent 89a0c4d commit d3af0e5
Show file tree
Hide file tree
Showing 2 changed files with 116 additions and 18 deletions.
133 changes: 116 additions & 17 deletions docs/root/configuration/http_conn_man/headers.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,12 @@ the :option:`--service-node` option.
x-envoy-external-address
------------------------

It is a common case where a service wants to perform analytics based on the client IP address. Per
the lengthy discussion on :ref:`XFF <config_http_conn_man_headers_x-forwarded-for>`, this can get
quite complicated. A proper implementation involves forwarding XFF, and then choosing the first non
RFC1918 address *from the right*. Since this is such a common occurrence, Envoy simplifies this by
setting *x-envoy-external-address* during decoding if and only if the request ingresses externally
(i.e., it's from an external client). *x-envoy-external-address* is not set or overwritten for
internal requests. This header can be safely forwarded between internal services for analytics
It is a common case where a service wants to perform analytics based on the origin client's IP
address. Per the lengthy discussion on :ref:`XFF <config_http_conn_man_headers_x-forwarded-for>`,
this can get quite complicated, so Envoy simplifies this by setting *x-envoy-external-address*
to the :ref:`trusted client address <config_http_conn_man_headers_x-forwarded-for_trusted_client_address>`
if the request is from an external client. *x-envoy-external-address* is not set or overwritten
for internal requests. This header can be safely forwarded between internal services for analytics
purposes without having to deal with the complexities of XFF.

.. _config_http_conn_man_headers_x-envoy-force-trace:
Expand Down Expand Up @@ -156,8 +155,10 @@ operates in a transparent mode where it does not modify XFF.
.. attention::

In general, *use_remote_address* should be set to true when Envoy is deployed as an edge
node, whereas it may need to be set to false when Envoy is used as an internal service node
in a mesh deployment.
node (aka a front proxy), whereas it may need to be set to false when Envoy is used as
an internal service node in a mesh deployment.

.. _config_http_conn_man_headers_x-forwarded-for_trusted_client_address:

The value of *use_remote_address* controls how Envoy determines the *trusted client address*.
Given an HTTP request that has traveled through a series of zero or more proxies to reach
Expand All @@ -166,24 +167,116 @@ accurate. The source IP address of the immediate downstream node's connection to
trusted. XFF *sometimes* can be trusted. Malicious clients can forge XFF, but the last
address in XFF can be trusted if it was put there by a trusted proxy.

Envoy's rules for determining the trusted client address are:
Envoy's default rules for determining the trusted client address (*before* appending anything
to XFF) are:

* If *use_remote_address* is false and an XFF containing at least one IP address is
present in the request, the trusted client address is the *last* (rightmost) IP address in XFF.
* Otherwise, the trusted client address is the source IP address of the immediate downstream
node's connection to Envoy.

In an environment where there are one or more trusted proxies in front of an edge
Envoy instance, the *xff_num_trusted_hops* configuration option can be used to trust
additional addresses from XFF:

* If *use_remote_address* is false and *xff_num_trusted_hops* is set to a value *N* that is
greater than zero, the trusted client address is the (N+1)th address from the right end
of XFF. (If the XFF contains fewer than N+1 addresses, Envoy falls back to using the
immediate downstream connection's source address as trusted client address.)
* If *use_remote_address* is true and *xff_num_trusted_hops* is set to a value *N* that is
greater than zero, the trusted client address is the Nth address from the right end
of XFF. (If the XFF contains fewer than N addresses, Envoy falls back to using the
immediate downstream connection's source address as trusted client address.)

Envoy uses the trusted client address contents to determine whether a request originated
externally or internally. This influences whether the
:ref:`config_http_conn_man_headers_x-envoy-internal` header is set.

Testing IPv6 in a large multi-hop system can be difficult from a change management perspective. For
testing IPv6 compatibility of upstream services which parse XFF header values,
:ref:`represent_ipv4_remote_address_as_ipv4_mapped_ipv6
<envoy_api_field_config.filter.network.http_connection_manager.v2.HttpConnectionManager.represent_ipv4_remote_address_as_ipv4_mapped_ipv6>`
can be enabled in the v2 API. Envoy will append an IPv4 address in mapped IPv6 format, e.g.
::FFFF:50.0.0.1. This change will also apply to
:ref:`config_http_conn_man_headers_x-envoy-external-address`.
Example 1: Envoy as edge proxy, without a trusted proxy in front of it
Settings:
| use_remote_address = true
| xff_num_trusted_hops = 0
Request details:
| Downstream IP address = 192.0.2.5
| XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1"
Result:
| Trusted client address = 192.0.2.5 (XFF is ignored)
| X-Envoy-External-Address is set to 192.0.2.5
| XFF is changed to "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5"
| X-Envoy-Internal is removed (if it was present in the incoming request)
Example 2: Envoy as internal proxy, with the Envoy edge proxy from Example 1 in front of it
Settings:
| use_remote_address = false
| xff_num_trusted_hops = 0
Request details:
| Downstream IP address = 10.11.12.13 (address of the Envoy edge proxy)
| XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5"
Result:
| Trusted client address = 192.0.2.5 (last address in XFF is trusted)
| X-Envoy-External-Address is not modified
| X-Envoy-Internal is removed (if it was present in the incoming request)
Example 3: Envoy as edge proxy, with two trusted external proxies in front of it
Settings:
| use_remote_address = true
| xff_num_trusted_hops = 2
Request details:
| Downstream IP address = 192.0.2.5
| XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1"
Result:
| Trusted client address = 203.0.113.10 (2nd to last address in XFF is trusted)
| X-Envoy-External-Address is set to 203.0.113.10
| XFF is changed to "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5"
| X-Envoy-Internal is removed (if it was present in the incoming request)
Example 4: Envoy as internal proxy, with the edge proxy from Example 3 in front of it
Settings:
| use_remote_address = false
| xff_num_trusted_hops = 2
Request details:
| Downstream IP address = 10.11.12.13 (address of the Envoy edge proxy)
| XFF = "203.0.113.128, 203.0.113.10, 203.0.113.1, 192.0.2.5"
Result:
| Trusted client address = 203.0.113.10
| X-Envoy-External-Address is not modified
| X-Envoy-Internal is removed (if it was present in the incoming request)
Example 5: Envoy as an internal proxy, receiving a request from an internal client
Settings:
| use_remote_address = false
| xff_num_trusted_hops = 0
Request details:
| Downstream IP address = 10.20.30.40 (address of the internal client)
| XFF is not present
Result:
| Trusted client address = 10.20.30.40
| X-Envoy-External-Address remains unset
| X-Envoy-Internal is set to "true"
Example 6: The internal Envoy from Example 5, receiving a request proxied by another Envoy
Settings:
| use_remote_address = false
| xff_num_trusted_hops = 0
Request details:
| Downstream IP address = 10.20.30.50 (address of the Envoy instance proxying to this one)
| XFF = "10.20.30.40"
Result:
| Trusted client address = 10.20.30.40
| X-Envoy-External-Address remains unset
| X-Envoy-Internal is set to "true"
A few very important notes about XFF:

Expand All @@ -206,6 +299,12 @@ A few very important notes about XFF:
Envoy will not consider it internal. This is a known "bug" due to the simplification of how
XFF is parsed to determine if a request is internal. In this scenario, do not forward XFF and
allow Envoy to generate a new one with a single internal origin IP.
3. Testing IPv6 in a large multi-hop system can be difficult from a change management perspective.
For testing IPv6 compatibility of upstream services which parse XFF header values,
:ref:`represent_ipv4_remote_address_as_ipv4_mapped_ipv6 <envoy_api_field_config.filter.network.http_connection_manager.v2.HttpConnectionManager.represent_ipv4_remote_address_as_ipv4_mapped_ipv6>`
can be enabled in the v2 API. Envoy will append an IPv4 address in mapped IPv6 format, e.g.
::FFFF:50.0.0.1. This change will also apply to
:ref:`config_http_conn_man_headers_x-envoy-external-address`.

.. _config_http_conn_man_headers_x-forwarded-proto:

Expand Down
1 change: 0 additions & 1 deletion envoy/config/filter/network/http_connection_manager/v2/http_connection_manager.proto
View file
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,6 @@ message HttpConnectionManager {
// :ref:`config_http_conn_man_headers_x-envoy-external-address` for more information.
google.protobuf.BoolValue use_remote_address = 14;

// [#not-implemented-hide:]
// The number of additional ingress proxy hops from the right side of the
// :ref:`config_http_conn_man_headers_x-forwarded-for` HTTP header to trust when
// determining the origin client's IP address. The default is zero if this option
Expand Down

0 comments on commit d3af0e5

Please sign in to comment.