caddyhttp: Security enhancements for client IP parsing

[nebez]

This is the first stab at addressing #5804

Commits are readable in order. This review is easiest done by going through them one-by-one:

• cover existing client_ip_headers functionality
• cover existing trusted_proxy functionality with custom ranges
• write tests covering both scenarios above in combination
• introduce a failing test proving the potential security weakness of existing trusted_proxy + client_ip_headers configuration
• introduce a new configuration option (default: false) to opt-in to a safer method of parsing client_ip_headers

The first 3 commits provide confidence in maintaining backwards compatibility for those who rely on the existing, left-most parsing of XFF headers. The remaining commits introduce tests + config options (default off) to enable the new right-most, first untrusted IP XFF parsing.

[CLAassistant]

All committers have signed the CLA.

[mholt]

Awesome, thanks for working on this! :)

[nebez]

@francislavoie I've renamed the options, I like this new name better. Ty for suggestion.

@francislavoie @mholt can I suggest this be a patch version of current 2.7.x release? This is a backwards compatible change and keeps the left-most parsing as a default. As-is, the trusted_proxies default parsing of XFF headers opts users into a non-configurable behaviour that injects insecure client_ip headers (and then forwards them down). Unless you have 100% trust in every upstream, this is the least secure approach to XFF parsing per the 2 sources in my original issue.

For my web services behind caddy, I've disabled trusted_proxies and manually pass XFF headers to downstreams. This lets me extract the right-most client IP myself downstream, but at the cost of losing client_ip's in caddy. For us this is an acceptable tradeoff off until the mode can be made strict (right-most parsing) via this option. I imagine more users would also expect similar behaviour.

In addition, if you point me to the right direction, I will also include changes to the documentation of the website to mention this new opt-in option. Users should understand the behaviour of left-most vs. right-most XFF parsing and the security dangers of using left-most on untrusted upstreams.

edit: I've put up a PR for documentation here caddyserver/website#346

[mholt]

Sorry for my latency on this, been a very busy time -- it is still on my radar though! I will circle back soon enough.

can I suggest this be a patch version of current 2.7.x release?

I think we can do that since this is basically a bugfix / not a new feature.

[nebez]

No worries.

When you've got the time, can you please also review caddyserver/website#346 as part of the same change?

[mholt]

Yes we will try to do that :D

(I will be a little busy for a while with some family obligations.)

[lachlan2k]

Great PR! I look forward to support for this case landing in Caddy.

I must say though, I prefer the option that was suggested in the original issue (#5804) of instead having trusted_proxy_count <n>, as this adds a lot more flexibility, and conveniently matches the terminology of the MDN docs on XFF.

There are a lot of use cases where there could be an intermediary proxy, such as WAF -> Load Balancer -> Caddy, where it is useful to trust the second right-most value. I.e., trusting C in X-Forwarded-For: A, B, C, D.

Matching the functionality of this PR would be achieved by setting trusted_proxy_count 1, and the use case I described above would be trusted_proxy_count 2.

I'd love to hear other perspectives on this.

[mholt]

I'll also be able to revisit this PR soon @nebez

[francislavoie]

LGTM! Sorry for the delay. We had a feature freeze which we eventually cancelled, then the holidays.

Thanks for the thorough tests. This is only an opt-in behaviour change, so I'm comfortable to merge this safely. If there's any issues that crop up, they can be fixed in a followup.

[francislavoie]

Actually, I'll make a tiny adjustment to this right now as per @lachlan2k's comments.

I think it's valid to say that count should be supported. Since the current "strict" behaviour effectively maps to count of "1", we can change the type from bool to int and make it 1 if set. That way, we can later implement trusted_proxies_strict <count> without it being a breaking change.

I don't want to block merging this any longer for an additional feature that can be added after the fact.

[nebez]

Actually, I'll make a tiny adjustment to this right now as per @lachlan2k's comments.

I think it's valid to say that count should be supported. Since the current "strict" behaviour effectively maps to count of "1", we can change the type from bool to int and make it 1 if set. That way, we can later implement trusted_proxies_strict <count> without it being a breaking change.

I don't want to block merging this any longer for an additional feature that can be added after the fact.

That sounds quite pragmatic to me. Thanks for tackling that @francislavoie

Once you're comfortable merging + tagging it, I'll update caddyserver/website#346 and then it will be ready for your review