Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set a bounded default for http.max_warning_header_count #33479

Closed
tomcallahan opened this issue Sep 6, 2018 · 4 comments
Closed

Set a bounded default for http.max_warning_header_count #33479

tomcallahan opened this issue Sep 6, 2018 · 4 comments
Labels

Comments

@tomcallahan
Copy link
Contributor

Today we do not by default limit the number of HTTP Warning headers emitted when users of Elasticsearch trigger deprecated functionality. As of 6.4.0 we will emit a warning for each docvalue_fields item without a format parameter in an msearch query, which is currently leveraged
by Kibana when using the Discover feature for instance.

Here is an example message, of which there can be many:
#! Deprecation: Doc-value field [date2] is not using a format. The output will change in 7.0 when doc value fields get formatted based on mappings by default. It is recommended to pass [format=use_field_mapping] with the doc value field in order to opt in for the future behaviour and ease the migration to 7.0.

In the context of accessing Elasticsearch through a proxy this can be problematic, as a large number of header records can cause proxies to fail.
This can be difficult behavior to debug/diagnose, as proxies convey errors such as Bad Gateway. I'd like to discuss setting a bounded default (at a minimum by header count, but perhaps also by header bytes).

The Kibana side of this issue is located in elastic/kibana#22484.

Further reading on user impact: https://discuss.elastic.co/t/bad-gateway-errors-in-discover-in-kibana-6-4-on-some-indices/146576

@tomcallahan tomcallahan added >enhancement blocker :Distributed/Network Http and internode communication implementations v7.0.0 v6.4.1 labels Sep 6, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@jasontedor jasontedor added :Core/Infra/Core Core issues without another label and removed :Distributed/Network Http and internode communication implementations v6.4.1 labels Sep 6, 2018
@jasontedor
Copy link
Member

I disagree with bounding these by default as it leads to silent truncation. It is much easier for a user to discover and solve the problem of "there are too many warning headers coming back and it's blowing up my proxy" versus the lack of ability to easily discover "there are too many warnings headers and they are silently being truncated". I have explained this previously here.

And no matter what we agree on, we would never change this in a minor let alone a patch release. Therefore I am removing the v6.4.1 label.

@jasontedor
Copy link
Member

jasontedor commented Sep 6, 2018

And as I explained in my comment, this does not preclude Cloud from introducing a default, and Cloud is one of the use-cases that we had in mind for these limits being configurable. Our intention from the beginning in #28301 is that this be unbounded by default in our distributions though. Note that the use of a proxy is not guaranteed, a default would be harmful to users without proxies, and harmful to users that have proxies that do not impose limits here.

@jasontedor
Copy link
Member

Superseded by #33508

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants