Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Header parsing for Rest versioning #52370

Open
pgomulka opened this issue Feb 14, 2020 · 7 comments
Open

Header parsing for Rest versioning #52370

pgomulka opened this issue Feb 14, 2020 · 7 comments
Labels
:Core/Infra/REST API REST infrastructure and utilities feedback_needed Team:Core/Infra Meta label for core/infra team v9.0.0

Comments

@pgomulka
Copy link
Contributor

pgomulka commented Feb 14, 2020

when accessing a compatible API a user has to specify a version its client is compatible with
Some people prefer to use a custom header for that purpose but it is also a common practice on the internet to use Accept and/or Content-Type for that purpose.
The argument against custom header is that it might often be filtered by firewalls or proxies.
This however might also happen with non IANA registered custom values of Accept/Content-Type.
There are several 'patterns' that people follow for values of Accept/Content-Type. Most popular are in a form:
application/vnd.elasticsearch.v7+json - however if we ever decide to register our subtype then we would have to do it for every new version. Also does not indicate well enought that the API is only compatible.

application/vnd.elasticsearch+json; compatibleWith= 7 - this seems to be the most preferable to me, it indicates the intention that the API is not version 7 but only compatible. Also registration would be easier. The subtype will not change - vnd.elasticsearch and we will just define that compatibleWith expects a major version number.

relates #51816

@pgomulka pgomulka added :Core/Infra/REST API REST infrastructure and utilities v8.0.0 labels Feb 14, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/REST API)

@pgomulka pgomulka changed the title Implement header parsing for Rest versioning Header parsing for Rest versioning Feb 18, 2020
@bpintea
Copy link
Contributor

bpintea commented Feb 18, 2020

Do you refer to the Accept with "Accept-Type"? Asking since I couldn't find any reference to an HTTP Accept-Type header.

Also, wondering about considerations about (or thoughts against) using query strings (or URL params) to signal the desired compatibility, besides using already registered media types in the Accept/Content-Type headers (the content still is "plain" JSON/CBOR etc.). This way the use of HTTP would remain vanilla and thus maybe safe in face of mid-way undesired modifications.

@pgomulka
Copy link
Contributor Author

@bpintea right Accept - fixed the first comment. Thank you

I think the path param - or any path modification - will be problematic as some users will hardcode permalinks to the api within their code.

@bpintea
Copy link
Contributor

bpintea commented Feb 18, 2020

some users will hardcode permalinks to the api within their code.

Wouldn't the code that adds a custom media type in the header be in "proximity" to the code that handles the URL? I'm not in the clear why one would be significantly more difficult than the other, but I admit I don't have a great overview in this regard.

However, not having to mangle the URL and have the header "separation" does feel cleaner. Furthermore I see that RFC6838 does mandate the proposed format so it should be standards-compliant to a certain degree even without the actual IANA registration. (And one can find further endorsement in literature on the topic.)

Which are arguments in favour of the original proposal.

pgomulka added a commit that referenced this issue Mar 25, 2020
Providing a REST infrastructure to handle compatible APIs. Rest APIs that were removed in 8 will be available only if an HTTP request had a compatible header. At the moment is is expecting an Accept header to contain application/vnd.elasticsearch+json;compatible-with=7. This might be changed though when #52370 is resolved.
The REST changes contain

enriching RestRequest with a parameter Compatible-With. The value is taken from Accept header. See RestRequest creation. Used when deciding if a request to a compatible handler can be dispatched. See RestController.
//TODO this at the moment can be simplified to only use a value from a header. However in rest layer we often have only access to Params, so headers won't be accessible when implementing some compatible code

enriching XContentBuilder with a compatible version value. See AbstractRestChannel. Used when shape of a response is changed. see DocWriteResponse and GetResult

MediaType parsing change - see XContentType. because we expect a different media type now application/vnd.elasticsearch+json;compatible-with=7 (when previously application/json or similar)

testing changes - compat tests will have a compatible header set. see AbstractCompatRestTest.java

Compatible rest handlers - based on RestIndexAction and RestGetAction. See modules/rest-compatibility. These extend already existing handlers and register them in a RestCompatPlugin under a typed paths. They will only be accessible when a compatible header is present (use of compatibilityRequired method from RestHandler interface)

Without this PR 283 tests from 7.x are failing. - rest api spec tests only
with this PR 228 tests are passing - with almost all tests from index/* package passed (missing 2 tests that require change for include_type_param, to be added in a new PR)

relates #51816
@ezimuel
Copy link
Contributor

ezimuel commented Apr 21, 2020

We discussed in a meeting with @Mpdreamz and I'm also in favor of having a separate option as compatibleWith instead of a specific subtype. That said, I've some concern about the naming, since compatible sounds almost like act as. If I understood right, this Elasticsearch feature is to format the json output according to a specific version (e.g. 7).
For instance, you are using Elasticsearch 8 with and a client perform a request with compatibleWith=7. Since its is a new major, maybe, the ES engine will respond with a different result, even if the output will be formatted as 7. Here, I see a potential issue for BC break even if the format is ok but the data (the meaning) is different.

My proposal is to use a different naming, for instance format=7 that is more related to the real meaning, supporting specific output format. Moreover, I think we should document this feature carefully, explaining what really means "compatible".

@rjernst rjernst added the Team:Core/Infra Meta label for core/infra team label May 4, 2020
@rjernst rjernst added the needs:triage Requires assignment of a team area label label Dec 3, 2020
@jaymode
Copy link
Member

jaymode commented Dec 10, 2020

@pgomulka @jakelandis ping regarding this issue and the comment from @ezimuel. I know that most of the work recently has been using compatible with and IMO I think we should close this as code has landed to do this parsing. Do you see a reason to keep this issue open?

@jaymode jaymode added feedback_needed and removed needs:triage Requires assignment of a team area label labels Dec 10, 2020
@arteam arteam added v8.1.0 and removed v8.0.0 labels Jan 12, 2022
@mark-vieira mark-vieira added v8.2.0 and removed v8.1.0 labels Feb 2, 2022
@mark-vieira mark-vieira added v8.5.0 and removed v8.4.0 labels Jul 27, 2022
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@csoulios csoulios added v8.6.0 and removed v8.5.0 labels Sep 21, 2022
@kingherc kingherc removed the v8.6.0 label Nov 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/REST API REST infrastructure and utilities feedback_needed Team:Core/Infra Meta label for core/infra team v9.0.0
Projects
None yet
Development

No branches or pull requests