Support compressed HTTP request bodies in REST API

[guilload]

Implementation pointers:

• the HTTP server for the REST API is set up here.
• add this Tower layer should do the trick

[etolbakov]

Hey Adrien,
I'm interested in looking into it.

Regards, Eugene

[unpervertedkid]

@guilload can i take this up?

[fmassot]

@unpervertedkid yes, you can!

[etolbakov]

@unpervertedkid please have a look at the #4003 comment section. Hopefully, you'll find it useful! 👍

[unpervertedkid]

@etolbakov Cool lemme pick it up and lemme also take a look at #4003 thanks.

[guilload]

Hello @unpervertedkid,

I saw you opened a PR to add support for decompression directly in warp. Super cool and thank you! I don't know how long it'll take for the review/merge/release process to complete. In the meantime, we could use your warp::Filter in Quickwit directly. The only missing piece is to dynamically decompress the payload depending on the value of the Content-Encoding header (or its absence).

[unpervertedkid]

Sure @guilload . Although I might take a week or two pick it up so in the mean time if anyone is willing to pick it up feel free to go ahead.
Refer to this pr.

[DCjanus]

A malicious user can compose a request using gzip that has a compression ratio > 1000x, meaning a body of 1MB could expand >1GB. When the server processes these requests, it's very likely to saturate the memory capacity and cause an outage.
-- https://medium.com/@abhinav.ittekot/why-http-request-compression-is-almost-never-supported-5de68067b245

Maybe we should be carefully handle this situation.

[fmassot]

Friendly ping @unpervertedkid

[unpervertedkid]

@fmassot I think you should assign this to someone else.
My bandwidth is currently pretty limited😥.