Skip to content

Commit

Permalink
Remove support for deflate compression
Browse files Browse the repository at this point in the history
Closes #215

Per https://zlib.net/zlib_faq.html#faq39

> What's the difference between the "gzip" and "deflate" HTTP 1.1 encodings?
>
> "gzip" is the gzip format, and "deflate" is the zlib format. They
> should probably have called the second one "zlib" instead to avoid
> confusion with the raw deflate compressed data format. While the HTTP
> 1.1 RFC 2616 correctly points to the zlib specification in RFC 1950 for
> the "deflate" transfer encoding, there have been reports of servers and
> browsers that incorrectly produce or expect raw deflate data per the
> deflate specification in RFC 1951, most notably Microsoft. So even
> though the "deflate" transfer encoding using the zlib format would be
> the more efficient approach (and in fact exactly what the zlib format
> was designed for), using the "gzip" transfer encoding is probably more
> reliable due to an unfortunate choice of name on the part of the HTTP
> 1.1 authors.
>
> Bottom line: use the gzip format for HTTP 1.1 encoding.
  • Loading branch information
wojtekmach committed Aug 11, 2023
1 parent 0c88cc4 commit e19d89f
Show file tree
Hide file tree
Showing 5 changed files with 9 additions and 15 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Req is a batteries-included HTTP client for Elixir.

* Extensibility via request, response, and error steps.

* Request body compression and automatic response body decompression (via [`compress_body`], [`compressed`], and [`decompress_body`] steps). Supports gzip, deflate, brotli and zstd decompression.
* Request body compression and automatic response body decompression (via [`compress_body`], [`compressed`], and [`decompress_body`] steps). Supports gzip, brotli, and zstd decompression.

* Request body encoding and automatic response body decoding (via [`encode_body`] and [`decode_body`] steps.)

Expand Down
4 changes: 2 additions & 2 deletions lib/req/request.ex
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ defmodule Req.Request do
Req.get!(req, url: "https://httpbin.org/json", print_headers: true).status
# Outputs:
# > accept-encoding: br, gzip, deflate
# > accept-encoding: br, gzip
# > user-agent: req/0.3.0-dev
# < date: Wed, 11 May 2022 11:10:47 GMT
# < content-type: application/json
Expand All @@ -228,7 +228,7 @@ defmodule Req.Request do
req = Req.new() |> PrintHeaders.attach(print_headers: true)
Req.get!(req, url: "https://httpbin.org/json").status
# Outputs:
# > accept-encoding: br, gzip, deflate
# > accept-encoding: br, gzip
# ...
200
Expand Down
8 changes: 1 addition & 7 deletions lib/req/steps.ex
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,6 @@ defmodule Req.Steps do
Supported formats:
* `gzip`
* `deflate`
* `br` (if [brotli] is installed)
* `zstd` (if [ezstd] is installed)
Expand Down Expand Up @@ -272,7 +271,7 @@ defmodule Req.Steps do
end

defp supported_accept_encoding do
value = "gzip, deflate"
value = "gzip"
value = if brotli_loaded?(), do: "br, " <> value, else: value
if ezstd_loaded?(), do: "zstd, " <> value, else: value
end
Expand Down Expand Up @@ -812,7 +811,6 @@ defmodule Req.Steps do
| Format | Decoder |
| ------------- | ----------------------------------------------- |
| gzip, x-gzip | `:zlib.gunzip/1` |
| deflate | `:zlib.unzip/1` |
| br | `:brotli.decode/1` (if [brotli] is installed) |
| zstd | `:ezstd.decompress/1` (if [ezstd] is installed) |
| identity | Returns data as is |
Expand Down Expand Up @@ -868,10 +866,6 @@ defmodule Req.Steps do
:zlib.gunzip(body)
end

defp decompress_with_algorithm("deflate", body) do
:zlib.unzip(body)
end

defp decompress_with_algorithm("br", body) do
if brotli_loaded?() do
{:ok, decompressed} = :brotli.decode(body)
Expand Down
2 changes: 1 addition & 1 deletion test/req/request_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ defmodule Req.RequestTest do

assert [
{"user-agent", "req/" <> _},
{"accept-encoding", "zstd, br, gzip, deflate"},
{"accept-encoding", "zstd, br, gzip"},
{"authorization", ^authorization}
] = request.headers
end
Expand Down
8 changes: 4 additions & 4 deletions test/req/steps_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,8 @@ defmodule Req.StepsTest do
test "multiple codecs", c do
Bypass.expect(c.bypass, "GET", "/", fn conn ->
conn
|> Plug.Conn.put_resp_header("content-encoding", "gzip, deflate")
|> Plug.Conn.send_resp(200, "foo" |> :zlib.gzip() |> :zlib.zip())
|> Plug.Conn.put_resp_header("content-encoding", "gzip, zstd")
|> Plug.Conn.send_resp(200, "foo" |> :zlib.gzip() |> :ezstd.compress())
end)

assert Req.get!(c.url).body == "foo"
Expand All @@ -257,12 +257,12 @@ defmodule Req.StepsTest do
{:ok, socket} = :gen_tcp.accept(listen_socket)
assert {:ok, "GET / HTTP/1.1\r\n" <> _} = :gen_tcp.recv(socket, 0)

body = "foo" |> :zlib.gzip() |> :zlib.zip()
body = "foo" |> :zlib.gzip() |> :ezstd.compress()

data = """
HTTP/1.1 200 OK
content-encoding: gzip
content-encoding: deflate
content-encoding: zstd
content-length: #{byte_size(body)}
#{body}
Expand Down

0 comments on commit e19d89f

Please sign in to comment.