Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add gzip support for /metrics #477

Merged
merged 1 commit into from
Aug 18, 2022
Merged

Conversation

fatelei
Copy link
Contributor

@fatelei fatelei commented Jun 17, 2022

What is pr do ?

Let pushgateway server handle gzip content-encoding content.

Tests

Accept gzip

  • server

image

  • client

image

Forbid gzip

  • server

image

  • client

image

@fatelei fatelei force-pushed the issue-84 branch 2 times, most recently from e75ffb5 to 37609ec Compare June 22, 2022 07:45
Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this. This seems to work in general, but most importantly, we have to add it in a way that the default behavior is the same as the current one (no compression, even if requested). See comments (which also include some nit-picking).

main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, you used github.com/NYTimes/gziphandler , but now you dropped it. Any reason for that? It seems reasonable to me to reuse that battle-tested middleware instead of implementing our own here.

In different news, you need to switch off gzip compression in the promhttp.HandlerOpts (line 137). Otherwise you might get double gzip compression.

@fatelei
Copy link
Contributor Author

fatelei commented Aug 10, 2022

Initially, you used github.com/NYTimes/gziphandler , but now you dropped it. Any reason for that? It seems reasonable to me to reuse that battle-tested middleware instead of implementing our own here.

In different news, you need to switch off gzip compression in the promhttp.HandlerOpts (line 137). Otherwise you might get double gzip compression.

#84 need a feature, send Content-Encoding: gzip content to push gateway, so pushgateway need handle gzip format content in request. While github.com/NYTimes/gziphandler just make a gzip format response, it can not handle gzip format request.

Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies for only realizing now that #84 only talks about gzip'ing the body of a request, which is not handled at all by the usual gzip handlers (which are only concerned about response compression).

I think I understand now. Good news is that it makes things easier in terms of behavior change. We don't even need a flag because the request encoding isn't really connected to content negotiations. So far, a gzip'd request was just failing, so we can simply always allow it now. The response encoding will be untouched by all of that.

For remaining things to do, see comments below that, hopefully, make more sense now. My apologies again.

Also, could you add a short section to the README.md about the new feature? Perhaps at the end of the API section? Along the lines: "The body of a POST or PUT request may be gzip-compressed. Add a header Content-Encoding: gzip to do so." And then perhaps a curl example like echo "some_metric 3.14" | gzip | curl -H 'Content-Encoding: gzip' --data-binary @- http://pushgateway.example.org:9091/metrics/job/some_job

main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
@fatelei
Copy link
Contributor Author

fatelei commented Aug 11, 2022

My apologies for only realizing now that #84 only talks about gzip'ing the body of a request, which is not handled at all by the usual gzip handlers (which are only concerned about response compression).

I think I understand now. Good news is that it makes things easier in terms of behavior change. We don't even need a flag because the request encoding isn't really connected to content negotiations. So far, a gzip'd request was just failing, so we can simply always allow it now. The response encoding will be untouched by all of that.

For remaining things to do, see comments below that, hopefully, make more sense now. My apologies again.

Also, could you add a short section to the README.md about the new feature? Perhaps at the end of the API section? Along the lines: "The body of a POST or PUT request may be gzip-compressed. Add a header Content-Encoding: gzip to do so." And then perhaps a curl example like echo "some_metric 3.14" | gzip | curl -H 'Content-Encoding: gzip' --data-binary @- http://pushgateway.example.org:9091/metrics/job/some_job

OK

Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your contribution and your patience. I only have a few formatting nits left for the README.md file.

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
Let pushgateway can handle gzip format request.

Signed-off-by: fatelei <[email protected]>
Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much again. Merging now.

@beorn7 beorn7 merged commit a148ab0 into prometheus:master Aug 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants