Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ghz should not include unmarshaling when calculating response time #426

Open
blwt opened this issue Oct 31, 2023 · 2 comments
Open

ghz should not include unmarshaling when calculating response time #426

blwt opened this issue Oct 31, 2023 · 2 comments

Comments

@blwt
Copy link

blwt commented Oct 31, 2023

Describe the bug
It seems ghz includes unmarshaling of the gRPC response when reporting response time. As a result, the results are misleading, especially if the gRPC response is large (> 5MB). When analyzing the response of a large gRPC response using TCPDUMP, the time from first byte sent to the status OK is in milliseconds, while ghz reports a response in seconds.

To Reproduce
Run ghz with a method that returns a large gRPC response (> 5MB).

Expected behavior
ghz should not include unmarshaling of the gRPC response when calculating the response time

Environment

  • OS: macOS 14.0
  • ghz: 0.117.0
@bojand
Copy link
Owner

bojand commented Nov 1, 2023

This seems similar as #125.
This is by design. Measuring deserialization is more meaningful as it would be indicative of the practical lag the service clients would experience in the real world, until they can actually do something useful with the response.
Generally it's considered a good practice to keep protobuf messages less than 1MB in size.
This could be an improvement and a feature for future.
A workaround would be to design your test case scenario using smaller message size.

@blwt
Copy link
Author

blwt commented Nov 1, 2023

I, concur, it is good practice to keep protobuf message small but for reason I can not disclose, this is not an option.

While, I concur, measuring deserialization is meaningful, currently the way ghz is implement, the calculation is how fast ghz deserialize. In my case, I have various gRPC clients written in Golang, Python, Ruby, and Java, each having different deserialization performance.

Maybe ghz future feature is show a breakdown consisting of actual response time + deserialization or an option to exclude deserialization in the response calculation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants