You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
It seems ghz includes unmarshaling of the gRPC response when reporting response time. As a result, the results are misleading, especially if the gRPC response is large (> 5MB). When analyzing the response of a large gRPC response using TCPDUMP, the time from first byte sent to the status OK is in milliseconds, while ghz reports a response in seconds.
To Reproduce
Run ghz with a method that returns a large gRPC response (> 5MB).
Expected behavior
ghz should not include unmarshaling of the gRPC response when calculating the response time
Environment
OS: macOS 14.0
ghz: 0.117.0
The text was updated successfully, but these errors were encountered:
This seems similar as #125.
This is by design. Measuring deserialization is more meaningful as it would be indicative of the practical lag the service clients would experience in the real world, until they can actually do something useful with the response.
Generally it's considered a good practice to keep protobuf messages less than 1MB in size.
This could be an improvement and a feature for future.
A workaround would be to design your test case scenario using smaller message size.
I, concur, it is good practice to keep protobuf message small but for reason I can not disclose, this is not an option.
While, I concur, measuring deserialization is meaningful, currently the way ghz is implement, the calculation is how fast ghz deserialize. In my case, I have various gRPC clients written in Golang, Python, Ruby, and Java, each having different deserialization performance.
Maybe ghz future feature is show a breakdown consisting of actual response time + deserialization or an option to exclude deserialization in the response calculation.
Describe the bug
It seems ghz includes unmarshaling of the gRPC response when reporting response time. As a result, the results are misleading, especially if the gRPC response is large (> 5MB). When analyzing the response of a large gRPC response using TCPDUMP, the time from first byte sent to the status OK is in milliseconds, while ghz reports a response in seconds.
To Reproduce
Run ghz with a method that returns a large gRPC response (> 5MB).
Expected behavior
ghz should not include unmarshaling of the gRPC response when calculating the response time
Environment
The text was updated successfully, but these errors were encountered: