Repeated requests in a loop with a timeout. GRPC DEADLINE_EXCEEDED -> Next request HTTP/2 error: (errorCode: 1)

[RubenGarcia]

- I am using dart both for server and client. - I am awaiting repeated GRPC requests with a timeout of 200ms in a for loop - I am creating a new request after a response, a timeout or an error is provided by GRPC

After receiving a timeout (GRPC code 4, codeName DEADLINE_EXCEEDED)
The next request almost always errors with

• code 2
• codeName UNKNOWN
• message: HTTP/2 error: Connection error: Connection is being forcefully terminated. (errorCode: 1)
  HTTP/2 errorCode 1 is PROTOCOL_ERROR according to https://www.iana.org/assignments/http2-parameters/http2-parameters.xhtml#error-code

<version of the grpc-dart packages used; see your pubspec.lock file>
grpc version: both "3.2.3" and "3.2.4" tested, both present the issue.

Tested with two Android phones connected to the same wifi.

Repro steps

Create a server, run it and run this code in the client:

Request request = Request();
for (int i = 0; i< size; i++) {
    try {
       CallOptions callOptions=CallOptions(timeout: Duration (milliseconds: 50));
       MyResponse response = await myGRPC!.getResponse(request, options: callOptions);
       // log response
    } on GrpcError catch (e) {
          // log e
    }
}

Expected result: Some good responses, some timeouts, but no consistent HTTP/2 protocol errors after a timeout

Actual result: Every timeout

GRPC code 4, codeName DEADLINE_EXCEEDED

is followed by an HTTP/2 error

HTTP/2 error: Connection error: Connection is being forcefully terminated. (errorCode: 1)

Details

I suspect the dart GRPC libraries need to verify that on the event of a timeout, the HTTP/2 protocol is still being followed so no PROTOCOL_ERROR is produced on subsequent calls.

Otherwise, if this is desired behaviour, it should be documented at CallOptions class in package:grpc/src/client/call.dart

Any input is appreciated regarding this.

[RubenGarcia]

I created a minimal example of the issue with working server and client code at
https://github.com/RubenGarcia/dartGrpcTimeoutIssue
Please check.

[mosuem]

I could not reproduce this with https://github.com/grpc/grpc-dart/blob/reproDeadlineError/test/timeout_repro_test.dart - needs more investigation.

[RubenGarcia]

Can you confirm your results using my repo? You may need to lower your timeout depending on your wifi.
Example output after pressing the button a few times.

Started 192.168.178.37, timeout: 50
GRPC error: code 4, codename DEADLINE_EXCEEDED, message Deadline exceeded
GRPC error: code 4, codename DEADLINE_EXCEEDED, message Deadline exceeded
GRPC error: code 4, codename DEADLINE_EXCEEDED, message Deadline exceeded
GRPC error: code 2, codename UNKNOWN, message HTTP/2 error: Connection error: Connection is being forcefully terminated. (errorCode: 1)
GRPC error: code 4, codename DEADLINE_EXCEEDED, message Deadline exceeded

[mosuem]

Can you confirm your results using my repo?

I don't have any phones to test this with. I could try to set it up for Linux instead.

[RubenGarcia]

Ok, let's check under which platforms this is an issue.
I used a Huawei with EMUI-Version 12.0.0 (client) and a Pixel 3 with Android 12 (server) -> Shows the issue.