Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

client: reset backoff to 0 after a connection is established #2669

Merged
merged 1 commit into from
Mar 8, 2019

Conversation

menghanl
Copy link
Contributor

@menghanl menghanl commented Mar 6, 2019

@@ -1032,6 +1032,7 @@ func (ac *addrConn) resetTransport() {
continue
}

backoffFor = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works fine for us to reconnect immediately after connection closed.
But I think this also resets backoff duration when reconnecting failed repeated in the server down, which resulted in trying to reconnect aggressively.

So the backoff duration should be reset only in first retry like this?

if ac.backoffIdx == 0 {
	backoffFor = 0
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If reconnecting failed, we will continue above, and won't reach this.
This only happens when we successfully create a connection, and we want to skip any backoff if this connection breaks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry it's my bad. I tried again with this branch and it does retry to connect with backoff correctly.

@dfawley
Copy link
Member

dfawley commented Mar 7, 2019

@kazegusuri

If the connection fails to be created, this code will be skipped.

Did you apply this patch to master@HEAD or another branch/commit?

Are you setting GRPC_GO_REQUIRE_HANDSHAKE=off in your environment? If so, please try not setting it.

@@ -1032,6 +1032,7 @@ func (ac *addrConn) resetTransport() {
continue
}

backoffFor = 0
Copy link
Member

@dfawley dfawley Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this should also reset ac.backoffIdx = 0.

EDIT: Oh we already do this below; maybe we should move it up here though for consistency and to avoid taking the lock an extra time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have cleanup changes in another branch, was trying to keep this change simple so we can cherrypick it to previous releases.

@menghanl menghanl merged commit 5abb357 into grpc:master Mar 8, 2019
@menghanl menghanl deleted the reset_backoff branch March 8, 2019 18:16
@dfawley dfawley added this to the 1.20 Release milestone Mar 14, 2019
srenatus added a commit to chef/automate that referenced this pull request May 21, 2019
Looks like we've missed a few releases -- here they are. The interesting
releases are:

- v1.20.0, as it has a few new features and bug fixes
- v1.19.1 has a bug fix for grpc/grpc-go#2669,
  which could have happened to us as well.

For all the notes, see the releases page:
https://github.com/grpc/grpc-go/releases

Signed-off-by: Stephan Renatus <[email protected]>
srenatus added a commit to chef/automate that referenced this pull request May 22, 2019
Looks like we've missed a few releases -- here they are. The interesting
releases are:

- v1.20.0, as it has a few new features and bug fixes
- v1.19.1 has a bug fix for grpc/grpc-go#2669,
  which could have happened to us as well.

For all the notes, see the releases page:
https://github.com/grpc/grpc-go/releases

Signed-off-by: Stephan Renatus <[email protected]>
srenatus added a commit to chef/automate that referenced this pull request May 22, 2019
* deps: bump go-grpc (1.19.0 -> 1.20.1)

Looks like we've missed a few releases -- here they are. The interesting
releases are:

- v1.20.0, as it has a few new features and bug fixes
- v1.19.1 has a bug fix for grpc/grpc-go#2669,
  which could have happened to us as well.

For all the notes, see the releases page:
https://github.com/grpc/grpc-go/releases

Signed-off-by: Stephan Renatus <[email protected]>
@lock lock bot locked as resolved and limited conversation to collaborators Sep 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants