-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add rate limit handling for GitHub client #4926
base: main
Are you sure you want to change the base?
feat: Add rate limit handling for GitHub client #4926
Conversation
@@ -121,23 +122,28 @@ func NewGithubClient(hostname string, credentials GithubCredentials, config Gith | |||
return nil, errors.Wrap(err, "error initializing github authentication transport") | |||
} | |||
|
|||
transportWithRateLimit, err := github_ratelimit.NewRateLimitWaiterClient(transport.Transport, github_ratelimit.WithTotalSleepLimit(time.Minute, nil)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
regarding WithTotalSleepLimit(time.Minute, nil)
- not sure how long we want to keep retry, and if this should be configurable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are trying to limit as much as possible adding more flags, so maybe we can add this to the repo.Yaml instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is something like this acceptable? open to suggestions
...
git:
github:
secondary_rate_limit_duration: 60
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, repo config would be a good idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I suggest starting with the hard-coded 1 minute value that you currently have, so that we can get this PR finished. It can always be refined later to make it configurable.
Signed-off-by: Simon Heather <[email protected]>
@@ -121,23 +122,28 @@ func NewGithubClient(hostname string, credentials GithubCredentials, config Gith | |||
return nil, errors.Wrap(err, "error initializing github authentication transport") | |||
} | |||
|
|||
transportWithRateLimit, err := github_ratelimit.NewRateLimitWaiterClient(transport.Transport, github_ratelimit.WithTotalSleepLimit(time.Minute, nil)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about adding a callback to these that log an info message so that we can see when they are being triggered?
@@ -121,23 +122,28 @@ func NewGithubClient(hostname string, credentials GithubCredentials, config Gith | |||
return nil, errors.Wrap(err, "error initializing github authentication transport") | |||
} | |||
|
|||
transportWithRateLimit, err := github_ratelimit.NewRateLimitWaiterClient(transport.Transport, github_ratelimit.WithTotalSleepLimit(time.Minute, nil)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I suggest starting with the hard-coded 1 minute value that you currently have, so that we can get this PR finished. It can always be refined later to make it configurable.
Signed-off-by: Simon Heather <[email protected]>
What
Why
Implementation
go-github-ratelimit
library, as recommended in the official go-github README:Testing
References