Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[stable9.1] Enable GDrive retries and backoff for rate limit #27530

Merged
merged 1 commit into from
Apr 3, 2017

Conversation

PVince81
Copy link
Contributor

Makes it possible to still get successful calls after delayed retries
when hitting the API limits.

To test:

  1. Mount GDrive as "/gdrive"
  2. Setup local sync client
  3. Create a lot of files in the local "gdrive" folder: for I in $(seq 1 200); do echo "test" > file$I.txt; done
  4. Start sync client

Before the fix: lots of API rate limit errors.
After the fix: lots more successes and rare API limit errors

Please review @jvillafanez @butonic @davicente

Backports:

  • master
  • stable9 ? (if the correct lib version is used there)

Makes it possible to still get successful calls after delayed retries
when hitting the API limits.
@mention-bot
Copy link

@PVince81, thanks for your PR! By analyzing the history of the files in this pull request, we identified @MTGap, @AdamWill, @icewind1991 and @Xenopathic to be potential reviewers.

@PVince81 PVince81 changed the title Enable GDrive retries and backoff for rate limit [stable9.1] Enable GDrive retries and backoff for rate limit Mar 29, 2017
@davitol
Copy link
Contributor

davitol commented Apr 3, 2017

Applying the PR the logs (403) User Rate Limit Exceeded\" are not longer shown, but the sync speed is too slow, at least in my configuration:

screen shot 2017-04-03 at 09 58 26

Tested with oC 2.2.4 Desktop Client

@PVince81
Copy link
Contributor Author

PVince81 commented Apr 3, 2017

@davitol hmm, that's possibly because for each PHP request it will redo the backoff dance: fire a lot of requests, get the errors, then slow down, try again, slow down, etc. Instead of settling on a request speed and reuse it for every request.

That's unfortunately a limitation of the library.

@PVince81
Copy link
Contributor Author

PVince81 commented Apr 3, 2017

I guess slow sync is still an improvement over lots of errors ?

It sounds like we'll eventually need to write our own library... or reimplement exponential backoff in a way that works more efficiently 😦

@davitol
Copy link
Contributor

davitol commented Apr 3, 2017

IMO it is better this approach than the one with errors, but It should not be the definitive solution.

@PVince81 PVince81 merged commit d973c12 into stable9.1 Apr 3, 2017
@PVince81 PVince81 deleted the stable9.1-gdrive-retries-backoff branch April 3, 2017 13:39
@PVince81
Copy link
Contributor Author

PVince81 commented Apr 3, 2017

master: 776f031

@lock
Copy link

lock bot commented Aug 3, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants