-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: media operation retries can be configured using the same interface as with non-media operation #447
feat: media operation retries can be configured using the same interface as with non-media operation #447
Changes from all commits
9a0eede
f43f630
0d3d260
265d3bc
a820538
20afb8e
4b37672
c051371
39f03b1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -23,6 +23,7 @@ | |
import os | ||
|
||
from six.moves.urllib.parse import urlsplit | ||
from google import resumable_media | ||
from google.cloud.storage.constants import _DEFAULT_TIMEOUT | ||
from google.cloud.storage.retry import DEFAULT_RETRY | ||
from google.cloud.storage.retry import DEFAULT_RETRY_IF_METAGENERATION_SPECIFIED | ||
|
@@ -45,6 +46,12 @@ | |
("if_source_metageneration_not_match", "ifSourceMetagenerationNotMatch"), | ||
) | ||
|
||
_NUM_RETRIES_MESSAGE = ( | ||
"`num_retries` has been deprecated and will be removed in a future " | ||
"release. Use the `retry` argument with a Retry or ConditionalRetryPolicy " | ||
"object, or None, instead." | ||
) | ||
|
||
|
||
def _get_storage_host(): | ||
return os.environ.get(STORAGE_EMULATOR_ENV_VAR, _DEFAULT_STORAGE_HOST) | ||
|
@@ -524,3 +531,37 @@ def _bucket_bound_hostname_url(host, scheme=None): | |
return host | ||
|
||
return "{scheme}://{host}/".format(scheme=scheme, host=host) | ||
|
||
|
||
def _api_core_retry_to_resumable_media_retry(retry, num_retries=None): | ||
"""Convert google.api.core.Retry to google.resumable_media.RetryStrategy. | ||
|
||
Custom predicates are not translated. | ||
|
||
:type retry: google.api_core.Retry | ||
:param retry: (Optional) The google.api_core.Retry object to translate. | ||
|
||
:type num_retries: int | ||
:param num_retries: (Optional) The number of retries desired. This is | ||
supported for backwards compatibility and is mutually exclusive with | ||
`retry`. | ||
|
||
:rtype: google.resumable_media.RetryStrategy | ||
:returns: A RetryStrategy with all applicable attributes copied from input, | ||
or a RetryStrategy with max_retries set to 0 if None was input. | ||
""" | ||
|
||
if retry is not None and num_retries is not None: | ||
raise ValueError("num_retries and retry arguments are mutually exclusive") | ||
|
||
elif retry is not None: | ||
return resumable_media.RetryStrategy( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It looks like error types are not translatable in this case? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You mean custom predicates? Yes, those are not translatable. It'll be documented wherever possible. |
||
max_sleep=retry._maximum, | ||
max_cumulative_retry=retry._deadline, | ||
initial_delay=retry._initial, | ||
multiplier=retry._multiplier, | ||
) | ||
elif num_retries is not None: | ||
return resumable_media.RetryStrategy(max_retries=num_retries) | ||
else: | ||
return resumable_media.RetryStrategy(max_retries=0) |
Large diffs are not rendered by default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this clear to the end user?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll have this warning in every single public method docstring, so with that it should be clear.