Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Fix backoff stategy docs #18143

Merged
merged 2 commits into from
Oct 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -648,7 +648,7 @@
}
}
],
"description": "\n Error handler that sequentially iterates over a list of `ErrorHandler`s\n\n Sample config chaining 2 different retriers:\n error_handler:\n type: \"CompositeErrorHandler\"\n error_handlers:\n - response_filters:\n - predicate: \"{{ 'codase' in response }}\"\n action: RETRY\n backoff_strategies:\n - type: \"ConstantBackoffStrategy\"\n backoff_time_in_seconds: 5\n - response_filters:\n - http_codes: [ 403 ]\n action: RETRY\n backoff_strategies:\n - type: \"ConstantBackoffStrategy\"\n backoff_time_in_seconds: 10\n Attributes:\n error_handlers (List[ErrorHandler]): list of error handlers\n "
"description": "\n Error handler that sequentially iterates over a list of `ErrorHandler`s\n\n Sample config chaining 2 different retriers:\n error_handler:\n type: \"CompositeErrorHandler\"\n error_handlers:\n - response_filters:\n - predicate: \"{{ 'codase' in response }}\"\n action: RETRY\n backoff_strategies:\n - type: \"ConstantBackoff\"\n backoff_time_in_seconds: 5\n - response_filters:\n - http_codes: [ 403 ]\n action: RETRY\n backoff_strategies:\n - type: \"ConstantBackoff\"\n backoff_time_in_seconds: 10\n Attributes:\n error_handlers (List[ErrorHandler]): list of error handlers\n "
},
"DefaultErrorHandler": {
"allOf": [
Expand Down Expand Up @@ -677,24 +677,24 @@
"items": {
"anyOf": [
{
"$ref": "#/definitions/ConstantBackoffStrategy"
"$ref": "#/definitions/ConstantBackoff"
},
{
"$ref": "#/definitions/ExponentialBackoffStrategy"
"$ref": "#/definitions/ExponentialBackoff"
},
{
"$ref": "#/definitions/WaitTimeFromHeaderBackoffStrategy"
"$ref": "#/definitions/WaitTimeFromHeader"
},
{
"$ref": "#/definitions/WaitUntilTimeFromHeaderBackoffStrategy"
"$ref": "#/definitions/WaitUntilTimeFromHeader"
}
]
}
}
}
}
],
"description": "\n Default error handler.\n\n By default, the handler will only retry server errors (HTTP 5XX) and too many requests (HTTP 429) with exponential backoff.\n\n If the response is successful, then return SUCCESS\n Otherwise, iterate over the response_filters.\n If any of the filter match the response, then return the appropriate status.\n If the match is RETRY, then iterate sequentially over the backoff_strategies and return the first non-None backoff time.\n\n Sample configs:\n\n 1. retry 10 times\n `\n error_handler:\n max_retries: 10\n `\n 2. backoff for 5 seconds\n `\n error_handler:\n backoff_strategies:\n - type: \"ConstantBackoffStrategy\"\n backoff_time_in_seconds: 5\n `\n 3. retry on HTTP 404\n `\n error_handler:\n response_filters:\n - http_codes: [ 404 ]\n action: RETRY\n `\n 4. ignore HTTP 404\n `\n error_handler:\n - http_codes: [ 404 ]\n action: IGNORE\n `\n 5. retry if error message contains `retrythisrequest!` substring\n `\n error_handler:\n response_filters:\n - error_message_contain: \"retrythisrequest!\"\n action: IGNORE\n `\n 6. retry if 'code' is a field present in the response body\n `\n error_handler:\n response_filters:\n - predicate: \"{{ 'code' in response }}\"\n action: IGNORE\n `\n\n 7. ignore 429 and retry on 404\n `\n error_handler:\n - http_codes: [ 429 ]\n action: IGNORE\n - http_codes: [ 404 ]\n action: RETRY\n `\n\n Attributes:\n response_filters (Optional[List[HttpResponseFilter]]): response filters to iterate on\n max_retries (Optional[int]): maximum retry attempts\n backoff_strategies (Optional[List[BackoffStrategy]]): list of backoff strategies to use to determine how long\n to wait before retrying\n "
"description": "\n Default error handler.\n\n By default, the handler will only retry server errors (HTTP 5XX) and too many requests (HTTP 429) with exponential backoff.\n\n If the response is successful, then return SUCCESS\n Otherwise, iterate over the response_filters.\n If any of the filter match the response, then return the appropriate status.\n If the match is RETRY, then iterate sequentially over the backoff_strategies and return the first non-None backoff time.\n\n Sample configs:\n\n 1. retry 10 times\n `\n error_handler:\n max_retries: 10\n `\n 2. backoff for 5 seconds\n `\n error_handler:\n backoff_strategies:\n - type: \"ConstantBackoff\"\n backoff_time_in_seconds: 5\n `\n 3. retry on HTTP 404\n `\n error_handler:\n response_filters:\n - http_codes: [ 404 ]\n action: RETRY\n `\n 4. ignore HTTP 404\n `\n error_handler:\n - http_codes: [ 404 ]\n action: IGNORE\n `\n 5. retry if error message contains `retrythisrequest!` substring\n `\n error_handler:\n response_filters:\n - error_message_contain: \"retrythisrequest!\"\n action: IGNORE\n `\n 6. retry if 'code' is a field present in the response body\n `\n error_handler:\n response_filters:\n - predicate: \"{{ 'code' in response }}\"\n action: IGNORE\n `\n\n 7. ignore 429 and retry on 404\n `\n error_handler:\n - http_codes: [ 429 ]\n action: IGNORE\n - http_codes: [ 404 ]\n action: RETRY\n `\n\n Attributes:\n response_filters (Optional[List[HttpResponseFilter]]): response filters to iterate on\n max_retries (Optional[int]): maximum retry attempts\n backoff_strategies (Optional[List[BackoffStrategy]]): list of backoff strategies to use to determine how long\n to wait before retrying\n "
},
"HttpResponseFilter": {
"type": "object",
Expand Down Expand Up @@ -745,7 +745,7 @@
},
"description": "InterpolatedBoolean(condition: str, options: dataclasses.InitVar[typing.Mapping[str, typing.Any]])"
},
"ConstantBackoffStrategy": {
"ConstantBackoff": {
"allOf": [
{
"$ref": "#/definitions/BackoffStrategy"
Expand All @@ -767,7 +767,7 @@
"properties": {},
"description": "\n Backoff strategy defining how long to wait before retrying a request that resulted in an error.\n "
},
"ExponentialBackoffStrategy": {
"ExponentialBackoff": {
"allOf": [
{
"$ref": "#/definitions/BackoffStrategy"
Expand All @@ -784,7 +784,7 @@
],
"description": "\n Backoff strategy with an exponential backoff interval\n\n Attributes:\n factor (float): multiplicative factor\n "
},
"WaitTimeFromHeaderBackoffStrategy": {
"WaitTimeFromHeader": {
"allOf": [
{
"$ref": "#/definitions/BackoffStrategy"
Expand All @@ -804,7 +804,7 @@
],
"description": "\n Extract wait time from http header\n\n Attributes:\n header (str): header to read wait time from\n regex (Optional[str]): optional regex to apply on the header to extract its value\n "
},
"WaitUntilTimeFromHeaderBackoffStrategy": {
"WaitUntilTimeFromHeader": {
"allOf": [
{
"$ref": "#/definitions/BackoffStrategy"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ class CompositeErrorHandler(ErrorHandler, JsonSchemaMixin):
- predicate: "{{ 'codase' in response }}"
action: RETRY
backoff_strategies:
- type: "ConstantBackoffStrategy"
- type: "ConstantBackoff"
backoff_time_in_seconds: 5
- response_filters:
- http_codes: [ 403 ]
action: RETRY
backoff_strategies:
- type: "ConstantBackoffStrategy"
- type: "ConstantBackoff"
backoff_time_in_seconds: 10
Attributes:
error_handlers (List[ErrorHandler]): list of error handlers
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class DefaultErrorHandler(ErrorHandler, JsonSchemaMixin):
`
error_handler:
backoff_strategies:
- type: "ConstantBackoffStrategy"
- type: "ConstantBackoff"
backoff_time_in_seconds: 5
`
3. retry on HTTP 404
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ Schema:

### Constant Backoff

When using the `ConstantBackoffStrategy`, the requester will backoff with a constant interval.
When using the `ConstantBackoff` strategy, the requester will backoff with a constant interval.

Schema:

Expand All @@ -202,7 +202,7 @@ Schema:

### Wait time defined in header

When using the `WaitTimeFromHeaderBackoffStrategy`, the requester will backoff by an interval specified in the response header.
When using the `WaitTimeFromHeader`, the requester will backoff by an interval specified in the response header.
In this example, the requester will backoff by the response's "wait_time" header value:

Schema:
Expand Down Expand Up @@ -230,7 +230,7 @@ requester:
error_handler:
<...>
backoff_strategies:
- type: "WaitTimeFromHeaderBackoffStrategy"
- type: "WaitTimeFromHeader"
header: "wait_time"
```

Expand All @@ -244,14 +244,14 @@ requester:
error_handler:
<...>
backoff_strategies:
- type: "WaitTimeFromHeaderBackoffStrategy"
- type: "WaitTimeFromHeader"
header: "wait_time"
regex: "[-+]?\d+"
```

### Wait until time defined in header

When using the `WaitUntilTimeFromHeaderBackoffStrategy`, the requester will backoff until the time specified in the response header.
When using the `WaitUntilTimeFromHeader` backoff strategy, the requester will backoff until the time specified in the response header.
In this example, the requester will wait until the time specified in the "wait_until" header value:

Schema:
Expand Down Expand Up @@ -281,7 +281,7 @@ requester:
error_handler:
<...>
backoff_strategies:
- type: "WaitUntilTimeFromHeaderBackoffStrategy"
- type: "WaitUntilTimeFromHeader"
header: "wait_until"
regex: "[-+]?\d+"
min_wait: 5
Expand All @@ -302,9 +302,9 @@ requester:
error_handler:
<...>
backoff_strategies:
- type: "WaitTimeFromHeaderBackoffStrategy"
- type: "WaitTimeFromHeader"
header: "wait_time"
- type: "ConstantBackoffStrategy"
- type: "ConstantBackoff"
backoff_time_in_seconds: 5
```

Expand Down Expand Up @@ -340,13 +340,13 @@ requester:
- predicate: "{{ 'code' in response }}"
action: RETRY
backoff_strategies:
- type: "ConstantBackoffStrategy"
- type: "ConstantBackoff"
backoff_time_in_seconds: 5
- response_filters:
- http_codes: [ 403 ]
action: RETRY
backoff_strategies:
- type: "ExponentialBackoffStrategy"
- type: "ExponentialBackoff"
```

## More readings
Expand Down