Skip to content

Commit

Permalink
Prevent user removing required headers by mistake
Browse files Browse the repository at this point in the history
  • Loading branch information
@Colin-b
Colin-b committed Feb 11, 2024
1 parent b953364 commit 13c44a7
Show file tree
Hide file tree
Showing 5 changed files with 78 additions and 143 deletions.
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- As the AWS documentation might be wrong or not exhaustive enough, feel free to open issues, should you encounter edge cases.

### Changed
- `httpx_auth.AWS4Auth.default_include_headers` is not available anymore, use `httpx_auth.AWS4Auth` `include_headers` parameter instead to change the list of included headers if the default does not fit your need (refer to documentation for an exhaustive list).
- `httpx_auth.AWS4Auth.default_include_headers` is not available anymore, use `httpx_auth.AWS4Auth` `include_headers` parameter instead to include additional headers if the default does not fit your need (refer to documentation for an exhaustive list).
- `httpx_auth.AWS4Auth` `include_headers` values will not be stripped anymore, meaning that you can now include headers prefixed and/or suffixed with blank spaces.
- `httpx_auth.AWS4Auth` query fragment (`#` and everything following) is not considered as part of the canonical query string anymore. Feel free to open an issue if this is one.
- `httpx_auth.AWS4Auth` does not includes `date` header by default anymore. You will have to provide it via `include_headers` yourself if you need to.
- Note that it should not be required as `httpx_auth.AWS4Auth` is sending `x-amz-date` by default and AWS documentation states that the request date can be specified by using either the HTTP `Date` or the `x-amz-date` header. If both headers are present, `x-amz-date` takes precedence.
- `httpx_auth.AWS4Auth` `include_headers` does not needs to include `host`, `content-type` or `x-amz-*` anymore as those headers will always be included. It is now expected to be provided as a list of additional headers.

## [0.19.0] - 2024-01-09
### Added
Expand Down
16 changes: 8 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -697,14 +697,14 @@ Note that the following changes were made compared to `requests-aws4auth`:

### Parameters

| Name | Description | Mandatory | Default value |
|:-------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:---------------------------------------------------------------------------------------------------------------------|
| `access_id` | AWS access ID. | Mandatory | |
| `secret_key` | AWS secret access key. | Mandatory | |
| `region` | The region you are connecting to, as per [this list](http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region). For services which do not require a region (e.g. IAM), use us-east-1. | Mandatory | |
| `service` | The name of the service you are connecting to, as per [this list](http://docs.aws.amazon.com/general/latest/gr/rande.html). e.g. elasticbeanstalk. | Mandatory | |
| `security_token` | Used for the `x-amz-security-token` header, for use with STS temporary credentials. | Optional | |
| `include_headers` | Set of headers to include in the canonical and signed headers. Specific values are `x-amz-*` that matches any header starting with `x-amz-` (except for `x-amz-client-context`) and `*` that include every provided header. | Optional | {"host", "content-type", "x-amz-*"} if `security_token` is provided, `x-amz-security-token` will always be included. |
| Name | Description | Mandatory | Default value |
|:-------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:-------------------------------------------------------------------------------------------------------------------------|
| `access_id` | AWS access ID. | Mandatory | |
| `secret_key` | AWS secret access key. | Mandatory | |
| `region` | The region you are connecting to, as per [this list](http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region). For services which do not require a region (e.g. IAM), use us-east-1. | Mandatory | |
| `service` | The name of the service you are connecting to, as per [this list](http://docs.aws.amazon.com/general/latest/gr/rande.html). e.g. elasticbeanstalk. | Mandatory | |
| `security_token` | Used for the `x-amz-security-token` header, for use with STS temporary credentials. | Optional | |
| `include_headers` | Set of headers to include in the canonical and signed headers (in addition to the default). Note that `x-amz-client-context` is not included by default and `*` will include all headers. | Optional | {"host", "content-type", "x-amz-*"} and if `security_token` is provided, `x-amz-security-token`. |

## API key in header

Expand Down
138 changes: 68 additions & 70 deletions httpx_auth/aws.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,11 @@ def __init__(
http://docs.aws.amazon.com/general/latest/gr/rande.html
e.g. elasticbeanstalk.
:param security_token: Used for the x-amz-security-token header, for use with STS temporary credentials.
:param include_headers: Set of headers to include in the canonical and signed headers.
{"host", "content-type", "x-amz-*"} by default.
Note that if security_token is provided, x-amz-security-token will always be included.
Specific values:
- "x-amz-*" matches any header starting with 'x-amz-' except for x-amz-client context.
- "*" will include every provided header.
:param include_headers: Set of headers to include in the canonical and signed headers, in addition to:
* host
* content-type
* Every header prefixed with x-amz- (except for x-amz-client-context)
Providing {"*"} as value will include all headers.
"""
self.secret_key = secret_key
if not self.secret_key:
Expand All @@ -57,17 +56,11 @@ def __init__(
# https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
# For the purpose of calculating an authorization signature, only the host and any x-amz-* headers are required;
# however, in order to prevent data tampering, you should consider including all the headers in the signature calculation.
include_headers = {"host", "content-type", "x-amz-*"}

self.include_headers = {
header.lower() for header in kwargs.get("include_headers", include_headers)
header.lower() for header in kwargs.get("include_headers", [])
}

# https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
# if you are using temporary security credentials, you need to include x-amz-security-token in your request.
# You must add this header in the list of CanonicalHeaders
if self.security_token:
self.include_headers.add("x-amz-security-token")
self.include_headers.add("host")
self.include_headers.add("content-type")

def auth_flow(
self, request: httpx.Request
Expand All @@ -89,10 +82,13 @@ def auth_flow(

# https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
# if you are using temporary security credentials, you need to include x-amz-security-token in your request.
# You must add this header in the list of CanonicalHeaders
if self.security_token:
request.headers["x-amz-security-token"] = self.security_token

canonical_headers, signed_headers = self._canonical_headers(request.headers)
canonical_headers, signed_headers = canonical_and_signed_headers(
request.headers, self.include_headers
)
canonical_request = self._canonical_request(
request, canonical_headers, signed_headers
)
Expand Down Expand Up @@ -127,71 +123,73 @@ def _canonical_request(
]
)

def _canonical_headers(self, headers: httpx.Headers) -> Tuple[str, str]:
"""
See https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html for more details.

CanonicalHeaders is a list of request headers with their values.
Individual header name and value pairs are separated by the newline character ("\n").
Header names must be in lowercase.
You must sort the header names alphabetically to construct the string, as shown in the following example:
def canonical_and_signed_headers(
headers: httpx.Headers, include_headers: set[str]
) -> tuple[str, str]:
"""
See https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html for more details.
CanonicalHeaders is a list of request headers with their values.
Individual header name and value pairs are separated by the newline character ("\n").
Header names must be in lowercase.
You must sort the header names alphabetically to construct the string, as shown in the following example:
Lowercase(<HeaderName1>)+":"+Trim(<value>)+"\n"
Lowercase(<HeaderName2>)+":"+Trim(<value>)+"\n"
...
Lowercase(<HeaderNameN>)+":"+Trim(<value>)+"\n"
Lowercase(<HeaderName1>)+":"+Trim(<value>)+"\n"
Lowercase(<HeaderName2>)+":"+Trim(<value>)+"\n"
...
Lowercase(<HeaderNameN>)+":"+Trim(<value>)+"\n"
The Lowercase() and Trim() functions used in this example are described in the preceding section.
The Lowercase() and Trim() functions used in this example are described in the preceding section.
The CanonicalHeaders list must include the following:
- HTTP host header.
- If the Content-Type header is present in the request, you must add it to the CanonicalHeaders list.
- Any x-amz-* headers that you plan to include in your request must also be added.
The CanonicalHeaders list must include the following:
- HTTP host header.
- If the Content-Type header is present in the request, you must add it to the CanonicalHeaders list.
- Any x-amz-* headers that you plan to include in your request must also be added.
For example, if you are using temporary security credentials, you need to include x-amz-security-token in your request.
You must add this header in the list of CanonicalHeaders.
For example, if you are using temporary security credentials, you need to include x-amz-security-token in your request.
You must add this header in the list of CanonicalHeaders.
Note
The x-amz-content-sha256 header is required for all AWS Signature Version 4 requests.
It provides a hash of the request payload.
If there is no payload, you must provide the hash of an empty string.
Note
The x-amz-content-sha256 header is required for all AWS Signature Version 4 requests.
It provides a hash of the request payload.
If there is no payload, you must provide the hash of an empty string.
The following is an example CanonicalHeaders string.
The header names are in lowercase and sorted.
The following is an example CanonicalHeaders string.
The header names are in lowercase and sorted.
host:s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20130708T220855Z
host:s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20130708T220855Z
Note
For the purpose of calculating an authorization signature, only the host and any x-amz-* headers are required;
however, in order to prevent data tampering, you should consider including all the headers in the signature calculation.
Note
For the purpose of calculating an authorization signature, only the host and any x-amz-* headers are required;
however, in order to prevent data tampering, you should consider including all the headers in the signature calculation.
SignedHeaders is an alphabetically sorted, semicolon-separated list of lowercase request header names.
The request headers in the list are the same headers that you included in the CanonicalHeaders string.
For example, for the previous example, the value of SignedHeaders would be as follows:
SignedHeaders is an alphabetically sorted, semicolon-separated list of lowercase request header names.
The request headers in the list are the same headers that you included in the CanonicalHeaders string.
For example, for the previous example, the value of SignedHeaders would be as follows:
host;x-amz-content-sha256;x-amz-date
"""
included_headers = {}
for header, header_value in headers.items():
if (header or "*") in self.include_headers or (
"x-amz-*" in self.include_headers
and header.startswith("x-amz-")
# x-amz-client-context break mobile analytics auth if included
and not header == "x-amz-client-context"
):
included_headers[header] = _amz_norm_whitespace(header_value)

canonical_headers = ""
signed_headers = []
for header in sorted(included_headers):
signed_headers.append(header)
canonical_headers += f"{header}:{included_headers[header]}\n"

signed_headers = ";".join(signed_headers)

return canonical_headers, signed_headers
host;x-amz-content-sha256;x-amz-date
"""
included_headers = {}
for header, header_value in headers.items():
if (header or "*") in include_headers or (
header.startswith("x-amz-")
# x-amz-client-context break mobile analytics auth if included
and not header == "x-amz-client-context"
):
included_headers[header] = _amz_norm_whitespace(header_value)

canonical_headers = ""
signed_headers = []
for header in sorted(included_headers):
signed_headers.append(header)
canonical_headers += f"{header}:{included_headers[header]}\n"

signed_headers = ";".join(signed_headers)

return canonical_headers, signed_headers


def _string_to_sign(request: httpx.Request, canonical_request: str, scope: str) -> str:
Expand Down
32 changes: 0 additions & 32 deletions tests/aws_signature_v4/test_aws4auth_async.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,7 @@ async def test_aws_auth_share_security_tokens_between_instances(
)
assert auth2.include_headers == {
"content-type",
"x-amz-security-token",
"host",
"x-amz-*",
}

httpx_mock.add_response(
Expand Down Expand Up @@ -199,12 +197,8 @@ async def test_aws_auth_allows_to_include_custom_and_default_forbidden_header(
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
"cusTom",
"x-aMz-client-context",
"x-amz-*",
],
)

Expand Down Expand Up @@ -239,11 +233,7 @@ async def test_aws_auth_does_not_strips_header_names(
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
" cusTom ",
"x-amz-*",
],
)

Expand Down Expand Up @@ -277,11 +267,7 @@ async def test_aws_auth_header_with_multiple_values(
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
"cusTom",
"x-amz-*",
],
)

Expand Down Expand Up @@ -318,11 +304,7 @@ async def test_aws_auth_header_performances_with_spaces_in_value(
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
"custom_with_spaces",
"x-amz-*",
],
)

Expand Down Expand Up @@ -362,11 +344,7 @@ async def test_aws_auth_header_performances_without_spaces_in_value(
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
"custom_without_spaces",
"x-amz-*",
],
)

Expand Down Expand Up @@ -425,11 +403,7 @@ async def test_aws_auth_headers_encoded_values(
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
"My-Header1",
"x-amz-*",
],
)

Expand Down Expand Up @@ -462,12 +436,6 @@ async def test_aws_auth_host_header_with_port(httpx_mock: HTTPXMock):
secret_key="wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY",
region="us-east-1",
service="iam",
include_headers=[
"Host",
"content-type",
"date",
"x-amz-*",
],
)

httpx_mock.add_response(
Expand Down
Loading

0 comments on commit 13c44a7

Please sign in to comment.