boto3.client("s3").head_bucket does not throw NoSuchBucket as per documentation

[ashaik687]

Contrary to documentation, boto3.client("s3").head_bucket(Bucket=bucket_name) does not throw S3.Client.exceptions.NoSuchBucket. It throws An error occurred (404) when calling the HeadBucket operation: Not Found
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.head_bucket

import boto3
client = boto3.client("s3")
try:
    client.head_bucket(Bucket='bucket-does-not-exist')
except client.exceptions.NoSuchBucket:
    print('exception caught')

The above code snippet should print 'exception caught'. However, It throws the below exception.

  File "/Users/ashaik/.local/share/virtualenvs/loadenv38-el3BARCC/lib/python3.8/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/ashaik/.local/share/virtualenvs/loadenv38-el3BARCC/lib/python3.8/site-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (404) when calling the HeadBucket operation: Not Found

Please align documentation and behavior of exception thrown.

[swetashre]

@ashaik687 - Thank you for your post. The API model provided by S3 that the AWS SDKs use to generate the code and documentation for their S3 clients shows NoSuchBucket as the error shape for a HeadBucket operation.
Since HEAD operations don't return an HTTP response body the SDK can't convert the response to a modeled error so instead it just returns a generic HTTP 404 response. So it need to be fixed on s3 side.
According to the s3 team service will not be changing its behavior regarding the error returned on HeadBucket requests to nonexistent resources.

Unfortunately we won't be able to make the requested change in our documentation to account for this. All of the AWS SDKs use the same model provided to us by the S3 team to generate the code and documentation for their S3 clients, unfortunately the way in which the model is written by the S3 team does not allow for removing the constant used for NoSuchBucket from this operation without creating a breaking change.

[benkehoe]

There's no way for the S3 model to indicate that a 404 error from HeadBucket should be converted to a NoSuchBucket error?

[swetashre]

@benkehoe - Since HeadBucket operations doesn't return an HTTP response body the SDK can't convert the response to a modeled error so instead it just returns a generic HTTP 404 response.
In this case s3 will have to update their model with the correct exception as AWS SDKs use those api model to generate documentation.

[BastianZim]

@stobrien89 Sorry to be pinging you out of the blue but would it be possible to add a note or something somewhere for this, as I just stumbled over the same thing. Also, can we expect this to never change or might this be something that gets fixed in the future?

[stobrien89]

Hi @BastianZim,

My apologies for the delayed response. It seems like something could be done from a documentation standpoint since it seems they won't be changing the API behavior (I'll double-check on that), so I'll see if they're able to fix this upstream.

[BastianZim]

Thank you @stobrien89!

[stobrien89]

Hi again @BastianZim,

Had a chance to look into this further this morning and it looks like the documentation has been updated:

If the bucket does not exist or you do not have permission to access it, the HEAD request returns a generic 404 Not Found or 403 Forbidden code. A message body is not included, so you cannot determine the exception beyond these error codes.

From what I understand, they won't be able to remove the error from the model, as it would be a breaking change, so it will remain listed under the exceptions section, which is derived from the s3 model. Wish there was more we could do here!

[BastianZim]

Hi @stobrien89 ok thanks for looking into this again though!

[m-skarzyn]

Method which works:

import boto3
import botocore

client = boto3.client("s3")
try:
    client.head_bucket(Bucket='bucket-does-not-exist')
except botocore.exceptions.ClientError:
    print('exception caught')