GetBucketWebsite Details - NotImplemented error with third party S3 implementation #14645

eldhosepaul7 · 2020-08-13T21:53:43Z

We are using netapp S3 and automate bucket creation and support using terraform. Bucket gets created but terraform fails due to

Error: error getting S3 Bucket website configuration: XNotImplemented: The request you provided implies functionality that is not implemented.

Affected Resource(s)

aws_s3_bucket

Terraform tf file

  access_key                  = "mockaccess"
  region                      = "us-east-1"
  secret_key                  = "mockkey"
  s3_force_path_style         = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  skip_get_ec2_platforms = true
  endpoints {
     s3 = "https://xxxxxxxx:18082"
  }
}

resource "aws_s3_bucket" "b" {
  bucket = "tftest-bucket"
  acl="private"
}

Debug output:

2020-08-13T17:31:53.307-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/08/13 17:31:53 [DEBUG] [aws-sdk-go] DEBUG: Request s3/GetBucketWebsite Details:
2020-08-13T17:31:53.307-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2020-08-13T17:31:53.307-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: GET /tftest-bucket?website= HTTP/1.1
2020-08-13T17:31:53.307-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Host: xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2020-08-13T17:31:53.307-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: User-Agent: aws-sdk-go/1.32.12 (go1.13.7; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.13.0 (+https://www.terraform.io)
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Authorization: AWS4-HMAC-SHA256 Credential=xxxxxxxxxxx/20200813/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=50295faba214b882a9c48010626cf2683cd3b6a9c9bee89023ddbd6ba71ba7ee
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Date: 20200813T213153Z
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Accept-Encoding: gzip
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-08-13T17:31:53.308-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: -----------------------------------------------------
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/08/13 17:31:53 [DEBUG] [aws-sdk-go] DEBUG: Response s3/GetBucketWebsite Details:
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: ---[ RESPONSE ]--------------------------------------
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: HTTP/1.1 501 Not Implemented
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Connection: close
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Length: 263
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Content-Type: application/xml
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Date: Thu, 13 Aug 2020 21:31:53 GMT
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: Server: StorageGRID/11.3.0.3
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Id-2: 12725169
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: X-Amz-Request-Id: 1597354313310942
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: -----------------------------------------------------
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/08/13 17:31:53 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: <Error><Code>XNotImplemented</Code><Message>The request you provided implies functionality that is not implemented.</Message><Resource>/tftest-bucket?website</Resource><RequestId>1597354313310942</RequestId></Error>
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4: 2020/08/13 17:31:53 [DEBUG] [aws-sdk-go] DEBUG: Validate Response s3/GetBucketWebsite failed, attempt 0/25, error XNotImplemented: The request you provided implies functionality that is not implemented.
2020-08-13T17:31:53.324-0400 [DEBUG] plugin.terraform-provider-aws_v2.70.0_x4:  status code: 501, request id: 1597354313310942, host id: 12725169
2020/08/13 17:31:53 [ERROR] eval: *terraform.EvalRefresh, err: error getting S3 Bucket website configuration: XNotImplemented: The request you provided implies functionality that is not implemented.
        status code: 501, request id: 1597354313310942, host id: 12725169
2020/08/13 17:31:53 [ERROR] eval: *terraform.EvalSequence, err: error getting S3 Bucket website configuration: XNotImplemented: The request you provided implies functionality that is not implemented.
        status code: 501, request id: 1597354313310942, host id: 12725169
2020/08/13 17:31:53 [TRACE] [walkRefresh] Exiting eval tree: aws_s3_bucket.b
2020/08/13 17:31:53 [TRACE] vertex "aws_s3_bucket.b": visit complete
2020/08/13 17:31:53 [TRACE] vertex "aws_s3_bucket.b": dynamic subgraph encountered errors
2020/08/13 17:31:53 [TRACE] vertex "aws_s3_bucket.b": visit complete
2020/08/13 17:31:53 [TRACE] vertex "aws_s3_bucket.b (expand)": dynamic subgraph encountered errors
2020/08/13 17:31:53 [TRACE] vertex "aws_s3_bucket.b (expand)": visit complete
2020/08/13 17:31:53 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" errored, so skipping
2020/08/13 17:31:53 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020/08/13 17:31:53 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info

Error: error getting S3 Bucket website configuration: XNotImplemented: The request you provided implies functionality that is not implemented.
        status code: 501, request id: 1597354313310942, host id: 12725169

Steps to Reproduce

Can be reproduced on custom endpoint implementation of s3.

Proposed Solution

Add a flag which can skip features that are not implemented on this third party s3 providers.

The text was updated successfully, but these errors were encountered:

command-tab · 2021-09-07T02:41:57Z

I'm in the same situation — using a NetApp for object storage via its S3 API — and I'm getting the same error:

Error: error getting S3 Bucket website configuration: XNotImplemented:
The request you provided implies functionality that is not implemented.

My provider configuration looks something like:

provider "aws" {
  region                      = "us-east-1"
  access_key                  = "redacted"
  secret_key                  = "redacted"
  skip_credentials_validation = true
  skip_get_ec2_platforms      = true
  skip_region_validation      = true
  skip_requesting_account_id  = true
  skip_metadata_api_check     = true
  endpoints {
    s3 = "https://storagegrid.mycompany.com"
  }
}

resource "aws_s3_bucket" "mybucket" {
  bucket   = "mybucket"
  acl      = "public-read"
}

It would be great to be able to skip the fetch of S3 bucket website configuration, as the aws provider does actually create the expected buckets on the NetApp via its S3 API, but then it fails when trying to read back the static website config. So close!

wica128 · 2021-11-08T08:18:34Z

Hi,

We use S3 from Ceph, Where we don't ~~which also does not~~ support S3 website's.
Some of our customers, which trying to use terraform, hit this problem.
It would be really nice if the customers can configure this.

Maybe better if terraform detects 'NotImplemented' it would just skipped it.

Grtz, Jiri

JqckB · 2021-11-17T15:57:37Z

I have the same issue, on Ceph RGW but I don't have NotImplemented but MethodNotAllowed.

-----------------------------------------------------: timestamp=2021-11-17T16:50:48.240+0100
2021-11-17T16:50:48.299+0100 [INFO]  provider.terraform-provider-aws_v3.65.0_x5: 2021/11/17 16:50:48 [DEBUG] [aws-sdk-go] DEBUG: Response s3/GetBucketWebsite Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 405 Method Not Allowed
Connection: close
Content-Length: 193
Accept-Ranges: bytes
Content-Type: application/xml
Date: Wed, 17 Nov 2021 15:50:48 GMT
X-Amz-Request-Id: tx000000000000012f9c423-00619524d8-70ec24-default


-----------------------------------------------------: timestamp=2021-11-17T16:50:48.299+0100
2021-11-17T16:50:48.299+0100 [INFO]  provider.terraform-provider-aws_v3.65.0_x5: 2021/11/17 16:50:48 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?><Error><Code>MethodNotAllowed</Code><RequestId>tx000000000000012f9c423-00619524d8-70ec24-default</RequestId><HostId>70ec24-default-default</HostId></Error>: timestamp=2021-11-17T16:50:48.299+0100
2021-11-17T16:50:48.299+0100 [INFO]  provider.terraform-provider-aws_v3.65.0_x5: 2021/11/17 16:50:48 [DEBUG] [aws-sdk-go] DEBUG: Validate Response s3/GetBucketWebsite failed, attempt 0/25, error MethodNotAllowed:
	status code: 405, request id: tx000000000000012f9c423-00619524d8-70ec24-default, host id:: timestamp=2021-11-17T16:50:48.299+0100
╷
│ Error: error getting S3 Bucket website configuration: MethodNotAllowed:
│ 	status code: 405, request id: tx000000000000012f9c423-00619524d8-70ec24-default, host id:
│
│   with module.test-backup-cyril.aws_s3_bucket.bucket,
│   on modules/s3_bucket/bucket.tf line 1, in resource "aws_s3_bucket" "bucket":
│    1: resource "aws_s3_bucket" "bucket" {
│

lweberru · 2022-01-24T13:23:13Z

Having the same problem - are there any workarounds possible, like sending empty block?

Third party S3 might reply `NotImplemented` error when creating a bucket. This error prevent the successful apply of the plan. Example when checking policy: ``` ╷ │ Error: error getting S3 bucket ([redacted]) policy: NotImplemented: The requested resource is not implemented │ status code: 501, request id: [redacted], host id: [redacted] │ │ with aws_s3_bucket.lts, │ on main.tf line 22, in resource "aws_s3_bucket" "lts": │ 22: resource "aws_s3_bucket" "lts" { │ ╵ ``` Example when checking for payment: ``` ╷ │ Error: error getting S3 Bucket request payment: NotImplemented: The requested resource is not implemented │ status code: 501, request id: txc25b2f0fe509409081b8b-00620fcb0c, host id: txc25b2f0fe509409081b8b-00620fcb0c │ │ with aws_s3_bucket.lts, │ on main.tf line 23, in resource "aws_s3_bucket" "lts": │ 23: resource "aws_s3_bucket" "lts" { │ ╵ ``` Closes: hashicorp#14645, hashicorp#13726 Signed-off-by: Wilfried Roset <[email protected]>

anGie44 · 2022-02-18T18:25:02Z

Hi @eldhosepaul7, thank you for raising this issue. To look into this a bit further, is the error code you (and @command-tab) are seeing literally XNotImplemented as seen in the logs provided <Error><Code>XNotImplemented</Code><Message> ? or was that X prepended to help locate the error? I ask because looking to the s3_bucket resource code, there is already logic to catch NotImplemented and I believe that's been around since v1.37.0 of the aws provider so I'm curious if there's a difference in the error code the AWS API is sending back

Reference: #5842.

command-tab · 2022-02-18T18:42:33Z

The error code in the response is literally XNotImplemented — that was not a prepended X to locate the error. Thanks for looking into this!

eldhosepaul7 · 2022-02-18T18:45:42Z

@anGie44 yes, the error code for most of the custom AWS providers has XNotImplemented and X is not added to locate the error :). thanks for checking

anGie44 · 2022-02-18T19:26:55Z

Wow, TIL ! thank you for confirming @command-tab @eldhosepaul7

cwalkatron · 2022-02-18T20:39:04Z

This appears to still be an issue in v4 of the AWS provider.

anGie44 · 2022-02-18T22:29:15Z

Hi all, noting here we've added handling for both XNotImplemented and MethodNotAllowed when fetching the bucket website as part of the aws_s3_bucket resource.

github-actions · 2022-02-19T00:08:40Z

This functionality has been released in v4.2.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions · 2022-05-12T02:31:07Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

ghost added the service/s3 Issues and PRs that pertain to the s3 service. label Aug 13, 2020

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Aug 13, 2020

breathingdust added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Sep 17, 2021

wilfriedroset mentioned this issue Feb 18, 2022

Improve support for third party S3 #23278

Merged

anGie44 added the waiting-response Maintainers are waiting on response from community or contributor. label Feb 18, 2022

anGie44 self-assigned this Feb 18, 2022

github-actions bot removed the waiting-response Maintainers are waiting on response from community or contributor. label Feb 18, 2022

anGie44 closed this as completed in #23278 Feb 18, 2022

github-actions bot added this to the v4.2.0 milestone Feb 18, 2022

command-tab mentioned this issue Feb 19, 2022

GetBucketAccelerateConfiguration Details - NotImplemented error with third party S3 implementation #23291

Closed

github-actions bot locked as resolved and limited conversation to collaborators May 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GetBucketWebsite Details - NotImplemented error with third party S3 implementation #14645

GetBucketWebsite Details - NotImplemented error with third party S3 implementation #14645

eldhosepaul7 commented Aug 13, 2020

command-tab commented Sep 7, 2021

wica128 commented Nov 8, 2021 •

edited

Loading

JqckB commented Nov 17, 2021 •

edited

Loading

lweberru commented Jan 24, 2022

anGie44 commented Feb 18, 2022 •

edited

Loading

command-tab commented Feb 18, 2022

eldhosepaul7 commented Feb 18, 2022 •

edited

Loading

anGie44 commented Feb 18, 2022

cwalkatron commented Feb 18, 2022

anGie44 commented Feb 18, 2022 •

edited

Loading

github-actions bot commented Feb 19, 2022

github-actions bot commented May 12, 2022

GetBucketWebsite Details - NotImplemented error with third party S3 implementation #14645

GetBucketWebsite Details - NotImplemented error with third party S3 implementation #14645

Comments

eldhosepaul7 commented Aug 13, 2020

Affected Resource(s)

Terraform tf file

Debug output:

Steps to Reproduce

Proposed Solution

command-tab commented Sep 7, 2021

wica128 commented Nov 8, 2021 • edited Loading

JqckB commented Nov 17, 2021 • edited Loading

lweberru commented Jan 24, 2022

anGie44 commented Feb 18, 2022 • edited Loading

command-tab commented Feb 18, 2022

eldhosepaul7 commented Feb 18, 2022 • edited Loading

anGie44 commented Feb 18, 2022

cwalkatron commented Feb 18, 2022

anGie44 commented Feb 18, 2022 • edited Loading

github-actions bot commented Feb 19, 2022

github-actions bot commented May 12, 2022

wica128 commented Nov 8, 2021 •

edited

Loading

JqckB commented Nov 17, 2021 •

edited

Loading

anGie44 commented Feb 18, 2022 •

edited

Loading

eldhosepaul7 commented Feb 18, 2022 •

edited

Loading

anGie44 commented Feb 18, 2022 •

edited

Loading