Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r/aws_dynamodb_table_export - support DynamoDB ExportTableToPointInTime API. #30399

Merged
merged 10 commits into from
Mar 20, 2024

Conversation

mtt88
Copy link
Contributor

@mtt88 mtt88 commented Apr 2, 2023

Description

Add aws_dynamodb_table_export resource.

This change introduces support for the DynamoDB ExportTableToPointInTime API.

As issue #28917 describes, that this is not a conventional CRUD resources/API but will prove useful for ops tasks. Here are some observations and notes relating to this implementation:

  • AWS provides no mechanism for deleting export table tasks. Even after deleting the table and S3 bucket used in an export, it's still visible in the AWS API - see the AWS Docs

You can find information about export tasks you've run in the past by using the list-exports command. This command returns a list of all exports you've created in the past 90 days. Note that although export task metadata expires after 90 days and jobs older than that are no longer returned by the list-exports command, the objects in your S3 bucket remain as long as their bucket policies allow. DynamoDB never deletes any of the objects it creates in your S3 bucket during an export.
even once we delete a DynamoDB table or the S3 bucket that we export to, the export will still be returned by this API.

  • I was having difficulty writing a specific test for the export_time attribute as this needs to be greater than or equal to the time Point-in-time recovery was enabled. This value is returned from the https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PointInTimeRecoveryDescription.html API but it's not currently accessible from any Terraform datasource/resource. It might be useful to add this?

  • The provider has been configured to wait for either a COMPLETED or a FAILED status when creating the resource. This means that we will wait for the task to finish - AWS suggests that this will usually be less than 30minutes. I have only tested on empty or small DynamoDB tables. It would be good to understand how long these tasks may take on a large table - the default create timeout may need to be reviewed accordingly.

  • I have not tested 'cross account' export (using the s3_bucket_owner attribute). Any help testing this would be useful.

Relations

Closes #28917

References

Output from Acceptance Testing

make testacc TESTS=TestAccDynamoDBTableExport PKG=dynamodb
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/dynamodb/... -v -count 1 -parallel 20 -run='TestAccDynamoDBTableExport'  -timeout 180m
=== RUN   TestAccDynamoDBTableExport_basic
=== PAUSE TestAccDynamoDBTableExport_basic
=== RUN   TestAccDynamoDBTableExport_kms
=== PAUSE TestAccDynamoDBTableExport_kms
=== RUN   TestAccDynamoDBTableExport_s3Prefix
=== PAUSE TestAccDynamoDBTableExport_s3Prefix
=== CONT  TestAccDynamoDBTableExport_basic
=== CONT  TestAccDynamoDBTableExport_s3Prefix
=== CONT  TestAccDynamoDBTableExport_kms
--- PASS: TestAccDynamoDBTableExport_kms (946.40s)
--- PASS: TestAccDynamoDBTableExport_s3Prefix (951.34s)
--- PASS: TestAccDynamoDBTableExport_basic (954.40s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/dynamodb   954.470s

...

@github-actions
Copy link

github-actions bot commented Apr 2, 2023

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added documentation Introduces or discusses updates to documentation. generators Relates to code generators. service/dynamodb Issues and PRs that pertain to the dynamodb service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. size/XL Managed by automation to categorize the size of a PR. labels Apr 2, 2023
@rwlodarczyk-xealth
Copy link

What's the status of this PR? Is it planned for a release of the AWS provider? If so, which release?

@YakDriver YakDriver self-assigned this Mar 19, 2024
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Mar 19, 2024
Copy link
Member

@YakDriver YakDriver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎉

% make t T=TestAccDynamoDBTableExport_ K=dynamodb        
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.21.8 test ./internal/service/dynamodb/... -v -count 1 -parallel 20 -run='TestAccDynamoDBTableExport_'  -timeout 360m
=== RUN   TestAccDynamoDBTableExport_basic
=== PAUSE TestAccDynamoDBTableExport_basic
=== RUN   TestAccDynamoDBTableExport_kms
=== PAUSE TestAccDynamoDBTableExport_kms
=== RUN   TestAccDynamoDBTableExport_s3Prefix
=== PAUSE TestAccDynamoDBTableExport_s3Prefix
=== CONT  TestAccDynamoDBTableExport_basic
=== CONT  TestAccDynamoDBTableExport_s3Prefix
=== CONT  TestAccDynamoDBTableExport_kms
--- PASS: TestAccDynamoDBTableExport_basic (887.94s)
--- PASS: TestAccDynamoDBTableExport_kms (923.29s)
--- PASS: TestAccDynamoDBTableExport_s3Prefix (941.83s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/dynamodb	944.132s

@YakDriver YakDriver merged commit f766b58 into hashicorp:main Mar 20, 2024
44 checks passed
@github-actions github-actions bot added this to the v5.42.0 milestone Mar 20, 2024
@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Mar 22, 2024
Copy link

This functionality has been released in v5.42.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. generators Relates to code generators. service/dynamodb Issues and PRs that pertain to the dynamodb service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement]: Export dynamodb table to S3
3 participants