AWS - tagging of resources

[radeksimko]

As requested by @catsby in #1080 (comment) I'm creating this meta-issue to track all tagging functionality for AWS in TF.

EC2-related tagging

Existing

It's worth pointing out that the existing ./builtin/providers/aws/tags.go should really be called ec2_tags.go as it in fact uses the EC2 API only and deals with tags that are related to EC2 only which are currently following:

• aws_instance
• aws_internet_gateway
• aws_network_acl
• aws_route_table
• aws_security_group
• aws_subnet
• aws_vpc
• aws_vpc_peering_connection

Missing

Here's the full list of EC2 resources that can be tagged using the same API endpoint which means that we're missing tagging functionality in the EC2 world for these:

• AMI - probably unrelated to Terraform
• aws_customer_gateway - Added in Add support for AWS customer gateways #1746
• aws_dhcp_options - not supported by TF (yet)
• EBS Volume - tagging not implemented, most likely should go under aws_instance.block_device.tags
• aws_network_interface - currently auto-managed along with EC2 instances to assign public IP, not sure if this would be useful as a separate resource, maybe related to Support AWS Elastic Network Interfaces (ENI) #1149 ? Support AWS Elastic Network Interfaces (ENI) #1149 was merged
• aws_reserved_instances - not supported by TF (yet)
• aws_spot_instance_request - provider/aws: tag the spot instance. #4380
• Snapshot - probably unrelated to Terraform
• aws_vpn_gateway - not supported by TF (yet) Added in provider/aws: Add Support For AWS VPN Gateways using aws-sdk-go #1137
• aws_vpn_connection - not supported by TF (yet) - Adding CRUD support for Customer Gateway for AWS VPC. #838 Resources to manage AWS vpn connections, customer and virtual gateways #1011 AWS VPN Resources  #551

EC2-UNrelated tags

I think most of the resources having the simple tag structure as below 🔽

type Tag struct {
    Key   aws.StringValue `ec2:"Key" xml:"key"`
    Value aws.StringValue `ec2:"Value" xml:"value"`
}

may theoretically work with the existing tagging logic as long as we can abstract the EC2-specific API calls & data-types that we currently have in ./builtin/providers/aws/tags.go.

grep -i 'ec2' ./builtin/providers/aws/tags.go

./builtin/providers/aws/tags.go:    "github.com/hashicorp/aws-sdk-go/gen/ec2"
./builtin/providers/aws/tags.go:func setTags(conn *ec2.EC2, d *schema.ResourceData) error {
./builtin/providers/aws/tags.go:            err := conn.DeleteTags(&ec2.DeleteTagsRequest{
./builtin/providers/aws/tags.go:            err := conn.CreateTags(&ec2.CreateTagsRequest{
./builtin/providers/aws/tags.go:func diffTags(oldTags, newTags []ec2.Tag) ([]ec2.Tag, []ec2.Tag) {
./builtin/providers/aws/tags.go:    var remove []ec2.Tag
./builtin/providers/aws/tags.go:func tagsFromMap(m map[string]interface{}) []ec2.Tag {
./builtin/providers/aws/tags.go:    result := make([]ec2.Tag, 0, len(m))
./builtin/providers/aws/tags.go:        result = append(result, ec2.Tag{
./builtin/providers/aws/tags.go:func tagsToMap(ts []ec2.Tag) map[string]string {

Resources that would benefit from this as these have the same Tag structure but not part of the EC2 group:

• cloudformation - provider/aws: Add aws_cloudformation_stack #2636
• elasticbeanstalk - provider/aws: Elastic Beanstalk Application and Environment #3157
• elb - tagging not implemented, but possible - provider/aws: Add tag support to ELB #1289
• emr - not supported by TF (yet)
• kinesis - provider/aws: Allow tags for kinesis streams #3397
• rds - tagging not implemented, but possible - Add Tags to AWS RDS #1292
• redshift - not supported by TF (yet)

EC2-UNrelated requiring extra care

All these will most likely require resource-specific code for handling tags

autoscaling - #1080

type Tag struct {
    Key               aws.StringValue  `query:"Key" xml:"Key"`
    PropagateAtLaunch aws.BooleanValue `query:"PropagateAtLaunch" xml:"PropagateAtLaunch"`
    ResourceID        aws.StringValue  `query:"ResourceId" xml:"ResourceId"`
    ResourceType      aws.StringValue  `query:"ResourceType" xml:"ResourceType"`
    Value             aws.StringValue  `query:"Value" xml:"Value"`
}

route53

type Tag struct {
    XMLName xml.Name

    Key   aws.StringValue `xml:"Key"`
    Value aws.StringValue `xml:"Value"`
}

s3 - #1312

type Tag struct {
    XMLName xml.Name

    Key   aws.StringValue `xml:"Key"`
    Value aws.StringValue `xml:"Value"`
}

[radeksimko]

@catsby feel free to modify and/or reorganise things if needed.

[catsby]

Excellent, thanks @radeksimko !

[catsby]

Right now I've duplicated the tags.go for the non-EC2 resource(s) I've added, ELB and RDS.

I plan to introduce a simple Tag struct like you've shown above. The idea is to make diffTags(), tagsToMap() and tagsFromMap() work on that generic tag.

Each resource to be responsible for converting from the simple Tag to their own version (ex. rds.Tag), and maintain their own setTags func as needed.

Most of the ec2 resources should be easy and not require their own file.
The more complicated ones like ASG will need a lot more in their own tags.go file, as @radeksimko has done in #1080 with it's own diff'ing.

In the end, all this is to reuse those three funcs mentioned, as much as possible.

[wazoo]

It is great that you are tracking this, now that AWS has implemented the Resource Groups UI tags will become even more valuable going forward.

[catsby]

ASG Tagging was just merged with #1319

[arothian]

Any update on this? Eagerly awaiting tagging for EBS volumes so I can convince my boss that terraform supports all our use cases. 👍

[jtopper]

👍 This is one of the things we're waiting on too.

[radeksimko]

@arothian @jtopper #2135

[arothian]

@radeksimko Awesome thanks! Is it significantly different to provide tagging for the volumes from the block_device configuration in aws_instance? I guess I need to learn Go.

[jtopper]

That's my use case too - I want to be able to tag ebs_block_device attributes in the aws_instance resource type, and my go-fu is similarly weak :)

[phinze]

@jtopper @arothian The ebs_block_device attribute on instances uses the AWS API's BlockDeviceMapping functionality, which I don't believe supports tagging (feel free to link the docs if you can find it though! [1]).

So if you want to tag a volume, you'll need to use this ebs_volume and the new volume_attachment resource (#2050) to attach the volume after the instance is created.

[1] I was looking here https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html

[jtopper]

@phinze: Ah yes, you're quite right. I misread how we're doing that in our existing fog-based tooling, where we're actually doing something similar to what you describe here. Thanks! (and thanks @radeksimko - hit me up for a beer next time we're in the same place!)

[pdecat]

Tags are now also supported on CloudFront distributions (since 2016/08/01 apparently):

• https://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/tags-tag-resource.html
• https://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/tags-list-tags-for-resource.html
• https://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/distribution-create-with-tags.html

I did not find a corresponding issue, should I create one?

[catsby]

Hey friends I'm going to close this meta-issue now. If you find a resource that is missing tag support, please open a new issue for it.

Thanks!

[smicha]

is it possible to update tags on a terraform managed ASG from the cli

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.