Add ability to remove egress rules from eks_cluster default security group

[vladimirtiukhtin]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS started creating a security group for it's EKS clusters which is fine. But it is out of terraform and there is no way currently to remove permissive egress rules from it rather than manually. I wish to have an option within aws_eks_cluster resource or a specific data source which allows this

New or Affected Resource(s)

• aws_eks_cluster

[bflad]

Hi @vladimirtiukhtin 👋 Thank you for filing this feature request. On the surface it is a reasonable ask, however there are some details which complicate the maintainer's position on this.

Most Terraform Providers, including this one, follow the HashiCorp Provider Design Principles which states that resource and attribute schema should closely match the underlying API. In this case, the EKS API does not provide this functionality, so typically this is not something the Terraform AWS Provider would include with the existing EKS functionality (e.g. aws_eks_cluster resource). If the EKS API did natively implement this type of functionality, we would gladly revisit this type of implementation.

Another conceptual issue is that Terraform is typically designed to manage the full lifecycle of an infrastructure/API component and in particular, be declarative in the creation and/or full configuration of the component. Terraform does not currently have the declarative concept of ensuring something is always destroyed and does not handle partial configurations well (without special logic encoded in the Terraform resource). To bring this around to this specific use case, we couldn't have the existing aws_security_group_rule resource that always ensured it was deleted. Instead we would have to consider the usage of the existing aws_security_group resource, but that requires importing the resource into Terraform, which is not likely to be changed and introduces manual steps into handling a single Terraform configuration.

We could introduce something like an aws_ec2_managed_security_group resource that presumes another AWS service created the EC2 Security Group and takes the security group identifier as a required argument while not managing anything but the rules within the group, e.g.

# Design sketch for non-existent Terraform AWS Provider resource.
# This sketch does not represent an accepted proposal, but is provided for discussion purposes.
resource "aws_ec2_managed_security_group" "eks-cluster-default" {
  id = aws_eks_cluster.example.vpc_config[0].cluster_security_group_id

  # one or more, similar to aws_security_group
  egress {
    # ...
  }
}

However this introduces its own challenges for practitioners as all of the egress rules would need to be configured as this project would not accept the maintenance burden of special logic for all situations (other AWS services also have these "managed" EC2 Security Groups with their own rules). Requiring operators to configuration all ingress and/or egress rule handling could be very error prone and change over time as AWS services introduce their own changes to the EC2 Security Groups. This resource may also need an argument that allows deleting the EC2 Security Group on destroy (some managed groups would allow this with depending on resources that create the ENI attachments; others would not).

The answer that follows our typical expectations in this project would be to file a feature request in the aws/containers-roadmap repository that outlines the problems you are facing with the EKS created EC2 Security Group and suggest either the ability for EKS to not create that EC2 Security Group automatically, allow the ability to specify a custom EC2 Security Group for this purpose, or the ability for EKS to manage that EC2 Security Group within its own API.

I'm not going to immediately close this though as I do not want to discount the fact that a separate Terraform resource may be in our best interests to cover some of these (albeit generally more esoteric) use cases. We recently ran into another similar case where it was desirable to destroy an automatically created EC2 Security Group since the service does not automatically do that (#14486).

What do you think about the potential separate resource?

[vladimirtiukhtin]

Thanks @bflad for a such extensive explanation. I am already using similar resource for VPC default security group management, and having a generic kind of a resource that does the same seems like a good idea

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.