Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

delete a cluster - CloudFormation Stack finish with DELETE_FAILED status #616

Closed
eladh opened this issue Mar 7, 2019 · 1 comment
Closed

Comments

@eladh
Copy link

eladh commented Mar 7, 2019

What happened?
after creating simple eks cluster and running eksctl delete the CloudFormation Stack finish with . DELETE_FAILED status.
i attached the relevant logs but also i can see that the ec2 volumes, load balancers and security groups were not deleted properly.

17:22:21 UTC+0200DELETE_FAILEDAWS::CloudFormation::Stackeksctl-demo2-platform-cluster-clusterThe following resource(s) failed to delete: [InternetGateway, VPC, VPCGatewayAttachment].17:22:20 UTC+0200DELETE_FAILEDAWS::EC2::VPCVPCThe vpc 'vpc-0a285bb13425fa14f' has dependencies and cannot be deleted. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: 9554fa04-a267-4e7c-99cc-2c9663f7c00f)Physical ID:vpc-0a285bb13425fa14f17:14:21 UTC+0200DELETE_FAILEDAWS::EC2::InternetGatewayInternetGatewayThe internetGateway 'igw-08d4a0e3f4d402776' has dependencies and cannot be deleted. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: de7c2759-d5a1-4195-b573-b767d3542029)17:03:48 UTC+0200DELETE_IN_PROGRESSAWS::EC2::VPCVPC17:03:47 UTC+0200DELETE_COMPLETEAWS::EC2::SubnetSubnetPublicUSEAST1A17:03:21 UTC+0200DELETE_COMPLETEAWS::EC2::SubnetSubnetPublicUSEAST1B16:57:24 UTC+0200DELETE_IN_PROGRESSAWS::EC2::InternetGatewayInternetGateway16:57:23 UTC+0200DELETE_FAILEDAWS::EC2::VPCGatewayAttachmentVPCGatewayAttachmentNetwork vpc-0a285bb13425fa14f has some mapped public address(es). Please unmap those public address(es) before detaching the gateway. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: 1d3dfd1f-d9e5-4000-944d-a2b978a9940a) |   | 17:22:21 UTC+0200 | DELETE_FAILED | AWS::CloudFormation::Stack | eksctl-demo2-platform-cluster-cluster | The following resource(s) failed to delete: [InternetGateway, VPC, VPCGatewayAttachment]. |   | 17:22:20 UTC+0200 | DELETE_FAILED | AWS::EC2::VPC | VPC | The vpc 'vpc-0a285bb13425fa14f' has dependencies and cannot be deleted. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: 9554fa04-a267-4e7c-99cc-2c9663f7c00f) |   | Physical ID:vpc-0a285bb13425fa14f |   | 17:14:21 UTC+0200 | DELETE_FAILED | AWS::EC2::InternetGateway | InternetGateway | The internetGateway 'igw-08d4a0e3f4d402776' has dependencies and cannot be deleted. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: de7c2759-d5a1-4195-b573-b767d3542029) |   | 17:03:48 UTC+0200 | DELETE_IN_PROGRESS | AWS::EC2::VPC | VPC |   |   | 17:03:47 UTC+0200 | DELETE_COMPLETE | AWS::EC2::Subnet | SubnetPublicUSEAST1A |   |   | 17:03:21 UTC+0200 | DELETE_COMPLETE | AWS::EC2::Subnet | SubnetPublicUSEAST1B |   |   | 16:57:24 UTC+0200 | DELETE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway |   |   | 16:57:23 UTC+0200 | DELETE_FAILED | AWS::EC2::VPCGatewayAttachment | VPCGatewayAttachment | Network vpc-0a285bb13425fa14f has some mapped public address(es). Please unmap those public address(es) before detaching the gateway. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: 1d3dfd1f-d9e5-4000-944d-a2b978a9940a)
  | 17:22:21 UTC+0200 | DELETE_FAILED | AWS::CloudFormation::Stack | eksctl-demo2-platform-cluster-cluster | The following resource(s) failed to delete: [InternetGateway, VPC, VPCGatewayAttachment].
  | 17:22:20 UTC+0200 | DELETE_FAILED | AWS::EC2::VPC | VPC | The vpc 'vpc-0a285bb13425fa14f' has dependencies and cannot be deleted. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: 9554fa04-a267-4e7c-99cc-2c9663f7c00f)
  | Physical ID:vpc-0a285bb13425fa14f
  | 17:14:21 UTC+0200 | DELETE_FAILED | AWS::EC2::InternetGateway | InternetGateway | The internetGateway 'igw-08d4a0e3f4d402776' has dependencies and cannot be deleted. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: de7c2759-d5a1-4195-b573-b767d3542029)
  | 17:03:48 UTC+0200 | DELETE_IN_PROGRESS | AWS::EC2::VPC | VPC |  
  | 17:03:47 UTC+0200 | DELETE_COMPLETE | AWS::EC2::Subnet | SubnetPublicUSEAST1A |  
  | 17:03:21 UTC+0200 | DELETE_COMPLETE | AWS::EC2::Subnet | SubnetPublicUSEAST1B |  
  | 16:57:24 UTC+0200 | DELETE_IN_PROGRESS | AWS::EC2::InternetGateway | InternetGateway |  
  | 16:57:23 UTC+0200 | DELETE_FAILED | AWS::EC2::VPCGatewayAttachment | VPCGatewayAttachment | Network vpc-0a285bb13425fa14f has some mapped public address(es). Please unmap those public address(es) before detaching the gateway. (Service: AmazonEC2; Status Code: 400; Error Code: DependencyViolation; Request ID: 1d3dfd1f-d9e5-4000-944d-a2b978a9940a)

What you expected to happen?
A full assets clean should occur

How to reproduce it?
Just run the following :
eksctl create cluster --config-file ./group-nodes.yaml
eksctl delete cluster --name=demo-cluster --region=us-east-1

group-nodes.yaml file

---
apiVersion: eksctl.io/v1alpha4
kind: ClusterConfig

metadata:
  name: demo-cluster
  region: us-east-1

nodeGroups:
  - name: ng1-public2
    instanceType: m5.4xlarge
    minSize: 2
    volumeSize: 50
    volumeType: gp2
    maxSize: 3
    allowSSH: true
    ami: auto
    labels:
      nodegroup-type: demo-cluster-workloads
    preBootstrapCommand:
          # allow docker registries to be deployed as cluster service
          - 'echo "{\"bridge\":\"none\",\"log-driver\":\"json-file\",\"log-opts\":{\"max-size\":\"10m\",\"max-file\":\"10\"},\"live-restore\":true,\"max-concurrent-downloads\":10,\"insecure-registries\":[\"172.20.0.0\/16\",\"10.100.0.0\/16\"]}" > /etc/docker/daemon.json'
          - "systemctl restart docker"

availabilityZones: ["us-east-1a", "us-east-1b"]

Anything else we need to know?
What OS are you using, are you using a downloaded binary or did you compile eksctl, what type of AWS credentials are you using (i.e. default/named profile, MFA) - please don't include actual credentials though!

Versions
Please paste in the output of these commands:

[ℹ]  version.Info{BuiltAt:"", GitCommit:"", GitTag:"0.1.23"}
Darwin local 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}

Also include your version of heptio-authenticator-aws

Logs
Include the output of the command line when running eksctl. If possible, eksctl should be run with debug logs. For example:
eksctl get clusters -v 4
Make sure you redact any sensitive information before posting.
If the output is long, please consider a Gist.

@errordeveloper
Copy link
Contributor

Hi @eladh! This is a long standing issue we need to fix soon, see #103 and #274. Hope you don't mind that I close this.

torredil pushed a commit to torredil/eksctl that referenced this issue May 20, 2022
Update hack/run-e2e-test to be more idempotent and pleasant to use
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants