Pending load balancer security groups leak in AWS legacy cloud provider

[2opremio]

What happened:

A security group is leaked when a <pending> (due to having reached the ELB limit) type: LoadBalancer Service is deleted.

What you expected to happen:

The Security group should be deleted when deleting the service.

How to reproduce it (as minimally and precisely as possible):

1. Create file memcache-svc.yaml:

---
apiVersion: v1
kind: Service
metadata:
  name: service$I
spec:
  ports:
    - name: memcached
      port: 11211
  type: LoadBalancer
  selector:
    name: memcached

2. Create enough services to surpass the ELB limit of your account. In my case 25 is enough to reach it, since the cluster is in a region with the default limit of 20 ELBS:

for I in `seq 25`; do cat memcache-svc.yaml | I=$I envsubst  | kubectl apply -f - ; done

3. Wait for the the services to be created (check with kubectl get services), the EXTERNAL-IP of some of them will remain pending due to the ELB limit having been reached (in my case service21, service22, service23 service24 and service25 are in that state)

4. Delete all the services

for I in `seq 25`; do kubectl delete service service$I ; done

5. Wait for their corresponding ELBs to be deleted by AWS the cloud provider. Wait some more.

6. List the security groups in the cluster's AWS region. You will encounter that the ones matching the PENDING services (service21, service22, service23 service24 and service25) remain there and won't be deleted.

Anything else we need to know?:

I discovered this while working on eksctl-io/eksctl#1010

Environment:

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.7-eks-c57ff8", GitCommit:"c57ff8e35590932c652433fab07988da79265d5b", GitTreeState:"clean", BuildDate:"2019-06-07T20:43:03Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider or hardware configuration: Amazon EKS (Kubernetes 1.13)
• Other: The cluster was created with version Release 0.1.39, runnig eksctl create cluster -f cluster.yaml with:

cluster.yaml:

---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: fons-cluster-4
  region: eu-north-1

nodeGroups:
  - name: ng-1
    instanceType: m5.large
    desiredCapacity: 2

[2opremio]

/sig AWS
/sig cloud-provider

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.