error resolving name when attempting to create the dnsendpoint object

[raffaelespazzoli]

What happened:

I am getting this error when the federation control plane is attempting to create a dnsendpoint object related to a servicednsrecord:

E0503 16:33:11.838916       1 common.go:71] Failed to resolve af1d01e4c6dc011e99c520672c4b2178-1584149087.us-west-1.elb.amazonaws.com, err: lookup af1d01e4c6dc011e99c520672c4b2178-1584149087.us-west-1.elb.amazonaws.com on 172.30.0.10:53: no such host
E0503 16:33:11.838961       1 controller.go:162] Error processing fns1/httpd (will retry): lookup af1d01e4c6dc011e99c520672c4b2178-1584149087.us-west-1.elb.amazonaws.com on 172.30.0.10:53: no such host

attempting the same resolution from the container terminals gives this result:

nslookup af1d2ee956dc011e9bdde023a7fb9768-592033900.us-east-2.elb.amazonaws.com
nslookup: can't resolve '(null)': Name does not resolve

Name:      af1d2ee956dc011e9bdde023a7fb9768-592033900.us-east-2.elb.amazonaws.com
Address 1: 52.14.123.115 ec2-52-14-123-115.us-east-2.compute.amazonaws.com
Address 2: 18.219.167.21 ec2-18-219-167-21.us-east-2.compute.amazonaws.com
Address 3: 18.223.8.153 ec2-18-223-8-153.us-east-2.compute.amazonaws.com

the name seems to be resolved correctly, but at notice the initial error:
nslookup: can't resolve '(null)': Name does not resolve

digging on it, it seems to be related to how name resolution works in an alpine image, and there are issues on it: gliderlabs/docker-alpine#476

I am not 100% sure this is the root cause.
Also this issue is happening in AWS because in AWS loadbalancers return a name instead of an IP.
Overall though this looks like a bug to me because this should work.

Environment:

• Kubernetes version (use kubectl version)

kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+81fc896", GitCommit:"81fc896", GitTreeState:"clean", BuildDate:"2019-04-21T23:18:54Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

• Federation version

Federation v2 controller-manager version: version.Info{Version:"v0.0.9", GitCommit:"442e8868c6bd02bb61871af02269cb18a58804d4", GitTreeState:"clean", BuildDate:"2019-04-30T05:36:16Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}

• Install method (helm vs script)
  helm
• Install configuration (namespaced vs cluster-scoped)
  namespaces
• Others
  cluster running on AWS

/kind bug

[shashidharatd]

@raffaelespazzoli, Thanks for filing the issue. This is legit issue and affects the environment involving AWS loadbalancers.

[shashidharatd]

Related Issue: kubernetes/kubernetes#56903 (comment)

[raffaelespazzoli]

@shashidharatd I run a test with a different base image. The issue remains, so I can confirm that it's not caused by the alpine base image.
Are there any plans to fix this issue?

[shashidharatd]

@raffaelespazzoli, this need more investigation to find the root cause.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[raffaelespazzoli]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.