segfault after node reboot

[thatsmydoing]

This looks similar to #914 but fails in a different place.

Kubernetes: v1.15.11-eks-af3caf
AWS CNI: 1.6.0

starting IPAM daemon in background ... ok.
checking for IPAM connectivity ... ok.
copying CNI plugin binaries and config files ...  ok.
foregrounding IPAM daemon ... 
failed (process terminated)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1427e61]
goroutine 328 [running:
github.com/aws/amazon-vpc-cni-k8s/pkg/awsutils.(*EC2InstanceMetadataCache).DescribeENI(0xc00012efc0, 0xc000a99020, 0x15, 0x1, 0x1, 0xc0003b2850, 0x1d91580, 0xc00062ac30, 0x0, 0x8)
    /go/src/github.com/aws/amazon-vpc-cni-k8s/pkg/awsutils/awsutils.go:911 +0x531
github.com/aws/amazon-vpc-cni-k8s/pkg/awsutils.(*EC2InstanceMetadataCache).freeENI(0xc00012efc0, 0xc000a99020, 0x15, 0x77359400, 0xdf8475800, 0x0, 0xc000046558)
    /go/src/github.com/aws/amazon-vpc-cni-k8s/pkg/awsutils/awsutils.go:807 +0xed
github.com/aws/amazon-vpc-cni-k8s/pkg/awsutils.(*EC2InstanceMetadataCache).FreeENI(0xc00012efc0, 0xc000a99020, 0x15, 0x1, 0x1)
    /go/src/github.com/aws/amazon-vpc-cni-k8s/pkg/awsutils/awsutils.go:800 +0x57
github.com/aws/amazon-vpc-cni-k8s/pkg/ipamd.(*IPAMContext).tryFreeENI(0xc0000e80f0)
    /go/src/github.com/aws/amazon-vpc-cni-k8s/pkg/ipamd/ipamd.go:517 +0x11d
github.com/aws/amazon-vpc-cni-k8s/pkg/ipamd.(*IPAMContext).updateIPPoolIfRequired(0xc0000e80f0)
    /go/src/github.com/aws/amazon-vpc-cni-k8s/pkg/ipamd/ipamd.go:477 +0x71
github.com/aws/amazon-vpc-cni-k8s/pkg/ipamd.(*IPAMContext).StartNodeIPPoolManager(0xc0000e80f0)
    /go/src/github.com/aws/amazon-vpc-cni-k8s/pkg/ipamd/ipamd.go:463 +0x39
created by main._main
    /go/src/github.com/aws/amazon-vpc-cni-k8s/cmd/aws-k8s-agent/main.go:67 +0x453
stream closed

We've recently had problems with nodes spontaneously rebooting but I've only seen this once so far so it's not really consistent.

[mogren]

Hi @thatsmydoing, thanks for opening a ticket!

I see that you use v1.6.0, are you absolutely sure it is not the same as issue #914? This is the line from your log:

amazon-vpc-cni-k8s/pkg/awsutils/awsutils.go

Line 911 in 20c3a65

return result.NetworkInterfaces[0].PrivateIpAddresses, tags, result.NetworkInterfaces[0].Attachment.AttachmentId, nil

It seems to assume that result.NetworkInterfaces[0].Attachment will never be nil, which not the case. Could you try updating to v1.6.2 and see if that fixes the issue?

[thatsmydoing]

Ah, sorry. The linked issue mentioned the fix being in 1.6, but I didn't realize that it was merged in 1.6.1 instead. I'll just close the issue and reopen it if I see it again after updating.