Fix prometheus cadvisor authentication

[imranismail]

Building upon #8762, the kubelet API requires auth if --anonymous-auth flag is set to false for kubelet (which is the case for EKS).

Enabling that flag is a huge security risk

Closes #9408 #9349

Required for all PRs:

• Updated associated README.md.
• Wrote appropriate unit tests.

Before

$ curl -v -k "https://10.38.177.113:10250/pods"
*   Trying 10.38.177.113:10250...
* Connected to 10.38.177.113 (10.38.177.113) port 10250 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):    
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): 
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: O=system:nodes; CN=system:node:ip-10-38-177-113.ap-southeast-1.compute.internal
*  start date: Jul  8 06:58:00 2021 GMT
*  expire date: Jul  8 06:58:00 2022 GMT
*  issuer: CN=kubernetes
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55f65d1620e0)
> GET /pods HTTP/2
> Host: 10.38.177.113:10250
> user-agent: curl/7.77.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 401
< content-type: text/plain; charset=utf-8
< content-length: 12
< date: Tue, 13 Jul 2021 19:21:15 GMT
<
* Connection #0 to host 10.38.177.113 left intact

After

$ export TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
$ curl -v -k -H "Authorization: Bearer $TOKEN" "https://10.38.177.113:10250/pods"
*   Trying 10.38.177.113:10250...
* Connected to 10.38.177.113 (10.38.177.113) port 10250 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):    
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): 
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: O=system:nodes; CN=system:node:ip-10-38-177-113.ap-southeast-1.compute.internal
*  start date: Jul  8 06:58:00 2021 GMT
*  expire date: Jul  8 06:58:00 2022 GMT
*  issuer: CN=kubernetes
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55f84ff27500)
> GET /pods HTTP/2
> Host: 10.38.177.113:10250
> user-agent: curl/7.77.0
> accept: */*
> authorization: Bearer <redacted>
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< content-type: application/json
< date: Tue, 13 Jul 2021 19:19:37 GMT
<
<redacted>
* Connection #0 to host 10.38.177.113 left intact

[telegraf-tiger]

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

[imranismail]

!signed-cla

[telegraf-tiger]

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

• windows_amd64.zip

• windows_i386.zip

• darwin_amd64.tar.gz

• ppc64le.rpm

• linux_ppc64le.tar.gz

• ppc64el.deb

• linux_mipsel.tar.gz

• mipsel.deb

• armel.rpm

• linux_armel.tar.gz

• armel.deb

• linux_mips.tar.gz

• mips.deb

• x86_64.rpm

• freebsd_amd64.tar.gz

• linux_amd64.tar.gz

• amd64.deb

• armv6hl.rpm

• freebsd_armv7.tar.gz

• linux_armhf.tar.gz

• armhf.deb

• i386.rpm

• freebsd_i386.tar.gz

• linux_i386.tar.gz

• i386.deb

• aarch64.rpm

• linux_arm64.tar.gz

• arm64.deb

• static_linux_amd64.tar.gz

• s390x.rpm

• linux_s390x.tar.gz

• s390x.deb

[srebhan]

Seems to also resolve #9408.

[srebhan]

Looks good to me. Thanks for fixing this @imranismail!

[srebhan]

@imranismail can you please change your PR description such that it includes a line

resolves #9408

so that the issue is automatically closed when this is merged.

[imranismail]

@srebhan, added those keywords