Get non-AAD credentials from a AAD-enabled Kubernetes cluster setup

[schroedan]

What happened:

• we try to setup an automated CI workflow with Travis CI and an Azure Kubernetes cluster setup with enabled AAD integration
• we try to use the login credentials from az aks get-credentials --resource-group <resource-group> --name <name> -f - which returns:

# ...
users:
- name: <username>
  user:
    auth-provider:
      config:
        apiserver-id: <apiserver-id>
        client-id: <client-id>
        tenant-id: <tenant-id>
      name: azure

• but these credentials are incompatible to a sane automated workflow because a user interaction is necessary: in order to get the access/refresh tokens a user must open the login page and paste the provided code into the form and press the submit button and so on

What you expected to happen:

• az aks get-credentials --resource-group <resource-group> --name <name> -f - returns:

# ...
users:
- name: <username>
  user:
    token: <user-token>

as az aks get-credentials --resource-group <resource-group> --name <name> -a -f - does but without the admin privileges

How to reproduce it (as minimally and precisely as possible):

kubectl config set-cluster ${KUBE_CLUSTER_NAME} \
    --server=${KUBE_CLUSTER_SERVER} \
    --certificate-authority=${HOME}/kube_ca.crt --embed-certs=true

## Token authentication does not work with Azure
#kubectl config set-credentials ${KUBE_USER} \
#    --token=${KUBE_TOKEN}

kubectl config set-credentials ${KUBE_USER} \
    --auth-provider=azure \
    --auth-provider-arg=environment=AzurePublicCloud \
    --auth-provider-arg=client-id=${KUBE_AZURE_CLIENT_ID} \
    --auth-provider-arg=tenant-id=${KUBE_AZURE_TENANT_ID} \
    --auth-provider-arg=apiserver-id=${KUBE_AZURE_APISERVER_ID}

kubectl config set-context ${KUBE_CONTEXT} \
    --cluster=${KUBE_CLUSTER_NAME} \
    --user=${KUBE_USER} \
    --namespace=${KUBE_NAMESPACE}

kubectl config use-context ${KUBE_CONTEXT}

kubectl get nodes

Anything else we need to know?:

• The following constellation seem to work but we did not verify if the privileges are the correct ones (admin or user) for now:

# ...
users:
- name: <username>
  user:
    client-certificate-data: <admin-client-certificate-data>
    client-key-data: <admin-client-key-data>
    token: <user-token>

The admin-client-certificate-data and admin-client-key-data was taken from az aks get-credentials --resource-group <resource-group> --name <name> -a -f -.

Environment:

• Kubernetes version (use kubectl version): v1.11.1

[timwebster9]

@schroedan I raised something sort of similar in #556 . I was redirected to the feedback forum, and was just thinking about raising something there.

Ideally we'd be able to do this without having to distribute credentials, which is sort of why I didn't bother investigating further. I really do not want to do that.

[timwebster9]

@schroedan raised on feedback forum if you want to vote :-)

https://feedback.azure.com/forums/914020-azure-kubernetes-service-aks/suggestions/35146387-support-non-interactive-login-for-aad-integrated-c

[davenewza]

Is there a workaround for this problem?

[sshkel]

@davenewza it appears that you can bypass it by requesting admin credentials as pointed out in the comment here https://feedback.azure.com/forums/914020-azure-kubernetes-service-aks/suggestions/35146387-support-non-interactive-login-for-aad-integrated-c
Not an ideal workaround though...

[DenisBiondic]

Why don't you simply create a service account and use this to login to k8s? https://devopscube.com/kubernetes-api-access-service-account/

[haodeon]

I came across this issue, linked from Stack Overflow, trying to get Azure Pipelines working with AAD enabled AKS.

Thanks to @DenisBiondic for the suggestion to use k8s service account. Created one as per the link and tested it in Azure Pipelines, works really well. Now I just need to scope the service account so it has sane permissions.

[jpoizat]

Faced the same issue, the solution provided by @DenisBiondic seems to be the only current solution...

[jnoller]

Closing this issue as old/stale.

If this issue still comes up, please confirm you are running the latest AKS release. If you are on the latest release and the issue can be re-created outside of your specific cluster please open a new github issue.

If you are only seeing this behavior on clusters with a unique configuration (such as custom DNS/VNet/etc) please open an Azure technical support ticket.