Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undocumented breaking change in release v1.8.1 regarding EKS MastersRole #712

Open
Feder1co5oave opened this issue Jun 5, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@Feder1co5oave
Copy link
Contributor

Describe the bug

In the latest release v1.8.1, cdk-lib was upgraded to v2.81.0 which included PR aws/aws-cdk#25473 fixing the security issue aws/aws-cdk#25674

This breaking change was reconciled in EKS blueprints by creating another MastersRole with an equally permissive trust policy in
https://github.com/aws-quickstart/cdk-eks-blueprints/pull/702/files#diff-2deda5874d3f9e558bf44658acfe823824902966d1c87c6c0eb87660f43d4d61

The effect of this is twofold:

  1. when upgrading an existing CDK EKS application to blueprints 1.8.1 from a previous version, the masters role was created as MastersRole* by CDK EKS. This is used in both the aws eks update-kubeconfig command and aws eks get-token command outputs. Since the master role used to authenticate to the cluster is now changed to something like AccessRole*, the commands changed, and the user must update their kubeconfig with the new commands in order to be able to authenticate to the cluster again (if they had updated their kubeconfig with the output command earlier, using the MastersRole).
  2. the security concern addressed in ‼️ NOTICE: eks overly permissive trust policies aws/aws-cdk#25674 is completely ignored, since we still create a masters role with cluster-admin privileges in the cluster, with an unsafe * trust policy.

Expected Behavior

This breaking change (the change in EKS master role) should be documented in the release notes.

Current Behavior

A user would lose their kubectl access to the cluster if they created their kubeconfig via the default commands.

Reproduction Steps

Deploy a cluster with blueprints < 1.8.1.
Update kubeconfig via the aws eks update-kubeconfig command output by cdk deploy.
Make sure you have access to the cluster via kubectl cluster-info.
Upgrade the blueprints and linked dependencies to 1.8.1. Run npm i.
Deploy the stack with the new version.
kubectl commands will not work anymore. An error is returned because the previously configured role does not exist anymore.

Re-run the aws eks update-kubeconfig command output from the second cdk deploy.
You should now be able to issue kubectl commands again.

Possible Solution

The change in master role between versions should be documented in the release notes.

Furthermore, we should addressed the security concerns of creating by default a cluster master role with an insecure wildcard trust policy.

Additional Information/Context

No response

CDK CLI Version

2.81.0 (build bd920f2)

EKS Blueprints Version

1.8.1

Node.js Version

v16.20.0

Environment details (OS name and version, etc.)

Linux

Other information

No response

@Feder1co5oave Feder1co5oave added the bug Something isn't working label Jun 5, 2023
@Feder1co5oave
Copy link
Contributor Author

Feder1co5oave commented Jun 5, 2023

For more information: https://aws.github.io/aws-eks-best-practices/security/docs/iam/

Please note that, since I use SSO on my laptop to login to AWS services, I created an IAM identity mapping in aws-auth, mapping the SSO-based role I assume in that AWS account to the cluster-admin user in the EKS cluster.

I then configured an aws cli profile to assume that role. When configuring the kubeconfig, I should have something like this:

- name: arn:aws:eks:eu-central-1:########:cluster/eks-cluster
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - --region
      - eu-central-1
      - eks
      - get-token
      - --cluster-name
      - eks-cluster
      - --role
      - arn:aws:iam::########:role/AWSReservedSSO_EKSClusterAdminAccess_6a316cc66d154241
      command: aws
      env:
      - name: AWS_PROFILE
        value: eks-profile

or, in the case the profile role and the one that is in the aws-auth mapping are the same, I should omit entirely the --role argument.

https://aws.amazon.com/blogs/containers/a-quick-path-to-amazon-eks-single-sign-on-using-aws-sso/

In this scenario, the change in master role should not cause any issues, since I am not using it to authenticate.
The security concern still stands, since the master role still exist, and can be assumed by anyone in the AWS account.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant