Feature Request: Support for sts:assumerole/source_profile

[geekifier]

In a MSP setting we often interact with many AWS client accounts. In order to streamline management of the account, we set up an IAM role in each client's account, which we then can temporarily 'assume' using our IAM credential.

See: https://blogs.aws.amazon.com/security/post/Tx2Q5SSG3SFRRHO/How-to-Use-External-ID-When-Granting-Access-to-Your-AWS-Resources.

When attempting to use downstream tools utilizing aws-sdk-go and trying to read profiles, we see errors such as this one:

Error: SharedCredsAccessKey: shared credentials in /Users//.aws/credentials did not contain aws_access_key_id.

My ~/.aws/config:

[default]
output = json
region = us-east-1
[profile PROFILENAME]
source_profile = default
role_arn = arn:aws:iam::SECRET:role/SECRET-sts
region = us-east-1

In ~/.aws/credentials, the 'default' profile has the credentials for my IAM User, in our AWS Account (not client's).

As you can see, the credential file does not contain an access key for that specific profile, because it simply sources the default profile (using source_profile), which then uses the assumes the sts:assumerole function.

All of the native AWS CLI tools work natively with this, and it would be great if aws-sdk-go would too! I realize this is not a super widespread problem, and appreciate anyone who would be willing to look into this.

[jasdel]

@geekifier thanks for creating this feature request. The SDK does have a stscreds.AssumeRoleProvider which allows using external IDs for assuming roles and can be used as a credential provider in the SDK. This could be used if your client has access to the role arn and external ID.

sess := session.New()
sess.Config.Credentials = &aws.Config{Credentials: stscreds. NewCredentials(sess, roleArn, func(p * AssumeRoleProvider) {
    p.ExternalID = externalAccountID
})

// Use sess to create service clients and make API operation calls with.

To simplify this the SDK would need to support a portion of the CLI's config file. (#384) Portions of the config file are specific to how the CLI functions and may not translate well to the SDK. Having parity where possible would be helpful. To implement the config file more fully this may also require #414 for service specific configuration injected into service clients.

[geekifier]

I came across another OSS project, TerraForm, which is also affected by this issue. Unfortunately, this is a bit of a show stopper when it comes to using these useful tools. We do not want to create/store IAM credentials for each client's account, as that is contrary to AWS best practices. Any chance of this request being on a roadmap in the near future?

[xibz]

Hello @geekifier, we are doing our best to ensure that customer demanded features are of highest priority. I can definitely see this being on our roadmap, but when, I don't know. I will let you know when we are looking to implement this feature.

[kquinsland]

@xibz If it's any help, I'd like to add a +1 to this because two tools that I use

• Terraform
• cli53

are both effected by this issue. Lots of indirect Go SDK users are effected by this :/

So does that count as a +3? ;)

[aaroncaito]

Need here as well, hashicorp tools, cli53 and every other aws targeted go project would benefit.

[freefood89]

+1

This affects the awslogs driver on moby/moby#23698

As for tools like Terraform and Ansible, the current work around I use is to wrap Terraform commands using a python script that uses boto3 to retrieve the AWS secret/key/sessiontoken and then set them as process variables prior to calling the wrapped command: ./wrapper.py terraform plan

This may quickly get annoying unless you're running terraform from an automation tool like Jenkins or Drone

[bmurphy1976]

@freefood89 would you mind sharing that script?

[freefood89]

@bmurphy1976 no guarantees that it's bullet proof 😛
Suggestions for improvement are also welcome

https://gist.github.com/freefood89/7446e54dfc486f420a386a895ae1c49f

[bmurphy1976]

Thanks! Doesn't have to be bullet proof, I just need a good starting point. We were already planning to do something similar so if it saves us some effort, great!!

[jasdel]

Linking to #384 which tracks the SDKs support general support of the ~/aws/config file. Once the SDK supports the config file we should be able to support assume role.

Thanks for voting for this issue, it helps drive our backlog priority.

[5330]

Would use this feature daily to support AWS using Terraform (tf) instead of using a wrapper. (wtf)

👍

[STRogers]

This would also be of use to us. (Using cli53).

[jasdel]

Thanks for the feedback, and feature request. In release v1.3.0 we added support for the using assume role with the shared config's source_profile and role_arn fields. This feature is an opt-in and needs to be enabled. The easiest way to enable the functionality is via the AWS_SDK_LOAD_CONFIG environment variable. See the session package docs for more information.

Let us know if you have any additional feedback, questions, or issues.

[geekifier]

@jasdel Thanks for the implementation. One thing that is worth mentioning, is that the "opt-in" feature you have mentioned was implemented, and promptly abandoned by the aws-sdk-ruby team. See aws/aws-sdk-ruby#1257.

It might make sense for various SDKs to have feature parity/predictable behavior, but I am not sure about the dynamics of the various teams at AWS, so I am not suggesting either way, just referencing the other discussion :).

[jasdel]

Thanks for the feedback @geekifier, we're talking about this idea driven by @awood45 between our SDKs based on customer feedback.

[jasdel]

Hi everyone, based on the feedback we received, along with the discussion over on aws/aws-sdk-ruby#1257 we've decided to enable the functionality by default. This will remove the needed for the AWS_SDK_LOAD_CONFIG env var as it would no longer do anything. A new flag AWS_SDK_CONFIG_OPT_OUT.

PR #788

[jasdel]

Per the discussion in PR #788, we've held off on switching this functionality from opt in to opt out while determining the viability, determining impact of opt in to opt out change, and demand of this feature. With that said, we're listening for feedback on this issue.

[johnrengelman]

My thoughts on this:

1. if the AWS docs at any point say that source_profile etc is a valid field for the shared credentials file then the SDKs should support it. It's not realistic to expect people to know the difference (i.e I can use these settings when using the aws CLI, but not when using an application that uses the SDK because that causes an error)
2. This allows for centralizing credentials configuration via profiles and the shared credential file. Tools that use the SDK would no longer need to implement there own configuration/credential chain to handle things the AWS CLI does natively.
3. The only caveat to this when enabling MFA. I'm not sure the SDK could effectively implement prompting the user for the MFA code b/c it was configured in the profile.

[dprime]

I've recently run into this limitation and not only is the default behaviour confusing, it took me an hour to find this issue and realise what was actually happening. Default enabled gets my vote, for sure.

[mcfedr]

Found similar problem when trying to use amazon-ecr-credential-helper for docker, and it cannot read my profiles that use source_profile, I have had to manually call assume role, and export the AWS_ACCESS_KEY etc

[devops-expanse]

i think it's strange to call this a feature. Shouldn't the profile be read properly from this library?