provider/aws: iam_instance_profile not ready when TF creates the LaunchConfig of an ASG #2136

stefancocora · 2015-05-29T15:00:17Z

I've finally hit this issue as well !
I'm on

terraform version
Terraform v0.5.3-dev (a6b8b65e6e0b5cf1f0b4fcf3f6abde3b7db21a97)

which has been built 5 days after the PR #2037 was merged.
The above PR #2037 does not fix the issue for me.

Am I missing something obvious ?

When I apply all resources on the 1st run ( from scratch - no resources present ) TF fails because of the slow IAM propagation(see error output below)
When I apply the plan the 2nd time, naturally TF sees the resources that haven't been created ( 3 ASGs+LC ) and creates them since the IAM instance profile has propagated by the end of the first TF run.

Error output when creating all VPC resources in one run:

...
aws_launch_configuration.jenkinsmaster-cps-LC: Creating...
  associate_public_ip_address:               "" => "0"
  ebs_block_device.#:                        "" => "<computed>"
  ebs_optimized:                             "" => "<computed>"
  iam_instance_profile:                      "" => "ec2describe_profile"
  image_id:                                  "" => "ami-fd4fda8a"
  instance_type:                             "" => "t2.medium"
  key_name:                                  "" => "ops-ec2-provision"
  name:                                      "" => "jenkinsmaster-cps-LC"
  root_block_device.#:                       "" => "1"
  root_block_device.0.delete_on_termination: "" => "1"
  root_block_device.0.iops:                  "" => "<computed>"
  root_block_device.0.volume_size:           "" => "64"
  root_block_device.0.volume_type:           "" => "gp2"
  security_groups.#:                         "" => "2"
  security_groups.14195040:                  "" => "sg-84ac82e1"
  security_groups.4045803087:                "" => "sg-bbac82de"
  user_data:                                 "" => "545c0308b7b339c0a68c4b902777506379e370bb"
aws_launch_configuration.jenkinsmaster-cps-LC: Error: 1 error(s) occurred:

* Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: ec2describe_profile
        status code: 400, request id: [96bbced4-03c1-11e5-99a5-b17026acf417]
aws_launch_configuration.jenkinsmaster-ops-LC: Error: 1 error(s) occurred:

* Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: ec2describe_profile
        status code: 400, request id: [96bf5089-03c1-11e5-a281-51c7f0bf4f0e]
aws_elb.jmaster: Creation complete
...
Error applying plan:

3 error(s) occurred:

* 1 error(s) occurred:

* 1 error(s) occurred:

* Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: ec2describe_profile
        status code: 400, request id: [96292ba9-03c1-11e5-ba98-45c0c04e3ac5]
* 1 error(s) occurred:

* 1 error(s) occurred:

* Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: ec2describe_profile
        status code: 400, request id: [96bbced4-03c1-11e5-99a5-b17026acf417]
* 1 error(s) occurred:

* 1 error(s) occurred:

* Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: ec2describe_profile
        status code: 400, request id: [96bf5089-03c1-11e5-a281-51c7f0bf4f0e]

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

The text was updated successfully, but these errors were encountered:

catsby · 2015-05-29T15:48:21Z

Hey @stefancocora – sorry to hear you ran into this. Can you share the configuration you used to reproduce this? Be sure to omit any secrets, and if possible trim the config down to the bare minimum needed to reproduce.

Thank you!

itsaryuk · 2015-05-29T20:16:59Z

Same issue here.

module.iam.aws_iam_instance_profile.instance-profile: Creation complete
aws_launch_configuration.apache-redirector: Creating...
associate_public_ip_address: "" => "0"
ebs_block_device.#: "" => ""
ebs_optimized: "" => ""
iam_instance_profile: "" => "ec2-apache-redirector-prod"
image_id: "" => "ami-..."
instance_type: "" => "t2.small"
key_name: "" => ".."
name: "" => "apache-redirector-prod"
root_block_device.#: "" => ""
security_groups.#: "" => "1"
security_groups.640532882: "" => "sg-.."
user_data: "" => "4189ff5e1a3c238ba47bc7e91ae429fac7a5daa4"
module.iam.aws_iam_role_policy.base-policy: Creation complete
aws_launch_configuration.apache-redirector: Error: 1 error(s) occurred:

Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: ec2-apache-redirector-prod

stefancocora · 2015-05-30T11:05:48Z

@catsby I will post some TF config to test this in a couple of days when I'm back in the office.

stefancocora · 2015-06-05T15:40:32Z

Since I've reported this bug I've migrated to TF 0.5.3

terraform version
Terraform v0.5.3

I've created a github gist with:

minimum terraform source code to reproduce the issue main.tf
a capture of the first terraform apply that fails with 404

...
Error creating launch configuration: ValidationError: Invalid IamInstanceProfile

a capture of the second terraform apply that finishes the creation of the resources

How to use it:

export your access_key and secret_key credentials
run terraform plan passing it the above credentials
run terraform apply passing it the above credentials

I hope this is enough to allow you to test !
@catsby ^^

mikeyhill · 2015-06-11T14:54:54Z

I've got the same issue here, my workaround is also to re-run Terraform.

mitchellh · 2015-06-25T05:57:04Z

Dup of #2349. I know thats newer, but we have more discussion going on there.

carlsverre · 2016-03-18T21:56:13Z

Not sure which issue to bring this up - but I am hitting this issue in Terraform v0.6.13.

Setting up remote state...
Downloading remote modules...
Refreshing Terraform state prior to plan...
module.medium-runners.aws_iam_role.primary: Creating...
  arn:                "" => "<computed>"
  assume_role_policy: "" => "{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Action\": \"sts:AssumeRole\",\n            \"Principal\": {\"AWS\": \"*\"},\n            \"Effect\": \"Allow\",\n            \"Sid\": \"\"\n        }\n    ]\n}\n"
  name:               "" => "atmos-runner-m"
  path:               "" => "/"
  unique_id:          "" => "<computed>"
module.medium-runners.aws_iam_role.primary: Creation complete
module.medium-runners.aws_iam_role_policy.test_policy: Creating...
  name:   "" => "atmos-runner-m"
  policy: "" => "{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Action\": [\n                \"ec2:CreateSnapshot\",\n                \"ec2:CreateTags\",\n                \"ec2:CreateVolume\",\n                \"ec2:DeleteVolume\",\n                \"ec2:AttachVolume\",\n                \"ec2:DetachVolume\",\n                \"ec2:DescribeAvailabilityZones\",\n                \"ec2:DescribeSnapshots\",\n                \"ec2:DescribeTags\",\n                \"ec2:DescribeVolumeAttribute\",\n                \"ec2:DescribeVolumeStatus\",\n                \"ec2:DescribeVolumes\",\n                \"s3:*\"\n            ],\n            \"Resource\": \"*\",\n            \"Effect\": \"Allow\"\n        }\n    ]\n}\n"
  role:   "" => "atmos-runner-m"
module.medium-runners.aws_iam_instance_profile.primary: Creating...
  arn:              "" => "<computed>"
  create_date:      "" => "<computed>"
  name:             "" => "atmos-runner-m"
  path:             "" => "/"
  roles.#:          "" => "1"
  roles.2907203487: "" => "atmos-runner-m"
  unique_id:        "" => "<computed>"
module.medium-runners.aws_iam_role_policy.test_policy: Creation complete
module.medium-runners.aws_iam_instance_profile.primary: Creation complete
module.medium-runners.aws_launch_configuration.primary: Creating...
  associate_public_ip_address:               "" => "1"
  ebs_block_device.#:                        "" => "<computed>"
  ebs_optimized:                             "" => "1"
  enable_monitoring:                         "" => "1"
  iam_instance_profile:                      "" => "atmos-runner-m"
  image_id:                                  "" => "OMITTED"
  instance_type:                             "" => "c4.2xlarge"
  key_name:                                  "" => "atmos"
  name:                                      "" => "<computed>"
  name_prefix:                               "" => "atmos-runner-m-"
  root_block_device.#:                       "" => "1"
  root_block_device.0.delete_on_termination: "" => "1"
  root_block_device.0.iops:                  "" => "<computed>"
  root_block_device.0.volume_size:           "" => "256"
  root_block_device.0.volume_type:           "" => "gp2"
  security_groups.#:                         "" => "1"
  security_groups.985038555:                 "" => "OMITTED"
  user_data:                                 "" => "11ba1e67ac6aee845bc06d67d20ded6b02949c23"
Error applying plan:

1 error(s) occurred:

* aws_launch_configuration.primary: Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: atmos-runner-m
    status code: 400, request id: 480bf5c7-ed53-11e5-9130-09787fb57398

sstarcher · 2016-03-22T11:44:41Z

After upgrading from 0.6.12 to 0.6.13 this issue appears and I have to downgrade.

* aws_launch_configuration.coreoscluster-launch_config: Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: coreos-influx
    status code: 400, request id: f8a7833e-f022-11e5-8e8e-7ff6b962d555

justinhennessy · 2016-03-23T01:03:10Z

Yep, I can confirm this, made me a sad panda, I have rolled back to 0.6.11 and the issue has gone away.

Have also confirmed 0.6.14 has the same issue. We are using it on Mac installed via brew if that is of any use.

alexander-balashov · 2016-03-23T09:02:51Z

I faced with same problem after updating from v0.6.11 to v0.6.14.

* aws_launch_configuration.config: Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: icm-238-cluster-instance-profile
    status code: 400, request id: 01479765-f0d4-11e5-b7f1-ade14b5d9466

part of config:

/* Create instance profile */
resource "aws_iam_instance_profile" "profile" {
    name = "${var.cluster_name}-cluster-instance-profile"
    roles = ["${aws_iam_role.role.name}"]
}

/* Create launch configuration */
resource "aws_launch_configuration" "config" {
    name = "${var.cluster_name}-cluster-launch-configuration"
    image_id = "${var.ami}"
    instance_type = "${var.instance_type}"
    iam_instance_profile = "${aws_iam_instance_profile.profile.name}"
    key_name = "${var.key_name}"
    security_groups = ["${var.security_group}"]
    user_data = <<EOF
#!/bin/bash
echo ECS_CLUSTER="${aws_ecs_cluster.cluster.name}" >> /etc/ecs/ecs.config
EOF
}

log list after executed terraform apply

aws_iam_instance_profile.profile: Creation complete
aws_launch_configuration.config: Creating...
  associate_public_ip_address: "" => "0"
  ebs_block_device.#:          "" => "<computed>"
  ebs_optimized:               "" => "<computed>"
  enable_monitoring:           "" => "1"
  iam_instance_profile:        "" => "icm-238-cluster-instance-profile"
  image_id:                    "" => "ami-dc1ffbbc"
  instance_type:               "" => "c3.xlarge"
  key_name:                    "" => "microservices-common-us-og-v2"
  name:                        "" => "icm-238-cluster-launch-configuration"
  root_block_device.#:         "" => "<computed>"
  security_groups.#:           "" => "1"
  security_groups.1648137709:  "" => "microservices-common-sec-group-us-og"
  user_data:                   "" => "49831a139e565ac42b2f326e05165294d769367a

Error applying plan:

1 error(s) occurred:

* aws_launch_configuration.config: Error creating launch configuration: ValidationError: Invalid IamInstanceProfile: icm-238-cluster-instance-profile
    status code: 400, request id: 01479765-f0d4-11e5-b7f1-ade14b5d9466

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Error disappears when second execute terraform apply

phinze · 2016-03-25T17:55:06Z

Hey folks, tracking the new bug and the incoming fix in #5862

ghost · 2020-04-27T01:57:52Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

catsby added bug waiting-response An issue/pull request is waiting for a response from the community provider/aws labels May 29, 2015

stefancocora closed this as completed Jun 5, 2015

stefancocora reopened this Jun 5, 2015

zollie mentioned this issue Jun 24, 2015

Add retry to aws_launch_configuration Create to wait for IAM instance profile propagation #2452

Merged

mitchellh closed this as completed Jun 25, 2015

mitchellh mentioned this issue Jun 25, 2015

aws_launch_configuration does not wait for aws_iam_instance_profile #2349

Closed

radeksimko mentioned this issue Dec 27, 2015

helper: Add ContinuousTargetOccurence to work around inconsistency #4447

Merged

vikas027 mentioned this issue Feb 20, 2017

AWS ECR Policy - need to run terraform apply twice #12108

Closed

ghost locked and limited conversation to collaborators Apr 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

provider/aws: iam_instance_profile not ready when TF creates the LaunchConfig of an ASG #2136

provider/aws: iam_instance_profile not ready when TF creates the LaunchConfig of an ASG #2136

stefancocora commented May 29, 2015

catsby commented May 29, 2015

itsaryuk commented May 29, 2015

stefancocora commented May 30, 2015

stefancocora commented Jun 5, 2015

mikeyhill commented Jun 11, 2015

mitchellh commented Jun 25, 2015

carlsverre commented Mar 18, 2016

sstarcher commented Mar 22, 2016

justinhennessy commented Mar 23, 2016

alexander-balashov commented Mar 23, 2016

phinze commented Mar 25, 2016

ghost commented Apr 27, 2020

provider/aws: iam_instance_profile not ready when TF creates the LaunchConfig of an ASG #2136

provider/aws: iam_instance_profile not ready when TF creates the LaunchConfig of an ASG #2136

Comments

stefancocora commented May 29, 2015

catsby commented May 29, 2015

itsaryuk commented May 29, 2015

stefancocora commented May 30, 2015

stefancocora commented Jun 5, 2015

mikeyhill commented Jun 11, 2015

mitchellh commented Jun 25, 2015

carlsverre commented Mar 18, 2016

sstarcher commented Mar 22, 2016

justinhennessy commented Mar 23, 2016

alexander-balashov commented Mar 23, 2016

phinze commented Mar 25, 2016

ghost commented Apr 27, 2020