Bug: leading whitespace causes aws_iam_policy to incorrectly report valid JSON policies as invalid

[WintersMichael]

Terraform Version

0.10.7, 0.9.11

Affected Resource(s)

• aws_iam_role
• aws_iam_policy

Terraform Configuration Files

resource "aws_iam_policy" "nodes_sqs_policy" {
    name        = "nodes_sqs_policy"
    description = "nodes SQS"
    policy = <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "sqs:GetQueueAttributes"
          ],
          "Resource": [
            "arn:aws:sqs:us-east-1:123123123:myapp-dev-us-east-1*"
          ]
        }
      ]
    }
EOF
}

Expected Behavior

The policy was applied

Actual Behavior

1 error(s) occurred:

* aws_iam_policy.nodes_sqs_policy: "policy" contains an invalid JSON policy

Important Factoids

According to RFC 4627, "Insignificant whitespace is allowed before or after any of the six structural characters."

Removing the whitespace before the first character in the policy allows it to be applied:

data "template_file" "nodes_iam_sqs" {
    name        = "nodes_sqs_policy"
    description = "nodes SQS"
    policy = <<EOF
{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "sqs:GetQueueAttributes"
          ],
          "Resource": [
            "arn:aws:sqs:us-east-1:123123123:myapp-dev-us-east-1*"
          ]
        }
      ]
    }
EOF
}

References

Terraform #11906 is where the JSON validation was applied.

[xocasdashdash]

Adding this here as docs but this can cause bugs on resources that depend on this policy and the warning is extremely disconcerting.

The example would be if you have an aws_iam_role_policy_attachment depend on your policy it will tell you that the policy does not exist.

[nodesocket]

I came across this today as well. This is a bug right?

    policy = <<CONFIG
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject"
                ],
                "Resource": [
                    "arn:aws:s3:::our-org-secrets",
                    "arn:aws:s3:::our-org-secrets/*"
                ]
            }
        ]
    }
    CONFIG

"policy" contains an invalid JSON policy

[mplanchard]

Also affects terraform 0.11.4, aws provider 1.13.0

[lakshmigk01]

+1 I encountered this as well

[woodcockjosh]

#trim

[larroy]

+1

[nwipfli]

+1 Same issue for me.

[whatisaphone]

[WintersMichael]

I finally had a minute to write #5887 but I don't currently have an environment I can run acceptance tests in. If someone can pull my branch, run make testacc TEST=./aws TESTARGS='-run=TestAccAWSLaunchTemplate_', and post results in the PR thread, that might help get this merged.

[WintersMichael]

I found a donor account, test results are added.

[mgasner]

👍

[goetzc]

As a workaround on using the ugly JSON inline Heredoc, the aws_iam_policy_document data source works great, HCL to JSON transformer.

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[goetzc]

Can be in a stale state, but it's still a bug 😄

[himanshubora12]

OMG!! Just a small whitespace in starting is stopping!!! harsh

[yogeshbodke]

jsonencode() solved my issue

policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:BatchCheckLayerAvailability",
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetAuthorizationToken"
            ],
            "Resource": "*"
        }
    ]
  })

[anirudhdggl]

Even with jsonencode, it's giving me MalformedPolicyDocument: JSON strings must not have leading spaces

[devopsrick]

FWIW I have switched almost exclusively to the aws_iam_policy_document data resources for these policies. JSON blocks cause more trouble than they are worth.

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document#basic-example

Here is a very hand utility you can use to generate them from the json policy themselves in more or less one line with echo.

https://github.com/flosell/iam-policy-json-to-terraform

[jbasement]

Any updates on this?

[srinisakh]

Indented heredoc should work here: instead of <<CONFIG use <<-CONFIG

ref: https://developer.hashicorp.com/terraform/language/expressions/strings#indented-heredocs

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.43.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.