Throwing Errors when parsing nested brackets in HCL

[Shubhammathur22]

• terrascan version: v0.2.0
• Python version: Python 3.6.9
• Operating System: Ubuntu 18.04.1 LTS (Bionic Beaver)

Description

I was trying to scan one of my terraform directory. It contains a line where nested brackets exists, its giving errors there.

What I Did

I ran the terrascan on below code

locals {
  map = {
    r1         = "21.0.0.0/16"
    r2         = "22.4.0.0/16"
        }

  sg = {
  "test" = [
    "r1",
    "r2"
  ]
}

}

resource "aws_security_group" "test" {
  name        = "testshubham"
  vpc_id      = "vpc-xxxxxxxxx"

  dynamic "ingress" {
    for_each = local.sg.test
    content {
      from_port = 80
      to_port   = 80
      protocol  = "tcp"
      cidr_blocks = split(",", lookup(local.map, ingress.value, ingress.value))
    }
  }
}

Error (Truncated)

Processed on 05/06/2020 at 11:49
Results (took 0.92 seconds):

Failures: (0)

Errors: (1)
[high] Traceback (most recent call last):
  File "/home/shubham/myterrascan/terrascan/terrascan/embedded/terraform_validate/terraform_validate.py", line 573, in readDir
    self.loadFileByDir(fileName, relativeFileName, d, d, terraform_string)
  File "/home/shubham/myterrascan/terrascan/terrascan/embedded/terraform_validate/terraform_validate.py", line 586, in loadFileByDir
    hclSubDirDict[path] = hcl.loads(terraform_string)
  File "/home/shubham/.local/lib/python3.6/site-packages/hcl/api.py", line 81, in loads
    return HclParser().parse(s, export_comments=export_comments)
  File "/home/shubham/.local/lib/python3.6/site-packages/hcl/parser.py", line 643, in parse
    s, lexer=Lexer(export_comments=export_comments), debug=DEBUG
  File "/home/shubham/.local/lib/python3.6/site-packages/hcl/ply/yacc.py", line 503, in parse
    tok = self.errorfunc(errtoken)
  File "/home/shubham/.local/lib/python3.6/site-packages/hcl/parser.py", line 634, in p_error
    raise ValueError(msg)
ValueError: Line 26, column 407: unexpected LEFTPAREN; expected ASTERISK_PERIOD, RIGHTBRACKET, COMMA, RIGHTPAREN, ADD, MINUS, MULTIPLY, DIVIDE

[Shubhammathur22]

@cesar-rodriguez

[cesar-rodriguez]

@Shubhammathur22 Thanks for opening this. It seems to be an issue with pyhcl which is used by terrascan to parse hcl files. I opened an issue on that repository virtuald/pyhcl#74.

$ cat test.tf 
locals {
  map = {
    r1 = "21.0.0.0/16"
    r2 = "22.4.0.0/16"
  }

  sg = {
    "test" = [
      "r1",
      "r2"
    ]
  }

}

resource "aws_security_group" "test" {
  name   = "testshubham"
  vpc_id = "vpc-xxxxxxxxx"

  dynamic "ingress" {
    for_each = local.sg.test
    content {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = split(",", lookup(local.map, ingress.value, ingress.value))
    }
  }
}

$ python
Python 3.7.5 (default, Jan  1 2020, 15:46:16) 
[Clang 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hcl
>>> with open('test.tf', 'r') as fp:
...   obj = hcl.load(fp)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/therasec/.pyenv/versions/terrascan/lib/python3.7/site-packages/hcl/api.py", line 62, in load
    return loads(fp.read(), export_comments=export_comments)
  File "/Users/therasec/.pyenv/versions/terrascan/lib/python3.7/site-packages/hcl/api.py", line 81, in loads
    return HclParser().parse(s, export_comments=export_comments)
  File "/Users/therasec/.pyenv/versions/terrascan/lib/python3.7/site-packages/hcl/parser.py", line 643, in parse
    s, lexer=Lexer(export_comments=export_comments), debug=DEBUG
  File "/Users/therasec/.pyenv/versions/terrascan/lib/python3.7/site-packages/hcl/ply/yacc.py", line 503, in parse
    tok = self.errorfunc(errtoken)
  File "/Users/therasec/.pyenv/versions/terrascan/lib/python3.7/site-packages/hcl/parser.py", line 634, in p_error
    raise ValueError(msg)
ValueError: Line 26, column 397: unexpected LEFTPAREN; expected ASTERISK_PERIOD, RIGHTBRACKET, COMMA, RIGHTPAREN, ADD, MINUS, MULTIPLY, DIVIDE

[jonesy1234]

@cesar-rodriguez - From the update to virtuald/pyhcl#74 this is not going to be supported. Your linter now seems to have been included in the GitHub/Superliter which is great but the lack of support for HCL2 is a real issue moving forward. Do you have a way forward?

Quote from pyhcl repo readme :-

pyhcl does not support HCL2 (which is what modern terraform uses). You might try https://pypi.org/project/python-hcl2/ instead (though I've never personally tried it).

[cesar-rodriguez]

Yes. We're actively working on a new release that addresses this issue, among others. Should be out in a few weeks. I'll post and update here and will close this issue once it's out.

[3h4x]

superlinter brought me here. Thanks @cesar-rodriguez waiting for new release with fix.

[cesar-rodriguez]

This is fixed with v1.0.0. The version included in super-linter still needs to be updated.