Transform Modules to Constructs

[skorfmann]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Right now, the support of Terraform Modules is quite limited. Besides from only being available via the Terraform Registry, we parse the metadata only. From that metadata, the type information is derived and a Construct class for that Module interface is created. When synthesized, it ends up being treated as a classical Terraform module where Terraform will fetch the actual module source code.

There are a few drawbacks to this from my point of view:

• Special handing is required when parsing output (e.g. see Prettify module output to match a tree format. #113)
• Resources in the module are hidden from the constructs tree
  • No dynamic changes to these resources (e.g. thinking of tagging aspects)
  • No way of accessing this in unit tests
  • Supportive tooling will likely be limited (e.g. Moving / Renaming a Project Leads to Symlink Error #189)

Proposal

A Terraform module could be fully transformed into a Construct and enable the use-cases described above. A prerequisite for this to happen would be:

• A hcl2cdktf tool
• Full coverage of the HCL syntax in cdktf

[thesurlydev]

@skorfmann Would this enable using a module such as the following?

module "api_gateway_cors_proxy" {
  source = "github.com/kyeotic/terraform-api-gateway-cors-module.git?ref=1.1"
  resource_id = aws_api_gateway_resource.proxy.id
  rest_api_id = aws_api_gateway_rest_api.rest_api.id
}

I'm trying to determine which of the issues here. Thanks.

[jsteinich]

@digitalsanctum I think what you're after is covered by #16.
This is going farther and converting a terraform module into a construct tree. Writing in code would look very similar to using a terraform module in hcl, but the generated terraform json would actually just contain all of the resources that would have been in the module.

[skorfmann]

While working on #522 I stumbled upon this regarding submodules.

Terraform will still extract the entire package to local disk, but will read the module from the subdirectory. As a result, it is safe for a module in a sub-directory of a package to use a local path to another module as long as it is in the same package.

That's something to keep in mind here.

[benkeil]

Could we add in the meantime a Testing.toHaveResourceWithProperties method to test at least the input of a module?

[jsteinich]

Could we add in the meantime a Testing.toHaveResourceWithProperties method to test at least the input of a module?

Unfortunately modules aren't compatible with how the tests work as the module contents aren't part of the cdktf output.

Extracting the information from Terraform would either involve doing everything needed to implement this or some tricky workarounds with reading existing state and planned changes.

[benkeil]

But I have the input of the module in the Synth output. That’s better than nothing.

[jsteinich]

But I have the input of the module in the Synth output. That’s better than nothing.

Ah, I'd probably opt for having a Testing.toHaveModuleWithProperties to clarify that it is checking something different than resources (could also have a Testing.toHaveConstructWithProperties for more general checking). Definitely possible to add something like that.

[benkeil]

Sounds good. I think about to write an extension for kotest