text templating nested maps

[hansbogert]

I'm trying to pretty print a nested map something like this:

taints: {
    workload: {
        key: "workload"
        value: "data"
        effect: "NoSchedule"
    }
}

However with the current text/template, I think I don't have enough mechanisms to know when to make a recursive call because I can't check if a variable is of type map.

How would you implement a pretty printer for a nested map of unknown depth?
For context, I'm trying to pretty print to Terraform's HCL2 syntax. So maybe there's a better approach in the first place

[infogulch]

I have some Y answers to your X question, if you'll forgive me.

Are you sure you need to export to HCL? All the Hashicorp products accept json in addition to hcl files. You could export to json with encoding/json and then use json directly.

Another alternative, you could export to json and then use a json to hcl2 tool. Here are a few I found of unknown quality:

• https://developer.hashicorp.com/packer/tutorials/configuration-language/hcl2-upgrade
• https://github.com/kvz/json2hcl
• https://www.convertsimple.com/convert-json-to-hcl/

[hansbogert]

well, I wanted it to be contained to just Cue, but perhaps that's simply a bridge too far. On the other hand, if we want to be the lingua franca of config, we should be able to do this. But perhaps terraform's HCL should be an Cue encoding like openapi and json-schema.

hashicorp's usage of json is quite an afterthought and would be a pain to read for my colleagues.

Further, quality is indeed an issue with the tooling provided, lists of dicts isn't possible what that 3rd one for example. But I'll guess that I can use the first one for now.

[infogulch]

Reading between the lines I'm guessing your goal is to:

1. Write and manage config for hashicorp products in CUE (which products? packer/terraform/nomad?)
2. Render CUE to HCL2 for read-only review by users who are accustomed to using HCL2 to configure hashicorp product(s)
3. Render CUE to a format to configure and run said products

You may think of 2 and 3 as the same, but here they are split out for clarity.

From what I found people were saying that HCL2 is too underspecified to write a generalized format conversion tool (maybe its representation depends on which plugins are loaded? idk the details), so I'm not sure that such a thing would even be possible. In general JSON is a better choice for api communication between programs. Afaik all of the hashicorp products have first class support for json input.

[hansbogert]

Yeah at least for my day job we're bound to terraform. It's doing itself a disservice with the rather lacking type system, but it serves its purpose. For Cue, I have multiple use-cases, my main purpose so far is refactoring of HCL

1. Take in HCL, especially repeating object definitions
2. map it to Cue
3. use Cue's functionality like trim
4. for now, manually make higher-level Cue structs and its unification using Terraform objects and the merge() function.

So yeah, it's basically all your points .1, .2, .3.

I've used this effectively to get rid of a lot of boilerplate already, including the infernal redeclares of default values. But what puts me off is the incredibly opaque pipeline of sed scripts, hcl2json (yet another tool! not the one you mentioned), and all the small edge cases.

Terraform's typing system is so incredibly poor that I'm starting to see it as merely a render target, however, what's stopping me is a roundtrip conversion from CUE->HCL->CUE->... if we'd have that, than checking a model's validity would be easy.

From what I found people were saying that HCL2 is too underspecified to write a generalized format conversion tool (maybe its representation depends on which plugins are loaded? idk the details), so I'm not sure that such a thing would even be possible. In general JSON is a better choice for api communication between programs. Afaik all of the hashicorp products have first class support for json input.

Hashicorp's use of json is isomorphic to HCL2, it's just syntax, it's not dependent on plugins. It's just rather hard to read.
I do concur that having an encoder in Cue which is purely for HCL2 wouldn't work. The render target would need to be something like hcl2+terraform.

[infogulch]

I used terraform since release and I couldn't agree more with your take on its terrible type system.

It would be handy if you could round-trip hcl2 to json and back. I think this would be a better target than direct to CUE, in part because it would be easier to convince people to support it. That said if you wanted to preserve vars inputs and expressions etc I guess you'd need direct translation to CUE.

This tf contributor agrees with you that a hcl2<->json transformer would have to be terraform-specific: hashicorp/hcl#294 (comment)

I found some hcl to json conversion tools:

• https://github.com/tmccombs/hcl2json This one looks pretty good. (There's an issue about going in reverse.)
• https://github.com/hashicorp/hcl/tree/main/cmd/hcldec This is published by hashicorp but requires an hclspec file (??)

There have been some experiments with driving terraform from CUE. The CUE team uses it to manage github resources via terraform providers, there was a presentation and slides about it last year. I'd recommend if you haven't seen it yet: #2462