Can this be used as a library for rendering TF JSON?

[ebrusseau]

Hello, this project is great! I have a use case however where I need to write custom code around the generation of TF JSON, and would rather use as a library than calling the Pretf binary.

It doesn't seem obvious how this could be done; as everything seems to be pretty tightly coupled around the workflow etc. Seems like it could work by building collections/blocks and then passing them to some render() function, but I couldn't figure out how to achieve it.

Is it possible to do this?

[ndarwincorn]

Hey @ebrusseau I was just looking at doing the same. I'll update when I finish but have you tried importing pretf.render's Renderer class and more or less copying the function in create_files?

i.e.

main.py

...
from pretf.render import Renderer
...
[the 29 lines I linked above]

main.tf.py

from pretf.api import block
[your pretf Terraform blocks]

[raymondbutcher]

Thanks! Yes, this is fairly tightly coupled with files and workflows, I think mainly to support Terraform variables. I never really thought about using this as an API before.

Firstly, have you considered running pretf validate just to generate and validate files? You can set up a pretf.workflow.py file that generates files and then doesn't delete them afterwards.

If you're still keen on using it like an API, then I think the collections feature might be the best option, as you suggested. This way it won't involve any files and won't try to read variables (from *.tf files on disk, environment variables, and CLI args) which it normally does.

Something like this might work for you. This is importing undocumented functions, and I don't really like it, so we might want want to improve it more and include it in Pretf before you rely on it for serious work. Still, have a look/try and let me know what you think.

import json
from collections.abc import Iterable
from typing import Union

from pretf.api import block
from pretf.collections import collect
from pretf.render import Block, json_default, unwrap_yielded


@collect
def security_group(var):

    # Inputs.
    yield block("variable", "name", {})
    yield block("variable", "type", {})
    yield block("variable", "protocol", {})
    yield block("variable", "cidrs", {"default": []})
    yield block("variable", "ports", {"default": []})

    # Group resource.
    group = yield block("resource", "aws_security_group", var.name, {
        "name": var.name,
    })

    # Rule resources.
    for cidr in sorted(set(var.cidrs)):
        cidr_label = cidr.replace(".", "_").replace("/", "_")
        for port in var.ports:
            rule_label = f"{var.name}_{port}_from_{cidr_label}"
            yield block("resource", "aws_security_group_rule", rule_label, {
                "security_group_id": group.id,
                "type": "ingress",
                "protocol": var.protocol,
                "from_port": port,
                "to_port": port,
                "cidr_blocks": [cidr],
            })

    # Outputs.
    yield block(f"output", "group", {"value": group})


def json_dumps(contents: Union[Block, dict, Iterable]):
    # Contents can be a block, dict, collection, or sequence of any of them.
    # The following will flatten it into a sequence of blocks or dicts.
    blocks = tuple(unwrap_yielded(sg))
    # Now dump it using json_default which converts blocks into dicts,
    # and handles interpolations.
    return json.dumps(blocks, indent=2, default=json_default)


if __name__ == "__main__":

    sg = security_group(
        name="web",
        type="ingress",
        cidrs=["10.0.0.0/24", "192.168.0.0/24"],
        protocol="tcp",
        ports=[80, 443],
    )

    print(json_dumps(sg))