Upgrading to Terraform 0.12: separate configuration file for Terragrunt?

[apparentlymart]

Hi! I'm one of the engineers at HashiCorp who works on Terraform Core.

As you might be aware, we've been working for some time now on various improvements to the Terraform configuration language, to address a number of usability problems that have been reported over the years. The new changes are broadly compatible with existing configurations, and particularly those using the common idioms reflected in the Terraform documentation.

Unfortunately resolving some of these problems required tightening up some things that were previously rather loose. Current and previous versions of Terraform ignore certain things that are considered invalid, often producing surprising results, and so to improve usability the new implementation has some more error checking in order to catch mistakes and produce error messages that are hopefully more actionable.

We're planning to include a migration helper tool in the next release in order to help detect and fix some of these rough edges. The details of what this tool does will be largely informed by real-world usage, and so we're planning to release an early preview of the new implementation as soon as it is in a usable state in the hope that we can gather as many examples as possible of patterns in the wild that the migration helper tool should be looking for.

With all of that context in mind, we know that there are lots of users wrapping Terraform with Terragrunt, and so we want to make sure there's a good upgrade path. I'm opening this issue to discuss a specific problem I've noticed, but I'd like to discuss any other potential hurdles too, in whatever forum makes sense.

---

Terragrunt currently uses an extension of the terraform.tfvars file as its own configuration file, relying on the fact that Terraform itself ignores definitions in that file that are not defined as variables, and then using a special variable name terragrunt whose value is not, by Terraform's rules, valid: it includes mixed-typed nested structures, has nested blocks, etc.

The new implementation has a few differences that will cause friction here if we proceed naively:

• Terraform will consider the mixed-type nested block structures as invalid and will generate error messages about them during validation, since they are invalid by Terraform's rules.
• The new configuration parser distinguishes more strongly between object expressions and nested blocks, in order to resolve various confusing behaviors that resulted from the ambiguity of that in the original HCL design.
• The expression evaluator and the structural language were previously entirely separate (HIL vs. HCL), but they are combined in the new implementation, which means that interpolation sequences ${ are interpreted in all contexts, with the difference now being whether variables and functions are available, rather than whether interpolations are possible at all.

Looking at the "live infrastructure" example, a naive syntax-level pass of migration would produce the following rewritten version:

# Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules,
# remote state, and locking: https://github.com/gruntwork-io/terragrunt
terragrunt = {
  # Configure Terragrunt to automatically store tfstate files in an S3 bucket
  remote_state = {
    backend = "s3"
    config = {
      encrypt        = true
      bucket         = "terragrunt-example-prod-terraform-state"
      key            = "$${path_relative_to_include()}/terraform.tfstate"
      region         = "us-west-2"
      dynamodb_table = "terraform-locks"
    }
  }
  # Configure root level variables that all resources can inherit
  terraform = {
    extra_arguments = {
      # (this part will need to do some heuristic processing to decide whether to use
      # direct objects or lists of objects, mimicking the behavior of the current hcl.Decode
      # into an interface{}, so the details may not exactly match this in practice.)
      bucket = {
        commands = ["$${get_terraform_commands_that_need_vars()}"]
        optional_var_files = [
          "$${get_tfvars_dir()}/$${find_in_parent_folders("account.tfvars", "ignore")}"
        ]
      }
    }
  }
}

Since Terragrunt is using the HCL parser directly to process its configuration file today, it has the freedom to interpret the tree of HCL AST objects inside here as desired, and so I expect there are expectations here that are different than how Terraform generally deals with HCL structures it has no specific schema for. Also, due to the fact that interpolation syntax is now accepted in all quoted strings (variables and functions are just not available in some contexts), the naive migration tool will escape the ${ sequences under the assumption that the user intended them to be treated literally, since that was Terraform's own interpretation of a .tfvars file before this change.

Therefore it feels to me like the most practical path to take here is for our migration tool to have a special case for a variable called terragrunt (possibly activated only if there is no variable "terragrunt" block in configuration) which can then have some light understanding of Terragrunt's configuration structure and make some allowances for it so that we can avoid unintentional breakage of the Terragrunt configuration.

Terragrunt is currently relying on syntax features that are considered invalid in the new implementation, so leaving the configuration untouched by the migration tool would inevitably lead to syntax errors during parsing.

---

In order to avoid conflicts between Terraform's and Terragrunt's interpretation of the same file, I'd like to propose having Terragrunt look for a Terragrunt-specific configuration filename (e.g. terragrunt.conf) in preference to terraform.tfvars, and then Terraform would entirely ignore this file and leave Terragrunt's own parser to deal with it. Terragrunt could continue to use the original implementation of HCL, avoiding the need to deal with any of the new processing rules from the new implementation.

If we were to go down this path, we could build the configuration migration tool so that its special case for the terragrunt variable name is to migrate the contents out into a terragrunt.conf file that retains the block structure and original-HCL syntax that Terragrunt is expecting, and then deletes this value from the terraform.tfvars file altogether so that Terraform itself won't try to parse it.

I understand that this is a pretty drastic shift in the design and usage of Terragrunt, so I wanted to start talking about this early so we have some time to think through the implications and any other alternative approaches. I hope that together we can find a reasonable path forward!

[brikis98]

Hi @apparentlymart! First, a big thank you for checking in to make sure there's a migration path for Terragrunt! 🍺

Second, could you provide more details on what HCL and Terraform changes are coming? Terragrunt exists as a quick way to fill in gaps in Terraform, but as Terraform fills in those gaps itself, we're only too happy to reduce the need for Terragrunt, and hopefully have it go away entirely in the future.

Once that's clear, we can think about whether changing the config format or moving to a different config file makes more sense.

[apparentlymart]

Hi @brikis98!

We will have a comprehensive summary of all of the planned configuration changes coming soon. This is in progress but is waiting for the early preview release to be more complete before we finalize it so we can ensure we're describing its true scope/behavior. I wanted to get this conversation started early in case some adjustments need to be made either to Terraform itself (prior to the final release) or to Terragrunt.

However, I can say that the focus of this release is on configuration changes rather than workflow changes, and so I believe all of Terragrunt's use-cases continue to make sense for the moment. Once the configuration language is in good shape we are planning to divert our attention to workflow for a while, at which point we'll definitely be interested in discussing if some or all of Terragrunt's features make sense for incorporation into Terraform. There'll be at least one more major release cycle before any changes that might reduce the scope of Terragrunt, and so I think it's worthwhile to ensure there's a viable path forward .

If you'd rather pause this conversation until we have the more complete digest of changes ready, I'm happy to do so. I know it's hard to discuss a hypothetical situation. We're working on getting the new configuration language implementation integrated right now, and once we're at a point where we're back to feature-complete (bugs notwithstanding) I'll be able to run some real Terragrunt configurations through it and get a more concrete sense of how it behaves and whether some more surgical fixes might be possible.

My main motivation with this proposal was to try to minimize the need for tight co-ordination between the Terraform release and any Terragrunt changes, and that was why I lent towards a separate file as a way to leave Terragrunt in control of its own destiny with regards to its configuration parsing, rather than having a single file that needs to be understandable both to Terraform's new parser (based on HCL2) and Terragrunt's existing parser (based on the original HCL implementation).

[brikis98]

However, I can say that the focus of this release is on configuration changes rather than workflow changes, and so I believe all of Terragrunt's use-cases continue to make sense for the moment. Once the configuration language is in good shape we are planning to divert our attention to workflow for a while, at which point we'll definitely be interested in discussing if some or all of Terragrunt's features make sense for incorporation into Terraform. There'll be at least one more major release cycle before any changes that might reduce the scope of Terragrunt, and so I think it's worthwhile to ensure there's a viable path forward .

Understood. Thx for providing the context.

My main motivation with this proposal was to try to minimize the need for tight co-ordination between the Terraform release and any Terragrunt changes, and that was why I lent towards a separate file as a way to leave Terragrunt in control of its own destiny with regards to its configuration parsing, rather than having a single file that needs to be understandable both to Terraform's new parser (based on HCL2) and Terragrunt's existing parser (based on the original HCL implementation).

The main concern I have with a separate file is that it will double the number of files necessary to use Terragrunt. One of the main complaints we have is that there are already too many files/folders to manage, so I'm a bit hesitant to make it worse.

I'm going to take a minute to think out loud and throw out some ideas:

1. One of the primary use cases for Terragrunt "remote Terraform configurations", where you can have a single, versioned definition of your Terraform module in one place and deploy that module across environments by defining a separate terraform.tfvars file for each environment (see, Keep your Terraform code DRY). With this use case, Terragrunt checks out your Terraform code into a temp folder, copies the terraform.tfvars file into that temp folder, and then runs terraform. With this use case, it would be easy for Terragrunt to remove the terragrunt-specific code from terraform.tfvars before running terraform. This allows us to do whatever we want in terraform.tfvars and decouples the release.

2. The approach above doesn't solve the use case for people who don't use the remote Terraform configurations feature. We could update Terragrunt to always copy code to a temp folder before running; might be a bit clunky and have side effects.

3. If the HCL2 parser supports parsing of interpolation, that would actually be a great help, as currently, Terragrunt uses the HCL parser to read the general structure of the .tfvars file, but processes interpolations through hacky, sometimes buggy regular expressions 😁

4. On the other hand, while the structural changes to the Terragrunt configuration you posted (mainly adding = in a few places?) all seem fairly minor, the double dollar sign for interpolation is a bit annoying and would not allow Terragrunt to use HCL2 to parse the interpolations.

5. On the topic of interpolations, what will be allowed in terraform.tfvars?

[lorengordon]

The main concern I have with a separate file is that it will double the number of files necessary to use Terragrunt. One of the main complaints we have is that there are already too many files/folders to manage, so I'm a bit hesitant to make it worse.

fwiw, I know I'd have no problem using a separate config file. I rather like the clarity of separating the two more cleanly. Personally, I don't use terraform.tfvars with terragrunt to also define terraform variables anyway, since I like maintaining the separation of concerns. Instead, I use a separate file <foo>.auto.tfvars, which terraform reads automatically...

And technically, a separate config file wouldn't necessarily double the number of files, since terraform.tfvars would no longer be required...

How about a poll, maybe just using reactions on the issue, or perhaps publishing a poll through the newsletter?

[brikis98]

And technically, a separate config file wouldn't necessarily double the number of files, since terraform.tfvars would no longer be required...

How so? You still need to configure Terragrunt, so that's one file. I suppose if your module needed no variables, then there wouldn't be a second, but that doesn't seem like a common use case.

[lorengordon]

Pass the vars inside the terragrunt config, using extra_arguments. :)

[brikis98]

Ah, I gotcha. That's something worth considering, I suppose. Terragrunt as it's own, completely separate format for (a) downloading Terraform configurations from somewhere, (b) configuring remote state for them, and (c) running apply with a bunch of arguments passed using -var...

Then we're decoupled entirely from Terraform's .tfvars files. I'll have to think on this some more.

[apparentlymart]

Thanks for sharing that extra context, @brikis98! I didn't fully understand the "remote Terraform configurations" workflow on my first pass, but I think I have a better handle on what's going on there now, and can see why a Terragrunt-specific file would -- without other changes -- lead to needing two separate files, where only one is needed today.

The idea of including an extra map of variables inside the Terragrunt file sounds like a potential compromise, though we'd need to think through what the configuration migration tool should do here in order to leave the user with a valid Terragrunt configuration. That is, should it (when a terragrunt definition is present) also move all of the variables from the terraform.tfvars file into the Terragrunt-specific file? Or perhaps it must recognize when the "remote configurations" feature is in use and behave differently in that case?

---

Some additional background on the situation with interpolations in the .tfvars file:

The new "HCL2" implementation has interpolation built into the core language, rather than treating it as a separate processing pass (with the HIL library) on the strings produced by the HCL parser. A consequence of this is that interpolations are parsed during the initial parsing step, and so the AST for them (but not the value) is built much earlier than it would've been under the prior implementation.

An unintended consequence of this is that the .tfvars files, which are also read using the HCL2 parser, can now include expressions and interpolations, since that step is not separable from the structural parsing:

foo = "bar"
bar = "${ 1 + 2 }"
# HCL2 also allows "naked" expressions, without the interpolation markers
baz = 1 + 2

However, Terraform must still evaluate these at a stage where no variables or functions are available, so in practice the interpolations are limited to just basic computations on literal values. I think in practice there's little reason to use expressions in these files, and so users are unlikely to make use of this capability, but it's just a natural consequence of the design of the new parser.

# this will produce a "variables are not allowed here" error message
baz = "${any.variable}"

Because of this, the migration tool will need to conservatively escape any ${ sequences already present in existing files in order to ensure that they are interpreted the same way in the new parser as they would've been in the old parser. (unless we make some sort of exception for the terragrunt variable, of course.)

Another important consequence of the earlier handling of these is that Terraform will evaluate the given expression early in its initialization, when it produces the map of available input variables, since expression evaluation is how attribute values are read in HCL2. This means that a "function calls are not allowed here" message would be produced if given an unescaped Terragrunt configuration with the Terragrunt-specific function calls in it.

[brikis98]

I've been thinking about this a bit more and moving to a separate file, along with some changes to Terragrunts "ergonomics", may make sense.

Here's a rough idea of what a typical "child" terragrunt.hcl may look like:

# Include all the settings from another Terragrunt config
include = "${find_in_parent_folders()}"

# The Terraform code to use. If not specified, it's assumed the code is in pwd.
source = "git::[email protected]:foo/modules.git//app?ref=v0.0.3"

# Variables to pass to the Terraform code
inputs = {
  instance_type  = "t2.micro"
  instance_count = 5

  # A new helper to get output variables from other Terragrunt / Terraform modules
  # This implicitly creates dependencies that will be respected by the apply-all command
  vpc_id = "${get_output("../vpc", "vpc_id")}"

  # A new helper to get input variables from other Terragrunt HCL files to help keep things DRY
  account_id = "${get_input("../account.hcl", "account_id")}" 
}

# Keep your CLI flags DRY
extra_arguments = {
  retry_lock = {
    commands  = ["${get_terraform_commands_that_need_locking()}"]
    arguments = ["-lock-timeout=20m"]
  }
}

# Keep your remote state config DRY
remote_state {
  backend = "s3"
  config {
    bucket = "my-terraform-bucket"
    region = "us-east-1"
    key    = "${path_relative_to_include()}/terraform.tfstate"
  }
}

The main changes are:

1. The config lives in a separate .hcl file. The default name should be terragrunt.hcl.
2. Most of the config settings are now flattened, top-level items to avoid unnecessary nesting.
3. The variables that used to be listed directly in terraform.tfvars now go in a variables = { ... } block.
4. There are new helpers to read variables from other Terragrunt .hcl files, which have their own variables = { ... } blocks, and new helpers to read outputs, by running terraform output or terragrunt output, from other Terraform/Terragrunt modules.

Note that this requires code changes in Terragrunt itself. We don't need all of these for it to be useful, but at the very least, what file it reads, and the structure of that file would need to be available immediately. The new helpers could come later.

I'd love to hear feedback from any Terragrunt users.

Also, @apparentlymart, would we be able to use the HCL2 parser to parse the new format above? Would it parse the "interpolations" and give us back a reasonable AST?

[brikis98]

@apparentlymart What is the rough timeline for HCL2?

[lorengordon]

Seems reasonable to me!

I suppose the new "hooks" feature would be specified something like extra_arguments? But as a list of dictionaries, or an ordered dictionary, since execution order might matter... Not sure I have the syntax right for that in HCL2, but maybe:

after_hooks = [
  "foo" = {
    commands = [...]
    execute = [...]
  }

  "bar" = {
    commands = [...]
    execute = [...]
  }

before_hooks = [
  "abc" = {
    commands = [...]
    execute = [...]
  }

  "bcd" = {
    commands = [...]
    execute = [...]
  }
]

[brikis98]

Ah, right, I left out hooks, and yes, they would look exactly like that.

[apparentlymart]

That looks great to me, @brikis98! The HCL2 parser should be able to accept that format, though I'd make some small adjustments to some details for "idiomatic" HCL2:

# No need to wrap a single expression in interpolation
include = find_in_parent_folders()

source = "git::[email protected]:foo/modules.git//app?ref=v0.0.3"

# Fixed elements (part of the structure of the format) tend to be defined as blocks rather than
# attributes, so no = sign here.
inputs {
  instance_type  = "t2.micro"
  instance_count = 5
  vpc_id         = get_output("../vpc", "vpc_id")
  account_id     = get_input("../account.hcl", "account_id")
}

extra_arguments {
  retry_lock {
    commands  = get_terraform_commands_that_need_locking()
    arguments = ["-lock-timeout=20m"]
  }
}

remote_state {
  backend = "s3"
  config {
    bucket = "my-terraform-bucket"
    region = "us-east-1"
    key    = "${path_relative_to_include()}/terraform.tfstate"
  }
}

# Not exactly sure about these, but their structure suggests something I'd model
# with repeated nested blocks rather than single attributes.
hook_after "foo" {
  commands = [ ... ]
  execute  = [ ... ]
}
hook_before "abc" {
  commands = [ ... ]
  execute  = [ ... ]
}

We're not promising compatibility for the HCL2 package APIs until some time after Terraform is released using it, in case we find design flaws during implementation in Terraform. With that said, we've already completed most of the work in Terraform that interacts with the HCL2 parsing APIs, and so in practice significant changes from here on out are not very likely.

The structure here seems rigid enough that it could probably be processed using the higher-level gohcl package, which uses Go's reflect package to drive decoding via structs. (Terraform's own processing is a little more involved and so it goes to the lower-level API so it can evaluate things gradually during the graph walk.) Since Terraform's needs are the main thing driving HCL2 development right now the gohcl package is not as well documented as it ought to be, but I think its complete enough to be able to process the format you described here, using DecodeBody with a set of structs that represent the intended structure, using functions defined in the EvalContext. Either way, the first step is to parse a file to get a "body" using the parser API.

We are currently in the process of updating Terraform Core to support HCL2's new type system, and so getting to the other end of that is the main blocker for us releasing the initial preview release, and there's still quite a lot of work to do here. How long it'll be until the final release after that depends on what sort of feedback we get from that preview release, but making time for the usual beta and release candidate process we do for any major release means that it'll be another couple months at least before we'll be final here. (The work is driving the schedule rather than the other way around, so unfortunately I can't be more precise at this stage.)

[brikis98]

Thanks for the feedback @apparentlymart. The new syntax looks really nice, especially for simple expressions/interpolations 👍

I think I'm on board with moving to this format. We're a bit overloaded now, so probably won't be able to dig into this for at least a few more weeks, but it sounds like Terraform moving to HCL2 is still a little ways down the road, so I think we have time.

Two more questions for you:

1. You mentioned some sort of migration tool. Are you building something to parse terraform.tfvars and transform it accordingly? If so, it would be very helpful if there was a way to use that tool to detect terraform.tfvars files with a terragrunt = { ... } block and transform them to the new terraform.hcl format automatically.

2. Do the HCL2 libraries support parsing only, or also modifying the AST in memory, and writing it back out? The original HCL library could do pretty-printing of the AST, but not if you made any modifications to it.

[apparentlymart]

The planned migration tool is primarily for updating .tf files, but we were planning to also have it do some simple processing on .tfvars files just to deal with the fact that the HCL2 parser will now interpret ${ sequences in there. We had honestly not planned to do any super-complex processing of .tfvars files, but if there's a reasonably straightforward and well-defined way to map a terragrunt variable assignment onto a terraform.hcl file I'm certainly game to try it in order to smooth the transition here.

The original HCL parser did, as you noted, have a few issues with the generation portion. For HCL2, given the increased complexity of handling both structural and expression parsing in both HCL native and JSON syntaxes, we decided not to build writing support into the main API. Instead, we have a separate package hclwrite which offers an API more tailored for generation and for making surgical changes to existing files. However, this package is not yet fully-functional since again Terraform's needs are driving the priorities for now and the terraform fmt needs can be mostly met by the relatively-simpler Format function. The hclwrite package design and functionality will be finished up in due course, probably driven by some configuration-generation use-cases like hashicorp/terraform#15608.

For the sake of the migration helper tool, we are planning to generate HCL2 syntax output in a more template-ish way, rather than generating it with the hclwrite package, just because the needs there are pretty constrained and it reduces the size of the work required for what will hopefully be a pretty short-lived codepath. Our prototypes so far produced roughly-formatted HCL syntax using templates and then ran the result through Format to get back to idiomatic style.

[Indigenuity]

A question on the inputs block--will all inputs blocks be be merged from all included terragrunt.hcl files? Seems like that would make it so you could reference those values similarly to terraform's locals, like ${input.myvalue} rather than having to specify what file it comes from with ${get_input("my_terragrunt_source", "myvalue")}. Or was get_input intended to access more than just values in the inputs block?

[brikis98]

A question on the inputs block--will all inputs blocks be be merged from all included terragrunt.hcl files?

Good question. Currently, Terragrunt only allows you to include a single file. I suppose inputs would be merged from that one file.

Seems like that would make it so you could reference those values similarly to terraform's locals, like ${input.myvalue}

An interesting idea. I hadn't thought through what the syntax would be likely to reference inputs defined in the current file or imported from the parent. We could use get_input("myvalue") to indicate the current file or input.myvalue.

[Indigenuity]

Oh I was under the impression you could have more than a single layer of includes. Is that not what this test does?
https://github.com/gruntwork-io/terragrunt/tree/master/test/fixture-parent-folders/multiple-terragrunt-in-parents

[ebarault]

just as @lorengordon explained, i also separate the concerns by making sure any variable required by my modules are placed in separate files pulled by terragrunt using extra_arguments as in

    extra_arguments "conditional_vars" {
      commands = ["${get_terraform_commands_that_need_vars()}"]

      required_var_files = [
        "${get_tfvars_dir()}/${find_in_parent_folders("global.tfvars")}",
        "${get_tfvars_dir()}/${find_in_parent_folders("env.tfvars")}"
      ]

      optional_var_files = [
        "${get_tfvars_dir()}/component.tfvars"
      ]
    }

Having all the variables in a terragrunt/terraform-logic-free separate file enables a clear separation between the modules code and the variables and eases their use by ops users on a day-to-day basis (read: I build modules and set up overall solutions <--> end-users fine-tunes the variables)

So i won't use the new input block much.

Reading the lines in this thread, i expect this won't be impacted much by the proposed changes so I have no concerns to use a separate terragrunt config file.

---

Now, regarding the new helper get_output(), I wonder: could we make this interpolation function work inside the .tfvars files used to store the modules variables only?

I guess the separation of concerns postulate would dictate along these lines not to interfere with terraform files anymore, then it would be more a question of an additional feature to add to terraform's core, and the proposed terragrunt implementation should be seen as an interim solution until then.

Thoughts?

[lorengordon]

@ebarault I'd love to see some interpolation in .tfvars files. Looking up ENV values, and many of the "data" resources would be killer. I'd especially like to be able to retrieve values from the SSM Parameter Store. It's quite a pain to expose these implementation-specific "value lookup" constructs in a module (especially if you publish it publicly). Getting this in terraform core would be awesome! 👍 👍 👍

[ebarault]

as of now, i use terragrunt hooks to interpolate "remote" vars in .tfvars files: more specifically all vars nested in more or less complex data structures than cannot be easily used with remote data sources and current terraform's HCL implementation.

I'm yet not quite sure how much terraform's next major release will break this.

[jbergknoff]

Has there been any progress on this? It sounds like TF 0.12 will be released within the next month or so. It would be great to have a migration path for continuing to use Terragrunt with it.

[brikis98]

No progress yet.

[apparentlymart]

Terraform 0.12 will go through an alpha release, at least one beta release, and at least one release candidate before final, which'll add up to at least a month and a half of release process, so please don't worry about it just surprisingly appearing one day!

The terragrunt-specific configuration migration mechanism we discussed in this thread is unlikely to make it in before the alpha, because the focus of the alpha is in getting early feedback from maintainers of re-usable modules we don't have access to (because they are in private repositories), but the Terraform side of this is definitely still on our radar and likely to go in during a beta, assuming what we discussed above still seems approximately right (we can make small tweaks before final, of course).

[lorengordon]

Looks like there is a new, and pretty stable beta release of tf 0.12. Probably at a good place to start testing whether any changes might be needed to terragrunt itself.

[ekini]

While we are here, would it be possible to allow to specify path to terraform binary in terragrunt.hcl to ease migration? So that terraform version can be set on per-module basis, or globally for a project. Should fix #265 as well.

[brikis98]

Update: I've submitted a PR that adds Terraform 0.12 support. I think it's good to go, but a few tests are failing, possibly due to a bug in Terraform 0.12. Follow the PR for details.

[brikis98]

Alright, #731 is merged. Please give https://github.com/gruntwork-io/terragrunt/releases/tag/v0.19.0 a try (the new binaries should show up in a few minutes)!

[ozbillwang]

@brikis98

Thanks to make terragrunt ready for terraform 0.12

But I got error:

/apps # terragrunt --version
terragrunt version v0.19.0

/apps # terraform --version
Terraform v0.12.1

/apps # terragrunt init
[terragrunt] [/apps] 2019/06/11 06:20:44 Running command: terraform --version
[terragrunt] 2019/06/11 06:20:44 Reading Terragrunt config file at /apps/terragrunt.hcl
[terragrunt] 2019/06/11 06:20:44 Error reading file at path /apps/terragrunt.hcl: open /apps/terragrunt.hcl: no such file or directory
[terragrunt] 2019/06/11 06:20:44 Unable to determine underlying exit code, so Terragrunt will exit with error code 1
/apps # 

Update 1

Seems the README and migration guide have been updated:

https://github.com/gruntwork-io/terragrunt#quick-start

https://github.com/gruntwork-io/terragrunt/blob/master/_docs/migration_guides/upgrading_to_terragrunt_0.19.x.md

We need prepare a new terragrunt.hcl file now. Let me have a try

Update 2

A real sample to deploy terraform 0.12 with terragrunt 0.19

https://github.com/gruntwork-io/terragrunt-infrastructure-live-example

Now it makes sense for me.

Really appreciated.

[brikis98]

@ozbillwang Glad to hear you got it sorted!

[ozbillwang]

@brikis98

And I can confirm now, I run on a complex terraform stack with 7 terraform modules, the part to do 0.12upgrade in terraform takes time to make it work, but the part to run terragrunt after 0.12upgrade, it works perfect.

[kamsz]

@brikis98 While declaring an empty remote state in a way:

terraform {
  # The configuration for this backend will be filled in by Terragrunt
  backend "gcs" {}
}

I'm getting:

Error: Missing key/value separator

  on terraform.tfvars line 2:
   1:
   2:   terraform {

Expected an equals sign ("=") to mark the beginning of the attribute value.

Content of directory terragrunt.hcl:

terraform {
  source = "."

  extra_arguments "vars" {
    commands = [
      "apply",
      "plan",
      "import",
      "push",
      "refresh",
      "destroy",
      "validate",
    ]

    arguments = [
      "-var-file=${get_terragrunt_dir()}/../tfvars/terraform.tfvars",
    ]
  }
}

include {
  path = "${find_in_parent_folders()}"
}

Content of root terragrunt.hcl:

remote_state {
  backend = "gcs"
  config = {
    bucket = "terraform-state"
    prefix = "${path_relative_to_include()}/terraform.tfstate"
  }
}

terragrunt version v0.19.2
Terraform v0.12.2
+ provider.google v2.6.0

Edit: So apparently it tries to parse terragrunt.hcl too, that's why there's an error. After commenting out terraform block from terragrunt.hcl it works.

[brikis98]

on terraform.tfvars line 2:

You still have a terraform.tfvars file hanging around?

[kamsz]

The only file that's called terraform.tfvars is included here:

"-var-file=${get_terragrunt_dir()}/../tfvars/terraform.tfvars",

[brikis98]

And does it have a terraform { ... } block starting on line 2?

[kamsz]

No. For some weird reason the error is related to terragrunt.hcl file, but it's showing terraform.tfvars. I've managed to overcome this by moving all *.tf files to subdirectory and point to it in terragrunt.hcl.

[brikis98]

That error is from Terraform itself, not Terragrunt...

Are you passing terragrunt.hcl as a var file to Terraform somewhere? Check all your extra_arguments declarations.

[kamsz]

Nevermind, I had leftover environment variables set... Excuse my stupidity.

[brikis98]

No prob! Thx for following up.

[outofcoffee2]

Hi @brikis98, I cant find any guide on below link... please help

https://github.com/gruntwork-io/terragrunt/blob/master/_docs/migration_guides/upgrading_to_terragrunt_0.19.x.md

[yorinasub17]

@shivamheller Try https://github.com/gruntwork-io/terragrunt/blob/v0.19.0/_docs/migration_guides/upgrading_to_terragrunt_0.19.x.md

The docs have been revamped on master, so the version pinned link will work better.