Terraform Overrides - Choose which file to override the default variables.tf

[AleksandarTokarev]

I am using Terraform v0.11.11.

I have few override files for each of my environments - examples:
dev_override.tf
stage_override.tf
prod_override.tf

According to the documentation - the override files load in alphabetical order.
Is it possible somehow to choose only one file with a variable or something ?
For example: use dev_override.tf only to override values (and not stage and prod).
Because with the current alphabetical solution - stage is the last one in alphabetical order and this one is loaded always.

Proposal

Would be good terraform to accept argument variables on apply/plan to support this. Or maybe do this based on a variable with a default value.

[bennycornelissen]

Hi @AleksandarTokarev,

out of curiosity: what kinds of things are you configuring in your various override files? Is it mostly about variables or implementation choices that could be reduced to variables? In that case it could be well worth looking in to using Terraform workspaces in conjunction with Locals to configure certain variables depending on the environment you're using (and you get state management for free).

[AleksandarTokarev]

Thanks for the response @bennycornelissen.
My use case is the following:
I am having endpoints for different environments (like DB urls, etc)
So i wanted to have the separate Urls in each override file - and control this using a flag or something (which file to have precedence - to load last).
But i guess i could use a single file with different properties - and use locals to achieve this.

[bennycornelissen]

@AleksandarTokarev I tend to hang out in the issues section out of pure curiosity, especially issues regarding the limitations of Terraform. 😄

But what you're describing can indeed be solved with locals and workspaces, or if you're not using the workspaces functionality, you can still work with .tfvars files and specify which one to use. Or use locals and a single input var to specify the environment.

I've used a Makefile approach in the past, where all the required Terraform flags would be dealt with and you could just run make apply-prd or make plan-tst. It wasn't perfect, but it's fairly easy to implement. It also handled AWS credentials (since 'tst' and 'prd' were on different AWS accounts).

[apparentlymart]

Hi @AleksandarTokarev!

In addition to what's been said already: usually this override files feature is used to combine hand-written base files with machine-generated override files, as a way to keep the hand-written part human-friendly and override only the small part that needs to be generated. As such, the feature is implemented under the expectation that the override files will be generated on the fly before running Terraform, rather than kept under version control alongside the files being overridden. This feature is generally used only for some unusual workflows where Terraform is run in automation.

For the more common situation of just setting variable values a little differently depending on environment, the -var-file argument with .tfvars files is the more common answer, as @bennycornelissen mentioned. In future we are planning to introduce a feature where default variable values can be provided on a per-workspace basis to avoid the need to set them explicitly, but as he noted users commonly handle this today with some sort of wrapper script or makefile to make up for the lack of an automatic feature to customize variables on a per-workspace basis.

If you have more differences between your environments than just variable values, then it may be better to instead have a separate root module per environment and factor out the common parts into shared modules that you can combine in different ways per environment. There is an overview of the current capabilities of the workspaces feature and tradeoffs for its use in When to use Multiple Workspaces.

[AleksandarTokarev]

Ok great, -var-file and .tfvars works fine for my problem :)
Thanks for your help guys!

[apparentlymart]

Thanks for following up, @AleksandarTokarev! I'm glad you found a working solution.

[ryancalhoun]

I came looking here because of #1439.

I want to use a variable in a module source, but can't. I need a different module source per workspace. I thought maybe override files would be an option, but it doesn't sound like it. If I could specify the override file to use, I could switch out the module source per environment in a workspace-specific override file.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.