"Warning: Value for undeclared variable"

[jmervine]

Current Terraform Version

Terraform v0.12.3

Use-cases

In our current project, we use a common.tfvars file to store shared configurations across multiple modules and plans. Up until recently, this has worked very well in allowing us to dry up configurations to a single location. With recent upgrades we've started seeing Warning: Value for undeclared variable, as not all "common" configurations are used in every plan.

Proposal

I'd like to request that a feature / flag is included to allow for ignoring the "Value for undeclared variable" warning condition, and additionally request that you not make it a breaking error in the future -- as the output indicates is the intention. Being able to include a collection of configurations and have a portion of the ignored is a valuable feature. I do understand that there is also a valid use-case for having it behave in a stricter way to many -- perhaps most. A flag or configuration allowing for opt out of strict checking seems to be an acceptable alternative.

[apparentlymart]

Hi @jmervine,

The warning is there for transition to give you an opportunity to migrate to the new recommended approach, which is to use TF_VAR_ environment variables for this sort of "global" or "ambient" setting. The environment variables are not and will never be subject to this existence check, because we presume that they are being set in the session with the intent of ignoring them when they are not relevant.

Using environment variables in this way allows a distinction between the ones that are set globally and the ones that are set locally, and thus allows us to have the check to report when the local ones are set incorrectly. That's important because otherwise users get no feedback when they accidentally make a typo in a variable name.

Environment variables are, therefore, the "opt-out" that is already available and what you should plan to transition to.

[jmervine]

@apparentlymart,

Thank you for the quick reply.

In my dealings, I've found that TF_VARS_ quickly breaks down on a distributed team or when using CI/CD pipelines to automate interactions. If required, it will be possible to work around this by checking in a .env file and leveraging something like godotenv. That said, the .tfvars file pattern is greatly preferred.

I understand that you have a large audience to consider here, just wanted to weigh in.

Cheers!
Josh

[kayrus]

I also vote for the flag. These warning messages spoil our secrets, which are used in different modules.

[svyotov]

👍

[ekini]

If the only argument for keeping the warning in place is to give feedback to users who mistype, then there might be a solution that suits everyone.

Terraform has access to both declared variables and specified via named files/auto.tfvars and env vars, and therefore to detect a mistype, Damerau–Levenshtein distance can be used. I've done some testing and exploration, and it seems the distance <=2 is sufficient to catch most if not all errors.

This way:

1. The users will get not only a warning that they mistyped, but also suggestions what variables they should use.
2. This will restore the previous functionality and allow keeping common variables source checked. There is no sense in turning terraform to infrastructure-as-environment-variable!

An example:

Warning: Value for undeclared variable

  on /Users/ekini/project_dir/preview/service-app/../env.tfvars line 11:
  11: ecs_service_container_prot   = 8000

The root module does not declare a variable named
"ecs_service_container_prot". Did you mean one of
[ecs_service_container_port]?


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

I have a working prototype, but it seems pointless to make a PR if it's going to be rejected.
@apparentlymart

[jkinred]

Another use case affected is the sourcing of externally generated JSON files and loading them unmodified using -var-file. I discovered this when re-factoring my code away from using jq to set TF_VAR_'s.

A flag to silence these warnings would be great.

[deweysasser]

Similar to jkinred, I organize my terraform into sets of related projects that build on each other to create a full stack (e.g. network, infrastructure, services, application). I use a shared secrets.tfvars file between these projects. Not all projects reference all of the variables, so this "feature" substantially impacts my working pattern.

Creating a mechanism to load these into environment variables would be a poor security practice as well as substantially more complicated.

Terraform needs some mechanism to provide "here is a body of knowledge that can be used during evaluation" -- variable files have been this for a long time. Adding this restriction effectively removes a very useful mechanism and will require either it's replacement with something else, or hacking around what becomes a tool limitation.

[wrsenn]

Another perspective on JSON front: we were hoping to use a single set of JSON files to serve as "set it once" configuration for both our Terraform variables and application configuration (as most of these values are used in both cases) but without being able to conditionally ignore these warnings, that's not really feasible.

[wwentland]

Much like @jmervine we utilise various optional var files to group shared settings for different modules, regions and environments.

Please allow users to suppress these warnings and, in particular, do not make them an error in future releases. Using TF_VAR_'s as suggested by @apparentlymart appears to not be an option and the approach outlined above seems to be the only way to achieve this very common use-case in a way that works for teams and CI/CD pipelines.

[zswanson]

The terrraform documentation itself states:

to set lots of variables, it is more convenient to specify their values in a variable definitions file (with a filename ending in either .tfvars or .tfvars.json) and then specify that file on the command line

as a fallback for the other ways of defining variables, Terraform searches the environment of its own process for environment variables named TF_VAR_

I don't see why an unused variable in a tfvars is outside the very use case you outline in your docs. Also kind of ridiculous when those variables could be complex maps and long lists.

[0xDones]

I have the same problem, but in my case I just want to override a variable in variables.tf file, but looks like it's not working according the documentation.

.
├── backend.tf
├── main.tf
├── modules
│   ├──api
│   │   ├── main.tf
│   │   ├── output.tf
│   │   ├── test.tfvars
│   │   └── variables.tf
├── provider.tf
└── terraform.out

variables.tf

variable "test_var" { }

test.tfvars

test_var = "my-test"

cli

# Terraform Version
terraform --version
# Output
Terraform v0.12.9
+ provider.aws v2.28.1

# Terraform plan
terraform plan -var-file=modules/api/test.tfvars -out terraform.out
# Output
Acquiring state lock. This may take a few moments...

Warning: Value for undeclared variable

  on modules/api/test.tfvars line 1:
   1: test_var = "my-test"

The root module does not declare a variable named "test_var". To use this
value, add a "variable" block to the configuration.

Using a variables file to set an undeclared variable is deprecated and will
become an error in a future release. If you wish to provide certain "global"
settings to all configurations in your organization, use TF_VAR_...
environment variables to set these instead.

[lpakula]

@dpolicastro

variable "test_var" { default="" } is solving the issue

You can then override default value

[MCyprien]

Hi @apparentlymart,

From what I understand the solution Hashicorp is pushing is to use scripts to source instead of tfvars file.

The need for that 'feature' is to spot typo errors.

So I need to change my tfvars file from :

MY_VAR1 = "toto"
MY_VAR2 = "tata"

to a script:

export TF_VAR_MY_VAR1 = "toto"
export TF_VAR_MY_VAR2 = "tata"

Here is the problem
Now imagine, I am making a typo error here:

export TF_VAR_MY_VOR1 = "toto"
export TF_VAR_MY_VIR2 = "tata"

Terraform will not print warnings here because those are environment variables. So the whole point of this feature, to spot typo error, is not fulfilled and is causing constraints and frustrations for your users as you can read in this thread. I also think that a lot of people does not clearly see that will be a problem in futur versions. As you can see in #19424, this was already an issue.

I think good development practices is the key to avoid typo error (pair-prog, review, etc.). It looks to me you are trying to design something idiot-proof (no offense here) and I will just quote Douglas Adams:

"a common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools"

Next time for this kind of breaking changes, can you propose a feature proposal to get feedbacks?

[nicerobot]

I wrote about my objections to this feature in #19424 (comment) but that issue was locked, effectively silencing further discussion so I'm going to raise my objections and supporting arguments again here.

I strongly disagree with the rationale for this warning/error! I've been using terraform for years and this goes against almost everything I've come to consider as best practices.

First, the recommendation to use TF_VAR_ goes against the philosophy of infrastructure as code! TF_VAR_ are easily overlooked when source controlling and not source controlled with .tf nor .tfvars files. They should always be discouraged or at least never recommended. And source-controlling scripts of TF_VAR_ variables is significantly less elegant than remaining in HCL.

Second, it makes no sense to be concerned about typos in the tool itself since i can simply use a map for my top-level variables and i'm right back in the same boat with typos. Linters are the right tools to try and detect typos. Something like terraform vet would be the right way to alert about this.

Third, re: #19424 (comment)

your use-case of potentially sharing files between multiple configurations; that is not something we ever intended be possible

Expecting a single .tfvars to be associated to a single definition is utter nonsense. A practice I generally prefer and promote is to separate definitions based on frequency of use. For example, networks may rarely apply but application services may apply many times a day so it makes sense to separate these definitions. Extrapolating that to a full, complex infrastructure can result in dozens of folders of terraform definitions (making use of dozens of modules), most with very similar, but rarely identical, variables needs. To require a custom, unique .tfvars for each of these is simply ridiculous! .tfvars files should not align with the .tf definitions! .tf files align with the infrastructure design (like deployment frequency already mentioned) while .tfvars align more with application layout like allowing developers to define variables needed for their services. So .tf and .tfvars are and should be organized differently. To require that developers define only the values used in the infrastructure results in a disastrously tight coupling between the developers and the infrastructure and also means they'll have to maintain multiple .tfvars (which almost certainly means duplicating values) just so they can be used in various deployment scenarios.

Lastly, the diff output of terraform is absolutely critical. It's immensely more important than warning about a typo and I would even argue that it's the most important aspect of the tool. In lieu of a specific command to report problems, like validate, vet, ..., the diff is actually the right place to discover that there is a typo. To clutter that output with warnings about situations that the user may intentionally desire is amazingly irresponsible.

This has a very serious negative impact on productivity. I don't completely disagree with the intent of providing safer infrastructure definitions. But I strongly disagree with the proposed solution. And while I do also agree with your concern about restraining the complexity of the CLI flags, I believe this is a valid reason for an exception of making this a selectable feature. Something like -[no]-strict-mode, or even just TF_STRICT_MODE=true|false. To simply ignore all the valid use-cases outlined in the comments posted here and elsewhere and to believe your usage is the only right use-case is incredibly heavy-handed and makes me seriously question continuing to rely on this tool in the long term.

cc: @apparentlymart

[udondan]

These warnings confused our users when running our tooling (based on terragrunt). We're now using a wrapper script to get rid of these warnings. Leaving it here as others might find this helpful:

#!/usr/bin/env bash
ansi="(\x1b\\[[0-9;]+[a-zA-Z])*"
terraform "$@" | pcregrep -Mv "\n?${ansi}Warning: ${ansi}Values? for undeclared variables?(\n|.)*set these instead.\n?"

If you use terragrunt, you can just point to this script via TERRAGRUNT_TFPATH.

^{On MacOS you can install pcregrep via brew install pcre}

[kolly83]

Hello Guys

Can somebody tell how can be fixed this error\warning message and run the script?

Interestingly I have used the totally same terraform.tfvars file and procider.tf file in my previous learning lessons and it did work.

Though I am struggling with this warn message in my current learning lesson.

PS: I have tried the below but now I've got an Error message: Invalid block definition.

The equals sign "=" indicates an argument definition, and must not be used
when defining a block.

export TF_VAR_MY_VAR1 = "toto"
export TF_VAR_MY_VAR2 = "tata"

It is really annoying.

Terraform version: 0.12.9

Thank you.

[udondan]

fixed this error\warning message and run the script?

It only is a warning not an error (yet). So your code is still working. It might not in some future version of terraform.

export TF_VAR_MY_VAR1 = "toto"
export TF_VAR_MY_VAR2 = "tata"

This would not go into your terraform files. You need to export these vars before calling terraform, like so:

export TF_VAR_MY_VAR1="toto"
export TF_VAR_MY_VAR2="tata"
terraform plan ....

or putting it right in front of the tf call:

TF_VAR_MY_VAR1="toto" TF_VAR_MY_VAR2="tata" terraform plan ....

[kolly83]

Hello udondan

Thanks for your message, though after I amended the terraform.tfvars as you suggested I've still got an error:
Error: Invalid block definition

The equals sign "=" indicates an argument definition, and must not be used
when defining a block.

Thanks.

[udondan]

The whole thing should not be in your tfvars. You have to export it outside of tf.

[kolly83]

What do you mean?

Like that?

terraform plan -out out.terraform # terraform plan and write the plan to out file

In the lesson it is not said that it has to be exported.

Thank you.

[kolly83]

I could sorted out it.

Thank you for your help anyway.

[DTTerastar]

Please just add an option to ignore these! Pulumi is going to get a lot of new users unless you listen...

[mingzhaodotname]

Instead of forcing it, a better option might be to provide a command flag (like many other terraform flags), e.g. --allow-undeclared-variable or --allow-set-undeclared-variable

In this way, users have the judgement call to enforce or not. I think this will have a better user experience. As a tool, terraform should make it easy for users to do their job, instead of blocking them. Terraform can have some best practices, and can make them as the default behavior too. However, it should give users option to do their work, as there are so many other use cases.

[lqueryvg]

Sadly we are at 0.12.13 and still no resolution for this.

But it doesn't make any sense...

The use case for being able to specify input variables which don't have to be declared and used by all configurations is fairly real and obvious. Yet Terraform seems to be trying to prevent users from doing this.

The Terraform .11 behaviour was:

• If I don't want one of my configurations to use a tvfars input variable, I don't declare it (i.e. no variable declaration in a .tf file) in that configuration.
• If I try to use a variable in a configuration without declaring it, Terraform gives me an error.
• If I try to use a variable in a configuration which I have declared but haven't specified a value for in a tfvars file, Terraform will prompt for a value.

All bases are covered, aren't they ? No problem, correct ?
So, what problem are Hashicorp trying to solve with this warning in Terraform .12 ?

Also, if there is a real problem which I've missed, then how does forcing users to use TF_VAR_ variables help ? We'll surely still hit the same problem (whatever that is!) after suffering the inconvenience of having to export environment variables (in countless wrapper scripts) because we are no longer allowed to use tfvars files for this use case.

What's wrong with giving the users the flexibility and power to structure their project in their own way ?

I'm baffled !

[voltzop]

Another use case I have not seen mentioned in these threads:

We are doing multithreaded Terraform deployments on our workers as part of our CICD pipeline - requiring system wide environment variables will simply not work if we don't otherwise work around this unnecessary, breaking change. We supply tfvars to components based on arguments, environment files, and platform specifications and may be running one or more deployments to one or more accounts at any given time.

I sincerely hope HashiCorp listens on this one.

[SerhiiSokolov]

I have big variables (objects for environments). One map may include 20 environments and each environment should include about 50 properties. Move those variables to ENV is terrifying me.

[kayrus]

Just in case, there is another trend, like removing of using env vars: kubernetes/kubernetes#81117

[jmervine]

Seconding what @smeena667 said, we're in the process of moving to Terragrunt to resolve this issue, as well as many others.

[conzy]

This is a really disappointing change. I typically use terragrunt to paper over some of the cracks in functionality missing from terraform. Thats fine. On a current project I didn't want to introduce the additional complexity of another tool like terragrunt and wanted to give "vanilla" terraform a go.

After some battling and creative use of symlinks and terraform.tfvars I finally reach a reasonably elegant solution and I was hit with this deprecation warning.

I'm not trying to do anything groundbreaking. Multiple AWS accounts with a handful of states each with some shared configuration to keep the code DRY. Its looking like adding terragrunt to this project is actually far less complex than trying to achieve this with vanilla terraform

[tphan18]

In my case, I integrated the SAM CLI tool to build my lambdas:

1. SAM uses template.json as its configuration to build and upload lambdas to S3.
2. Terraform uses template.json as variable file so I can dynamically build my lambda resources.

I don't want to declare unused variables like AWSTemplateFormatVersion and Transform.

[mvejmelk]

Hello - we have bumped into this "problem" recently. I thought we would have nicely aggregated site specific variables in site.tfvar files and could deduplicate our codebase a lot. Now I am considering some dirty workaround so that our terraform wrapper would load files based on the modules names (something like platform/service.tfvars would be loaded by our terraform wrapper when deploy platform/service module would be invoked). If possible, consider not making this breaking change for the future.

With Terraform 0.12 version it started to look like terraform starts to be more "mature" language where one can refactor and build more abstractions (and deduplicate), but this change seems to go against these visions.

[matt-brewster]

Another perspective on JSON front: we were hoping to use a single set of JSON files to serve as "set it once" configuration for both our Terraform variables and application configuration (as most of these values are used in both cases) but without being able to conditionally ignore these warnings, that's not really feasible.

This is similar to what we're doing: we store variables in dynamodb, and these variables are used by both Terraform and Ansible. We have a wrapper script that exports the variables from dynamodb as JSON and passes them in to Terraform (and Ansible). We really don't want to see these warnings as they are just noise. Some of these variables are maps which are awkward to put into environment variables, indeed from the Terraform documentation: "For readability, and to avoid the need to worry about shell escaping, we recommend always setting complex variable values via variable definitions files."

[henrytk]

A -strict flag for enforcing this seems like a good solution. I actually like having an explicit set of variables for a Terraform run, so would turn this flag on to ensure undeclared variables cause tests to fail, but I totally appreciate the use case for common variables, and they should be usable if people's setup requires it - for example, if not using a common set of variables results in massive amounts of duplication.

[alanhagerman]

+1 for solving this. I too am using this for a shared secrets file common to terraforms that is stored outside the repo. Why would I want to create a shell script to move all these to environments. I also use this for MAP variables to keep the tfvars manageable. If you are going to manage variable quality just make a variable output command and show me counts for references or something. this warning is just noise

[ranokarno]

is it possible to introduce parameter like TF_WARN_OUTPUT_ERRORS
this will allow user to ignore undeclared variable from -var-file

e.g. TF_WARN_INPUT_ERROR

[TheYorkshireDev]

@apparentlymart Can you confirm if this warning will become an error for the v0.13.0 release which I see is in beta.

If not, do the terraform team have an update to address the concerns raised by the community for this change. We have been talking about it since v0.12.0 alpha in 2018? hashicorp/terraform#19424

[jhaood]

@apparentlymart
It is very important to our deployment stack and TF architecture that this ERROR be abandoned for TF13. Preventing typo's is a terrible reason to effect everyone's deployment architecture. Please allow a command-line arg to disable the ERROR - and the WARNING as well.

We do not copy-paste our TF for each new ENV. I have one DRY source tree for all regions and all accounts. We deploy our Disaster Recovery region from the exact same TF as the primary ENV. I parameterize the variables per region & account separately from the terraform/modules tree. These regional-params define vars like "ami_id" or "rds_parameter_group_name". I use the same regional-params for deploying RDS or DDB.

Furthermore, the same regional-params tfvars are used by the TerraTest suite, local dev & our CD pipelines that push to DEV. For a given deployment environment, we have one tfvars file at the regional level, then other tfvars inside the terraform source tree for stack-specific variables, not regional vars.

Hashicorp's goal of protecting newer developers against type-o's would would force us to make things more complex with hierarchical regional tfvars files or generating the vars at runtime.

Thank you for your consideration.

• jim

This is an example of our source-tree:
./deployments/dev/us-east-1/aws-regional-params.tfvars
./deployments/dev/us-west-2/aws-regional-params.tfvars
./deployments/stage/us-east-1/aws-regional-params.tfvars
./deployments/prod/us-east-1/aws-regional-params.tfvars
....
./examples/rds_cluster_aurora_example
./scripts
./terraform/modules/rds
./terraform/modules/rds/cluster
./terraform/modules/rds/cluster_instance
./tests

[wdec]

I would like to add support for not deprecate support for undeclared variables, or at least have a flag allowing a backward compatible "-ignore-undeclared" flag (or the like).

We are running a fairly sized network partially orchestrated via Terraform. In it, we use a comprehensive json variable file for all of the network inventory that's used by different groups and across different tools. This makes our inventory independent of the tool used and also ensures coherency.
As such, the inventory contains json attributes which are of no significance to our terrafrom runs.
Should this feature be pulled out, we'd need to create a specialized Terrafrom inventory which is undesirable, when treating the inventory as a managed code.
Terraform should simply continue to ignore undeclared variables even if encountered in the variable file.
A change to this will break existing deployments, for no apparent benefit.

[breml]

We ran into this issue as well and we would highly appreciate any official reaction from hashicorp why there is no attention to this issue and why it is not an option to add an opt-out flag to terraform.

There is at least one additional issue, which looks like a duplicate: #22337.

[TheToxicDuck]

We're also hitting into this "Warning". If this becomes an error our environment will need redoing and it's a shame because I have yet to see another way of doing things which provides us with the hierarchy we want for our variables.

Please provide an opt out or remove this warning altogether as all it does is constrain the users of Terraform.

[breml]

We found a possible solution to ignore the undeclared variables by using https://github.com/shihanng/tfvar

This tool allows to set the necessary (it does inspect the terraform code) variables as env vars with the following command:

eval $(tfvar -e --var-file ../config/xyz.tfvar .)
terraform apply

CC: @shihanng

[MrMickS]

We are using Terraform Cloud with remote execution. This rules out environmental variables so the only choice we have is to use an auto.tfvars file.

In order not to have to use specific pipelines for each Git repository we use a common auto.tfvars file which does contain some values that aren't used in the Terraform in some of them.

We would also like to have the ability to suppress the warnings.

[DrStrangepork]

I need to chime in on this issue as well. My team has dozens of distinct Terraform repos managing dozens of services in our environments, and we have gone through considerable effort to consolidate our variables in a single set of tfvars, so that we don't have to recreate and support duplicate variables for every repo. The idea of a "strict" compliance of set->declared variables is absurd. Variables are declared for the purpose of identifying them to Terraform as variables it needs to be concerned with, so why should Terraform care about variables which are not declared? Are we to believe that future development of Terraform will render it incapable of discerning declared variables from undeclared variables in such a way as to put users' code and infrastructure at risk? For those environments where strict compliance is necessary, this is a fine additional feature, but who would want to make the same variable value change across 30+ repos when the same change can be accomplished by changing it in 1 file where it can be used by those 30+ repos and be ignored by the other 100+ that will ignore it because they don't have that variable declared? What is so inherently insecure or programmatically unsound about allowing undeclared variables to simply be ignored like they always have?

[DTTerastar]

I think the writing is on the wall, terraform has jumped the shark... They no longer listen to their users.

[Juludut]

While this warning may be useful in general, it would be nice to be able to ignore it.
I would appreciate the TF_WARN_OUTPUT_ERRORS idea, or anything to knowingly disable these, without using a wrapper of some kind.

[wonderphil]

@apparentlymart be really good to get an update from hashicorp about this, does hashicorp still plan on making this happen?

[djsuszi]

This warning is so "pain in the ..."
I have variable files like
wc -l /
95 account/development.tfvars
57 account/preprod.tfvars
57 account/prod.tfvars
299 environment/development.tfvars
308 environment/preprod.tfvars
284 environment/prod.tfvars
233 global.tfvars

so to terraform one thing I get 95 lines + 299 lines + 233 lines of vars
It's so stupid to put all this settings to env variable first.
Right now I often end with something like that:
Warning: Values for undeclared variables
In addition to the other similar warnings shown, 82 other variable(s) defined
without being declared.

And what with situation, when I want to terraform from one console
first development and next prod ?
I need to clean all env variables to make sure, nothing will pass from one env to another.

Please add that IGNORE / STRICT / WARN_OUTPUT_ERRORS or ... anything and don't ignore your users.

If you want it could be even paid feature, so many people need it !!!
--allow-undeclared-variable-paid-feature=S0M3BA5E64TIME8AS3DC0D3

[cp-jennifer]

I have a similar design pattern as many people here and I would also like to know if there are any plans to avoid this impending doom.

[ekini]

If you use Arch Linux, feel free to use https://aur.archlinux.org/packages/terraform-nowarning/

It has a patch that still gives users feedback as per #22004 (comment) but doesn't display unnecessary warnings.

[pkolyvas]

I hope I bring good news for all here.

Terraform 0.15+ is shipping with support for undeclared variables values, but without the deprecation notice.

Yes, you read that right, we've heard you loud and clear. We will protect this workflow for the long term. We will continue to support using undeclared variable values in .tfvars files as per the original submission's common.tfvars example for the foreseeable future.

PR details are here: #27795 (which I'll use to close the issue).

We won't be adding a method to silence the warnings however; we believe the consolidation of repetitive warning messages, coupled with the -compact-warnings option (#23632) should suffice for most users, while still giving those who wish to catch errors with their variable value definitions the opportunity to do so.

Thank you for your patience and all your input.

[lorengordon]

I feel tears coming on

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.