After 0.12.0 upgrade teraform spawns 1000+ process and uses 7GB of RAM

[tomaxsas]

Terraform Version

$ terraform.exe -v
Terraform v0.12.0

Actual Behavior

After upgrading to new terrafrom and running apply to config 16 VM's terrafomr spawns 1000+ process and use 7GB of RAM

Additional Context

Win 10 OS

[jbardin]

Hi @tomaxsas,

Thanks for letting us know!
Can you provide any more information about your system? Is there an unusually large number of providers or provisioners in your configuration? Are these processes left running after the main process has exited, or do they clean up afterwards? Can you inspect the arguments of these processes to determine what the processes are for?

Thanks!

[tomaxsas]

one provider, it's, vSphere, and latest version (1.11.0). Only 3 provisioners per VM. All processes are exited after execution
And here is command line args

[tomaxsas]

0.12.1 did not fixed problem

[tomaxsas]

so turns out terraform creates process for every provisioner which it founds in main.tf even if i target only one module out of many

[apparentlymart]

Thanks for that additional context, @tomaxsas.

I suspect this is happening because Terraform must launch a provisioner to validate configurations for it. However, because that validation happens before the provisioner has been fully configured in principle we should be able to share a single process per provisioner type for the entire configuration, and only launch the individual processes per configuration block during the apply phase. We'll investigate that for a future release.

[apparentlymart]

In #21827 we can see a variant of this problem on a Unix-like OS. Because of the fork/exec model I suspect it takes a lot more provisioners to hit the limit there, but it's still excessive to perform validation of each block in a separate process.

[tomaxsas]

Are there ETA on this? As it's really a huge bug for users with few provisioners...

[stahs]

@tomaxsas agreed. This is completely blocking us from upgrading to 0.12 and I imagine it's a big problem for any deployments with an appreciable number of provisioners.

[Gerr-Alt]

Same problem. Can you please give some ETA? That is the huge problem for us.

[tomaxsas]

Thanks, now resource updates is 5-10 times faster and ram only 60-130MB

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.