Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform CRASH after updating terraform version to 1.6 #33977

Closed
henriquebrunialti opened this issue Oct 4, 2023 · 14 comments · Fixed by #33996
Closed

Terraform CRASH after updating terraform version to 1.6 #33977

henriquebrunialti opened this issue Oct 4, 2023 · 14 comments · Fixed by #33996
Assignees
Labels
bug config confirmed a Terraform Core team member has reproduced this issue upstream v1.6 Issues (primarily bugs) reported against v1.6 releases
Milestone

Comments

@henriquebrunialti
Copy link

henriquebrunialti commented Oct 4, 2023

Terraform Version

1.6.0

Terraform Configuration Files

I don't think the configuration has any impact here, since this crash has started after the release of the new version.

We aways install the lastest version of terraforms in our pipeline machines.

Providers:

2023-10-04T18:57:06.0116711Z ├── provider[registry.terraform.io/hashicorp/http]
2023-10-04T18:57:06.0117376Z ├── provider[registry.terraform.io/betr-io/mssql] 0.2.7
2023-10-04T18:57:06.0117815Z ├── provider[registry.terraform.io/mastercard/restapi] 1.18.2
2023-10-04T18:57:06.0119706Z ├── provider[registry.terraform.io/hashicorp/azurerm] 3.71.0
2023-10-04T18:57:06.0120458Z ├── provider[registry.terraform.io/cloudflare/cloudflare] 4.14.0
2023-10-04T18:57:06.0120978Z ├── provider[registry.terraform.io/hashicorp/template]
2023-10-04T18:57:06.0123309Z ├── provider[registry.terraform.io/hashicorp/time] 0.9.1
2023-10-04T18:57:06.0123865Z ├── provider[registry.terraform.io/hashicorp/local]
2023-10-04T18:57:06.0124345Z ├── provider[registry.terraform.io/hashicorp/azuread]

Debug Output

2023-10-04T18:57:12.9958114Z !!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-10-04T18:57:12.9958301Z 
2023-10-04T18:57:12.9973941Z value is marked, so must be unmarked first
2023-10-04T18:57:12.9974281Z goroutine 471 [running]:
2023-10-04T18:57:12.9974506Z runtime/debug.Stack()
2023-10-04T18:57:12.9974779Z 	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x5e
2023-10-04T18:57:12.9975197Z runtime/debug.PrintStack()
2023-10-04T18:57:12.9975463Z 	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x13
2023-10-04T18:57:12.9975751Z github.com/hashicorp/terraform/internal/logging.PanicHandler()
2023-10-04T18:57:12.9976027Z 	/home/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x13b
2023-10-04T18:57:12.9976526Z panic({0x2c47000?, 0x39833a0?})
2023-10-04T18:57:12.9976799Z 	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x270
2023-10-04T18:57:12.9977428Z github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
2023-10-04T18:57:12.9977896Z 	/home/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/marks.go:141
2023-10-04T18:57:12.9978388Z github.com/zclconf/go-cty/cty.Value.AsString({{{0x39c0fb0?, 0xc000122cb1?}}, {0x2ed3ce0?, 0xc002c872f0?}})
2023-10-04T18:57:12.9979020Z 	/home/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1385 +0x47
2023-10-04T18:57:12.9980041Z github.com/zclconf/go-cty/cty.Value.Range({{{0x39c0fb0?, 0xc000122cb1?}}, {0x2ed3ce0?, 0xc002c872f0?}})
2023-10-04T18:57:12.9980469Z 	/home/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_range.go:53 +0x285
2023-10-04T18:57:12.9980775Z github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0xc0006d4150, 0xc002c86c60)
2023-10-04T18:57:12.9981096Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:745 +0xd35
2023-10-04T18:57:12.9981447Z github.com/hashicorp/hcl/v2/hcldec.(*AttrSpec).decode(0xc0033d7470, 0xc002a54680?, {0xc002392c50?, 0x4?, 0xc0006346a0?}, 0xc002c86c60)
2023-10-04T18:57:12.9981961Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/spec.go:214 +0x1f5
2023-10-04T18:57:12.9982278Z github.com/hashicorp/hcl/v2/hcldec.ObjectSpec.decode(0xc0033d7380, 0xc0033d7380?, {0x5232260, 0x0, 0x0}, 0x10?)
2023-10-04T18:57:12.9982617Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/spec.go:88 +0x1f7
2023-10-04T18:57:12.9983132Z github.com/hashicorp/hcl/v2/hcldec.decode({0x39c2360, 0xc000b79080}, {0x5232260, 0x0, 0x0}, 0xc000a621f0?, {0x39c1838, 0xc0033d7380}, 0x0)
2023-10-04T18:57:12.9983473Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/decode.go:24 +0x10f
2023-10-04T18:57:12.9983801Z github.com/hashicorp/hcl/v2/hcldec.(*BlockSetSpec).decode(0xc0033d74a0, 0xc002c91440, {0xc002392538?, 0x4?, 0xc0006346a0?}, 0xc002c86c60?)
2023-10-04T18:57:12.9984712Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/spec.go:759 +0x1ff
2023-10-04T18:57:12.9985065Z github.com/hashicorp/hcl/v2/hcldec.ObjectSpec.decode(0xc0033d65d0, 0xc0033d65d0?, {0x0, 0x0, 0x0}, 0xc0030a8000?)
2023-10-04T18:57:12.9985565Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/spec.go:88 +0x1f7
2023-10-04T18:57:12.9985913Z github.com/hashicorp/hcl/v2/hcldec.decode({0x39c2360, 0xc000b78500}, {0x0, 0x0, 0x0}, 0x0?, {0x39c1838, 0xc0033d65d0}, 0x0)
2023-10-04T18:57:12.9986414Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/decode.go:24 +0x10f
2023-10-04T18:57:12.9986917Z github.com/hashicorp/hcl/v2/hcldec.Decode(...)
2023-10-04T18:57:12.9987448Z 	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hcldec/public.go:18
2023-10-04T18:57:12.9988168Z github.com/hashicorp/terraform/internal/lang.(*Scope).EvalBlock(0xc002c5fcb0, {0x39c1db0, 0xc003569cb0}, 0xc002a55178?)
2023-10-04T18:57:12.9988511Z 	/home/runner/work/terraform/terraform/internal/lang/eval.go:71 +0x1ea
2023-10-04T18:57:12.9988920Z github.com/hashicorp/terraform/internal/terraform.(*BuiltinEvalContext).EvaluateBlock(0xc0029e7920?, {0x39c1598, 0xc0006d3550}, 0xc001c5f9b0?, {0x0?, 0x0?}, {{{{0x0, 0x0}}, {0x0, 0x0}}, ...})
2023-10-04T18:57:12.9989344Z 	/home/runner/work/terraform/terraform/internal/terraform/eval_context_builtin.go:282 +0x145
2023-10-04T18:57:12.9990669Z github.com/hashicorp/terraform/internal/terraform.(*NodeValidatableResource).validateResource(0xc0012ce990, {0x39de640, 0xc000b640e0})
2023-10-04T18:57:12.9991198Z 	/home/runner/work/terraform/terraform/internal/terraform/node_resource_validate.go:372 +0x666
2023-10-04T18:57:12.9991559Z github.com/hashicorp/terraform/internal/terraform.(*NodeValidatableResource).Execute(0xc0012ce990, {0x39de640, 0xc000b640e0}, 0x0?)
2023-10-04T18:57:12.9991914Z 	/home/runner/work/terraform/terraform/internal/terraform/node_resource_validate.go:52 +0x55
2023-10-04T18:57:12.9992290Z github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc0026f8480, {0x39de640, 0xc000b640e0}, {0x214fa4d8a20, 0xc0012ce990})
2023-10-04T18:57:12.9992653Z 	/home/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:143 +0xbe
2023-10-04T18:57:12.9993125Z github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x3190340, 0xc0012ce990})
2023-10-04T18:57:12.9993452Z 	/home/runner/work/terraform/terraform/internal/terraform/graph.go:78 +0x375
2023-10-04T18:57:12.9993788Z github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc00087f860, {0x3190340, 0xc0012ce990}, 0xc000e55640)
2023-10-04T18:57:12.9994219Z 	/home/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2e5
2023-10-04T18:57:12.9994534Z created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 21
2023-10-04T18:57:12.9994822Z 	/home/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xde8

Expected Behavior

The terraform apply should have run successfully.

We had releases that ran fine 6 hours ago.

Actual Behavior

Terraform Apply is crashing even before the execution plan is made.

Steps to Reproduce

terraform init
terraform apply

with terraform 1.6.0

Additional Context

No response

References

No response

@henriquebrunialti henriquebrunialti added bug new new issue not yet triaged labels Oct 4, 2023
@henriquebrunialti henriquebrunialti changed the title Error on terraform apply 1.6 Terraform CRASH after updating to 1.6 Oct 4, 2023
@henriquebrunialti henriquebrunialti changed the title Terraform CRASH after updating to 1.6 Terraform CRASH after updating to terraform version to 1.6 Oct 4, 2023
@henriquebrunialti henriquebrunialti changed the title Terraform CRASH after updating to terraform version to 1.6 Terraform CRASH after updating terraform version to 1.6 Oct 4, 2023
@henriquebrunialti
Copy link
Author

Confirming that after I rollback the version to 1.5.7 the apply command is working fine.

@jrogers-dlp
Copy link

I can confirm that this is also happening for terraform validate

@crw
Copy link
Collaborator

crw commented Oct 4, 2023

Thanks for the reports. It may not be necessary pending review by the devs, but if you do figure out a minimal config that causes the crash, that is always helpful.

@Erouan50
Copy link
Contributor

Erouan50 commented Oct 4, 2023

I found a way to reproduce:

output "test" {
  value = var.test != null ? var.test : ""
}

variable "test" {
  type      = string
  default   = null
  sensitive = true
}

Try a terraform plan and it crashes.

@jbardin jbardin added upstream config and removed new new issue not yet triaged labels Oct 4, 2023
@jbardin
Copy link
Member

jbardin commented Oct 4, 2023

Thanks @Erouan50!

I'm also going to add this stack trace, which is slightly different but will probably be addressed in the same upstream patch:

cannot refine an unknown value of an unknown type
goroutine 85 [running]:
runtime/debug.Stack()
	/opt/homebrew/Cellar/go/1.21.1/libexec/src/runtime/debug/stack.go:24 +0x64
runtime/debug.PrintStack()
	/opt/homebrew/Cellar/go/1.21.1/libexec/src/runtime/debug/stack.go:16 +0x1c
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/Users/jbardin/terraform/internal/logging/panic.go:58 +0x164
panic({0x1056ae8a0?, 0x105c9e630?})
	/opt/homebrew/Cellar/go/1.21.1/libexec/src/runtime/panic.go:920 +0x26c
github.com/zclconf/go-cty/cty.Value.Refine({{{0x105cdc3a8?, 0x107494520?}}, {0x10578d240?, 0x10744f540?}})
	/Users/jbardin/go/pkg/mod/github.com/zclconf/[email protected]/cty/unknown_refinement.go:47 +0x498
github.com/zclconf/go-cty/cty.Value.RefineNotNull({{{0x105cdc3a8?, 0x107494520?}}, {0x10578d240?, 0x10744f540?}})
	/Users/jbardin/go/pkg/mod/github.com/zclconf/[email protected]/cty/unknown_refinement.go:123 +0x2c
github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0x140007bc8c0, 0x14000aa54e8)
	/Users/jbardin/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:749 +0xa8c

@ubaid-github

This comment was marked as off-topic.

@thepoppingone
Copy link

Attaching Stack Trace here as well

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

value is marked, so must be unmarked first
goroutine 8[126](https://github.com/SPHTech-Data/terraform-datahub-infra/actions/runs/6415569076/job/17417686559#step:16:127) [running]:
runtime/debug.Stack()
	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x5e
runtime/debug.PrintStack()
	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x13
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/home/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x13b
panic({0x2b92ec0?, 0x38c4c30?})
	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x270
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
	/home/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/marks.go:141
github.com/zclconf/go-cty/cty.Value.AsString({{{0x3902720?, 0xc000012c69?}}, {0x2e1de40?, 0xc00ab9b0f8?}})
	/home/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1385 +0x47
github.com/zclconf/go-cty/cty.Value.Range({{{0x3902720?, 0xc000012c69?}}, {0x2e1de40?, 0xc00ab9b0f8?}})
	/home/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_range.go:53 +0x285
github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0xc0006eae70, 0xc00ab9b0e0)
	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:746 +0xd6a
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalExpr(0xc00542a480, {0x3902df8?, 0xc0006eae70}, {{0x3902f48?, 0x5[130](https://github.com/SPHTech-Data/terraform-datahub-infra/actions/runs/6415569076/job/17417686559#step:16:131)520?}})
	/home/runner/work/terraform/terraform/internal/lang/eval.go:175 +0x1bd
github.com/hashicorp/terraform/internal/terraform.(*BuiltinEvalContext).EvaluateExpr(0x76?, {0x3902df8, 0xc0006eae70}, {{0x3902f48?, 0x5130520?}}, {0x0?, 0x0?})
	/home/runner/work/terraform/terraform/internal/terraform/eval_context_builtin.go:289 +0xab
github.com/hashicorp/terraform/internal/terraform.(*NodeLocal).Execute(0x7f04744dce60?, {0x391fb60, 0xc003fdc620}, 0x50?)
	/home/runner/work/terraform/terraform/internal/terraform/node_local.go:158 +0x592
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc0044e78c0, {0x391fb60, 0xc003fdc620}, {0x7f0474052c80, 0xc00492c840})
	/home/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:[143](https://github.com/SPHTech-Data/terraform-datahub-infra/actions/runs/6415569076/job/17417686559#step:16:144) +0xbe
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x2f593a0, 0xc00492c840})
	/home/runner/work/terraform/terraform/internal/terraform/graph.go:78 +0x375
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc003f16d80, {0x2f593a0, 0xc00492c840}, 0xc0051fbc40)
	/home/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2e5
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 2496
	/home/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xde8

@ogusak
Copy link

ogusak commented Oct 5, 2023

For us it crashes on 1.6 on terraform plan with the error message and stack trace similar to the comment above

@bosch-ch
Copy link

bosch-ch commented Oct 5, 2023

Same bug on a destroy.

@pjhampton
Copy link

Our CI wasn't pinned at a specific TF version and upgraded us automatically. We've rolled back to v1.5.7 for now.
Commenting to follow.

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

goroutine 23934 [running]:
runtime/debug.Stack()
	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x5e
runtime/debug.PrintStack()
	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x13
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/home/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x13b
panic({0x2b92ec0?, 0x38c4c30?})
	/opt/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x270
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
	/home/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/marks.go:141
github.com/zclconf/go-cty/cty.Value.AsString({{{0x3902720?, 0xc0000d2b41?}}, {0x2e1de40?, 0xc007014378?}})
	/home/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_ops.go:1385 +0x47
github.com/zclconf/go-cty/cty.Value.Range({{{0x3902720?, 0xc0000d2b41?}}, {0x2e1de40?, 0xc007014378?}})
	/home/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_range.go:53 +0x285
github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0xc000892070, 0xc000c430c8)
	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hclsyntax/expression.go:745 +0xd35
github.com/hashicorp/hcl/v2/hcldec.(*AttrSpec).decode(0xc0067ab860, 0xc0067957b8?, {0xc006101f30?, 0x10?, 0xc001376d00?}, 0xc000c430c8)
	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hcldec/spec.go:214 +0x1f5
github.com/hashicorp/hcl/v2/hcldec.ObjectSpec.decode(0xc0067ab4d0, 0xc0067ab4d0?, {0x0, 0x0, 0x0}, 0xc001bdb860?)
	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hcldec/spec.go:88 +0x1f7
github.com/hashicorp/hcl/v2/hcldec.decode({0x3903ae0, 0xc0019bc4a0}, {0x0, 0x0, 0x0}, 0x0?, {0x3902fb8, 0xc0067ab4d0}, 0x0)
	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hcldec/decode.go:24 +0x10f
github.com/hashicorp/hcl/v2/hcldec.Decode(...)
	/home/runner/go/pkg/mod/github.com/hashicorp/hcl/v2@v2.18.0/hcldec/public.go:18
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalBlock(0xc00104e990, {0x3903530, 0xc003e4e000}, 0xc006795c38?)
	/home/runner/work/terraform/terraform/internal/lang/eval.go:71 +0x1ea
github.com/hashicorp/terraform/internal/terraform.(*BuiltinEvalContext).EvaluateBlock(0x0?, {0x3902d18, 0xc001650d10}, 0xc00219b6e0?, {0x0?, 0x0?}, {{{{0x0, 0x0}}, {0x0, 0x0}}, ...})
	/home/runner/work/terraform/terraform/internal/terraform/eval_context_builtin.go:282 +0x145
github.com/hashicorp/terraform/internal/terraform.(*NodeAbstractResourceInstance).plan(0xc0066ffa00, {0x391fb60, 0xc00451ca80}, 0x0, 0x0, 0x0, {0x0, 0x0, 0x0?})
	/home/runner/work/terraform/terraform/internal/terraform/node_resource_abstract_instance.go:717 +0xb8b
github.com/hashicorp/terraform/internal/terraform.(*NodePlannableResourceInstance).managedResourceExecute(0xc0018f9290, {0x391fb60, 0xc00451ca80})
	/home/runner/work/terraform/terraform/internal/terraform/node_resource_plan_instance.go:282 +0x1795
github.com/hashicorp/terraform/internal/terraform.(*NodePlannableResourceInstance).Execute(0x7f0972990970?, {0x391fb60?, 0xc00451ca80?}, 0x78?)
	/home/runner/work/terraform/terraform/internal/terraform/node_resource_plan_instance.go:74 +0x8a
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc005412000, {0x391fb60, 0xc00451ca80}, {0x7f0972805ca0, 0xc0018f9290})
	/home/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:143 +0xbe
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x3135c40, 0xc0018f9290})
	/home/runner/work/terraform/terraform/internal/terraform/graph.go:78 +0x375
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc000e2ad20, {0x3135c40, 0xc0018f9290}, 0xc005b58f80)
	/home/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2e5
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 17000
	/home/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xde8

@NyanKiyoshi
Copy link

Not sure if that's exactly the same bug: putting a sensitive() value into the id field of an import block may cause to crash as well with the same error message "value is marked, so must be unmarked first" (only tried against random_password).

It can be reproduced like so:

locals {
  password = sensitive("abc")
}

resource "random_password" "test" {
  length = 3
}

import {
  to = random_password.test
  id = local.password  # crashes as well with `id = sensitive("abc")`
}

And then terraform plan will crash:

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

value is marked, so must be unmarked first
goroutine 104 [running]:
runtime/debug.Stack()
        /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x5e
runtime/debug.PrintStack()
        /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:16 +0x13
github.com/hashicorp/terraform/internal/logging.PanicHandler()
        /Users/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x13b
panic({0x379be40?, 0x44d1c70?})
        /Users/runner/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:920 +0x270
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
        /Users/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/marks.go:141
github.com/zclconf/go-cty/cty.Value.AsString({{{0x450fbe0?, 0xc000014c49?}}, {0x3a29120?, 0xc000c83560?}})
        /Users/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1385 +0x47
github.com/zclconf/go-cty/cty/gocty.fromCtyString({{{0x450fbe0?, 0xc000014c49?}}, {0x3a29120?, 0xc000c83560?}}, {0x379be40?, 0xc000baa940?, 0xc000cbb290?}, {0xc000accd10, 0x0, 0x0})
        /Users/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/gocty/out.go:245 +0x5f
github.com/zclconf/go-cty/cty/gocty.fromCtyValue({{{0x450fbe0, 0xc000014c49}}, {0x3a29120, 0xc000c83560}}, {0x370a7e0?, 0xc000baa940?, 0x10110a5?}, {0xc000accd10, 0x0, 0x0})
        /Users/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/gocty/out.go:87 +0x70b
github.com/zclconf/go-cty/cty/gocty.FromCtyValue({{{0x450fbe0?, 0xc000014c49?}}, {0x3a29120?, 0xc000c83560?}}, {0x370a7e0?, 0xc000baa940?})
        /Users/runner/go/pkg/mod/github.com/zclconf/[email protected]/cty/gocty/out.go:43 +0xca
github.com/hashicorp/terraform/internal/terraform.evaluateImportIdExpression({0x45100e8, 0xc0000e56e0}, {0x452d0a0?, 0xc0008f40e0?})
        /Users/runner/work/terraform/terraform/internal/terraform/eval_import.go:57 +0x35a
github.com/hashicorp/terraform/internal/terraform.(*NodePlannableResourceInstance).managedResourceExecute(0xc000cb51e0, {0x452d0a0, 0xc0008f40e0})
        /Users/runner/work/terraform/terraform/internal/terraform/node_resource_plan_instance.go:168 +0x476
github.com/hashicorp/terraform/internal/terraform.(*NodePlannableResourceInstance).Execute(0x7c65158?, {0x452d0a0?, 0xc0008f40e0?}, 0x0?)
        /Users/runner/work/terraform/terraform/internal/terraform/node_resource_plan_instance.go:74 +0x8a
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc000cdf8c0, {0x452d0a0, 0xc0008f40e0}, {0x7c6bbf8, 0xc000cb51e0})
        /Users/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:143 +0xbe
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x3d42900, 0xc000cb51e0})
        /Users/runner/work/terraform/terraform/internal/terraform/graph.go:78 +0x375
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc0007eec60, {0x3d42900, 0xc000cb51e0}, 0xc0007fa480)
        /Users/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2e5
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 119
        /Users/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xde8

But we can make it work by making the sensitive field non-sensitive:

locals {
  password = sensitive("abc")
}

resource "random_password" "test" {
  length = 3
}

import {
  to = random_password.test
  id = nonsensitive(local.password)  # will work too: `id = nonsensitive(sensitive("abc"))`
}

Results to:

random_password.test: Preparing import... [id=abc]
random_password.test: Refreshing state... [id=none]

Terraform will perform the following actions:

  # random_password.test will be imported
    resource "random_password" "test" {
        bcrypt_hash = (sensitive value)
        id          = "none"
        length      = 3
        lower       = true
        min_lower   = 0
        min_numeric = 0
        min_special = 0
        min_upper   = 0
        number      = true
        numeric     = true
        result      = (sensitive value)
        special     = true
        upper       = true
    }

Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
$ terraform version
Terraform v1.6.0
on darwin_amd64
+ provider registry.terraform.io/hashicorp/random v3.5.1

@znahas
Copy link

znahas commented Oct 5, 2023

Same also happening on:
terraform validate

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

value is marked, so must be unmarked first
goroutine 2932 [running]:
runtime/debug.Stack()
        /snap/go/10339/src/runtime/debug/stack.go:24 +0x5e
runtime/debug.PrintStack()
        /snap/go/10339/src/runtime/debug/stack.go:16 +0x13
github.com/hashicorp/terraform/internal/logging.PanicHandler()
        /build/terraform/parts/terraform/build/internal/logging/panic.go:58 +0x13b
panic({0x2b92ec0?, 0x38c4c50?})
        /snap/go/10339/src/runtime/panic.go:920 +0x270
github.com/zclconf/go-cty/cty.Value.assertUnmarked(...)
        /root/go/pkg/mod/github.com/zclconf/[email protected]/cty/marks.go:141
github.com/zclconf/go-cty/cty.Value.AsString({{{0x3902700?, 0xc000012c71?}}, {0x2e1de40?, 0xc003871b90?}})
        /root/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_ops.go:1385 +0x47
github.com/zclconf/go-cty/cty.Value.Range({{{0x3902700?, 0xc000012c71?}}, {0x2e1de40?, 0xc003871b90?}})
        /root/go/pkg/mod/github.com/zclconf/[email protected]/cty/value_range.go:53 +0x285
github.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0xc000b2abd0, 0xc003871b78)
        /root/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclsyntax/expression.go:746 +0xd6a
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalExpr(0xc003802bd0, {0x3902dd8?, 0xc000b2abd0}, {{0x3902f28?, 0x5127560?}})
        /build/terraform/parts/terraform/build/internal/lang/eval.go:175 +0x1bd
github.com/hashicorp/terraform/internal/terraform.(*nodeModuleVariable).evalModuleVariable(0xc000b38d20, {0x391fb40?, 0xc003104460?}, 0x80?)
        /build/terraform/parts/terraform/build/internal/terraform/node_module_variable.go:248 +0x225
github.com/hashicorp/terraform/internal/terraform.(*nodeModuleVariable).Execute(0xc000b38d20, {0x391fb40, 0xc003104460}, 0x4)
        /build/terraform/parts/terraform/build/internal/terraform/node_module_variable.go:183 +0x12c
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc002dd87e0, {0x391fb40, 0xc003104460}, {0x7f67012f5ff0, 0xc000b38d20})
        /build/terraform/parts/terraform/build/internal/terraform/graph_walk_context.go:143 +0xbe
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x2f07ca0, 0xc000b38d20})
        /build/terraform/parts/terraform/build/internal/terraform/graph.go:78 +0x375
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc000b38de0, {0x2f07ca0, 0xc000b38d20}, 0xc00385acc0)
        /build/terraform/parts/terraform/build/internal/dag/walk.go:384 +0x2e5
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 1180
        /build/terraform/parts/terraform/build/internal/dag/walk.go:307 +0xde8

@apparentlymart apparentlymart added the confirmed a Terraform Core team member has reproduced this issue label Oct 6, 2023
@apparentlymart apparentlymart self-assigned this Oct 6, 2023
@apparentlymart apparentlymart added this to the v1.6.1 milestone Oct 6, 2023
@apparentlymart
Copy link
Contributor

Thanks for all of the example stack traces, everyone.

We've narrowed down a few different problems here and are planning to fix them all in the forthcoming v1.6.1 release. There's no need to share any further examples here, since we have what we need to reproduce the problem.

These are all slightly incorrect details in the handling of the new v1.6.0 behaviors to perform more precise analysis of unknown values, such as tracking when an unknown value is known not to be null so that value != null can then return true rather than an unknown value. Unfortunately with a cross-cutting change like that it's relatively easy to miss some combinations of features when testing, and that's what's happened here.

We labelled this as "upstream" earlier because the bugs are actually in two libraries that Terraform depends on, rather than in Terraform itself: my personal cty library is what handles the low-level type and value details for the Terraform language and is responsible for the new unknown value tracking, and HCL provides the grammar, structure, and expression language that the Terraform language is built on and has at least one bug of its own in how it now handles unknown values in conditional expressions.

Therefore the fixes for these will actually appear in those upstream libraries, but we'll use this issue to represent upgrading Terraform's dependencies on both libraries to use the fixed versions, once they are available.

@apparentlymart
Copy link
Contributor

Thanks again everyone for sharing your stack traces above!

We've now merged some fixes that will be coming in the forthcoming v1.6.1 release.

As I mentioned above, these bugs resulted from some interactions between the new cross-cutting unknown value refinements feature and some other Terraform language features, and with cross-cutting changes like this one it's possible that similar things will crop up in other locations too. If you've seen a problem that seems similar to the ones described here even after upgrading to Terraform v1.6.1 (once it's available), please open a new issue to discuss it just because adding new bugs onto this closed issue is likely to cause us to miss things.

@hashicorp hashicorp locked as resolved and limited conversation to collaborators Oct 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug config confirmed a Terraform Core team member has reproduced this issue upstream v1.6 Issues (primarily bugs) reported against v1.6 releases
Projects
None yet
Development

Successfully merging a pull request may close this issue.