Allow import blocks within child modules

[kmoe]

Currently, import blocks may only appear in the root module. Should we allow them to appear in child modules?

Related: #33376

[Shocktrooper]

Appearing within child modules should be fine as it would only really apply to local modules and it would make passing providers to the import block easier or not needed

[dylan-shipwell]

i would appreciate the option to specify either a fully qualified namespace, or a local/relative namespace for import block within modules.

[arnvid]

This is a huge blocker for moving to providers like the AWS Provider v6.x which has announced breaking changed to aws_ssm_parameter which requires import of changed variables.

 Warning: Argument is deprecated
with module.this.aws_ssm_parameter.tfe_api_url
on .terraform/modules/this/ssm.tf line 17, in resource "aws_ssm_parameter" "tfe_api_url":

  overwrite = true

this attribute has been deprecated

[AMEvers]

Adding my 2 cents, this is also a major issue for us. We've been writing a lot of modules that deploy cloud infrastructure and modules being unable to import within a child module is incredibly burdensome.

[omarismail]

Hey @AMEvers @arnvid @dylan-shipwell, I'd love to learn more here about your workflows and use cases. Could you email me at [email protected] and we can schedule time to chat?

[arnvid]

Hey I'd love to learn more here about your workflows and use cases. Could you email me at [email protected] and we can schedule time to chat?

You can reach me on slack if you ping Robert G.

[markwellis]

i also have a use case.

i use the tailscale terraform provider, and it requires the tailscale_acl resource is imported before it can be used - https://registry.terraform.io/providers/tailscale/tailscale/latest/docs/resources/acl

I have a module that wraps that resource, and i need to import the tailscale_acl in that module.

[TrueCyberAxe]

I would also like to be able to use it within child modules, I hadn't realised it was only for the root module and have spent the last few weeks implementing imports into a number of modules so we can update them and move away from legacy code since some of our modules date back to terraform 0.12 before we had an internal standard set and such.

We also have other scenarios that would be solved with the ability to use them specifically for the following two scenarios

Changing Security Groups/Rules without it deleting and re-creating, this has been an annoying blocker for a lot of people within the Company.

Datadog Monitors for example we not too long ago had the Data Dog monitors all moved into a single company wide environment isntead of a different one for each aws environment

A lot of builds stopped working and had to have monitors manually deleted so they could be recreated by the module.

[dylan-shipwell]

off topic, work around
in case it helps anyone

i don't really believe hcl import will be a successor to proper state migrations. (even if they reverse their opinion on child import statements being illegal)

there aren't any great terraform migration frameworks out there that i know of. there is https://github.com/minamijoyo/tfmigrate but it struggles to accomplish obvious things in a lot of ways similar to terraform native import does.

for now, i'm sticking to manual migration script files
these generally read like this

#!/bin/sh -eux
# terraform/myproject/migrations/0001-some-state-migration.sh
: "${TERRAFORM_BIN:=terraform}" # use "terraform" from $PATH or $TERRAFORM_BIN if specified

do_forward=1
while [[ $# -gt 0 ]]; do
  case $1 in
    --reverse) do_forward=0;;
    *) echo "Unknown option $1"; exit 1;;
  esac
  shift
done

fn_tf_forward() {
    $TERRAFORM_BIN import foo.bar.baz my_uid
}

fn_tf_backward() {
    $TERRAFORM_BIN state rm foo.bar.baz my_uid
}

if [ $do_forward -eq 1 ]; then
    fn_tf_forward; 
else
    fn_tf_backward;
fi

the most annoying thing about this low tech solution is that there is no persistent database of which migration files have been completed. i just remember with my mind right now.

hcl import is nice because it shows a plan of what should happen if both the import step and apply step are happy, this is a great feature, and it remembers if an import is complete.

tfmigrate is nice because it remembers which migrations have been applied.

the nice thing about manual migration scripts is that you can write them in any language you like, meaning i have access to for loops, and if this then that logic control, and string format/templates, not to mention service provider apis like cloud vendors. very helpful when i have 1000 users of a module that aren't all homogeneous but are patterned. given that its also software, i could use these to generate correct import hcl blocks, and i have considered doing this, the most annoying part of this to me, is that the import hcl files must be in the same folder as the top level terraform.tf/main.tf, so i usually move them to a mgirations/ subfolder after they have been applied to not clutter the project. scripts that I run can live anywhere so i prefer that slightly.

[carloprone]

I also have a scenario where this feature would be very useful.
We have some child modules that dynamically crates (or not) and manages (or not) several resources depending on the environment they are deployed. This is controlled in the root module by passing some flags.
Some of these resources also need to be imported, and at the moment the only way to make this work is to have several complicated import blocks in the root module, where we use the for_each construct to replicate the flag-controlled configuration accordingly.
If we could push the import in the child module, our configuration would be way simpler, easier to maintain and less prone to errors.

Of course, the ID of the imported resource would be an input variable for the child module, while at the moment import blocks only accept literal strings. I can imagine this being a challenge, but it should be doable to relax such constraint.

[nikolaifa]

Another use case I'd love to be covered.
We have a module managing Azure network resources, due to how the azurerm provider implements some of its resources, and how we want to use Azure governance tooling, we cannot use the azurerm provider for a subset of these resources.
To bypass this limitation I want to use the Microsoft provided azapi provider. Since our module is widely used internally I'm interested in doing the state operations within the module itself to avoid the operational overhead of having to do it manually for each root module.

If I could use both the removed and imported block within the module itself, I can use the resource ID from the already deployed Terraform object to import the new one.

Can you give any sort of time frame for when such a feature will be available, or if it will be?

[omarismail]

Another use case I'd love to be covered. We have a module managing Azure network resources, due to how the azurerm provider implements some of its resources, and how we want to use Azure governance tooling, we cannot use the azurerm provider for a subset of these resources. To bypass this limitation I want to use the Microsoft provided azapi provider. Since our module is widely used internally I'm interested in doing the state operations within the module itself to avoid the operational overhead of having to do it manually for each root module.

If I could use both the removed and imported block within the module itself, I can use the resource ID from the already deployed Terraform object to import the new one.

Can you give any sort of time frame for when such a feature will be available, or if it will be?

Hey @nikolaifa , does the moved{} block help you here? Why is it not sufficient?

[omarismail]

@h2oearth @oliverwiegers we are actively exploring this problem, can you email me? [email protected]

[Blefish]

Using a module for AWS EC2 instances, wanted to refactor SG attachment from static to dynamic like following:

diff --git a/main.tf b/main.tf
index b5ee768..692436c 100644
--- a/main.tf
+++ b/main.tf
@@ -29,7 +29,6 @@ resource "aws_volume_attachment" "this" {
 
 resource "aws_network_interface" "this" {
   subnet_id         = var.subnet_id
-  security_groups   = var.security_group_ids
   private_ip        = var.private_ip
   source_dest_check = var.source_dest_check
 
@@ -38,6 +37,17 @@ resource "aws_network_interface" "this" {
       Name = "${local.hostname}-if"
     }
   )
+
+  lifecycle {
+    ignore_changes = [security_groups]
+  }
+}
+
+resource "aws_network_interface_sg_attachment" "this" {
+  count = length(var.security_group_ids)
+
+  security_group_id    = var.security_group_ids[count.index]
+  network_interface_id = aws_network_interface.this.id
 }

Something that could be as simple as

import {
  count = length(var.security_group_ids)
  to    = aws_network_interface_sg_attachment.this[count.index]
  id    = "${aws_network_interface.this.id}_${var.security_group_ids[count.index]}"
}

is currently impossible and I'd have to add this in every state that uses the module, changing references everywhere.
PS! Not sure if count is allowed, but according to documentation, for_each is, so it would be possible to use that instead.

[lijok]

Second. Here's where I'm struggling;

locals {
  service_linked_roles_file               = "${path.module}/service-linked-roles.json"
  service_linked_roles                    = jsondecode(file(local.service_linked_roles_file))
  service_linked_roles_service_principals = keys(local.service_linked_roles)

  service_linked_roles_to_provision = var.service_linked_roles_to_provision == ["*"] ? local.service_linked_roles_service_principals : var.service_linked_roles_to_provision
}

data "aws_caller_identity" "this" {}

resource "aws_iam_service_linked_role" "this" {
  for_each = toset(local.service_linked_roles_to_provision)

  aws_service_name = each.key
  tags             = var.tags

  lifecycle {
    ignore_changes = [
      # Description is auto-generated by AWS.
      description
    ]
  }
}

# Currently importing inside child modules is not supported: https://github.com/hashicorp/terraform/issues/33474
# import {
#   # Some of the service linked roles get automatically created by AWS when an account is created.
#   for_each = toset(local.service_linked_roles_to_provision)
#
#   to = aws_iam_service_linked_role.this[each.key]
#   id = "arn:aws:iam::${data.aws_caller_identity.this.account_id}:role/aws-service-role/${each.key}/${local.service_linked_roles[each.key]}"
# }

[robson90]

We would really appreciate this feature !

We are using the Keycloak Terraform Provider and every Instance has its own Import. That would make them 100% seperated. Right now we need to build a workaround with templatefile-Function.

[YevhenLodovyi]

I had a simple case: I wanted to import aws_cloudwatch_log_group resource in my lambda module, but:

Import blocks are only allowed in the root module.

I call this module XXX times and i wanted to import all at once. I hope it would be implemented someday and it would not be like all other issues: #25534

[findajay]

I also wanted to import a resource from security child module but its not possible.

[cah-louis-verzi]

Also waiting for this feature as we have tens of thousands of resources to import using child modules.

[markwellis]

another use case: we want to refactor module A, and move some of it's resources into module B without having to add import blocks to the parent terraform code

[zackarydev]

We're also facing an issue with this, especially in monorepo structures. If it doesn't work for "public" usages of Terraform, maybe it can be a configuration option?

Could be cool to be able to combine perhaps the import and moved blocks at the same time:

# root.tf
import {
  id = "MyTopic"
  to = google_pubsub_topic.my_topic
}

resource "google_pubsub_topic" "my_topic" {
  name = "MyTopic"
}

moved {
  from = google_pubsub_topic.my_topic
  to = module.xyz.google_pubsub_topic.my_topic
}

# xyz.tf
resource "google_pubsub_topic" "my_topic" {
  name = "MyTopic"
}

Alas, I think I'll have no way but to do 3 successive deployments to achieve this structure.

[bukzor]

I'll add my use case here. Maybe it will clarify the implementation and/or priority.

I'm working in GCP, creating a generic_project module that I want my team to use for all our GCP projects. There's a feature in GCP to enable querying logs with SQL with bigquery, but it requires modification of an object that's implicitly created by the project resource.

module "project_factory" {
  source  = "terraform-google-modules/project-factory/google"
  version = "~> 14.5"

  ...
}

locals {
  project_id         = one(module.project_factory[*].project_id)
}

resource "google_logging_project_bucket_config" "default" {
  project   = "projects/${local.project_id}"
  location  = "global"
  bucket_id = "_Default"

  enable_analytics = true   # a non-default value
  retention_days   = 30
  description      = "Default bucket"
}

resource "google_logging_linked_dataset" "default" {
  link_id = "logging_default"

  bucket   = google_logging_project_bucket_config.default.bucket_id
  parent   = google_logging_project_bucket_config.default.project
  location = google_logging_project_bucket_config.default.location
}

This "works" but I'm unsatisfied that the google_logging_project_bucket_config.default object is modified without any review of the plan, plus the fact that it's created out-of-band with respect to terraform-apply isn't clearly represented here. I had hoped to fix both problems by adding:

import {
  to = google_logging_project_bucket_config.default
  id = "projects/${local.project_id}/locations/global/buckets/_Default"
}

but I get this error during plan

│ Error: Invalid import configuration
│
│   on ../../module/generic_project/logging.tf line 33:
│   33: import {
│
│ An import block was detected in "module.project--my_project". Import
│ blocks are only allowed in the root module.

[lantoli]

we would also need to allow import inside modules, more info here: hashicorp/terraform-plugin-framework#1058.

Our main use case is to help our customers to move a resource from an old type to a new one using a combination of removed and import blocks.

Right now removed and moved blocks are allowed inside modules, I think there is not any technical issue to allow import as well.

In fact we can use import from the root modules for resources inside the module. The problem with this is that all module clients need to do the import breaking encapsulation, when it should be possible to be done inside the module.