[Bug]: Aws Provider 5.37.0 crashed when enabling TLS on ECS Service Connect

[jihed]

Terraform Core Version

1.7.3

AWS Provider Version

5.37.0

Affected Resource(s)

aws_ecs_service
aws_lb
aws_nat_gateway

Expected Behavior

It deploys ECS cluster/service with TLS and timeout configuration on.

Actual Behavior

Provider crashed before finish applying the stack.

Relevant Error/Panic Output Snippet

dule.ecs_service.aws_ecs_service.this[0]: Creating...
╷
│ Error: Plugin did not respond
│
│   with module.alb.aws_lb.this[0],
│   on .terraform/modules/alb/main.tf line 12, in resource "aws_lb" "this":
│   12: resource "aws_lb" "this" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange
│ call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with module.vpc.aws_nat_gateway.this[0],
│   on .terraform/modules/vpc/main.tf line 1059, in resource "aws_nat_gateway" "this":
│ 1059: resource "aws_nat_gateway" "this" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange
│ call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with module.ecs_service.aws_ecs_service.this[0],
│   on ../../modules/service/main.tf line 29, in resource "aws_ecs_service" "this":
│   29: resource "aws_ecs_service" "this" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange
│ call. The plugin logs may contain more details.
╵

Stack trace from the terraform-provider-aws_v5.37.0_x5 plugin:

panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 82 [running]:
github.com/hashicorp/terraform-provider-aws/internal/service/ecs.expandIssuerCertAuthority(...)
	github.com/hashicorp/terraform-provider-aws/internal/service/ecs/service.go:1548
github.com/hashicorp/terraform-provider-aws/internal/service/ecs.expandTLS({0xc002403bc0?, 0xc002401230?, 0x11329f9d?})
	github.com/hashicorp/terraform-provider-aws/internal/service/ecs/service.go:1532 +0x2fe
github.com/hashicorp/terraform-provider-aws/internal/service/ecs.expandServices({0xc002403ad0?, 0x1, 0x11331422?})
	github.com/hashicorp/terraform-provider-aws/internal/service/ecs/service.go:1500 +0x545
github.com/hashicorp/terraform-provider-aws/internal/service/ecs.expandServiceConnectConfiguration({0xc002403a70?, 0x114405f7?, 0xc0024009f0?})
	github.com/hashicorp/terraform-provider-aws/internal/service/ecs/service.go:1417 +0x249
github.com/hashicorp/terraform-provider-aws/internal/service/ecs.resourceServiceCreate({0x12c8f448, 0xc0024009f0}, 0xc0023ba680, {0x112d4d20?, 0xc000816000?})
	github.com/hashicorp/terraform-provider-aws/internal/service/ecs/service.go:625 +0xea6
github.com/hashicorp/terraform-provider-aws/internal/provider.New.(*wrappedResource).Create.interceptedHandler[...].func8(0x0?, {0x112d4d20?, 0xc000816000?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:112 +0x283
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x12c8f448?, {0x12c8f448?, 0xc001fc1aa0?}, 0xd?, {0x112d4d20?, 0xc000816000?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:773 +0x7a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc000d67b20, {0x12c8f448, 0xc001fc1aa0}, 0xc001fee9c0, 0xc0018ffc80, {0x112d4d20, 0xc000816000})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:909 +0xa89
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc003229fc8, {0x12c8f448?, 0xc001fc19b0?}, 0xc001d6fae0)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:1077 +0xdbc
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.(*muxServer).ApplyResourceChange(0x12c8f480?, {0x12c8f448?, 0xc001fc16b0?}, 0xc001d6fae0)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ApplyResourceChange.go:36 +0x193
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc000a575e0, {0x12c8f448?, 0xc001fc0ea0?}, 0xc000a187e0)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:845 +0x3d0
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x11097360?, 0xc000a575e0}, {0x12c8f448, 0xc001fc0ea0}, 0xc0018ff900, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:518 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0014de200, {0x12c8f448, 0xc001fc0e10}, {0x12cbd238, 0xc001c82680}, 0xc001fcd320, 0xc001c41890, 0x1a726698, 0x0)
	google.golang.org/[email protected]/server.go:1385 +0xe03
google.golang.org/grpc.(*Server).handleStream(0xc0014de200, {0x12cbd238, 0xc001c82680}, 0xc001fcd320)
	google.golang.org/[email protected]/server.go:1796 +0xfec
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	google.golang.org/[email protected]/server.go:1029 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 27
	google.golang.org/[email protected]/server.go:1040 +0x135

Error: The terraform-provider-aws_v5.37.0_x5 plugin crashed!

Terraform Configuration Files

resource "aws_kms_key" "test" {
  description             = %[1]q
  deletion_window_in_days = 7
  policy                  = data.aws_iam_policy_document.test.json
}


data "aws_iam_policy_document" "test" {
  policy_id = "KMSPolicy"

  statement {
    sid    = "Root User Permissions"
    effect = "Allow"
    principals {
      type = "AWS"
      identifiers = [
      "arn:${data.aws_partition.current.partition}:iam::${data.aws_caller_identity.current.account_id}:root"]
    }
    actions = [
    "kms:*"]
    resources = ["*"]
  }

  statement {
    sid    = "EC2 kms permissions"
    effect = "Allow"
    principals {
      type        = "AWS"
      identifiers = [aws_iam_role.test.arn]
    }
    actions = [
      "kms:Encrypt",
      "kms:Decrypt",
      "kms:GenerateDataKey",
    "kms:GenerateDataKeyPair"]
    resources = ["*"]
  }
}

resource "aws_iam_role" "test" {
  name = %[1]q

  assume_role_policy  = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "ecs.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
  managed_policy_arns = ["arn:${data.aws_partition.current.partition}:iam::aws:policy/service-role/AmazonECSInfrastructureRolePolicyForServiceConnectTransportLayerSecurity"]
}

resource "aws_service_discovery_http_namespace" "test" {
  name = %[1]q
}

resource "aws_ecs_cluster" "test" {
  name = %[1]q
}

resource "aws_ecs_task_definition" "test" {
  family       = %[1]q
  network_mode = "bridge"

  container_definitions = <<DEFINITION
[
  {
    "cpu": 128,
    "essential": true,
    "image": "mongo:latest",
    "memory": 128,
    "name": "mongodb",
    "portMappings": [
    {
      "hostPort": 0,
      "appProtocol": "http",
      "containerPort": 27017,
      "name": "tf-test",
      "protocol": "tcp"
    }
    ]
  }
]
DEFINITION
}

resource "aws_ecs_service" "test" {
  name            = %[1]q
  cluster         = aws_ecs_cluster.test.id
  task_definition = aws_ecs_task_definition.test.arn
  desired_count   = 1

  service_connect_configuration {
    enabled   = true
    namespace = aws_service_discovery_http_namespace.test.arn

    log_configuration {
      log_driver = "json-file"

      options = {
        key = "value"
      }
    }

    service {
      client_alias {
        dns_name = "example.com"
        port     = 8080
      }

      discovery_name        = "test"
      ingress_port_override = 8443
      port_name             = "tf-test"
      tls {
        issuer_cert_authority {
          aws_pca_authority_arn = aws_acmpca_certificate_authority.test.arn
        }
        kms_key  = aws_kms_key.test.arn
        role_arn = aws_iam_role.test.arn
      }
      timeout {
        idle_timeout_seconds        = 120
        per_request_timeout_seconds = 60
      }
    }
  }
}

resource "aws_acmpca_certificate_authority_certificate" "test" {
  certificate_authority_arn = aws_acmpca_certificate_authority.test.arn

  certificate       = aws_acmpca_certificate.test.certificate
  certificate_chain = aws_acmpca_certificate.test.certificate_chain
}

resource "aws_acmpca_certificate" "test" {
  certificate_authority_arn   = aws_acmpca_certificate_authority.test.arn
  certificate_signing_request = aws_acmpca_certificate_authority.test.certificate_signing_request
  signing_algorithm           = "SHA512WITHRSA"

  template_arn = "arn:${data.aws_partition.current.partition}:acm-pca:::template/RootCACertificate/V1"

  validity {
    type  = "YEARS"
    value = 1
  }
}

resource "aws_acmpca_certificate_authority" "test" {
  permanent_deletion_time_in_days = 7
  type                            = "ROOT"
  usage_mode                      = "SHORT_LIVED_CERTIFICATE"
  certificate_authority_configuration {
    key_algorithm     = "RSA_4096"
    signing_algorithm = "SHA512WITHRSA"

    subject {
      common_name = %[1]q
    }
  }
  tags = {
    AmazonECSManaged = "true"
  }
}

data "aws_partition" "current" {}
data "aws_caller_identity" "current" {}

Steps to Reproduce

Enable TLS configuration on aws_ecs_service with provider 5.37.0

Debug Output

Uploading tf_tarce.log…

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[dgr237]

@jihed I have tested the provider and I don't see the same issue. I have had to do some tweaks to the tf to get my task running (as the TF does not create a container instance for it to run) but I do not get an error that you are seeing.

[github-actions]

This functionality has been released in v5.41.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.