[Bug]: aws_elasticache_replication_group recreates resource on engine_version upgrade

[madpipeline]

Terraform Core Version

v1.3.5

AWS Provider Version

v4.40.0

Affected Resource(s)

• aws_elasticache_replication_group

Expected Behavior

When upgrading the aws_elasticache_replication_group engine_version from any version to any version to upgrade the resource in-place, just like in the AWS Console.

Actual Behavior

When upgrading from version 2.8.24 to any other higher version, the plan wants to destroy and recreate the resource.

Upgrading from any other higher version to a higher version works as expected.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

provider "aws" {
  region = var.aws_region
}

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = local.prefix
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
  elasticache_subnets = ["10.0.31.0/24", "10.0.32.0/24"]

  enable_nat_gateway = true
  create_elasticache_subnet_group = true

  tags = local.common_tags
}

resource "aws_elasticache_replication_group" "session_mgm" {
  replication_group_id  = local.prefix
  description           = local.prefix
  node_type             = "cache.t2.micro"
  number_cache_clusters = "1"
  port                  = "6379"
  engine                = "redis"
  engine_version        = "6.2" # "3.2.6" # "2.8.24"

  availability_zones         = ["eu-west-1a"]
  automatic_failover_enabled = false
  subnet_group_name          = module.vpc.elasticache_subnet_group_name
  # security_group_ids         = [var.default_sg]
  parameter_group_name       = "default.redis6.x" # "default.redis3.2" # "default.redis2.8"

  auto_minor_version_upgrade = "true"
  maintenance_window         = "sun:03:00-sun:04:00"
  snapshot_window            = "00:00-01:00"
  snapshot_retention_limit   = "1"
  apply_immediately          = "false"

  tags = local.common_tags
}

Steps to Reproduce

• Deploy the provided snippet with engine version set to 2.8.24
• Change the engine version to any higher version
• run terraform plan

The plan will attempt to destroy-recreate the resource instead of upgrading in place.

Debug Output

No response

Panic Output

No response

Important Factoids

According to the CloudFormation documentation on the Engine version, the upgrade should happen in place, with no down time.

References

• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticache-replicationgroup.html#cfn-elasticache-replicationgroup-engineversion

Would you like to implement a fix?

Yes

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[apanzerj]

I'm seeing this issue when upgrading from 6 => 7 as well.

It does a drop and replace.

[josacar]

I was able to replicate the bug:

package main

import (
	"fmt"
	gversion "github.com/hashicorp/go-version"
	"math"
	"regexp"
)

const (
	redisVersionPreV6RegexpRaw  = `[1-5](\.[[:digit:]]+){2}`
	redisVersionPostV6RegexpRaw = `(([6-9])\.x)|([6-9]\.[[:digit:]]+)`

	redisVersionRegexpRaw = redisVersionPreV6RegexpRaw + "|" + redisVersionPostV6RegexpRaw
)

const (
	redisVersionRegexpPattern       = "^" + redisVersionRegexpRaw + "$"
	redisVersionPostV6RegexpPattern = "^" + redisVersionPostV6RegexpRaw + "$"
)

var (
	redisVersionRegexp       = regexp.MustCompile(redisVersionRegexpPattern)
	redisVersionPostV6Regexp = regexp.MustCompile(redisVersionPostV6RegexpPattern)
)

func normalizeEngineVersion(version string) (*gversion.Version, error) {
	if matches := redisVersionPostV6Regexp.FindStringSubmatch(version); matches != nil {
		if matches[1] != "" {
			version = fmt.Sprintf("%s.%d", matches[2], math.MaxInt)
		} else if matches[3] != "" {
			version = matches[3]
		}
	}
	return gversion.NewVersion(version)
}

func main() {
	oVersion, _ := normalizeEngineVersion("6.x")
	nVersion, _ := normalizeEngineVersion("6.0")

	print(nVersion.LessThan(oVersion))
}

This return true as x is 'higher' than any digit.

When comparing two versions:

func main() {
        oVersion, _ := normalizeEngineVersion("6.0")
        nVersion, _ := normalizeEngineVersion("6.2")

        print(nVersion.LessThan(oVersion))
}

Works as expected returning false

[josacar]

Looks like changing this:

const (
  redisVersionRegexpPattern       = "^" + redisVersionRegexpRaw + "$"
  redisVersionPostV6RegexpPattern = "^" + redisVersionPostV6RegexpRaw + "$"
)

to:

const (
  redisVersionRegexpPattern       = "^(?:" + redisVersionRegexpRaw + ")$"
  redisVersionPostV6RegexpPattern = "^(?:" + redisVersionPostV6RegexpRaw + ")$"
)

works fine, as with the previous implementation 2.8.12 will be detected as 8.12 as minor is higher or equal than 6

[github-actions]

This functionality has been released in v5.22.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.