Add support for launch templates to aws_spot_fleet_request

[WheresWardy]

Now that support for EC2 launch templates has been added to Terraform via #2927, I'd like to request support for this be added to the aws_spot_fleet_request resource.

At the moment, the only way to add multiple variations of instances to spot fleet requests is with individual launch_specification parameters. This involves a lot of duplication of aws_instance parameters that are the same when you may just want to change the instance type across multiple launch specifications for example.

Launch templates will add the ability to supply the majority of consistent parameters via the template.

Terraform Version

Terraform v0.11.7
+ provider.aws v1.15.0
+ provider.template v1.0.0

Affected Resource(s)

Please list the resources as a list, for example:

• aws_spot_fleet_request

[gordonbondon]

I'm working on supporting launch templates in instance and asg resources. Spot fleet request support looks similar so I might as well work on it. PR should a arrive somewhere next week.

[ingwarsw]

@gordonbondon Working on that maybe?
Would be nice to have that options..

[gordonbondon]

@ingwarsw I'm waiting for #4543 to get merged so we can settle on how this launch_template option is implemented and then I'll add the same for spot fleet.

[yanivomc]

Thank you @gordonbondon this would be awesome!

[gordonbondon]

Looks like someone got to it before me #4866

[lossanarch]

Oh sorry @gordonbondon, hadn't seen this issue and that you were working on it. If anyone has any ideas on how to get the testing assertions against the random-ish numbers in the state working (see WIP section of #4866) I can finish up the PR.

[Jamie-BitFlight]

Would you be down for having the overrides being a list of maps?
So they can be passed in dynamically in version 0.11?

[lossanarch]

This is already implemented in #4866 and is exactly how I use it @Jamie-BitFlight
For example:

launch_template_configs {
    launch_template_specification {
      id      = "${aws_launch_template.ecs.id}"
      version = "${aws_launch_template.ecs.latest_version}"
    }

    overrides = ["${data.null_data_source.overrides.*.outputs}"]
  }

This actually works with any resource attr that isn't set as "required" when multiple map blocks can be passed in:

launch_template_configs {
    launch_template_specification {
      id = "${aws_launch_template.ecs_instance.id}"
      version = "${aws_launch_template.ecs_instance.latest_version}"
    }
    overrides {
      instance_type = "t2.micro"
      spot_price = "0.05"
    }
    overrides {
      instance_type = "t2.medium"
      spot_price = "0.07"
    }
  }

Such structures always allow a list of maps to be passed in as long as they're not set to required.
The problem with the ones set as required is that the input is validated before interpolation so they error out. Whenever it's not set as required though, you can pass in a list of maps dynamically. I really hope this gets addressed with HCL2.

If you're interested in getting the completed PR merged (it's finished now as long as no changes need to be made after review) please thumbs up or comment it or something so that the team can gauge interest.

[four43]

The Launch Template support in EC2 Fleets seems to work really well. Is that direction AWS seems to be going with provisioning like this (instead of spot fleets?) It sounds like Auto Scaling isn't working for those groups yet.

Either way I think that syntax would work well in for aws_spot_fleet_requests. This seems to match Amazon's API/JSON structure for defining those resources. The current way to do it is very verbose.

[almlys]

Is there are update or workaround to have it working with Terraform? I have searched a lot on the web and seems that the only solution is to keep with the "launch_specification" and we would prefer to use the "launch_template" as it suits better with our use case.

[WheresWardy]

As an update, since AWS added support for launch templates and spot request pools to ASGs, we've switched to using these instead, as we get all the extra benefits of ASGs over spot fleet requests and it can all be done in Terraform too.

[deitch]

@WheresWardy , how did you do it in terraform?

[WheresWardy]

@deitch We created an aws_launch_template to describe some fairly generic things about the instance (image_id, user_data, block_device_mappings and tags) and then used the "Mixed Instances Policy" support in aws_autoscaling_group to attach the launch_template and specify overrides for the instance types, and instances_distribution to control the spot aspect of the ASG. The terraform config for the ASG itself allows you to do the more generic networking parts, like VPC subnets without the issues we encountered with spot fleets.

[deitch]

Nicely done @WheresWardy . Thanks!

[thaiphv]

@gordonbondon when will you add support for using launch templates in a aws_spot_fleet_request?

[gordonbondon]

@thaiphv please check comments above. As I've mentioned, this has already been done in #4866

[thaiphv]

@gordonbondon thanks. I think we need to update the documentation of the spot_fleet_request resource.

[gammasts]

Just fyi, this isn't actually done, just that PR #4866 had been opened. It seems to be awaiting changes and another review.

[scalp42]

@WheresWardy so what would be the difference between ASG and Spot Fleet in your scenario? I believe Spot Fleet is that only one that can on-demand instances mixed with spot ones correct?

[WheresWardy]

@scalp42 This issue predates the ability to have mixed-instance-type spot fleets in ASGs, but since this is now supported in ASGs (and ASGs have support for launch templates in terraform) we've since switched to using ASGs, so this issue doesn't affect us anymore. They also support mixing on-demand with spot now too.

[DrFaust92]

addressed in #12732

[bflad]

Support for this functionality has been merged and will release with version 2.60.0 of the Terraform AWS Provider, later this week. Thanks to @lossanarch and @DrFaust92 for the implementation. 👍

[ghost]

This has been released in version 2.60.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!