provider/aws: Support ephemeral volumes in ecs_task_definition

[radeksimko]

via @jbrook from #1803 (comment)

I am playing with the ECS functionality and I ran into an issue with specifying volumes in a task definition. It should be possible to pass an empty host in which case the volume is not made persistent - the docker daemon manages its location and garbage collection:

http://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_data_volumes.html

Although closed & merged PRs aren't the best place to report issues (new one is almost always better), thanks for bringing this up, @jbrook 👍

It will be necessary to look into the SDK to confirm my theory, but I reckon the solution is just making host_path optional and leaving it out from API request in those cases.
Either way I feel this one should be fairly easy to fix.

[radeksimko]

Even though both the SDK & docs say it should allow an empty parameter, API keeps returning exception:

* ClientException: host.sourcePath should not be blank.

type Volume struct {
    // The path on the host container instance that is presented to the containers
    // which access the volume. If this parameter is empty, then the Docker daemon
    // assigns a host path for you.
    Host *HostVolumeProperties `locationName:"host" type:"structure"`

I'm not sure how to address that problem. I will probably have to isolate that piece of code and raise it with AWS.

[tj]

Awesome job getting that stuff in! Stoked to try it out.

Looking at it from the outside (haven't read all the threads in the PR) it seems odd that the abstraction stopped at volumes, why not have the whole task def in declarative form? Or the entire task def as JSON?

[radeksimko]

Looking at it from the outside (haven't read all the threads in the PR) it seems odd that the abstraction stopped at volumes, why not have the whole task def in declarative form? Or the entire task def as JSON?

I was actually thinking about that approach, but then I decided to mimic approach of other tools (e.g. AWS CLI) to make things easily portable - i.e. TD/Service definition in JSON is the building block you can eventually reuse between any tools that share that approach.
I'm not against the idea of having everything in DSL, but I don't see much benefits, so the extra effort (not only implementation, but maintenance too) is just not worth it.

The API AFAIK doesn't allow updating only certain fields - i.e. we have to send the whole definition to the API anyway, that's just the smallest building block recognised by API. The reason Go SDK crumbled the JSON completely is IMO merely because it's machine-generated code.

btw. Terraform doesn't expand IAM definitions either, even though it sometime causes problems.
Those decisions may be revisited as the time goes though. Nothing is set in the stone. 😉

[tj]

True yea good point, not dissimilar to the others – that makes sense with the CLI parity too I didn't notice that they made that split there, seems arbitrary but they must have a reason :D

[radeksimko]

See #3810

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.