inputs.ecs: add v3 metadata support.

[sergeykhegay]

Justification: v2 metadata and Docker stats are available to tasks that use the awsvpc network mode ... only [v2]. v3 metadata is available regardless of the network mode and provides the same statistics as the v2 metadata.

[v2] https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v2.html

Required for all PRs:

• Signed CLA.
• Associated README.md updated.
• Has appropriate unit tests.

[sergeykhegay]

Could you take a look, @danielnelson?

[sergeykhegay]

Maybe you could peek, @ssoroka?

[ssoroka]

Looks reasonable. 👍

[danielnelson]

I think having both endpoint_url and metadata_version options introduce option combinations that you wouldn't normally want and would make the plugin more difficult to setup. For example, during normal use you would never want to set metadata_version = 3 and leave the endpoint_url as the default value. Likewise setting an empty endpoint_url = "" and metadata_version = 2 wouldn't work either.

What if we removed the metadata_version and on startup we try to read the ECS_CONTAINER_METADATA_URI envvar, if this is set we use v3 and if not we try with v2 and the endpoint_url. We should update the endpoint_url option to mention that it is only for use with v2.

With this approach most users will be bumped forward to v3, so the idea requires that v3 needs to show the same results as v2. I believe this is the case. If one wanted to force v2, it would always be possible to unset the environment variable.

[danielnelson]

Nitpick but I prefer the old style:

c.statsURL = c.BaseURL.ResolveReference(getMetaStatsPath(c.version)).String()

[sergeykhegay]

Unfortunately this doesn't work with v3 metadata endpoint.

The url for v3 looks something like http://1.1.1.1/v3/metadata (I do not recall exact path, but the important part is that the path is present. IP is not relevant). When you use ResolveReference on such base URL, the path you provide will override the the original path. I faced this issue when I was testing the code in ECS.

A small demo:

package main

import (
	"fmt"
	"log"
	"net/url"
)

func main() {

	examples := []struct{
		base string
		path string
	} {
		{
			base: "http://1.1.1.1/v3/metadata",
			path: "/stats",
		},
		{
			base: "http://1.1.1.1/v3/metadata",
			path: "stats",
		},
	}
	
	for i, e := range examples {
		u, err := url.Parse(e.path)
		if err != nil {
			log.Fatal(err)
		}
		base, err := url.Parse(e.base)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println("Example ", i)
		fmt.Println(" base:   ", e.base)
		fmt.Println(" path:   ", e.path)
		fmt.Println(" result: ", base.ResolveReference(u))
		fmt.Println()
	}
}

Result:

Example  0
 base:    http://1.1.1.1/v3/metadata
 path:    /stats
 result:  http://1.1.1.1/stats

Example  1
 base:    http://1.1.1.1/v3/metadata
 path:    stats
 result:  http://1.1.1.1/v3/stats

[sergeykhegay]

Just noticed the change request. Will fix it. This is not abandoned.

[snemetz]

There is also metadata v4 now with a new variable name.
Would be nice if this could handle any of the versions

[danielnelson]

@snemetz Thanks for the heads up, and I sure hope these version changes settle down. It would be good to support all available versions if we can. Could you open a feature request issue for v4 metadata?

[snemetz]

v4 feature request created

[sergeykhegay]

@danielnelson, pushed requested changes. Please take a look.

[sergeykhegay]

@danielnelson did you have a chance to peek again?

[danielnelson]

Thank you!