ECS services_stable waiter does not await load balance registration.

[jstewmon]

The services_stable waiter succeeds when the number of deployments is 1 and the runningCount matches the desiredCount. Those conditions are satisfied before a container instance is registered with a load balancer (for services having a load balancer).

    "ServicesStable": {
      "delay": 15,
      "operation": "DescribeServices",
      "maxAttempts": 40,
      "acceptors": [
        {
          "expected": "MISSING",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "failures[].reason"
        },
        {
          "expected": "DRAINING",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "services[].status"
        },
        {
          "expected": "INACTIVE",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "services[].status"
        },
        {
          "expected": true,
          "matcher": "path",
          "state": "success",
          "argument": "services | [@[?length(deployments)!=`1`], @[?desiredCount!=runningCount]][] | length(@) == `0`"
        }
      ]
    }

I do not see any formal attribute that indicates when an ECS Service with a load balancer stabilized, but the first event message will end with has reached a steady state. following successful registration.

Can we check the event message with the jmespath expression to determine when the service has stabilized?

I found this expression produces the expected result:

services | [@[?length(deployments)!=`1`], @[?desiredCount!=runningCount], @[?!ends_with(events[0].message, 'reached a steady state.')]][] | length(@) == `0`

I suppose checking the event message actually obviates the need for the first two checks, so it could be simplified as:

services[?!ends_with(events[0].message, 'reached a steady state.')][] | length(@) == `0`

If checking the event message in the waiter is unacceptable, do you have any suggestions on how to use the ECS / ELB APIs to make this check?

[jstewmon]

@jamesls any thoughts on this?

[kyleknap]

Seems like it could work. One thing that I am a little hesitant about is if we can always rely on that message being there (and it is part of the first event) and that the message value will never change. Seems like a lot of assumptions, but if these assumptions can be confirmed then I think we can look to updating it. I think we need to do research though into what is the for-sure qualifier for when a ECS service is considered stable.

[jstewmon]

@kyleknap thanks for the response. I share your reservations about relying on the event message text...

I filed a support case for which I think an inquiry as been submitted to the ECS team to provide advice on this issue.

Would it help if I shared that case number with you (so as not to duplicate inquiries)?

[kyleknap]

No worries about the case number. We will probably be looped in at some point with the case you opened. Thanks for driving this!

[tim-finnigan]

Checking in as this issue hasn't received any comments in a few years. Waiter models are now owned by service teams as those model definitions are used across AWS SDKs. To request waiter updates going forward we recommend reaching out through AWS Support or creating an issue here in our cross-SDK repository: https://github.com/aws/aws-sdk/issues.