feature request: make container startup timeout configurable

[jameslamb]

Description

canary validate fails if the container takes more than 10 seconds to start up.

That 10 second threshold is currently hard-coded.

container-canary/internal/container/docker.go

Lines 76 to 82 in a4f5c79

	if time.Since(startTime) > (time.Second * 10) {
	err := c.Remove()
	if err != nil {
	return err
	}
	return errors.New("container failed to start after 10 seconds")
	}

That timeout should be configurable.

Benefits of this work

Would allow the use of canary validate with images that take longer than 10 seconds to start up, for example:

• containers that do some heavy work on startup, like starting a process and polling until it passes health checks)
• docker daemon that is running slowly, e.g. because it's performing I/O on a network filesystem or is in some other way resource-constrainted

Acceptance Criteria

• it's possible to modify, via command-line argument(s), how long canary validate waits for a container to start up before timing out

Approach

Modify this

container-canary/internal/container/docker.go

Lines 76 to 82 in a4f5c79

	if time.Since(startTime) > (time.Second * 10) {
	err := c.Remove()
	if err != nil {
	return err
	}
	return errors.New("container failed to start after 10 seconds")
	}

such that that 10 second timeout can be altered via configuration.

For example, I'd like to be able to run the following:

canary validate \
   --file ./checks.yaml \
   --startup-timeout 30 \
   ${IMAGE_URI}

Notes

Created this after observing this exact timeout while testing RAPIDS images over in rapidsai/docker#670.

[jacobtomlinson]

Yup this sounds great!