Downscaling causes whole stack to shutdown

[umurkontaci]

If you start a compose stack with one of the items with n nodes and then downgrade to anything <n nodes, the console will output that one of the nodes had quit with error code 137 and gracefully stop everything.

However, if you start with n nodes, upscale to >n nodes then downscale to n nodes, everything works as intended and the stack will not crash.

Example:

---

web:
  build: ./web
  expose:
    - "3000"
  restart: always
  environment:
    JOB: web
  links:
    - soa
lb:
  build: ./lb
  ports:
    - "80:80"
  restart: always
  environment:
    JOB: lb
  links:
    - web
    - soa
soa:
  build: ./soa
  environment:
    JOB: soa
  restart: always
  expose:
    - "8301"
    - "8302"
    - "8400"
    - "8500"
    - "8600"

Then (everything is scale=1)

docker-compose up
docker-compose scale web=10
docker-compose scale web=1
docker-compose scale web=10
docker-compose stop

This works, scales, and stops as expected. However after the previous command (that is, when the last scale value for web is 10), if you execute the following

docker-compose up
docker-compose scale web=1

This will say somenode_1 exited with code 137 and gracefully stop the whole thing.

Is this the intended behavior?

[brianlund]

I'd like to know also, it was unexpected for me and I can't seem to find that behavior described

[aanand]

It sounds like it's the behaviour of docker-compose up when run interactively that's causing this. Definitely a bug. We want to change that behaviour anyhow - see #741.

Does this happen when containers are run in the background with docker-compose up -d?

[brianlund]

No. I have tested and it only happens when you do not run in detached mode.

[aanand]

Right, as I suspected.

I think we should definitely not shut down containers in docker-compose up when a container has died. As for when the user presses Ctrl-C, that can be another discussion.

[umurkontaci]

I think if we start docker-compose up on the foreground (without -d), SIGINT, SIGSTOP, SIGKILL etc. should also propagate the signal to the containers. I believe most of the startup process managers like upstart and supervisor relies on that behavior by default.

However, if a container dies while there are at least one more of the same type (that is, scaled to >1), then I think the rest of the containers should continue. But there might be a case where a setup would expect at least N of the same containers (scaled to N) to be running, then it might make sense if one container dies, the whole stack dies as well.

[dnephin]

Fixed by #1754