Skip to content
This repository has been archived by the owner on Feb 28, 2024. It is now read-only.

Update docker in production to something more recent #1

Closed
iby opened this issue Jun 8, 2015 · 24 comments
Closed

Update docker in production to something more recent #1

iby opened this issue Jun 8, 2015 · 24 comments

Comments

@iby
Copy link

iby commented Jun 8, 2015

Not sure if this is the right place to ask, I've contacted the support on this recent, but it went silent. Please consider updating docker to something more recent. Circle right now runs 1.4.1, the most recent version is 1.6.2 which is light years ahead.

@notnoopci
Copy link

Sorry about that. We are planning to update pre-instlaled Docker version very soon. Meanwhile, you can install 1.6.2 binary as part of the build by adding the following steps to circle.yml

machine:
  pre:
    - sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci' && sudo chmod 0755 /usr/bin/docker

[You may need to remove docker from the machine.services section]

Let me know how it goes.

@iby
Copy link
Author

iby commented Jun 8, 2015

Not much luck, if I do just that I end up with this problem. If I add docker -d it all goes down with fire:

sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
sudo docker -d
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

100 14.7M  100 14.7M    0     0  28.6M      0 --:--:-- --:--:-- --:--:-- 29.0M
INFO[0000] +job init_networkdriver()                    
INFO[0000] +job serveapi(unix:///var/run/docker.sock)   
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock) 
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1 
INFO[0000] -job init_networkdriver() = OK (0)           
WARN[0000] mountpoint for memory not found              
FATA[0000] Shutting down daemon due to errors: Error loading docker apparmor profile: exit status 243 (Warning from profile docker-default (docker) ptrace rules not enforced
Warning from profile docker-default (docker) signal rules not enforced
Warning from profile docker-default (docker) mount rules not enforced
Unable to open /sys/kernel/security/apparmor/.replace - Permission denied
/sbin/apparmor_parser: Unable to replace "docker-default".  Permission denied; attempted to load a profile while confined?
Warning failed to create cache: docker

sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
sudo docker -d
 returned exit code 1

Action failed: sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
) sudo docker -d

@iby
Copy link
Author

iby commented Jun 8, 2015

Though. That's strange. It seems the error is also with downloading / replacing the docker? When I ran this for the first time it didn't argue:

sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
sudo chmod 0755 /usr/bin/docker
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

100 14.7M 100 14.7M 0 0 29.9M 0 --:--:-- --:--:-- --:--:-- 30.4M

@notnoopci
Copy link

I believe docker will already be auto-started as part of the apt-get/deb install - so you don't need to invoke it directly yourself.

Also, in CircleCI, we only support btrfs storage and lxc execution engine; so the command is sudo docker -d -e lxc -s btrfs (but it will bail out probably if the service is up already).

@iby
Copy link
Author

iby commented Jun 8, 2015

Did the trick:

machine:
  pre:
    - |
      sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci'
      sudo chmod 0755 /usr/bin/docker
      sudo start docker

@notnoopci
Copy link

Thanks! I'll keep it in mind next I time I suggest it! Let us know how else we can help!

@iby
Copy link
Author

iby commented Jun 8, 2015

Sadly updating it didn't help. I think it's the lxc execution engine you've mentioned. I'm trying to debug the container with docker exec -ti and get:

FATA[0000] Error response from daemon: Unsupported: Exec is not supported by the lxc driver 

Assume there's no workaround for that? Is there any other way?

@notnoopci
Copy link

I'm afraid that docker exec isn't supported by lxc - and we cannot support native (a.k.a. libcontainer) yet. We are planning to support docker exec - but for the time-being you can use the workaround documented at https://circleci.com/docs/docker#docker-exec in CircleCI builds . Would that work you?

@iby
Copy link
Author

iby commented Jun 8, 2015

That is perfect, thanks so much for pointing this out! I can attach in the same way as with docker exec -ti … bash and do what I need. There's a typo in the docs, missing use:

To work around this, you can use the following command, customized for your container name and the command you want to run, using LXC directly:

@Dynom
Copy link

Dynom commented Jul 5, 2015

I need a more recent version too (to specify an alternate Dockerfile). This ticket helped me with setting it up. The image building is different from our development situation vs our online environments, mostly mounting local paths at runtime vs embedding artifacts in the images.

Docker 1.6 offer the features we currently need.

Our circle.yml starts like this:

machine:
  pre:
    - |
      test $(docker --version | awk '{ print $3 }' | head -c3) != "1.6" && {
        sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci';
        sudo chmod 0755 /usr/bin/docker;
        sudo start docker;
      }
# Restore this once docker >1.6.2 is available.
#  services:
#    - docker

Builds start failing once 1.6.x becomes available on Circle, in which we restore the services section.

@notnoopci
Copy link

@Dynom fwiw - you can change it to make it work every after >1.6.2 becomes available (planned for the next few days)

machine:
  pre:
    - |
      test $(docker --version | awk '{ print $3 }' | head -c3) != "1.6" && {
        sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.6.2-circleci';
        sudo chmod 0755 /usr/bin/docker;
      }
    - sudo service docker start
# Restore this once docker >1.6.2 is available.
#  services:
#    - docker

@Dynom
Copy link

Dynom commented Jul 7, 2015

Hi @notnoopci,

Thanks for the comment! I want it to fail because then I know when I can cleanup my circle.yml and notify my team that you've updated Docker on your hosts. So it was intentional this time, thanks for the tip though! (-:

@Dynom
Copy link

Dynom commented Jul 9, 2015

At voila, Circle supports Docker 1.6 now! Fixing my Circle file (-;

affablebloke added a commit to affablebloke/chatops that referenced this issue Aug 5, 2015
@notnoopci
Copy link

Just pushed a docker 1.8.1 version - and could use someone testing it first. To enable it you can run with:

machine:
  pre:
    - |
      sudo curl -L -o /usr/bin/docker 'http://s3-external-1.amazonaws.com/circle-downloads/docker-1.8.1-circleci'
      sudo chmod 0755 /usr/bin/docker
      sudo service docker start

(but without machine.services.docker line)

@bromanko
Copy link

I can confirm that this solves an issue I was having with docker-compose and docker build.

@mrfelton
Copy link

I can't get docker-compose to start with this version in conjunction with docker-compose 1.4.0

Traceback (most recent call last):
File "", line 3, in
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 39, in main
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command
File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 407, in start
File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 198, in start
File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 132, in start
File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 110, in containers
File "/code/build/docker-compose/out00-PYZ.pyz/docker.client", line 202, in containers
File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 86, in _get
File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 477, in get
File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 465, in request
File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 573, in send
File "/code/build/docker-compose/out00-PYZ.pyz/requests.adapters", line 433, in send
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=120)

@neilellis
Copy link

I can confirm this also works for me using docker-compose 1.3.3

@neilellis
Copy link

Spoke too soon:

docker-compose up && [ -f ./docker-test/test.txt ]
Pulling test1 (nginx:latest)...
latest: Pulling from library/nginx













Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 39, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 495, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.project", line 265, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 369, in execute_convergence_plan
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.service", line 440, in start_container
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.container", line 136, in start
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.utils.decorators", line 20, in wrapped
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.client", line 797, in start
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 128, in _post_json
  File "/code/build/docker-compose/out00-PYZ.pyz/docker.clientbase", line 83, in _post
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 508, in post
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 465, in request
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.sessions", line 573, in send
  File "/code/build/docker-compose/out00-PYZ.pyz/requests.adapters", line 433, in send

requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60) docker-compose up && [ -f ./docker-test/test.txt ] returned exit code 255

@ecstasy2
Copy link

ecstasy2 commented Sep 1, 2015

Having the same issue.

requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60) (cd test/e2e && docker-compose up -d) returned exit code 1

@gitu
Copy link

gitu commented Sep 10, 2015

Hi @notnoopci

When using this version (Docker version 1.8.1-circleci, build 076fb94)
The following errors occur when building my image:

...
---> caa0020ada3f
Step 1 : MAINTAINER Florian Schrag <[email protected]>
 ---> Running in 0ea869e2fe39
 ---> 831642999a7e
Error removing intermediate container 0ea869e2fe39: Cannot destroy container 0ea869e2fe391fa9cb79db137e918d5b52ba9b33f512cec1f084c9553acde6ea: Driver btrfs failed to remove root filesystem 0ea869e2fe391fa9cb79db137e918d5b52ba9b33f512cec1f084c9553acde6ea: Failed to destroy btrfs snapshot: operation not permitted
Step 2 : VOLUME /imports
 ---> Running in 650c7eb0128a
 ---> b6faca93db81
Error removing intermediate container 0ea869e2fe39: no such id: 0ea869e2fe391fa9cb79db137e918d5b52ba9b33f512cec1f084c9553acde6ea
...

@skippy
Copy link

skippy commented Sep 10, 2015

I'm having the same issue with docker-1.8.1-circleci and docker-compose 1.4.0. I can get the default circle-ci docker bin (v 1.6.1) and docker-compose 1.3.3 to work, but not the latest combination.

@gitu I believe that is 'ok' behavior here. There is a note somewhere in the circle-ci docker documentation about lxc not being allowed to delete intermediate containers... but that shouldn't prevent a successful build

arm4b pushed a commit to arm4b/st2-packages that referenced this issue Oct 21, 2015
kimh pushed a commit that referenced this issue Nov 4, 2015
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.

Quoting MkdirAll documentation:

> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.

This means two things:

1. If a directory to be created already exists, no error is returned.

2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.

The above is a theory, based on quoted documentation and my UNIX
knowledge.

3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.

Because of #1, IsExist check after MkdirAll is not needed.

Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.

Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.

[v2: a separate aufs commit is merged into this one]

[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <[email protected]>
@minodisk
Copy link

Is v1.9.0 ready?

@kimh
Copy link

kimh commented Nov 10, 2015

yup!

@minodisk
Copy link

@kimh 👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests