Struggling to use data volume

[anentropic]

"You need to ensure the postgres_data directory has sufficient permissions for the docker process to read / write it."

$ mkdir .postgres_data
$ chmod -R 777 .postgres_data/
$ fig up
postgres_1 | Creating Postgres data at /var/lib/postgresql/9.3/main
postgres_1 | mkdir: cannot create directory '/var/lib/postgresql/9.3/main': Permission denied
postgres_1 | ls: cannot access /var/lib/postgresql/9.3/main: No such file or directory
postgres_1 | Initializing Postgres Database at /var/lib/postgresql/9.3/main
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.

The ./postgres_data/9.3 dir gets created but it fails as above

How do I do this?

[anentropic]

The advice seems to be to use a data volume container, rather than a dir on the host:
boot2docker/boot2docker#581

But this has it's own difficulties...

eg in fig.yml I have

postgres:
  image: kartoza/postgis
  volumes_from:
    - postgresdata
  expose:
    - 5432

postgresdata:
  image: dockerfile/ubuntu
  volumes:
    - /var/lib/postgresql

but I still get:

postgres_1 | Creating Postgres data at /var/lib/postgresql/9.3/main
postgres_1 | mkdir: cannot create directory '/var/lib/postgresql/9.3': Permission denied
postgres_1 | ls: cannot access /var/lib/postgresql/9.3/main: No such file or directory

it seems related to moby/moby#6137 though that is marked fixed

some non-fig experiments:

$ docker run -it --volumes-from 505013795eeb kartoza/postgis bash
postgres@0ce4adaa145e:/$ ls -la /var/lib
drwxr-xr-x  2 root    root    4096 Dec  4 06:51 postgresql

$ docker run -it kartoza/postgis bash
postgres@901104399bd3:/$ ls -la /var/lib
drwxr-xr-x  4 postgres postgres 4096 Dec  3 04:41 postgresql

in other words postgres expects this dir to be owned by postgres:postgres

ok, so what if I make a Dockerfile like this for my data volume container:

FROM dockerfile/ubuntu

MAINTAINER anentropic

RUN useradd postgres

VOLUME /var/lib/postgresql

COPY fix_perms.sh /home/docker/
RUN chmod +x /home/docker/fix_perms.sh

CMD ["/home/docker/fix_perms.sh"]

and fix_perms.sh looks like:

#!/bin/sh

chown -R postgres:postgres /var/lib/postgresql

then I get:

$ docker run -it --volumes-from 485f702784ee kartoza/postgis bash
postgres@36f6aaf544c7:/$ ls -l /var/lib
drwxr-xr-x  2    1000    1000 4096 Dec  4 07:32 postgresql

but initdb still fails with a permission error

It looks to me like something in kartoza/postgis should be doing a chown on /var/lib/postgresql otherwise this can't work

[anentropic]

your Dockerfile has:

USER postgres
CMD /start-postgis.sh

adding chown -R postgres:postgres /var/lib/postgresql to start-postgis.sh doesn't work because the script is run as postgres user which doesn't have enough permissions

it has to happen in the CMD i.e. runtime rather than in a build-time RUN step because it's only at run-time that the permissions of the newly mounted volume can be overwritten

[anentropic]

I made a workaround here: anentropic@e465078

what do you think of this approach?

[timlinux]

Hi @anentropic

I used a host volume (not a volume container though that should work fine too) successfully both with fig and with bash scripts to orchestrate. e.g. https://github.com/AIFDR/inasafe-django uses fig and host volume.

Note that fig has an irritation in that I don't see how to get it to wait for the db cluster to be initialised before it brings up subsequent containers.

Using your approach seems fine - we can probably fix the SSL permissions error in that script too (see #3 )

Feel free to make a PR and I will apply it.

Regards

Tim

[anentropic]

thanks, I think the problem with host volume may be specific to running under boot2docker (even with /perms_wrapper.sh it fails at Creating Postgres data at /var/lib/postgresql/9.3/main mkdir: cannot create directory '/var/lib/postgresql/9.3/main': Permission denied)

AFAIK fig will use the links and volumes_from declarations to build a dependency graph to determine which order to bring up the containers (docker/compose#235)... but of course this won't wait for any initialisation run by CMD to complete. I guess in that case the dependent containers need to poll or the primary container could use the docker api to signal the other containers when ready or something...

[timlinux]

Hmm I am running Linux here so I don't have a way to test with boot2docker.

[anentropic]

I think it's probably not fixable in the context of this project, needs fixing within boot2docker or manual hacks of the virtualbox vm configuration

[timlinux]

Ok - I'll close this issue out - feel free to re-open if you have some fresh ideas on how I can structure things to make it work for you.

Regards

Tim