touching docker.ini on startup does not respond well to a readonly filesystem

[guest20]

Overwriting local.d/docker.ini from k8s causes the couch container to go into CrashLoopBackOff because touch: cannot touch '/opt/couchdb/etc/local.d/docker.ini': Read-only file system

Expected Behavior

I'd expect to just be able to mount my own docker.ini over the top of the one in the container so I can skip the setup wizard (in my case I only n=1)

Current Behavior

CrashLoopBackOff because couchdb crashes when it attempts to touch docker.ini

Steps to Reproduce (for bugs)

The single-node.ini seen here came from clicking "single node cluster" in the ui, and then grabbing the config from the running contgainer.

apiVersion: v1
kind: ConfigMap
metadata:
  name: couchdb-ini
data:
  single-node.ini: |
    [admins]
    admin = -pbkdf2-████████████████████████
    [couchdb]
    uuid = ████████████████████████████████
    [couch_httpd_auth]
    secret = ████████████████████████████████
    [chttpd]
    bind_address = 0.0.0.0
    port = 5984
    [log]
    level = debug
    writer = stderr
    [cluster]
    n = 1

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: couchdb
spec:
  selector:
    matchLabels:
      app: couchdb
  replicas: 1
  template:
    metadata:
      labels:
        app: couchdb
    spec:
      containers:
      - name: couchdb
        image: couchdb:3.0.1
        ports:
        - containerPort: 5984
        envFrom:
          - configMapRef:
              name: couchdb-config
        volumeMounts:
          - name: config-volume
            mountPath:  /opt/couchdb/etc/local.d/docker.ini
            subPath: single-node.ini
      volumes:
        - name: config-volume
          configMap:
            name: couchdb-ini
            items:
            - key: single-node.ini
              path: single-node.ini 

1. kubectl apply -f ...
2. kubectl get pods

Context

my cluster was built via k3s, and runs fluxcd to apply changes to my manifests - I think i've managed to pare it down.

Your Environment

• Version used: 3.0.1, from dockerhub
• Operating System and version (desktop or mobile): Linux dave 4.19.0-8-amd64 apache/couchdb-docker#1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 GNU/Linux

 kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3+k3s1", GitCommit:"96653e8d86b74a1f73eadf09d4edc44778c2f8dd", GitTreeState:"clean", BuildDate:"2020-06-01T16:47:11Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3+k3s1", GitCommit:"96653e8d86b74a1f73eadf09d4edc44778c2f8dd", GitTreeState:"clean", BuildDate:"2020-06-01T16:47:11Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/amd64"}      

[willholley]

Can you just change the mountpath to e.g. /opt/couchdb/etc/local.d/zz-single-node.ini? As described in the docs, settings in successive config files override the settings in earlier entries so you just need to put your overrides into a file that sorts later than docker.ini within local.d.

[guest20]

I did try calling it single-node.ini, which is ascibetically later than docker.ini. In that case couchdb just refused to start, and fell into CrashLoopBackOff without any message at all (so I gave up and created a ticket)

Edit: And that's where the [log] section in the config came from

[wohali]

We can file a PR to only touch the file if it does not exist, but then wouldn't you get stuck here?

https://github.com/apache/couchdb-docker/blob/master/3.1.0/docker-entrypoint.sh#L59

[wohali]

The issue Will is bringing up is that CouchDB requires write access to the last file in the config chain. If you make your file last ascibetically, you'll break CouchDB. If you place it prior to docker.ini then any changes it writes will go inside the container and be lost.

If you're OK with this, then that's the best workaround, and we'll document it.

We'd recommend having a writeable config file placed in permanent storage outside of the system, but I wouldn't know exactly how to do that in your k3s setup.

[guest20]

Thanks for mulling this over with me!

mount a read-write file ascibetically after ...

The files i'm mounting come from a ConfigMap, I'm not sure if I can make them rw.

If you make your file last ascibetically, you'll break CouchDB.

Yep, that's definitely what's happening here.

If you place it prior to docker.ini then any changes it writes will go inside the container and be lost.

This sounds like exactly the what I'm looking for.

Is there also a magical file I can poke to solve the "you need to create _global_changes, _replicator and _users manually" thing?

[wohali]

If you place it prior to docker.ini then any changes it writes will go inside the container and be lost.

This sounds like exactly the what I'm looking for.

Great! That's the recommended approach, then. 😉

Is there also a magical file I can poke to solve the "you need to create _global_changes, _replicator and _users manually" thing?

Try adding the following to your ini file:

[couchdb]
single_node=true

Assuming, of course, you're deploying this as a single node and not a cluster.

[wohali]

@guest20 I've moved this over to our new GitHub Discussions since this is a help request that didn't result in a change to our code. Please mark this as solved if we've answered your question!

Thanks for using CouchDB 👯‍♂️

[guest20]

Thanks for the pointers @wohali !

Assuming, of course, you're deploying this as a single node and not a cluster.

Is single_node=true different than [cluster] n = 1?

[wohali]

Yes! single_node=true will automatically create those databases that you were asking for. You'll also want to set [cluster] n=1.