Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Central v2023.2 upgrade to Postgres 14 #1577

Merged
merged 33 commits into from
Mar 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
cf61d52
Add special upgrade instructions for v2023.2 (postgres upgrade!)
Feb 9, 2023
bbba3a7
Update step 10: check ./postgres14-upgrade/upgrade-completed-ok exists
Feb 15, 2023
2503d1d
Move check-available-space script
Feb 15, 2023
07c8ac0
Update instructions for custom database servers
Feb 15, 2023
32a988c
Update digital ocean instructions' ref to postgres version
Feb 15, 2023
76a642b
Improve console example formatting
lognaturel Feb 17, 2023
44516b7
Add header for versioned instructions
lognaturel Feb 20, 2023
c0708ee
Use tabs to separate standard and custom instructions
lognaturel Feb 21, 2023
a4a9410
Improve formatting of customization instructions
lognaturel Feb 21, 2023
e6149e6
Add option for those comfortable with resolving conflicts
lognaturel Feb 21, 2023
7d3673a
Upgrade file locations inline with central/postgres-upgrade PR
Feb 21, 2023
f5d6929
Feedback from @yanokwa
lognaturel Feb 27, 2023
9a45848
Add a bit more context, bold steps
ktuite Mar 1, 2023
0a043e5
Add instructions to upgrade Docker and nginx
lognaturel Mar 8, 2023
c93f777
Add instructions on migrating configurations
lognaturel Mar 9, 2023
1b110c7
Improve standard upgrade instructions
lognaturel Mar 9, 2023
3c61f77
@yanokwa feedback
lognaturel Mar 9, 2023
12ab91a
Update how to identify database type
lognaturel Mar 9, 2023
62c617e
Simplifications recommended by @yanokwa
lognaturel Mar 9, 2023
217eddf
Update install instructions
lognaturel Mar 9, 2023
55324e7
Update customization instructions
lognaturel Mar 9, 2023
5d9a322
Improve formatting
lognaturel Mar 9, 2023
ede8b2c
Improve instructions for upgrading Docker
lognaturel Mar 9, 2023
7f21bbe
Use docker compose and improve formatting
lognaturel Mar 9, 2023
0e471ab
Add more about copying config files
lognaturel Mar 9, 2023
2f6228a
Use forum post introduction mentioning Docker
lognaturel Mar 9, 2023
20f42c6
Update container names
lognaturel Mar 9, 2023
39960fe
Fix advanced section link
lognaturel Mar 9, 2023
6a78eed
Improve no-reply note
lognaturel Mar 9, 2023
7d1678e
Improve command line styling
lognaturel Mar 9, 2023
bed57f0
Update EMAIL_FROM var name
lognaturel Mar 9, 2023
e7362a1
Add database SSL option
lognaturel Mar 9, 2023
e1d0c0f
Simplify DB SSL instructions
lognaturel Mar 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/central-backup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,13 @@ Restoring a Direct Backup file to a Central instance will entirely replace all o
cd
cd central
docker-compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/backup-2018-01-01T00:00:00Z.zip 'SECRET_PASSPHRASE'
docker compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/backup-2018-01-01T00:00:00Z.zip 'SECRET_PASSPHRASE'
You'll have to replace the filename following :file:`/data/transfer` with your own snapshot filename, and the text ``SECRET_PASSPHRASE`` with the passphrase you typed when backups were first set up. If you did not set up a passphrase, immediately press Enter after you have finished putting the :file:`.zip` filename in:

.. code-block:: console
docker-compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/backup-2018-01-01T00:00:00Z.zip
docker compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/backup-2018-01-01T00:00:00Z.zip
#. The server will think for a while, and then print some more instructions. You will have to refresh any browser windows you have open to ODK Central to proceed. If you run into error messages at this step, please read them carefully and then seek help on the `ODK Forum <https://forum.getodk.org/>`_ if you are not sure what to do.

16 changes: 8 additions & 8 deletions docs/central-command-line.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ Central Command Line Tools

Central allows some administrative actions to be performed by anybody with direct console access to the server itself. Usually, this means only the person who set up the server and installed Central onto it. These tools can be used to:

- create new user accounts,
- reset passwords,
- and manage user permissions.
- create new user accounts,
- reset passwords,
- and manage user permissions.

All of these actions can be done through the website, and if everything is working normally we strongly recommend that you use the web interface to perform these actions. But things happen, and if something gets broken (like everybody forgets their password, or someone deletes all the users), you can use the command line tools to get things working again.

Expand All @@ -22,8 +22,8 @@ Once you have a command line in front of you (it should say something like ``roo

.. code-block:: console
cd central
docker-compose exec service odk-cmd
$ cd central
$ docker compose exec service odk-cmd
If you see instructions appear with the section headings *Usage*, *Options*, and *Commands*, you'll know you're in the right place. If you are familiar with command line tools in general, those instructions are probably all you need to get going. Otherwise, please see the sections below for a short guide and example on how to use each one.

Expand All @@ -36,7 +36,7 @@ If you followed our :doc:`DigitalOcean installation steps <central-install-digit

.. code-block:: console
docker-compose exec service odk-cmd --email [email protected] user-create
$ docker compose exec service odk-cmd --email [email protected] user-create
You will be asked for a password for the account, and if everything worked correctly you should see some data printed out that among other things lists the email you entered a moment ago. The next thing you'll need to do is to :ref:`make the new account an administrator <central-command-line-user-promote>`, which is normally automatically done by the web interface. If you don't, the new user will not be able to do anything.

Expand All @@ -51,7 +51,7 @@ Please start by performing the steps above in the :ref:`central-command-line-bas

.. code-block:: console
docker-compose exec service odk-cmd --email [email protected] user-set-password
$ docker compose exec service odk-cmd --email [email protected] user-set-password
You will be prompted for a new password. Type it in and press Enter, and if you see text that says ``true``, the action succeeded. You can then use the website to log into that user account.

Expand All @@ -66,7 +66,7 @@ Please start by performing the steps above in the :ref:`central-command-line-bas

.. code-block:: console
docker-compose exec service odk-cmd --email [email protected] user-promote
$ docker compose exec service odk-cmd --email [email protected] user-promote
If the action succeeded, you will see text that reads ``{"success":"true"}``.

350 changes: 223 additions & 127 deletions docs/central-install-digital-ocean.rst

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/central-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ If you want to self-host but you're not sure what cloud provider to choose, we r
Installing elsewhere
~~~~~~~~~~~~~~~~~~~~

Central is entirely defined via **Docker Compose**, which means the ``docker-compose`` command will be all you need to manage the entire system.
Central is entirely defined via **Docker Compose**, which means the ``docker compose`` command will be all you need to manage the entire system.

.. warning::
We verify each version of Central on DigitalOcean and confirm that upgrades are possible. However, we do not verify them on other cloud providers and generally can't provide free support for installations that deviate from the DigitalOcean instructions. You may find other community members able to help `on the forum <https://forum.getodk.org/>`_.
Expand Down
82 changes: 41 additions & 41 deletions docs/central-troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ If Central is behaving in an unexpected way, it is often helpful to read the Doc

The containers that are most likely to be helpful for troubleshooting are the `service` container and the `nginx` container. Use the `--tail` and `--since` options to help filter the logs. For example:

* ``docker logs --tail 100 central_service_1`` to see the last 100 lines of the service logs.
* ``docker logs --since 5m central_nginx_1`` to see the last 5 minutes of the nginx logs.
* ``docker logs --tail 100 central-service-1`` to see the last 100 lines of the service logs.
* ``docker logs --since 5m central-nginx-1`` to see the last 5 minutes of the nginx logs.

Other commands that are helpful are:

Expand Down Expand Up @@ -46,21 +46,21 @@ You may run into a "Could not connect with Server" error message when previewing

To resolve this problem, first identify your upstream DNS servers. Run ``cat /run/systemd/resolve/resolv.conf`` to see your current list of nameservers with their IP addresses. They will look like this:

.. code-block:: console
.. code-block:: console
lognaturel marked this conversation as resolved.
Show resolved Hide resolved
nameserver 1.2.3.4
nameserver 9.8.7.6
nameserver 1.2.3.4
nameserver 9.8.7.6
Now, run ``nano /etc/docker/daemon.json`` to make those nameservers and, optionally, the Google DNS (8.8.8.8) as a fallback available to Docker. Your ``daemon.json`` file will look like the snippet below, but you will need to replace ``1.2.3.4`` and ``9.8.7.6`` with your nameservers' IP addresses.

.. code-block:: console
.. code-block:: bash
{
"dns": ["1.2.3.4", "9.8.7.6", "8.8.8.8"]
}
{
"dns": ["1.2.3.4", "9.8.7.6", "8.8.8.8"]
}
Finally, stop the containers, restart Docker, and bring the containers back up with ``docker-compose stop``, ``systemctl restart docker`` and ``docker-compose up -d``.
Finally, stop the containers, restart Docker, and bring the containers back up with ``docker compose stop``, ``systemctl restart docker`` and ``docker compose up -d``.

.. _migration-fails-due-to-out-of-memory-error:

Expand Down Expand Up @@ -91,13 +91,13 @@ If you get an error `413` when trying to upload a submission or when trying to u

If you absolutely must upload files over 100MB, you can change the `client_max_body_size` `nginx` directive:

.. code-block:: console
.. code-block:: console
cd ~/central
docker-compose stop
nano files/nginx/odk.conf.template
<modify the nginx conf value for client_max_body_size>
docker-compose up -d
$ cd ~/central
$ docker compose stop
$ nano files/nginx/odk.conf.template
<modify the nginx conf value for client_max_body_size>
$ docker compose up -d
.. _troubleshooting-docker-compose-down:

Expand All @@ -110,57 +110,57 @@ Database reset after running Docker command
If you do not have a backup, do not reboot or restart the machine. Instead, take a live, full disk backup of the machine so you have a fallback. If you are using DigitalOcean, see `how to create snapshots <https://docs.digitalocean.com/products/images/snapshots/how-to/snapshot-droplets/>`_.


It is possible to accidentally reset the database by running the down command with docker-compose. We are working on a way to prevent this error in the future. For now, if you have run this command and your database has reset, follow these steps to restore your data.
Prior to Central v2023.2, it is possible to accidentally reset the database by running the ``down`` command with ``docker-compose``. This no longer happens in Central v2023.2+ because the default database is stored on a named volume. If you are running an older Central version, you have run this command and your database has reset, follow these steps to restore your data.

The instructions below assume you installed Central on an Ubuntu LTS server. If you did not, or do not feel confident following the steps below, email [email protected] for assistance.

1. Capture the location of the new (and empty) database.

.. code-block:: console
.. code-block:: console
$ CENTRAL_NEW_DB=$(docker inspect --type container central_postgres_1 \
-f '{{(index .Mounts 0).Source}}' | cut -d / -f 6)
CENTRAL_NEW_DB=$(docker inspect --type container central_postgres_1 \
-f '{{(index .Mounts 0).Source}}' | cut -d / -f 6)
2. Next, find any additional databases you have. You should get the number one (``1``) back. If you get anything else, stop and email [email protected] for assistance.

2. Next, find any additional databases you have. You should get the number one (`1`) back. If you get anything else, stop and email [email protected] for assistance.
.. code-block:: console
.. code-block:: console
find /var/lib/docker/volumes/ -name pg_hba.conf \
| grep -v "$CENTRAL_NEW_DB" | wc -l
$ find /var/lib/docker/volumes/ -name pg_hba.conf \
| grep -v "$CENTRAL_NEW_DB" | wc -l
3. Now that you've confirmed you have only one additional database, capture the location of the old database you wish to restore.

.. code-block:: console
.. code-block:: console
CENTRAL_OLD_DB=$(find /var/lib/docker/volumes/ -name pg_hba.conf \
| grep -v "$CENTRAL_NEW_DB" | cut -d / -f 6)
$ CENTRAL_OLD_DB=$(find /var/lib/docker/volumes/ -name pg_hba.conf \
| grep -v "$CENTRAL_NEW_DB" | cut -d / -f 6)
4. Stop the Docker containers to prepare for restoration.

.. code-block:: console
.. code-block:: console
cd ~/central
docker-compose stop
$ cd ~/central
$ docker-compose stop
lognaturel marked this conversation as resolved.
Show resolved Hide resolved
5. Backup the new database and restore the old database.

.. code-block:: console
.. code-block:: console
cd /var/lib/docker/volumes/
mv "$CENTRAL_NEW_DB" "$CENTRAL_NEW_DB"-backup
mv "$CENTRAL_OLD_DB" "$CENTRAL_NEW_DB"
$ cd /var/lib/docker/volumes/
$ mv "$CENTRAL_NEW_DB" "$CENTRAL_NEW_DB"-backup
$ mv "$CENTRAL_OLD_DB" "$CENTRAL_NEW_DB"
6. Now rebuild and restart the containers.

.. code-block:: console
.. code-block:: console
cd ~/central
docker-compose build
docker-compose up -d
$ cd ~/central
$ docker-compose build
$ docker-compose up -d
lognaturel marked this conversation as resolved.
Show resolved Hide resolved
7. Go to your site in a browser and try to log in with an account that previously existed. If everything works as expected, consider deleting the backup of the new database. You can find it with the following command.

.. code-block:: console
.. code-block:: console
find /var/lib/docker/volumes/ -name *-backup
$ find /var/lib/docker/volumes/ -name *-backup
Loading