Skip to content

install

Jump to bottom Edit New page
Ryan Hodges edited this page Aug 11, 2020 · 27 revisions

Installing COMPASS

Independent Server Setup

install dependencies (deprecated):

  • git
  • python-dev
  • python-pip
  • python-virtualenv
  • postgresql-9.3
  • postgresql-9.3-postgis-2.1
  • postgresql-server-dev-9.3
  • libjpeg-dev

Install dependencies (Ubuntu 18.04 LTS, python 3 2019 overhaul)

sudo apt install git python3 python3-dev python3-virtualenv python3-pip postgresql postgresql-contrib postgis postgresql-server-dev-10 libjpeg-dev gdal-bin python-gdal python3-gdal libgdal-dev -y

Create Apps directory:

sudo mkdir /usr/local/apps
sudo chown ubuntu /usr/local/apps

Check out the code into /usr/local/apps/:

cd /usr/local/apps/
git clone https://github.com/Ecotrust/COMPASS.git

Configure PostgreSQL

Replace USERNAME with an appropriate username for your database user below.

You will be prompted to enter a password for the new user as well. Keep track of these for the following steps as well.

sudo -u postgres createuser -s -P USERNAME
sudo -u postgres createdb -O USERNAME compass
sudo -u postgres psql -c "CREATE EXTENSION postgis; CREATE EXTENSION postgis_topology;" compass

Add db user to pg_hba.conf:

  • vim /etc/postgresql/10/main/pg_hba.conf
  • add line to bottom: 
    local   compass    USERNAME                  trust
  • save and close
  • restart postgres: sudo service postgresql restart

Create Python VirtualEnv

python3 -m pip install --user virtualenv
cd /usr/local/apps/COMPASS/
python3 -m virtualenv env
source /usr/local/apps/COMPASS/env/bin/activate

Install Python Dependencies

pip install -r /usr/local/apps/COMPASS/requirements.txt

Installing PyGDAL

Currently PyGDAL is a bit of a pain to install...

pip uninstall numpy
gdal-config --version
  • Capture the output of that last command (such as 2.2.3)
  • Increment the last digit by 1 and plug it in to pip install "pygdal<X.Y.Z" (such as pip install "pygdal<2.2.4")

Set local Django settings

cp /usr/local/apps/COMPASS/mp/settings_local.template /usr/local/apps/COMPASS/mp/settings_local.py

Edit /usr/local/apps/COMPASS/mp/settings_local.py:

  • SECRET_KEY = [something new]
  • DATABASES
    • 'NAME': 'compass'
    • 'USER': '[compassdbuser]'
    • 'PASSWORD': '[add password]',
    • 'HOST': 'localhost',
    • 'PORT': 5432,
  • MEDIA_ROOT = '/usr/local/apps/COMPASS/mediaroot/'
  • MEDIA_URL = '/media/'
  • SOCKET_URL = ''
  • SOCIAL_AUTH_GOOGLE_PLUS_KEY = ''
  • SOCIAL_AUTH_GOOGLE_PLUS_SECRET = ''
  • ADMIN_MEDIA_PATH = '/usr/local/apps/COMPASS/env/python3/site-packages/django/contrib/admin/static/admin/'

Prep logfile

sudo chown -R {username} /usr/local/apps/COMPASS/mp/logs/

Create dj Alias

sudo vim ~/.bashrc add this to the bottom:

alias dj="/usr/local/apps/COMPASS/env/bin/python3 /usr/local/apps/COMPASS/mp/manage.py"
alias djrun="dj runserver 0:8000 --noreload"

exit out of the ssh session and ssh back in again

Enable layer fixture import/export

mkdir /usr/local/apps/COMPASS/mediaroot/data_manager_uploads
chgrp www-data /usr/local/apps/COMPASS/mediaroot/data_manager_uploads

Final Django setup steps

dj migrate
dj collectstatic
dj enable_sharing all
dj loaddata /usr/local/apps/COMPASS/mp/fixtures/layer_data.json
dj loaddata /usr/local/apps/COMPASS/mp/fixtures/project_settings.json

use djrun to test that things are working as expected

Loading a PostgreSQL dump file

If you are loading data from an existing database (created with something like pg_dump compass > ./compass_dump_20190529.sql) and are starting from scratch on a new install, do the following:

  • Identify the users/owners of the database that was dumped
    • Update both your django settings and PostgreSQL settings (pg_hba) to include those users/owners
    • restart PostgreSQL
  • Change to postgres user: sudo su postgres
    • Drop your old DB: dropdb compass
    • Create an empty DB: createdb compass
    • Read in the file: psql compass < ./compass_dump_20190529.sql
    • Exit back to your primary user: exit
  • Run migrations, but you need to fake the initial DB creation: dj migrate --fake-initial

Installing on NGINX/uWSGI

For reference go here

TODO: Update with newer emperor approach Figure out why uwsgi isn't firing on boot

From inside your virualenv:

  • sudo apt-get install nginx uwsgi uwsgi-plugin-python3 -y
  • pip install uwsgi
  • test uwsgi install with uwsgi --http :8000 --wsgi-file /usr/local/apps/COMPASS/mp/wsgi.py
  • sudo cp /usr/local/apps/COMPASS/deploy/compass.conf /etc/nginx/sites-available/compass
  • sudo rm /etc/nginx/sites-enabled/default
  • sudo ln -s /etc/nginx/sites-available/compass /etc/nginx/sites-enabled/compass
  • sudo cp /usr/local/apps/COMPASS/deploy/emperor.ini /etc/uwsgi/
  • sudo cp /usr/local/apps/COMPASS/deploy/uwsgi.service /etc/systemd/system/
  • sudo systemctl enable uwsgi.service
  • sudo cp /usr/local/apps/COMPASS/deploy/compass_uwsgi.ini /etc/uwsgi/apps-enabled/compass.ini
  • sudo service nginx start
  • sudo service nginx restart
  • copy relevant content from deploy/rc.local.template to /etc/rc.local
  • sudo chmod +x /usr/local/apps/COMPASS/deploy/restart_nginx.sh
  • sudo chmod 666 /usr/local/apps/COMPASS/mp/logs/main.log <- TODO: make this not necessary
  • sudo reboot
  • In a few minutes, test your URL in a browser to see that everything came up as expected

Additional Production Server Setup Tasks:

Google Analytics

Please review the following:

  • Login Credentials for accessing an appropriate Google Account
  • Google's documentation on inserting a script into your HTML's head block to enable tacking
    • This should be done manually to keep keys out of the repository
    • This should be inserted into the planner.html file

Munin

sudo apt-get install munin munin-node -y

Swapspace

sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo cp /etc/fstab /etc/fstab.bak
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Email

  • sudo apt install postfix -y
    • Follow steps for "Internet Site" - hopefully you have a url already. If not you can change these settings later
  • Sudo apt install postgrey -y
  • When you have a url (if you didn’t enter is for ‘System mail name’ while installing postfix) you can edit /etc/postfix/main.cf and update ‘myhostname’ with your url.
  • You should also set up spf records in your dns manager. Doing so is beyond the scope of this document.
  • The same with an Mx record
  • Update your Site’s name in the “site” section of the django admin to reflect your URL.
  • Restart postfix

Unattended Updates

  • sudo apt install unattended-upgrades
  • sudo vim /etc/apt/apt.conf.d/50unattended-upgrades
    • Uncomment the "...-updates" line
    • Uncomment and configure:
      • Mail
      • MailOnlyOnError
    • Remove-Unused-Kernel-Packages "true";
    • Remove-Unused-Dependencies "true";
    • Automatic-Reboot "true";
    • Automatic-Reboot-Time "8:00";
      • The above assumes a UTC server with assumed 1 or 2 AM Pacific time downtime
  • sudo vim /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "7";
APT::Periodic::Unattended-Upgrade "1";
  • sudo unattended-upgrades --dry-run --debug

PUBLIC DRAWING

If you are intending to allow public drawing on your install, you will want to follow these additional steps:

  • In the admin, create a user with username 'public'
  • chown www-data the following directories:
    • media/extracted/
    • media/planning_unit_sql/
    • mediaroot/csvs/
  • Use the admin hex-import script to import new base data
  • Be sure that you have the correct species lookups (manage.py import_report_lookups)
  • Set a cron job to clean out the mediaroot/csvs/ directory
    • sudo crontab -e
    • see deploy/cron_job.sh for suggested content
  • Set settings_local.py with PUBLIC_CSV_DIR = '/home/vagrant/marine-planner/mediaroot/csvs/' or similar
  • Be sure to install the public-drawing branch of Madrona in your environment
    • py3 branch contains public-drawing by default

(Optional) Install a Dev (Stage) Instance of Compass

Follow along with this document.

Old PUBLIC DRAWING NOTES (Deprecated)

  • get fetch/merge
  • the scenarios/migrations/0003_auto__del... file does not seem to play well. Grab the version on the ODFW production server
  • manage.py syncdb, migrate, and install_media

Fonts

  • locate an internal copy of the file "compass_fonts.css"
    • copy it to /usr/local/apps/COMPASS/media/compass/css/
  • locate an internal copy of the 'fonts' directory
    • this contains bebasneue files, opensans files, and a trade-gothic directory
    • copy it to /usr/local/apps/COMPASS/media/compass/
  • run dj collectstatic on the new Compass server

VM Setup (deprecated)

Dependencies

  • git

  • vagrant

  • Check out the code

  • Create and update mp/settings_local.py

    • MEDIA_ROOT = '/home/vagrant/marine-planner/media'

    • MEDIA_URL = '/media/'
    • SOCKET_URL = ''
    • SOCIAL_AUTH_GOOGLE_PLUS_KEY = ''
    • SOCIAL_AUTH_GOOGLE_PLUS_SECRET = ''
    • ADMIN_MEDIA_PATH = '/home/vagrant/.virtualenvs/marine-planner/lib/python2.7/site-packages/django/contrib/admin/static/admin/'

  • vagrant up

    • (ignore) Exceptions: Exception: Your MEDIA_ROOT setting has to be a directory other than your madrona or project media folder!
    • Likely this will choke on not being able to find ElementTree
      • Copy ElementTree libraries from a live source
      • run vagrant provision

  • vagrant ssh

    • dj syncdb
      • create superuser now
    • dj migrate
    • dj loaddata data_manager/fixtures/ofr_data_manager.json [ dj is alias for python manage.py ]
    • dj loaddata mp-settings/fixtures/ofr_mp_settings.json
    • dj loaddata ../mp/fixtures/project_settings.json
    • dj loaddata ../mp/fixtures/layer_data_proxy.json

  • dj site localhost:8000

  • dj createsuperuser

  • djrun [ same as python manage.py runserver 0.0.0.0:8000 ]

  • Point a browser at http://localhost:8000

Add a custom footer
Add a custom sidebar
Clone this wiki locally