Creates a symlink to the .so file in postint for mod_wsgi

[kushaldas]

Status

Ready for review

Description of Changes

Fixes in part of #4768

On Focal, mod_wsgi expects a shared library for the python module at:

$VENV/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.so

But, the real shared library path is

$VENV/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so"

So we are creating a symlink with the proper path if we are using Python 3.8 (that is on Focal)

Testing

• make build-deps on this branch
• make staging to create a fresh staging environment from the debian packages. There should not be any error in the postinstall section
• Login to app molecule -s libvirt-staging-xenial -h app-staging (on mac point to virtualbox) and see that the following file does not exist.

/opt/venvs/securedrop-app-code/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.so

• sudo touch /usr/bin/python3.8 to emulate a python3.8 env in focal
• reinstall the securedrop-app-code package or reconfigure it, it should create a symlink as mentioned above.

Deployment

Any special considerations for deployment? Consider both:

1. Upgrading existing production instances.
2. New installs.

Checklist

If you made changes to the server application code:

• Linting (make lint) and tests (make test) pass in the development container

If you made changes to securedrop-admin:

• Linting and tests (make -C admin test) pass in the admin development container

If you made changes to the system configuration:

• Configuration tests pass

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

If you made changes to documentation:

• Doc linting (make docs-lint) passed locally

If you added or updated a code dependency:

Choose one of the following:

• I have performed a diff review and pasted the contents to the packaging wiki
• I would like someone else to do the diff review

[kushaldas]

Failed app-tests on the docker step, restarted the job again.

[conorsch]

The proposed approach for resolving the symlink path conflicts with the related postinst changes introduced in #4622. The wsgi module path is read from /etc/apache2/mods-available/wsgi.load, which we write to in postinst on L162: https://github.com/freedomofpress/securedrop/pull/4622/files#diff-4dd8e904f4b653b022dd495265bd22bdR156

Rather than add a symlink as a second indication of where the module should be imported from, can we instead revise the existing postinst logic to address the problem under Focal?

[conorsch]

The test -f conditional on L100 will always fail, since the target is a symlink, not a file. Rather than -f, -L would be a better test.

[kushaldas]

@conorsch updated the PR based on feedback.

[emkll]

@kushaldas independently of the way this symlink is applied, would it be appropriate here to add a testinfra test to ensure the symlink is properly applied (or not) depending on the targeted platform?

[rmol]

I tried the on_focal branch with the Qubes staging environment (changed select-staging-env to preserve securedrop_platform_suffix under Qubes, copied qubes-staging to qubes-staging-focal and changed securedrop_staging_install_target_distro to focal) and wsgi.load was generated correctly by the postinst invocation of mod_wsgi-express module-config:

sdadmin@sd-staging-app:~$ cat /etc/apache2/mods-enabled/wsgi.load
LoadModule wsgi_module "/opt/venvs/securedrop-app-code/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so"
WSGIPythonHome "/opt/venvs/securedrop-app-code"
sdadmin@sd-staging-app:~$ ls -al /opt/venvs/securedrop-app-code/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so
-rw-r--r-- 1 root root 256904 Jun 18 21:58 /opt/venvs/securedrop-app-code/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so

The site's working. 🤔 I'll go see if the way I installed dh-virtualenv in the builder image (building its .deb in a clone of its repo, because apt complained about the signature on the sid package) could have affected this.

[conorsch]

Given the changes in #5444 & #5458, we may not need the symlink approach here at all. @rmol plans to follow up with additional investigations. For now, I'm leaving the "Changes requested" review to block merge. We can revisit after more folks have chimed in on what's needed for the new build logic.

[rmol]

Investigation complete, and the mod_wsgi config is still getting generated correctly, so I think the symlink is unnecessary. I'm closing this; we can of course reopen if someone else sees the problem.

[kushaldas]

It seems this path modification is happening only in the virtual environment of the dh-virtualenv.

✦ ❯ docker run --rm -it quay.io/freedomofpress/sd-docker-builder-focal:2020_09_10
root@8c7d0635b0db:/# cd 
root@8c7d0635b0db:~# python3 -m venv hello
root@8c7d0635b0db:~# source hello/bin/activate
(hello) root@8c7d0635b0db:~# python3 -m pip install --no-binary :all: mod-wsgi==4.6.7
Collecting mod-wsgi==4.6.7
  Downloading mod_wsgi-4.6.7.tar.gz (497 kB)
     |████████████████████████████████| 497 kB 771 kB/s 
Skipping wheel build for mod-wsgi, due to binaries being disabled for it.
Installing collected packages: mod-wsgi
    Running setup.py install for mod-wsgi ... done
Successfully installed mod-wsgi-4.6.7
(hello) root@8c7d0635b0db:~# mod_wsgi-express module-config
LoadModule wsgi_module "/root/hello/lib/python3.8/site-packages/mod_wsgi/server/mod_wsgi-py38.cpython-38-x86_64-linux-gnu.so"
WSGIPythonHome "/root/hello"

[conorsch]

Reopening based on comments by @kushaldas & @rmol in standup today. Once the dust settles a bit on the build system choice for dh-virtualenv on Focal (#5484), looks like we will indeed need to munge the so path.

[conorsch]

Reopening based on comments by @kushaldas & @rmol in standup today.

GitHub wouldn't let us reopen the PR because the underlying branch had been modified, so this same PR was resubmitted as #5489.