Build issues with docker-compose

[robsavoye]

There's two slightly related bugs trying to build a docker container from the develop branch.

COPY failed: file not found in build context or excluded by .dockerignore: stat frontend: file does not exist
ERROR: Service 'frontend' failed to build : Build failed

If I remove frontend from .dockerignore, docker-compose still fails trying to build with:

Collecting PyYAML==6.0
Downloading PyYAML-6.0.tar.gz (124 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.0/125.0 kB 1.7 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [48 lines of output]
running egg_info
writing lib/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib/PyYAML.egg-info/top_level.txt
Traceback (most recent call last):

[Aadesh-Baral]

Building wheel for PyYAML 6.0.0 seems to fail with Python 3.10. Our lock file is generated using python 3.9.x as we have set Python version to be between >=3.9 and <=3.11. If we can set the Python version to be within the range of >=3.10 and <=3.11 in the pyproject.toml file, PyYAML will be upgraded to version 6.0.1 in the lock file, which should allow the build to pass successfully.

@eternaltyro What is the python version we are using in prod and staging?

[eternaltyro]

@Aadesh-Baral v3.10

[Aadesh-Baral]

@robsavoye Can you try building backend container from branch fix/pyyaml-build-fail? I will merge the PR once you verify it's working.

[Aadesh-Baral]

Installing scipy is failing now due to some missing build dependencies. I would always avoid using alpine-based images for Python projects due to these errors. Also, build times are significantly higher while using alpine-based images in Python projects.

764.3 Collecting scipy==1.10.1
764.4   Downloading scipy-1.10.1.tar.gz (42.4 MB)
769.2      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.4/42.4 MB 8.4 MB/s eta 0:00:00
777.4   Installing build dependencies: started
790.8   Installing build dependencies: finished with status 'done'
790.8   Getting requirements to build wheel: started
790.9   Getting requirements to build wheel: finished with status 'done'
790.9   Installing backend dependencies: started
793.1   Installing backend dependencies: finished with status 'done'
793.1   Preparing metadata (pyproject.toml): started
795.3   Preparing metadata (pyproject.toml): finished with status 'error'
795.3   error: subprocess-exited-with-error
795.3   
795.3   × Preparing metadata (pyproject.toml) did not run successfully.
795.3   │ exit code: 1
795.3   ╰─> [40 lines of output]
795.3       The Meson build system
795.3       Version: 1.2.0
795.3       Source dir: /tmp/pip-install-zt4uph40/scipy_7bfe37b571ab4871ba103e770f6547c5
795.3       Build dir: /tmp/pip-install-zt4uph40/scipy_7bfe37b571ab4871ba103e770f6547c5/.mesonpy-gy7ipia5/build
795.3       Build type: native build
795.3       Project name: SciPy
795.3       Project version: 1.10.1
795.3       C compiler for the host machine: cc (gcc 12.2.1 "cc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924")
795.3       C linker for the host machine: cc ld.bfd 2.39
795.3       C++ compiler for the host machine: c++ (gcc 12.2.1 "c++ (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924")
795.3       C++ linker for the host machine: c++ ld.bfd 2.39
795.3       Cython compiler for the host machine: cython (cython 0.29.36)
795.3       Host machine cpu family: x86_64
795.3       Host machine cpu: x86_64
795.3       Compiler for C supports arguments -Wno-unused-but-set-variable: YES
795.3       Compiler for C supports arguments -Wno-unused-function: YES
795.3       Compiler for C supports arguments -Wno-conversion: YES
795.3       Compiler for C supports arguments -Wno-misleading-indentation: YES
795.3       Compiler for C supports arguments -Wno-incompatible-pointer-types: YES
795.3       Library m found: YES
795.3       
795.3       ../../meson.build:63:0: ERROR: Unknown compiler(s): [['gfortran'], ['flang'], ['nvfortran'], ['pgfortran'], ['ifort'], ['ifx'], ['g95']]
795.3       The following exception(s) were encountered:
795.3       Running `gfortran --version` gave "[Errno 2] No such file or directory: 'gfortran'"
795.3       Running `gfortran -V` gave "[Errno 2] No such file or directory: 'gfortran'"
795.3       Running `flang --version` gave "[Errno 2] No such file or directory: 'flang'"
795.3       Running `flang -V` gave "[Errno 2] No such file or directory: 'flang'"
795.3       Running `nvfortran --version` gave "[Errno 2] No such file or directory: 'nvfortran'"
795.3       Running `nvfortran -V` gave "[Errno 2] No such file or directory: 'nvfortran'"
795.3       Running `pgfortran --version` gave "[Errno 2] No such file or directory: 'pgfortran'"
795.3       Running `pgfortran -V` gave "[Errno 2] No such file or directory: 'pgfortran'"
795.3       Running `ifort --version` gave "[Errno 2] No such file or directory: 'ifort'"
795.3       Running `ifort -V` gave "[Errno 2] No such file or directory: 'ifort'"
795.3       Running `ifx --version` gave "[Errno 2] No such file or directory: 'ifx'"
795.3       Running `ifx -V` gave "[Errno 2] No such file or directory: 'ifx'"
795.3       Running `g95 --version` gave "[Errno 2] No such file or directory: 'g95'"
795.3       Running `g95 -V` gave "[Errno 2] No such file or directory: 'g95'"
795.3       
795.3       A full log can be found at /tmp/pip-install-zt4uph40/scipy_7bfe37b571ab4871ba103e770f6547c5/.mesonpy-gy7ipia5/build/meson-logs/meson-log.txt
795.3       + meson setup --prefix=/usr/local /tmp/pip-install-zt4uph40/scipy_7bfe37b571ab4871ba103e770f6547c5 /tmp/pip-install-zt4uph40/scipy_7bfe37b571ab4871ba103e770f6547c5/.mesonpy-gy7ipia5/build --native-file=/tmp/pip-install-zt4uph40/scipy_7bfe37b571ab4871ba103e770f6547c5/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
795.3       [end of output]
795.3   
795.3   note: This error originates from a subprocess, and is likely not a problem with pip.
795.3 error: metadata-generation-failed
795.3 
795.3 × Encountered error while generating package metadata.
795.3 ╰─> See above for output.
795.3 
795.3 note: This is an issue with the package mentioned above, not pip.
795.3 hint: See above for details.
795.8 
795.8 [notice] A new release of pip is available: 23.0.1 -> 23.2.1
795.8 [notice] To update, run: pip install --upgrade pip
------
failed to solve: process "/bin/sh -c pip install --user --no-warn-script-location     --no-cache-dir -r /opt/python/requirements.txt" did not complete successfully: exit code: 1

cc @eternaltyro cc @dakotabenjamin

[cyrossignol]

We used a similar fix for tasks.opensidewalks.com. I rebased onto 55b6570 from #6013, and things continue to build successfully. I also needed to add these two patches in addition to the PyYAML bump:

• OpenSidewalks/tasking-manager@bc893e8
• OpenSidewalks/tasking-manager@a46e35a

I can PR if needed. Thanks.

[Aadesh-Baral]

@cyrossignol Thanks for your input! we're working on migrating the backend container from an Alpine-based image to a Debian image. You can find the work in progress in PR #4646. Thanks again for your interest!