Debugging wheel‐build issues

Problem

• https://github.com/single-cell-data/TileDB-SOMA/blob/main/.github/workflows/python-packaging.yml uses pypa/cibuildwheel which invokes various commands
• These are layered within various docker run
• The pypy/cibuildwheel YAML is opaque and does not say everything it's doing
• This means that if you want to go into a failed wheel-build CI run, click through to the logs, and copy/paste commands to try to repro a CI wheel-build failure, you simply cannot

Running wheel-build CI at will

• Our "sdist & wheels" CI only auto-runs on releases and on nightly crons. (Please also see our established procedure.)
• In this repo to go Actions
• Then on the left pick the "sdist & wheels" action after "show more workflows"
• At the right "Run workflow", then your branch

Solution

The below is a known-good method to reproduce wheel-build issues outside of CI. They are a truthful record of commands that have been invoked.

• Suggestion: use a clean docker run environment for greatest fidelity. If you use an existing system then you make pick up localisms that don't reflect a clean CI-like environment.

$ alias dreb
alias dreb='docker run --rm -it --entrypoint=/bin/bash'

$ dreb python:3.10
root@1892dc61dd4d:/#

• Build a source dist, then extract it to somewhere else, then install from there. This step is crucial as it verifies that the install will be seeing only files listed in MANIFEST.in, not everything in your local checkout (which is the way the CI YAML works)

cd /tmp/

git clone https://github.com/single-cell-data/TileDB-SOMA

cd TileDB-SOMA/apis/python/

tag=1.9.0 # or whatever tag

git checkout $tag

python setup.py sdist

ls -lrt dist/

mkdir /tmp/install

cp dist/tiledbsoma-${tag}.tar.gz /tmp/install

cd /tmp/install

tar zxf tiledbsoma-${tag}.tar.gz

cd tiledbsoma-${tag}

apt-get update

apt install -y cmake zip

pip install pybind11

python setup.py bdist_wheel

ls -l dist/tiledbsoma-*.whl

pip install dist/tiledbsoma-*.whl

cd /tmp/TileDB-SOMA/test

tar zxf soco.tgz

cd /tmp/TileDB-SOMA/apis/python

pip install -r requirements_dev.txt

python -m pytest tests/

MacOS

All of the above, except between building the .whl file and installing the .whl file:

$ ls -l dist/*.whl
-rw-r--r--  1 johnkerl  staff  415420 Mar 27 11:15 dist/tiledbsoma-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl

If

$ pip install dist/tiledbsoma-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl

then

ERROR: tiledbsoma-1.9.0-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.

Note that my Mac is current at v14:

$ sw_vers
ProductName:        macOS
ProductVersion:     14.4.1
BuildVersion:       23E224

But renaming 11_0 to 14_4 is, surprisingly, not the right thing to do. It turns out that the installer is using get_platform() as follows:

>>> from distutils import util
>>> util.get_platform()
'macosx-10.9-x86_64'

which is bizarre. Nonetheless the right thing is

mv tiledbsoma-1.9.0-cp39-cp39-macosx_11_0_x86_64.whl tiledbsoma-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl

at which point

pip install tiledbsoma-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl

succeeds.

Note however #2620 wherein it appears that this works for Python 3.9 but not Python 3.11.