Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added meson build system files #260

Merged
merged 45 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
5a87919
Added meson build system files
JBorrow Jun 28, 2024
009864f
Massage paths into relative
JBorrow Jun 28, 2024
d09aa96
Add libmath as a dependency?
JBorrow Jun 28, 2024
f440061
See what happens without 38
JBorrow Jul 1, 2024
13f7529
Remove setup.py
JBorrow Jul 1, 2024
4f1f27b
Remove references to setup.py
JBorrow Jul 1, 2024
cf2e571
Use build tool not setup.py in wheel builder
JBorrow Jul 1, 2024
3f8cfa9
Remedy typo in build_wheelsh.sh
JBorrow Jul 1, 2024
1873f84
Refactor tests
JBorrow Jul 1, 2024
12c48c5
Fix test pathing
JBorrow Jul 1, 2024
3a168de
Remove pixell in test path
JBorrow Jul 1, 2024
9187d48
Potential bug in pillow causing test failures?
JBorrow Jul 1, 2024
a676fe9
Attempt to use correct pyproject.toml dependency specification syntax
JBorrow Jul 1, 2024
ad7987c
Use build library to build wheels
JBorrow Jul 1, 2024
7d81f1d
Use cibuildwheel runner
JBorrow Jul 1, 2024
e5358a4
Remove old wheel builders
JBorrow Jul 1, 2024
acd12d9
Correctly link build targets
JBorrow Jul 1, 2024
147aca3
explicitly set fortran compiler
JBorrow Jul 1, 2024
7117ce2
Give gforaran-12 to correct step
JBorrow Jul 1, 2024
eb9db50
Force gcc-12 as compiler
JBorrow Jul 1, 2024
5996b80
Try without arm macs (for now)
JBorrow Jul 1, 2024
536b230
Add macos arm wheels and change min deployment
JBorrow Jul 1, 2024
43d7554
Set MACOSX_DEPLOYMENT_TARGET to be string
JBorrow Jul 1, 2024
8179282
Add correct test command for cibuildwheel
JBorrow Jul 1, 2024
53d9b74
Simplify build instruction
JBorrow Jul 1, 2024
3a7afbc
MUST use editable install for coverage to work
JBorrow Jul 1, 2024
0f1c60d
Attempt to remedy issue with editable install
JBorrow Jul 1, 2024
d7b0e43
try upgrading pip
JBorrow Jul 1, 2024
c33c94d
Remove upper limit on h5py
JBorrow Jul 1, 2024
06f18be
Add meson no build isolation to editable installs
JBorrow Jul 1, 2024
70b3514
Pre-install cythona nd numpy
JBorrow Jul 1, 2024
29dd8e5
Actually run coverage... That would help
JBorrow Jul 1, 2024
1caf1ad
Remove some unused source files and make benchmark easier to use
JBorrow Jul 2, 2024
2a28308
Allow for dynamic versioning with versioneer
JBorrow Jul 2, 2024
64eef4e
Main Makefile compatible with new build system
JBorrow Jul 2, 2024
0d05360
Docs update
JBorrow Jul 2, 2024
08aa7b3
Don't build i686 wheelsrem
JBorrow Jul 3, 2024
7c75b0c
Remove NERSC section
JBorrow Jul 3, 2024
b5e7f8c
Fix numpy 2.0 errors
JBorrow Jul 3, 2024
778a8fc
Don't build musllinux wheels
JBorrow Jul 3, 2024
8298c05
Add editable install info
JBorrow Jul 3, 2024
760b21b
Swap order of editable args
JBorrow Jul 3, 2024
d4eddf6
Add space in editable install
JBorrow Jul 3, 2024
6972ee7
Add note that you need to include build dependencies before no build …
JBorrow Jul 3, 2024
43e5b1d
Merge pull request #261 from simonsobs/numpy-2p0-changes
JBorrow Jul 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 37 additions & 121 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:

strategy:
matrix:
python: ["3.11", "3.10", "3.9", "3.8"]
python: ["3.12", "3.11", "3.10", "3.9"]

steps:
- uses: actions/checkout@v1
Expand All @@ -19,11 +19,9 @@ jobs:

- name: Run Tests (Linux)
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install pytest-cov numpy "Cython<3.0.4"
python setup.py build_ext -i
python -m pip install .
pytest --cov-report html --cov-report xml --cov-report annotate --cov=pixell pixell/tests/ -s
python -m pip install --upgrade pip setuptools wheel meson ninja meson-python cython numpy
python -m pip install --no-build-isolation --editable '.[test]'
pytest --cov --cov-report html --cov-report xml --cov-report annotate -s

- uses: codecov/codecov-action@v2
with:
Expand All @@ -44,7 +42,7 @@ jobs:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
with:
python-version: "3.9"
python-version: "3.10"

- name: Install Dependencies (MacOS)
run: |
Expand All @@ -58,141 +56,58 @@ jobs:
run: |
ln -s $FC $(dirname $(which $FC))/gfortran
echo "Using FC=$FC CXX=$CXX CC=$CC"
python setup.py build_ext -i
python -m pip install .

- name: Run Tests (MacOS)
run: |
pytest pixell/tests/ -s

pytest -s

build_wheels_linux:
name: Build wheels on Linux
runs-on: ubuntu-latest
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
# Ensure that a wheel builder finishes even if another fails
fail-fast: false
matrix:
os: [ubuntu-20.04]
cp: [cp38, cp39, cp310, cp311]
include:
- cp: cp38
numpyver: "1.20"
- cp: cp39
numpyver: "1.20"
- cp: cp310
numpyver: "1.22"
- cp: cp311
numpyver: "1.22"
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-latest, macos-13]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: actions/setup-python@v2
name: Install Python
with:
python-version: '3.8'

- name: Install cibuildwheel and other dependencies
run: |
python -m pip install -U pip
python -m pip install -U setuptools
python -m pip install --upgrade pip setuptools wheel
python -m pip install pytest-cov numpy "Cython<3.0.4"
python -m pip install cibuildwheel

- name: Test and build wheels
run: |
ln -s $FC $(dirname $(which $FC))/gfortran
bash scripts/build_wheels.sh
- name: Build wheels
uses: pypa/[email protected]
env:
CIBW_ENVIRONMENT: OMP_NUM_THREADS=2
OPENBLAS_NUM_THREADS=2
CIBW_BEFORE_BUILD: "python -m pip install numpy==${{ matrix.numpyver }} scipy 'cython<3.0.4'"
CIBW_BUILD_VERBOSITY: 3
# If users are still on 32 bit systems or PPC they should build from source.
CIBW_BUILD: "${{ matrix.cp }}-manylinux_{x86_64,aarch64}"
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: >
cd /
pytest {project}/pixell/tests -s
PYTHON: "python"
PIP: "pip"
FC: gfortran-12
CC: gcc-12
MACOSX_DEPLOYMENT_TARGET: "13.0"

- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

build_wheels_macos:
name: Build wheels on MacOS
runs-on: macos-12
env:
CC: gcc-12
CXX: gcc-12
FC: gfortran-12
DUCC0_NUM_THREADS: 2
build_wheels_macos_arm:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
# Ensure that a wheel builder finishes even if another fails
fail-fast: false
matrix:
os: [macos-12]
# We don't build 3.7 wheels for MacOS because that's x86 only.
cp: [cp38, cp39, cp310, cp311]
include:
- cp: cp38
numpyver: "1.20"
- cp: cp39
numpyver: "1.20"
- cp: cp310
numpyver: "1.22"
- cp: cp311
numpyver: "1.22"
# macos-13 is an intel runner, macos-14 is apple silicon
os: [macos-14]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: actions/setup-python@v2
name: Install Python
with:
python-version: '3.9'

- name: Install cibuildwheel and other dependencies
run: |
python -m pip install -U pip
python -m pip install -U setuptools
python -m pip install --upgrade pip setuptools wheel
python -m pip install pytest-cov numpy "Cython<3.0.4"
python -m pip install cibuildwheel

- name: Test and build wheels
run: |
ln -s $FC $(dirname $(which $FC))/gfortran
bash scripts/build_wheels.sh
- name: Build wheels
uses: pypa/[email protected]
env:
CIBW_ENVIRONMENT: OMP_NUM_THREADS=2
OPENBLAS_NUM_THREADS=2
CIBW_BEFORE_BUILD: "python -m pip install numpy==${{ matrix.numpyver }} scipy 'cython<3.0.4'"
CIBW_BUILD_VERBOSITY: 3
# To build for both x86_64 and arm64, uncomment the following lines.
# That would require a fortran compiler that can cross-comile from
# x86_64 to arm64, as well as an OMP library that's cross-compiled.
# CIBW_BUILD: "${{ matrix.cp }}-macosx_{x86_64,arm64}"
CIBW_BUILD: "${{ matrix.cp }}-macosx_x86_64"
#CIBW_ARCHS_MACOS: "x86_64 arm64"
CIBW_ARCHS_MACOS: "x86_64"
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: >
cd /
pytest {project}/pixell/tests -s
# Cannot test arm64 on non-native runner
CIBW_TEST_SKIP: "*_arm64"
PYTHON: "python"
PIP: "pip"
FC: gfortran-12
CC: gcc-12
MACOSX_DEPLOYMENT_TARGET: "14.0"

- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl


build_sdist:
name: Build source distribution
Expand All @@ -203,7 +118,7 @@ jobs:
- uses: actions/setup-python@v2
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'

- name: Build sdist
run: |
Expand All @@ -212,14 +127,15 @@ jobs:
python -m pip install --upgrade pip setuptools wheel
python -m pip install "Cython<3.0.4"
python -m pip install numpy
python setup.py sdist
python -m pip install build
python -m build . --sdist

- uses: actions/upload-artifact@v2
with:
path: dist/*.tar.gz

upload_pypi:
needs: [build_wheels_linux, build_wheels_macos, build_sdist]
needs: [build_wheels, build_sdist, build_wheels_macos_arm]
runs-on: ubuntu-latest
# upload to PyPI on every tag starting with 'v'
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ version: 2
build:
os: ubuntu-22.04
tools:
python: "3.7"
python: "3.9"

# Build documentation in the docs/ directory with Sphinx
sphinx:
Expand Down
24 changes: 0 additions & 24 deletions MANIFEST.in

This file was deleted.

15 changes: 5 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,10 @@ export PRINT_HELP_PYSCRIPT
BROWSER := python -c "$$BROWSER_PYSCRIPT"

python = python
-include options.mk

# Main targets:

develop:
OPT="" $(python) setup.py build_ext --inplace $(build_opts)
OPT="" $(python) -m pip install --no-build-isolation -e .

help:
@$(python) -c "$$PRINT_HELP_PYSCRIPT" < $(MAKEFILE_LIST)
Expand All @@ -52,12 +50,10 @@ lint: ## check style with flake8
flake8 pixell tests

test: ## run tests quickly with the default Python
$(python) setup.py test $(build_opts)
pytest

coverage: ## check code coverage quickly with the default Python
coverage run --source pixell setup.py test
coverage report -m
coverage html
pytest --cov --cov-report=html
$(BROWSER) htmlcov/index.html

docs: ## generate Sphinx HTML documentation, including API docs
Expand All @@ -75,9 +71,8 @@ release: dist ## package and upload a release
twine upload dist/*

dist: clean ## builds source and wheel package
$(python) setup.py sdist $(build_opts)
$(python) setup.py bdist_wheel $(build_opts)
$(python) build
ls -l dist

install: clean ## install the package to the active Python's site-packages
$(python) setup.py install $(build_opts)
pip install .
Loading
Loading