Skip to content

🎑 Build Python wheels for all the platforms on CI with minimal configuration.

License

Notifications You must be signed in to change notification settings

EwoutH/cibuildwheel

Β 
Β 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

cibuildwheel

PyPI Documentation Status Actions Status Travis Status Appveyor status CircleCI Status Azure Status

Documentation

Python wheels are great. Building them across Mac, Linux, Windows, on multiple versions of Python, is not.

cibuildwheel is here to help. cibuildwheel runs on your CI server - currently it supports GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI - and it builds and tests your wheels across all of your platforms.

What does it do?

macOS Intel macOS Apple Silicon Windows 64bit Windows 32bit Windows Arm64 manylinux
musllinux x86_64
manylinux
musllinux i686
manylinux
musllinux aarch64
manylinux
musllinux ppc64le
manylinux
musllinux s390x
CPythonΒ 3.6 βœ… N/A βœ… βœ… N/A βœ… βœ… βœ… βœ… βœ…
CPythonΒ 3.7 βœ… N/A βœ… βœ… N/A βœ… βœ… βœ… βœ… βœ…
CPythonΒ 3.8 βœ… βœ… βœ… βœ… N/A βœ… βœ… βœ… βœ… βœ…
CPython 3.9 βœ… βœ… βœ… βœ… βœ…Β² βœ…Β³ βœ… βœ… βœ… βœ…
CPythonΒ 3.10 βœ… βœ… βœ… βœ… βœ…Β² βœ… βœ… βœ… βœ… βœ…
CPythonΒ 3.11⁴ βœ… βœ… βœ… βœ… βœ…Β² βœ… βœ… βœ… βœ… βœ…
PyPyΒ 3.7 v7.3 βœ… N/A βœ… N/A N/A βœ…ΒΉ βœ…ΒΉ βœ…ΒΉ N/A N/A
PyPyΒ 3.8 v7.3 βœ… N/A βœ… N/A N/A βœ…ΒΉ βœ…ΒΉ βœ…ΒΉ N/A N/A
PyPyΒ 3.9 v7.3 βœ… N/A βœ… N/A N/A βœ…ΒΉ βœ…ΒΉ βœ…ΒΉ N/A N/A

ΒΉ PyPy is only supported for manylinux wheels.
Β² Windows arm64 support is experimental.
Β³ Alpine 3.14 and very briefly 3.15's default python3 was not able to load musllinux wheels. This has been fixed; please upgrade the python package if using Alpine from before the fix.
⁴ CPython 3.11 is available using the CIBW_PRERELEASE_PYTHONS option.

  • Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
  • Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI
  • Bundles shared library dependencies on Linux and macOS through auditwheel and delocate
  • Runs your library's tests against the wheel-installed version of your library

See the cibuildwheel 1 documentation if you need to build unsupported versions of Python, such as Python 2.

Usage

cibuildwheel runs inside a CI service. Supported platforms depend on which service you're using:

Linux macOS Windows Linux ARM
GitHub Actions βœ… βœ… βœ… βœ…ΒΉ
Azure Pipelines βœ… βœ… βœ…
Travis CI βœ… βœ… βœ…
AppVeyor βœ… βœ… βœ…
CircleCI βœ… βœ…
Gitlab CI βœ…

ΒΉ Requires emulation, distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.

Example setup

To build manylinux, musllinux, macOS, and Windows wheels on GitHub Actions, you could use this .github/workflows/wheels.yml:

name: Build

on: [push, pull_request]

jobs:
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-20.04, windows-2019, macOS-10.15]

    steps:
      - uses: actions/checkout@v2

      # Used to host cibuildwheel
      - uses: actions/setup-python@v2

      - name: Install cibuildwheel
        run: python -m pip install cibuildwheel==2.6.0

      - name: Build wheels
        run: python -m cibuildwheel --output-dir wheelhouse
        # to supply options, put them in 'env', like:
        # env:
        #   CIBW_SOME_OPTION: value

      - uses: actions/upload-artifact@v2
        with:
          path: ./wheelhouse/*.whl

For more information, including PyPI deployment, and the use of other CI services or the dedicated GitHub Action, check out the documentation and the examples.

How it works

The following diagram summarises the steps that cibuildwheel takes on each platform.

Explore an interactive version of this diagram in the docs.

Options

Option Description
Build selection CIBW_PLATFORM Override the auto-detected target platform
CIBW_BUILD
CIBW_SKIP
Choose the Python versions to build
CIBW_ARCHS Change the architectures built on your machine by default.
CIBW_PROJECT_REQUIRES_PYTHON Manually set the Python compatibility of your project
CIBW_PRERELEASE_PYTHONS Enable building with pre-release versions of Python if available
Build customization CIBW_BUILD_FRONTEND Set the tool to use to build, either "pip" (default for now) or "build"
CIBW_ENVIRONMENT Set environment variables needed during the build
CIBW_ENVIRONMENT_PASS_LINUX Set environment variables on the host to pass-through to the container during the build.
CIBW_BEFORE_ALL Execute a shell command on the build system before any wheels are built.
CIBW_BEFORE_BUILD Execute a shell command preparing each wheel's build
CIBW_REPAIR_WHEEL_COMMAND Execute a shell command to repair each built wheel
CIBW_MANYLINUX_*_IMAGE
CIBW_MUSLLINUX_*_IMAGE
Specify alternative manylinux / musllinux Docker images
CIBW_DEPENDENCY_VERSIONS Specify how cibuildwheel controls the versions of the tools it uses
Testing CIBW_TEST_COMMAND Execute a shell command to test each built wheel
CIBW_BEFORE_TEST Execute a shell command before testing each wheel
CIBW_TEST_REQUIRES Install Python dependencies before running the tests
CIBW_TEST_EXTRAS Install your wheel for testing using extras_require
CIBW_TEST_SKIP Skip running tests on some builds
Other CIBW_BUILD_VERBOSITY Increase/decrease the output of pip wheel

These options can be specified in a pyproject.toml file, as well; see configuration.

Working examples

Here are some repos that use cibuildwheel.

Name CI OS Notes
scikit-learn github icon windows icon apple icon linux icon The machine learning library. A complex but clean config using many of cibuildwheel's features to build a large project with Cython and C++ extensions.
Tornado travisci icon apple icon linux icon Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed.
NumPy github icon travisci icon windows icon apple icon linux icon The fundamental package for scientific computing with Python.
pytorch-fairseq github icon apple icon linux icon Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
Matplotlib github icon windows icon apple icon linux icon The venerable Matplotlib, a Python library with C++ portions
Kivy github icon windows icon apple icon linux icon Open source UI framework written in Python, running on Windows, Linux, macOS, Android and iOS
NCNN github icon windows icon apple icon linux icon ncnn is a high-performance neural network inference framework optimized for the mobile platform
Prophet github icon windows icon apple icon linux icon Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.
MyPy github icon apple icon linux icon windows icon The compiled version of MyPy using MyPyC.
pydantic github icon apple icon linux icon windows icon Data parsing and validation using Python type hints

ℹ️ That's just a handful, there are many more! Check out the Working Examples page in the docs.

Legal note

Since cibuildwheel repairs the wheel with delocate or auditwheel, it might automatically bundle dynamically linked libraries from the build machine.

It helps ensure that the library can run without any dependencies outside of the pip toolchain.

This is similar to static linking, so it might have some license implications. Check the license for any code you're pulling in to make sure that's allowed.

Changelog

v2.6.0

25 May 2022

  • 🌟 Added the ability to test building wheels on CPython 3.11! Because CPython 3.11 is in beta, these wheels should not be distributed, because they might not be compatible with the final release, but it's available to build for testing purposes. Use the flag --prerelease-pythons or CIBW_PRERELEASE_PYTHONS to test. This version of cibuildwheel includes CPython 3.11.0b1. (#1109)
  • πŸ“š Added an interactive diagram showing how cibuildwheel works to the docs (#1100)

v2.5.0

29 April 2022

  • ✨ Added support for building ABI3 wheels. cibuildwheel will now recognise when an ABI3 wheel was produced, and skip subsequent build steps where the previously built wheel is compatible. Tests still will run on all selected versions of Python, using the ABI3 wheel. Check this entry in the docs for more info. (#1091)
  • ✨ You can now build wheels directly from sdist archives, in addition to source directories. Just call cibuildwheel with an sdist argument on the command line, like cibuildwheel mypackage-1.0.0.tar.gz. For more details, check the --help output (#1096)
  • πŸ› Fix a bug where cibuildwheel would crash when no builds are selected and --allow-empty is passed (#1086)
  • πŸ› Workaround a permissions issue on Linux relating to newer versions of git and setuptools_scm (#1095)
  • πŸ“š Minor docs improvements

v2.4.0

2 April 2022

  • ✨ cibuildwheel now supports running locally on Windows and macOS (as well as Linux). On macOS, you'll have to install the versions of Pythons that you want to use from Python.org, and cibuildwheel will use them. On Windows, cibuildwheel will install it's own versions of Python. Check out the documentation for instructions. (#974)
  • ✨ Added support for building PyPy 3.9 wheels. (#1031)
  • ✨ Listing at the end of the build now displays the size of each wheel (#975)
  • πŸ› Workaround a connection timeout bug on Travis CI ppc64le runners (#906)
  • πŸ› Fix an encoding error when reading setup.py in the wrong encoding (#977)
  • πŸ›  Setuptools updated to 61.3.0, including experimental support for reading config from pyproject.toml(PEP 621). This could change the behaviour of your build if you have a pyproject.toml with a [project] table, because that takes precedence over setup.py and setup.cfg. Check out the setuptools docs and the project metadata specification for more info.
  • πŸ›  Many other dependency updates.
  • πŸ“š Minor docs improvements

v2.3.1

14 December 2021

  • πŸ› Setting pip options like PIP_USE_DEPRECATED in CIBW_ENVIRONMENT no longer adversely affects cibuildwheel's ability to set up a Python environment (#956)
  • πŸ“š Docs fixes and improvements

v2.3.0

26 November 2021

  • πŸ“ˆ cibuildwheel now defaults to manylinux2014 image for linux builds, rather than manylinux2010. If you want to stick with manylinux2010, it's simple to set this using the image options. (#926)
  • ✨ You can now pass environment variables from the host machine into the Docker container during a Linux build. Check out the docs for CIBW_ENVIRONMENT_PASS_LINUX for the details. (#914)
  • ✨ Added support for building PyPy 3.8 wheels. (#881)
  • ✨ Added support for building Windows arm64 CPython wheels on a Windows arm64 runner. We can't test this in CI yet, so for now, this is experimental. (#920)
  • πŸ“š Improved the deployment documentation (#911)
  • πŸ›  Changed the escaping behaviour inside cibuildwheel's option placeholders e.g. {project} in before_build or {dest_dir} in repair_wheel_command. This allows bash syntax like ${SOME_VAR} to passthrough without being interpreted as a placeholder by cibuildwheel. See this section in the docs for more info. (#889)
  • πŸ›  Pip updated to 21.3, meaning it now defaults to in-tree builds again. If this causes an issue with your project, setting environment variable PIP_USE_DEPRECATED=out-of-tree-build is available as a temporary flag to restore the old behaviour. However, be aware that this flag will probably be removed soon. (#881)
  • πŸ› You can now access the current Python interpreter using python3 within a build on Windows (#917)

That's the last few versions.

ℹ️ Want more changelog? Head over to the changelog page in the docs.


Contributing

For more info on how to contribute to cibuildwheel, see the docs.

Everyone interacting with the cibuildwheel project via codebase, issue tracker, chat rooms, or otherwise is expected to follow the PSF Code of Conduct.

Maintainers

Credits

cibuildwheel stands on the shoulders of giants.

Massive props also to-

  • @zfrenchee for help debugging many issues
  • @lelit for some great bug reports and contributions
  • @mayeut for a phenomenal PR patching Python itself for better compatibility!
  • @czaki for being a super-contributor over many PRs and helping out with countless issues!
  • @mattip for his help with adding PyPy support to cibuildwheel

See also

Another very similar tool to consider is matthew-brett/multibuild. multibuild is a shell script toolbox for building a wheel on various platforms. It is used as a basis to build some of the big data science tools, like SciPy.

If you are building Rust wheels, you can get by without some of the tricks required to make GLIBC work via manylinux; this is especially relevant for cross-compiling, which is easy with Rust. See maturin-action for a tool that is optimized for building Rust wheels and cross-compiling.

About

🎑 Build Python wheels for all the platforms on CI with minimal configuration.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 99.9%
  • Shell 0.1%