Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare Python 3.11 support #31170

Merged
merged 52 commits into from
Jan 7, 2023
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
7a18111
python3.11 prep
csko Dec 17, 2022
9255b7a
fix recursion limit
csko Dec 17, 2022
65e77a9
update cicd, buildkite
csko Dec 17, 2022
d5fa276
upgrade cython to 0.29.32
csko Dec 18, 2022
4adad69
pydantic
csko Dec 18, 2022
814581b
update cloudpickle
csko Dec 18, 2022
5ccaa98
condition recursion_depth on <3.11
csko Dec 20, 2022
1f0015c
lint
csko Dec 20, 2022
01dcb87
condition pip packages on py311
csko Dec 20, 2022
1d9daf8
fix wheels
csko Dec 20, 2022
466a835
lint
csko Dec 20, 2022
ddc700c
add 3.11 to wheel scripts
csko Dec 21, 2022
f751360
add numpy version to wheel script
csko Dec 21, 2022
4d33b4f
more specific pydantic versions
csko Dec 21, 2022
9726e0a
use newer pypa images with 311
csko Dec 21, 2022
f33ea25
fix macos build
csko Dec 21, 2022
350a69f
Merge branch 'ray-project:master' into py311
csko Dec 21, 2022
7dc33cd
add 311 to wheel test
csko Dec 21, 2022
fb05c4a
try build without 311
csko Dec 21, 2022
90cb548
Revert "use newer pypa images with 311"
csko Dec 22, 2022
7823dc1
try 2022-02-24-3876535 manylinux2014
csko Dec 22, 2022
722760e
try 2022-01-02-2b36e6e manylinux2014
csko Dec 22, 2022
4d54aa3
try 2021-12-05-142ef77 manylinux2014
csko Dec 22, 2022
c50db92
try 2021-12-19-a4d7f5a manylinux2014
csko Dec 22, 2022
d8005c6
try 2021-12-30-cb9fd5b manylinux2014
csko Dec 23, 2022
fac2e49
Update ci.sh
csko Dec 24, 2022
7013382
use pypa version that supports py3.11
csko Dec 25, 2022
247d3f6
fix typo
csko Dec 25, 2022
8e997a6
try conda-forge
csko Dec 25, 2022
7666831
fix gpu docker images
csko Dec 25, 2022
562ab4f
add back py3.11 to build-wheel-manylinux2014
csko Dec 25, 2022
9763dc6
condition test packages on python3.11
csko Jan 2, 2023
e227548
revert conda-forge
csko Jan 2, 2023
173963f
don't build docker for 3.11
csko Jan 2, 2023
53cf70a
don't build macos and windows wheels
csko Jan 2, 2023
bda9a36
revert cython upgrade in release test
csko Jan 2, 2023
bec10c4
Update documentation
csko Jan 2, 2023
76a3011
Merge branch 'ray-project:master' into py311
csko Jan 2, 2023
8f87f0f
remove purelib reference in commit check
csko Jan 4, 2023
0f2b123
remove mac 3.11 wheels
csko Jan 4, 2023
aa313c7
Merge branch 'master' into py311
csko Jan 4, 2023
d3ea5fd
condition llvmlite on py<3.11
csko Jan 4, 2023
44006df
don't check for 3.11 docker image
csko Jan 4, 2023
c09c347
remove recursion_depth ref
csko Jan 5, 2023
d4c8ee2
remove unused build step
csko Jan 5, 2023
71dfb8a
match __commit__ only on the beginning of the line in wheel check
csko Jan 5, 2023
22538e6
Merge branch 'ray-project:master' into py311
csko Jan 5, 2023
00d7138
remove other commented-out 3.11 references
csko Jan 6, 2023
f22eaab
remove gitpod (merge upstream)
csko Jan 6, 2023
513b97b
Merge branch 'ray-project:master' into py311
csko Jan 6, 2023
241dc32
update docs
csko Jan 6, 2023
d72e1d7
fix table in documentation
csko Jan 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .buildkite/pipeline.build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,14 @@
- ./ci/ci.sh test_minimal 3.10
- ./ci/ci.sh test_latest_core_dependencies 3.10

- label: ":python: Minimal install 3.11"
conditions: ["RAY_CI_PYTHON_AFFECTED"]
instance_size: medium
commands:
- cleanup() { if [ "${BUILDKITE_PULL_REQUEST}" = "false" ]; then ./ci/build/upload_build_info.sh; fi }; trap cleanup EXIT
- ./ci/ci.sh test_minimal 3.11
- ./ci/ci.sh test_latest_core_dependencies 3.11

- label: ":python: Default install"
conditions: ["RAY_CI_PYTHON_AFFECTED"]
instance_size: small
Expand Down
4 changes: 2 additions & 2 deletions bazel/ray_deps_setup.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,8 @@ def ray_deps_setup():
auto_http_archive(
name = "cython",
build_file = True,
url = "https://github.com/cython/cython/archive/3028e8c7ac296bc848d996e397c3354b3dbbd431.tar.gz",
sha256 = "31ea23c2231ddee8572a2a5effd54952e16a1b44e9a4cb3eb645418f8accf20d",
url = "https://github.com/cython/cython/archive/c48361d0a0969206e227ec016f654c9d941c2b69.tar.gz",
sha256 = "37c466fea398da9785bc37fe16f1455d2645d21a72e402103991d9e2fa1c6ff3",
)

auto_http_archive(
Expand Down
7 changes: 4 additions & 3 deletions ci/build/build-docker-images.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,9 @@
"cu112": "nvidia/cuda:11.2.0-cudnn8-devel-ubuntu20.04",
"cu111": "nvidia/cuda:11.1.1-cudnn8-devel-ubuntu20.04",
"cu110": "nvidia/cuda:11.0.3-cudnn8-devel-ubuntu20.04",
"cu102": "nvidia/cuda:10.2-cudnn8-devel-ubuntu20.04",
rickyyx marked this conversation as resolved.
Show resolved Hide resolved
"cu101": "nvidia/cuda:10.1-cudnn8-devel-ubuntu20.04",
# there is no ubuntu20.04 image for cuda 10.2 and 10.1
"cu102": "nvidia/cuda:10.2-cudnn8-devel-ubuntu18.04",
"cu101": "nvidia/cuda:10.1-cudnn8-devel-ubuntu18.04",
"cpu": "ubuntu:focal",
}

Expand Down Expand Up @@ -619,7 +620,7 @@ def push_readmes(merge_build: bool):
default="py37",
nargs="*",
help="Which python versions to build. "
"Must be in (py36, py37, py38, py39, py310)",
"Must be in (py36, py37, py38, py39, py310, py311)",
)
parser.add_argument(
"--device-types",
Expand Down
3 changes: 2 additions & 1 deletion ci/build/test-wheels.sh
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,8 @@ elif [[ "$platform" == "macosx" ]]; then
"3.7"
"3.8"
"3.9"
"3.10")
"3.10"
)

for ((i=0; i<${#PY_MMS[@]}; ++i)); do
PY_MM="${PY_MMS[i]}"
Expand Down
7 changes: 3 additions & 4 deletions ci/ci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -414,8 +414,7 @@ validate_wheels_commit_str() {
continue
fi

folder=${basename%%-cp*}
WHL_COMMIT=$(unzip -p "$whl" "${folder}.data/purelib/ray/__init__.py" | grep "__commit__" | awk -F'"' '{print $2}')
WHL_COMMIT=$(unzip -p "$whl" | grep "^__commit__" | awk -F'"' '{print $2}')

if [ "${WHL_COMMIT}" != "${EXPECTED_COMMIT}" ]; then
echo "Error: Observed wheel commit (${WHL_COMMIT}) is not expected commit (${EXPECTED_COMMIT}). Aborting."
Expand Down Expand Up @@ -459,7 +458,7 @@ build_wheels() {
# This command should be kept in sync with ray/python/README-building-wheels.md,
# except the "${MOUNT_BAZEL_CACHE[@]}" part.
docker run --rm -w /ray -v "${PWD}":/ray "${MOUNT_BAZEL_CACHE[@]}" \
quay.io/pypa/manylinux2014_x86_64:2021-11-07-28723f3 /ray/python/build-wheel-manylinux2014.sh
quay.io/pypa/manylinux2014_x86_64:2022-12-20-b4884d9 /ray/python/build-wheel-manylinux2014.sh
else
rm -rf /ray-mount/*
rm -rf /ray-mount/.whl || true
Expand All @@ -469,7 +468,7 @@ build_wheels() {
docker run --rm -v /ray:/ray-mounted ubuntu:focal ls /
docker run --rm -v /ray:/ray-mounted ubuntu:focal ls /ray-mounted
docker run --rm -w /ray -v /ray:/ray "${MOUNT_BAZEL_CACHE[@]}" \
quay.io/pypa/manylinux2014_x86_64:2021-11-07-28723f3 /ray/python/build-wheel-manylinux2014.sh
quay.io/pypa/manylinux2014_x86_64:2022-12-20-b4884d9 /ray/python/build-wheel-manylinux2014.sh
cp -rT /ray-mount /ray # copy new files back here
find . | grep whl # testing

Expand Down
2 changes: 2 additions & 0 deletions ci/env/install-minimal.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ else
PYTHON_VERSION="3.9"
elif [ "$1" = "3.10" ]; then
PYTHON_VERSION="3.10"
elif [ "$1" = "3.11" ]; then
PYTHON_VERSION="3.11"
else
echo "Unsupported Python version."
exit 1
Expand Down
16 changes: 11 additions & 5 deletions doc/source/ray-overview/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,12 @@ You can install the nightly Ray wheels via the following links. These daily rele
==================== ==================== =======================
Linux MacOS Windows (beta)
==================== ==================== =======================
`Linux Python 3.10`_ `MacOS Python 3.10`_ `Windows Python 3.10`_
`Linux Python 3.9`_ `MacOS Python 3.9`_ `Windows Python 3.9`_
`Linux Python 3.8`_ `MacOS Python 3.8`_ `Windows Python 3.8`_
`Linux Python 3.7`_ `MacOS Python 3.7`_ `Windows Python 3.7`_
`Linux Python 3.6`_ `MacOS Python 3.6`_
`Linux Python 3.11`_ `MacOS Python 3.10`_ `Windows Python 3.10`_
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we avoid this "off by one" format? maybe just put 3.11 at the bottom (and label as experimental if necessary?)

`Linux Python 3.10`_ `MacOS Python 3.9`_ `Windows Python 3.9`_
`Linux Python 3.9`_ `MacOS Python 3.8`_ `Windows Python 3.8`_
`Linux Python 3.8`_ `MacOS Python 3.7`_ `Windows Python 3.7`_
`Linux Python 3.7`_ `MacOS Python 3.6`_
`Linux Python 3.6`_
==================== ==================== =======================

.. note::
Expand All @@ -68,6 +69,11 @@ You can install the nightly Ray wheels via the following links. These daily rele

:ref:`Usage stats <ref-usage-stats>` collection is enabled by default (can be :ref:`disabled <usage-disable>`) for nightly wheels including both local clusters started via ``ray.init()`` and remote clusters via cli.

.. note::

Python 3.11 support is experimental.

.. _`Linux Python 3.11`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp311-cp311-manylinux2014_x86_64.whl
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the process for adding this artifact? The LinkCheck check if failing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we could ignore the soft failure - the wheel will get automatically uploaded to the URL if it's a CI job in master.

.. _`Linux Python 3.10`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp310-cp310-manylinux2014_x86_64.whl
.. _`Linux Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp39-cp39-manylinux2014_x86_64.whl
.. _`Linux Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp38-cp38-manylinux2014_x86_64.whl
Expand Down
2 changes: 1 addition & 1 deletion docker/base-deps/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ RUN sudo apt-get update -y && sudo apt-get upgrade -y \
&& $HOME/anaconda3/bin/conda clean -y --all \
&& $HOME/anaconda3/bin/pip install --no-cache-dir \
flatbuffers \
cython==0.29.26 \
cython==0.29.32 \
# Necessary for Dataset to work properly.
numpy\>=1.20 \
psutil \
Expand Down
2 changes: 1 addition & 1 deletion docker/ray-worker-container/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ RUN apt-get update -y && sudo apt-get upgrade -y \
&& $HOME/anaconda3/bin/conda clean -y --all \
&& $HOME/anaconda3/bin/pip install --no-cache-dir \
flatbuffers \
cython==0.29.26 \
cython==0.29.32 \
numpy==1.15.4 \
psutil \
# To avoid the following error on Jenkins:
Expand Down
2 changes: 1 addition & 1 deletion python/build-wheel-macos-arm64.sh
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ for ((i=0; i<${#PY_VERSIONS[@]}; ++i)); do
# Setuptools on CentOS is too old to install arrow 0.9.0, therefore we upgrade.
# TODO: Unpin after https://github.com/pypa/setuptools/issues/2849 is fixed.
$PIP_CMD install --upgrade setuptools==58.4
$PIP_CMD install -q cython==0.29.26
$PIP_CMD install -q cython==0.29.32
# Install wheel to avoid the error "invalid command 'bdist_wheel'".
$PIP_CMD install -q wheel
# Set the commit SHA in __init__.py.
Expand Down
14 changes: 9 additions & 5 deletions python/build-wheel-macos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,27 @@ PY_VERSIONS=("3.6.2"
"3.7.0"
"3.8.2"
"3.9.1"
"3.10.4")
"3.10.4"
csko marked this conversation as resolved.
Show resolved Hide resolved
)
PY_INSTS=("python-3.6.2-macosx10.6.pkg"
"python-3.7.0-macosx10.6.pkg"
"python-3.8.2-macosx10.9.pkg"
"python-3.9.1-macosx10.9.pkg"
"python-3.10.4-macos11.pkg")
"python-3.10.4-macos11.pkg"
)
PY_MMS=("3.6"
"3.7"
"3.8"
"3.9"
"3.10")
"3.10"
)

NUMPY_VERSIONS=("1.14.5"
"1.14.5"
"1.14.5"
"1.19.3"
"1.22.0")
"1.22.0"
)

./ci/env/install-bazel.sh

Expand Down Expand Up @@ -93,7 +97,7 @@ for ((i=0; i<${#PY_VERSIONS[@]}; ++i)); do
$PIP_CMD install -q setuptools_scm==3.1.0
# Fix the numpy version because this will be the oldest numpy version we can
# support.
$PIP_CMD install -q numpy=="$NUMPY_VERSION" cython==0.29.26
$PIP_CMD install -q numpy=="$NUMPY_VERSION" cython==0.29.32
# Install wheel to avoid the error "invalid command 'bdist_wheel'".
$PIP_CMD install -q wheel
# Set the commit SHA in __init__.py.
Expand Down
6 changes: 4 additions & 2 deletions python/build-wheel-manylinux2014.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@ PYTHONS=("cp36-cp36m"
"cp37-cp37m"
"cp38-cp38"
"cp39-cp39"
"cp310-cp310")
"cp310-cp310"
"cp311-cp311")
rickyyx marked this conversation as resolved.
Show resolved Hide resolved

NUMPY_VERSIONS=("1.14.5"
"1.14.5"
"1.14.5"
"1.19.3"
"1.22.0"
"1.22.0")

yum -y install unzip zip sudo
Expand Down Expand Up @@ -87,7 +89,7 @@ for ((i=0; i<${#PYTHONS[@]}; ++i)); do
pushd python
# Fix the numpy version because this will be the oldest numpy version we can
# support.
/opt/python/"${PYTHON}"/bin/pip install -q numpy=="${NUMPY_VERSION}" cython==0.29.26
/opt/python/"${PYTHON}"/bin/pip install -q numpy=="${NUMPY_VERSION}" cython==0.29.32
# Set the commit SHA in __init__.py.
if [ -n "$TRAVIS_COMMIT" ]; then
sed -i.bak "s/{{RAY_COMMIT_SHA}}/$TRAVIS_COMMIT/g" ray/__init__.py && rm ray/__init__.py.bak
Expand Down
3 changes: 2 additions & 1 deletion python/build-wheel-windows.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ WORKSPACE_DIR="${ROOT_DIR}/.."
PY_VERSIONS=("3.7"
"3.8"
"3.9"
"3.10")
"3.10"
csko marked this conversation as resolved.
Show resolved Hide resolved
)

bazel_preclean() {
"${WORKSPACE_DIR}"/ci/run/bazel.py preclean "mnemonic(\"Genrule\", deps(//:*))"
Expand Down
3 changes: 2 additions & 1 deletion python/ray/_raylet.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@ cdef extern from "Python.h":

# You can find the cpython definition in Include/cpython/pystate.h#L59
ctypedef struct CPyThreadState "PyThreadState":
int recursion_depth
int recursion_limit
int recursion_remaining

# From Include/ceveal.h#67
int Py_GetRecursionLimit()
Expand Down
12 changes: 11 additions & 1 deletion python/ray/_raylet.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -261,9 +261,19 @@ cdef increase_recursion_limit():
"""Double the recusion limit if current depth is close to the limit"""
cdef:
CPyThreadState * s = <CPyThreadState *> PyThreadState_Get()
int current_depth = s.recursion_depth
int current_limit = Py_GetRecursionLimit()
int new_limit = current_limit * 2
cdef extern from *:
rickyyx marked this conversation as resolved.
Show resolved Hide resolved
"""
#if PY_VERSION_HEX >= 0x30B00A4
#define CURRENT_DEPTH(x) ((x)->recursion_limit - (x)->recursion_remaining)
#else
#define CURRENT_DEPTH(x) ((x)->recursion_depth)
#endif
"""
int CURRENT_DEPTH(CPyThreadState *x)

int current_depth = CURRENT_DEPTH(s)

if current_limit - current_depth < 500:
Py_SetRecursionLimit(new_limit)
Expand Down
2 changes: 1 addition & 1 deletion python/ray/autoscaler/aws/development-example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ setup_commands:
- git clone https://github.com/ray-project/ray || true
- ray/ci/env/install-bazel.sh
- cd ray/python/ray/dashboard/client; npm ci; npm run build
- pip install boto3==1.4.8 cython==0.29.26 aiohttp grpcio psutil setproctitle
- pip install boto3==1.4.8 cython==0.29.32 aiohttp grpcio psutil setproctitle
- cd ray/python; pip install -e . --verbose

# Command to start ray on the head node. You don't need to change this.
Expand Down
Loading