Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support Python 3.13.0b1 (PEP 667 fix) #5127

Merged
merged 5 commits into from
May 24, 2024

Conversation

henryiii
Copy link
Collaborator

@henryiii henryiii commented May 10, 2024

Description

Since beta 1 is out, time to start testing all the time on Python 3.13, not just PR’s with a dev label.

Suggested changelog entry:

* Run CI on Python 3.13

@henryiii
Copy link
Collaborator Author

henryiii commented May 23, 2024

Some destructors are not being called when they used to be. The following three tests break:

FAILED ../../tests/test_factory_constructors.py::test_init_factory_alias - assert [3, 3] == [0, 0]
FAILED ../../tests/test_factory_constructors.py::test_init_factory_dual - assert [9, 6] == [7, 4]
FAILED ../../tests/test_virtual_functions.py::test_override - assert 2 == 0

In these, .alive() is called, which runs the gc, then the number of times .destroyed() is called is less on 3.13.0b1 than 3.13.0a6.

Running git bisect between v3.13.0a6 and v3.13.0b1 with the follow script:

#/usr/bin/env bash
set -euo pipefail

./configure --prefix $PWD/../python
make -j8
make install -j8

cd ..
uv pip install --python=./Python/bin/python3 pytest
cmake -S. -Bbuild -DPython_ROOT_DIR=Python -DPYBIND11_TEST_OVERRIDE='test_factory_constructors.py;test_virtual_functions.py'
cmake --build build --target pytest

Showed this broke with python/cpython#115153, PEP 667.

These are now replaced with new functions:

$ git grep -E 'PyEval_GetLocals|PyEval_GetGlobals|PyEval_GetBuiltins|\.f_locals'
include/pybind11/detail/internals.h:446:    state_dict = reinterpret_borrow<object>(PyEval_GetBuiltins());
include/pybind11/pybind11.h:1347:    PyObject *p = PyEval_GetGlobals();
include/pybind11/pybind11.h:2773:            PyObject *locals = PyEval_GetLocals();
include/pybind11/pybind11.h:2820:                       "        self_caller = frame.f_locals[frame.f_code.co_varnames[0]]\n"
tests/constructor_stats.h:116:        PyObject *globals = PyEval_GetGlobals();

These are now no-ops:

$ git grep -E 'PyFrame_FastToLocalsWithError|PyFrame_FastToLocals|git grep PyFrame_LocalsToFast'
include/pybind11/pybind11.h:2797:        PyFrame_FastToLocals(frame);

If I update to use the new functions (only a couple of instances, the rest are for PyPy), then nothing changes (Edit: never mind, messed up the version number, forgot a 0).

@henryiii henryiii force-pushed the henryiii/ci/py313 branch 2 times, most recently from 0a457ea to b5a125a Compare May 23, 2024 21:59
@henryiii henryiii changed the title ci: add Python 3.13 fix: support Python 3.13.0b1 (PEP 667 fix) May 23, 2024
@henryiii
Copy link
Collaborator Author

FYI, four tests fail (related to refcounts) when using free-threaded 3.13 but holding the GIL.

@henryiii henryiii merged commit 4d0fced into pybind:master May 24, 2024
82 checks passed
@henryiii henryiii deleted the henryiii/ci/py313 branch May 24, 2024 15:00
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label May 24, 2024
PyObject *locals = PyEval_GetLocals();
Py_INCREF(locals);
Copy link
Collaborator

@rwgk rwgk May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@henryiii This Py_INCREF(locals) appears before the locals != nullptr test, therefore I think it could segfault in object.h: op->ob_refcnt++;. Did you consider that already? Did you mean to use Py_XINCREF(locals) here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I did, thanks! I think it’s very unlikely that this will be null, but best to be correct!

rwgk pushed a commit to rwgk/pybind11 that referenced this pull request Jun 12, 2024
* Fix merge accident in pybind11/detail/descr.h (pybind#5086)

This was noticed only when manually reviewing the diffs with the Google review tools.

* Fix typo in changelog date (pybind#5096)

This was actually released in 2024!

* ci: macos-latest is changing to macos-14 ARM runners (pybind#5109)

Committed via https://github.com/asottile/all-repos

* chore: docs and nox bump (pybind#5071)

Signed-off-by: Henry Schreiner <[email protected]>

* docs(numpy): drop duplicated ndim (pybind#5119)

* chore(deps): bump idna from 3.6 to 3.7 in /docs (pybind#5121)

Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](kjd/idna@v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /docs (pybind#5122)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump the actions group with 1 update (pybind#5082)

* chore(deps): bump the actions group with 1 update

Bumps the actions group with 1 update: [actions/labeler](https://github.com/actions/labeler).


Updates `actions/labeler` from 4 to 5
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](actions/labeler@v4...v5)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <[email protected]>

* ci: fix labeler

Signed-off-by: Henry Schreiner <[email protected]>

* ci: move eigen to 64-bit only

Signed-off-by: Henry Schreiner <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Henry Schreiner <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <[email protected]>

* chore(deps): update pre-commit hooks (pybind#5123)

* chore(deps): update pre-commit hooks

updates:
- [github.com/pre-commit/mirrors-clang-format: v18.1.2 → v18.1.4](pre-commit/mirrors-clang-format@v18.1.2...v18.1.4)
- [github.com/astral-sh/ruff-pre-commit: v0.3.5 → v0.4.3](astral-sh/ruff-pre-commit@v0.3.5...v0.4.3)
- [github.com/pre-commit/mirrors-mypy: v1.9.0 → v1.10.0](pre-commit/mirrors-mypy@v1.9.0...v1.10.0)
- [github.com/pre-commit/pre-commit-hooks: v4.5.0 → v4.6.0](pre-commit/pre-commit-hooks@v4.5.0...v4.6.0)
- [github.com/python-jsonschema/check-jsonschema: 0.28.1 → 0.28.2](python-jsonschema/check-jsonschema@0.28.1...0.28.2)

* style: pre-commit fixes

* Update .pre-commit-config.yaml

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <[email protected]>

* --- (pybind#5130)

updated-dependencies:
- dependency-name: requests
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: support Python 3.13.0b1 (PEP 667 fix) (pybind#5127)

* ci: add Python 3.13

Signed-off-by: Henry Schreiner <[email protected]>

* tests: run the gc for 3.13+

Signed-off-by: Henry Schreiner <[email protected]>

* Revert "tests: run the gc for 3.13+"

This reverts commit fe8a3ce.

* ci: drop macos ARM for now, need pin updates

Signed-off-by: Henry Schreiner <[email protected]>

* fix: use Python 3.13 API if on 3.13

Signed-off-by: Henry Schreiner <[email protected]>

---------

Signed-off-by: Henry Schreiner <[email protected]>

* chore: some cleanup (pybind#5137)

* Tracking ci.yml changes from master.

---------

Signed-off-by: Henry Schreiner <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Tim Stumbaugh <[email protected]>
Co-authored-by: Henry Schreiner <[email protected]>
Co-authored-by: nobkd <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@henryiii henryiii removed the needs changelog Possibly needs a changelog entry label Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants