Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes for 3.13 #3005

Merged
merged 17 commits into from
Jun 13, 2024
Merged

Fixes for 3.13 #3005

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c37f0cd
Try out naive KI protection fix
A5rocks May 24, 2024
9c97949
Update generated files
A5rocks May 24, 2024
3af42bb
Account for `._local` in full name
A5rocks May 24, 2024
e4fff1c
Add 3.13 to CI matrix
A5rocks May 24, 2024
c1c4b72
Avoid cryptography
A5rocks May 24, 2024
dc372df
Avoid an evil operation, I guess
A5rocks May 24, 2024
7f64b79
un-inline ki protection decorator
richardsheridan May 27, 2024
9a2fee4
Advertise 3.12 + maybe cryptography works?
A5rocks Jun 10, 2024
b1c304c
Switch back to avoiding cryptography
A5rocks Jun 10, 2024
f774b19
Try cryptography one more time
A5rocks Jun 12, 2024
e15bf6f
Add symbols
A5rocks Jun 12, 2024
dedaf21
Try one other possibility for symbols
A5rocks Jun 12, 2024
460a1ce
More symbol trial and error
A5rocks Jun 12, 2024
b3f76b1
Try to pick up patterns in what allows what
A5rocks Jun 12, 2024
7a67fb3
Test hypothesis
A5rocks Jun 12, 2024
a9410d8
Finish off failures
A5rocks Jun 12, 2024
f62e81e
Remove trailing whitespace
A5rocks Jun 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python: ['pypy-3.9', 'pypy-3.10', '3.8', '3.9', '3.10', '3.11', '3.12']
python: ['pypy-3.9', 'pypy-3.10', '3.8', '3.9', '3.10', '3.11', '3.12', '3.13']
check_formatting: ['0']
no_test_requirements: ['0']
extra_name: ['']
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Networking",
"Typing :: Typed",
]
Expand Down
5 changes: 3 additions & 2 deletions src/trio/_core/_generated_instrumentation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# *************************************************************
from __future__ import annotations

import sys
from typing import TYPE_CHECKING

from ._ki import LOCALS_KEY_KI_PROTECTION_ENABLED
Expand All @@ -23,7 +24,7 @@ def add_instrument(instrument: Instrument) -> None:
If ``instrument`` is already active, does nothing.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.instruments.add_instrument(instrument)
except AttributeError:
Expand All @@ -43,7 +44,7 @@ def remove_instrument(instrument: Instrument) -> None:
deactivated.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.instruments.remove_instrument(instrument)
except AttributeError:
Expand Down
8 changes: 4 additions & 4 deletions src/trio/_core/_generated_io_epoll.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
# *************************************************************
from __future__ import annotations

import sys
from typing import TYPE_CHECKING

from ._ki import LOCALS_KEY_KI_PROTECTION_ENABLED
from ._run import GLOBAL_RUN_CONTEXT

if TYPE_CHECKING:
from .._file_io import _HasFileNo
import sys

assert not TYPE_CHECKING or sys.platform == "linux"

Expand Down Expand Up @@ -40,7 +40,7 @@ async def wait_readable(fd: int | _HasFileNo) -> None:
if another task calls :func:`notify_closing` while this
function is still working.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_readable(fd)
except AttributeError:
Expand All @@ -59,7 +59,7 @@ async def wait_writable(fd: int | _HasFileNo) -> None:
if another task calls :func:`notify_closing` while this
function is still working.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_writable(fd)
except AttributeError:
Expand Down Expand Up @@ -91,7 +91,7 @@ def notify_closing(fd: int | _HasFileNo) -> None:
step, so other tasks won't be able to tell what order they happened
in anyway.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.notify_closing(fd)
except AttributeError:
Expand Down
14 changes: 7 additions & 7 deletions src/trio/_core/_generated_io_kqueue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# *************************************************************
from __future__ import annotations

import sys
from typing import TYPE_CHECKING, Callable, ContextManager

from ._ki import LOCALS_KEY_KI_PROTECTION_ENABLED
Expand All @@ -14,7 +15,6 @@
from .. import _core
from .._file_io import _HasFileNo
from ._traps import Abort, RaiseCancelT
import sys

assert not TYPE_CHECKING or sys.platform == "darwin"

Expand All @@ -34,7 +34,7 @@ def current_kqueue() -> select.kqueue:
anything real. See `#26
<https://github.com/python-trio/trio/issues/26>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.current_kqueue()
except AttributeError:
Expand All @@ -48,7 +48,7 @@ def monitor_kevent(
anything real. See `#26
<https://github.com/python-trio/trio/issues/26>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.monitor_kevent(ident, filter)
except AttributeError:
Expand All @@ -62,7 +62,7 @@ async def wait_kevent(
anything real. See `#26
<https://github.com/python-trio/trio/issues/26>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_kevent(
ident, filter, abort_func
Expand Down Expand Up @@ -93,7 +93,7 @@ async def wait_readable(fd: int | _HasFileNo) -> None:
if another task calls :func:`notify_closing` while this
function is still working.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_readable(fd)
except AttributeError:
Expand All @@ -112,7 +112,7 @@ async def wait_writable(fd: int | _HasFileNo) -> None:
if another task calls :func:`notify_closing` while this
function is still working.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_writable(fd)
except AttributeError:
Expand Down Expand Up @@ -144,7 +144,7 @@ def notify_closing(fd: int | _HasFileNo) -> None:
step, so other tasks won't be able to tell what order they happened
in anyway.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.notify_closing(fd)
except AttributeError:
Expand Down
20 changes: 10 additions & 10 deletions src/trio/_core/_generated_io_windows.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# *************************************************************
from __future__ import annotations

import sys
from typing import TYPE_CHECKING, ContextManager

from ._ki import LOCALS_KEY_KI_PROTECTION_ENABLED
Expand All @@ -14,7 +15,6 @@
from .._file_io import _HasFileNo
from ._unbounded_queue import UnboundedQueue
from ._windows_cffi import CData, Handle
import sys

assert not TYPE_CHECKING or sys.platform == "win32"

Expand Down Expand Up @@ -54,7 +54,7 @@ async def wait_readable(sock: _HasFileNo | int) -> None:
if another task calls :func:`notify_closing` while this
function is still working.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_readable(sock)
except AttributeError:
Expand All @@ -73,7 +73,7 @@ async def wait_writable(sock: _HasFileNo | int) -> None:
if another task calls :func:`notify_closing` while this
function is still working.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_writable(sock)
except AttributeError:
Expand Down Expand Up @@ -105,7 +105,7 @@ def notify_closing(handle: Handle | int | _HasFileNo) -> None:
step, so other tasks won't be able to tell what order they happened
in anyway.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.notify_closing(handle)
except AttributeError:
Expand All @@ -118,7 +118,7 @@ def register_with_iocp(handle: int | CData) -> None:
<https://github.com/python-trio/trio/issues/26>`__ and `#52
<https://github.com/python-trio/trio/issues/52>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.register_with_iocp(handle)
except AttributeError:
Expand All @@ -131,7 +131,7 @@ async def wait_overlapped(handle_: int | CData, lpOverlapped: CData | int) -> ob
<https://github.com/python-trio/trio/issues/26>`__ and `#52
<https://github.com/python-trio/trio/issues/52>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_overlapped(
handle_, lpOverlapped
Expand All @@ -148,7 +148,7 @@ async def write_overlapped(
<https://github.com/python-trio/trio/issues/26>`__ and `#52
<https://github.com/python-trio/trio/issues/52>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.write_overlapped(
handle, data, file_offset
Expand All @@ -165,7 +165,7 @@ async def readinto_overlapped(
<https://github.com/python-trio/trio/issues/26>`__ and `#52
<https://github.com/python-trio/trio/issues/52>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.io_manager.readinto_overlapped(
handle, buffer, file_offset
Expand All @@ -180,7 +180,7 @@ def current_iocp() -> int:
<https://github.com/python-trio/trio/issues/26>`__ and `#52
<https://github.com/python-trio/trio/issues/52>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.current_iocp()
except AttributeError:
Expand All @@ -193,7 +193,7 @@ def monitor_completion_key() -> ContextManager[tuple[int, UnboundedQueue[object]
<https://github.com/python-trio/trio/issues/26>`__ and `#52
<https://github.com/python-trio/trio/issues/52>`__.
"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.io_manager.monitor_completion_key()
except AttributeError:
Expand Down
17 changes: 9 additions & 8 deletions src/trio/_core/_generated_run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# *************************************************************
from __future__ import annotations

import sys
from typing import TYPE_CHECKING, Any

from ._ki import LOCALS_KEY_KI_PROTECTION_ENABLED
Expand Down Expand Up @@ -55,7 +56,7 @@ def current_statistics() -> RunStatistics:
other attributes vary between backends.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.current_statistics()
except AttributeError:
Expand All @@ -72,7 +73,7 @@ def current_time() -> float:
RuntimeError: if not inside a call to :func:`trio.run`.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.current_time()
except AttributeError:
Expand All @@ -81,7 +82,7 @@ def current_time() -> float:

def current_clock() -> Clock:
"""Returns the current :class:`~trio.abc.Clock`."""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.current_clock()
except AttributeError:
Expand All @@ -94,7 +95,7 @@ def current_root_task() -> Task | None:
This is the task that is the ultimate parent of all other tasks.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.current_root_task()
except AttributeError:
Expand All @@ -119,7 +120,7 @@ def reschedule(task: Task, next_send: Outcome[Any] = _NO_SEND) -> None:
raise) from :func:`wait_task_rescheduled`.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.reschedule(task, next_send)
except AttributeError:
Expand Down Expand Up @@ -183,7 +184,7 @@ def spawn_system_task(
Task: the newly spawned task

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.spawn_system_task(
async_fn, *args, name=name, context=context
Expand All @@ -197,7 +198,7 @@ def current_trio_token() -> TrioToken:
:func:`trio.run`.

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return GLOBAL_RUN_CONTEXT.runner.current_trio_token()
except AttributeError:
Expand Down Expand Up @@ -262,7 +263,7 @@ async def test_lock_fairness():
print("FAIL")

"""
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
try:
return await GLOBAL_RUN_CONTEXT.runner.wait_all_tasks_blocked(cushion)
except AttributeError:
Expand Down
2 changes: 1 addition & 1 deletion src/trio/_core/_ki.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ def wrapper(*args: ArgsT.args, **kwargs: ArgsT.kwargs) -> RetT: # type: ignore[

@wraps(fn)
def wrapper(*args: ArgsT.args, **kwargs: ArgsT.kwargs) -> RetT:
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = enabled
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = enabled
return fn(*args, **kwargs)

return wrapper
Expand Down
11 changes: 2 additions & 9 deletions src/trio/_core/_run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -623,6 +623,7 @@ def _close(self, exc: BaseException | None) -> BaseException | None:
self._cancel_status = None
return exc

@enable_ki_protection
def __exit__(
self,
etype: type[BaseException] | None,
Expand All @@ -633,10 +634,6 @@ def __exit__(
# so __exit__() must be just _close() plus this logic for adapting
# the exception-filtering result to the context manager API.

# This inlines the enable_ki_protection decorator so we can fix
# f_locals *locally* below to avoid reference cycles
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True

# Tracebacks show the 'raise' line below out of context, so let's give
# this variable a name that makes sense out of context.
remaining_error_after_cancel_scope = self._close(exc)
Expand All @@ -658,10 +655,6 @@ def __exit__(
# see test_cancel_scope_exit_doesnt_create_cyclic_garbage
# Note: still relevant
del remaining_error_after_cancel_scope, value, _, exc
# deep magic to remove refs via f_locals
locals()
# TODO: check if PEP558 changes the need for this call
# https://github.com/python/cpython/pull/3640
Comment on lines -661 to -664
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Back when we were making reference cycle tests, calling locals() when an exception was around would make a reference cycle for every exception (i.e. every Cancelled) unless you did both del and removed it from the f_locals dict. if the test is passing on all versions without it, though, no reason to keep it around

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test_cancel_scope_exit_doesnt_create_cyclic_garbage test fails on Fedora with Python 3.12 but passes with 3.13 :/

(However, we are stuck at 0.23.1, we are backporting this on top.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps something like:

if sys.version_info < (3, 13):
    locals()

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replacing locals() with sys._getframe().f_locals also does the trick for both Python 3.12 and Python 3.13. The code is too magical for me to tell whether it's factually correct.

Copy link
Contributor Author

@A5rocks A5rocks May 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm: latest state of the PR fails tests on fedora with Python 3.12? (That's not a platform in CI so very possible)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused cause when I used Python 3.12.3 on Fedora (Docker, I built CPython 3.12.3 from source) and pytest --pyargs trio --skip-optional-imports on this PR, the tests pass. Am I missing something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said, we are stuck at 0.23.1, we are backporting this PR on top of it, so perhaps that makes a difference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said, we are stuck at 0.23.1, we are backporting this PR on top of it, so perhaps that makes a difference.

what's the reason you're stuck at 0.23.1? Looking at the changelog of 0.23.2 doesn't look like it should be introducing anything controversial.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python-trio/trio-websocket#187 and possibly others

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that should only be affected from 0.25.0 though. If you're at the point where you're cherrypicking commits you could instead revert cfc0755


def __repr__(self) -> str:
if self._cancel_status is not None:
Expand Down Expand Up @@ -2476,7 +2469,7 @@ def unrolled_run(
args: tuple[Unpack[PosArgT]],
host_uses_signal_set_wakeup_fd: bool = False,
) -> Generator[float, EventResult, None]:
locals()[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
sys._getframe().f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = True
__tracebackhide__ = True

try:
Expand Down
2 changes: 2 additions & 0 deletions src/trio/_path.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,8 @@ def __repr__(self) -> str:
write_text = _wrap_method(pathlib.Path.write_text)
if sys.version_info < (3, 12):
link_to = _wrap_method(pathlib.Path.link_to)
if sys.version_info >= (3, 13):
full_match = _wrap_method(pathlib.Path.full_match)


@final
Expand Down
Loading
Loading