gh-76785: Crossinterp utils additions #111530

ericsnowcurrently · 2023-10-30T23:09:31Z

This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers).

Specifically:

_Py_excinfo, etc.: the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c)
_PyXI_exception_info, etc.: helpers for passing an exception beween interpreters (wraps _Py_excinfo)
_PyXI_namespace, etc.: helpers for copying a dict of attrs between interpreters
_PyXI_Enter(), _PyXI_Exit(): functions that abstract out the transitions between one interpreter and a second that will do some work temporarily

Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations. I plan on proposing these as public API at some point.

Issue: [subinterpreters] PEP 554 implementation: add interpreters module #76785

…rror_type().

There were a few corner cases I didn't handle properly in gh-111530, which I've noticed while working on a follow-up PR. This fixes those cases.

This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers). Specifically: * _Py_excinfo, etc.: the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c) * _PyXI_exception_info, etc.: helpers for passing an exception beween interpreters (wraps _Py_excinfo) * _PyXI_namespace, etc.: helpers for copying a dict of attrs between interpreters * _PyXI_Enter(), _PyXI_Exit(): functions that abstract out the transitions between one interpreter and a second that will do some work temporarily Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations. I plan on proposing these as public API at some point.

There were a few corner cases I didn't handle properly in pythongh-111530, which I've noticed while working on a follow-up PR. This fixes those cases.

…1715) I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in gh-111530 (9322ce9). Since then I've had a nagging sense that I should have added the type and functions in its own PR. While I do plan on using _Py_excinfo outside crossinterp.c very soon (see gh-111572/gh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR. Hence, here we are. (FWIW, I may combine that with gh-111572, which I may, in turn, combine with gh-111573. We'll see.)

…ythongh-111715) I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in pythongh-111530 (9322ce9). Since then I've had a nagging sense that I should have added the type and functions in its own PR. While I do plan on using _Py_excinfo outside crossinterp.c very soon (see pythongh-111572/pythongh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR. Hence, here we are. (FWIW, I may combine that with pythongh-111572, which I may, in turn, combine with pythongh-111573. We'll see.)

This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers). Specifically: * _Py_excinfo, etc.: the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c) * _PyXI_exception_info, etc.: helpers for passing an exception beween interpreters (wraps _Py_excinfo) * _PyXI_namespace, etc.: helpers for copying a dict of attrs between interpreters * _PyXI_Enter(), _PyXI_Exit(): functions that abstract out the transitions between one interpreter and a second that will do some work temporarily Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations. I plan on proposing these as public API at some point.

There were a few corner cases I didn't handle properly in pythongh-111530, which I've noticed while working on a follow-up PR. This fixes those cases.

…ythongh-111715) I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in pythongh-111530 (9322ce9). Since then I've had a nagging sense that I should have added the type and functions in its own PR. While I do plan on using _Py_excinfo outside crossinterp.c very soon (see pythongh-111572/pythongh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR. Hence, here we are. (FWIW, I may combine that with pythongh-111572, which I may, in turn, combine with pythongh-111573. We'll see.)

This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers). Specifically: * _Py_excinfo, etc.: the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c) * _PyXI_exception_info, etc.: helpers for passing an exception beween interpreters (wraps _Py_excinfo) * _PyXI_namespace, etc.: helpers for copying a dict of attrs between interpreters * _PyXI_Enter(), _PyXI_Exit(): functions that abstract out the transitions between one interpreter and a second that will do some work temporarily Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations. I plan on proposing these as public API at some point.

There were a few corner cases I didn't handle properly in pythongh-111530, which I've noticed while working on a follow-up PR. This fixes those cases.

…ythongh-111715) I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in pythongh-111530 (9322ce9). Since then I've had a nagging sense that I should have added the type and functions in its own PR. While I do plan on using _Py_excinfo outside crossinterp.c very soon (see pythongh-111572/pythongh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR. Hence, here we are. (FWIW, I may combine that with pythongh-111572, which I may, in turn, combine with pythongh-111573. We'll see.)

ericsnowcurrently added 6 commits October 30, 2023 16:54

Factor out _Py_excinfo.

67a88c2

Extract _PyXI_errcode.

a9ea9ac

Extract _PyXI_exception_info.

33d91af

Extract _PyXI_namespace.

2424e33

Factor out _enter_interpreter(), _exit_interpreter(), etc.

23d6959

Move enter/exit to crossinterp.c.

b08249f

ericsnowcurrently added the skip news label Oct 30, 2023

bedevere-app bot mentioned this pull request Oct 30, 2023

[subinterpreters] PEP 554 implementation: add interpreters module #76785

Open

bedevere-app bot added the awaiting core review label Oct 30, 2023

ericsnowcurrently added 17 commits October 30, 2023 17:32

Factor out _sharednsitem_set_value().

773f5ab

Add _PyXI_NamespaceFromNames().

cf7354e

Add a default arg to _PyXI_ApplyNamespace().

6b43620

Allocate xid dynamically when in target interpreter.

caef717

Add _PyXI_FillNamespaceFromDict().

0bd42e0

Add xid_state structs and lifecycle funcs.

a230f77

Add PyExc_NotShareableError.

5675f86

Propagate the ValueError when a value is not shareable.

45488f2

Propagate errors in _PyXI_Enter() directly.

6f07364

Factor out _init_not_shareable_error_type() and _fini_not_shareable_e…

0201b7f

…rror_type().

Drop some duplicate lines.

d32a918

Fix a comment.

1d4fc87

Call _PyXI_Fini() *before* the interpreter is cleared.

88c9d54

Fix init/fini.

2edcb49

Add _get_not_shareable_error_type().

8e53752

Export fewer symbols.

53764c1

Merge branch 'main' into crossinterp-utils-additions

cacf969

ericsnowcurrently merged commit 9322ce9 into python:main Nov 1, 2023
28 checks passed

bedevere-app bot removed the awaiting core review label Nov 1, 2023

ericsnowcurrently deleted the crossinterp-utils-additions branch November 1, 2023 23:36

ericsnowcurrently mentioned this pull request Nov 3, 2023

gh-76785: Minor Fixes in crossinterp.c #111671

Merged

ericsnowcurrently added a commit that referenced this pull request Nov 3, 2023

gh-76785: Minor Fixes in crossinterp.c (gh-111671)

93206d1

There were a few corner cases I didn't handle properly in gh-111530, which I've noticed while working on a follow-up PR. This fixes those cases.

ericsnowcurrently mentioned this pull request Nov 3, 2023

gh-76785: Move _Py_excinfo Functions Out of the Internal C-API #111715

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-76785: Crossinterp utils additions #111530

gh-76785: Crossinterp utils additions #111530

ericsnowcurrently commented Oct 30, 2023 •

edited

Loading

gh-76785: Crossinterp utils additions #111530

gh-76785: Crossinterp utils additions #111530

Conversation

ericsnowcurrently commented Oct 30, 2023 • edited Loading

ericsnowcurrently commented Oct 30, 2023 •

edited

Loading