[C API] PEP 741: Add PyInitConfig C API to customize the Python initialization #107954

vstinner · 2023-08-14T22:57:37Z

Links to previous discussion of this feature:

https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004

Summary of proposal:

Add a text-based configuration to Python initialization

Pitch:

An API should be added to the limited C API customize the Python initialization (PyConfig API, PEP 587).

The problem is that the PyConfig C API is excluded from the limited C API, whereas the legacy initialization API is deprecated (ex: Py_VerboseFlag in Python 3.12) and being removed in Python 3.13 (ex: PySys_SetPath()).

Linked PRs

LostTime76 · 2023-08-15T02:26:10Z

As discussed in discourse, I do not think the text file is the best option for embedding. Depending on the embedding scenario, I think we will run into issues of finding the file due to how python is currently resolving paths. I have a very specific scenario that may be solved in easier ways. Even if you by chance get python to find the file in my particular scenario, I don't want the user of my embedded python exe to have to specify package paths that my exe should be able to set itself.

Its not as easy as putting the file in the same directory as the embedded exe or the main script. Take the scenario that the embedding exe is using the global python install or the embedding exe is launched as a child process from an external process where the process path is not the path of the embedded exe.

I think there may be much easier ways to solve my specific scenario. I would like to use as few APIs as possible and not duplicate how python is already parsing the options from the command line.

What about adding additional command line options that can be set from argv? If we could set additional package paths from argv, it would allow us to directly point to where we need to without hassle.

Environment variables do not work nicely in the embedded scenario, even though we have PYTHONPATH, because there is no way to cleanly set an environment variable from an already running process to be accessible from a loaded python dll.

vstinner · 2023-08-15T12:25:23Z

I'm thinking about a function taking a string, not a file.

Add a specification of the PyConfig structure to factorize the code.

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetString(config, key, value) * PyInitConfig_SetList(config, key, length, items) * PyInitConfig_GetErrorMsg(config) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) Changes: * Add Include/initconfig.h header.

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetStr(config, key, value) * PyInitConfig_SetWStr(config, key, value) * PyInitConfig_SetStrList(config, key, length, items) * PyInitConfig_SetWStrList(config, key, length, items) * PyInitConfig_GetErrorMsg(config) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) Changes: * Add Include/initconfig.h header.

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetStr(config, key, value) * PyInitConfig_SetWStr(config, key, value) * PyInitConfig_SetStrList(config, key, length, items) * PyInitConfig_SetWStrList(config, key, length, items) * PyInitConfig_GetErrorMsg(config) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) Add these 11 functions to the limited C API. Changes: * Add Include/initconfig.h header.

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetStr(config, key, value) * PyInitConfig_SetWStr(config, key, value) * PyInitConfig_SetStrList(config, key, length, items) * PyInitConfig_SetWStrList(config, key, length, items) * PyInitConfig_GetErrorMsg(config) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) * Add functions to get runtime config: * PyConfig_GetInt() * PyConfig_GetStr() * PyConfig_GetStrList() Add these functions to the limited C API. Changes: * Add Doc/c-api/config.rst. * Add Include/initconfig.h header. * Add Modules/_testcapi/config.c * Add Lib/test/test_capi/test_config.py.

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetStr(config, key, value) * PyInitConfig_SetWStr(config, key, value) * PyInitConfig_SetStrList(config, key, length, items) * PyInitConfig_SetWStrList(config, key, length, items) * PyInitConfig_Exception(config) * PyInitConfig_GetError(config, &err_msg) * PyInitConfig_GetExitCode(config, &exitcode) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) * Add functions to get runtime config: * PyConfig_GetInt() * PyConfig_GetStr() * PyConfig_GetStrList() Add these functions to the limited C API. Changes: * Add Doc/c-api/config.rst. * Add Include/initconfig.h header. * Add Modules/_testcapi/config.c * Add Lib/test/test_capi/test_config.py.

Add PyConfig_Get() and PyConfig_GetInt() functions to get the current Python configuration.

Add PyConfig_Get() and PyConfig_GetInt() functions to get the current Python configuration. _PyConfig_AsDict() now converts PyConfig.xoptions as a dictionary.

_PyConfig_AsDict() now returns bool objects for options using the new PyConfig_MEMBER_BOOL type. Update tests for these changes.

vstinner · 2024-03-07T17:34:59Z

See also #99872 "Allow efficient read access to PyConfig options".

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetStr(config, key, value) * PyInitConfig_SetWStr(config, key, value) * PyInitConfig_SetStrList(config, key, length, items) * PyInitConfig_SetWStrList(config, key, length, items) * PyInitConfig_Exception(config) * PyInitConfig_GetError(config, &err_msg) * PyInitConfig_GetExitCode(config, &exitcode) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) Add these functions to the limited C API. Changes: * Add Doc/c-api/config.rst. * Add Include/initconfig.h header.

…n#116359) _PyConfig_AsDict() now returns bool objects for options using the new PyConfig_MEMBER_BOOL type. Update tests for these changes.

Add PyInitConfig functions: * PyInitConfig_Python_New() * PyInitConfig_Isolated_New() * PyInitConfig_Free(config) * PyInitConfig_SetInt(config, key, value) * PyInitConfig_SetStr(config, key, value) * PyInitConfig_SetWStr(config, key, value) * PyInitConfig_SetStrList(config, key, length, items) * PyInitConfig_SetWStrList(config, key, length, items) * PyInitConfig_Exception(config) * PyInitConfig_GetError(config, &err_msg) * PyInitConfig_GetExitCode(config, &exitcode) Add also functions using it: * Py_InitializeFromInitConfig(config) * Py_ExitWithInitConfig(config) Add these functions to the limited C API. Changes: * Add Doc/c-api/config.rst. * Add Include/initconfig.h header.

Add PyConfig_Get(), PyConfig_GetInt(), PyConfig_Set() and PyConfig_Names() functions to get and set the current runtime Python configuration. _PyConfig_AsDict() now converts PyConfig.xoptions as a dictionary.

Add Doc/c-api/config.rst documentation.

Add a specification of the PyConfig structure to factorize the code.

Add PyConfig_Get(), PyConfig_GetInt(), PyConfig_Set() and PyConfig_Names() functions to get and set the current runtime Python configuration. Add visibility and "sys spec" to config and preconfig specifications. _PyConfig_AsDict() now converts PyConfig.xoptions as a dictionary. Co-authored-by: Bénédikt Tran <[email protected]>

Add Doc/c-api/config.rst documentation.

Setting dev_mode to 1 in an isolated configuration now enables also faulthandler. Moreover, setting "module_search_paths" option with PyInitConfig_SetStrList() now sets "module_search_paths_set" to 1.

vstinner · 2024-09-06T14:32:24Z

PEP 741 is now fully implemented, I close the issue.

vstinner added type-feature A feature request or enhancement topic-C-API labels Aug 14, 2023

rokm mentioned this issue Aug 16, 2023

Configure and initialize python via PEP 587 API pyinstaller/pyinstaller#7847

Merged

vstinner mentioned this issue Sep 4, 2023

Issues with the Stable ABI capi-workgroup/problems#4

Open

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Add _PyConfig_Parse()

fa98a1c

bedevere-app bot mentioned this issue Sep 30, 2023

[WIP] gh-107954: Add _PyConfig_Parse() #110145

Closed

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Refactor initconfig.c, add CONFIG_SPEC

01fcc4c

Add a specification of the PyConfig structure to factorize the code.

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Refactor initconfig.c: add CONFIG_SPEC

92d80d6

Add a specification of the PyConfig structure to factorize the code.

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Refactor initconfig.c: add CONFIG_SPEC

c09b9a4

Add a specification of the PyConfig structure to factorize the code.

bedevere-app bot mentioned this issue Sep 30, 2023

gh-107954: Refactor initconfig.c: add CONFIG_SPEC #110146

Merged

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Add _PyConfig_Parse()

96cb693

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Refactor initconfig.c: add CONFIG_SPEC

7191629

Add a specification of the PyConfig structure to factorize the code.

vstinner added a commit that referenced this issue Sep 30, 2023

gh-107954: Refactor initconfig.c: add CONFIG_SPEC (#110146)

f3bb00e

Add a specification of the PyConfig structure to factorize the code.

vstinner added a commit to vstinner/cpython that referenced this issue Sep 30, 2023

pythongh-107954: Add _PyConfig_Parse()

e6e4e5c

bedevere-app bot mentioned this issue Oct 1, 2023

[PEP 741] gh-107954: Add PyInitConfig C API #110176

Closed

vstinner added a commit to vstinner/cpython that referenced this issue Dec 1, 2023

pythongh-107954: Add PyConfig_Get() function

fba1c0f

vstinner added a commit to vstinner/cpython that referenced this issue Dec 1, 2023

pythongh-107954: Add PyConfig_Get() function

30af159

vstinner added a commit to vstinner/cpython that referenced this issue Dec 1, 2023

pythongh-107954: Add PyConfig_Get() function

3bc322f

Add PyConfig_Get() and PyConfig_GetInt() functions to get the current Python configuration.

bedevere-app bot mentioned this issue Mar 5, 2024

gh-107954: Add PyConfig_MEMBER_BOOL type to PyConfigSpec #116359

Merged

vstinner added a commit that referenced this issue Mar 6, 2024

gh-107954: Add PyConfig_MEMBER_BOOL type to PyConfigSpec (#116359)

2b37996

_PyConfig_AsDict() now returns bool objects for options using the new PyConfig_MEMBER_BOOL type. Update tests for these changes.

bedevere-app bot mentioned this issue Aug 29, 2024

gh-107954, PEP 741: Add PyConfig_Get() function #123472

Merged

vstinner added a commit to vstinner/cpython that referenced this issue Aug 30, 2024

pythongh-107954, PEP 741: Add PyInitConfig C API

0eb3fd0

Add Doc/c-api/config.rst documentation.

vstinner changed the title ~~[C API] PEP 741: Add stable ABI to customize Python initialization~~ [C API] PEP 741: Add PyInitConfig C API to customize the Python initialization Aug 30, 2024

bedevere-app bot mentioned this issue Aug 30, 2024

gh-107954, PEP 741: Add PyInitConfig C API #123502

Merged

vstinner added a commit to vstinner/cpython that referenced this issue Aug 30, 2024

pythongh-107954, PEP 741: Add PyInitConfig C API

c745977

Add Doc/c-api/config.rst documentation.

Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024

pythongh-107954: Refactor initconfig.c: add CONFIG_SPEC (python#110146)

5afbfdc

Add a specification of the PyConfig structure to factorize the code.

vstinner added a commit that referenced this issue Sep 3, 2024

gh-107954, PEP 741: Add PyInitConfig C API (#123502)

ef9d547

Add Doc/c-api/config.rst documentation.

bedevere-app bot mentioned this issue Sep 4, 2024

gh-107954, PEP 741: Adjust Python initialization config #123663

Merged

vstinner added a commit to vstinner/cpython that referenced this issue Sep 4, 2024

pythongh-107954, PEP 741: Add PyInitConfig_AddModule() function

3a62386

vstinner added a commit to vstinner/cpython that referenced this issue Sep 4, 2024

pythongh-107954, PEP 741: Add PyInitConfig_AddModule() function

1ca74e2

vstinner added a commit to vstinner/cpython that referenced this issue Sep 4, 2024

pythongh-107954, PEP 741: Add PyInitConfig_AddModule() function

5241126

bedevere-app bot mentioned this issue Sep 4, 2024

gh-107954, PEP 741: Add PyInitConfig_AddModule() function #123668

Merged

vstinner added a commit that referenced this issue Sep 4, 2024

gh-107954, PEP 741: Add PyInitConfig_AddModule() function (#123668)

0d6b6e3

vstinner closed this as completed Sep 6, 2024

pablogsal added a commit to pablogsal/cpython that referenced this issue Sep 26, 2024

pythongh-107954: Fix configuration type for the perf profiler

3fb0be9

bedevere-app bot mentioned this issue Sep 26, 2024

gh-107954: Fix configuration type for the perf profiler #124636

Merged

pablogsal added a commit to pablogsal/cpython that referenced this issue Sep 26, 2024

pythongh-107954: Fix configuration type for the perf profiler

6092199

ambv pushed a commit that referenced this issue Sep 27, 2024

gh-107954: Fix configuration type for the perf profiler (#124636)

02b49c5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[C API] PEP 741: Add PyInitConfig C API to customize the Python initialization #107954

[C API] PEP 741: Add PyInitConfig C API to customize the Python initialization #107954

vstinner commented Aug 14, 2023 •

edited by bedevere-app bot

Loading

LostTime76 commented Aug 15, 2023 •

edited

Loading

vstinner commented Aug 15, 2023

vstinner commented Mar 7, 2024

vstinner commented Sep 6, 2024

[C API] PEP 741: Add PyInitConfig C API to customize the Python initialization #107954

[C API] PEP 741: Add PyInitConfig C API to customize the Python initialization #107954

Comments

vstinner commented Aug 14, 2023 • edited by bedevere-app bot Loading

Links to previous discussion of this feature:

Summary of proposal:

Pitch:

Linked PRs

LostTime76 commented Aug 15, 2023 • edited Loading

vstinner commented Aug 15, 2023

vstinner commented Mar 7, 2024

vstinner commented Sep 6, 2024

vstinner commented Aug 14, 2023 •

edited by bedevere-app bot

Loading

LostTime76 commented Aug 15, 2023 •

edited

Loading