Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 667: Clarify specification ambiguities and the expected impact #3845

Merged
merged 54 commits into from
Oct 21, 2024
Merged
Changes from 4 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
ab71e9a
Add impact on exec() for PEP 667
gaogaotiantian Jun 19, 2024
3d86025
Fix lint
gaogaotiantian Jun 19, 2024
8729874
Apply suggestions from code review
gaogaotiantian Jun 21, 2024
e2f0b94
Apply @carljm suggestions from review
willingc Jun 21, 2024
98edffa
Incorporate feedback and PyEval_GetLocals PR
ncoghlan Jun 25, 2024
9a2a7d3
Syntax fixes, PEP 558 comparison tweaks
ncoghlan Jun 25, 2024
e67a37d
Move f_locals spec note to spec section
ncoghlan Jun 25, 2024
a2b6b33
Add missing space
ncoghlan Jun 25, 2024
6be7b28
Indent ctypes example
ncoghlan Jun 25, 2024
f2423f0
Minor syntax and wording edits
ncoghlan Jun 25, 2024
82f94fb
Reword mention of "real" locals.
ncoghlan Jun 25, 2024
4a85e39
PEP link fix, minor text edits
ncoghlan Jun 25, 2024
6115fbd
Add note about the extra PEP 558 C APIs being omitted
ncoghlan Jun 25, 2024
669a0bc
Another link syntax fix
ncoghlan Jun 25, 2024
c68d02c
Merge branch 'main' into pep-667-exec-impact
ncoghlan Jun 25, 2024
9a966dd
Fix incorrect prefix in function name
ncoghlan Jun 25, 2024
98eddaf
Improve locals() implementation note
ncoghlan Jun 25, 2024
a8a700d
Clarify the note about deoptimization
ncoghlan Jun 25, 2024
7808759
Be clear there is no `is_function` frame method
ncoghlan Jun 25, 2024
6a89a65
More syntax and wording tweaks
ncoghlan Jun 25, 2024
22494e4
Further updates
ncoghlan Jun 25, 2024
828daa2
No joy on mixed syntax highlighting
ncoghlan Jun 25, 2024
1bc2eaa
Remove leftover comma
ncoghlan Jun 27, 2024
a924f4b
Reword note about better optimization support
ncoghlan Jun 28, 2024
c5d35e1
Merge branch 'main' into pep-667-exec-impact
ncoghlan Jul 4, 2024
32fc5fb
Address review comments from Barry & Tian
ncoghlan Jul 4, 2024
6449585
Syntax fix
ncoghlan Jul 4, 2024
a0881b1
Fix typo
ncoghlan Jul 4, 2024
8dec696
f_locals is mutable, not read-only!
ncoghlan Jul 4, 2024
4151acb
Misc proofreading edits
ncoghlan Jul 4, 2024
4871a39
Define exec/eval, fix proxy mutability
ncoghlan Jul 9, 2024
ad391ea
Fix syntax, update implementation note
ncoghlan Jul 9, 2024
2269f65
Another syntax fix
ncoghlan Jul 9, 2024
1539941
Make snapshot code consistent
ncoghlan Jul 9, 2024
0c5aea9
Use 3.13 signature for eval spec
ncoghlan Jul 16, 2024
a846667
Merge branch 'main' into pep-667-exec-impact
ncoghlan Jul 26, 2024
8abca04
Delay `PyEval_GetLocals` hard deprecation to 3.14
ncoghlan Jul 27, 2024
d3bff7b
Apply suggestions from code review
ncoghlan Jul 30, 2024
40e59cb
Merge branch 'main' into pep-667-exec-impact
ncoghlan Jul 30, 2024
1d4447d
Move locals rationale details to PEP 558
ncoghlan Aug 5, 2024
80e115b
Merge remote-tracking branch 'origin/main' into pep-667-exec-impact
ncoghlan Aug 5, 2024
c84fc0d
Merge remote-tracking branch 'origin/main' into pep-667-exec-impact
ncoghlan Aug 5, 2024
b7b427f
Merge branch 'main' into pep-667-exec-impact
ncoghlan Aug 24, 2024
544de0f
Merge branch 'main' into pep-667-exec-impact
ncoghlan Sep 4, 2024
ac5c974
Apply simple suggestions from code review
ncoghlan Oct 19, 2024
e21a718
Merge remote-tracking branch 'origin/main' into pep-667-exec-impact
ncoghlan Oct 19, 2024
709c345
Address initial set of recent review comments
ncoghlan Oct 19, 2024
db15bf0
Extra keys can be removed
ncoghlan Oct 19, 2024
ea92f36
Split 'Summary of Changes' out from 'Specification'
ncoghlan Oct 19, 2024
9d812fd
Avoid duplicate heading name
ncoghlan Oct 19, 2024
56727f0
Extra key removal is implementation dependent
ncoghlan Oct 19, 2024
a27332d
Add refs for del/pop regression
ncoghlan Oct 19, 2024
713ec6f
Fix typo
ncoghlan Oct 19, 2024
f875410
Merge branch 'main' into pep-667-exec-impact
ncoghlan Oct 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions peps/pep-0667.rst
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,70 @@ C API
As with all functions that return a borrowed reference, care must be taken to
ensure that the reference is not used beyond the lifetime of the object.

Impact on ``exec()``
====================

Even though this PEP does not modify ``exec()`` directly, the semantic change
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
to ``locals()`` impacts the behavior of ``exec()``.

``exec(object)`` has been equivalent to ``exec(object, globals(), locals())``
for a long time, but it's not officially documented. We plan to keep this
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
relation, which means existing code using ``exec()`` will see some different
behaviors.

The core difference is that the current implementation returns the same dict
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
when ``locals()`` is called multiple times in function scope, so the following
code magically works::

def f():
exec('a = 0') # equivalent to exec('a = 0', globals(), locals())
exec('print(a)') # equivalent to exec('print(a)', globals(), locals())
print(locals()) # {'a': 0}
# However, print(a) will not work here
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
f()

``locals()`` in a function currently returns the same persisted dict for every call. This allows stashing
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
extra "fake locals" in that dict, which aren't real locals known by the compiler, but can still be seen in
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
``locals()`` and shared between multiple ``exec()`` in the same function scope.

This behavior gets harder to predict when the code in ``exec()`` tries to write
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
to an existing local variable::
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved

def f():
a = None
exec('a = 0') # equivalent to exec('a = 0', globals(), locals())
exec('print(a)') # equivalent to exec('print(a)', globals(), locals())
print(locals()) # {'a': None}
f()

``print(a)`` will print ``None`` because the implicit ``locals()`` call in
``exec()`` refreshes the cached dict with the actual values on the frame.
So "real" locals, unlike "fake" locals created via ``exec()``, can't easily be
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
modified by ``exec()``.

With the semantic changes to ``locals()`` in this PEP, it's easier to explain the behavior
of ``exec()`` - it will never affect local function scopes. Any assignment
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved
to the local variables will be discarded when ``exec()`` returns.

However, you can still achieve a shared namespace across `exec()` calls with explicit namespaces::
ncoghlan marked this conversation as resolved.
Show resolved Hide resolved

def f():
scope = {}
exec('a = 0', locals=scope)
exec('print(a)', locals=scope)
f()

You can even change the variables in the local scope by explicitly using
``frame.f_locals``, which was not possible before::

def f():
a = None
exec('a = 0', locals=sys._getframe().f_locals)
print(a) # 0
f()

The behavior of ``exec()`` for module and class scopes is not changed.

Impact on PEP 709 inlined comprehensions
========================================

Expand Down
Loading