…thonGH-120994)

…h-121041)

Remove fallthrough warnings

…le (python#121017)

…on#120643)

* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()

…tails (python#121009)

When testing IDLE, don't create a Tk to avoid side effects such as
installing a PyOS_InputHook hook.

…n#121053)

Remove the const qualifier of the argument of functions:

* _PyLong_IsCompact()
* _PyLong_CompactValue()

Py_TYPE() argument is not const.

Fix the compiler warning:

  Include/cpython/longintrepr.h: In function ‘_PyLong_CompactValue’:
  Include/pyport.h:19:31: error: cast discards ‘const’ qualifier from
  pointer target type [-Werror=cast-qual]
    (...)
  Include/cpython/longintrepr.h:133:30: note: in expansion of macro
  ‘Py_TYPE’
    assert(PyType_HasFeature(Py_TYPE(op), Py_TPFLAGS_LONG_SUBCLASS));

…n#121054)

…18450)

This PR sets up tagged pointers for CPython.

The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly.

Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out.

This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it.

The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs.

Please read Include/internal/pycore_stackref.h for more information!

---------

Co-authored-by: Mark Shannon <[email protected]>

…ython#120940)

Co-authored-by: Hugo van Kemenade <[email protected]>

…e() (pythongh-121060)

This change makes things a little less painful for some users.  It also fixes a failing assert (pythongh-120765), by making sure all subinterpreters are destroyed before the main interpreter.  As part of that, we make sure Py_Finalize() always runs with the main interpreter active.

…QueueHandler` (pythonGH-120872)

…-120993)

Co-authored-by: Pradyun Gedam <[email protected]>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: Alyssa Coghlan <[email protected]>
Co-authored-by: T. Wouters <[email protected]>

Fix warnings when using -Wimplicit-fallthrough compiler flag.

Annotate explicitly "fall through" switch cases with a new
_Py_FALLTHROUGH macro which uses __attribute__((fallthrough)) if
available. Replace "fall through" comments with _Py_FALLTHROUGH.

Add _Py__has_attribute() macro. No longer define __has_attribute()
macro if it's not defined. Move also _Py__has_builtin() at the top
of pyport.h.

Co-Authored-By: Nikita Sobolev <[email protected]>

Check the usage of the 'const' qualifier in the Python C API in
test_cext.

…ystats` arguments in configuration command after python#118450 (python#121083)



Signed-off-by: Manjusaka <[email protected]>
Co-authored-by: Ken Jin <[email protected]>

…ython#121086)

* Move pprinter parameters description to the table

The change improves readability.
Suggested in the GH#116085 PR discussion.

* Make pprint doc with params markup

* Fix formatting
Indentation of code blocks made them nested
"Version changed" is better placed after the code block

* Fix formatting for tests

* fix code indentation for autotests

* Fix identation for autotests

* Remove duplication of the parameters' description

* Rearrange parameters description in a correct order

---------

Co-authored-by: Hugo van Kemenade <[email protected]>

…ndler (pythonGH-105887)

The check for whether the log file is a real file is expensive on NFS
filesystems.  This commit reorders the rollover condition checking to
not do the file type check if the expected file size is less than the
rotation threshold.

Co-authored-by: Oleg Iarygin <[email protected]>

…ython#121100)

…handler. (pythonGH-121036)

…afe (pythongh-121051)

…tation (python#120995)

…ython#121097)

…rs implementation in asyncio (python#121124)

On Windows, test_cext and test_cppext now pass /WX flag to the MSC
compiler to treat all compiler warnings as errors. In verbose mode,
these tests now log the compiler commands to help debugging.

Change Py_BUILD_ASSERT_EXPR implementation on Windows to avoid a
compiler warning about an unnamed structure.

…rror=False (pythonGH-121056)

* parse_intermixed_args() now raises ArgumentError instead of calling
  error() if exit_on_error is false.
* Internal code now always raises ArgumentError instead of calling
  error(). It is then caught at the higher level and error() is called if
  exit_on_error is true.

…ythonGH-121118)

Fix a few wrong steals in bytecodes.c

…pickle.c (python#121136)

PyObject_GetAttr returns a new reference, but this reference is never decremented using Py_DECREF, so Py_DECREF calls to this referece are added

…ythonGH-120820)

…n#121102)

In some cases, previously computed as (nan+nanj), we could
recover meaningful component values in the result, see
e.g. the C11, Annex G.5.2, routine _Cdivd().

…ib docs. (python#120970)

Add dedicated subsection for `home()`, `expanduser()`, `cwd()`,
`absolute()`, `resolve()` and `readlink()`. The position of this section
keeps all the `Path` constructors (`Path()`, `Path.from_uri()`,
`Path.home()` and `Path.cwd()`) near the top. Within the section, closely
related methods are kept adjacent. Specifically:

-.`home()` and `expanduser()` (the former calls the latter)
- `cwd()` and `absolute()` (the former calls the latter)
- `absolute()` and `resolve()` (both make paths absolute)
- `resolve()` and `readlink()` (both read symlink targets)
- Ditto `cwd()` and `absolute()`
- Ditto `absolute()` and `resolve()`

The "Other methods" section is removed.

…121158)

Co-authored-by: Hugo van Kemenade <[email protected]>

… to patch out problematic code (python#120984)

* Add --app-store-compliance configuration option.

* Added blurb.

* Correct tab-vs-spaces formatting issue.

* Correct source file name in docs.

Co-authored-by: Nice Zombies <[email protected]>

* Correct source code reference in Mac docs

Co-authored-by: Nice Zombies <[email protected]>

* Only apply the patch forward, and ensure the working directory is correct.

* Make patching reslient to multiple builds.

* Documentation fixes found during review

Co-authored-by: Alyssa Coghlan <[email protected]>

* Documentation and configure.ac syntax improvements

Co-authored-by: Erlend E. Aasland <[email protected]>

* Regenerate configure script.

* Silence the patch echo output.

---------

Co-authored-by: Nice Zombies <[email protected]>
Co-authored-by: Alyssa Coghlan <[email protected]>
Co-authored-by: Erlend E. Aasland <[email protected]>

)

…ythonGH-121182)

Update element positions and styles.

….simplefilter() (python#121164)

Add support for ``all`` as an valid alias for ``always`` in ``warnings.simplefilter()``
and ``warnings.filterswarnings()``.

…python#121126)

asyncio earlier relied on subprocess module to send signals to the process, this has some drawbacks one being that subprocess module unnecessarily calls waitpid on child processes and hence it races with asyncio implementation which internally uses child watchers. To mitigate this, now asyncio sends signals directly to the process without going through the subprocess on non windows systems. On Windows it fallbacks to subprocess module handling but on windows there are no child watchers so this issue doesn't exists altogether.

…ython#119816)

1. Use pkg-config to check for ncursesw/panelw. If that fails, use 
   pkg-config to check for ncurses/panel.
2. Regardless of pkg-config output, search for curses/panel headers, so
   we're sure we have all defines in pyconfig.h.
3. Regardless of pkg-config output, check if libncurses or libncursesw
   contains the 'initscr' symbol; if it does _and_ pkg-config failed
   earlier, add the resulting -llib linker option to CURSES_LIBS.
   Ditto for 'update_panels' and PANEL_LIBS.
4. Wrap the rest of the checks with WITH_SAVE_ENV and make sure we're 
   using updated LIBS and CPPFLAGS for those.

Add the PY_CHECK_CURSES convenience macro.

…121195)

When creating the JUnit XML file, regrtest now escapes characters
which are invalid in XML, such as the chr(27) control character used
in ANSI escape sequences.

)

Use the _Py__has_attribute() macro in timemodule.c and
bootstrap_hash.c to fix a build error on old GCC versions (GCC 4.8.5
on s390x).

…hon#120894)

Example:

```pycon
>>> import ctypes
>>> ctypes.__STDC_IEC_559_COMPLEX__
1
>>> libm = ctypes.CDLL('libm.so.6')
>>> libm.clog.argtypes = [ctypes.c_double_complex]
>>> libm.clog.restype = ctypes.c_double_complex
>>> libm.clog(1+1j)
(0.34657359027997264+0.7853981633974483j)
```

Co-authored-by: Nice Zombies <[email protected]>
Co-authored-by: Bénédikt Tran <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>

…ython#121191)

dash_R_cleanup() now calls _abc_registry_clear() before calling again
register().

Use subTest() to log the pwd entry in test_expanduser_pwd2() of
test_posixpath to help debugging.

…621 in /Tools (python#121215)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…1216)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

….20240627 in /Tools (python#121217)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…HANDSHAKE (pythonGH-117785)

With this change, builds with OpenSSL forks that don't have this functionalty
(like AWS-LC or BoringSSL) will require less patching.

…vy (python#121221)

Mark test_threaded_weak_value_dict_copy() and
test_threaded_weak_key_dict_copy() of test_weakref as
CPU-heavy tests

…read safe (pythonGH-120490)

…#121228)

Call getpwnam() to get pw_dir, since it can be different than
getpwall() pw_dir.

Soft deprecate os.popen() and os.spawn*() functions.

…time behavior (python#121175)

…set_each (pythongh-121236)

This will allow Py_TRACE_REFS builds to pass the test suite, until the underlying issue can be resolved.

…ython#120914)

Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed
atomic loads in the free-threaded build.

After this change, the TSAN doesn't report data races for this method.

…121197)

…ythonGH-121254)

* Move get_signal_name() from test.libregrtest to test.support.
* Use get_signal_name() in support.script_helper.
* support.script_helper now decodes stdout and stderr from UTF-8,
  instead of ASCII, if a command failed.

Co-authored-by: Miro Hrončok <[email protected]>

…compare` (python#121211)

…while(0) (python#121166)

…ymtable (python#121273)

…1244)

Avoids the extra conversion from stack refs to PyObjects.

… dark/light modes. (pythonGH-121265)

…ython#120807)

Check for `ERROR_INVALID_PARAMETER` when calling `_winapi.CopyFile2()` and
raise `UnsupportedOperation`. In `Path.copy()`, handle this exception and
fall back to the `PathBase.copy()` implementation.

pythonGH-121089)

Co-authored-by: d.grigonis <[email protected]>

…E in rotatingtree.c (python#121260)

…r.ForkingMixIn` (python#118134)

…n#121218)

Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.100.2 to 6.104.2.
- [Release notes](https://github.com/HypothesisWorks/hypothesis/releases)
- [Commits](HypothesisWorks/hypothesis@hypothesis-python-6.100.2...hypothesis-python-6.104.2)

---
updated-dependencies:
- dependency-name: hypothesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

)

Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Blaise Pabon <[email protected]>

Add more offsets to _Py_DebugOffsets

We add a few more offsets that are required by some out-of-process
tools, such as [Austin](https://github.com/p403n1x87/austin).

… module (python#121248)

This amends 6988ff0: memory allocation for
stginfo->ffi_type_pointer.elements in PyCSimpleType_init() should be
more generic (perhaps someday fmt->pffi_type->elements will be not a
two-elements array).

It should finally resolve python#61103.

Co-authored-by: Victor Stinner <[email protected]>
Co-authored-by: Bénédikt Tran <[email protected]>

…on#121297)

compiler no longer modifies the symtable after this.

…-121270)

Macro-ify most stackref functions for MSVC

…on#121259)

…-121323)

…21328)

Disable perf_trampoline on riscv64 for now

Until support is added in perf_jit_trampoline.c

pythongh-120089 was incomplete.

Restore the private _PyArg_Parser structure and the private
_PyArg_ParseTupleAndKeywordsFast() function, previously removed
in Python 3.13 alpha 1.

Recreate Include/cpython/modsupport.h header file.

…reasonable (python#118725)

…ython#120233)

As noted in pythongh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.

An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.

For reference, here are benchmarks for the current main branch:

```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       9.7 ms ±   0.7 ms    [User: 7.7 ms, System: 1.8 ms]
  Range (min … max):     8.4 ms …  13.1 ms    313 runs
```

And with this patch:

```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       8.4 ms ±   0.7 ms    [User: 6.8 ms, System: 1.4 ms]
  Range (min … max):     7.2 ms …  11.7 ms    352 runs
```

Compare to:

```
❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
  Time (mean ± σ):       7.6 ms ±   0.6 ms    [User: 5.9 ms, System: 1.6 ms]
  Range (min … max):     6.7 ms …  11.3 ms    390 runs
```

This roughly halves the import time of importlib.util.

…121162)

…se (python#120755)

This reduces the system call count of a simple program[0] that reads all
the `.rst` files in Doc by over 10% (5706 -> 4734 system calls on my
linux system, 5813 -> 4875 on my macOS)

This reduces the number of `fstat()` calls always and seek calls most
the time. Stat was always called twice, once at open (to error early on
directories), and a second time to get the size of the file to be able
to read the whole file in one read. Now the size is cached with the
first call.

The code keeps an optimization that if the user had previously read a
lot of data, the current position is subtracted from the number of bytes
to read. That is somewhat expensive so only do it on larger files,
otherwise just try and read the extra bytes and resize the PyBytes as
needeed.

I built a little test program to validate the behavior + assumptions
around relative costs and then ran it under `strace` to get a log of the
system calls. Full samples below[1].

After the changes, this is everything in one `filename.read_text()`:

```python3
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3`
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0`
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

This does make some tradeoffs
1. If the file size changes between open() and readall(), this will
still get all the data but might have more read calls.
2. I experimented with avoiding the stat + cached result for small files
in general, but on my dev workstation at least that tended to reduce
performance compared to using the fstat().

[0]

```python3
from pathlib import Path

nlines = []
for filename in Path("cpython/Doc").glob("**/*.rst"):
    nlines.append(len(filename.read_text()))
```

[1]
Before small file:

```
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
ioctl(3, TCGETS, 0x7ffe52525930)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

After small file:

```
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

Before large file:

```
openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
ioctl(3, TCGETS, 0x7ffe52525930)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104
read(3, "", 1)                          = 0
close(3)                                = 0
```

After large file:

```
openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104
read(3, "", 1)                          = 0
close(3)                                = 0
```

Co-authored-by: Shantanu <[email protected]>
Co-authored-by: Erlend E. Aasland <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>