New python312dev.yml (to replace upstream.yml) (#30006)

* Replace upstream.yml with new python312.yml

* Revert "Replace upstream.yml with new python312.yml"

This reverts commit 9a6b9b4e6fee7a4b9d88f09f666f7cba9773265a.

* Bring back python312.yml as upstream.yml

* Move upstream.yml back to python312.yml

* Move python312.yml to python312dev.yml

* Change label python312dev to gha_wf_python312dev_enable; introduce label gha_wf_ci_disable.

* FlakyException failure_point_init debugging prints

* Revert "FlakyException failure_point_init debugging prints"

This reverts commit d38c9da.

* Skip failing test specifically for 3.12alpha6

Root problem:

* Up to and including 3.12alpha3: `PyErr_Fetch()` produces the `FlakyException` type.

* With 3.12alpha6: `PyErr_Fetch()` produces the `ValueError` type instead.

Additional detail:

Up to and including 3.12alpha3: only `PyErr_NormalizeException()` hits the `ValueError` in `FlakyException._init__`, but not `PyErr_Fetch()`.

* Remove test code that does not exercise anything in pybind11, but breaks between Python 3.12alpha3 (still working) and 3.12alpha6 (broken):

```
str(OrderedDict([(1, "a"), (2, "b")]))
```

Old:
```
OrderedDict([(1, 'a'), (2, 'b')])
```

New:
```
OrderedDict({1: 'a', 2: 'b'})
```

* Document that test_embed is still failing (tested only interactively with locally install 3.12alpha6, but the behavior is exactly the same as previously observed in the GH workflow).

* Try again: gha_wf_ci_disable

* Give up on trying to use label to disable entire workflow file.

.github/workflows/python312dev.yml

@@ -0,0 +1,143 @@
+name: python312dev
+
+on:
+  workflow_dispatch:
+  pull_request:
+
+concurrency:
+  group: python312dev-${{ github.ref }}
+  cancel-in-progress: false
+
+env:
+  PYTEST_TIMEOUT: 300
+
+jobs:
+  standard:
+    name: "🐍 3.12-dev • ubuntu-latest • x64"
+    runs-on: ubuntu-latest
+
+    # Hint: Select the label in the PR web view.
+    if: "contains(github.event.pull_request.labels.*.name, 'gha_wf_python312dev_enable')"
+
+    steps:
+    - name: Show env
+      run: env
+
+    - uses: actions/checkout@v3
+
+    - name: Setup Python 3.12-dev
+      uses: actions/setup-python@v4
+      with:
+        python-version: "3.12-dev"
+
+    - name: Setup Boost
+      run: sudo apt-get install libboost-dev
+
+    - name: Update CMake
+      uses: jwlawson/[email protected]
+
+    - name: Run pip installs
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install --prefer-binary -r tests/requirements.txt
+    #   python -m pip install --prefer-binary numpy  # SLOW
+    #   python -m pip install --prefer-binary scipy  # FAILED ~Nov 2022
+
+    - name: Show platform info
+      run: python -m platform
+
+    - name: Show CMake version
+      run: cmake --version
+
+    # FIRST BUILD
+    - name: Configure C++11
+      run: >
+        cmake -S . -B build11
+        -DCMAKE_VERBOSE_MAKEFILE=ON
+        -DPYBIND11_WERROR=ON
+        -DDOWNLOAD_CATCH=ON
+        -DDOWNLOAD_EIGEN=ON
+        -DCMAKE_CXX_STANDARD=11
+        -DCMAKE_BUILD_TYPE=Debug
+
+    - name: Build C++11
+      run: cmake --build build11 -j 2
+
+    - name: Python tests C++11
+      run: cmake --build build11 --target pytest -j 2
+
+    # python-3.12.0-alpha.1-linux-22.04-x64.tar.gz  PASSED
+    # python-3.12.0-alpha.2-linux-22.04-x64.tar.gz  FAILED
+    # python-3.12.0-alpha.3-linux-22.04-x64.tar.gz  FAILED
+    # python-3.12.0-alpha.6-linux-22.04-x64.tar.gz  FAILED
+    #     free(): invalid pointer
+    #     Custom PyConfig
+    #     /home/runner/work/pybind11/pybind11/tests/test_embed/test_interpreter.cpp:175
+    #     /home/runner/work/pybind11/pybind11/tests/test_embed/test_interpreter.cpp:179: FAILED:
+    # gdb traceback points here:
+    #     https://github.com/python/cpython/blame/54289f85b2af1ecf046089ddf535dda1bdf6af24/Python/import.c#L524
+    # - name: C++ tests C++11
+    #   run: cmake --build build11 --target cpptest -j 2
+
+    - name: Interface test C++11
+      run: cmake --build build11 --target test_cmake_build
+
+    - name: Clean directory
+      run: git clean -fdx
+
+    # SECOND BUILD
+    - name: Configure C++17
+      run: >
+        cmake -S . -B build17
+        -DCMAKE_VERBOSE_MAKEFILE=ON
+        -DPYBIND11_WERROR=ON
+        -DDOWNLOAD_CATCH=ON
+        -DDOWNLOAD_EIGEN=ON
+        -DCMAKE_CXX_STANDARD=17
+        -DCMAKE_BUILD_TYPE=Debug
+
+    - name: Build C++17
+      run: cmake --build build17 -j 2
+
+    - name: Python tests C++17
+      run: cmake --build build17 --target pytest
+
+    # - name: C++ tests C++17
+    #   run: cmake --build build17 --target cpptest
+
+    - name: Interface test C++17
+      run: cmake --build build17 --target test_cmake_build
+
+    - name: Clean directory
+      run: git clean -fdx
+
+    # THIRD BUILD
+    - name: Configure C++17 max DPYBIND11_INTERNALS_VERSION
+      run: >
+        cmake -S . -B build17max
+        -DCMAKE_VERBOSE_MAKEFILE=ON
+        -DPYBIND11_WERROR=ON
+        -DDOWNLOAD_CATCH=ON
+        -DDOWNLOAD_EIGEN=ON
+        -DCMAKE_CXX_STANDARD=17
+        -DCMAKE_BUILD_TYPE=Debug
+        -DPYBIND11_INTERNALS_VERSION=10000000
+
+    - name: Build C++17 max DPYBIND11_INTERNALS_VERSION
+      run: cmake --build build17max -j 2
+
+    - name: Python tests C++17 max DPYBIND11_INTERNALS_VERSION
+      run: cmake --build build17max --target pytest
+
+    # - name: C++ tests C++17 max DPYBIND11_INTERNALS_VERSION
+    #   run: cmake --build build17max --target cpptest
+
+    - name: Interface test C++17 max DPYBIND11_INTERNALS_VERSION
+      run: cmake --build build17max --target test_cmake_build
+
+    # Ensure the setup_helpers module can build packages using setuptools
+    - name: Setuptools helpers test
+      run: pytest tests/extra_setuptools
+
+    - name: Clean directory
+      run: git clean -fdx

tests/test_exceptions.py

@@ -317,6 +317,11 @@ def test_error_already_set_what_with_happy_exceptions(
     assert what == expected_what
 
 
+@pytest.mark.skipif(
+    # Intentionally very specific:
+    "sys.version_info == (3, 12, 0, 'alpha', 6)",
+    reason="Suspected bug in 3.12alpha6",
+)
 @pytest.mark.skipif("env.PYPY", reason="PyErr_NormalizeException Segmentation fault")
 def test_flaky_exception_failure_point_init():
     with pytest.raises(RuntimeError) as excinfo:

tests/test_modules.py

@@ -63,7 +63,6 @@ def test_importing():
     from pybind11_tests.modules import OD
 
     assert OD is OrderedDict
-    assert str(OD([(1, "a"), (2, "b")])) == "OrderedDict([(1, 'a'), (2, 'b')])"
 
 
 def test_pydoc():