setup.py breaks when run through new pip due to somehow missing Cython, works outside of pip and worked in old pip

[ell1e]

Description

Running this setup.py of mine, which unchanged used to work some weeks ago before a python and pip upgrade, and still works when directly running python3 setup.py install --user, breaks when invoked through a pip install of the package:

$ pip3 install --user -U .
Processing /home/user/Develop/myprj
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      Traceback (most recent call last):
        File "/usr/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-imi4zebj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-imi4zebj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-imi4zebj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 488, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-imi4zebj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 254, in <module>
        File "<string>", line 66, in extensions
      ModuleNotFoundError: No module named 'Cython'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behavior

Installing the package works fine as it does outside of pip and does with older Python + pip.

pip version

pip-23.1.2, pip-23.2.1

Python version

3.11.4

OS

postmarketOS 23.06, used to work with pip + Python from postmarketOS 22.12

How to Reproduce

1. Currently the package isn't public, but you just need to try to install it to break it: cd /home/user/Develop/myprj && pip3 install --user -U .

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

See #8368 and #8559. The latter link also contains possible migration paths away from this.

[ell1e]

My apologies for maybe being a bit dense, but the tickets only seems to contain workarounds for the user but doesn't seem to have any info on how the package should be changed. The problem seems to be that setup.py imports Cython (intentionally) and apparently that's no longer possible now, but that's basically already the whole extent of my knowledge (or lack thereof) so far on how to actually fix it.

[notatallshaw]

The problem seems to be that setup.py imports Cython (intentionally) and apparently that's no longer possible now, but that's basically already the whole extent of my knowledge (or lack thereof) so far on how to actually fix it.

You need to add build dependencies in a pyproject.toml, e.g.

[build-system]
build-backend = 'setuptools.build_meta'
requires = [
    'setuptools',
    'cython'
]

[ell1e]

That's not practical though, it depends on another package's pxd headers and that other package takes half an hour to build. It's not really feasible to just randomly switch over to an isolated virtualenv if that's what's going on. How would I tell pip not to do that from the package's side?

[notatallshaw]

that other package takes half an hour to build

Then use --no-build-isolation