[FR] Disallow "requirements.txt" as a file: input for install_requires/dependencies

[pradyunsg]

What's the problem this feature will solve?

This will avoid encouraging users to follow the mixing/bad-practice of reading a requirements.txt file into their setup.py file; to "deal with the duplication". We've known about this anti-pattern for over a decade and we shouldn't repeat the same mistake after spending a non-trivial amount of energy to get to a cleaner dependency specification model.

https://caremad.io/posts/2013/07/setup-vs-requirement/

Describe the solution you'd like

Disallow using *.txt filenames as a file: source for the dependencies when using pyproject.toml-based metadata, with the error pointing users to documentation about why this is a bad practice.

Alternative Solutions

Not doing anything, and continuing to make the situation with the misunderstanding around "duplicated information" spread more.

Additional context

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct

[pradyunsg]

https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html explicitly documents this as the pattern it is meant to enable:

dependencies | file | subset of the requirements.txt format (# comments and blank lines excluded) BETA

Even though it does document that "hey, don't use this footgun to shoot yourself in the foot", I'd argue we shouldn't be giving users a footgun in the first place. (as demonstrated in #3511 and friends)

[bunny-therapist]

Currently, we have lots of projects which are apps, so the dependencies should be pinned. We use pip-compile to automatically update a requirements.txt file, and we link that file from pyproject.toml. (We also have other projects which are libraries, so we do not pin requirements and we can just list the required dependencies in pyproject.toml.)

Is this bad practice and should be stopped?

[sinoroc]

@bunny-therapist Maybe this will help: #1951 (comment)

[ThiefMaster]

TBH I think it makes perfect sense to put pinned deps in install_requires when you have an application which is expected to be installed into a clean (virtual) environment. pipx makes it easy and if you have a more sophisticated application with detailed installation instructions those can easily include creation of such an environment.

And in all those cases, being able to pip install XXX is convenient. ANY other option is likely to be more cumbersome:

• Download a requirements file and install from that? Yeah no, that's convoluted and someone will probably try to pip install the package first and then get versions that may not work at all
• Package using something else? Fine for desktop apps, but for stuff you install on a server (or just simple CLI tools) you REALLY don't want to deal with weird installer tools...

Obviously unmaintained applications are a different topic, but it doesn't really matter here because either they break because their old pinned dependencies might no longer work (or are no longer secure), or they break because the application is not compatible with the newer releases of some dependencies.

[fenchu]

The day I dreaded came with pip 33.1 release. We can no longer use setup.py and dynamically load requirements.txt.
I had to make a little script that can convert content of requirements.txt into pyproject.toml dependency array. no problem.

But this is definitely not an anti-pattern. we use deptracker, safety, Snyk etc to check dependencies and they all work on requirements.txt files. Now I just have to sent the the entire messy pyproject.toml file.

I would have started using pyptoject.toml years ago it this had worked, like it did some years ago, but I always prefered setup.py:

[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}

What we need just more fragmentation.

[pradyunsg]

https://blog.ganssle.io/articles/2023/01/attractive-nuisances.html#footnote-reference-1 is a good pointer as to the mistake we're repeating by allowing this.

[sinoroc]

@fenchu What were you doing before that is now not possible to do anymore with pip 23.1 (I guess you meant 23.1 instead of 33.1)? As far as I know it is possible to read a requirements.txt file from a setup.py. What is not working in your case? Can you show an example?

[pradyunsg]

The ship has sailed on this -- there's too many people relying on this behaviour now and we'll now have to deal with the confusion this'll continue to cause around mixing the concepts of abstract and concrete dependencies. :(