-
-
Notifications
You must be signed in to change notification settings - Fork 612
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pip-compile doesn't cache local requirements correctly. #1545
Comments
Thanks @mattdee123 . FWIW #1519, which is awaiting review, replaces |
Nice, that sounds like it'll help somewhat. Though having to load the file multiple times in ROUND 1 is still slower than I would think is technically required. Feels to me like a technically correct solution might be to add a layer of caching somewhere along the call path? Locally, I've tried it with the following monkeypatch to add a cache around
|
This is biting me as well; we have a bunch of local packages that use pyproject.toml, and this makes building extremely painfully slow. I'm resorting to building wheels and editing the generated requirements.txt to patch the |
You may wish to try #1539, adding |
@AndydeCleyre That works great! Hopefully that lands soon. |
This has been fixed in #1539 with the backtracking resolver, try |
I have a large repository with multiple
.in
files that depend on each other with-r
. There is a shared library which is included with-e
, and it is included by a number of sub-files. This makes generating files take a long time.It seems to me like this is due to the caching in _get_ireq_with_name using the full object. Despite all the imports being of the same library, they have some different fields which cause them to be unequal. I suspect that caching on
ireq.local_file_path
would speed things up.Environment Versions
Python 3.8.10
pip 21.3.1
6.4.1
Steps to replicate
req.in
in the parent directory of this repository (or any local package; just using this one as a convenient example)(obviously this kind of file should never happen in real life, but it's possible with multiple files included via
-r
to repeat)2. Run
pip-compile --verbose req.in
Expected result
This should run fairly quickly
Actual result
This takes longer than I'd expect, and with the
--verbose
flag, you can see that in each round it spends most of its time while printing something likeThis would be much faster if it didn't have to repeatedly do all this work for the same package. That should be possible, since this is just happening inside of
_get_ireq_with_name
to get the name of the package.Happy to submit a PR using
ireq.local_file_path
as a cache key, though I'm not sure if that's correct.The text was updated successfully, but these errors were encountered: