-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Surprising behaviour of relative directory references in requirement files #8765
Comments
The requirement objects are constructed before the resolver is reached, and both implementations get the same things. It is possible to only fix in the new resolver, but that feels hacky to me. Maybe it’d be better to deprecate relative paths entirely, and introduce a couple of custom environment variables to use (e.g. |
Or... we could maintain status quo where (almost) no one is complaining, and the explanation of how the feature works is straightforward enough. :) |
Well, that might be straightforward to explain but quirky nevertheless. Especially if you consider requirement files located at a remote URL. requirements.txt:
subreq.txt:
|
That feels like a case special enough that I'm personally not interested in solving this until someone comes in to say "it's important for my workflow to improve behaviour in this situation" and even then, only if they ask nicely. :P |
### Description Standard requirements.txt files have a few issues with a third party library installed from source: 1. pip does not handle [relative paths](pypa/pip#8765) in a predictable way. It is relative to current working directory instead of relative to the path of the requirements.txt. As we cannot constrain that, it leads to unpredictable behavior. 2. `pip freeze` does not generate the correct output when there is a local library. ### Test plan 1. tried to deploying to elastic beanstalk, and it seems to work. 2. ran `make update-deps` 3. ran `make update-deps` with a local library and `Pipfile.lock` references that instead of just a version number.
### Description Standard requirements.txt files have a few issues with a third party library installed from source: 1. pip does not handle [relative paths](pypa/pip#8765) in a predictable way. It is relative to current working directory instead of relative to the path of the requirements.txt. As we cannot constrain that, it leads to unpredictable behavior. 2. `pip freeze` does not generate the correct output when there is a local library. ### Test plan 1. tried to deploying to elastic beanstalk, and it seems to work. 2. ran `make update-deps` 3. ran `make update-deps` with a local library and `Pipfile.lock` references that instead of just a version number.
### Description Standard requirements.txt files have a few issues with a third party library installed from source: 1. pip does not handle [relative paths](pypa/pip#8765) in a predictable way. It is relative to current working directory instead of relative to the path of the requirements.txt. As we cannot constrain that, it leads to unpredictable behavior. 2. `pip freeze` does not generate the correct output when there is a local library. ### Test plan 1. tried to deploying to elastic beanstalk, and it seems to work. 2. ran `make update-deps` 3. ran `make update-deps` with a local library and `Pipfile.lock` references that instead of just a version number.
### Description Standard requirements.txt files have a few issues with a third party library installed from source: 1. pip does not handle [relative paths](pypa/pip#8765) in a predictable way. It is relative to current working directory instead of relative to the path of the requirements.txt. As we cannot constrain that, it leads to unpredictable behavior. 2. `pip freeze` does not generate the correct output when there is a local library. ### Test plan 1. tried to deploying to elastic beanstalk, and it seems to work. 2. ran `make update-deps` 3. ran `make update-deps` with a local library and `Pipfile.lock` references that instead of just a version number.
### Description Standard requirements.txt files have a few issues with a third party library installed from source: 1. pip does not handle [relative paths](pypa/pip#8765) in a predictable way. It is relative to current working directory instead of relative to the path of the requirements.txt. As we cannot constrain that, it leads to unpredictable behavior. 2. `pip freeze` does not generate the correct output when there is a local library. ### Test plan 1. tried to deploying to elastic beanstalk, and it seems to work. 2. ran `make update-deps` 3. ran `make update-deps` with a local library and `Pipfile.lock` references that instead of just a version number.
Suppose we have the following dependencies in the environment.yml: ``` dependencies: - pip - pip: - -r requirements.txt - -e deps/my_local_package ``` Without this commit, it is impossible to install such dependencies with micromamba. That is because micromamba was creating its own temporary requirements.txt file in the /tmp directory and called pip to install them. For some reason pip resolves `-r` paths as relative to the file that contains them, but `-e` paths as relative to the current working directory. It is a known issue in pip: pypa/pip#8765. So in the case of mamba, pip looks for `/tmp/requirements.txt` and `directory/of/env/yml/deps/my_local_package` and unless whole project is thrown right into `/tmp` this installation will fail. This commit makes it so installing pip deps matches the behavior of conda.
Suppose we have the following dependencies in the environment.yml: ``` dependencies: - pip - pip: - -r requirements.txt - -e deps/my_local_package ``` Without this commit, it is impossible to install such dependencies with micromamba. That is because micromamba was creating its own temporary requirements.txt file in the /tmp directory and called pip to install them. For some reason pip resolves `-r` paths as relative to the file that contains them, but `-e` paths as relative to the current working directory. It is a known issue in pip: pypa/pip#8765. So in the case of mamba, pip looks for `/tmp/requirements.txt` and `directory/of/env/yml/deps/my_local_package` and unless whole project is thrown right into `/tmp` this installation will fail. This commit makes it so installing pip deps matches the behavior of conda.
Suppose we have the following dependencies in the environment.yml: ``` dependencies: - pip - pip: - -r requirements.txt - -e deps/my_local_package ``` Without this commit, it is impossible to install such dependencies with micromamba. That is because micromamba was creating its own temporary requirements.txt file in the /tmp directory and called pip to install them. For some reason pip resolves `-r` paths as relative to the file that contains them, but `-e` paths as relative to the current working directory. It is a known issue in pip: pypa/pip#8765. So in the case of mamba, pip looks for `/tmp/requirements.txt` and `directory/of/env/yml/deps/my_local_package` and unless whole project is thrown right into `/tmp` this installation will fail. This commit makes it so installing pip deps matches the behavior of conda. --------------------- Co-authored-by: Wolf <[email protected]>
Environment
Description
In requirement files, relative references to other requirement files (via
-r
or-c
) are interpreted to be relative to the base url of the including file. However requirements expressed as relative directories (with our without-e
) are interpreted to be relative to the current working directory.Expected behavior
Relative directory requirements should be interpreted as relative to the base url of the requirement file.
How to Reproduce
The following bash scripts fails installing
./pkga
, where I'd expect it to succeed.output:
Compatibility considerations
Modifying this behaviour would be a breaking change.
Assuming there is agreement this needs changing, could this be introduced as part of the
2020-resolver
feature flag?The text was updated successfully, but these errors were encountered: