Fix #5889: AttributeError: 'NoneType' object has no attribute 'netloc'

[aloosley]

Hello,

If accepted, this trivial change solves the issue I started to describe March 14 2019 in issue 5889. To be self contained, I elaborate everything below:

• pip: 19.0.3
• OS: Windows 10 (using git bash)
• python: 3.6

I am installing a private package called deep-nlp-embedding (pip install -e .) that depends on another private package called deep-nlp-core that was previously installed with pip install -e .

The package deep-nlp-core has install_requires:

install_requires = [
    "gensim>=3.4.0",
    "torch@ https://download.pytorch.org/whl/cu90/torch-1.0.0-cp36-cp36m-win_amd64.whl"
]

The package deep-nlp-embeddings has install_requires:

install_requires = [
    "deep-nlp-core"
]

Finally installing deep-nlp-embedding breaks pip:

$ pip install -e deep-nlp-embedding/
Obtaining file:///E:/Repos/flamingo-nlp/deep-nlp-embedding
Requirement already satisfied: deep-nlp-core in e:\repos\flamingo-nlp\deep-nlp-core (from deep-nlp-embedding==0.0.2) (0.0.2)
Exception:
Traceback (most recent call last):
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\cli\base_command.py", line 179, in main
    status = self.run(options, args)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\commands\install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\resolve.py", line 357, in _resolve_one
    add_req(subreq, extras_requested=available_requested)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\resolve.py", line 314, in add_req
    use_pep517=self.use_pep517
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\req\constructors.py", line 328, in install_req_from_req_string
    if req.url and comes_from.link.netloc in domains_not_allowed:
AttributeError: 'NoneType' object has no attribute 'netloc'

Because this change is trivial, I assume it doesn't require a NEWS.rst entry.

[xavfernandez]

Thanks for the PR.

Because this change is trivial, I assume it doesn't require a NEWS.rst entry.

Well, while the change is trivial, it is still a bugfix for #5889 and deserve an entry in the change log...
And ideally a test failing before this change.

[aloosley]

Well, while the change is trivial, it is still a bugfix for #5889 and deserve an entry in the change log...

I'm on it.

And ideally a test failing before this change.

I'm on it too.

[aloosley]

Hi @xavfernandez,
The only tests failing seem to be because imports are out of order. Should I continue to try and solve this? If so, is there a code style guide that I can use?

Thank you.

[xavfernandez]

@aloosley yes please.

tox -e lint-py3 should tell you what imports to fix.

[aloosley]

@aloosley yes please.

tox -e lint-py3 should tell you what imports to fix.

Doing this just tells me the same as the CI, that Imports are incorrectly sorted. After trying several import orders, sorry, I cannot figure out which order is required for this test to go away.

lint-py3 installed: entrypoints==0.3,flake8==3.7.6,isort==4.3.4,mccabe==0.6.1,pip==19.0.3,pycodestyle==2.5.0,pyflakes==2.1.1,setuptools==40.8.0,wheel==0.33.1
lint-py3 run-test-pre: PYTHONHASHSEED='424820836'
lint-py3 runtests: commands[0] | flake8
lint-py3 runtests: commands[1] | isort --check-only --diff
ERROR: /Users/a.loosley/Alex/Repos/pip/tests/unit/test_req_install.py Imports are incorrectly sorted.
--- /Users/a.loosley/Alex/Repos/pip/tests/unit/test_req_install.py:before	2019-03-18 12:17:28.994151
+++ /Users/a.loosley/Alex/Repos/pip/tests/unit/test_req_install.py:after	2019-03-18 12:17:37.316608
@@ -2,13 +2,12 @@
 import tempfile
 
 import pytest
+from pip._vendor.packaging.requirements import Requirement
 
 from pip._internal.req.constructors import (
-    install_req_from_line, install_req_from_req_string
+    install_req_from_line, install_req_from_req_string,
 )
 from pip._internal.req.req_install import InstallRequirement
-
-from pip._vendor.packaging.requirements import Requirement
 
 
 class TestInstallRequirementBuildDirectory(object):

[xavfernandez]

-    install_req_from_line, install_req_from_req_string
+    install_req_from_line, install_req_from_req_string,

indicates the missing final ,

 import pytest
+from pip._vendor.packaging.requirements import Requirement
[...]
-from pip._vendor.packaging.requirements import Requirement

indicates that this line should move right after import pytest

FWIW, pip install isort==4.3.4 followed by running isort should guide you with the expected changes.

[cjerdonek]

A few random comments.

[aloosley]

@cjerdonek , thanks for comments on the code. I hope they are all solved now.

[xavfernandez]

Thanks a lot for your work 👍

[aloosley]

Thanks a lot for your work 👍

Thanks also for the intro to isort / tox.

[cjerdonek]

Some minor nits before merging.

[aloosley]

Some minor nits before merging.

Done.

[cjerdonek]

You're going to get a style error here, but note that you don't need a + (parentheses concatenate without that).

[cjerdonek]

You might need to do something like the following to address linter complaints--

wheel_url = (
    "https://download.pytorch.org/whl/cu90/..."
)

[aloosley]

Now done?

[xavfernandez]

Thanks a lot @aloosley 🎉

[WeatherGod]

Is this fix in a released pip yet? Or perhaps there is a conda channel I can use to double-check this fix? Looking at the test, I don't think it properly exercises the bug I originally reported.

Also, as a side-note. I am not familiar with pip's testing system, but it looks like the test added here depends upon an external resource? Why not use a fixture approach instead of fetching from the internet?

[cjerdonek]

Is this fix in a released pip yet?

No, but it will be in the next release (19.1).

Looking at the test, I don't think it properly exercises the bug I originally reported.

It looks fine to me. For example, without the code change, one of the added tests will raise:

AttributeError: 'NoneType' object has no attribute 'netloc'

I am not familiar with pip's testing system, but it looks like the test added here depends upon an external resource? Why not use a fixture approach instead of fetching from the internet?

It doesn't look to me like it fetches from the internet. It just calls install_req_from_req_string(), which is essentially a constructor / factory function.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.