Vendoring of tenacity is leaky

[bdarnell]

Description

Tenacity contains a conditional import of tornado. This makes the behavior of pip sensitive to third-party packages outside of its vendor directory. Specifically, if a version of tornado that does not include the tornado.gen.sleep function is installed, pip will fail to start. (This is unlikely since this function has been around a long time and we have no plans of deprecating it or removing it. But we do have a report of this happening in tornadoweb/tornado#3034)

Expected behavior

Pip should not be affected by the presence or absence of any other third-party packages. Any conditional imports in its vendored dependencies should be modified to unconditionally fail (e.g. replace the above-linked block with tornado = None).

pip version

21.1.2

Python version

3.8

OS

linux

How to Reproduce

1. pip install tornado==4.0.0
2. pip --version

Output

~/ImpressionableVoluminousCategories$ pip --version
pip 21.1.2 from /opt/virtualenvs/python3/lib/python3.8/site-packages/pip (python 3.8)
~/ImpressionableVoluminousCategories$ pip install tornado==4.0.0
Collecting tornado==4.0.0
  Downloading tornado-4.0.tar.gz (313 kB)
     |████████████████████████████████| 313 kB 4.5 MB/s 
Requirement already satisfied: certifi in /opt/virtualenvs/python3/lib/python3.8/site-packages (from tornado==4.0.0) (2020.12.5)
Building wheels for collected packages: tornado
  Building wheel for tornado (setup.py) ... done
  Created wheel for tornado: filename=tornado-4.0-cp38-cp38-linux_x86_64.whl size=344556 sha256=d9c5e6911e5bdac5b90db4b33d01891562365e235396bd336380dd45cb61a9b7
  Stored in directory: /home/runner/.cache/pip/wheels/9a/d7/93/a846246f95067512a78899329bdb84a695d693e67c28a4e71f
Successfully built tornado
Installing collected packages: tornado
Successfully installed tornado-4.0
~/ImpressionableVoluminousCategories$ pip --version
Traceback (most recent call last):
  File "/opt/virtualenvs/python3/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
    from pip._internal.cli import cmdoptions
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 23, in <module>
    from pip._internal.cli.parser import ConfigOptionParser
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/cli/parser.py", line 12, in <module>
    from pip._internal.configuration import Configuration, ConfigurationError
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/configuration.py", line 27, in <module>
    from pip._internal.utils.misc import ensure_dir, enum
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 38, in <module>
    from pip._vendor.tenacity import retry, stop_after_delay, wait_fixed
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_vendor/tenacity/__init__.py", line 523, in <module>
    from pip._vendor.tenacity.tornadoweb import TornadoRetrying
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_vendor/tenacity/tornadoweb.py", line 26, in <module>
    class TornadoRetrying(BaseRetrying):
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/pip/_vendor/tenacity/tornadoweb.py", line 27, in TornadoRetrying
    def __init__(self, sleep=gen.sleep, **kwargs):
AttributeError: module 'tornado.gen' has no attribute 'sleep'
~/ImpressionableVoluminousCategories$ 

### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).

[uranusjr]

Thanks for raising the issue. I think an easy way would be to vendor a patch to delete those imports. We already have a few of those here. Would you be interested in contributing one for this?

[bdarnell]

Sure: #10029

[pradyunsg]

@sbidoul do you think this is important enough for a bugfix release? 😅

[pfmoore]

Is this a new problem that was introduced in 21.1, or has it been around for a while and just not been noticed before? If it's not a new problem, I doubt it's worth a bugfix release (although it's the RM's call). If it is a new issue, I'm still ambivalent - it seems pretty rare in practice 🤷

[bdarnell]

It appears to be new in 21.1 (introduced in 64ecfc8). It is indeed rare (it requires using a version of Tornado from before Feb 2015), but it has happened and when it happens it breaks things in a way that you can't really recover from without blowing away the virtualenv. (but overall I'm ambivalent too - personally I'd probably wait for a second occurrence before starting a release).

[pfmoore]

Ah yes, I'd forgotten we'd switched libraries. But equally, it's worth remembering that 21.2 isn't that far off (July) so we're not talking about a huge delay either way.

[pradyunsg]

Well, if someone else hits this, it'd probably make sense to expidite this. Otherwise, we'll leave it be until the next release.

[bdarnell]

FYI we've had a second commenter on tornadoweb/tornado#3034 so it does look like this is affecting more than one person.

[pradyunsg]

Heyo @bdarnell! Gentle nudge to not mention the issue link/number in the commit message.

That results in a lot of spam in this issue, as people rebase that PR or cherry pick changes or just do anything with that commit on GitHub.

[bdarnell]

Ah, OK, I can remove that if I have to update the commit again. I'm well aware of the spam that can come from an @username mention in a commit message, but most of the projects I've contributed to embrace the commit-message-to-issue linking.

[Eeyhan]

I just solve，If you want to solve this problem quickly，pull down the git source code for Tornado>=4.0 and above and install it with the command:
python setup.py install