Drop support for Python 2.7

[ghost]

Python will drop support for 2.7 in January 1st, 2020, I think DVC should too, it will ease the maintenance (spending less time dealing with str/unicode issues, default support for type checking syntax, access to a more robust standard library, etc.).

Countdown for Python 2.7 EOL: https://pythonclock.org/

I don't know how many users are currently running on Python 2.7, but we can get a clue by analyzing pypi stats with BigQuery or using pypinfo.

There are some maintenance tasks to migrate to a Python 3.x only codebase:

• Remove 2.7 and 3.4 from our conda package
• Remove Python 2.7 from TravisCI (EDIT(30 Dec) removed from travis.yaml already, but need to cleanup scripts/ci/* from py2 code)
• Upgrade AppVeyor's Python version
• Modify python_requires and explicit version classifiers (kudos to @hugovk !)
• Update the package
• Remove all calls/dependencies to future
• Remove all code linked to utils.compat
• Refactor code with comments about python 2 backwards compatibility
• Update super() to new syntax
• Refactor DvcExceptions to use default traceback instead of custom one
  • Refactor logger.py. Remove logging formatter parse_exc logic and use formatException and formatStack instead
• Refactor system.py module to use the default os methods for linking, as they include Windows support
• Make keyword params kwonly in dvc.api. Consider the same in other places. (api: use only kwargs #3108)

Feel free to edit this comment, modifying the list of tasks that should happen to make a clean transition.

NOTE: This issue was created to keep track of what should be done. I'm not implying that this will happen in a near future, neither expressing the ideas of the rest of the maintainers.
It would be great if you can write down your opinion so we reach a consensus between the maintainers and the community.

[hugovk]

I'd recommend adding python_requires and explicit version classifiers, especially useful when 2.7 is dropped, and also useful now. Please see PR #1845.

---

I don't know how many users are currently running on Python 2.7, but we can get a clue by analyzing pypi stats with BigQuery or using pypinfo.

Here's the pip installs for dvc from PyPI for March 2019:

category	percent	downloads
3	73.99%	6,320
2	17.53%	1,497
null	8.49%	725
Total		8,542

Source: pypistats python_major dvc --last-month # pip install pypistats

category	percent	downloads
3.6	42.39%	3,621
3.5	20.21%	1,726
2.7	17.51%	1,496
3.7	11.33%	968
null	8.49%	725
3.4	0.06%	5
2.6	0.01%	1
Total		8,542

Source: pypistats python_minor dvc --last-month # pip install pypistats

That's a very low proportion of Python 2, see https://dev.to/hugovk/python-version-share-over-time-4-26hm for some other projects.

[ghost]

Thank you so much for the detailed stats and for contributing with #1845 and #1847 , @hugovk ! 🔥

[ghost]

Time flies when you're having fun)
EOL is around the corner, I'm eager to close this issue))

[efiop]

For the record: trying to drop support for 2.7 (and 3.4, as it has reached EOL already) in conda-forge/dvc-feedstock#54 , as it is starting to cause problems during the build.

[ghost]

what do you mean, @efiop ? is it happening? are we finally dropping 2.7?
I can happily spend my weekend on the migration 😅

[efiop]

@MrOutis No, just dropping it where it causes serious issues. The main feast will come later, you won't miss it 😄

[ghost]

I'm starting to feel the christmas spirit, we all joyful writing python 3.x code 🙂

[efiop]

@MrOutis The joy probably won't fully come until py35 is dead, which is 1 more year 🙂

[ghost]

https://github.com/asottile/pyupgrade

[casperdcl]

Eek python_requires is the most important version-related keyword. Really bad that it wasn't included all this time. Definitely need at least one release explicitly allowing py>=3,<=3.4 and py<3 otherwise things could break in future. Looks like that's happened already though.

[efiop]

@casperdcl Last release was 0.78.1 and it has py2 support, so not broken. We'll limit to >py3.4 in the next one. Plus would be probably nice to put some kind of sanity check for that in CLI and when importing dvc

[ghost]

It is happening!

[ghost]

yay or nay for relative imports?

(cc: @pared , @Suor)

[ghost]

what about using ABCs?

[casperdcl]

Are there py3-specific ABC features you refer to?

[ghost]

No, @casperdcl; I thought that ABCs weren't implemented in Python 2.7, don't know why we are not using them for Remotes and Outputs, then 🤔

[efiop]

@MrOutis Let's not do that within this sprint, but rather create a separate ticket for that for later. We'll keep in mind them next time we are going to create appropriate classes. Relative imports could be moved to a separate ticket as well, as it is a bit unrelated and we currently mismatch relative and abs imports all over the codebase 🙂

[casperdcl]

I agree with @efiop - both features were supported in py2 (may have needed things like __future__.absolute_import) but not related to deprecation. I'm fine with removal of __future__ (which has already happened) but as @efiop said would leave actually using relative imports and ABCs everywhere for another issue.

[ghost]

Great, @efiop , @casperdcl ; should check better next time, I thought they were not supported in Python 2.7, this is why I was asking 😅

[ghost]

The last thing to address is using kwargs only on api.py module.

This is how it looks like:

>>> def hello(*, name='world'): print('hello', name)
>>> hello()
"hello world"
>>> hello("people")
TypeError: hello() takes 0 positional arguments but 1 was given

It makes the interface more explicit.
To make it work you only need to put an asterisk before the keyword arguments.

I can do it for api.py but not sure if make sense for other modules. I would leave it as a separate issue. (#3108)

[ghost]

Should we close this one or anyone has any other suggestion? @iterative/engineering

[ghost]

Let's not leave this one open, we can handle any further suggestions on separate issues :)