Add type annotations and mypy compliance

[abought]

🦑 Ready for review, but watch for sea monsters (my first WB PR)

Ticket

https://openscience.atlassian.net/browse/SVCS-351

Purpose

• Add type annotations to assist with debugging (core classes and some, but not all, providers)
• Provide an option (currently not on by default) to use mypy, a linter for gradual type hinting- catches weird usages of methods

Notes for reviewers

To try this out, run inv mypy.

In the future this can be used in Travis CI, but is currently hidden behind a flag by default:
inv test --types

Many of these types are a "best guess" and subject to evolve in the future. Python types are informational only and do not affect runtime, but tools such as mypy (and pycharm!) support automatic linting to catch errors during development.

I am new to waterbutler. If there is a better type to use, let me know. The ignore settings in mypy may be too aggressive and are worth revisiting in the future.

Specific rules that we ignore

Mypy does not appear to support selective ignoring of certain rules. Therefore, ignore statements are scattered throughout the code more than I would wish. See notes and rationale:

• Mypy doesn't seem to do very well with functions that can return more than one type of thing. (so basically every call to metadata gets ignored) See:
  https://github.com/python/mypy/issues/1693
  • This could be resolved using dependent types / @overload, if we some day refactor so that files and folders are represented via different classes.
• Method call signatures or return types are not consistent with parent classes. We routinely violate the Liskov substitution principle, of necessity as there is no uniform abstraction for all providers. See: https://github.com/python/mypy/issues/1237
  • There are a lot of ignores for this. It's terrible; I'm sorry.

Summary of changes

• Add type annotations to code
• Update docstrings
• Update method signatures that did not reflect the common usages across all subclasses

Testing notes

In theory this is a pure developer facing change. One or two minor changes were made to method call signatures.

Recommend watching for any new errors during common operations. I have done light manual testing consisting of uploads/downloads/renames/cross provider moves (osf storage to google drive).

[felliott]

Looks good, @abought! One minor nitpick per our poorly-advertised CONTRIBUTING.md.

[felliott]

Nitpick: Per our CONTRIBUTING.md, imports should be split into three groups (core, external, internal) and each group ordered by line length.

[abought]

Updated in 3705f2a

I haven't yet manually tested this; is there a page with suggestions on common operations to verify?

Some tests are passing locally but failing on travis; still investigating why. In particular, as part of the cleanup of abc stub classes, I added async keyword to a few methods of the base class (when all child classes were using it). Want to make sure this doesn't cause unintended side effects.

[abought]

Initial manual testing ok. Merge conflicts with googledrive refactor; I will update and revalidate before next week.

[coveralls]

Coverage increased (+0.4%) to 75.6% when pulling e3fc183 on abought:feature/svcs340-waterbutlerpath into 753bdd2 on CenterForOpenScience:develop.

[abought]

Merge conflicts resolved and basic retesting done. Let me know if there are additional checks requested!