Dev Tool Updates #379
Merged
Dev Tool Updates #379
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Drop `nose` from package requirements and update all tests to be compatible with `pytests`. Changes for compatibility are pretty minor. For the most part all that was necessary is updating some `nose` specific asserts to the `pytest` equivalent. There were some issues with configuration being corrupted by test modules that didn't adequately clean up after execution. I assume this issue pops up due to the way `pytest` executes tests compared to `nose`. Note, we should work on updating all our tests to use the `__init__.TestFile` class for automatically dealing with temporary files. Our current approaches are a bit all over the place at the moment.
Move all optional / development dependencies under `dev` instead of splitting it across `tests` and `docs`. Add a number of extra dependencies for updating our development tooling including black, flake8, mypy, and tox.
Add an initial pre-commit pipeline and run across all files in the repo. Add various fixes for mypy related errors including adding stubs, ignoring certain dependencies in setup.cfg (for those where stubs couldn't be located), and manually add type ignore comments where necessary. These mypy related issues should be revisited in the future to see if they can be cleaned up in a more coherent way.
!! NOTE !! Some of these fixes are backwards incompatible changes in API function and / or argument names.
!! NOTE !! These changes drop the deprecated "old" database API functions. These have been marked deprecated since #85 / version 1.1.0 and are multiple years out of date.
Address linting errors in DMC. Where possible, fix casing issues, add typing info, and clean up docstrings. !! NOTE !! Many of these changes are breaking API changes due to casing adjustments on the public API. This will require updates in other packages / modules.
!! NOTE !! Many of these changes are backwards incompatible API changes due to casing fixes in the public API.
!! NOTE !! This includes backwards incompatible changes due to public API changes.
!! NOTE !! These changes include backwards incompatible public API changes.
Cleanup "missing" function calls due to API changes, minor bugs in val.py from bad variable name cleanup, and get the test suite running cleanly again.
Note, this changes the default kwarg zmq_args handling to avoid using a mutable valuable as a keyword default. The unittest suite runs clean but this was NOT run in a full data flow use case.
Note, this removes default kwarg zmq_args definitions that use a mutuable object. This was not tested in a full end to end data flow test.
Add tox config for building our docs, executing our test suite, and running our pre-commit linter pipeline. Move all our tests into a top level tests/ directory to address annoying issues with tox / pytest and the wrong files being located. Add MANIFEST.in so our data files get picked up properly in source distributions.
jasonmlkang
approved these changes
Oct 26, 2021
+1 on the docs. Changes are in bound in a PR soon. I have another one coming up with a number of packaging changes and then a bunch of docs updates will follow shortly. |
nttoole
approved these changes
Oct 27, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is part 1 of part [number > 1] of updates for #364. The changes here cover the vast majority of the updates necessary to expand our dev tooling for Core. There's some changes in here related to updating our packaging but I'm more fully in a follow on PR. See the "work to do" list below for what's missing here.
What's here
pytest, update the test suite as necessary, and dropnose.blackand run a reformatting of the entire code baseflake8and useful config, cleanup all linting errors in the code base.mypyand attempt to address issues in the code base that is raises. A non-trivial amount pop up because of the dynamic handling that we have for our logging and config. We'll have to figure out a nicer way to address that.pre-commitconfig that executes our entire linting pipeline on commit and includes YAML dictionary validation and test execution on push.devextra_requireWork to Go
!!! NOTE !!!
Some of the changes here represent backwards incompatible changes due to API name changes. Could we have marked all these issue as
noqaand dealt with them later? Yes. But, it needs done eventually and I was already there fixing them. Might as well bite the bullet and get it over with.