[MRG] use tox for running tests locally #696

luizirber · 2019-07-10T03:54:45Z

Benefits:

Only need to install tox, and it takes care of installing all other deps.
Isolate code and temp files, avoiding testing the code in the dir (instead of the installed code)

(This is a slim version of #680, skipping the fix_lint parts that change A LOT of code)

Checklist

Is it mergeable?
make test Did it pass the tests?
make coverage Is the new code covered?
Did it change the command-line interface? Only additions are allowed without a major version increment. Changing file formats also requires a major version number increment.
Was a spellchecker run on the source code and documentation after
changes were made?

codecov · 2020-08-04T23:49:56Z

Codecov Report

Merging #696 (42ba7e5) into latest (12c7da8) will increase coverage by 10.63%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           latest     #696       +/-   ##
===========================================
+ Coverage   83.32%   93.95%   +10.63%     
===========================================
  Files         103       98        -5     
  Lines        9600    14615     +5015     
  Branches        0     1434     +1434     
===========================================
+ Hits         7999    13732     +5733     
+ Misses       1601      637      -964     
- Partials        0      246      +246

Flag	Coverage Δ
python	`93.95% <100.00%> (?)`
rusttests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/sourmash/__init__.py	`100.00% <ø> (ø)`
src/sourmash/__main__.py	`83.33% <ø> (ø)`
src/sourmash/cli/__init__.py	`95.65% <ø> (ø)`
src/sourmash/cli/categorize.py	`100.00% <ø> (ø)`
src/sourmash/cli/compare.py	`100.00% <ø> (ø)`
src/sourmash/cli/compute.py	`100.00% <ø> (ø)`
src/sourmash/cli/gather.py	`100.00% <ø> (ø)`
src/sourmash/cli/import_csv.py	`100.00% <ø> (ø)`
src/sourmash/cli/index.py	`100.00% <ø> (ø)`
src/sourmash/cli/info.py	`100.00% <ø> (ø)`
... and 161 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 12c7da8...42ba7e5. Read the comment docs.

luizirber · 2020-11-01T22:41:34Z

This branch runs black and some other of the pre-commit formatters, and shows how it will look like. I'm not adding to this PR because... it is massive =]

move most of setup.py to pyproject.toml and setup.cfg use isolated build for package creation isolate test running add some pyscaffold-inspired changes update precommit isort config tox.ini updates based on tox-dev/tox fix benchmarks for CI, add pypy wheels add nix.shell for env setup move sourmash to src/sourmash coverage fixes

ctb · 2020-12-10T14:51:22Z

reviewing tox part of things

I started out by creating a new conda environment with python3.8 in it, and running tox; got:

ERROR:  py39: InterpreterNotFound: python3.9
ERROR:   py38: InvocationError for command /Users/t/dev/sourmash.tox/.tox/py38/bin/python -m pip install --exists-action w '/Users/t/dev/sourmash.tox/.tox/.tmp/package/1/sourmash-4.0.0a4.dev2+g12c7da8.zip[test,storage]' (exited with code 2)
ERROR:  py37: InterpreterNotFound: python3.7
ERROR:  docs: InterpreterNotFound: python3.7

(I think the py38 / InvocationError is just a network error, so that's ok; will try again later.)

I'm not a fan of having the default be that a user has to have python 3.7, 3.8, and 3.9 installed just to run basic tests, so I suggest making that clear; I'll revisit the docs with that in mind.

`tox -e py38` fails in naive conda environment

did pip install tox and got this;
not sure if it is expected:

python -m pytest --cov=. --cov-report term-missing --cov-report=xml
/Users/t/dev/sourmash.tox/.tox/py38/bin/python: No module named pytest
make: *** [coverage] Error 1
ERROR: InvocationError for command /usr/bin/make coverage (exited with code 2)

`tox -e dev` doesn't work

this one is listed in the developer docs...?

% tox -e dev
ERROR: unknown environment 'dev'

ctb · 2020-12-10T14:55:53Z

as far as moving code under src/sourmash goes, thanks, I hate it, but that's just because I'm a conservative kinda guy and hate learning new things if I don't have to :). Let me see how much slower it makes running the tests - that's my only remaining concern.

ctb · 2020-12-10T14:56:07Z

(as far as breaking existing PRs... most of them are yours!)

luizirber · 2020-12-10T17:22:56Z

reviewing tox part of things
ERROR:  py39: InterpreterNotFound: python3.9
ERROR:   py38: InvocationError for command /Users/t/dev/sourmash.tox/.tox/py38/bin/python -m pip install --exists-action w '/Users/t/dev/sourmash.tox/.tox/.tmp/package/1/sourmash-4.0.0a4.dev2+g12c7da8.zip[test,storage]' (exited with code 2)
ERROR:  py37: InterpreterNotFound: python3.7
ERROR:  docs: InterpreterNotFound: python3.7
(I think the py38 / InvocationError is just a network error, so that's ok; will try again later.)

I'm not a fan of having the default be that a user has to have python 3.7, 3.8, and 3.9 installed just to run basic tests, so I suggest making that clear; I'll revisit the docs with that in mind.

Hmm, what version of tox are you running? I set up tox.ini to skip missing versions, it shouldn't be erroring with py39 and py37...

(is this running on this branch? or latest? Because docs should always be using 3.8, and I see it trying to use 3.7 🤔)

`tox -e py38` fails in naive conda environment

did pip install tox and got this;
not sure if it is expected:

python -m pytest --cov=. --cov-report term-missing --cov-report=xml
/Users/t/dev/sourmash.tox/.tox/py38/bin/python: No module named pytest
make: *** [coverage] Error 1
ERROR: InvocationError for command /usr/bin/make coverage (exited with code 2)

It makes sense that running pytest directly will fail, because it is not installed. But this is the command that was executed inside tox -e py38, right?

This worked for me:

conda create -n tox_test python=3.8
conda activate tox_test
python -m pip install tox
python -m tox -e py38

(I used python -m {pip,tox} because I had them installed somewhere else, and my environment was not picking the conda-installed ones)

tox -e dev doesn't work

this one is listed in the developer docs...?
% tox -e dev
ERROR: unknown environment 'dev'

Hmm, I'm increasingly thinking that you're not in the build/tox_travis branch, or it is an older revision... dev is the last item in tox.ini, can you check if it is there?

ctb · 2020-12-10T17:32:38Z

On Thu, Dec 10, 2020 at 09:23:13AM -0800, Luiz Irber wrote: (is this running on this branch? or `latest`? Because `docs` should always be using 3.8, and I see it trying to use 3.7 :thinking:)

oh. sorry. I think you nailed it :). trying again!

luizirber · 2020-12-10T17:39:24Z

as far as moving code under src/sourmash goes, thanks, I hate it, but that's just because I'm a conservative kinda guy and hate learning new things if I don't have to :). Let me see how much slower it makes running the tests - that's my only remaining concern.

I get that, and it is a big change, but there are good reasons to use the src/ layout, especially to be sure we are testing/measuring coverage for the right files...

(as far as breaking existing PRs... most of them are yours!)

Good, so it is up to me to fix them =]

ctb

Looks good overall, nice work!

I think it would be good to update the docs to point at src/sourmash/ rather than sourmash/.

Also, is there any documentation for the nix stuff?

luizirber · 2020-12-15T18:42:17Z

I think it would be good to update the docs to point at src/sourmash/ rather than sourmash/.

Yup, I'll do that now.

Also, is there any documentation for the nix stuff?

I'll add docs and CI for that, it's what I've been using for local dev in my laptop (setting up Rust, python and tox), but since I do most of my dev in the remote workstation it is not as well tested/organized yet.

ctb · 2020-12-15T18:43:22Z

On Tue, Dec 15, 2020 at 10:42:31AM -0800, Luiz Irber wrote: > Also, is there any documentation for the `nix` stuff? I'll add docs and CI for that, it's what I've been using for local dev in my laptop (setting up Rust, python and tox), but since I do most of my dev in the remote workstation it is not as well tested/organized yet.

ok, no worries/no need to invest in that now, but might be good to have an issue placeholder for this.

luizirber force-pushed the build/tox_travis branch 2 times, most recently from c8160be to 015bf25 Compare July 10, 2019 03:58

luizirber mentioned this pull request Sep 14, 2019

[RFC] CI and precommit hooks ctb/speet#1

Closed

luizirber changed the title ~~use tox for running tests~~ use tox for running tests locally Nov 22, 2019

luizirber added this to the post-3.0 milestone Dec 30, 2019

luizirber mentioned this pull request Feb 9, 2020

readthedocs fails to auto-import API docs #894

Closed

luizirber force-pushed the build/tox_travis branch from 015bf25 to e8f7afa Compare August 4, 2020 23:44

luizirber changed the base branch from master to latest August 5, 2020 00:29

luizirber force-pushed the build/tox_travis branch from b5aefe5 to 8be429d Compare August 5, 2020 22:23

luizirber changed the title ~~use tox for running tests locally~~ [WIP] use tox for running tests locally Aug 5, 2020

luizirber force-pushed the build/tox_travis branch 6 times, most recently from a47cd64 to 1eb8804 Compare August 12, 2020 21:47

luizirber mentioned this pull request Oct 21, 2020

ci: add flake to tests, format with black, and add precommit hooks spacegraphcats/spacegraphcats#326

Merged

luizirber mentioned this pull request Nov 1, 2020

[EXP] Set up black and pre-commit hooks #680

Closed

7 tasks

luizirber added the 4.0 issues to address for a 4.0 release label Nov 1, 2020

luizirber force-pushed the build/tox_travis branch from 1e2a83c to 7707359 Compare November 1, 2020 21:51

luizirber force-pushed the build/tox_travis branch 3 times, most recently from 6f1c084 to b68ab80 Compare November 2, 2020 15:59

luizirber force-pushed the build/tox_travis branch from b68ab80 to 154bf2f Compare December 8, 2020 22:13

luizirber mentioned this pull request Dec 9, 2020

CI: small build fixes #1252

Merged

5 tasks

luizirber force-pushed the build/tox_travis branch from d3510e4 to c1ec864 Compare December 9, 2020 18:02

luizirber added 2 commits December 9, 2020 10:06

use darker for progressive code formatting

1b32cb5

luizirber force-pushed the build/tox_travis branch 4 times, most recently from d8cd27d to 10ead56 Compare December 10, 2020 00:40

run docs and package_description on py38

74a5bd9

luizirber force-pushed the build/tox_travis branch from 10ead56 to 74a5bd9 Compare December 10, 2020 00:45

include tests in coverage

74de27d

luizirber added 4 commits December 10, 2020 09:46

remove 10x configs, more explicit dev instructions

fe02376

add tests to coverage tracking

55d4166

move hypothesis and khmer tests into tox

f580595

keep khmer and hypothesis in separate CI jobs

27ed5be

luizirber force-pushed the build/tox_travis branch from 6465359 to 27ed5be Compare December 10, 2020 20:19

update setuptools_scm version

ab891a9

luizirber changed the title ~~[WIP] use tox for running tests locally~~ [MRG] use tox for running tests locally Dec 14, 2020

ctb approved these changes Dec 15, 2020

View reviewed changes

doc and path updates

4640996

luizirber mentioned this pull request Dec 15, 2020

Document and CI nix usage #1254

Open

small doc fixes

42ba7e5

luizirber merged commit ca201cf into latest Dec 15, 2020

luizirber deleted the build/tox_travis branch December 15, 2020 19:57

This was referenced Feb 7, 2021

Draft release notes for v4.0 #1309

Closed

[MRG] backport github actions to stable branch (3.5.x) #1317

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MRG] use tox for running tests locally #696

[MRG] use tox for running tests locally #696

luizirber commented Jul 10, 2019 •

edited

Loading

codecov bot commented Aug 4, 2020 •

edited

Loading

luizirber commented Nov 1, 2020 •

edited

Loading

ctb commented Dec 10, 2020

ctb commented Dec 10, 2020

ctb commented Dec 10, 2020

luizirber commented Dec 10, 2020

reviewing tox part of things

`tox -e py38` fails in naive conda environment

`tox -e dev` doesn't work

ctb commented Dec 10, 2020 via email

luizirber commented Dec 10, 2020

ctb left a comment

luizirber commented Dec 15, 2020

ctb commented Dec 15, 2020 via email

[MRG] use tox for running tests locally #696

[MRG] use tox for running tests locally #696

Conversation

luizirber commented Jul 10, 2019 • edited Loading

Checklist

codecov bot commented Aug 4, 2020 • edited Loading

Codecov Report

luizirber commented Nov 1, 2020 • edited Loading

ctb commented Dec 10, 2020

reviewing tox part of things

tox -e py38 fails in naive conda environment

tox -e dev doesn't work

ctb commented Dec 10, 2020

ctb commented Dec 10, 2020

luizirber commented Dec 10, 2020

reviewing tox part of things

tox -e py38 fails in naive conda environment

tox -e dev doesn't work

ctb commented Dec 10, 2020 via email

luizirber commented Dec 10, 2020

ctb left a comment

Choose a reason for hiding this comment

luizirber commented Dec 15, 2020

ctb commented Dec 15, 2020 via email

luizirber commented Jul 10, 2019 •

edited

Loading

codecov bot commented Aug 4, 2020 •

edited

Loading

luizirber commented Nov 1, 2020 •

edited

Loading

`tox -e py38` fails in naive conda environment

`tox -e dev` doesn't work

`tox -e py38` fails in naive conda environment

`tox -e dev` doesn't work