diff --git a/.coveragerc b/.coveragerc deleted file mode 100644 index 09b80d8275..0000000000 --- a/.coveragerc +++ /dev/null @@ -1,10 +0,0 @@ -[run] -include= - sourmash/*.py - sourmash_lib/*.py -omit = - doc/conf.py - setup.py - tests/* - .tox/* - benchmarks/* diff --git a/.github/workflows/asv.yml b/.github/workflows/asv.yml index 124d453307..9d0106f794 100644 --- a/.github/workflows/asv.yml +++ b/.github/workflows/asv.yml @@ -19,6 +19,10 @@ jobs: uses: actions/setup-python@v1 with: python-version: 3.7 + - uses: actions-rs/toolchain@v1 + with: + toolchain: stable + override: true - name: Install dependencies run: | python -m pip install --upgrade pip diff --git a/.gitignore b/.gitignore index d42fcdc7e7..919cdccd05 100644 --- a/.gitignore +++ b/.gitignore @@ -18,10 +18,10 @@ sourmash.egg-info .coverage .pytest_cache .python-version -sourmash/version.py +src/sourmash/version.py *.DS_Store .tox -sourmash/_lowlevel*.py +src/sourmash/_lowlevel*.py .env Pipfile Pipfile.lock diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000000..c4c2868f0e --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,62 @@ +default_language_version: + python: python3 +repos: +- repo: https://github.com/pre-commit/pre-commit-hooks + rev: v3.2.0 + hooks: + - id: check-ast +# - id: check-builtin-literals + - id: check-docstring-first + - id: check-merge-conflict + - id: check-yaml + - id: check-toml + - id: debug-statements +# - id: end-of-file-fixer +# exclude: 'tests/test-data' +# - id: trailing-whitespace +# exclude: 'tests/test-data' +#- repo: https://github.com/asottile/pyupgrade +# rev: v2.7.2 +# hooks: +# - id: pyupgrade +#- repo: https://github.com/pre-commit/mirrors-isort +# rev: v5.4.2 +# hooks: +# - id: isort +# additional_dependencies: [toml] +#- repo: https://github.com/psf/black +# rev: 20.8b1 +# hooks: +# - id: black +# args: +# - --safe +# language_version: python3.8 +#- repo: https://github.com/asottile/blacken-docs +# rev: v1.8.0 +# hooks: +# - id: blacken-docs +# additional_dependencies: +# - black==19.10b0 +# language_version: python3.8 +#- repo: https://github.com/asottile/add-trailing-comma +# rev: v2.0.1 +# hooks: +# - id: add-trailing-comma +#- repo: https://github.com/pre-commit/pygrep-hooks +# rev: v1.6.0 +# hooks: +# - id: rst-backticks +#- repo: https://github.com/asottile/setup-cfg-fmt +# rev: v1.11.0 +# hooks: +# - id: setup-cfg-fmt +# args: +# - --min-py3-version +# - '3.7' +#- repo: https://gitlab.com/pycqa/flake8 +# rev: 3.8.3 +# hooks: +# - id: flake8 +# additional_dependencies: +# - flake8-bugbear == 20.1.2 +# language_version: python3.8 diff --git a/MANIFEST.in b/MANIFEST.in index 1d206dc896..1d2b9be999 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -1,10 +1,4 @@ -include LICENSE Makefile Dockerfile LICENSE Makefile README.md requirements.txt -include index.ipynb -include sourmash VERSION -recursive-include sourmash_lib * -recursive-include sourmash * -recursive-include src *.rs -recursive-include benches *.rs +include LICENSE Makefile README.md requirements.txt include Cargo.toml include include/sourmash.h prune .eggs @@ -12,5 +6,5 @@ global-exclude *.rlib global-exclude *.orig global-exclude *.pyc global-exclude *.so -prune tests/test-data/ -global-exclude *.git/ +prune tests/test-data +global-exclude *.git diff --git a/Makefile b/Makefile index 95dbe62eae..65b55db68b 100644 --- a/Makefile +++ b/Makefile @@ -18,13 +18,12 @@ install: all dist: FORCE $(PYTHON) setup.py sdist -test: all - $(PYTHON) -m pip install -e '.[test]' - $(PYTHON) -m pytest +test: + tox -e py38 cargo test -doc: build .PHONY - cd doc && make html +doc: .PHONY + tox -e docs include/sourmash.h: src/core/src/lib.rs \ src/core/src/ffi/hyperloglog.rs \ @@ -36,8 +35,7 @@ include/sourmash.h: src/core/src/lib.rs \ RUSTUP_TOOLCHAIN=nightly cbindgen -c cbindgen.toml . -o ../../$@ coverage: all - $(PYTHON) setup.py build_ext -i - $(PYTHON) -m pytest --cov=. --cov-report term-missing --cov-report=xml + tox -e coverage benchmark: asv continuous latest `git rev-parse HEAD` diff --git a/nix.shell b/nix.shell new file mode 100644 index 0000000000..57ba5d1039 --- /dev/null +++ b/nix.shell @@ -0,0 +1,51 @@ +let + moz_overlay = import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz); + nixpkgs = import { overlays = [ moz_overlay ]; }; + ruststable = (nixpkgs.latest.rustChannels.stable.rust); + + mach-nix = import ( + builtins.fetchGit { + url = "https://github.com/DavHau/mach-nix/"; + ref = "2.0.0"; + } + ); + + customPython = mach-nix.mkPython { + python = nixpkgs.python38; + requirements = '' + screed>=0.9 + cffi>=1.14.0 + numpy + matplotlib + scipy + deprecation>=2.0.6 + cachetools >=4,<5 + setuptools>=38.6.0 + milksnake + setuptools_scm>=3.2.0 + setuptools_scm_git_archive + pytest + pytest-cov + hypothesis + tox + ''; + }; + +in + with nixpkgs; + + nixpkgs.mkShell { + buildInputs = [ + customPython + git + stdenv + ruststable + stdenv.cc.cc.lib + ]; + + shellHook = '' + # workaround for https://github.com/NixOS/nixpkgs/blob/48dfc9fa97d762bce28cc8372a2dd3805d14c633/doc/languages-frameworks/python.section.md#python-setuppy-bdist_wheel-cannot-create-whl + export SOURCE_DATE_EPOCH=315532800 # 1980 + export LD_LIBRARY_PATH="${stdenv.cc.cc.lib}/lib64:$LD_LIBRARY_PATH"; + ''; +} diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000000..f941122703 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,21 @@ +[build-system] +requires = [ + "setuptools >= 48", + "setuptools_scm[toml] >= 4, <5", + "setuptools_scm_git_archive", + "milksnake", + "wheel >= 0.29.0", +] +build-backend = 'setuptools.build_meta' + +[tool.setuptools_scm] +write_to = "src/sourmash/version.py" +git_describe_command = "git describe --dirty --tags --long --match v* --first-parent" + +[tool.isort] +known_third_party = ["deprecation", "hypothesis", "mmh3", "numpy", "pkg_resources", "pytest", "screed", "setuptools", "sourmash_tst_utils"] +multi_line_output = 3 +include_trailing_comma = true +force_grid_wrap = 0 +line_length = 88 +known_first_party = ["sourmash"] diff --git a/pytest.ini b/pytest.ini deleted file mode 100644 index 3b174ff089..0000000000 --- a/pytest.ini +++ /dev/null @@ -1,4 +0,0 @@ -[pytest] -addopts = --doctest-glob='doc/*.md' --ignore=setup.py -python_files = sourmash/*.py tests/*.py -norecursedirs = utils build buildenv .tox .asv .eggs diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 0000000000..f4e2d072df --- /dev/null +++ b/setup.cfg @@ -0,0 +1,102 @@ +[metadata] +name = sourmash +description = tools for comparing DNA sequences with MinHash sketches +long_description = file: README.md +long_description_content_type = text/markdown; charset=UTF-8 +url = https://github.com/dib-lab/sourmash +author = C. Titus Brown +author_email = titus@idyll.org +license = BSD 3-clause +license_file = LICENSE +classifiers = + Development Status :: 5 - Production/Stable + Environment :: Console + Environment :: MacOS X + Intended Audience :: Science/Research + License :: OSI Approved :: BSD License + Natural Language :: English + Operating System :: POSIX :: Linux + Operating System :: MacOS :: MacOS X + Programming Language :: Rust + Programming Language :: Python :: 3.7 + Programming Language :: Python :: 3.8 + Topic :: Scientific/Engineering :: Bio-Informatics +project_urls = + Documentation = https://sourmash.readthedocs.io + CI = https://travis-ci.com/dib-lab/sourmash + Source = https://github.com/dib-lab/sourmash + Tracker = https://github.com/dib-lab/sourmash/issues + +[options] +zip_safe = False +packages = find: +platforms = any +include_package_data = True +install_requires = + screed>=1.0 + cffi>=1.14.0 + numpy + matplotlib + scipy + deprecation>=2.0.6 + cachetools>=4,<5 +python_requires = >=3.7 + +[bdist_wheel] +universal = 1 + +[options.packages.find] +where = src + +# for explanation of %(extra)s syntax see: +# https://github.com/pypa/setuptools/issues/1260#issuecomment-438187625 +# this syntax may change in the future +[options.extras_require] +test = + pytest>=6 + pytest-cov<2.6 + recommonmark + hypothesis +demo = + jupyter + jupyter_client + ipython +doc = + sphinx + myst-parser[sphinx]>=0.12.2 + alabaster + sphinxcontrib-napoleon + nbsphinx + ipython +10x = + bam2fasta==1.0.4 +storage = + ipfshttpclient>=0.4.13 + redis +all = + %(test)s + %(demo)s + %(doc)s + %(10x)s + %(storage)s + +[options.entry_points] +console_scripts = + sourmash = sourmash.__main__:main + +[tool:pytest] +addopts = + --doctest-glob='doc/*.md' +norecursedirs = + utils + build + buildenv + .tox + .asv + .eggs +python_files = + src/sourmash/*.py + tests/*.py +testpaths = + tests + doc diff --git a/setup.py b/setup.py index eda4c80037..f416ff3336 100644 --- a/setup.py +++ b/setup.py @@ -1,7 +1,8 @@ import os -from setuptools import setup, find_packages import sys +from setuptools import setup + DEBUG_BUILD = os.environ.get("SOURMASH_DEBUG") == "1" @@ -26,66 +27,7 @@ def build_native(spec): rtld_flags=rtld_flags, ) - -CLASSIFIERS = [ - "Environment :: Console", - "Environment :: MacOS X", - "Intended Audience :: Science/Research", - "License :: OSI Approved :: BSD License", - "Natural Language :: English", - "Operating System :: POSIX :: Linux", - "Operating System :: MacOS :: MacOS X", - "Programming Language :: Rust", - "Programming Language :: Python :: 3.7", - "Programming Language :: Python :: 3.8", - "Topic :: Scientific/Engineering :: Bio-Informatics", -] - -CLASSIFIERS.append("Development Status :: 5 - Production/Stable") - -with open("README.md", "r") as readme: - LONG_DESCRIPTION = readme.read() - -SETUP_METADATA = { - "name": "sourmash", - "description": "tools for comparing DNA sequences with MinHash sketches", - "long_description": LONG_DESCRIPTION, - "long_description_content_type": "text/markdown", - "url": "https://github.com/dib-lab/sourmash", - "author": "C. Titus Brown", - "author_email": "titus@idyll.org", - "license": "BSD 3-clause", - "packages": find_packages(exclude=["tests", "benchmarks"]), - "entry_points": {'console_scripts': [ - 'sourmash = sourmash.__main__:main' - ] - }, - "install_requires": ['screed>=0.9', 'cffi>=1.14.0', 'numpy', - 'matplotlib', 'scipy', 'deprecation>=2.0.6', - 'cachetools >=4,<5'], - "setup_requires": [ - "setuptools>=38.6.0", - "milksnake", - "setuptools_scm>=3.2.0", - "setuptools_scm_git_archive", - ], - "use_scm_version": { - "write_to": "sourmash/version.py", - "git_describe_command": "git describe --dirty --tags --long --match v* --first-parent" - }, - "zip_safe": False, - "platforms": "any", - "extras_require": { - 'test' : ['pytest', 'pytest-cov', 'recommonmark', 'hypothesis'], - 'demo' : ['jupyter', 'jupyter_client', 'ipython'], - 'doc' : ['sphinx', 'myst-parser[sphinx]>=0.12.2', 'alabaster', - "sphinxcontrib-napoleon", "nbsphinx", - "ipython"], - 'storage': ["ipfshttpclient>=0.4.13", "redis"] - }, - "include_package_data": True, - "classifiers": CLASSIFIERS, - "milksnake_tasks": [build_native], -} - -setup(**SETUP_METADATA) +setup( + milksnake_tasks=[build_native], + package_dir={"": "src"}, +) diff --git a/sourmash/__init__.py b/src/sourmash/__init__.py similarity index 100% rename from sourmash/__init__.py rename to src/sourmash/__init__.py diff --git a/sourmash/__main__.py b/src/sourmash/__main__.py similarity index 100% rename from sourmash/__main__.py rename to src/sourmash/__main__.py diff --git a/sourmash/cli/.gitignore b/src/sourmash/cli/.gitignore similarity index 100% rename from sourmash/cli/.gitignore rename to src/sourmash/cli/.gitignore diff --git a/sourmash/cli/__init__.py b/src/sourmash/cli/__init__.py similarity index 100% rename from sourmash/cli/__init__.py rename to src/sourmash/cli/__init__.py diff --git a/sourmash/cli/categorize.py b/src/sourmash/cli/categorize.py similarity index 100% rename from sourmash/cli/categorize.py rename to src/sourmash/cli/categorize.py diff --git a/sourmash/cli/compare.py b/src/sourmash/cli/compare.py similarity index 100% rename from sourmash/cli/compare.py rename to src/sourmash/cli/compare.py diff --git a/sourmash/cli/compute.py b/src/sourmash/cli/compute.py similarity index 100% rename from sourmash/cli/compute.py rename to src/sourmash/cli/compute.py diff --git a/sourmash/cli/gather.py b/src/sourmash/cli/gather.py similarity index 100% rename from sourmash/cli/gather.py rename to src/sourmash/cli/gather.py diff --git a/sourmash/cli/import_csv.py b/src/sourmash/cli/import_csv.py similarity index 100% rename from sourmash/cli/import_csv.py rename to src/sourmash/cli/import_csv.py diff --git a/sourmash/cli/index.py b/src/sourmash/cli/index.py similarity index 100% rename from sourmash/cli/index.py rename to src/sourmash/cli/index.py diff --git a/sourmash/cli/info.py b/src/sourmash/cli/info.py similarity index 100% rename from sourmash/cli/info.py rename to src/sourmash/cli/info.py diff --git a/sourmash/cli/lca/__init__.py b/src/sourmash/cli/lca/__init__.py similarity index 100% rename from sourmash/cli/lca/__init__.py rename to src/sourmash/cli/lca/__init__.py diff --git a/sourmash/cli/lca/classify.py b/src/sourmash/cli/lca/classify.py similarity index 100% rename from sourmash/cli/lca/classify.py rename to src/sourmash/cli/lca/classify.py diff --git a/sourmash/cli/lca/compare_csv.py b/src/sourmash/cli/lca/compare_csv.py similarity index 100% rename from sourmash/cli/lca/compare_csv.py rename to src/sourmash/cli/lca/compare_csv.py diff --git a/sourmash/cli/lca/gather.py b/src/sourmash/cli/lca/gather.py similarity index 100% rename from sourmash/cli/lca/gather.py rename to src/sourmash/cli/lca/gather.py diff --git a/sourmash/cli/lca/index.py b/src/sourmash/cli/lca/index.py similarity index 100% rename from sourmash/cli/lca/index.py rename to src/sourmash/cli/lca/index.py diff --git a/sourmash/cli/lca/rankinfo.py b/src/sourmash/cli/lca/rankinfo.py similarity index 100% rename from sourmash/cli/lca/rankinfo.py rename to src/sourmash/cli/lca/rankinfo.py diff --git a/sourmash/cli/lca/summarize.py b/src/sourmash/cli/lca/summarize.py similarity index 100% rename from sourmash/cli/lca/summarize.py rename to src/sourmash/cli/lca/summarize.py diff --git a/sourmash/cli/migrate.py b/src/sourmash/cli/migrate.py similarity index 100% rename from sourmash/cli/migrate.py rename to src/sourmash/cli/migrate.py diff --git a/sourmash/cli/multigather.py b/src/sourmash/cli/multigather.py similarity index 100% rename from sourmash/cli/multigather.py rename to src/sourmash/cli/multigather.py diff --git a/sourmash/cli/plot.py b/src/sourmash/cli/plot.py similarity index 100% rename from sourmash/cli/plot.py rename to src/sourmash/cli/plot.py diff --git a/sourmash/cli/sbt_combine.py b/src/sourmash/cli/sbt_combine.py similarity index 100% rename from sourmash/cli/sbt_combine.py rename to src/sourmash/cli/sbt_combine.py diff --git a/sourmash/cli/search.py b/src/sourmash/cli/search.py similarity index 100% rename from sourmash/cli/search.py rename to src/sourmash/cli/search.py diff --git a/sourmash/cli/sig/__init__.py b/src/sourmash/cli/sig/__init__.py similarity index 100% rename from sourmash/cli/sig/__init__.py rename to src/sourmash/cli/sig/__init__.py diff --git a/sourmash/cli/sig/cat.py b/src/sourmash/cli/sig/cat.py similarity index 100% rename from sourmash/cli/sig/cat.py rename to src/sourmash/cli/sig/cat.py diff --git a/sourmash/cli/sig/describe.py b/src/sourmash/cli/sig/describe.py similarity index 100% rename from sourmash/cli/sig/describe.py rename to src/sourmash/cli/sig/describe.py diff --git a/sourmash/cli/sig/downsample.py b/src/sourmash/cli/sig/downsample.py similarity index 100% rename from sourmash/cli/sig/downsample.py rename to src/sourmash/cli/sig/downsample.py diff --git a/sourmash/cli/sig/export.py b/src/sourmash/cli/sig/export.py similarity index 100% rename from sourmash/cli/sig/export.py rename to src/sourmash/cli/sig/export.py diff --git a/sourmash/cli/sig/extract.py b/src/sourmash/cli/sig/extract.py similarity index 100% rename from sourmash/cli/sig/extract.py rename to src/sourmash/cli/sig/extract.py diff --git a/sourmash/cli/sig/filter.py b/src/sourmash/cli/sig/filter.py similarity index 100% rename from sourmash/cli/sig/filter.py rename to src/sourmash/cli/sig/filter.py diff --git a/sourmash/cli/sig/flatten.py b/src/sourmash/cli/sig/flatten.py similarity index 100% rename from sourmash/cli/sig/flatten.py rename to src/sourmash/cli/sig/flatten.py diff --git a/sourmash/cli/sig/ingest.py b/src/sourmash/cli/sig/ingest.py similarity index 100% rename from sourmash/cli/sig/ingest.py rename to src/sourmash/cli/sig/ingest.py diff --git a/sourmash/cli/sig/intersect.py b/src/sourmash/cli/sig/intersect.py similarity index 100% rename from sourmash/cli/sig/intersect.py rename to src/sourmash/cli/sig/intersect.py diff --git a/sourmash/cli/sig/merge.py b/src/sourmash/cli/sig/merge.py similarity index 100% rename from sourmash/cli/sig/merge.py rename to src/sourmash/cli/sig/merge.py diff --git a/sourmash/cli/sig/overlap.py b/src/sourmash/cli/sig/overlap.py similarity index 100% rename from sourmash/cli/sig/overlap.py rename to src/sourmash/cli/sig/overlap.py diff --git a/sourmash/cli/sig/rename.py b/src/sourmash/cli/sig/rename.py similarity index 100% rename from sourmash/cli/sig/rename.py rename to src/sourmash/cli/sig/rename.py diff --git a/sourmash/cli/sig/split.py b/src/sourmash/cli/sig/split.py similarity index 100% rename from sourmash/cli/sig/split.py rename to src/sourmash/cli/sig/split.py diff --git a/sourmash/cli/sig/subtract.py b/src/sourmash/cli/sig/subtract.py similarity index 100% rename from sourmash/cli/sig/subtract.py rename to src/sourmash/cli/sig/subtract.py diff --git a/sourmash/cli/sketch/__init__.py b/src/sourmash/cli/sketch/__init__.py similarity index 100% rename from sourmash/cli/sketch/__init__.py rename to src/sourmash/cli/sketch/__init__.py diff --git a/sourmash/cli/sketch/dna.py b/src/sourmash/cli/sketch/dna.py similarity index 100% rename from sourmash/cli/sketch/dna.py rename to src/sourmash/cli/sketch/dna.py diff --git a/sourmash/cli/sketch/protein.py b/src/sourmash/cli/sketch/protein.py similarity index 100% rename from sourmash/cli/sketch/protein.py rename to src/sourmash/cli/sketch/protein.py diff --git a/sourmash/cli/sketch/translate.py b/src/sourmash/cli/sketch/translate.py similarity index 100% rename from sourmash/cli/sketch/translate.py rename to src/sourmash/cli/sketch/translate.py diff --git a/sourmash/cli/storage/__init__.py b/src/sourmash/cli/storage/__init__.py similarity index 100% rename from sourmash/cli/storage/__init__.py rename to src/sourmash/cli/storage/__init__.py diff --git a/sourmash/cli/storage/convert.py b/src/sourmash/cli/storage/convert.py similarity index 100% rename from sourmash/cli/storage/convert.py rename to src/sourmash/cli/storage/convert.py diff --git a/sourmash/cli/utils.py b/src/sourmash/cli/utils.py similarity index 100% rename from sourmash/cli/utils.py rename to src/sourmash/cli/utils.py diff --git a/sourmash/cli/watch.py b/src/sourmash/cli/watch.py similarity index 100% rename from sourmash/cli/watch.py rename to src/sourmash/cli/watch.py diff --git a/sourmash/command_compute.py b/src/sourmash/command_compute.py similarity index 100% rename from sourmash/command_compute.py rename to src/sourmash/command_compute.py diff --git a/sourmash/command_sketch.py b/src/sourmash/command_sketch.py similarity index 100% rename from sourmash/command_sketch.py rename to src/sourmash/command_sketch.py diff --git a/sourmash/commands.py b/src/sourmash/commands.py similarity index 100% rename from sourmash/commands.py rename to src/sourmash/commands.py diff --git a/sourmash/compare.py b/src/sourmash/compare.py similarity index 100% rename from sourmash/compare.py rename to src/sourmash/compare.py diff --git a/sourmash/exceptions.py b/src/sourmash/exceptions.py similarity index 100% rename from sourmash/exceptions.py rename to src/sourmash/exceptions.py diff --git a/sourmash/fig.py b/src/sourmash/fig.py similarity index 100% rename from sourmash/fig.py rename to src/sourmash/fig.py diff --git a/sourmash/hll.py b/src/sourmash/hll.py similarity index 100% rename from sourmash/hll.py rename to src/sourmash/hll.py diff --git a/sourmash/index.py b/src/sourmash/index.py similarity index 100% rename from sourmash/index.py rename to src/sourmash/index.py diff --git a/sourmash/lca/__init__.py b/src/sourmash/lca/__init__.py similarity index 100% rename from sourmash/lca/__init__.py rename to src/sourmash/lca/__init__.py diff --git a/sourmash/lca/__main__.py b/src/sourmash/lca/__main__.py similarity index 100% rename from sourmash/lca/__main__.py rename to src/sourmash/lca/__main__.py diff --git a/sourmash/lca/command_classify.py b/src/sourmash/lca/command_classify.py similarity index 100% rename from sourmash/lca/command_classify.py rename to src/sourmash/lca/command_classify.py diff --git a/sourmash/lca/command_compare_csv.py b/src/sourmash/lca/command_compare_csv.py similarity index 100% rename from sourmash/lca/command_compare_csv.py rename to src/sourmash/lca/command_compare_csv.py diff --git a/sourmash/lca/command_gather.py b/src/sourmash/lca/command_gather.py similarity index 100% rename from sourmash/lca/command_gather.py rename to src/sourmash/lca/command_gather.py diff --git a/sourmash/lca/command_index.py b/src/sourmash/lca/command_index.py similarity index 100% rename from sourmash/lca/command_index.py rename to src/sourmash/lca/command_index.py diff --git a/sourmash/lca/command_rankinfo.py b/src/sourmash/lca/command_rankinfo.py similarity index 100% rename from sourmash/lca/command_rankinfo.py rename to src/sourmash/lca/command_rankinfo.py diff --git a/sourmash/lca/command_summarize.py b/src/sourmash/lca/command_summarize.py similarity index 100% rename from sourmash/lca/command_summarize.py rename to src/sourmash/lca/command_summarize.py diff --git a/sourmash/lca/lca_db.py b/src/sourmash/lca/lca_db.py similarity index 100% rename from sourmash/lca/lca_db.py rename to src/sourmash/lca/lca_db.py diff --git a/sourmash/lca/lca_utils.py b/src/sourmash/lca/lca_utils.py similarity index 100% rename from sourmash/lca/lca_utils.py rename to src/sourmash/lca/lca_utils.py diff --git a/sourmash/logging.py b/src/sourmash/logging.py similarity index 100% rename from sourmash/logging.py rename to src/sourmash/logging.py diff --git a/sourmash/minhash.py b/src/sourmash/minhash.py similarity index 100% rename from sourmash/minhash.py rename to src/sourmash/minhash.py diff --git a/sourmash/nodegraph.py b/src/sourmash/nodegraph.py similarity index 100% rename from sourmash/nodegraph.py rename to src/sourmash/nodegraph.py diff --git a/sourmash/np_utils.py b/src/sourmash/np_utils.py similarity index 100% rename from sourmash/np_utils.py rename to src/sourmash/np_utils.py diff --git a/sourmash/sbt.py b/src/sourmash/sbt.py similarity index 100% rename from sourmash/sbt.py rename to src/sourmash/sbt.py diff --git a/sourmash/sbt_storage.py b/src/sourmash/sbt_storage.py similarity index 100% rename from sourmash/sbt_storage.py rename to src/sourmash/sbt_storage.py diff --git a/sourmash/sbtmh.py b/src/sourmash/sbtmh.py similarity index 100% rename from sourmash/sbtmh.py rename to src/sourmash/sbtmh.py diff --git a/sourmash/search.py b/src/sourmash/search.py similarity index 100% rename from sourmash/search.py rename to src/sourmash/search.py diff --git a/sourmash/sig/__init__.py b/src/sourmash/sig/__init__.py similarity index 100% rename from sourmash/sig/__init__.py rename to src/sourmash/sig/__init__.py diff --git a/sourmash/sig/__main__.py b/src/sourmash/sig/__main__.py similarity index 100% rename from sourmash/sig/__main__.py rename to src/sourmash/sig/__main__.py diff --git a/sourmash/signature.py b/src/sourmash/signature.py similarity index 100% rename from sourmash/signature.py rename to src/sourmash/signature.py diff --git a/sourmash/sourmash_args.py b/src/sourmash/sourmash_args.py similarity index 100% rename from sourmash/sourmash_args.py rename to src/sourmash/sourmash_args.py diff --git a/sourmash/utils.py b/src/sourmash/utils.py similarity index 100% rename from sourmash/utils.py rename to src/sourmash/utils.py diff --git a/tests/__init__.py b/tests/__init__.py deleted file mode 100644 index 1019c178fc..0000000000 --- a/tests/__init__.py +++ /dev/null @@ -1,2 +0,0 @@ -import matplotlib.pyplot as plt -plt.rcParams.update({'figure.max_open_warning': 0}) diff --git a/tests/conftest.py b/tests/conftest.py index 06bd835ff5..406ca43cb4 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,5 +1,8 @@ import os +import matplotlib.pyplot as plt +plt.rcParams.update({'figure.max_open_warning': 0}) + from hypothesis import settings, Verbosity import pytest diff --git a/tests/test__minhash.py b/tests/test__minhash.py index dbad4bba6b..96b26e8e90 100644 --- a/tests/test__minhash.py +++ b/tests/test__minhash.py @@ -49,7 +49,7 @@ ) from sourmash import signature -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils # add: # * get default params from Python diff --git a/tests/test_api.py b/tests/test_api.py index fd8a66cabd..8be3396085 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -1,7 +1,7 @@ import pytest import sourmash -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils def test_sourmash_signature_api(): diff --git a/tests/test_bugs.py b/tests/test_bugs.py index 20608f0a0d..4870f0faf3 100644 --- a/tests/test_bugs.py +++ b/tests/test_bugs.py @@ -1,4 +1,4 @@ -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils def test_bug_781(): with utils.TempDirectory() as location: diff --git a/tests/test_cmd_signature.py b/tests/test_cmd_signature.py index a1925a9d76..a65270ab0c 100644 --- a/tests/test_cmd_signature.py +++ b/tests/test_cmd_signature.py @@ -7,7 +7,7 @@ import pytest -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils import sourmash ## command line tests diff --git a/tests/test_compare.py b/tests/test_compare.py index 9555993c6b..005c96c575 100644 --- a/tests/test_compare.py +++ b/tests/test_compare.py @@ -6,7 +6,7 @@ import sourmash from sourmash.compare import compare_all_pairs, compare_parallel, compare_serial -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils @pytest.fixture() diff --git a/tests/test_deprecated.py b/tests/test_deprecated.py index 9e69f50694..fdd9acc53c 100644 --- a/tests/test_deprecated.py +++ b/tests/test_deprecated.py @@ -1,5 +1,5 @@ from sourmash import signature -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils def test_load_textmode(track_abundance): # ijson required a file in binary mode or bytes, diff --git a/tests/test_hll.py b/tests/test_hll.py index 19491ceec2..4e4ce365dd 100644 --- a/tests/test_hll.py +++ b/tests/test_hll.py @@ -6,7 +6,7 @@ from sourmash.hll import HLL -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils K = 21 # size of kmer ERR_RATE = 0.01 diff --git a/tests/test_index.py b/tests/test_index.py index 5933004c12..7174db282d 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -6,7 +6,8 @@ from sourmash import load_one_signature, SourmashSignature from sourmash.index import LinearIndex from sourmash.sbt import SBT, GraphFactory, Leaf -from . import sourmash_tst_utils as utils + +import sourmash_tst_utils as utils def test_simple_index(n_children): diff --git a/tests/test_jaccard.py b/tests/test_jaccard.py index 475f73a482..709f71b3aa 100644 --- a/tests/test_jaccard.py +++ b/tests/test_jaccard.py @@ -5,7 +5,8 @@ import pytest from sourmash import MinHash -from . import sourmash_tst_utils as utils + +import sourmash_tst_utils as utils # below, 'track_abundance' is toggled to both True and False by py.test -- # see conftest.py. diff --git a/tests/test_lca.py b/tests/test_lca.py index e530588197..2226c473ae 100644 --- a/tests/test_lca.py +++ b/tests/test_lca.py @@ -7,7 +7,7 @@ import pytest import glob -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils import sourmash from sourmash import load_one_signature, SourmashSignature diff --git a/tests/test_nodegraph.py b/tests/test_nodegraph.py index 94750c301e..68283dd620 100644 --- a/tests/test_nodegraph.py +++ b/tests/test_nodegraph.py @@ -4,7 +4,7 @@ from sourmash.nodegraph import Nodegraph, extract_nodegraph_info, calc_expected_collisions -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils def test_nodegraph_to_khmer_basic(): diff --git a/tests/test_sbt.py b/tests/test_sbt.py index 4d7bb51db6..2ae5f6c5e6 100644 --- a/tests/test_sbt.py +++ b/tests/test_sbt.py @@ -13,7 +13,7 @@ from sourmash.sbt_storage import (FSStorage, RedisStorage, IPFSStorage, ZipStorage) -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils def test_simple(n_children): diff --git a/tests/test_signature.py b/tests/test_signature.py index e3054d76e5..18654ce8a0 100644 --- a/tests/test_signature.py +++ b/tests/test_signature.py @@ -5,7 +5,7 @@ import sourmash from sourmash.signature import SourmashSignature, save_signatures, \ load_signatures, load_one_signature -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils def test_compare(track_abundance): diff --git a/tests/test_sourmash.py b/tests/test_sourmash.py index 6cde335e23..331a53da5d 100644 --- a/tests/test_sourmash.py +++ b/tests/test_sourmash.py @@ -12,7 +12,8 @@ import sys import zipfile -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils + import sourmash from sourmash import MinHash from sourmash.sbt import SBT, Node diff --git a/tests/test_sourmash_compute.py b/tests/test_sourmash_compute.py index 659d83b020..44210fcf5e 100644 --- a/tests/test_sourmash_compute.py +++ b/tests/test_sourmash_compute.py @@ -10,7 +10,8 @@ import csv import pytest -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils + import sourmash from sourmash import MinHash from sourmash.sbt import SBT, Node diff --git a/tests/test_sourmash_sketch.py b/tests/test_sourmash_sketch.py index e13ac5db8e..4eca0894ea 100644 --- a/tests/test_sourmash_sketch.py +++ b/tests/test_sourmash_sketch.py @@ -11,7 +11,7 @@ import pytest import screed -from . import sourmash_tst_utils as utils +import sourmash_tst_utils as utils import sourmash from sourmash import MinHash from sourmash.sbt import SBT, Node diff --git a/tox.ini b/tox.ini index b859ff129f..dfa82b59f1 100644 --- a/tox.ini +++ b/tox.ini @@ -1,28 +1,167 @@ [tox] -envlist=py39,py38,py37,docs +envlist = + py39, + coverage, + docs, + package_description + py38, + py37, +# fix_lint, +minversion = 3.12 +isolated_build = true +skip_missing_interpreters = true [testenv] -passenv = CI TRAVIS TRAVIS_* PYTHONDEVMODE -whitelist_externals= - make -extras = - test - storage -deps= - pip>=19.3.1 - codecov -commands= - make coverage +description = run the tests with pytest under {basepython} +setenv = + PIP_DISABLE_VERSION_CHECK = 1 + COVERAGE_FILE = {toxworkdir}/.coverage.{envname} + VIRTUALENV_NO_DOWNLOAD = 1 + PIP_EXTRA_INDEX_URL = https://antocuni.github.io/pypy-wheels/manylinux2010 +passenv = + CURL_CA_BUNDLE + http_proxy + https_proxy + no_proxy + REQUESTS_CA_BUNDLE + SSL_CERT_FILE + PYTEST_* + PIP_CACHE_DIR + CI + TRAVIS + TRAVIS_* + PYTHONDEVMODE +deps = + pip>=19.3.1 +extras = + test + storage +changedir = tests +commands = pytest \ + --cov "{envsitepackagesdir}/sourmash" \ + --cov-config "{toxinidir}/tox.ini" \ + --junitxml {toxworkdir}/junit.{envname}.xml \ + {posargs:.} + +[testenv:pypy] +deps = + pip >= 19.3.1 + psutil <= 5.6.7 [testenv:docs] -basepython = python3.7 +description = invoke sphinx-build to build the HTML docs +basepython = python3.8 extras = doc +whitelist_externals = pandoc passenv = HOME +changedir = {toxinidir} +#commands = sphinx-build -d "{toxworkdir}/docs_doctree" doc "{toxworkdir}/docs_out" --color -W -bhtml {posargs} commands = sphinx-build -d "{toxworkdir}/docs_doctree" doc "{toxworkdir}/docs_out" --color -bhtml {posargs} python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))' +[testenv:package_description] +description = check that the long description is valid +basepython = python3.8 +deps = twine >= 1.12.1 + # TODO installing readme-renderer[md] should not be necessary + readme-renderer[md] >= 24.0 + pip >= 19.1 +skip_install = true +changedir = {toxinidir} +extras = +commands = pip wheel -w {envtmpdir}/build --no-deps . + twine check {envtmpdir}/build/* + +[testenv:fix_lint] +description = format the code base to adhere to our styles, and complain about what we cannot do automatically +basepython = python3.8 +passenv = {[testenv]passenv} + # without PROGRAMDATA cloning using git for Windows will fail with an `error setting certificate verify locations` error + PROGRAMDATA + PRE_COMMIT_HOME +extras = lint +deps = pre-commit>=2 +skip_install = True +commands = pre-commit run --all-files --show-diff-on-failure {posargs} + python -c 'import pathlib; print("hint: run \{\} install to add checks as pre-commit hook".format(pathlib.Path(r"{envdir}") / "bin" / "pre-commit"))' + +[testenv:coverage] +description = [run locally after tests]: combine coverage data and create report; + generates a diff coverage against origin/latest (can be changed by setting DIFF_AGAINST env var) +deps = {[testenv]deps} + coverage >= 5.0.1 + diff_cover +skip_install = True +passenv = {[testenv]passenv} + DIFF_AGAINST +setenv = COVERAGE_FILE={toxworkdir}/.coverage +commands = coverage combine + coverage report -i -m + coverage xml -i -o {toxworkdir}/coverage.xml + coverage html -i -d {toxworkdir}/htmlcov + diff-cover --compare-branch {env:DIFF_AGAINST:origin/latest} {toxworkdir}/coverage.xml +depends = py39, py38, py37, pypy3 +parallel_show_output = True + +[testenv:codecov] +description = [only run on CI]: upload coverage data to codecov (depends on coverage running first) +passenv = {[testenv]passenv} + CODECOV_TOKEN +deps = codecov +skip_install = True +changedir = {toxinidir} +depends = coverage +commands = codecov --file "{toxworkdir}/coverage.xml" {posargs} + +[testenv:X] +description = print the positional arguments passed in with echo +commands = echo {posargs} + +[coverage:run] +branch = true +parallel = true + +[coverage:report] +skip_covered = True +show_missing = True +exclude_lines = + \#\s*pragma: no cover + ^\s*raise AssertionError\b + ^\s*raise NotImplementedError\b + ^\s*return NotImplemented\b + ^\s*raise$ + ^if __name__ == ['"]__main__['"]:$ + +[coverage:paths] +source = src/sourmash/ + */.tox/*/lib/python*/site-packages/sourmash + */.tox/pypy*/site-packages/sourmash + */.tox\*\Lib\site-packages\sourmash + */src/sourmash + *\src\sourmash + [gh-actions] python = - 3.7: py37,docs - 3.8: py38 - 3.9: py39 + 3.7: py37, docs, package_description, coverage, codecov + 3.8: py38, coverage, codecov + 3.9: py39, coverage, codecov + +[flake8] +max-complexity = 22 +max-line-length = 99 +ignore = E203, W503, C901, E402, B011 + +[pep8] +max-line-length = 99 + +[testenv:dev] +description = dev environment with all deps at {envdir} +extras = + test + 10x + storage + doc +deps = {[testenv]deps} +usedevelop = True +commands = python -m pip list --format=columns + python -c "print(r'{envpython}')"