Skip to content

Commit

Permalink
Introduce tox to automate tasks (#78)
Browse files Browse the repository at this point in the history
* Introduce tox.ini for project automation

* Update documentation
  • Loading branch information
hajimes authored Sep 15, 2024
1 parent d39ef3a commit cac73ac
Show file tree
Hide file tree
Showing 5 changed files with 126 additions and 23 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
run: |
pip install --upgrade pip
pip install .
pip install ".[test]"
pip install ".[test,type]"
- name: Test with pytest
run: python -m pytest
- name: Test type hints with mypy
Expand Down
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,18 @@ ENV/
# mkdocs documentation
/site

# pytest
.pytest_cache/

# mypy
.mypy_cache/

# macOS
.DS_Store

# vscode
.vscode/
.vscode/

# Directory from an external source created by git submodule update --init
# Adding this path is useful for other tools like markdwonlint and prettier
util/smhasher/
70 changes: 51 additions & 19 deletions docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,17 +43,53 @@ As of version 4.2.0, the project layout is structured as follows:
- `docs`: project documentation directory
- `.github/workflows`: GitHub Actions workflows

## Testing
## Project setup

Run:

```shell
git clone https://github.com/hajimes/mmh3.git
```

This project uses `tox` to automate testing and other tasks. You can install
`tox` by running:

```shell
pipx install tox
```

In addition, `npx` (included with `npm` >= 5.2.0) is required within the `tox`
environments to run linters.

## Testing and linting

Before submitting your changes, make sure to run the project's tests to ensure
that everything is working as expected.
everything is working as expected.

Try
To run all tests, use the following command:

```shell
pip install ".[test]"
pytest
mypy --strict tests
tox
```

During development, you can run the tests for a specific environment by
specifying the environment name. For example, to run tests for a specific
version of Python (e.g., Python 3.12), use:

```shell
tox -e py312
```

For type checking, run:

```shell
tox -e type
```

To run linters with automated formatting, use:

```shell
tox -e lint
```

### (Optional) Testing on s390x
Expand Down Expand Up @@ -83,18 +119,15 @@ The `util` directory contains C files that were generated from the
The idea of the subproject directory loosely follows the
[`hashlib` implementation of CPython](https://github.com/python/cpython/tree/main/Modules/_hacl).

### Updating mmh3 C code

Try `git submodule update --init` to fetch Appleby's original SMHasher project
as a git submodule. Then, run `python util/refresh.py` to generate PEP
7-compliant C code from the original project, instead of editing `murmurhash3.*`
files manually.
### Updating mmh3 core C code

To perform further edits, add transformation code to the `refresh.py` script.
Then, run `refresh.py` again to update the `murmurhash3.*` files.
Run `tox -e build-cfiles`. This will fetch Appleby's original SMHasher project
as a git submodule and then generate PEP 7-compliant C code from the original
project.

After file generation, use `clang-format` to format the generated code. Try
`clang-format -i src/mmh3/*.{c,h}` from the project's top-level directory.
To perform further edits, add transformation code to the `refresh.py` script,
instead of editing `murmurhash3.*` files manually.
Then, run `tox -e build-cfiles` again to update the `murmurhash3.*` files.

### Local files

Expand Down Expand Up @@ -141,7 +174,7 @@ After obtaining the benchmark results, you can plot graphs by `plot_graph.py`.
The following is an example of how to run the script:

```shell
pip install ".[benchmark]" ".[plot]"
pip install ".[benchmark,plot]"
python benchmark/plot_graph.py --output-dir docs/_static RESULT_DIR/*.json
```

Expand All @@ -158,8 +191,7 @@ located in the `docs`. The documentation is automatically built and
To build the documentation locally, use the following command:

```shell
pip install ".[docs]"
make -C docs html
tox -e docs
```

To check the result of the built documentation, open
Expand Down
14 changes: 12 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,17 @@ classifiers = [

[project.optional-dependencies]
test = [
"mypy == 1.11.2",
"pytest == 8.3.2"
"pytest == 8.3.2",
"pytest-sugar == 1.0.0"
]
lint = [
"black == 24.8.0",
"clang-format == 18.1.8",
"isort == 5.13.2",
"pylint == 3.2.7"
]
type = [
"mypy == 1.11.2"
]
docs = [
"myst-parser == 4.0.0",
Expand Down Expand Up @@ -64,6 +73,7 @@ ignore-paths = [
"^build",
"^venv",
"^.venv",
"^.tox",
"^src/mmh3/__init__.pyi"
]
# Use multiple processes to speed up Pylint.
Expand Down
54 changes: 54 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
[tox]
requires =
tox>=4
envlist = lint, type, py{38,39,310,311,312}

[testenv]
description = run unit tests
commands_pre =
pip install ".[test]"
commands =
pytest

[testenv:lint]
description = run linters with formatting
allowlist_externals =
find
npx
commands_pre =
pip install ".[lint]"
commands =
black .
isort .
find ./src/mmh3 -name '*.[ch]' -exec clang-format -i {} +
npx prettier --write .
pylint --recursive=y .
npx markdownlint --config .markdown-lint.yml \
--ignore-path .gitignore **/*.md

[testenv:type]
description = run type checks
commands_pre =
pip install ".[type]"
commands =
mypy --strict tests

[testenv:docs]
description = run documentation build
allowlist_externals =
make
commands_pre =
pip install ".[docs]"
commands =
make -C docs html

[testenv:build-cfiles]
allowlist_externals =
find
git
commands_pre =
pip install ".[lint]"
commands =
git submodule update --init
python util/refresh.py
find ./src/mmh3 -name '*.[ch]' -exec clang-format -i {} +

0 comments on commit cac73ac

Please sign in to comment.