Skip to content

Commit

Permalink
If no tests then Pytest should exit with 0 (not 5)
Browse files Browse the repository at this point in the history
- Add a 'test' target to run pytest against the 'tests' directory. If an
  exit code of 5 is found then exit with a 0 instead. Otherwise exit
  with the appropriate error code.

- Intercept 'pytest_sessionfinish' to do the same thing in conftest.py
  [1].

- Switch off 'disallowed_untyped_defs' in mypy.ini as hints should help
  readability where it makes sense (hence 'hints'). It should not be enforced
  all the time IMHO.

[1] pytest-dev/pytest#2393
  • Loading branch information
webventurer committed Jan 13, 2024
1 parent e435c93 commit f8c4fb5
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ jobs:
- name: Check types
run: make types
- name: Run tests
run: pytest -o console_output_style=classic -v
run: pytest -o console_output_style=classic -v || ([ $? = 5 ] && exit 0 || exit $?)
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,7 @@ format:
.PHONY: types
types:
mypy .

.PHONY: test
test:
sh -c 'pytest tests || ([ $$? = 5 ] && exit 0 || exit $$?)'
2 changes: 1 addition & 1 deletion test
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
set -e

make check
pytest "$@"
make test
7 changes: 7 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
NO_TESTS_COLLECTED = 5
SUCCESS = 0


def pytest_sessionfinish(session, exitstatus):
if exitstatus == NO_TESTS_COLLECTED and session.config.getoption("-k"):
session.exitstatus = SUCCESS

1 comment on commit f8c4fb5

@gma
Copy link

@gma gma commented on f8c4fb5 Jan 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This exit code 5 thing is odd isn't it. I see somebody made a pytest plugin that gives it a new command line option so it exits with 0 rather than 5, but that feels a bit brittle/likely to need updating in future. I've not looked at it though — I wonder if it does the same thing that your code in conftest.py does?

My main reason for commenting is that I wanted to point out a feature of my original version that I benefit from. The ./test script passed all its command line arguments through to pytest. That means I can do things like ./test -k blah to select some by name, or ./test --lf -x to re-run just the one that failed. And I still get the linting and type checks executed first.

That's why I didn't have a target for testing in Makefile. I only used make in the first place because it gave me an easy way of running each of the tools separately in CI. The fact that we're not really using any of the power of make (but have introduced it as a dependency) offends me slightly, and always has. Is it better than writing a more complicated shell script? 🤷

Please sign in to comment.