Release pytest 8.0.0rc1 (2023-12-30) · pytest-dev/pytest

See https://docs.pytest.org/en/latest/changelog.html#pytest-8-0-0rc1-2023-12-30 for the rendered changelog.

Breaking Changes

Old Deprecations Are Now Errors

#7363: PytestRemovedIn8Warning deprecation warnings are now errors by default.

Following our plan to remove deprecated features with as little disruption as possible, all warnings of type PytestRemovedIn8Warning now generate errors instead of warning messages by default.

The affected features will be effectively removed in pytest 8.1, so please consult the deprecations{.interpreted-text role="ref"} section in the docs for directions on how to update existing code.

In the pytest 8.0.X series, it is possible to change the errors back into warnings as a stopgap measure by adding this to your pytest.ini file:
```
[pytest]
filterwarnings =
    ignore::pytest.PytestRemovedIn8Warning
```
But this will stop working when pytest 8.1 is released.

If you have concerns about the removal of a specific feature, please add a comment to 7363{.interpreted-text role="issue"}.

Version Compatibility

#11151: Dropped support for Python 3.7, which reached end-of-life on 2023-06-27.
pluggy>=1.3.0 is now required.

Collection Changes

In this version we've made several breaking changes to pytest's collection phase, particularly around how filesystem directories and Python packages are collected, fixing deficiencies and allowing for cleanups and improvements to pytest's internals. A deprecation period for these changes was not possible.

#7777: Files and directories are now collected in alphabetical order jointly, unless changed by a plugin.
Previously, files were collected before directories.
See below for an example.
#8976: Running [pytest pkg/__init__.py]{.title-ref} now collects the [pkg/__init__.py]{.title-ref} file (module) only.
Previously, it collected the entire [pkg]{.title-ref} package, including other test files in the directory, but excluding tests in the [__init__.py]{.title-ref} file itself
(unless python_files{.interpreted-text role="confval"} was changed to allow [__init__.py]{.title-ref} file).

To collect the entire package, specify just the directory: [pytest pkg]{.title-ref}.
#11137: pytest.Package{.interpreted-text role="class"} is no longer a pytest.Module{.interpreted-text role="class"} or pytest.File{.interpreted-text role="class"}.

The Package collector node designates a Python package, that is, a directory with an [__init__.py]{.title-ref} file.
Previously Package was a subtype of pytest.Module (which represents a single Python module), the module being the [__init__.py]{.title-ref} file.
This has been deemed a design mistake (see 11137{.interpreted-text role="issue"} and 7777{.interpreted-text role="issue"} for details).

The path property of Package nodes now points to the package directory instead of the __init__.py file.

Note that a Module node for __init__.py (which is not a Package) may still exist, if it is picked up during collection (e.g. if you configured python_files{.interpreted-text role="confval"} to include __init__.py files).
#7777: Added a new pytest.Directory{.interpreted-text role="class"} base collection node, which all collector nodes for filesystem directories are expected to subclass.
This is analogous to the existing pytest.File{.interpreted-text role="class"} for file nodes.

Changed pytest.Package{.interpreted-text role="class"} to be a subclass of pytest.Directory{.interpreted-text role="class"}.
A Package represents a filesystem directory which is a Python package,
i.e. contains an __init__.py file.

pytest.Package{.interpreted-text role="class"} now only collects files in its own directory; previously it collected recursively.
Sub-directories are collected as their own collector nodes, which then collect themselves, thus creating a collection tree which mirrors the filesystem hierarchy.

Added a new pytest.Dir{.interpreted-text role="class"} concrete collection node, a subclass of pytest.Directory{.interpreted-text role="class"}.
This node represents a filesystem directory, which is not a pytest.Package{.interpreted-text role="class"}, that is, does not contain an __init__.py file. Similarly to Package, it only collects the files in its own directory.

pytest.Session{.interpreted-text role="class"} now only collects the initial arguments, without recursing into directories.
This work is now done by the recursive expansion process <pytest.Collector.collect>{.interpreted-text role="func"} of directory collector nodes.

session.name <pytest.Session.name>{.interpreted-text role="attr"} is now ""; previously it was the rootdir directory name.
This matches session.nodeid <_pytest.nodes.Node.nodeid>{.interpreted-text role="attr"} which has always been [""]{.title-ref}.

The collection tree now contains directories/packages up to the rootdir <rootdir>{.interpreted-text role="ref"}, for initial arguments that are found within the rootdir.
For files outside the rootdir, only the immediate directory/package is collected --note however that collecting from outside the rootdir is discouraged.

As an example, given the following filesystem tree:
```
myroot/
    pytest.ini
    top/
    ├── aaa
    │   └── test_aaa.py
    ├── test_a.py
    ├── test_b
    │   ├── __init__.py
    │   └── test_b.py
    ├── test_c.py
    └── zzz
        ├── __init__.py
        └── test_zzz.py
```
the collection tree, as shown by [pytest --collect-only top/]{.title-ref} but with the otherwise-hidden ~pytest.Session{.interpreted-text role="class"} node added for clarity, is now the following:
```
<Session>
  <Dir myroot>
    <Dir top>
      <Dir aaa>
        <Module test_aaa.py>
          <Function test_it>
      <Module test_a.py>
        <Function test_it>
      <Package test_b>
        <Module test_b.py>
          <Function test_it>
      <Module test_c.py>
        <Function test_it>
      <Package zzz>
        <Module test_zzz.py>
          <Function test_it>
```
Previously, it was:
```
<Session>
  <Module top/test_a.py>
    <Function test_it>
  <Module top/test_c.py>
    <Function test_it>
  <Module top/aaa/test_aaa.py>
    <Function test_it>
  <Package test_b>
    <Module test_b.py>
      <Function test_it>
  <Package zzz>
    <Module test_zzz.py>
      <Function test_it>
```
Code/plugins which rely on a specific shape of the collection tree might need to update.
#11676: The classes ~_pytest.nodes.Node{.interpreted-text role="class"}, ~pytest.Collector{.interpreted-text role="class"}, ~pytest.Item{.interpreted-text role="class"}, ~pytest.File{.interpreted-text role="class"}, ~_pytest.nodes.FSCollector{.interpreted-text role="class"} are now marked abstract (see abc{.interpreted-text role="mod"}).

We do not expect this change to affect users and plugin authors, it will only cause errors when the code is already wrong or problematic.

Other breaking changes

These are breaking changes where deprecation was not possible.

#11282: Sanitized the handling of the default parameter when defining configuration options.

Previously if default was not supplied for parser.addini <pytest.Parser.addini>{.interpreted-text role="meth"} and the configuration option value was not defined in a test session, then calls to config.getini <pytest.Config.getini>{.interpreted-text role="func"} returned an empty list or an empty string depending on whether type was supplied or not respectively, which is clearly incorrect. Also, None was not honored even if default=None was used explicitly while defining the option.

Now the behavior of parser.addini <pytest.Parser.addini>{.interpreted-text role="meth"} is as follows:
- If default is NOT passed but type is provided, then a type-specific default will be returned. For example type=bool will return False, type=str will return "", etc.
- If default=None is passed and the option is not defined in a test session, then None will be returned, regardless of the type.
- If neither default nor type are provided, assume type=str and return "" as default (this is as per previous behavior).
The team decided to not introduce a deprecation period for this change, as doing so would be complicated both in terms of communicating this to the community as well as implementing it, and also because the team believes this change should not break existing plugins except in rare cases.
#11667: pytest's setup.py file is removed.
If you relied on this file, e.g. to install pytest using setup.py install, please see Why you shouldn't invoke setup.py directly for alternatives.
#9288: ~pytest.warns{.interpreted-text role="func"} now re-emits unmatched warnings when the context closes -- previously it would consume all warnings, hiding those that were not matched by the function.

While this is a new feature, we announce it as a breaking change because many test suites are configured to error-out on warnings, and will
therefore fail on the newly-re-emitted warnings.

Deprecations

#10465: Test functions returning a value other than None will now issue a pytest.PytestWarning{.interpreted-text role="class"} instead of pytest.PytestRemovedIn8Warning{.interpreted-text role="class"}, meaning this will stay a warning instead of becoming an error in the future.
#3664: Applying a mark to a fixture function now issues a warning: marks in fixtures never had any effect, but it is a common user error to apply a mark to a fixture (for example usefixtures) and expect it to work.

This will become an error in pytest 9.0.

Features and Improvements

Improved Diffs

These changes improve the diffs that pytest prints when an assertion fails.
Note that syntax highlighting requires the pygments package.

#11520: The very verbose (-vv) diff output is now colored as a diff instead of a big chunk of red.

Python code in error reports is now syntax-highlighted as Python.

The sections in the error reports are now better separated.
#1531: The very verbose diff (-vv) for every standard library container type is improved. The indentation is now consistent and the markers are on their own separate lines, which should reduce the diffs shown to users.

Previously, the standard Python pretty printer was used to generate the output, which puts opening and closing markers on the same line as the first/last entry, in addition to not having consistent indentation.
#10617: Added more comprehensive set assertion rewrites for comparisons other than equality ==, with the following operations now providing better failure messages: !=, <=, >=, <, and >.

Separate Control For Assertion Verbosity

#11387: Added the new verbosity_assertions{.interpreted-text role="confval"} configuration option for fine-grained control of failed assertions verbosity.

If you've ever wished that pytest always show you full diffs, but without making everything else verbose, this is for you.

See Fine-grained verbosity <pytest.fine_grained_verbosity>{.interpreted-text role="ref"} for more details.

For plugin authors, config.get_verbosity <pytest.Config.get_verbosity>{.interpreted-text role="attr"} can be used to retrieve the verbosity level for a specific verbosity type.

Additional Support For Exception Groups and `notes`

These changes improve pytest's support for exception groups.

#10441: Added ExceptionInfo.group_contains() <pytest.ExceptionInfo.group_contains>{.interpreted-text role="func"}, an assertion helper that tests if an ExceptionGroup{.interpreted-text role="class"} contains a matching exception.

See assert-matching-exception-groups{.interpreted-text role="ref"} for an example.
#11227: Allow pytest.raises{.interpreted-text role="func"} match argument to match against [PEP-678 <https://peps.python.org/pep-0678/>]{.title-ref} __notes__.

Custom Directory collectors

#7777: Added a new hook pytest_collect_directory{.interpreted-text role="hook"},
which is called by filesystem-traversing collector nodes, such as pytest.Session{.interpreted-text role="class"}, pytest.Dir{.interpreted-text role="class"} and pytest.Package{.interpreted-text role="class"}, to create a collector node for a sub-directory.
It is expected to return a subclass of pytest.Directory{.interpreted-text role="class"}.
This hook allows plugins to customize the collection of directories <custom directory collectors>{.interpreted-text role="ref"}.

"New-style" Hook Wrappers

#11122: pytest now uses "new-style" hook wrappers internally, available since pluggy 1.2.0.
See pluggy's 1.2.0 changelog and the updated docs <hookwrapper>{.interpreted-text role="ref"} for details.

Plugins which want to use new-style wrappers can do so if they require pytest>=8.

Other Improvements

#11216: If a test is skipped from inside an xunit setup fixture <classic xunit>{.interpreted-text role="ref"}, the test summary now shows the test location instead of the fixture location.
#11314: Logging to a file using the --log-file option will use --log-level, --log-format and --log-date-format as fallback
if --log-file-level, --log-file-format and --log-file-date-format are not provided respectively.
#11610: Added the LogCaptureFixture.filtering() <pytest.LogCaptureFixture.filtering>{.interpreted-text role="func"} context manager which
adds a given logging.Filter{.interpreted-text role="class"} object to the caplog{.interpreted-text role="fixture"} fixture.
#11447: pytest.deprecated_call{.interpreted-text role="func"} now also considers warnings of type FutureWarning{.interpreted-text role="class"}.
#11600: Improved the documentation and type signature for pytest.mark.xfail <pytest.mark.xfail>{.interpreted-text role="func"}'s condition param to use False as the default value.
#7469: ~pytest.FixtureDef{.interpreted-text role="class"} is now exported as pytest.FixtureDef for typing purposes.
#11353: Added typing to ~pytest.PytestPluginManager{.interpreted-text role="class"}.

Bug Fixes

#10701: pytest.WarningsRecorder.pop{.interpreted-text role="meth"} will return the most-closely-matched warning in the list,
rather than the first warning which is an instance of the requested type.
#11255: Fixed crash on [parametrize(..., scope="package")]{.title-ref} without a package present.
#11277: Fixed a bug that when there are multiple fixtures for an indirect parameter,
the scope of the highest-scope fixture is picked for the parameter set, instead of that of the one with the narrowest scope.
#11456: Parametrized tests now really do ensure that the ids given to each input are unique - for
example, a, a, a0 now results in a1, a2, a0 instead of the previous (buggy) a0, a1, a0.
This necessarily means changing nodeids where these were previously colliding, and for
readability adds an underscore when non-unique ids end in a number.
#11563: Fixed a crash when using an empty string for the same parametrized value more than once.
#11712: Fixed handling NO_COLOR and FORCE_COLOR to ignore an empty value.
#9036: pytest.warns and similar functions now capture warnings when an exception is raised inside a with block.

Improved Documentation

#11011: Added a warning about modifying the root logger during tests when using caplog.
#11065: Use pytestconfig instead of request.config in cache example to be consistent with the API documentation.

Trivial/Internal Changes

#11208: The (internal) FixtureDef.cached_result type has changed.
Now the third item cached_result[2], when set, is an exception instance instead of an exception triplet.
#11218: (This entry is meant to assist plugins which access private pytest internals to instantiate FixtureRequest objects.)

~pytest.FixtureRequest{.interpreted-text role="class"} is now an abstract class which can't be instantiated directly.
A new concrete TopRequest subclass of FixtureRequest has been added for the request fixture in test functions, as counterpart to the existing SubRequest subclass for the request fixture in fixture functions.
#11315: The pytester{.interpreted-text role="fixture"} fixture now uses the monkeypatch{.interpreted-text role="fixture"} fixture to manage the current working directory.
If you use pytester in combination with monkeypatch.undo() <pytest.MonkeyPatch.undo>{.interpreted-text role="func"}, the CWD might get restored.
Use monkeypatch.context() <pytest.MonkeyPatch.context>{.interpreted-text role="func"} instead.
#11333: Corrected the spelling of Config.ArgsSource.INVOCATION_DIR.
The previous spelling INCOVATION_DIR remains as an alias.
#11638: Fixed the selftests to pass correctly if FORCE_COLOR, NO_COLOR or PY_COLORS is set in the calling environment.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pytest 8.0.0rc1 (2023-12-30)

Breaking Changes

Old Deprecations Are Now Errors

Version Compatibility

Collection Changes

Other breaking changes

Deprecations

Features and Improvements

Improved Diffs

Separate Control For Assertion Verbosity

Additional Support For Exception Groups and `notes`

Custom Directory collectors

"New-style" Hook Wrappers

Other Improvements

Bug Fixes

Improved Documentation

Trivial/Internal Changes

pytest 8.0.0rc1 (2023-12-30)

Breaking Changes

Old Deprecations Are Now Errors

Version Compatibility

Collection Changes

Other breaking changes

Deprecations

Features and Improvements

Improved Diffs

Separate Control For Assertion Verbosity

Additional Support For Exception Groups and __notes__

Custom Directory collectors

"New-style" Hook Wrappers

Other Improvements

Bug Fixes

Improved Documentation

Trivial/Internal Changes

Additional Support For Exception Groups and `notes`