Cannot run unit tests with Python 3.10

[michaelkuhn]

Steps to reproduce

$ spack unit-test   
==> Error: required field "lineno" missing from alias

This seems to be pytest-dev/pytest#8540

Error message

$ spack --debug --stacktrace unit-test
lib/spack/spack/cmd/__init__.py:123 ==> [2022-06-04-12:58:33.721422] Imported unit_test from built-in commands
lib/spack/spack/cmd/__init__.py:123 ==> [2022-06-04-12:58:33.722049] Imported unit_test from built-in commands
lib/spack/spack/config.py:1017 ==> [2022-06-04-12:58:33.725607] Reading config file $SPACK/etc/spack/defaults/bootstrap.yaml
lib/spack/spack/config.py:1017 ==> [2022-06-04-12:58:33.730714] Reading config file $SPACK/etc/spack/defaults/config.yaml
lib/spack/spack/config.py:1017 ==> [2022-06-04-12:58:33.758275] Reading config file $SPACK/etc/spack/defaults/config.yaml
lib/spack/spack/database.py:382 ==> [2022-06-04-12:58:33.769446] DATABASE LOCK TIMEOUT: 3s
lib/spack/spack/database.py:386 ==> [2022-06-04-12:58:33.769576] PACKAGE LOCK TIMEOUT: No timeout
lib/spack/spack/config.py:1017 ==> [2022-06-04-12:58:33.769948] Reading config file $SPACK/etc/spack/defaults/bootstrap.yaml
lib/spack/spack/database.py:382 ==> [2022-06-04-12:58:33.773719] DATABASE LOCK TIMEOUT: 3s
lib/spack/spack/database.py:386 ==> [2022-06-04-12:58:33.773838] PACKAGE LOCK TIMEOUT: No timeout
lib/spack/spack/bootstrap.py:613 ==> [2022-06-04-12:58:33.773955] [BOOTSTRAP CONFIG SCOPE] name=_builtin
lib/spack/spack/bootstrap.py:629 ==> [2022-06-04-12:58:33.774258] [BOOTSTRAP CONFIG SCOPE] name=defaults, path=$SPACK/etc/spack/defaults
lib/spack/spack/bootstrap.py:630 ==> [2022-06-04-12:58:33.774345] [BOOTSTRAP CONFIG SCOPE] name=defaults/linux, path=$SPACK/etc/spack/defaults/linux
lib/spack/spack/bootstrap.py:629 ==> [2022-06-04-12:58:33.774431] [BOOTSTRAP CONFIG SCOPE] name=bootstrap, path=$DOTSPACK/bootstrap/config
lib/spack/spack/bootstrap.py:630 ==> [2022-06-04-12:58:33.774518] [BOOTSTRAP CONFIG SCOPE] name=bootstrap/linux, path=$DOTSPACK/bootstrap/config/linux
lib/spack/spack/config.py:1017 ==> [2022-06-04-12:58:33.774905] Reading config file $DOTSPACK/bootstrap/config/linux/compilers.yaml
lib/spack/spack/bootstrap.py:769 ==> [2022-06-04-12:58:33.802348] [BOOTSTRAP ROOT SPEC] clingo-bootstrap@spack+python %gcc target=x86_64
lib/spack/spack/bootstrap.py:522 ==> [2022-06-04-12:58:33.802477] [BOOTSTRAP MODULE clingo] Try importing from Python
lib/spack/spack/config.py:1017 ==> [2022-06-04-12:58:33.899979] Reading config file $SPACK/etc/spack/defaults/config.yaml
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'AssertionRewritingHook' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/main.py", line 110, in wrap_session
    config._do_configure()
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/config.py", line 935, in _do_configure
    self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 730, in call_historic
    self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 594, in execute
    res = hook_impl.function(*args)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/terminal.py", line 54, in pytest_configure
    reporter = TerminalReporter(config, sys.stdout)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/terminal.py", line 140, in __init__
    self._tw = self.writer = _pytest.config.create_terminal_writer(config,
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/config.py", line 1375, in create_terminal_writer
    tw = py.io.TerminalWriter(*args, **kwargs)
  File "$SPACK/lib/spack/external/pytest-fallback/py/_apipkg.py", line 125, in __makeattr
    result = importobj(modpath, attrname)
  File "$SPACK/lib/spack/external/pytest-fallback/py/_apipkg.py", line 48, in importobj
    module = __import__(modpath, None, None, ['__doc__'])
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 915, in _find_spec_legacy
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/assertion/rewrite.py", line 139, in find_module
    source_stat, co = _rewrite_test(self.config, fn_pypath)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/assertion/rewrite.py", line 335, in _rewrite_test
    co = compile(tree, fn.strpath, "exec", dont_inherit=True)
TypeError: required field "lineno" missing from alias

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'AssertionRewritingHook' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "$SPACK/./bin/spack", line 98, in <module>
    sys.exit(spack.main.main())
  File "$SPACK/lib/spack/spack/main.py", line 893, in main
    return _main(argv)
  File "$SPACK/lib/spack/spack/main.py", line 848, in _main
    return finish_parse_and_run(parser, cmd_name, env_format_error)
  File "$SPACK/lib/spack/spack/main.py", line 876, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
  File "$SPACK/lib/spack/spack/main.py", line 529, in _invoke_command
    return_val = command(parser, args, unknown_args)
  File "$SPACK/lib/spack/spack/cmd/unit_test.py", line 218, in unit_test
    return pytest.main(pytest_args)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/config.py", line 58, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 725, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/vendored_packages/pluggy.py", line 594, in execute
    res = hook_impl.function(*args)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/main.py", line 143, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/main.py", line 126, in wrap_session
    config.notify_exception(excinfo, config.option)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/config.py", line 965, in notify_exception
    excrepr = excinfo.getrepr(funcargs=True,
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/_code/code.py", line 427, in getrepr
    py.std.traceback.format_exception(
  File "$SPACK/lib/spack/external/pytest-fallback/py/_apipkg.py", line 125, in __makeattr
    result = importobj(modpath, attrname)
  File "$SPACK/lib/spack/external/pytest-fallback/py/_apipkg.py", line 48, in importobj
    module = __import__(modpath, None, None, ['__doc__'])
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 915, in _find_spec_legacy
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/assertion/rewrite.py", line 139, in find_module
    source_stat, co = _rewrite_test(self.config, fn_pypath)
  File "$SPACK/lib/spack/external/pytest-fallback/_pytest/assertion/rewrite.py", line 335, in _rewrite_test
    co = compile(tree, fn.strpath, "exec", dont_inherit=True)
TypeError: required field "lineno" missing from alias

Information on your system

• Spack: 0.19.0.dev0 (f3893b3)
• Python: 3.10.4
• Platform: linux-fedora36-zen2
• Concretizer: clingo

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have searched the issues of this repo and believe this is not a duplicate
• I have run the failing commands in debug mode and reported the output

[alalazo]

Workaround:

$ pip install pytest

and you'll get all the benefits of recent pytest versions 😉

[haampie]

We now run 3.10 tests in ci

[michaelkuhn]

spack unit-test still doesn't work out-of-the-box, though (or did I miss something?):

$ spack unit-test                     
==> Error: required field "lineno" missing from alias

[michaelkuhn]

OK, I found #25371, which implies that the vendored pytest should only be used as a last resort. I have now installed pytest locally and everything works. It might make sense to simply remove the vendored pytest since it won't work with any up-to-date installation anyway and might cause confusion (for me, at least).