Pytest Report fails due to AttributeError

[johnmarston-nhs]

Type: Bug

Behaviour

Expected vs. Actual

Expected behaviour:
When a test fails, the pytest report should be printed to the terminal.

Actual:
INTERNALERROR is reported to the terminal, along with a stacktrace as follows:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/_pytest/main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/_pytest/main.py", line 348, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 126, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 221, in call_and_report
INTERNALERROR>     hook.pytest_exception_interact(node=item, call=call, report=report)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "<projectdir>/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "<homedir>/.vscode/extensions/ms-python.python-2023.14.0/pythonFiles/vscode_pytest/__init__.py", line 74, in pytest_exception_interact
INTERNALERROR>     if type(report) == pytest.CollectReport:
INTERNALERROR> AttributeError: module 'pytest' has no attribute 'CollectReport'
pytest session has finished, exit status:  ExitCode.INTERNAL_ERROR in discovery?  False

Steps to reproduce:

1. Debug a failing pytest test using VS Code.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-08-08 16:05:11.883 [info] Experiment 'pythonPromptNewFormatterExt' is active
2023-08-08 16:05:11.883 [info] Experiment 'pythonPromptNewToolsExt' is active
2023-08-08 16:05:11.883 [info] Experiment 'pythonTestAdapter' is active
2023-08-08 16:05:11.883 [info] VS Code was launched from an activated environment: '.venv', selecting it as the interpreter for workspace.
2023-08-08 16:05:11.883 [info] Test server listening.
2023-08-08 16:05:11.883 [info] Python interpreter path: ./.venv/bin/python
2023-08-08 16:05:13.173 [info] > conda info --json
2023-08-08 16:05:13.449 [info] > /usr/share/anaconda/bin/conda info --json
2023-08-08 16:05:13.707 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2023.14.0/pythonFiles/printEnvVariables.py
2023-08-08 16:05:13.708 [info] shell: bash
2023-08-08 16:05:13.836 [info] Discover tests for workspace name: api-management-infrastructure - uri: /home/jon/APIM/repos/api-management-infrastructure
2023-08-08 16:05:13.836 [info] Running discovery for pytest using the new test adapter.
2023-08-08 16:05:13.858 [info] > ./.venv/bin/python -m pytest -p vscode_pytest --collect-only -v docker/proxygen
2023-08-08 16:05:13.858 [info] cwd: .
2023-08-08 16:05:13.967 [info] LINTING: Skipping linting from Python extension, since Flake8 extension is installed and enabled.
2023-08-08 16:05:13.967 [info] LINTING: Skipping linting from Python extension, since Pylint extension is installed and enabled.
2023-08-08 16:05:15.022 [info] Starting Pylance language server.
2023-08-08 16:05:16.654 [info] Test server connected to a client.
2023-08-08 16:05:16.728 [info] Using result resolver for discovery
2023-08-08 16:05:21.728 [info] Running PYTEST execution for the following test ids: docker/proxygen/proxygen/tests/integration/test_specs.py::TestApiSpecsEndpoints::test_spec_only_validation
2023-08-08 16:05:21.728 [info] Server listening on port 33945
2023-08-08 16:05:21.729 [info] Running DEBUG pytest with arguments: --rootdir /home/jon/APIM/repos/api-management-infrastructure -v --capture no

2023-08-08 16:05:21.731 [info] Using launch configuration in launch.json file.
2023-08-08 16:05:22.136 [info] DAP Server launched with command: /home/jon/APIM/repos/api-management-infrastructure/.venv/bin/python /home/jon/.vscode/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter
2023-08-08 16:05:22.486 [info] Send text to terminal: source /home/jon/APIM/repos/api-management-infrastructure/.venv/bin/activate
2023-08-08 16:05:23.488 [info] Send text to terminal:  /usr/bin/env /home/jon/APIM/repos/api-management-infrastructure/.venv/bin/python /home/jon/.vscode/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 56395 -- /home/jon/.vscode/extensions/ms-python.python-2023.14.0/pythonFiles/vscode_pytest/run_pytest_script.py --rootdir /home/jon/APIM/repos/api-management-infrastructure -v --capture no 
2023-08-08 16:05:30.391 [info] Test server connected to a client.
2023-08-08 16:05:30.404 [info] Test server connected to a client.
2023-08-08 16:05:31.284 [info] Client disconnected

User Settings

languageServer: "Pylance"

linting
• flake8Enabled: true

formatting
• provider: "none"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• pytestPath: "<placeholder>"

Extension version: 2023.14.0
VS Code version: Code 1.81.0 (6445d93c81ebe42c4cbd7a60712e0b17d9463e97, 2023-08-02T12:36:11.334Z)
OS version: Linux x64 6.4.7-200.fc38.x86_64
Modes:

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 x 3497)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	1, 1, 1
Memory (System)	62.43GB (48.92GB free)
Process Argv	. --crash-reporter-id ed7cda52-d9be-4e64-8def-511b71321e86
Screen Reader	no
VM	0%
DESKTOP_SESSION	gnome-xorg
XDG_CURRENT_DESKTOP	GNOME
XDG_SESSION_DESKTOP	gnome-xorg
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263cf:30335440
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
py29gd2263:30792226
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
vscccc:30803845
2e4cg342:30602488
f6dab269:30613381
showlangstatbar:30737416
vsctsb:30748421
03d35959:30757346
pythonfmttext:30731395
pythoncmvfstrcf:30756944
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
gsofa:30804715
pythonnosmt12:30797651
pythonidxptcf:30805731
pythonnoceb:30805159
e537b577:30795824
dsvsc013:30795093
dsvsc014:30804076

[karthiknadig]

@johnmarston-nhs What is the version of pytest that you have?

[johnmarston-nhs]

@karthiknadig 6.2.5.

Just to add to this, it works fine if I pin the version of vscode-python to 2023.12.0.

[eleanorjboyd]

Thank you for your issue report. If you add this to your settings does this fix it? user settings: "python.experiments.optOutFrom": ["pythonTestAdapter"],. If that does fix it then it is likely on the rewrite which will give me more information I can use to fix this. Thanks

[johnmarston-nhs]

Hi @eleanorjboyd , yes, opting out from pythonTestAdapter fixes the issue, thank you for your suggestion. Is leaving this disabled gonna bite me in the butt later down the line? 😄

As an addition, I was looking into the exception - it seems to be down to the CollectReport class not being 'exported' in versions of pytest below v7. It got added in this commit: pytest-dev/pytest@b0aa870

Maybe you can work around this with a str() around the call to type() or something, as I think pytest 6 is still quite widely used.

[eleanorjboyd]

Thank you for that investigation! I will look into that as a workaround.

Also no, the setting shouldn't be a problem later as once we are fully confident in the re-write and move it to 100% of stable we will get rid of the experiment and that setting will be ignored.

[andreamah]

Are there verification steps? Or @johnmarston-nhs, do you still see this in the latest Python prerelease?

[eleanorjboyd]

1. set your pytest version to 6.2.5
2. create a test in a file like test_file.py

def test_a:
     assert True

4. run pytest discovery (testing tab, configure testing with pytest)
5. run this pytest test
6. confirm you do not see any errors like above in your "python testing log" section of the output panel

[sandy081]

To verify I have to update the above sample to following

def test_a():
     assert True

[collinwr]

I am running pytest 6.2.5, and I am seeing the error even with the "python.experiments.optOutFrom": ["pythonTestAdapter"], setting configured in my user settings.

[eleanorjboyd]

to confirm @collinwr you are seeing INTERNALERROR> AttributeError: module 'pytest' has no attribute 'CollectReport' right? Do you see this even when you opt into the experiment? It will currently only work on insiders as this is unreleased in stable.