Flaky test test_asyncio_repl_is_ok in test_repl

[colesbury]

The test fails because the running the asyncio repl exits with:

Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads

I'm not sure if this is just a flaky test (i.e, _enter_buffered_busy failures are expected), or if it's a bug in the underlying repl.

The exit code -6 means the process crashes due to an abort(), I think.

Example: https://github.com/python/cpython/actions/runs/9331095550/job/25685390386?pr=119908

Test Output

======================================================================
FAIL: test_asyncio_repl_is_ok (test.test_repl.TestInteractiveInterpreter.test_asyncio_repl_is_ok)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_repl.py", line 199, in test_asyncio_repl_is_ok
    assert_python_ok("-m", "asyncio")
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/script_helper.py", line 180, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/script_helper.py", line 165, in _assert_python
    res.fail(cmd_line)
    ~~~~~~~~^^^^^^^^^^
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/script_helper.py", line 75, in fail
    raise AssertionError("Process return code is %d\n"
    ...<13 lines>...
                            err))
AssertionError: Process return code is -6
command line: ['/home/runner/work/cpython/cpython-builddir/hypovenv/bin/python', '-X', 'faulthandler', '-I', '-m', 'asyncio']

stdout:
---

---

stderr:
---
asyncio REPL 3.14.0a0 (remotes/pull/119908/merge-dirty:43e023a, Jun  1 2024, 14:47:23) [GCC 11.4.0] on linux
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
Exception in thread Interactive thread:
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/asyncio/__main__.py", line 109, in run
    raise OSError(errno.ENOTTY, "tty required", "stdin")
OSError: [Errno 25] tty required: 'stdin'

During handling of the above exception, another exception occurred:

exiting asyncio REPL...
Traceback (most recent call last):
Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads
Python runtime state: finalizing (tstate=0x000055910f31bbf0)

Current thread 0x00007fa49bcdb740 (most recent call first):
  <no Python frame>
---

----------------------------------------------------------------------
Ran 8 tests in 3.[354](https://github.com/python/cpython/actions/runs/9331095550/job/25685390386?pr=119908#step:22:355)s

FAILED (failures=1)
test test_repl failed

Linked PRs

• gh-119909: Fix NameError in asyncio REPL #121341
• [3.13] gh-119909: Fix NameError in asyncio REPL (GH-121341) #121447

[gvanrossum]

Maybe @ambv or @pablogsal has a clue?

[Eclips4]

It's look related to the 2237946

[ambv]

The REPL fails to start because of stdin is not a tty:

 stderr:
---
asyncio REPL 3.14.0a0 (remotes/pull/119908/merge-dirty:43e023a, Jun  1 2024, 14:47:23) [GCC 11.4.0] on linux
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
Exception in thread Interactive thread:
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/asyncio/__main__.py", line 109, in run
    raise OSError(errno.ENOTTY, "tty required", "stdin")
OSError: [Errno 25] tty required: 'stdin'

I'm fixing that case to revert to the classic REPL in this scenario.

[ambv]

I will need a better way to reproduce this. Inside asyncio/__main__.py the raised OSError in my code example is being handled by an except: block that indeed runs the old REPL in the case of a missing TTY. So there's nothing to fix there. I can force this path locally with nohup ./python.exe -m asyncio and it does what's expected.

So to see the exception bubble up as in the failing test, this must be a combination of this code running from a secondary daemon thread at interpreter shutdown.

[Eclips4]

Can we skip it for now? I've seen it very often when when I running the test suite in "hunt refleaks" mode

[vstinner]

When _enter_buffered_busy hangs, I also get a different error message if stdin is not a TTY: NameError: name 'exit_message' is not defined.

asyncio REPL 3.14.0a0 (heads/unicode_native-dirty:f3857d87958, Jul  1 2024, 17:40:44) [GCC 14.1.1 20240620 (Red Hat 14.1.1-6)] on linux
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
Exception in thread Interactive thread:
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/asyncio/__main__.py", line 110, in run
    raise OSError(errno.ENOTTY, "tty required", "stdin")
OSError: [Errno 25] tty required: 'stdin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/threading.py", line 1039, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
exiting asyncio REPL...
  File "/home/vstinner/python/main/Lib/asyncio/__main__.py", line 120, in run
    console.interact(banner="", exitmsg=exit_message)
                                        ^^^^^^^^^^^^
NameError: name 'exit_message' is not defined
Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads
Python runtime state: finalizing (tstate=0x0000000000988b28)

Current thread 0x00007fab8fda4740 (most recent call first):
  <no Python frame>

@ambv added console.interact(banner="", exitmsg=exit_message) line in commit 2237946 one month ago.

[vstinner]

Fixed by change 1143894: I can no longer reproduce the bug.