Regression in help(sqlite3) between Python 3.10.8 and Python 3.11.0

[tom-kacvinsky]

Bug report

When running the Python interactive interpreter as follows

$ ./python3
Python 3.10.8 (tags/v3.10.8:aaaf517424, Nov  2 2022, 07:29:14) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> help(sqlite3)

Python 3.10.8 displays the help for sqlite3, but 3.11.0 gives a trackback

$ ./python3
Python 3.11.0 (main, Nov  2 2022, 07:13:10) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> help(sqlite3)
Traceback (most recent call last):
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2219, in wrap_value
    value = eval(s, module_dict)
            ^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'ConnectionType' is not defined. Did you mean: 'Connection'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2222, in wrap_value
    value = eval(s, sys_module_dict)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'ConnectionType' is not defined. Did you mean: 'ConnectionError'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen _sitebuiltins>", line 103, in __call__
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 2001, in __call__
    self.help(request)
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 2060, in help
    else: doc(request, 'Help on %s:', output=self._output)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 1784, in doc
    pager(render_doc(thing, title, forceload))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 1778, in render_doc
    return title % desc + '\n\n' + renderer.document(object, name)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 479, in document
    if inspect.ismodule(object): return self.docmodule(*args)
                                        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 1275, in docmodule
    contents.append(self.document(value, key, name))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 481, in document
    if inspect.isroutine(object): return self.docroutine(*args)
                                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/pydoc.py", line 1494, in docroutine
    signature = inspect.signature(object)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 3272, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 3020, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2512, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2324, in _signature_from_builtin
    return _signature_fromstr(cls, func, s, skip_bound_arg)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2272, in _signature_fromstr
    p(name, default)
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2254, in p
    default_node = RewriteSymbolics().visit(default_node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/ast.py", line 410, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2246, in visit_Name
    return wrap_value(node.id)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/LOCAL/tjk/python-3.11.0/lib/python3.11/inspect.py", line 2224, in wrap_value
    raise RuntimeError()
RuntimeError

Your environment

• CPython versions tested on: 3.10.8 and 3.11.0
• Operating system and architecture: CentOS 7, x86_64

Built with GCC 12.1.0 and sqlite3 3.37.2

[AlexWaygood]

I get a different traceback on my Windows machine on main (EDIT: my traceback is the same on main after rebuilding CPython locally.)

Reproduced on main.

[AlexWaygood]

This bisects to a3d4d15 on main, and 5d3f2fb on the 3.11 branch. That's these PRs:

• gh-95132: Correctly relay *args and **kwds from sqlite3.connect to factory #95146
• [3.11] gh-95132: Correctly relay *args and **kwds from sqlite3.connect to factory (GH-95146) #95158

Both PRs are linked to:

• Python3.11: sqlite3.connect passes more arguments to factory() #95132

Cc. @erlend-aasland

[erlend-aasland]

Thanks for the report @tom-kacvinsky, and thanks for the bisect @AlexWaygood! I'll look into it.

[AlexWaygood]

I was curious, so I ran the following script to see if help(module) failed for any other stdlib modules:

Click to see the script

import io
import sys
import warnings
from contextlib import redirect_stdout

warnings.filterwarnings("ignore", category=DeprecationWarning)

bad_modules, unimportable_modules = [], []

for name in sys.stdlib_module_names:
    if name in {"antigravity", "this"}:
        continue  # no
    try:
        mod = __import__(name)
    except:
        unimportable_modules.append(name)
        continue
    with redirect_stdout(io.StringIO()):
        try:
            help(mod)
        except:
            bad_modules.append(name)

print(f"Modules with broken help: {bad_modules}")
print(f"Unimportable modules: {unimportable_modules}")

Results on my Windows machine:

Modules with broken help: ['_ssl', 'sqlite3', '_sqlite3']
Unimportable modules: ['resource', 'crypt', 'curses', 'posix', 'grp', '_crypt', '_scproxy', '_posixshmem', 'readline', 'tty', 'pwd', 'nis', '_gdbm', '_dbm', 'termios', 'syslog', '_posixsubprocess', 'pty', 'fcntl', '_curses', 'spwd', '_curses_panel', 'ossaudiodev']

python -c "import _ssl; help(_ssl)" crashes with a very similar traceback:

Click to see the traceback

C:\Users\alexw\coding\cpython>python -c "import _ssl; help(_ssl)"
Running Debug|x64 interpreter...
Traceback (most recent call last):
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2207, in wrap_value
    value = eval(s, module_dict)
            ^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'Encoding' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2210, in wrap_value
    value = eval(s, sys_module_dict)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'Encoding' is not defined. Did you mean: 'encodings'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\alexw\coding\cpython\Lib\_sitebuiltins.py", line 103, in __call__
    return pydoc.help(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 2001, in __call__
    self.help(request)
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 2062, in help
    else: doc(request, 'Help on %s:', output=self._output)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 1783, in doc
    pager(render_doc(thing, title, forceload))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 1777, in render_doc
    return title % desc + '\n\n' + renderer.document(object, name)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 479, in document
    if inspect.ismodule(object): return self.docmodule(*args)
                                        ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 1268, in docmodule
    contents.append(self.document(value, key, name))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 480, in document
    if inspect.isclass(object): return self.docclass(*args)
                                       ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 1434, in docclass
    attrs = spill("Methods %s:\n" % tag, attrs,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 1383, in spill
    push(self.document(value,
         ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 481, in document
    if inspect.isroutine(object): return self.docroutine(*args)
                                         ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\pydoc.py", line 1493, in docroutine
    signature = inspect.signature(object)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 3264, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 3008, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2500, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2312, in _signature_from_builtin
    return _signature_fromstr(cls, func, s, skip_bound_arg)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2260, in _signature_fromstr
    p(name, default)
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2242, in p
    default_node = RewriteSymbolics().visit(default_node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\ast.py", line 413, in visit
    return visitor(node)
           ^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2229, in visit_Attribute
    return wrap_value(value)
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\inspect.py", line 2212, in wrap_value
    raise RuntimeError()
RuntimeError

[tom-kacvinsky]

With the same script, these modules on Linux have problems with help(<module>)

$ /home/PUBLIC/tjk/python-3.11.0/bin/python3 help.py
Modules with broken help: ['sqlite3', '_sqlite3', '_ssl']
Unimportable modules: ['turtle', 'winsound', 'tkinter', 'msilib', 'nt', '_msi', '_tkinter', 'msvcrt', 'winreg', '_winapi', '_overlapped', '_scproxy']

Some of the ones that don't import are ones that either don't apply to Linux, or I didn't build - tkinter, for instance.

[erlend-aasland]

@AlexWaygood: Nice. A possible enhancement for the check-modules script that's run at the end of the build?

[erlend-aasland]

Part of the problem is inconsistencies in the docstrings themselves. I've fixed these, but the problems still persist. I suspect a bug in inspect.

[erlend-aasland]

See also gh-83685

[erlend-aasland]

See also gh-83685

I confirm that the fix for the above issue, gh-98796, resolves the remaining problems reported in this issue. The docstring fixups I mentioned does not affect this issue at all, so I'll merge them as an isolated change. I'll mark this as a duplicate of gh-83685.

[erlend-aasland]

Duplicate of gh-83685