always_inline makes build fail with -Og and --without-pydebug

[mgorny]

Bug report

Bug description:

We're trying to build CPython to debug a crash in third-party extension. To do that, I've set CFLAGS="-Og -g" and passed --with-assertions. However, we're not using --with-pydebug since that has had side effects that broke multiple third-party packages in the past. Unfortunately, in this configuration CPython fails to build due to use of always_inline:

gcc -c -fno-strict-overflow -fstack-protector-strong -Wtrampolines -Wsign-compare -g -O3 -Wall -Og   -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE -o Objects/dictobject.o Objects/dictobject.c
In function ‘do_lookup’,
    inlined from ‘unicodekeys_lookup_unicode’ at Objects/dictobject.c:1137:12:
Objects/dictobject.c:1120:1: error: inlining failed in call to ‘always_inline’ ‘compare_unicode_unicode’: function not considered for inlining
 1120 | compare_unicode_unicode(PyDictObject *mp, PyDictKeysObject *dk,
      | ^~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1052:30: note: called from here
 1052 |             Py_ssize_t cmp = check_lookup(mp, dk, ep0, ix, key, hash);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1120:1: error: inlining failed in call to ‘always_inline’ ‘compare_unicode_unicode’: function not considered for inlining
 1120 | compare_unicode_unicode(PyDictObject *mp, PyDictKeysObject *dk,
      | ^~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1068:30: note: called from here
 1068 |             Py_ssize_t cmp = check_lookup(mp, dk, ep0, ix, key, hash);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘do_lookup’,
    inlined from ‘unicodekeys_lookup_generic’ at Objects/dictobject.c:1116:12:
Objects/dictobject.c:1085:1: error: inlining failed in call to ‘always_inline’ ‘compare_unicode_generic’: function not considered for inlining
 1085 | compare_unicode_generic(PyDictObject *mp, PyDictKeysObject *dk,
      | ^~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1052:30: note: called from here
 1052 |             Py_ssize_t cmp = check_lookup(mp, dk, ep0, ix, key, hash);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1085:1: error: inlining failed in call to ‘always_inline’ ‘compare_unicode_generic’: function not considered for inlining
 1085 | compare_unicode_generic(PyDictObject *mp, PyDictKeysObject *dk,
      | ^~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1068:30: note: called from here
 1068 |             Py_ssize_t cmp = check_lookup(mp, dk, ep0, ix, key, hash);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘do_lookup’,
    inlined from ‘dictkeys_generic_lookup’ at Objects/dictobject.c:1171:12:
Objects/dictobject.c:1141:1: error: inlining failed in call to ‘always_inline’ ‘compare_generic’: function not considered for inlining
 1141 | compare_generic(PyDictObject *mp, PyDictKeysObject *dk,
      | ^~~~~~~~~~~~~~~
Objects/dictobject.c:1052:30: note: called from here
 1052 |             Py_ssize_t cmp = check_lookup(mp, dk, ep0, ix, key, hash);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Objects/dictobject.c:1141:1: error: inlining failed in call to ‘always_inline’ ‘compare_generic’: function not considered for inlining
 1141 | compare_generic(PyDictObject *mp, PyDictKeysObject *dk,
      | ^~~~~~~~~~~~~~~
Objects/dictobject.c:1068:30: note: called from here
 1068 |             Py_ssize_t cmp = check_lookup(mp, dk, ep0, ix, key, hash);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make: *** [Makefile:3051: Objects/dictobject.o] Error 1

CPython versions tested on:

3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-121266: Remove Py_ALWAYS_INLINE in dictobject.c #121493
• gh-121266: Change dict check_lookup() return type to int #121581
• [3.13] gh-121266: Change dict check_lookup() return type to int (GH-121581) #121949
• [3.13] gh-121266: Remove Py_ALWAYS_INLINE in dictobject.c (GH-121493) #122095

[mgorny]

I've confirmed this with 3.13 @ 78e96bd, main @ 6343486. This is Gentoo Linux amd64 with GCC 14.1.1_p20240622 p2.

[vstinner]

However, we're not using --with-pydebug since that has had side effects that broke multiple third-party packages in the past.

Would you mind to elaborate? The ABI is now the same between release and debug build since Python 3.8.

[mgorny]

Back when I've tried debugging an assertion failure and enabled --with-pydebug, I've ended up hitting different failures in dependencies even before I managed to get to "my" failure. It was #108295 (comment) back then.

[vstinner]

I proposed PR gh-121493 to remove Py_ALWAYS_INLINE. I'm not sure if my PR is correct.

[chris-eibl]

If there are problems with Py_ALWAYS_INLINE in compare_unicode_generic(), compare_unicode_unicode() and compare_generic(), I assume their *_threaded versions are affected, too, since they also have those asserts?
Except for compare_generic_threadsafe(), where the
assert(ep->me_key != NULL);
IMHO would be an improvement (or at least not hurt)?

Or maybe just for symmetry those should't use Py_ALWAYS_INLINE if their non-threaded versions don't?

[chris-eibl]

OTOH, if those asserts are the culprit and you are unsure about the performance impact of removing the Py_ALWAYS_INLINE (see #issuecomment-2213996110), you could use an #ifdef NDEBUG, but is it worth the hassle?

Since there is the convenient pyperformance infrastructure let's measure?

[vstinner]

The issue should now be fixed in 3.13 and main branches. Thanks for the report @mgorny.

[mgorny]

Thanks a lot!