RFC: Enforcing abi3 compliance?

[woodruffw]

First of all, thanks a ton for maintaining auditwheel! I've been a happy user for years.

Has anybody considered extending auditwheel to double-check that a wheel's abi3 tag is actually consistent with its contents? I can see this being useful/valuable to the community for a handful of reasons:

1. The presence of the abi3 tag in a wheel filename is just metadata, not a strong signal that the wheel is actually abi3 compatible with a particular Python version.
2. One of the most common ways of tagging a wheel as abi3 compatible is to use setuptools' --py-limited-abi=... flag, which doesn't actually set the underlying Py_LIMITED_API macro on any native extension builds. As such, it's possible that some wheels currently tagged as abi3 on PyPI and other indices are not actually abi3, and are working purely by luck.

In both cases, being tagged as abi3 but not actually being abi3 has reliability (hard to debug crashes because of ABI changes) and security (memory corruption from the same) implications.

If there's interest in this, it's something I can look into.

[woodruffw]

More context:

• setuptools does not guarantee that --py-limited-api corresponds to the right CPython macro, per the changelog: https://setuptools.pypa.io/en/latest/history.html?highlight=py_limited_api#v25-4-0

• CPython itself will try to load abi3-tagged shared objects without actually confirming that they're abi3 compatible: https://docs.python.org/3/c-api/stable.html#c.Py_LIMITED_API

[woodruffw]

Just to post an update: I'm currently working on a standalone tool that scans wheels for abi3 violations. I think most of the code there will be reusable and integratable into auditwheel if there's maintainer interest, however! As such, I'll leave this open.

[mayeut]

@woodruffw, I read 2 different things:

• enforcing abi3 compliance
• scans wheels for abi3 violations

The first one is not something that can be done I think.
The second one can but let's see what it would look like as a standalone tool before considering this for auditwheel.

[woodruffw]

Sorry for the late response!

Yes, I agree that enforcing abi3 compliance isn't something that can be done.

I'm currently working on the aforementioned standalone tool: https://github.com/trailofbits/abi3audit

[woodruffw]

Just following up: my company published a writeup of our findings using the tool above: https://blog.trailofbits.com/2022/11/15/python-wheels-abi-abi3audit/

In particular, a significant fraction (1/6th) of all packages containing ABI3 wheels have at least one mis-tagged wheel, suggesting that there would be a significant ecosystem benefit to automated detection.

[mayeut]

Thanks for the update.

I saw that it only supports cp310+, it probably is a showstopper for now if we want to make that check mandatory ?
I think a good starting point would be for auditwheel to call abi3audit CLI when asked to (by adding a flag) ?
Or should we aim directly to make that check mandatory when abi3audit CLI is present ?

[woodruffw]

Yep, I wrote it for 3.10+ because it was greenfield. I'm happy to port it downwards to 3.7 or 3.8, though, if it would help with integrations.

I was thinking that it probably makes sense to make the check mandatory, since users otherwise won't know to opt into it and it shouldn't have any false positives. But I'm open to either approach!

[woodruffw]

JFYI: We've cut a release (0.0.8) of abi3audit that's compatible with 3.8+, which means that it should be much more broadly applicable now.