Weird problem with IntEnum

[jolaf]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

Typeguard version

4.1.3

Python version

3.10.12

What happened?

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    import_module('A')
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/home/jolaf/.local/lib/python3.10/site-packages/typeguard/_importhook.py", line 98, in exec_module
    super().exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "A.py", line 2, in <module>
    class A(IntEnum):
  File "/usr/lib/python3.10/enum.py", line 289, in __new__
    enum_member = __new__(enum_class, *args)
  File "A.py", line 3, in __new__
    def __new__(cls, value: int, phrase: str = "") -> "A":
NameError: name 'A' is not defined

How can we reproduce the bug?

test.py:

from importlib import import_module
from typeguard import install_import_hook
with install_import_hook(('A',)):
    import_module('A')

A.py:

from enum import IntEnum
class A(IntEnum):
    def __new__(cls, value: int, phrase: str = "") -> "A":
        obj = int.__new__(cls, value)
        obj._value_ = value
        obj.phrase = phrase
        return obj
    v = 5

$ python3 test.py

[jolaf]

Originally the problem appeared on import httpx, I've just narrowed it down.

[agronholm]

This happens when __new__() is invoked at v = 5 before the class definition has finished. The interpreter therefore raises NameError because the class is not present in the global scope (or anywhere else really) . I don't know if there's anything I can do to fix this.

[agronholm]

I suppose I could special case __new__() and alias "A" to cls.

[jolaf]

Somehow interpreter is fine with this when typeguard is not involved.

[agronholm]

Of course, since this happens when A.__new__() is instrumented by typeguard.

[agronholm]

By the way, I suggest you familiarize yourself with the typeguard.config.debug_instrumentation flag. If you run into trouble, it will often give you valuable insight into the instrumentation process. For example, in this particular case:

Source code of '/home/alex/workspace/typeguard/foo2.py' after instrumentation:
----------------------------------------------
from typeguard import TypeCheckMemo
from typeguard._functions import check_argument_types, check_return_type
from enum import IntEnum

class A(IntEnum):

    def __new__(cls, value: int, phrase: str='') -> 'A':
        memo = TypeCheckMemo(globals(), locals(), self_type=cls)
        check_argument_types('A.__new__', {'value': (value, int), 'phrase': (phrase, str)}, memo)
        obj = int.__new__(cls, value)
        obj._value_ = value
        obj.phrase = phrase
        return check_return_type('A.__new__', obj, A, memo)
    v = 5
----------------------------------------------

[jolaf]

I'm afraid that looking at that dump I still don't see what is the cause of the problem. :(

[agronholm]

I'm afraid that looking at that dump I still don't see what is the cause of the problem. :(

Well, remember that classes only appear in the namespace where they are declared once the interpreter has finished executing their bodies? But that v = 5 line causes the __new__() method to be called before the class exists in the module's namespace. Now Typeguard has a problem: where is it going to get a reference to class A which it's supposed to check against? Without any special measures (which I mentioned earlier), the lookup for A would fail.

[agronholm]

As you can see from return check_return_type('A.__new__', obj, A, memo), it's referencing the class A here, but since it's not in the global namespace yet, you get a NameError.

[jolaf]

Okay, thanks, I see it now, after your explanation.

[agronholm]

The easiest fix I could think of involved replacing A with cls, but then I ran into a problem where the return type check was omitted entirely because the transformer considers all argument names as something you shouldn't check against.

Once I solve this issue I'll cut another patch release.

[agronholm]

Fixed via 888a8c5.

[agronholm]

Checked against httpx too, and it imports fine now.

[jolaf]

Thanks!!