Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-44975: [typing] Support issubclass for ClassVar data members #27883

Closed
5 changes: 5 additions & 0 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1284,6 +1284,11 @@ These are not used in annotations. They are building blocks for creating generic

.. versionadded:: 3.8

.. versionchanged:: 3.11
Protocols with data members annotated with :data:`ClassVar` now support
:func:`issubclass` checks. Subclasses must set these data members to pass.


Other special directives
""""""""""""""""""""""""

Expand Down
7 changes: 7 additions & 0 deletions Doc/whatsnew/3.11.rst
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,13 @@ sqlite3
experience.
(Contributed by Erlend E. Aasland in :issue:`45828`.)

typing
------

* Runtime protocols with data members now support :func:`issubclass` as long
as those members are annotated with :data:`typing.ClassVar`.
(Contributed by Ken Jin in :issue:`44975`).


sys
---
Expand Down
49 changes: 49 additions & 0 deletions Lib/test/test_typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1622,6 +1622,55 @@ def __init__(self):

Foo() # Previously triggered RecursionError

def test_runtime_issubclass_with_classvar_data_members(self):
@runtime_checkable
class P(Protocol):
x: ClassVar[int] = 1

class C: pass

class D:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also add a case with the same class prop, but different value. Example:

class E:
    x = 2

Because right now all names / values always match. It is not clear whether value is a part of the protocol or not.

Copy link
Member Author

@Fidget-Spinner Fidget-Spinner Dec 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

It is not clear whether value is a part of the protocol or not.

It shouldn't be. IMO, we should only care about the "shape". Trying to runtime-check the value is difficult and should be left to third-party libs to do.

Nevermind, I changed my mind. Let's do it!

x = 1

class E:
x = 2

class F:
x = 'foo'

self.assertNotIsSubclass(C, P)
self.assertIsSubclass(D, P)
Fidget-Spinner marked this conversation as resolved.
Show resolved Hide resolved
self.assertIsSubclass(E, P)
self.assertNotIsSubclass(F, P)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also test these classes:

class G:
   x: ClassVar[int] = 1
class H:
   x: 'ClassVar[int]' = 1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These aren't testing more code paths, the code doesn't look at the annotations of the first class argument in issubclass.

# String annotations (forward references).
@runtime_checkable
class P(Protocol):
# Special case, bare ClassVar, our checks should
# just skip these.
w: "ClassVar"
x: "ClassVar[int]" = 1
y: "typing.ClassVar[int]" = 2
z: "t.ClassVar[int]" = 3

class D:
w = 0
x = 1
y = 2
z = 3

self.assertNotIsSubclass(C, P)
self.assertIsSubclass(D, P)

# Make sure mixed are forbidden.
@runtime_checkable
class P(Protocol):
x: "ClassVar[int]" = 1
y = 2

self.assertRaises(TypeError, issubclass, C, P)
self.assertRaises(TypeError, issubclass, D, P)


class GenericTests(BaseTestCase):

Expand Down
52 changes: 43 additions & 9 deletions Lib/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1404,10 +1404,43 @@ def _get_protocol_attrs(cls):
attrs.add(attr)
return attrs

_CLASSVAR_PREFIXES = ("typing.ClassVar", "t.ClassVar", "ClassVar")

def _is_callable_members_only(cls):
def _is_callable_or_classvar_members_only(cls, instance):
"""Returns a 2-tuple signalling two things:
(Valid protocol?, If not valid protocol, was it due to ClassVar value mismatch?)
"""
attr_names = _get_protocol_attrs(cls)
annotations = getattr(cls, '__annotations__', {})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using typing.get_type_hints to resolve annotations for a class?

It will allow having ClassVar annotated as str:

@runtime_checkable
class P(Protocol):
    x: 'ClassVar[int]' = 1
Suggested change
annotations = getattr(cls, '__annotations__', {})
annotations = get_type_hints(cls)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! I'd considered that too but decided against it for two main reasons:

  1. get_type_hints is slow
  2. get_type_hints won't work with forward references/ PEP 563 if ClassVar is not defined in the caller's namespace

Consider the following:

# foo.py
from __future__ import annotations
from typing import *

@runtime_checkable
class X(Protocol):
 x: ClassVar[int] = 1
 y: SomeUndeclaredType = None
# bar.py
from .foo import X
class Y: ...

# Error! get_type_hints cannot resolve 'ClassVar' and 'SomeUndeclaredType'
issubclass(X, Y)

Nonetheless, your suggestion has reminded me of a basic workaround for string annotations. Thanks.

# PEP 544 prohibits using issubclass() with protocols that have non-method members.
return all(callable(getattr(cls, attr, None)) for attr in _get_protocol_attrs(cls))
# bpo-44975: Relaxing that restriction to allow for runtime-checkable
# protocols with class variables since those should be available at class
# definition time.
for attr_name in attr_names:
attr = getattr(cls, attr_name, None)
# Method-like.
if callable(attr):
continue
annotation = annotations.get(attr_name)
# ClassVar member
if getattr(annotation, '__origin__', None) is ClassVar:
instance_attr = getattr(instance, attr_name, None)
# If we couldn't find anything, don't bother checking value types.
if (instance_attr is not None
and attr is not None
and type(instance_attr) != type(attr)):
return False, True
continue
# ClassVar string annotations (forward references).
if isinstance(annotation, str) and annotation.startswith(_CLASSVAR_PREFIXES):
instance_attr = getattr(instance, attr_name, None)
if (instance_attr is not None
and attr is not None
and type(instance_attr) != type(attr)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. Consider this example:

class P(Protocol):
     x: ClassVar[object] = object()

class Y:
     x: ClassVar[object] = 1

Y implements protocol P, but your code rejects it. There are also problems with None here that could produce an incorrect result.

I'd recommend removing this whole check, and just checking that the attribute exists. Full runtime type checking is hard and the standard library shouldn't try to do it.

return False, True
continue
return False, False
return True, False


def _no_init_or_replace_init(self, *args, **kwargs):
Expand Down Expand Up @@ -1477,10 +1510,9 @@ def __instancecheck__(cls, instance):
):
raise TypeError("Instance and class checks can only be used with"
" @runtime_checkable protocols")

if ((not getattr(cls, '_is_protocol', False) or
_is_callable_members_only(cls)) and
issubclass(instance.__class__, cls)):
_is_callable_or_classvar_members_only(cls, instance)[0]) and
issubclass(instance.__class__, cls)):
return True
if cls._is_protocol:
if all(hasattr(instance, attr) and
Expand Down Expand Up @@ -1544,11 +1576,13 @@ def _proto_hook(other):
return NotImplemented
raise TypeError("Instance and class checks can only be used with"
" @runtime_checkable protocols")
if not _is_callable_members_only(cls):
if _allow_reckless_class_checks():
ok_members, classvar_mismatch = _is_callable_or_classvar_members_only(cls, other)
if not ok_members:
if _allow_reckless_class_checks() or classvar_mismatch:
return NotImplemented
raise TypeError("Protocols with non-method members"
" don't support issubclass()")
raise TypeError("Protocol members must be methods or data"
" attributes annotated with ClassVar to support"
" issubclass()")
if not isinstance(other, type):
# Same error message as for issubclass(1, int).
raise TypeError('issubclass() arg 1 must be a class')
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Runtime protocols with data members now support :func:`issubclass` as long
as those members are annotated with :data:`typing.ClassVar`.