Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for inheriting from built-in types other than PyObject (which is already supported) #169

Closed
hodgestar opened this issue Feb 16, 2021 · 2 comments
Assignees
Milestone

Comments

@hodgestar
Copy link
Contributor

HPy supports two kinds of memory layouts in extension types:

  • Legacy types have the same memory layout as in the old C API, and have structs that begin with PyObject_HEAD.
  • Pure types do not include PyObject_HEAD.

When a legacy type inherits from another legacy type, or when a pure type inherits from another pure type, the situation is straight forward -- the inheriting type includes the full struct of the base type and optionally extends the struct with additional members.

Ideally HPy would simply forbid legacy types inheriting from pure types or vice versa -- having HPy magically add or remove PyObject_HEAD as needed by C slots or methods in the two types would be error prone and complex.

However, pure types may need to extend built-in types like PyLongObject, PyDictObject, or PyUnicodeObject. This is currently only partially supported as follows:

  • PyObject: inheriting fully supported.
  • other built-in types: supported, but only if no extra members are added to the struct, and if HPy_AsStruct is not used and and no deallocator (i.e. HPy_tp_destroy) is defined on the extending type.

Ideally pure types would treat other built-in types in the same way as PyObject -- i.e. all of their internal memory layout would be hidden from the C extension -- but the C Python implementation of HPy does not yet support this.

Support for inheriting from other built-in types in the same way as from PyObject should be added.

It might be possible to allow pure types to extend any legacy type in the same way (by making the struct of the existing type inaccessible to the slots and methods of the pure type) but this is a stretch goal.

@hodgestar
Copy link
Contributor Author

Things are further complicated because determining which type an object is inheriting its memory layout from is rather complex until after one has called PyType_FromSpec.

@timfel timfel added this to the Version 0.9 milestone Nov 1, 2022
@fangerer fangerer self-assigned this Nov 22, 2022
@fangerer
Copy link
Contributor

I think the main part of it is solved by PR #335 .
However, some ideas/points are still unresolved or untested:

  • We don't test (and I'm not sure if that works): a legacy type inherits from a pure type. I'll investigate and open an issue if appropriate.
  • Although it is possible for pure types to extend any legacy type (in the sense that at the time of type creation, we won't issue an error), it is actually not supported. Using the <type_struct>_AsStruct function will most likely return the wrong pointer. This is maybe something we should prevent by default. It might still make sense to allow it since one can use _HPyType_GetBuiltinShape to get the shape of the base type (which will be HPyType_BuiltinShape_Legacy) and then it is possible to implement a custom *_AsStruct by manually adding the base's basicsize. I'll open a separate issue for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants