Question about returning unique pointer in virtual function #673
Comments
|
If we are returning a unique_ptr to the base class in a pure virtual member function, pybind11 gives the above compiler error. Just wondering if this is something pybind11 supports. However, if the member function is just a regular function (not a virtual function), everything works as expected. //Everything works great with the below!
class Animal {
public:
Animal(){}
~Animal() { }
std::unique_ptr<Animal> create()
{
return std::unique_ptr<Animal>(new Animal());
}
};
class Dog : public Animal {
public:
Dog(){}
~Dog(){}
std::unique_ptr<Animal> create() override
{
return std::unique_ptr<Animal>(new Dog());
}
};
PYBIND11_PLUGIN(animal) {
module m("animal", "pybind11 example plugin");
class_<Animal> animal(m, "Animal");
animal
.def(init<>())
.def("create", &Animal::create);
class_<Dog> dog(m, "Dog");
dog
.def(init<>())
.def("create", &Dog::create);
return m.ptr();
}//Compiler error!
class Animal {
public:
virtual ~Animal() { }
virtual std::unique_ptr<Animal> create()=0;
};
class PyAnimal : public Animal {
public:
using Animal::Animal;
std::unique_ptr<Animal> create() override {
PYBIND11_OVERLOAD_PURE(
std::unique_ptr<Animal> ,
Animal,
create,
);
}
};
class Dog : public Animal {
public:
Dog(){}
~Dog(){}
std::unique_ptr<Animal> create() override
{
return std::unique_ptr<Animal>(new Dog());
}
};
PYBIND11_PLUGIN(animal) {
module m("animal", "pybind11 example plugin");
class_<Animal, PyAnimal> animal(m, "Animal");
animal
.def(init<>())
.def("create", &Animal::create);
class_<Dog> dog(m, "Dog");
dog
.def(init<>())
.def("create", &Dog::create);
return m.ptr();
} |
|
The fundamental problem is that there isn't any way to get a unique_ptr to an instance returned by Python: the unique_ptr is an internal pybind11 holder created when the returned object was created in python (i.e. before the value gets returned). Pybind can't give up the unique_ptr because its tied to the Python instance. So while returning a std::unique_ptr from a C function is fine, it isn't going to work for a Python override. The easiest solution is to provide a custom wrapper around the parent method that returns a raw pointer instead, which will work for both overloaded and non-overloaded cases: class Animal {
public:
virtual std::unique_ptr<Whatever> f() { /* ... */ }
virtual std::unique_ptr<Whatever> pure() = 0;
};
class PyAnimal : public Animal {
public:
Whatever * f_wrapper() {
PYBIND11_OVERLOAD_INT(Whatever *, Animal, "f", /* extra args here, if any */);
return Animal::f().release();
}
Whatever * pure_wrapper() {
PYBIND11_OVERLOAD_PURE(Whatever *, Animal, pure);
}
};
// ... later, in binding code:
// virtual function that releases the returned pointer (if not overloaded):
animal.def("f", &PyAnimal::f_wrapper);
animal.def("pure", &PyAnimal::pure_wrapper);This works because, when pybind gets the raw pointer, the first thing it's going to do is a lookup to see if it's already stored internally; if it is, it'll use the already-known instance, and if it isn't it'll construct a new holder around the pointer. In essence what that means is that if there is no overload, pybind is going to get a pointer that it doesn't know about, and so construct a new unique_ptr around it; if it comes from Python, it's going to get a pointer it already knows about, and will thus reference that internal object. |
|
Thanks so much for your time and reply. The solution does not apply to our implementation(we have to return a unique pointer to the same type animal.cpp #include <string>
#include <iostream>
#include <pybind11/pybind11.h>
using namespace pybind11;
class Animal {
public:
virtual ~Animal() { }
virtual void call()=0;
virtual std::shared_ptr<Animal> create() { return nullptr; };
public:
std::string name;
};
class PyAnimal : public Animal {
public:
using Animal::Animal;
std::shared_ptr<Animal> create() override {
PYBIND11_OVERLOAD(
std::shared_ptr<Animal>,
Animal,
create,
);
}
void call() override {
PYBIND11_OVERLOAD_PURE(
void,
Animal,
call,
);
}
};
void testing(Animal & animal) {
auto t = animal.create();
t->call();
}
PYBIND11_PLUGIN(animal) {
module m("animal", "pybind11 example plugin");
class_<Animal, PyAnimal, std::shared_ptr<Animal> > animal(m, "Animal");
animal
.def(init<>())
.def("call", &Animal::call)
.def("create", &Animal::create)
.def_readwrite("name", &Animal::name);
m.def("testing", &testing);
return m.ptr();
}test.py from animal import Animal, testing
class A(Animal):
def __init__(self):
super(A, self).__init__()
self.b = A.B()
def call(self):
print('a')
def create(self):
return A.B()
class B(Animal):
def __init__(self):
super(A.B, self).__init__()
def call(self):
print('b')
def create(self):
return A()
testing(A())error: Is it has something to do with holder type for And the strange thing is if I set a breaking point just before from ipdb import set_trace; set_trace()and step into this Do you know what might cause this problem? Why step into the function help to remove the error? And what is the correct way to understand the error message? Thanks so much for your time and help. From my understanding the #include <string>
#include <pybind11/pybind11.h>
using namespace pybind11;
class Animal {
public:
virtual ~Animal() { }
virtual void call()=0;
virtual std::unique_ptr<Animal> create() { return nullptr; };
public:
std::string name;
};
class PyAnimal : public Animal {
public:
using Animal::Animal;
Animal* create_wrapper() {
PYBIND11_OVERLOAD_INT(Animal *, Animal, "create", /* extra args here, if any */);
return Animal::create().release();
}
void call() override {
PYBIND11_OVERLOAD_PURE(
void,
Animal,
call,
);
}
};
void testing(Animal & animal) {
auto t = animal.create();
t->call();
}
PYBIND11_PLUGIN(animal) {
module m("animal", "pybind11 example plugin");
class_<Animal, PyAnimal, std::shared_ptr<Animal> > animal(m, "Animal");
animal
.def(init<>())
.def("call", &Animal::call)
.def("create", &PyAnimal::create_wrapper)
.def_readwrite("name", &Animal::name);
m.def("testing", &testing);
return m.ptr();
}trace back: |
|
My mistake: I forgot that, of course, you'll also need an actual override of the You were on the right track with (You can see this working if you return a reference instead of a new instance, e.g. returning If returning references to existing objects won't work for you, your other option is to deal with storing the python objects as long as you need to. Basically, you'll have to hold the {
pybind11::gil_scoped_acquire gil; // Only necessary if this could be called from gil-released code
pybind11::function overload = pybind11::get_overload(static_cast<const Animal *>(this), "create");
if (overload) {
object o = overload();
auto shptr = pybind11::cast<std::shared_ptr<Animal>>(o);
// Virtual calls on shptr will work here (as long as `o` persists)
}
} |
|
Thanks so much for your help. Sorry for my poor understanding, would you mind elaborate on "hold the class PyAnimal : public Animal {
public:
using Animal::Animal;
std::shared_ptr<Animal> create() override {
pybind11::gil_scoped_acquire gil;
pybind11::function overload = pybind11::get_overload(static_cast<const Animal *>(this), "create");
object o = overload();
auto shptr = pybind11::cast<std::shared_ptr<Animal>>(o);
return shptr->create();
}
...
Sorry, I found a workaround for the local reference solution, it works just like magic! Thanks so much for you time! |
Sorry to bug you, I was trying to override a virtual function and return a unique pointer in that function, like a factory method:
The code above failed to compile and the error message shows:
And the same if I substitute
std::unique_ptr<Animal>to typedef.Is the right way to use pybind11? Did I miss something?
Thanks so much for your help.
The text was updated successfully, but these errors were encountered: