how to write c++ functions that accept numpy arrays with custom numeric types exposed via EigenPy

[ofloveandhate]

I'm looking for a bit of help.

I have C++ functions that accept Eigen vectors and matrices. And those functions are exposed to Python via Boost.Python, as are my numeric types, and I have exposed Eigen containers of my numeric types via EigenPy. So I think this is an eigenpy question.

But I'm struggling to provide inter-operability between the Eigen vectors for C++ calls and numpy containers with dtype my custom numeric type.

Do you have examples in eigenpy or pinnochio of a C++ function accepting Eigen data, but which when exposed to Python accepts numpy?

---

super cut-down example code

// in C++
def foo(Eigen::Vector v){
}

foo(Eigen::Vector(blabla))

# in Python

foo(np.array( [N(1), N(2), N(3) ])

where N is my wrapped custom numeric type.

But the call in Python fails because the C++ function expects an Eigen container, but it's getting a numpy container.

My specific error message:

Traceback (most recent call last):
  File "/Users/amethyst/github_repos/b2/python/test/tracking/amptracking_test.py", line 133, in test_tracker_quad
    tracker.track_path(y_end, t_start, t_end, y_start);
Boost.Python.ArgumentError: Python argument types in
    AMPTracker.track_path(AMPTracker, numpy.ndarray, Complex, Complex, numpy.ndarray)
did not match C++ signature:
    track_path(bertini::tracking::AMPTracker {lvalue}, Eigen::Matrix<boost::multiprecision::number<boost::multiprecision::backends::mpc_complex_backend<0u>, (boost::multiprecision::expression_template_option)1>, -1, 1, 0, -1, 1> {lvalue} result, boost::multiprecision::number<boost::multiprecision::backends::mpc_complex_backend<0u>, (boost::multiprecision::expression_template_option)1> start_time, boost::multiprecision::number<boost::multiprecision::backends::mpc_complex_backend<0u>, (boost::multiprecision::expression_template_option)1> end_time, Eigen::Matrix<boost::multiprecision::number<boost::multiprecision::backends::mpc_complex_backend<0u>, (boost::multiprecision::expression_template_option)1>, -1, 1, 0, -1, 1> start_point)

[ofloveandhate]

this SO post suggests copying in the data for the python --> c++ direction, but i dislike this. there should be a way to do it directly, to provide an overload / interoperability layer.

[ofloveandhate]

related open question, i think: https://stackoverflow.com/questions/76065711/converting-pyarray-to-eigenref-from-c

[ofloveandhate]

one more closely-related SO question, this time using eigency (but sadly not eigenpy)

[jcarpent]

Please look at this test

eigenpy/unittest/user_type.cpp

Lines 134 to 138 in e945a06

	template <typename Scalar>
	void print(const Eigen::Matrix<CustomType<Scalar>, Eigen::Dynamic,
	Eigen::Dynamic>& mat) {
	std::cout << mat << std::endl;
	}

[ofloveandhate]

Ooh, discussions! Thanks for moving this thread. I'll take a look at the example you indicated, thank you very much.

[ofloveandhate]

I don't think the selected answer solves the issue. It did lead me to add the following line of code, which helped some of the problems:

eigenpy::EigenFromPyConverter<Mat<T>>::registration();

BUT, I am still unable to pass by non-const reference.

[ofloveandhate]

I need (would strongly like) to be able to pass by non-const reference, in such a function

void track_path(Vec<T> & result, Vec<T> const& start);

I have const& working, but non-const & is not working. i'm missing something...

---

edit:

These functions worked in my Python side when I was using minieigen to expose Eigen types to Python. But then I lacked the numpy interoperability, which is a huge loss. So I switched.

Now with eigenpy I no longer have the eigen types directly in Python, and this causes all of my pass-by-mutable-reference functions to break. and I pass by mutable reference a LOT in my library. so getting it back without having to re-wrap all these functions would be awesome.

[ofloveandhate]

I am starting to suspect that I should be using Eigen::Ref<> and not non-const & in these functions...

Is that what this line of code is about?

[jcarpent]

You are right. Ref should be used to share properly the memory. While const reference is copy pasting quantities.

[ofloveandhate]

I played with it in the meantime. Switching my codebase from

template<typename T>
foo(Vec<T> & A)

to

template typename<T>
foo(Eigen::Ref<Vec<T>> A)

is causing all sorts of headache for me... Ugh.

[ofloveandhate]

I was able to come up with a workaround. It's not optimal. But it works.

I wrap the functions i want to expose to Python that pass by non-const-reference from the C++ side in the code that produces my bindings, like this:

# in hpp side, in my visitor
static
void rescale_wrap_inplace_mpfr(SystemBaseT const& self, Eigen::Ref<Vec<mpfr>> x){
    Vec<mpfr> result(x);
    self.RescalePointToFitPatchInPlace(result);
    x = result;}

#  in .cpp code
.def("rescale_point_to_fit_patch_in_place",&rescale_wrap_inplace_mpfr,"Re-scale the input point, in place, to fit the patch for the system.  This assumes you have properly set the variable groups and auto-patched the system.")

It lets me call my core code without changing to Eigen::Ref (which I will do, just not now) from Python via Eigenpy.

The cost:

• a copy into the temp result
• a copy back into the variable x

So definitely not good. But it will at least keep me from crying for now.

---

Bottom line: use Eigen::Ref to pass by non-const-reference. Eigenpy relies on it.

[ofloveandhate]

(i'm sorry, these should have been replies. this github feature is new to me)