Strange zeros in returned ndarray on python side that is correct on C++ side

[Chronum94]

I get leading zeros in arrays once they are returned to Python, and I know that these errant zeros don't exist on the C++ side (at least where I print out the vector elements).

m.def(
      "eigh",
      [](slate::HermitianMatrix<double> &A)
      {
        std::vector<blas::real_type<double>> eps(A.m());

        slate::heev(A, eps);
        auto rank = 0;
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
        if(rank == 0){
          for(auto i = 0; i < eps.size(); i++) {
            std::cout << i << " " << eps[i] << std::endl;
            if (i > 5)
              break;
          }
        }

        auto size = eps.size();
        size_t shape[1] = {size};

        return nb::ndarray<nb::numpy, double, nb::shape<nb::any>>(eps.data(), 1, shape);
      }
      // nb::rv_policy::copy
      );

This is what code looks like on the C++ side of things. I've included the full code snippet here since the last time I omitted things, the omitted details turned out to be important...

On running this from the python side, here is approximately the pattern of error I see:

C++
-15.9604
-15.6574
-15.1961
-14.1617
-13.9817
-13.613
-13.0724

Python:
0.0000     0.0000   -15.1961   -14.1617   -13.9817   -13.6130   -13.0724

The nb::rv_policy::copy is commented out, but it doesn't seem to make a difference.

I've been trying to sort this out for the past few days by looking at the docs and trying to see if I've missed anything, and I've not made much progress.

Any help or pointers to specfic parts of the docs that may be helpful with this is greatly appreciated.

[wjakob]

Please read the nd-array documentation carefully with respect to the owner argument.

[Chronum94]

Hi,

I've had a look at the documentation and have written up a potential solution. However the solution 'looks not great' which is my best feeling about it. I understand that this is an aesthetic/best practices question, with potentially no obvious right answer.

  m.def(
      "eigh",
      [](slate::HermitianMatrix<double> &A)
      {

        struct Temp {
          std::vector<blas::real_type<double>> eps;
        };

        Temp *temp = new Temp();

        slate::heev(A, temp->eps);

        auto size = temp->eps.size();
        auto epsdata = temp->eps.data();
        size_t shape[1] = {size};

        nb::capsule owner(temp, [](void *p) noexcept {
          delete (Temp *) p;
        });

        return nb::ndarray<nb::numpy, double, nb::shape<nb::any>>(temp->eps.data(), 1, shape, owner);
      }
      );

The above piece of code does the needful in that the strange zeros are gone, but I was wondering if you had recommendations on any way to clean it up. The slate::heev function takes in specifically a std::vector type. I've tried putting together solutions involving capsuling the std::vector directly, but those result in failures due to double frees that I do not understand how to fix.

If the above code cannot be made to look more idiomatic (or if you do not have the time to really go through it), please let me know and I'll be happy to accept this as the answer.

[jamesETsmith]

@Chronum94, thanks for figuring this out, I was struggling with the same problem. It's not clear to me why we need the indirection of the struct Temp, but my attempts to use std::vector weren't successful and my guess is that calling delete (std::vector<> *) data inside of nb::capsule owner is not playing nice in a way that's not obvious to me.