Ensure objects with __interface__ are converted to cupy/numpy arrays

[mroeschke]

Description

#16277 removed a universal cast to a cupy.array in _from_array. Although the typing suggested this method should only accept np.ndarray or cupy.ndarray, this method is called on any object implementing the __cuda_array_inferface__ or __array_interface__ (e.g. numba.DeviceArray) which caused a performance regression in cuspatial rapidsai/cuspatial#1413

closes #16434

In [1]: import cupy, numba.cuda

In [2]: import cudf

In [3]: cupy_array = cupy.ones((10_000, 100))

In [4]: %timeit cudf.DataFrame(cupy_array)
3.88 ms ± 52 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [5]: %timeit cudf.DataFrame(numba.cuda.to_device(cupy_array))
3.99 ms ± 35.4 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[bdice]

@mroeschke Is there a way we can test this effectively?

[mroeschke]

@mroeschke Is there a way we can test this effectively?

I am having trouble thinking how we could unit test this exactly, but I added a benchmark and a mypy type annotation in _from_array so hopefully those two can "test" this by proxy

[GregoryKimball]

@mroeschke Do you think this should go into 24.08?

[mroeschke]

@mroeschke Do you think this should go into 24.08?

I'm not sure how often users call cudf.DataFrame with a __cuda_array_interface__ that is not a cupy.ndarray. I suppose to be on the safer side to avoid performance regressions we should target for 24.08

[bdice]

I am okay with retargeting this to 24.08 to avoid perf regressions.

[bdice]

@mroeschke You'll need to rebase to remove the branch-24.10 changes.

[jakirkham]

Think we need an admin merge

[jakirkham]

Thanks all! 🙏