Common APIs across array libraries (1 year later)

[kgryte]

Overview

Similar to gh-6, this issue looks to identify commonalities across array libraries, but only addresses those APIs which are not already included in the array API specification.

Since gh-6 and its analysis,

1. Every array library added more APIs.
2. Continued convergence toward NumPy APIs (CuPy, PyTorch, TensorFlow.experimental, MXNet).
3. Greater agreement among accelerator libraries (e.g., CuPy, MXNet, Torch, TF) wrt special functions not available in NumPy, but available in SciPy.

Method

Similar to gh-6, the list was compiled by doing the following:

1. Generating a list of APIs based on publicly documented array APIs (e.g., by scraping website documentation).
2. Computing the intersection across the individual datasets.

The following libraries were analyzed:

• numpy
• cupy
• dask.array
• jax
• mxnet
• pytorch
• tensorflow

APIs

The following APIs were found to be common across the above libraries, but not already included in the array API specification:

cbrt
clip
copysign
count_nonzero

deg2rad
diff

erf (scipy)
erfc (scipy)
erfinv (scipy)
erfcinv (scipy)
exp2

gamma (scipy)
gammaln (scipy)

histogram
hypot

i0 (bessel)

logsumexp (accelerators)

nextafter

pad

rad2deg
reciprocal
repeat
rot90
rsqrt (accelerators)
rcbrt (accelerators)

sigmoid (accelerators)

take
tile
top_k (accelerators+dask)

xlogy (scipy)

We can split the APIs into the following categories...

Array Manipulation

pad
repeat
rot90
tile

Special Functions

cbrt
clip
copysign
deg2rad
erf
erfc
erfcinv
erfinv
exp2
gamma
gammaln
hypot
i0
nextafter
rad2deg
reciprocal
rsqrt
rcbrt
sigmoid
xlogy

Reductions

count_nonzero
histogram
logsumexp
top_k

Indexing

take

Other

diff

Next Steps

• Identify which APIs could be suitable candidates for standardization in the next version of the specification.

[rgommers]

Someone just proposed to add topk (or top_k) to NumPy, and I'm trying to find the best way to use the data that generated the above list. It looks like none of the make ... commands in https://github.com/data-apis/array-api-comparison show this?

[rgommers]

This issue is still useful; I'll remove the v2022 milestone given that we're done adding new APIs to that.

The list of common APIs here is probably longer than the list of things that make sense to add. The content here can be used as reference - one data point in future API extension conversations.

[TomNicholas]

FYI of this list xarray currently uses:

np.clip
np.diff
np.pad
np.repeat
np.take
np.tile

[rgommers]

Thanks @TomNicholas. take is implemented, and for clip there's gh-482, overall my reading is that we'll add clip for the next version.

The others need looking into, but seem to me to me among the most-often used numpy functions that we haven't included yet. repeat and tile are straightforward to implement, diff isn't too bad either, pad is a bit painful with its many options.

[tirthasheshpatel]

SciPy has a usecase for the nextafter function which is present in this list. Since it is a very basic function and implemented by all the array libraries, I think it should be added to the Array API.

[rgommers]

SciPy has a usecase for the nextafter function which is present in this list. Since it is a very basic function and implemented by all the array libraries, I think it should be added to the Array API.

Thanks for the proposal @tirthasheshpatel! That seems reasonable, and now that there's an identified need we can prioritize it. @steff456 volunteered to dig into this one.

[oleksandr-pavlyk]

What about binary elementwise functions that give smallest/largest of the two?

These can be implemented with where and less, sure, but direct implementation is much faster for non-JIT-ting implementations.