-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: support typetracer in ak.unflatten
#2293
Conversation
Codecov Report
Additional details and impacted files
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We talked about this today, and I thought that if threading a typetracer through the whole function is difficult, it could be a candidate for a specialized path. But it looks like you did it!
I don't see any tests, though. (Tests for the specific case of ak.unflatten
with typetracers.) Do they come up in dask-awkward?
current_offsets = backend.index_nplike.empty(len(counts) + 1, dtype=np.int64) | ||
current_offsets = backend.index_nplike.empty(counts.size + 1, dtype=np.int64) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So... counts
was originally obtained via to_layout
(layouts have .length
, not .size
), but then it was replaced by to_backend_array
. I assume that means it could be NumPy, CuPy, a JAX DeviceArray, or a TypeTracerArray. All of these have a .size
?
For a NumPy array, the .size
is the product of shape
, not its first element. Is there any way that the NumpyArray might be multidimensional? I suppose not, if it's a valid counts
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, a previous PR explicitly ensures that counts
is a backend array, all of which have .size
. We have an explicit check on counts.ndim
to ensure that it's 1D!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any tests, though. (Tests for the specific case of ak.unflatten with typetracers.) Do they come up in dask-awkward?
Dask-awkward does test this, because it brought this to my attention from their test suite. However, we should really test it for ourselves (I'm being lazy 🤕). I'll add some tests before we merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✔️ on tests/test_2293_unflatten_typetracer.py
Ah, I noticed a final corner-case: |
This PR fixes assumptions of length and known values that dask-awkward's test-suite fails.