Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix batched cov transform #541

Merged
merged 4 commits into from
Dec 19, 2023
Merged

Fix batched cov transform #541

merged 4 commits into from
Dec 19, 2023

Conversation

tbensonatl
Copy link
Collaborator

Fix a few issues with batched cov() transforms:

  • Use transpose_matrix instead of transpose for the batched transposed deviations. Thus, dimensions [0, RANK-2) match the sizes of the batched dimensions from the non-transposed deviations.
  • Use stream-oriented async allocators for temporaries and set the memory space to MATX_ASYNC_DEVICE_MEMORY.
  • Make some changes in dimension-checking for batched matmul() transforms. Compute the number of batches using the dimensions of C and, if necessary, broadcast A or B into the batched dimensions.
  • Add a cov() unit test case with batching and rectangular inputs.

One remaining issue is that we allocate 3-4 times the size of the input for temporary storage. This storage is part of a cached plan that currently is not evicted/freed, so if the dimensions of cov() inputs are changing over time, the cache will grow quickly.

Use transpose_matrix() rather than transpose() when creating
the batched tranposed matrices for cov(). Also allow broadcasting
for the matmul() transforms used for batched cov().
@cliffburdick
Copy link
Collaborator

/blossom-ci

@cliffburdick
Copy link
Collaborator

@tbensonatl looks like an example was removed:

Warning, treated as error:
/home/jenkins/workspace/unit-tests/docs_input/api/manipulation/rearranging/transpose.rst:15:start-after pattern not found: example-begin transpose-test-1

@tbensonatl
Copy link
Collaborator Author

/blossom-ci

@tbensonatl
Copy link
Collaborator Author

/blossom-ci

@tbensonatl tbensonatl merged commit b182a72 into main Dec 19, 2023
1 check passed
@tbensonatl tbensonatl deleted the fix-batched-cov-transform branch January 4, 2024 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants