-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#0: Use small vector instead of std::vector for shapes to optimize allocations #14281
base: main
Are you sure you want to change the base?
Conversation
e7f4bde
to
b06ec4d
Compare
Question on if we should push the SmallVector wrapper to tt_metal instead of ttnn? |
I think this is a great idea @sminakov-tt wdyt? |
@davorchap @tt-aho Yeah, I think we need small_vector everywhere :) The SmallVector wrapper over boost::small_vector can't really be pushed to tt_metal though. It has a python binding, allowing us to pass it to/from python. This binding must be specified in the same file as SmallVector itself, otherwise template resolution chaotically loses this overload and chaotically fails on method-per-method basis. And I don't think we should introduce python binding into tt_metal in any way. |
Ticket
Problem description
Currently we use std::vector to represent shapes, which causes a lot of allocation. However, all of the shapes are relatively small in practice, so instead we can use boost::small_vector, which pre-allocates a few elements on stack, avoiding allocations for the small vectors
What's changed
Introduced ttnn::SmallVector, a wrapper over boost::small_vector
Went through the whole codebase, and replaced std::vector, which was referring to a shape with either ttnn::SmallVector or std::span where appropriate
[🤓 PERF IMPACT INFO IN-PROGRESS]
Checklist