-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add more torch.nn modules #36
Conversation
@davoclavo are you planning to work on this in the near future? The reason I'm asking is that I want to implement a few transformer models and I need the If not, I might pull your implementation out from here into its own commit (keeping you as author of course) if that's OK for you. |
@sbrunk Hi! Thanks for checking. I have implemented some modules already but haven't had time to add documentation and other things. Let me upload what I have and feel free to pick up from there. |
1cc9bea
to
182b774
Compare
Awesome, thanks! |
182b774
to
3f94605
Compare
@sbrunk Done, I just uploaded the changes I had (ModuleList and LayerNorm), feel free to pick up the branch as is and add your changes. |
Move type parameter in all modules from apply to constructor for consistency and compat with Sequential. See sbrunk#47 for details about the current design.
Fix native LayerNorm initialization.
Merge main for tensor setters
Hey @davoclavo I've added tests and examples for the modules and a small fix for I think we should get this merged and tackle the remaining TODOs in later PRs. Would that be ok for you? |
@sbrunk Sorry for the delay, I was traveling and somehow missed the notification. Thanks for the fixes and the merge! The remaining TODOs can definitely be addressed in future PRs |
Implement more modules from https://pytorch.org/docs/stable/nn.html
TODO
export
(akin to ops)given Conversion
is a good approach to replace some extension methods