-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve allgather functions #9649
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good to me.
- Could you please look into the test failure?
- Is there a way to consolidate the interface into something more compact? I'm working on revamping the coll implementation and find it quite difficult to work on at the moment. There are many variants of the interface, some supported by GPU some others are not.
Not sure about the test failure. I can't reproduce it on my desktop, even after downgrade gcc from 12 to 9. Any ideas? Yeah the current interfaces evolved from the original rabit/nccl implementations. Need to invest some additional effort to try to unify them. |
@rongou I think it's a sanitizer error. You can enable it through the cmake flag |
@trivialfis Finally got the CI to pass. Please take another look. Thanks! |
AllgatherV