This repository has been archived by the owner on Mar 15, 2024. It is now read-only.
Gradient accumulation code #240
Comments
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Can you provide the gradient accumulation code? Since I only have 1 gpu for training, the max batch size that I can use is 256, and the final result can not match paper's result. So I'm thinking about doing gradient accumulation, thanks!!!
The text was updated successfully, but these errors were encountered: