-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Training freezes and gets stuck #6
Comments
Check your GPU memory, in some cases during training it goes haywire and GPU memory bust. I got this error while trying different combinations of parameters. |
Can you please specify the setup and the shape you are trying to run and also your environment? |
Hi, this is my conda env running on windows 10 cuda toolkit 11.8
|
one reason might be the relocation stage, it takes long time if there are a lot of "dead gaussian" in the scene (i suppose, not sure). I recommend lower the max_cap from 1M to some lower values. for me i just use 100K for simple scene. that might help. |
I had a similar issue and this fix resolved it for me. |
Thanks for the suggestion, I already switched to using the implementation in nerfstudio |
Training freezes and gets stuck, happens at low iterations
The text was updated successfully, but these errors were encountered: