Move camera optimization out of datamanager and parallelize dataloading #2092
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
https://github.com/nerfstudio-project/nerfstudio/blob/justin/camera_opt_refactor/nerfstudio/pipelines/base_pipeline.py#L363 may be fixed to |
…ere the messages are coming from, add default optimizer to optimizers.py for camera_opt param group to avoid crashes
|
I think this is ready for merge. Tested with Lerf repository out of the box, and warnings appeared as they should without breaking the pipeline. Users would have to pip install nerfstudio once again after this update, since viser version has been bumped up. |
brentyi
approved these changes
Sep 24, 2023
|
pytest failed :/ |
dmholtz
added a commit
to dmholtz/nerfstudio
that referenced
this pull request
Nov 29, 2023
The ParallelDataManager (see nerfstudio-project#2092) makes pytorch crash if the cameras instance's fx, fy, cx or cy tensors are loaded from a common shared tensor
dmholtz
added a commit
to dmholtz/nerfstudio
that referenced
this pull request
Nov 29, 2023
The ParallelDataManager (see nerfstudio-project#2092) makes pytorch crash if the cameras instance's fx, fy, cx or cy tensors are loaded from a common shared tensor. This PR fixes the issue by cloning the respective tensors before passing them to the Cameras(...) constructor.
dmholtz
added a commit
to dmholtz/nerfstudio
that referenced
this pull request
Nov 29, 2023
The ParallelDataManager (see nerfstudio-project#2092) makes pytorch crash if the cameras instance's fx, fy, cx or cy tensors are loaded from a common shared tensor. This PR fixes the issue by cloning the respective tensors before passing them to the Cameras(...) constructor.
dmholtz
added a commit
to dmholtz/nerfstudio
that referenced
this pull request
Nov 30, 2023
The ParallelDataManager (see nerfstudio-project#2092) makes pytorch crash if the cameras instance's fx, fy, cx or cy tensors are loaded from a common shared tensor. This PR fixes the issue by cloning the respective tensors before passing them to the Cameras(...) constructor.
brentyi
pushed a commit
that referenced
this pull request
Nov 30, 2023
The ParallelDataManager (see #2092) makes pytorch crash if the cameras instance's fx, fy, cx or cy tensors are loaded from a common shared tensor. This PR fixes the issue by cloning the respective tensors before passing them to the Cameras(...) constructor.
This was referenced Feb 23, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WIP for removing camera optimization from datamanger and moving it to the model. This will potentially allow significant speedups in dataloading as several people have already noted, since the most time consuming step in dataloading is ray generation, which currently depends on gradients from the camera optimizer and thus cannot be parallelized.
Currently this
Things that need to happen: