Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Automatic circular camera path rendering #3314

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

ginazhouhuiwu
Copy link
Contributor

Interface to generate a circular camera path! Object avoidance is WIP, but a vanilla circular path for now can be useful for object comparisons.

Screen.Recording.2024-07-17.at.4.55.47.PM.1.1.mov

TODO:

  • Fix the clicking. You can see that the clicking depth is kind of wonky. maybe a draggable sphere to indicate the center could be nice?
  • Add custom origin point input option so users can recreate all the params for a generate path easily

fars=torch.tensor(100).view(1, 1),
).to("cuda")

# Get the distance/depth to the intersection --> calculate 3D position of the click
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start! but this part should use the more general get_outputs_for_camera() function (a lot of the sampling, scaling stuff is method-specific to nerfacto). As long as the method outputs a 'depth' value it should work with this ray deprojection approach

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way, you would render a depth image from the viewer camera and deproject the click point with the intrinsics matrix + rendered depth, meaning it doesn't matter what the rendering backend is.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to note is that the 'depth' in splatfacto is actually z-depth and not ray-depth, so the math would need to be different for the two methods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok that makes sense! Thanks for the feedback appreciate it!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants