Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing for SDFormat camera sensors #19155

Closed
jwnimmer-tri opened this issue Apr 7, 2023 · 4 comments
Closed

Parsing for SDFormat camera sensors #19155

jwnimmer-tri opened this issue Apr 7, 2023 · 4 comments
Assignees
Labels
component: geometry perception How geometry appears in color, depth, and label images (via the RenderEngine API) priority: backlog type: feature request

Comments

@jwnimmer-tri
Copy link
Collaborator

SDFormat has a standard way of adding cameras to a scene with the sensor element on a link. It would be wonderful if Drake could support that, at least for the basics.

@jwnimmer-tri jwnimmer-tri added type: feature request component: geometry perception How geometry appears in color, depth, and label images (via the RenderEngine API) labels Apr 7, 2023
@jwnimmer-tri jwnimmer-tri self-assigned this Apr 7, 2023
@jwnimmer-tri
Copy link
Collaborator Author

jwnimmer-tri commented Apr 7, 2023

Rough design proposal:

(1) The parser converts the XML specification to a geometry.render.ColorRenderCamera (or DepthRenderCamera).

(2) On the MbP body (i.e., link), the parser registers a superfluous geometry instance with the perception role (only), something like a box geom set to zero alpha. Within its perception properties, we store the above item, keyed as something like (group_name="parser", name="color_render_camera").

(3) We provide a function that takes a Diagram and adds the RgbdSensor systems. It would scan all geometries for the magic "parser" property, and when it finds any such property it would add a System. It would return the collection of sensor output ports, somehow keyed or indexed by the original SDFormat sensor names.

@jwnimmer-tri
Copy link
Collaborator Author

@SeanCurtis-TRI when you have some time, I'd appreciate any feedback on the principle of using GeometryProperties to communicate sensors from the Parser to an RgbdSensor.

The other alternative would be passing the full DiagramBuilder pointer to the Parser, so it can add entire farms of new Systems instead of just MbP elements and geometries.

@SeanCurtis-TRI
Copy link
Contributor

Without pondering too long, it feels hacky. There's not actually a geometry with properties. We'd be turning geometry into a message passing service.

Generally, I'd vote for the parser being less MbP-centric if we're parsing formats that represent concepts that are more than MbP.

If we're looking for less churn on the current API, we can modify the idea of two passes (parser does its work, collecting additional data, followed by some other entity that discovers and operates on the additional data). The parser could collect all additional information in an explicit struct and the second pass can operate explicitly on that -- no discovery necessary.

@jwnimmer-tri
Copy link
Collaborator Author

At this point, I think we're happy with configuring cameras outside of SDFormat. For one, we load models from a wide variety of formats so SDFormat is nothing particularly special. But more importantly, there's a few more details about a camera than SDFormat denotes (e.g., capture offset, output delay, LCM topic, compression) so we would need to keep the yaml-based camera config in any case. We can re-open if we find the need to do this later.

@jwnimmer-tri jwnimmer-tri closed this as not planned Won't fix, can't repro, duplicate, stale Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: geometry perception How geometry appears in color, depth, and label images (via the RenderEngine API) priority: backlog type: feature request
Projects
None yet
Development

No branches or pull requests

2 participants