2D view of multiple DepthImages doesn't update with time scroll

[eskjorg]

Describe the bug
2D view with multiple DepthImages doesn't show the correct image when dragging the slider.

But hovering still works, showing the correct pixel for each layer. This indicates that the correct data is there.

To Reproduce

for i, depth_image in enumerate(depth_arr):
    rr.log(
        f"{phase}/depth_imgs/{i}",
        rr.DepthImage(depth_image, meter=1000.0, colormap=Colormap.Viridis),
    )

Then press play or drag slider through time.

Expected behavior
The 2D view should show each corresponding image when scrolling.

Screenshots

This is bad:

screen_rec.mov

This is good:

Desktop
macOS Sonoma 14.5 (23F79)

Rerun version
rerun-cli 0.18.2 [rustc 1.76.0 (07dca489a 2024-02-04), LLVM 17.0.6] aarch64-apple-darwin release-0.18.2 59ff15b, built 2024-08-29T13:55:42Z

[Wumpf]

Interesting find! So actually DepthImage is not supposed to support (transparent) layering at all: this is ever so slightly hinted by the fact that only Image and SegmentationImage come with an optional Opacity component.

This means that a depth image in front will always hide everything behind it. So unless you log new data to the front-most entity, you won't see anything.
Which in turn means that the actual bug here is unfortunately the thing you called out working: Multiple previews should not show the hover interaction is meant to be disabled for fully hidden things (just like a mesh being drawn in front of a point doesn't allow hovering the point).
Note also that the default DrawOrder is stable, but undefined, so you can't be sure which image is in front unless you specify it yourself.

But obviously your intention here is to get information out of several images so I'm wondering what you would like this to look like. Can you elaborate a bit on that?

So far we considered it not meaningful to have transparent layering on depth images - I have a hard time understanding how to get meaningful information out of such a visualization. Typically in those cases you'd either keep logging to the same entity path (then at every point in time you have a different image being shown) or, if all of them should show at once, you'd create more Views via a blueprint.