PointCloud XYZRGBL display draws points in incorrect z position relative to camera

[heuristicus]

This may be related to #1184, which I reported recently.

This is using rviz 1.12.16.

When visualising pointclouds with the XYZRGBL type, points which should be obscured by points in front of them from the current camera perspective are visible.

You can check this by starting a roscore, running rviz, adding a PointCloud2 display subscribed to cloud_pcd, and then running

rosrun pcl_ros pcd_to_pointcloud out.pcd 1

With the following pcd file:

out.pcd.zip

You will then have to set the colour transformer to intensity, and switch to an axis or label colouring (i.e. channel name set to x,y,z or label).

The first picture is a view of the cloud with label colouring, from approximately the same perspective in which it was taken. Blue is wall, pink is furniture and clutter, orange is the floor.

This picture from a different perspective shows the orange floor being displayed in front of the pink furniture/clutter, despite it actually being behind those points from the perspective of the camera.

This is another example, this time with the camera perspective behind the wall. From this view, it should only be possible to see the wall, but actually we can see both the floor and the furniture.

I wondered if this was related to the pointcloud being unorganised, so I got some raw data from the camera and tried to fake an unorganised cloud by simply changing the header from

WIDTH 640
HEIGHT 480

To

WIDTH 307200
HEIGHT 1

I'm not sure if this is the only change needed for the cloud to be read in as unorganised, but the effect does not appear when using this cloud. That may also be because it is just XYZRGB instead of XYZRGBL, which is another indicator that it might have something to do with the label part specifically.

unorganised_fake.zip

[xqms]

Happening here as well with XYZRGB clouds.

[heuristicus]

I noticed something potentially relevant while working with this cloud.

Rviz version:

[ INFO] [1583939036.276409936]: rviz version 1.13.7
[ INFO] [1583939036.276451382]: compiled against Qt version 5.9.5
[ INFO] [1583939036.276463120]: compiled against OGRE version 1.9.0 (Ghadamon)

Here's a gif of the effect in addition to the images in my previous message for a better understanding of the issue:

We place the camera in a position which produces the issue.

We make a selection

The points which are selected are the correct ones - the selection does not include the points which appear to be in the foreground but are actually in the background.

This seems to indicate that the point selection tool is correctly checking the Z-values of points relative to the camera, but the rendering is not.

[rhaschke]

Is it known that a past version of rviz did not have the issue? If so, I could track down the offending commit with git bisect.

[heuristicus]

I downloaded the ros:jade docker and installed ros-jade-rviz, which gave me:

[ INFO] [1583940262.467204132]: rviz version 1.11.18
[ INFO] [1583940262.467238280]: compiled against Qt version 4.8.6
[ INFO] [1583940262.467245964]: compiled against OGRE version 1.8.1 (Byatis)

The issue is present in that version as well.

I checked out version 1.11.10 in the same docker and tested, the issue was also present there.

Version 1.11.8 appears to be the first version in which RGB8 is available as a colour transformer on PointCloud2, and the problem is present there also.

I tried version 1.11.6, but the RGB8 colour transformer for PointCloud2 does not seem to be available there. Maybe it could be tested with a PointCloud topic but I'm using pcl_ros pcd_to_pointcloud which only does PointCloud2.

[rhaschke]

Thanks for this validation. So it looks like, the bug was there from the very beginning.