Ordered PointCloud from RGB-D Camera

[CWAndersn]

I was wondering if any more information could be provided on the support for ordered RGB-D PointClouds for RealSense depth cameras. According to the documentation, support for ordered pointclouds is included by the rgbd-launch ros package. When this package is used, what topic is meant to publish the ordered PointCloud? These are the topics that I see from rostopic list:

/camera/align_to_color/parameter_descriptions
/camera/align_to_color/parameter_updates
/camera/aligned_depth_to_color/camera_info
/camera/aligned_depth_to_color/image_raw
/camera/aligned_depth_to_color/image_raw/compressed
/camera/aligned_depth_to_color/image_raw/compressed/parameter_descriptions
/camera/aligned_depth_to_color/image_raw/compressed/parameter_updates
/camera/aligned_depth_to_color/image_raw/compressedDepth
/camera/aligned_depth_to_color/image_raw/compressedDepth/parameter_descriptions
/camera/aligned_depth_to_color/image_raw/compressedDepth/parameter_updates
/camera/aligned_depth_to_color/image_raw/theora
/camera/aligned_depth_to_color/image_raw/theora/parameter_descriptions
/camera/aligned_depth_to_color/image_raw/theora/parameter_updates
/camera/color/camera_info
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressed/parameter_descriptions
/camera/color/image_raw/compressed/parameter_updates
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/compressedDepth/parameter_descriptions
/camera/color/image_raw/compressedDepth/parameter_updates
/camera/color/image_raw/theora
/camera/color/image_raw/theora/parameter_descriptions
/camera/color/image_raw/theora/parameter_updates
/camera/color/image_rect_color
/camera/color/image_rect_color/compressed
/camera/color/image_rect_color/compressed/parameter_descriptions
/camera/color/image_rect_color/compressed/parameter_updates
/camera/color/image_rect_color/compressedDepth
/camera/color/image_rect_color/compressedDepth/parameter_descriptions
/camera/color/image_rect_color/compressedDepth/parameter_updates
/camera/color/image_rect_color/theora
/camera/color/image_rect_color/theora/parameter_descriptions
/camera/color/image_rect_color/theora/parameter_updates
/camera/color/metadata
/camera/color_rectify_color/parameter_descriptions
/camera/color_rectify_color/parameter_updates
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth/metadata
/camera/depth_registered/points
/camera/extrinsics/depth_to_color
/camera/motion_module/parameter_descriptions
/camera/motion_module/parameter_updates
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/auto_exposure_roi/parameter_descriptions
/camera/rgb_camera/auto_exposure_roi/parameter_updates
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/auto_exposure_roi/parameter_descriptions
/camera/stereo_module/auto_exposure_roi/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates
/diagnostics
/rosout
/rosout_agg
/tf
/tf_static

Unless I'm just missing it, the only additional topics I see here are the /camera/aligned_depth_to_color topics, which publish the aligned depth maps. What am I missing here? Thanks in advance.

[MartyG-RealSense]

Hi @CWAndersn A pointcloud topic associated with the rs_rgbd.launch RGBD launch mode is /camera/depth_registered/points, as described at #10222 (comment)

You can also publish an ordered pointcloud with the rs_camera.launch launch file (whose default is an unordered cloud) by adding the term ordered_pc:=true to the roslaunch instruction. For example:

roslaunch realsense2_camera rs_camera.launch filters:=pointcloud,colorizer align_depth:=true ordered_pc:=true

[CWAndersn]

Thank you! I see the ordered pointcloud in the original topics listed before now. However, when I run the rs_camera.launch file you suggest, I don't see that topic despite indicating that ordered_pc:=true

roslaunch realsense2_camera rs_camera.launch filters:=pointcloud,colorizer align_depth:=true ordered_pc:=true

[CWAndersn]

I believe when using rs_camera.launch the pointcloud is published to the topic /camera/depth/color/points instead. Thank you!

[MartyG-RealSense]

You are very welcome, @CWAndersn - thanks for the update!

[MikeHallettUK]

I do not get an Ordered PointCloud from "roslaunch realsense2_camera rs_rgbd.launch". It runs fine, but the array I get back from the topic /camera/depth_registered/points varies in size. e.g. I set both colour and depth to (480,848,3) so expect an array = 1,221,120... but get e.g. 1015911 or 1044357 etc ...

... I have successfully got the ordered pointcloud with the rs_camera.launch launch file by adding the term ordered_pc:=true to the roslaunch instruction. This works fine, but is not aligned. ( Is there a reason that this is not aligned by default out of the camera ?)...

Thanks. I am using a D455.

Mike

[MartyG-RealSense]

Hi @MikeHallettUK Whilst the rs_rgbd launch file enables depth-color alignment by default, the rs_camera one does not. There is not a provided reason for this. You can enable it by adding the term align_depth:=true to your roslaunch instruction.

roslaunch realsense2_camera rs_camera.launch align_depth:=true filters:=pointcloud ordered_pc:=true

[MikeHallettUK]

Yes, I suspected that. Sorry I did not ask the explicit question... the first point was the main query I had =

I do not get an Ordered PointCloud from "roslaunch realsense2_camera rs_rgbd.launch" = any idea why not ?

I could not see an explicit parameter like "ordered_pc:=true" for rs_rgbd... is there one ?

Thanks.

Mike

[MartyG-RealSense]

The RealSense ROS wrapper does not have built-in support for an RGBD launch and that support must be first installed manually using one of the instructions below, depending on which ROS1 branch you are using. Have you installed this support, please?

Kinetic
sudo apt-get install ros-kinetic-rgbd-launch

Melodic
sudo apt-get install ros-melodic-rgbd-launch

Noetic
sudo apt-get install ros-noetic-rgbd-launch

rs_rgbd.launch should by default generate a point cloud that is published at depth_image_proc and so does not have an option for an ordered cloud like rs_camera.launch has because it does not need it. There is a discussion about this and unorganized versus organized clouds in the rs_camera and rs_rgbd launches at IntelRealSense/realsense-ros#851

Information about depth_image_proc can be found at http://wiki.ros.org/depth_image_proc

[MikeHallettUK]

Yes thanks... ros-noetic-rgbd-launch is installed and runs.

BUT... I do not get an Ordered PointCloud from "roslaunch realsense2_camera rs_rgbd.launch" = any idea why not ?

It runs fine, but the array I get back from the topic /camera/depth_registered/points varies in size. e.g. I set both colour and depth to (480,848,3) so expect an array = 1,221,120... but get e.g. 1015911 or 1044357 etc ...

[MikeHallettUK]

AND ... I have tried "roslaunch realsense2_camera rs_camera.launch align_depth:=true filters:=pointcloud ordered_pc:=true"...

yes, "/camera/depth/color/points" is ordered ... and depth may be aligned...

BUT... the Pointcloud2 "/camera/depth/color/points" does NOT seem to be aligned ?

[MikeHallettUK]

I am using Python .. eg

pc = ros_numpy.point_cloud2.pointcloud2_to_xyz_array(msg)   # Ordered Pointcloud in msg
pc = pc.reshape(msg.height, msg.width, 3)

[MartyG-RealSense]

May I ask which version of the ROS wrapper you are using, please? The developer of the ROS1 wrapper states at IntelRealSense/realsense-ros#1886 (comment) that from wrapper 2.2.24 onwards (which matches with librealsense 2.44.0), using align_depth with rs_camera.launch should also generate a pointcloud that is aligned to color.

[MikeHallettUK]

I was on slightly older than those. Now compiled on my ODROID N2+ to latest versions:
RealSense ROS v2.3.2
Built with LibRealSense v2.50.0

This works well thanks, giving an Aligned and Ordered PointCloud. (i.e. from "roslaunch realsense2_camera rs_camera.launch align_depth:=true filters:=pointcloud ordered_pc:=true")

BUT, I notice the align_depth:=true option, disables the pointcloud filters (e.g. decimation).

Thanks for your help.

Mike

[MartyG-RealSense]

It's great rto hear that you achieved a working setup!

The issue of decimation not working when align_depth is enabled is discussed at IntelRealSense/realsense-ros#2269

[MikeHallettUK]

Useful discussion @ # 2269... but it would still be very nice to have filters (Decimation) working when align_depth:=true.

:)