Lag in point cloud streaming

[Silvia-Fatt]

Hello,

I have two PCs, one with Ubuntu 20.04 and a RealSense D435 publishing on ROS, the other with Windows 11 Pro and Unity 2022.3.4f1. I'd like to see the live stream of the point cloud from the RealSense in Unity. I'm using ROS-TCP-Connector to enstablish the connection between ROS and Unity and the Visualizations Package to visualize the incoming informations from ROS. I've managed to get the live stream of the point cloud by selecting the topic /camera/depth/color/points in the HUD of the Visualizations Package and I have changed some parameters in sensor_msgs/PointCloud2 to have a good size of the points and to see their colours.

The problem is that when I press Play in the Unity Editor, I often see quite a big lag in the stream of the point cloud (sometimes even around one minute). The amount of lag is a bit random: sometimes it remains constant in time, while sometimes it decreases in time. Also, I have noticed that if I press Stop in the Editor and I press Play again, without killing the processes in ROS (roslaunch realsense2_camera rs_camera.launch filters:=pointcloud and roslaunch ros_tcp_endpoint endpoint.launch), I have more lag in the beginning of the "new" stream. Therefore, I would like to know the reason behind this behaviour (is there a kind of queue of data that gets accumulated?).
I also think that the lag could be caused by the computer performance. Which are the right computer specs to have a nice and smooth stream with a set-up like this?

Environment:

• Unity Version: Unity 2022.3.4f1]
• Unity machine: Windows 11 Pro
• ROS machine: Ubuntu 20.04.6, ROS Noetic
• Camera: RealSense D435

[alextdbc]

Hello,

im facing the same issue. Did you already found a solution for the problem. I would be really happy if you could help me.