Unstable performance when using filters=pointcloud in d435 #1047

jediofgever · 2020-01-06T09:46:38Z

Hi,
When I start camera node with pointclouds enabled, I get very random performance. sometimes it can deliever a frame in 10 seconds and sometimes it performs as expected. I have built realsense with CUDA support and the computer specs are absolutely above required ones. I am using an additional usb cable to increase range to put camera on my robotic arm. I have checked the performance with and without this additional cable and I still have the same issue with original cable. Is there someone who met this problem ?

doronhi · 2020-01-06T12:58:37Z

I haven't met the problem you describe.
Could you begin by specifying the type of computer, OS version (including kernel version - uname -r) and the versions of librealsense2 and realsense2_camera (Printed by the realsense2_camera node)?

I assume you launch the node using roslaunch realsense2_camera rs_camera.launch filters:=pointcloud and measure the frame rate of the point cloud using rostopic hz /camera/depth/color/points. Do you see the frame rate drop from 30Hz (or whatever you specified) to around 1 or less or does the frame rate go up and down?

Can you run roslaunch realsense2_camera rs_camera.launch align_depth:=true and measure the frame rate of the aligned images: rostopic hz /camera/aligned_depth_to_color/image_raw?

Are there any warning messages printed by realsense2_camera node?

jediofgever · 2020-01-07T08:23:04Z

Hi @doronhi,
To start with info quickly ;

OS: UBUNTU 18.04 LTS
KERNEL:  5.0.0-37-generic
RealSense: ROS v2.2.11
LibRealSense: v2.31.0
Intel RealSense D435I

Here is output of when I listen to the average rate with point cloud;

no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 15.161
	min: 0.062s max: 0.067s std dev: 0.00175s window: 9
average rate: 15.048
	min: 0.028s max: 0.105s std dev: 0.01145s window: 24
average rate: 15.024
	min: 0.028s max: 0.105s std dev: 0.00892s window: 39
average rate: 15.012
	min: 0.027s max: 0.105s std dev: 0.01257s window: 54
average rate: 15.009
	min: 0.027s max: 0.105s std dev: 0.01111s window: 69
average rate: 15.005
	min: 0.027s max: 0.105s std dev: 0.01006s window: 84
average rate: 15.001
	min: 0.027s max: 0.105s std dev: 0.00925s window: 99
average rate: 14.885
	min: 0.027s max: 0.122s std dev: 0.01040s window: 107
no new messages
average rate: 11.798
	min: 0.027s max: 1.948s std dev: 0.18045s window: 109
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 7.027
	min: 0.027s max: 6.891s std dev: 0.65298s window: 117
average rate: 7.483
	min: 0.027s max: 6.891s std dev: 0.61493s window: 132
average rate: 7.889
	min: 0.027s max: 6.891s std dev: 0.58286s window: 147
average rate: 8.252
	min: 0.027s max: 6.891s std dev: 0.55532s window: 162
average rate: 8.582
	min: 0.027s max: 6.891s std dev: 0.53135s window: 177
average rate: 8.880
	min: 0.027s max: 6.891s std dev: 0.51023s window: 192
average rate: 9.151
	min: 0.027s max: 6.891s std dev: 0.49145s window: 207
average rate: 9.400
	min: 0.027s max: 6.891s std dev: 0.47461s window: 222
^Caverage rate: 9.539
	min: 0.027s max: 6.891s std dev: 0.46530s window: 231
ubscribed to [/camera/depth/color/points]

and the output when alignment is enabled;

subscribed to [/camera/aligned_depth_to_color/image_raw]
no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 9.206
	min: 0.109s max: 0.109s std dev: 0.00000s window: 2
average rate: 1.995
	min: 0.109s max: 0.894s std dev: 0.39272s window: 3
average rate: 1.476
	min: 0.109s max: 1.030s std dev: 0.40610s window: 4
average rate: 1.316
	min: 0.109s max: 1.030s std dev: 0.37945s window: 5
average rate: 1.243
	min: 0.109s max: 1.030s std dev: 0.35088s window: 6
average rate: 1.191
	min: 0.109s max: 1.030s std dev: 0.32982s window: 7
average rate: 1.160
	min: 0.109s max: 1.030s std dev: 0.31017s window: 8
average rate: 1.137
	min: 0.109s max: 1.030s std dev: 0.29400s window: 9
average rate: 1.124
	min: 0.109s max: 1.030s std dev: 0.27862s window: 10
average rate: 1.111
	min: 0.109s max: 1.030s std dev: 0.26623s window: 11
average rate: 1.097
	min: 0.109s max: 1.030s std dev: 0.25628s window: 12
average rate: 1.088
	min: 0.109s max: 1.030s std dev: 0.24683s window: 13
average rate: 1.079
	min: 0.109s max: 1.030s std dev: 0.23839s window: 14
average rate: 1.073
	min: 0.109s max: 1.030s std dev: 0.23051s window: 15
average rate: 1.069
	min: 0.109s max: 1.030s std dev: 0.22315s window: 16
no new messages
no new messages
average rate: 0.939
	min: 0.109s max: 3.007s std dev: 0.54605s window: 17
no new messages
average rate: 0.891
	min: 0.109s max: 3.007s std dev: 0.57618s window: 18
no new messages
no new messages
average rate: 0.813
	min: 0.109s max: 3.062s std dev: 0.71483s window: 19
average rate: 0.822
	min: 0.109s max: 3.062s std dev: 0.69791s window: 20
^Caverage rate: 0.829
	min: 0.109s max: 3.062s std dev: 0.68160s window: 21

Are there any warning messages printed by realsense2_camera node?

yes it prints out that;

No stream match for pointcloud chosen texture Process - Color

and time to time it gives this ERROR;

ERROR [140645725357824] (synthetic-stream.cpp:47) Exception was thrown during user processing callback!

As seen on logs, its performing very randomly

jediofgever · 2020-01-08T03:36:12Z

what is your kernel version ?, I tried on another computer where environmental setup is the same except for the kernel version being 4.15.0-72-generic, then the cam seems to perform as expected.
Kernel version might be triggering this, is it possible that you try on a system with kernel version 5.0.0-37-generic ?

and overall is there any recommended kernel version especially for Ubuntu 18.04 ?
Thank you for your time

doronhi · 2020-01-09T05:27:01Z

There is a problem with kernel 5.0.0-37-generic. The timestamps stop arriving after a random length of time. That causes issues with synchronizing depth and color images and hence the problems with aligning and pointcloud. I also encountered it a couple of days ago in 4.18.0-15-generic.
The issue is currently being investigated and I assume a fix will be published very soon.

lyubu · 2020-01-13T19:19:39Z

Hello! My version is Ubuntu18.04 with kernel 5.0.0-37.
When I run
roslaunch realsense2_camera rs_camera.launch filters:=pointcloud enable_infra1:=false enable_infra2:=false
and
rosrun rviz rviz
And I cant see my pointcloud2 topic. Is this what you mean (the problem is kernel 5.0.0-37)?
@doronhi @jediofgever

jediofgever · 2020-01-14T02:49:01Z

Hi @lyubu ,
As @doronhi confirmed that currently there is an issue with kernel 5.0.0-37,I suggest you to use another kernel version until a fix is published. You can switch kernel versions in the boot up menu of ubuntu. When you restart your machine, choose advanced options for Ubuntu, from the list choose a previous kernel version, for example 4.15.0-74-generic

ri-ceres · 2020-01-15T02:49:19Z

We are also experiencing the same issue here and have tried a number of different combinations of Ubuntu kernel, realsense firmware, and realsense sdk versions. One source of error seems to be related to firmware versions. 5.11.15.0 seems to work as expected with kernel 5.0.0-37-generic until a device with firmware 5.12.1.0 is plugged in. After this happens, both versions of firmware fail to produce a pointcloud at 30fps, and we see the same dropping of frames as reported by @jediofgever. This has been replicated across 3 different machines over the last week, and even after downgrading our cameras to 5.11.15.0 and recompiling the SDK we have been unable to recover the pointcloud functionality.

Any additional help with this issue would be appreciated.

OS: UBUNTU 18.04 LTS
KERNEL:  5.0.0-37-generic
RealSense: ROS v2.2.12
LibRealSense: v2.31.0, v2.32.0
Intel RealSense d435, d435i
Firmware: 5.11.15.0, 5.12.1.0

Unfortunately, our X399 motherboards have xHCI USB issues with Ubuntu kernels < 5.0, so we have not been able to test with kernel versions 4.15 or 4.18.

ri-ceres · 2020-01-15T03:38:21Z

Following up on the previous comment, it appears that issue 1024 is related in our case.

Downgrading to librealsense v2.30.1 has restored normal operation of our cameras. Further testing is needed, but after multiple restarts and extended uptime (~45min) there have been zero failure cases. This has been replicated on two separate machines.

Our stable configuration looks like this:

OS: UBUNTU 18.04 LTS
KERNEL:  5.0.0-37-generic
RealSense-ROS: v2.2.12
LibRealSense: v2.30.1
Firmware: 5.11.15.0

ri-ceres · 2020-01-15T17:29:02Z

After testing various firmware versions including 5.12.0 and 5.12.1, this appears to be entirely related to librealsense version 2.31.0 on our systems, and the solution is to rollback to 2.30.1. The potential firmware implication mentioned above was coincidence due to the systems running the older version of librealsense while testing firmware 5.11.15.0.

lyubu · 2020-01-15T18:55:55Z

I have uninstalled my librealsense-2.31.0 and installed librealsense-2.30.0 and point cloud has been displayed. Beside this, I did nothing extra. So this means, the display problem is only librealsense-2.31.0. Thanks for your advice so much!!
@ri-ceres
By the way, librealsense-2.30.1 doesn't suit my kernel. Because after I download the package and tried to install, it said that 2.30.1 suits only with kernel 4.4, so my kernel 5.0.0-37 isn't her Mr.Right.
But 2.30.0 also works.

I have not tried to change my kernel from 5.0.0-37 to 4.15, because the operating is a little complicate and I worried it will influence my other packages which I have installed. But thanks for your advice too!!
@jediofgever

Ubuntu 18.04 LTS
KERNEL: 5.0.0-37-generic
librealsense: 2.30.0

PeterJochem · 2020-01-31T18:04:53Z

How to I download the prior version of librealsense? Do I have to do this from source? Can I use apt somehow?
sudo apt-get install librealsense2=2.30.0 tells me that the version does not exist

RealSenseCustomerSupport · 2020-02-08T02:33:12Z

Hi @PeterJochem,

Before you're downgrading to some specific version of LRS please make sure that you have uninstalled all the other versions of LRS.
Using apt-cache showpkg librealsense2 you can see list of available versions for installation.
For example, if you want to downgrade LRS version to v2.30.0 you need to choose it from this list and then to run installation command using the version number together with the suffix that looks like this : ~realsense0."build number" .

For instance:
sudo apt install librealsense2=2.30.0-0~realsense0."build number"

Please pay attention, that if you want to use, let's assume, downgraded version of utils package
it should be installed in addition to librealsense2 package and in the same manner (using the version number together with the suffix).

sudo apt install librealsense2=2.30.0-0~realsense0."build number" librealsense2-utils=2.30.0-0~realsense0."build number"

@jediofgever, @lyubu, @ri-ceres ,@PeterJochem

Please let us know whether IntelRealSense/librealsense#5751 have fixed the problem.

Thank you!

RealSenseCustomerSupport · 2020-02-14T18:57:42Z

@PeterJochem,

Do you need any additional help with this question?

jediofgever changed the title ~~Unstable performance when using filers=pointcloud in d435~~ Unstable performance when using filters=pointcloud in d435 Jan 6, 2020

jediofgever mentioned this issue Jan 7, 2020

No support for ubuntu 18.04 on melodic ? #1021

Closed

This was referenced Jan 30, 2020

align_depth doesn't work #1060

Closed

WARNING [139954094954240] (ds5-timestamp.cpp:64) UVC metadata payloads not available. Please refer to installation chapter for details #492

Closed

ev-mp mentioned this issue Feb 2, 2020

Linux infrastructure fixes. Kernels v4.16+ IntelRealSense/librealsense#5751

Merged

doronhi mentioned this issue Feb 5, 2020

Pointcloud Unreliable (D435) #1071

Closed

RealSenseCustomerSupport closed this as completed Feb 20, 2020

MartyG-RealSense mentioned this issue Aug 10, 2020

Raspberry Pi 4 realsense-ros no pointcloud #1324

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unstable performance when using filters=pointcloud in d435 #1047

Unstable performance when using filters=pointcloud in d435 #1047

jediofgever commented Jan 6, 2020

doronhi commented Jan 6, 2020

jediofgever commented Jan 7, 2020 •

edited

Loading

jediofgever commented Jan 8, 2020 •

edited

Loading

doronhi commented Jan 9, 2020

lyubu commented Jan 13, 2020

jediofgever commented Jan 14, 2020 •

edited

Loading

ri-ceres commented Jan 15, 2020

ri-ceres commented Jan 15, 2020

ri-ceres commented Jan 15, 2020

lyubu commented Jan 15, 2020

PeterJochem commented Jan 31, 2020

RealSenseCustomerSupport commented Feb 8, 2020

RealSenseCustomerSupport commented Feb 14, 2020

Unstable performance when using filters=pointcloud in d435 #1047

Unstable performance when using filters=pointcloud in d435 #1047

Comments

jediofgever commented Jan 6, 2020

doronhi commented Jan 6, 2020

jediofgever commented Jan 7, 2020 • edited Loading

jediofgever commented Jan 8, 2020 • edited Loading

doronhi commented Jan 9, 2020

lyubu commented Jan 13, 2020

jediofgever commented Jan 14, 2020 • edited Loading

ri-ceres commented Jan 15, 2020

ri-ceres commented Jan 15, 2020

ri-ceres commented Jan 15, 2020

lyubu commented Jan 15, 2020

PeterJochem commented Jan 31, 2020

RealSenseCustomerSupport commented Feb 8, 2020

RealSenseCustomerSupport commented Feb 14, 2020

jediofgever commented Jan 7, 2020 •

edited

Loading

jediofgever commented Jan 8, 2020 •

edited

Loading

jediofgever commented Jan 14, 2020 •

edited

Loading