RealSense D455 / LibRealSense v2.48.0 / ROS2 / Ubuntu 20.02 / RPi 4b control_transfer error

[JHPHELAN]

Read "RealSense D455 FAIL..." for a synopsis of the error and system setup.
Read "...Experience" for a step-by-step log of the journey to see where I may have gone wrong.
Had a similar experience under Ubuntu 18.04 and ROS(1).
Search reveals others have had similar problems under other setups, eg Jetson NANO but no clear solution as suggested fixes failed to solve.

Will be happy to provide any other system information if you'll please give me the appropriate command line command to display it. I'm an experienced robotics hobbyist (NASA/JPL Open Source Rover) but certainly no expert. Any guidance would be appreciated!
JHP

Intel RealSense Camera Experience.pdf
RealSense D455 FAIL ROS2 Ubuntu2004.pdf

[MartyG-RealSense]

Hi @JHPHELAN As a starting point in investigating your case, I note from a section on page 6 of your kindly provided diagnostic documentation Intel.RealSense.Camera.Experience.pdf that you built the ROS2 wrapper first and then built librealsense secondly. It should be the other way around - librealsense should be built first and the ROS wrapper secondly. Also, if the ROS wrapper is built and then the librealsense version is subsequently updated, the ROS wrapper needs to be built again.

I would also recommend avoiding performing different types of installation together on the same computer (such as Method 1 and Method 2, or building from packages and then building from source code on a subsequent attempt. You could end up with conflicts otherwise.

It may be best to do a complete wipe and reinstall of the computer (including Ubuntu) and then reinstall Ubuntu, then librealsense from the Debian packages on Intel's Jetson instructions page, and then finally the ROS2 wrapper from source code.

If it is not practical for you to reinstall Ubuntu then you could uninstall all librealsense related packages from the computer using the Ubuntu command below:

dpkg -l | grep "realsense" | cut -d " " -f 3 | xargs sudo dpkg --purge

If librealsense is installed from source then going to the 'build' directory of the source code folder and running the CMake command below should remove it.

sudo make uninstall && make clean

In the case of RealSense ROS wrapper installations built from source, I believe that deleting the whole catkin workspace folder catkin_ws should be sufficient to remove it.

[MartyG-RealSense]

Hi @JHPHELAN Do you require further assistance with this case, please? Thanks!

[JHPHELAN]

@MartyG-RealSense
Probably. I've not had success following several trails but need to clarify the problem before I respond further. Thanks!!

[MartyG-RealSense]

Thanks very much @JHPHELAN for the update - I look forward to your next report. Good luck!

[MartyG-RealSense]

Hi @JHPHELAN Do you require further assistance with this case, please? Thanks!

[JHPHELAN]

2021.10.28 RealSense Log.pdf
@MartyG-RealSense
Yes! Please! Again, you may want to move this to a general install issue.
See attached log for excessive detail. This has been my most successful install so far.
SKD via "backend" (v2.49) which installed with warnings and errors (sample included) but didn't crash.
Wrapper via ros2 branch (which "supports SDK v2.48" concerned about that discrepancy).
The issues now come with launching.
Included are the terminal outputs captured with script with multiple line repeats deleted for the 4 sample launches.
There is no d455.yaml -- would Intel PLEASE provide one?
As do others, would very much appreciate an "apt install" for Pi4/Ubuntu 20.04/ROS2 to obviate this headache.
Thank you! Look forward to your insight!
Jim P

[MartyG-RealSense]

If you built librealsense from source code via RSUSB backend then I would recommend using SDK 2.48.0 instead of 2.49.0 as it is important to match the ROS wrapper version closely with the librealsense SDK version listed in a particular wrapper version's release notes. ROS2 wrapper 3.2.2. matches to 2.48.0, and there is not a ROS wrapper for 2.49.0 (the next wrapper release at the time of writing this should be for forthcoming SDK version 2.50.0). I note though that you have already tested with 2.48.0.

Looking at the contents of the ROS2 wrapper's d435i.yaml file in the link below, it looks as though it should function fine with a D455 (which is like a D435i with improved depth-color alignment and a faster 'global' shutter on the RGB sensor). There is not anything in this yaml file that is configuring settings incompatible with a D455.

https://github.com/IntelRealSense/realsense-ros/blob/ros2/realsense2_camera/config/d435i.yaml

In Try 4 it appears that there is a typing mistake in the yaml path that sets it to look for d435.yaml instead of d435i.yaml and so the launch cannot find the configuration file and applies the D455 camera model's default stream configuration instead (848x480 depth and 1280x720 color).

[JHPHELAN]

@MartyG-RealSense
Ok, a colleague showed me how to use Tags on the repo to find the appropriate SDK version. Will give that a try, then see if the camera will work witht the d435i.yaml.

[JHPHELAN]

2021.10.30.librealsense2.48.0FAIL.pdf
@MartyG-RealSense
OK, tried installing SDK v2.48.0 and immediately ran into problems. See attached log.
Has anybody successfully installed a D455 on a ROS2/Ubuntu 20.04/Raspberry Pi 4b system?
If so, could you point me to a reliable install path?
Thanks!
Jim P

[MartyG-RealSense]

@JHPHELAN I conducted extensive further research of your requirements. The closest fit that I could find though was a guide for installing ROS1 Noetic on a Pi 4 with a RealSense camera and Ubuntu 20.04.

https://ramith.fyi/setting-up-raspberry-pi-4-with-ubuntu-20-04-ros-intel-realsense/
https://answers.ros.org/question/363889/intel-realsens-on-ubuntu-2004-ros-noetic-installation-desription/

[JHPHELAN]

@MartyG-RealSense
Been down that road before.
The first link is just a basic OS install, now out of date. It points to the second for the install script.
The second has been deprecated and points back to the main librealsense page here:
https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md
I did note a couple things:
Here, and elsewhere, they installed librealsense before ros. Is that critical? I've been installing ros2 foxy first.
Also they install ddynamic-reconfigure manually using:
sudo apt install ros-noetic-ddynamic-reconfigure
When I tried before the install couldn't find the ddynamic-reconfigure package. Maybe there isn't one for ros2? I'll have to look back at the install and see if I can do it manually for foxy.

[MartyG-RealSense]

When the librealsense SDK and the ROS wrapper are installed separately, the correct build order is librealsense first and the ROS wrapper secondly after the librealsnse install is completed. Also, if the librealsense version is updated then the ROS wrapper must be built again.

You are correct that ROS2 does not use dynamic_reconfigure. Instead you would use rqt with the parameter reconfigure plugin to watch and modify parameters in ROS2.

[JHPHELAN]

@MartyG-RealSense
I didn't mean the wrapper, but the basic ROS2 foxy itself pre-wrapper. I understand that librealsense comes before wrapper. Or does anything ROS have to come after?

You are correct that ROS2 does not use dynamic_reconfigure. Instead you would use rqt with the parameter reconfigure plugin to watch and modify parameters in ROS2.

That's reassuring, but then what is "parameter reconfigure"? Is it included somewhere?
Or is is something else that needs to be installed, if so how? sudo apt install ??
What does it do, anyway?

[MartyG-RealSense]

As ROS is not dependent on librealsense - unlike the RealSense ROS wrapper - it will not matter if it is installed before or after librealsense.

Parameter reconfigure in ROS2 works through rqt_reconfigure with a similar rosrun command to the one used in ROS1 to access dynamic_reconfigure (using ros2 run instead of rosrun).

https://github.com/IntelRealSense/realsense-ros/tree/ros2#view-and-modify-camera-controls-params-in-runtime

[JHPHELAN]

Ah, got it. Thanks!!

[MartyG-RealSense]

Hi @JHPHELAN Do you require further assistance with this case, please? Thanks!

[JHPHELAN]

@MartyG-RealSense
Still working on it.
Will try to apply to the Pi4/Ubuntu 20 our recent success with the Jetson NANO / Ubuntu 18 / ROS2 dashing / libUVC backend (modified) / ROS2_Wrapper from source.
Attached if anyone is interested.
Will share when/if we're successful.
Thanks for your continued interest!
JHP
NANO RealSense SUCCESS.pdf

[MartyG-RealSense]

Thanks very much @JHPHELAN for the update and the sharing of your test documentation with the RealSene community - I look forward to your next report!

[JHPHELAN]

@MartyG-RealSense
Raspberry Pi 4b, Ubuntu 20.04, ROS2 foxy:
RealSense 2.48 seemed to install ok via modified libuvc backend. See attached log. When realsense-viewer is launched Stereo Module and RGB Camera display ok but still get transfer_control and other errors. The Motion Module worked ok in other situations but not in this one. Will go ahead and try to install realsense2_camera and see what happens.
JHP
RealSense control_transfer.pdf

[JHPHELAN]

realsense2_camera install FAIL
@MartyG-RealSense
realsense2_camera install FAIL.pdf
Following on from above, I tried installing realsense2_camera but ran into failure. See log attached.
JHP

[MartyG-RealSense]

Can you confirm please that the dependencies are successfully installed? The commands to do so are issued in the log but there is no mention of what happened when each install command was input.

https://github.com/IntelRealSense/realsense-ros/tree/ros2#step-4-install-dependencies

On the second colcon attempt after the initial build fail and the installation of the colcon extensions, the colcon build instruction is repeated, suggesting that it was assumed that the dependencies were already successfully installed and so those install commands did not need to be input again.

[JHPHELAN]

@MartyG-RealSense
Yes, they all installed without error.
JHP

[MartyG-RealSense]

Does it make any difference if you separately install diagnostic_updater for Foxy using the command below:

sudo apt-get install ros-foxy-diagnostic-updater

[JHPHELAN]

@MartyG-RealSense
The
sudo apt-get install ros-foxy-diagnostic-updater
Installed w/o errors. That tidbit needs to get included somehow in the install instructions or package as does the colcon dependency.

Now, at least, I'm getting a different error:
It says I'm requesting v2.50.0 but I thought I was requesting v2.48 because that was the version required by the ROS2 foxy version of realsense2_camera?! That incompatibility is what made me go to the libuvc backend 2.48 in the first place. Is it asking for the wrong version, or has it been updated to where I really do need the current 2.50.0 version? Do I need to do a work-around like I did with libuvc_installation.sh to get the install to accept 2.48?

--- stderr: realsense2_camera
CMake Warning at CMakeLists.txt:102 (find_package):
  Could not find a configuration file for package "realsense2" that is
  compatible with requested version "2.50.0".
  The following configuration files were considered but not accepted:
    /usr/local/lib/cmake/realsense2/realsense2Config.cmake, version: 2.48.0
CMake Error at CMakeLists.txt:104 (message):
   Intel RealSense SDK 2.0 is missing, please install it from https://github.com/IntelRealSense/librealsense/releases
---
Failed   <<< realsense2_camera [7.93s, exited with code 1]
Summary: 2 packages finished [12.0s]
  1 package failed: realsense2_camera
  1 package had stderr output: realsense2_camera

JHP

[MartyG-RealSense]

The newest ROS2 wrapper 3.2.3 matches to SDK 2.50.0 which was released in the past week.

https://github.com/IntelRealSense/realsense-ros/releases/tag/3.2.3
https://github.com/IntelRealSense/librealsense/releases/tag/v2.50.0

If you have SDK 2.48.0 installed then it should be matched with ROS2 wrapper 3.2.2.

[JHPHELAN]

LibRealSense 2.50 FAIL.pdf
@MartyG-RealSense
Thought it best to start over and install the latest. Was hoping for a simple "sudo apt install..." but no such luck!
Got through most of it, see attached log. There was some confusion, I think, mixing html code in the instructions, which I worked around. When it got to the end there was a missing "tag". Looking into the .sh file I couldn't determine what the missing tag was. Any suggestion?
JHP

[MartyG-RealSense]

As you are using Raspberry Pi, this may be an Arm related issue, as the error fatal: You need to specify a tag name has previously been reported on Raspberry Pi and Jetson in #1220 (Pi) and #1110 (Jetson).

Doronhi the RealSense ROS wrapper developer advised in the Jetson case to build from source with FORCE_RSUSB_BACKEND = true. I also suggested that course of action in the Pi case and it resolved that Pi user's tag error problem.

[JHPHELAN]

@MartyG-RealSense @doronhi
Thanks! I'll travel that path again and see what happens.
Will let y'all know.
JHP

[MartyG-RealSense]

Hi @JHPHELAN Do you have an update about this case that you can provide, please? Thanks!

[MartyG-RealSense]

Hi @JHPHELAN Do you require further assistance with this case, please? Thanks!

[JHPHELAN]

@MartyG-RealSense
Still working on it and having issues. Trying to find a good way to document the errors.
JHP

[MartyG-RealSense]

Okay, thanks very much @JHPHELAN for the update!

[JHPHELAN]

@MartyG-RealSense
@doronhi
See ROS_Wrapper.success.pdf for how I got here.
See RealSense.Camera.errors.pdf for current errors and questions.
Are there adjustments that can eliminate/reduce the control_transfer errors such as shutting off some function?
Where are the configurations for Fisheye 0 and Pose 0?
Where/how do I configure the ROS2 launch sequence to get Motion Module to work?
The goal being to pass information to Navigation for path control.
Also to enable Piloting on the Freedom Robotics dashboard which is not recognizing the camera.
JHP

[2021.12.05 RealSense Camera errors.pdf](https://github.com/IntelRealSense/realsense-ros/files/7656641/2021.12.05.RealSense.Camera.errors.pdf

2021.11.22 ROS_Wrapper success.pdf

[MartyG-RealSense]

Log messages about fisheye not being supported can be ignored. The fisheye sensors are only on the RealSense T265 Tracking Camera model and not 400 Series stereo depth models such as D455.

There may be nothing that can be done to completely eliminate control_transfer returned warnings. Reducing resolution / FPS or disabling unneeded stream types (Infra 2, for example) may help to reduce the number of warnings that are generated. But if the topics are being published and visualised correctly then control_transfer returned warnings can usually be ignored.

A good example of configuring a ROS2 roslaunch instruction is at #2068 (comment) - the Motion Module (IMU) related commands in the roslaunch featured in that link are enable_gyro:=true enable_accel:=true unite_imu_method:='linear_interpolation'

[MartyG-RealSense]

Hi @JHPHELAN Do you require further assistance with this case, please? Thanks!

[JHPHELAN]

@MartyG-RealSense
Yes, still working on clarifying the issues....
Thanks for following up!
Jim

[MartyG-RealSense]

Okay, thanks very much for the update. I look forward to your next report. Good luck!

[MartyG-RealSense]

This case will be closed after a week from the time of writing this if we do not receive further comment. Thanks!