Application using camera crashes with usbi_mutex_lock assertion and closes the ssh connection
#11573
Comments
|
Hi @maciejandrzejewski-digica It is recommended that when the librealsense SDK is installed on Rockchip devices, the SDK is built from source code with CMake in RSUSB backend mode by using the build flag -DFORCE_RSUSB_BACKEND=true. An RSUSB build of the SDK bypasses the kernel and so is not dependent on Linux versions or kernel versions and does not need to be kernel patched. |
|
@MartyG-RealSense The SDK has been built with following command which includes this flag: |
|
Could you check please whether there is a memory leak in your application. This is where the computer's memory is progressively consumed over time during the running of an application until it becomes unstable and may crash. You can do this by launching your application and then starting a Linux system monitoring tool such as htop. The link below describes how to install htop on Debian and use it over an ssh connection. https://www.cyberciti.biz/faq/how-to-install-htop-on-debian-linux-using-apt-get/ |
|
I have run the app with the After that I have run the app with Moreover I can run the app without the camera using the recorded data while whole processing is going on. In that case the app does not crash. |
|
Valgrind is not an ideal way to check for memory leaks in RealSense applications as it has been reported that it can provide false-positive results, as described at #3433 |
|
Hi @maciejandrzejewski-digica Do you require further assistance with this case, please? Thanks! |
|
Yes, the problem with USB error still persists (Issue 1). Issue number 2 has been resolved. It was a problem with proper power supply that is described here as a "My new ROCK 5B can not boot / stuck in infinite boot loop": You have stated that valgrind have problem with librealsense. I have checked the issue #3433 and it is irrelevant to my valgrind errors. The one from the issue is: |
|
As you are using a callback, did you place the word callback in the brackets of the pipeline start instruction to inform the program that it is a callback script? For example: pipe.start(callback) Or if cfg configuration instructions are used, put both callback and cfg in the brackets, separated by a comma but with no space between them: pipe.start(callback,cfg) It may also be worth trying to build librealsense in V4L2 backend mode instead of RSUSB backend mode to confirm whether or not libusb is causing the problem. This can be done by setting the flag -DFORCE_RSUSB_BACKEND=false instead of having it as true. #10188 (comment) is an example of a RealSense user who had greater stability with V4L2 backend than RSUSB. |
|
I'm using the callback and I have passed the function address of the callback into the start function. I can confirm that same libusb error is present on both RPI4 and Rock5B. Below is the code: int RealSenseVideoSource::open(const char *path)
{
int ret{};
rs2::config cfg;
cfg.enable_stream(RS2_STREAM_DEPTH);
cfg.enable_stream(RS2_STREAM_COLOR);
if(m_sensInertialEnabled)
{
rs2::config cfgInertial;
cfgInertial.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F, cRsAcclSampleRate); // Do not modify the FPS value - it is fixed for proper calculations.
cfgInertial.enable_stream(RS2_STREAM_GYRO, RS2_FORMAT_MOTION_XYZ32F, cRsGyroSampleRate); // Do not modify the FPS value - it is fixed for proper calculations.
auto callback = [&](const rs2::frame& frame)
{
inertialSensorPipelineCallback(frame, this);
};
rs2::pipeline_profile ppAccel;
try
{
ppAccel = m_inertial.start(cfgInertial, callback);
}
catch(const rs2::error &e)
{
processError(e);
}
if(! ppAccel)
{
msgError() << "Cannot open inertial sensors pipeline" << std::endl;
ret |= -2;
}
if(ret != 0)
{
m_isOpened = false;
return ret;
}
}
rs2::pipeline_profile pp;
try
{
pp = m_cam.start(cfg);
}
catch(const rs2::error &e)
{
processError(e);
ret = -1;
}
if(ret != 0 || !pp)
{
m_isOpened = false;
msgError() << "Cannot open video capture" << std::endl;
ret |= -1;
return ret;
}
if(msgIsDebug())
m_rprint = std::make_unique<rs2::rates_printer>();
m_isOpened = true;
return ret;
} |
|
The RealSense SDK has an example C++ program for callback called rs-callback. How it works is that it enables depth and color with the camera's default profile, which is what your own script above also does as you do not provide custom resolution and FPS settings in your cfg instructions. rs-callback also automatically enables the IMU streams if an IMU is present. |
|
So you suggest to configure only single pipeline for both video and IMU? |
|
When enabling depth, color and IMU simultaneously, using multiple pipelines is best suited to Python, whilst callbacks are recommended for C++ instead of multiple pipelines for depth-color-IMU streaming. |
|
Single pipeline for both video frames and IMU data in single callback solved the problem. I would consider this an SDK bug as it allows to have multiple pipelines. First I had a single video pipeline using wait_for_frames() as it was very handy in my implementation. |
|
You can create multiple pipelines for multiple cameras and place a camera on each, like with the RealSense SDK rs-multicam example program that automatically creates a separate pipeline for each attached camera. https://github.com/IntelRealSense/librealsense/tree/master/examples/multicam There are virtually no C++ script references for using depth + color + IMU on the same camera with multiple pipelines though. It is done with callback instead. |
|
There is nowhere written you can not do it. If it is forbidden then disallow it during the init phase. |
|
It is not impossible as far as I know and I have in the past seen one C++ script reference for using IMU with multiple pipeline on the same camera. Because of the lack of scripting references, callback is the recommendation because there are quotable script links for that method. I recall from that one case that the RealSense user who created a C++ two-pipeline script for depth-color-IMU said that it ran poorly compared to two pipelines on Python. |
Background description
C++ application is cross-compiled and run directly on the target hardware.
Camera is connected to the target hardware.
Two pipelines are running: RGB, Depth frames and IMU sensor read out with full speed.
The IMU pipeline is run as a callback.
Issue description no. 1
Application crashes randomly within 1-2 minutes during the gdb debug session with following error:
Not sure why is this happening. Looks like a mutex inside libusb driver?
Issue description no. 2
The issues might be connected to the one above.
The application is running without gdb session, started from the command line while connected through ssh.
It crashes randomly but does not stay in the command line as it should but it closes the current ssh connection. Very strange behavior. I have observer same situation when running
realsense-viewer. I have forced the error by keeping the camera opened in the application and running therealsense-viewerin the other connection. This was the output, please notice the the connection has been closed after therealsense-viewerexit with error:The text was updated successfully, but these errors were encountered: