How to verify the camera stream's status?

[Boatsure]

Required Info
Camera Model	{D435
Operating System & Version	Ubuntu18.04
Platform	NVIDIA Jetson TX2
Language	python

Issue Description

During the operation of a python program that enables the camera stream and waits for frames according to certain conditions, I observed that even after unplugging the cable between D435 and tx2 , the program on tx2 can still successfully get one frame of image (of course, it is not the current field of view of the camera but the last view when the cable is unplugged).

According to my understanding, because the last frame of the camera stream is stored in the tx2 memory, the image frame can still be obtained after unplugging the cable. So my question is, is there a way to judge whether the current stream inside the camera is normal or not before attempting to get the frame?

[MartyG-RealSense]

Hi @surefyyq When the camera is unplugged, the pipeline should close and then be reset to its default state when the camera is next plugged in. So there should not be a frame from the previous session still left in the pipeline when the camera is plugged back in.

It is my understanding though that if the camera disconnects then there is a five second time window in which, if the camera reconnects before that time elapses, the pipeline can resume streaming from where it left off.

It is possible to keep track of USB 'detatch and attach' events via the SDK's set_devices_changed_callback instruction. A RealSense team member provides a Python script that demonstrates this instruction at #4212 (comment)

[Boatsure]

Thanks for your reply @MartyG-RealSense ! I will try the set_devices_changed_callback feature.

But I'm kind of sure that no matter how long before the cable is reconnected to tx2, the python script can still get one frame and only one frame. And weather the cable is reconnected or not, the second attempt to wait_for_frames will go error "frame didn't arrive in 5000".

And by the way, I also observed that if a script, which wait_for_frames in a loop like align-depth2color.py, is terminated abnormally without running pipeline.stop(), this script will not be able to run again, but go error "resource temporarily unavailable" or "failed to set power state". Are these normal? And what's the right way to handle this condition, please?

Thank U!

[MartyG-RealSense]

Yes, it is possible for a program to experience such errors on a subsequent launch if the pipeline is not properly closed after the program's first run. You could test whether adding a hardware_reset() instruction to the script to reset the camera at the time of the program's launch resolves the errors. Activating this instruction will have a similar reset effect to unplugging and replugging the camera. Simple code for doing so in Python is shown below.

ctx = rs.context()
devices = ctx.query_devices()
for dev in devices:
dev.hardware_reset()

An example of implementing this code is in #9557 (comment)

[Boatsure]

Thanks, all right then, I will keep you posted about the set_devices_changed_callback and the cable thing.

[MartyG-RealSense]

Hi @surefyyq Do you have an update about this case that you can provide, please? Thanks!

[Boatsure]

Hi, @MartyG-RealSense sorry for the late.
Yes, the set_devices_changed_callback feature can detect USB 'detatch and attach' events.

However, It is observed that the 'frame didn't arrive in 5000' error occurs even if the cable is not unplugged, actually the error will occur under the condition where nothing has been changed.
To be more detailed, my python program will start the pipeline at launch and then monitor the keyboard input in a loop. When the keyboard input a space, wait_for_frames is called and save the current frame as an image. The issue is, it runs normally at the beginning, but when you don’t enter a space for a period of time and enter again, it will get the error 'frame didn't arrive in 5000' .

In addition, if I ret hardware at the time of the program's launch :

ctx = rs.context()
devices = ctx.query_devices()
for dev in devices:
    dev.hardware_reset()

it will go error segment fault or no device connect.

[MartyG-RealSense]

If your program is one where it is streaming continuously and pressing the space key causes a capture of the current frame, the application might work better if you ran a routine containing the save_single_frameset() instruction when the trigger conditions are met. Information about use of save_single_frameset in Python can be found in #9778 (comment)

If you would prefer to continue using your own code: I would suspect that making a call to wait_for_frames() be dependent on a keypress would be what could be breaking your program, and that wait_for_frames() should be outside of the keypress listener loop. If wait_for_frames() cannot operate until a keypress is received then the program is likely to timeout after 5 seconds (the '5000' mentioned in the error) because no new frames are being received unless wait_for_frames() is active.

#2422 (comment) provides a useful reference about how different types of _for_frames() instructions work.

[Boatsure]

@MartyG-RealSense Thanks for your citation, I'll try them.

[MartyG-RealSense]

Hi @surefyyq Do you have an update about this case that you can provide, please? Thanks!

[Boatsure]

Hi @MartyG-RealSense , before I can go further checking code-related problems, updating the camera hardware version solves the issue. No python code is changed, and now everything works fine.

Thanks a lot!

[MartyG-RealSense]

It's great to hear that your issue is resolved, @surefyyq - thanks very much for the update :)