-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Significant depth data mistakes, "Ghost Object" on D435 camera #6713
Comments
Hi @lipeize-pku I was going to suggest using the On-Chip Calibration function of the RealSense Viewer to try to improve your depth image results. I see though from the description at the top of this issue that you are using an extremely old RealSense SDK version from June 2018 (2.13.0). If you are unable to use a modern SDK version such as 2.35.2 for some reason then I would recommend using the Dynamic Calibrator calibration tool. Please note that the tool requires a USB 3 connection. Page 14 onward of the user guide for the tool explains how to install it on Ubuntu 16.04. https://www.intelrealsense.com/wp-content/uploads/2019/04/RealSense_D400_Dyn_Calib_User_Guide.pdf If you are using SDK 2.13.0 though then this could be why you are not getting best results, as the RealSense SDK has improved massively since that very early version. |
Thanks for your timely reply! I will update the SDK and try On-Chip Calibration. I will report back after that. One little thing:
Isn't the depth calculation supposed to be done on D435 itself? Then how come the SDK version can affect the result? |
The Vision Processor D4 hardware inside the camera performs a set of adjustments described in the PDF document linked to below. The data is then passed to the computer / computing device to be organised and used by applications. Operations performed on the data received by the computer, including post-processing, are changes that are performed in the computer's CPU / GPU chips by the RealSense SDK software and not the camera hardware. So the quality of the results can depend partly on the SDK software version and also the firmware version, as features are added and improved and bugs fixed. |
Hi @lipeize-pku Calibration can only help, but I believe what you seeing is caused by horizontal/vertical repetitive patterns in the scene. This is somewhat unfortunate, but well known issue with stereo matching technology. We published two white-papers you may find useful:
|
@dorodnic Yes I suspected repetitive pattern as well when I saw the outdoor far wall shaped like a series of fence posts, but was not so sure if the tree-tops were shaped similar enough to be perceived as a repetitive pattern or the vertical indoor window frames were close enough together. Thanks for the guidance! |
In the paper, one of the conclusions was that repetitive patterns can be introduced into the scene artificially by the process of rectification in hardware. When image is being resampled during rectification, this can introduce Moir patterns. These subtle patterns are sometimes enough to confuse the matching algorithm. In most cases using High Accuracy preset is enough to filter out these artefacts, the paper also presents "Super High Accuracy" JSON, and if all else fails, there is the opencv filter. It will invalidate almost all pixels, but will keep object edges, making it acceptable for collision avoidance. |
Hi @lipeize-pku |
Issue Description
I am using D435 and D435i to do robot obstacle avoidance. There are some mistakes in depth map that the robot misreport as obstacle.
Sample depth and RGB images:
Facing a wall more than 10 meters away.
Cyan “Ghost Object” on depth map, 1.47 meters away.
Facing real object 3.4 meters away.
Yellow “Ghost Objects” on depth map, 1.45 meters away.
Green “Ghost Objects” on depth map, 1.07 meters away.
“Ghost Cliff” on the ground, one side 1.04 meters, other side 1.77 meters
Blue “Ghost Object” top right corner, 0.718 meters away.
Facing a glass wall 2.5 meters away.
Blue “Ghost Objects” on depth map, 0.47 meters away.
Facing a glass wall 2.5 meters away.
Blue “Ghost Objects” on depth map, 0.47 meters away.
Facing a building more than 10 meters away.
Blue “Ghost Objects” on depth map, 0.84 meters away.
The text was updated successfully, but these errors were encountered: