-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wrong readings with D435i outdoors #2667
Comments
Hi @pepisg As the wall is made up of a row of similar looking vertical sections, this may be causing floating areas of incorrect depth due to a phenomenon called repetitive pattern. This is when the camera's depth sensing can become confused by a horizontal or vertical pattern of similar looking elements in a real-world scene. A RealSense team member offers advice at IntelRealSense/librealsense#6713 (comment) for negating repeitive patterns. Intel also have a PDF guide about the repetitive pattern phenomenon at the link below. https://dev.intelrealsense.com/docs/mitigate-repetitive-pattern-effect-stereo-depth-cameras |
Hi @pepisg Do you require further assistance with this case, please? Thanks! |
Hi! I have been trying the suggestions in the whitepaper but success has been limited: I have observed some partial improvement but the problem still occurs from time to time. I tried to enable the HDR and though this seemed to remove the misplaced points, I got an extremely noisy pointcloud overall. Am I missing something on how HDR should be used? Is there any way to completely solve this problem? or is something to be expected to happen occasionally regardless of the fine tuning you may perform on the camera? Thanks! |
If HDR is enabled then the image will be extremely noisy unless the post-processing filter HDR Merge is enabled. HDR Merge disabled HDR Merge enabled In the RealSense Viewer tool, the HDR Merge filter is enabled by default in the Post Processing list of filters. If you have created your own script and are enabling HDR in it then you will also need to program the HDR Merge filter into your script, as post-processing filters are all disabled by default and have to be deliberately coded into your own scripts. At the link below a RealSense user provides their code for enabling the filter with C++. https://support.intelrealsense.com/hc/en-us/community/posts/360053181133/comments/360015393773 Instead of using HDR, you could try reducing the Second Peak Threshold Advanced Mode setting from its default of '325' to a value of '0'. In the Viewer, this setting can be found in the Stereo Module > Advanced Controls > Depth Control section of the options side-panel. Second Peak Threshold can either be defined in a custom json camera configuration file or with code, such as a RealSense user's C++ script at IntelRealSense/librealsense#4993 If you have the option to change camera models, a new D435 model equipped with IMU and Infrared Pass filter called the D435if was released yesterday. The official information page for this camera model states that it "resolves repetitive pattern and false object detection due to light reflections". IntelRealSense/librealsense#11640 Shown below are the properties of the NIR filter on the D435if and D435f models, which you can compare to the filter that you are using on your D435i. |
Hi @pepisg Do you require further assistance with this case, please? Thanks! |
Hi @MartyG-RealSense. Does the hdr merge work in the same way in the ros wrapper? I tried using it but did not see the results you show in the picture. |
My understanding is that the ROS wrapper's implementation of HDR Merge is based on the librealsense one, as the wrapper acts as a ROS compatibility layer with librealsense in the background. A RealSense team member at #2486 (comment) advises to enable auto_exposure when using hdr_merge in the ROS wrapper. |
@pepisg except that recommendation of using enable_auto_exposure while using hdr and hdr_merge, it is recommended to be aligned to latest source code of ros2-developlemnt since it includes hdr and auto exposure fixes. |
Hi @MartyG-RealSense, HDR seems equally noisy for me in the realsense viewer even while having the hdr merge filter enabled: vokoscreenNG-2023-04-12_14-40-44.mp4Is there any specific firmware version or librealsense version that I should have? |
There is not much that can be done to prevent the depth noise in HDR mode, unfortunately. The settings associated with HDR are mostly intended to stop the infrared stream from flickering. The infrared flickering can be stopped by setting the Sequence options to 1, 2, 1. Then go to the Post Processing filter list and enable the HDR Merge filter and set the Filter By Sequnce ID filter to '1'. Metadata support should also be enabled in the Viewer if it is currently disabled in order to stop infrared from flickering. A couple of ways to close up holes in the depth image include:
Or:
The HDR mode requires firmware 5.12.8.100 or newer and SDK 2.39 or newer. |
Hi @MartyG-RealSense , unfortunately I did not got good results: What I'm seing in the realsense viewer is still quite different to your examples. May I be missing something else? Also, would it be useful if I was able to provide you with sample data so you can inspect it? Is the repetitive pattern problem something that ultimately I will be never able to remove completely? |
If you enable the Infrared stream as well as the depth stream, does the infrared image flicker even after setting the sequence ID values like in the images above? If infrared does flicker then it may mean that metadata support needs enabling. If it is currently disabled then when the Viewer is launched, a pop-up will usually appear in the top corner of the Viewer window where the firmware pop-up is, asking if you want to enable metadata. It looks as though a key difference between your images and mine is that your Viewer is in 3D pointcloud mode, whilst my images were taken in 2D mode. To test the settings in 2D mode, left-click on the '2D' option in the top corner of the Viewer window. If you can provide a bag file recording of your depth data then I will be happy to check it in my own Viewer. Also, would it be possible to post an RGB image of your scene into this discussion so that I can look for possible disruptive elements such as repetitive patterns or other environmental factors? |
Hi @pepisg Do you require further assistance with this case, please? Thanks! |
Case closed due to no further comments received. |
Hi!
I'm currently using a D435i with an IR filter on an outdoors robot and I'm getting very wrong readings in some scenarios. Here's an example:
As you can see in the camera pointcloud there are some small clusters (labeled NOISE) very close to the axes (the robot's position) which when aligned to the RGB image correspond to a wall that lies much further away, as confirmed by the aligned 2D lidar (labeled as REAL WALL POSITION). I also enclose a video to better grasp the depth of the points
noise.mp4
Is there any way of mitigating this behavior either by calibrating the camera or using a post processing filter?
Thanks in advance for your help!
The text was updated successfully, but these errors were encountered: