The measurement error of the plane using the D415 depth camera.

[ly5234]

Required Info
Camera Model	{ D415 }
Firmware Version	5.12.6
Operating System & Version	Win (10) / Linux
Kernel Version (Linux Only)
Platform	PC/ NVIDIA Jetson
SDK Version
Language	{C/python }

Issue Description

I used D415 to measure the depth of a plane at a distance of about 1.8m. I have made many measurements and found some phenomena that I did not quite understand. As shown in the figure below, there is always a deviation in depth measurement near the directly below the camera.
Is this phenomenon normal? Should I make some adjustments to improve it, or what is its principle?

[MartyG-RealSense]

Hi @ly5234 Would it be possible to provide an RGB image of the scene that you are observing on the depth image please? Thanks!

[ly5234]

Hi @ly5234 Would it be possible to provide an RGB image of the scene that you are observing on the depth image please? Thanks!

Thank you for your attention and reply.
The resolution of the depth image is 720p, and the resolution of the RGB image is 1080p.
The rectangle on the depth image is the visualization area that I showed in the issue description.

[MartyG-RealSense]

Thanks very much @ly5234 for the images. As the RGB image is for the purpose of showing me the scene and is not part of the program, the resolution of the RGB image will not matter.

The optimal depth accuracy resolution on the D415 camera model is 1280x720. As the depth resolution decreases, so will depth accuracy. I do not think that resolution is the cause of your measurement problems in this case though.

Looking at the RGB image, it is possible that the depth sensing of the camera is being confused by a repetitive pattern penomenon. This is where there are similarly shaped elements in a repeating horizontal and / or vertical pattern. An example would be a tiled floor (like in your RGB image) or a row of vertical fence posts (which your ruler resembles). So there may be two separate elements in the scene that could both generate a repetitive pattern and disrupt the depth image.

The link below discusses the repetitive pattern phenomenon and provides links to resources for reducing it that are suggested by Dorodnic the RealSense SDK Manager.

#6713

It is also worth mentioning that if you try to generate a depth image of the tiled floor then you may not have much success. This is because it is a physics principle that dark grey and black colors absorb light instead of reflecting it back, making such surfaces difficult for the camera to read depth detail from unless a strong light-source is cast onto the surface.

In dark scenes, having the IR Emitter enabled can improve depth image results. The images below show a scene with the IR emitter enabled (upper) and disabled (lower).

[ly5234]

Hi, @MartyG-RealSense . Thanks for your reply.
Based on your experience, if I use a standard chess board, does it cause the repetitive pattern penomenon and/or "dark light absorption" that you talked about？

I do the following experiment to verify the accuracy of the depth measurement.

• Place the target at the same height but different position and record the depth
• The target is composed of multiple boxes, try to avoid repetitive patterns
• Only measure the depth of a certain sampling point of the target, regardless of other positions

The result is that there are different measurement depth at different positions, as follow, errors is about +- 20mm，and the actual height is about 1780mm

The camera may have a slight tilt angle (less than 0.2 degree), which is not absolutely horizontal. Whether the measured value should conform to a linear distribution and the angle is equivalent to the actual camera's tilt angle.

Or maybe some non-linear functions can be used to fit the measurement errors, which are inevitable in stereo-vision-based depth camera.

[MartyG-RealSense]

Hi @ly5234 If the chessboard is being used for camera calibration purposes then the calibration process should not be affected by the repetitive pattern phenomenon.

I would speculate that the alternating black-white pattern would make the chessboard squares sharply distinctive enough not to be identified as a repeating element in the same way that a continuous row of similar-looking vertical fence posts or horizontal window-blind slats might. The chessboard is certainly not a disruptive element in Intel's box_dimensioner_multicam chessboard-based object measuring Python example program.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/box_dimensioner_multicam

I am reminded of a case from August 2020 that is similar to your setup of a box being observed from above. The expectation was that just the height of the top of the box would be detected. The RealSense user in that case found that there were a range of measurements though, apparently caused by the depth image 'bleeding out' from the edges of the top of the box and creating false depth readings.

#7021

Looking downwards from the elevated camera, 1 meter should represent the true distance from the camera of the top of the box and 4 meters the floor.

The solution that worked best for that particular RealSense user in the end was to use the Medium Density camera preset to provide a balance between measurement accuracy and "fill rate" on the image.

[ly5234]

@MartyG-RealSense Thanks for sharing the experience of ghost noise, it will be very useful for my future work.

However, the problem that confuses me now does not seem to be caused by ghosts.
Let me add a sketch to further express my problem.

I placed the target box in different positions to measure the distance between the target sample point (e.g. red point A, B, C, ...) on the box and the camera. The target point is located in the center of the upper surface of the box, and the size of the box is about 30cm x 20cm x 5cm.

In order to avoid the interference of repeated patterns, I used a conveyor to move the box horizontally and measure the depth of different positions in different depth imgae.

However, the depth of these points (A, B, C, ...) is different, I don’t quite understand their distribution trends.

[MartyG-RealSense]

Thanks very much for the illustration of the conveyor and the accompanying explanation. It was very helpful for clarification of your problem.

Just to check that I have the right idea: is the camera position fixed at the center, and the box positions A, B and D represent a single box being placed off-center from the camera, to the left and the right of it so that the box is towards the edge of the depth image when placed at position A, B and D?

If the above description is correct and the depth measurement is different when off to the side of the image: this is a phenonemon that has been experienced by other RealSense users. An example of such a case is in the link below. Please read downwards from the point that I have linked to in the discussion (skip past the script to the text underneath it):

#6749 (comment)

[ly5234]

Thanks so much for your reply.
I couldn't agree more, the depth measurements off to side of the image will be different. Indeed, I did not make the camera completely perpendicular to the conveyor, it have a slight tilt angle (< 0.2 degree).
But even so, I think if I placed the boxes in different positions with the illustration of the conveyor, measured values should increase / decrease linearly from left to right.

The X-axis of the diagram above is the position of X-axis in the depth image from left to right. If my above assumption is correct, it seems that these depth measurements shouldn’t be like this, that is, there are two unusual "prominence"
I have lift the conveyor, but the phenomenon is similar at different distances from the camera (e.g. 1.7m, 2m )

I will try to recalibrate to see if there is any improvement.

ps: the value obtained from the depth image should represent the distance along the Z-axis of camera, rather than the 3D spatial distance between the camera and the target?

[MartyG-RealSense]

This issue with measurements drifting significantly at the sides of the image tends to appear in cases where the instruction get_distance() is being used to retrieve the XY coordinates from the image. The link below has another example of it:

#7395

Given that you are using a conveyor, the link below may be useful. The Chief Technical Officer of the RealSense Group at Intel (agrunnet) provides advice about using the RealSense On-Chip calibration system with a conveyor.

#6102

In regard to how the depth value is generated, the data sheet document for the 400 Series cameras states that "the depth pixel value is a measurement from the parallel plane of the imagers and not the absolute range".

[MartyG-RealSense]

Hi @ly5234 Do you require further assistance with this case, please? Thanks!

[ly5234]

Thanks very much for your patient reply. I re-calibrated and adjusted the A-factor (=0.08) and depth units (=100) of Default mode. Although the phenomenon still exists, the error is currently within my acceptable range (about 10mm).
Of course, if other users have experience to share, or there are other possible reasons in the future, I'd be glad to continue the discussion.
Thanks again for sharing your experience @MartyG-RealSense

[MartyG-RealSense]

You are welcome :) Thanks very much for the update and the sharing of your solution for the benefit of other RealSense community members!