Inaccurate angle representation D435

[AlphaRalph]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	{D435}
Firmware Version	(05.12.05.00)
Operating System & Version	{Win10}
Kernel Version (Linux Only)
Platform	PC
SDK Version	{pyrealsense 2.35.2}
Language	{python}
Segment	{Robot}

Issue Description

Setup

D435 about 150mm above ground floor, looking down with ~30° from horizontal on the D435 box as test object.
Factory calibration unchanged, using "ShortRangePreset.json" with 848x480

Experiment

When streaming depth data with the realsense viewer (or my own app) the walls of the box on the ground floor are hanging towards the outside of the box, even though the walls of the (real) box are perpendicular to each other and to the floor. See images.
Looking from the right side:

Looking from the back side:

Looking from the front side (pink is backgroud->ignore):

Conclusion

All walls of the D435 box hang towards the outside, which makes an accurate mesaurement of the box dimensions nearly impossible.

Furthermore, when turning the camera 90° around its z-axis one wall of the box still hangs towards the outside, but the other wall then hangs towards the inside of the box.

Question

What has to be done to overcome this issue?

[MartyG-RealSense]

Hi @AlphaRalph This question has also been responded to at the Intel Support forum. I recommend handling the case here on the GitHub only so that we are not talking in 2 different places.

Do you have similar results if you place a box other than the RealSense box on the floor, please?

[AlphaRalph]

Hi @MartyG-RealSense
result with standard cardboard box:


Exact same behaviour

[MartyG-RealSense]

Thank you very much. And do you get similar results if you use the 'Default' preset instead of 'ShortRange', please?

[AlphaRalph]

Using Default:

Using HighAccuracy (Top Edge of box manually drawn for better visualisation understanding):

Note: Object is ~230mm away

[MartyG-RealSense]

Could you post an accompanying RGB image of the same scene please? This can convey a lot of useful diagnostic information about the colors, surface textures and light sources in the scene.

[AlphaRalph]

Using ShortRangePreset.json again

[MartyG-RealSense]

Thanks, those images make the scene easier to understand, as the outline of the box is much more visible.

It looks like a quite solid depth image with a lot of surface information, with the majority blue color probably due to the close distance from the camera of the surfaces.

If you are using the RealSense Viewer with a single camera and are not calibrating the camera, you should not need to have the checkerboard / chessboard pattern underneath the object.

[AlphaRalph]

The checkerboard is for my real application where I have to calibrate three D435 cameras to each other, based on the box_dimensioner_multicam example.
Nevertheless, the problem with the box walls hanging towards the outside remains.

[MartyG-RealSense]

Do you mean the black areas surrounding the box, please?

[AlphaRalph]

The sides of the box do not have an angle of 90° from the floor, even though the real (pysical) box has.

[MartyG-RealSense]

I am reminded of the case in the link below:

#5506

Can you switch to the Viewer's 3D mode and see what the point cloud version of the box looks like?

[AlphaRalph]

I've seen #5506 before, but it's not really the same problem, since he was able to solve it in his code/ app.
All screenshots I've shared in this discussion so far are taken from the 3D mode of the viewer.

[MartyG-RealSense]

What is not clear to me is why you have a good depth image of the box in the pictures where you have the depth, RGB and IR together, and a broken depth image in the pictures where it is the depth image on its own. Is the camera position / angle different in the depth-only images please? Thanks for your patience.

[AlphaRalph]

Sorry, several images have been taken with different settings (Presets) where at some the resulting fill factor was higher or lower.
Not the quality of the depth image itself is the problem. The fact that the sides of the box are not perpendicular to the ground floor bothers me, independant from the settings.
The position of the camera is equal for all images.

[MartyG-RealSense]

Can you test if you get a better result with a shallower camera tilt angle such as 20 degrees maximum please?

[AlphaRalph]

Meanwhile I tried something different. I exported the pointcloud from the viewer as ply file and loaded it into the standard Windows 3D-Viewer (Camera setup unchanged).
In the Windows 3D-Viewer the sides of the box seem to be in correct angle in relation to the floor, see image.

It seems to me, that the Intel Realsense 3D Viewer somehow "distortes" the pointcloud.
Furthermore, I was able to tune the advanced settings to be more strict and remove spatial artifacs, that influenced the measurement from the box_dimensioner_multicam example. Fortunately, the measurement result fits now pretty well to the expected result.
I'd like to share the adapted "ShortRangePreset.json" for anyone dealing with the same problem:

{
    "aux-param-autoexposure-setpoint": "1200",
    "aux-param-colorcorrection1": "0.298828",
    "aux-param-colorcorrection10": "-0",
    "aux-param-colorcorrection11": "-0",
    "aux-param-colorcorrection12": "-0",
    "aux-param-colorcorrection2": "0.293945",
    "aux-param-colorcorrection3": "0.293945",
    "aux-param-colorcorrection4": "0.114258",
    "aux-param-colorcorrection5": "-0",
    "aux-param-colorcorrection6": "-0",
    "aux-param-colorcorrection7": "-0",
    "aux-param-colorcorrection8": "-0",
    "aux-param-colorcorrection9": "-0",
    "aux-param-depthclampmax": "6000",
    "aux-param-depthclampmin": "0",
    "aux-param-disparityshift": "15",
    "controls-autoexposure-auto": "True",
    "controls-autoexposure-manual": "8500",
    "controls-color-autoexposure-auto": "True",
    "controls-color-autoexposure-manual": "156",
    "controls-color-backlight-compensation": "0",
    "controls-color-brightness": "0",
    "controls-color-contrast": "50",
    "controls-color-gain": "64",
    "controls-color-gamma": "300",
    "controls-color-hue": "0",
    "controls-color-power-line-frequency": "3",
    "controls-color-saturation": "64",
    "controls-color-sharpness": "50",
    "controls-color-white-balance-auto": "True",
    "controls-color-white-balance-manual": "4600",
    "controls-depth-gain": "16",
    "controls-laserpower": "360",
    "controls-laserstate": "on",
    "ignoreSAD": "0",
    "param-amplitude-factor": "0",
    "param-autoexposure-setpoint": "1200",
    "param-censusenablereg-udiameter": "9",
    "param-censusenablereg-vdiameter": "5",
    "param-censususize": "9",
    "param-censusvsize": "5",
    "param-depthclampmax": "6000",
    "param-depthclampmin": "0",
    "param-depthunits": "100",
    "param-disableraucolor": "0",
    "param-disablesadcolor": "0",
    "param-disablesadnormalize": "0",
    "param-disablesloleftcolor": "0",
    "param-disableslorightcolor": "0",
    "param-disparitymode": "0",
    "param-disparityshift": "15",
    "param-lambdaad": "2000",
    "param-lambdacensus": "39",
    "param-leftrightthreshold": "24",
    "param-maxscorethreshb": "2000",
    "param-medianthreshold": "320",
    "param-minscorethresha": "13",
    "param-neighborthresh": "1",
    "param-raumine": "6",
    "param-rauminn": "1",
    "param-rauminnssum": "1",
    "param-raumins": "1",
    "param-rauminw": "4",
    "param-rauminwesum": "8",
    "param-regioncolorthresholdb": "0.0234834",
    "param-regioncolorthresholdg": "0.0567515",
    "param-regioncolorthresholdr": "0.169276",
    "param-regionshrinku": "4",
    "param-regionshrinkv": "1",
    "param-robbinsmonrodecrement": "11",
    "param-robbinsmonroincrement": "1",
    "param-rsmdiffthreshold": "4.71875",
    "param-rsmrauslodiffthreshold": "0.03125",
    "param-rsmremovethreshold": "0.833333",
    "param-scanlineedgetaub": "849",
    "param-scanlineedgetaug": "515",
    "param-scanlineedgetaur": "736",
    "param-scanlinep1": "5",
    "param-scanlinep1onediscon": "341",
    "param-scanlinep1twodiscon": "248",
    "param-scanlinep2": "56",
    "param-scanlinep2onediscon": "28",
    "param-scanlinep2twodiscon": "415",
    "param-secondpeakdelta": "26",
    "param-texturecountthresh": "0",
    "param-texturedifferencethresh": "500",
    "param-usersm": "1",
    "param-zunits": "100",
    "stream-depth-format": "Z16",
    "stream-fps": "6",
    "stream-height": "480",
    "stream-ir-format": "Y8",
    "stream-width": "848"
}

@MartyG-RealSense thanks for your help, wouldn't be able to figure it out without you.

[MartyG-RealSense]

Great news that you found a solution - thank you for sharing the details with the RealSense community! :)