Very fluctuated point cloud and distorted objects

[sergorl]

---

Required Info
Camera Model	D455
Firmware Version	2.38.1.2223
Operating System & Version	Win 10
Platform	PC
SDK Version	2.38.1
Language	python
Segment	others

I use high accuracy preset in "RealSense Viewer". I got point cloud from D455 camera and see all objects (with heights 0.1, 0.2, 0.3 m ) are on distance more than 3 meters are very distorted (especially by height), also I have floor fluctuation with deviation amplitude 0.1 m and looks like point cloud "breathes" - I watch many fluctuations on floor, walls, and objects.

What's the nature of such fluctuations? I want to understand the physics. How can I fix it?

[MartyG-RealSense]

Hi @sergorl When using the High Accuracy preset, accuracy is prioritized over the amount of detail ('fill rate') in the scene. You may get a more detailed scene whilst still achieving accuracy if you use the Medium Density preset that balances between accuracy and fill-rate.

If you have a dark grey carpet as the RGB elements of the images would seem to suggest, you may not get much depth detail from the floor. This is because general physics dictates that dark grey and black colors absorb light, and this makes such surfaces hard for the camera to read depth from unless strong illumination is cast on the surface. The darker the shade of the surface is, the harder it is for the camera to analyze it for depth detail.

In the absence of a strong light source on the floor suface, casting the infrared dot pattern projection onto the floor may help to provide the camera with a texture source to analyze for depth detail.

If fluorescent ceiling lights (lights that have a heated gas inside them instead of a conventional bulb) are in the scene then this may generate noise on the image, because fluorescent lights can flicker at a rate that is hard to see with the human eye. If the lights are fluorescent, setting a camera FPS value close to the operating frequency of the lights may help to reduce the impact. For some lights the appropriate frequency may be 60 FPS and for others it may be 30 FPS. Having said that, I am not convinced that the ceiling lights are a significant factor in your particular scene.

There is the possibility though that your tiled ceiling is creating a repetitive pattern of similar horizontal / vertical arrangements that confuses the camera. This may also cause the kind of floaty 'ghost noise' areas that do not represent the location of actual objects that are appearing in your images. Resources that provide information about this phenomenon and ways to combat it can be found in the link below.

#7792 (comment)

Though surfaces can becomore more wavy on a point cloud as the camera becomes further from the observed surface, the extremely distorted waviness you are experiencing may be another form of ghost noise.

#4553

[sergorl]

@MartyG-RealSense , thank you for your assistanse! I have following up question:

how can I tune/choose the infrared dot pattern projection? Is it possible to tune it by real sense software?

[MartyG-RealSense]

If you are using Python then you can control the on / off state of the IR emitter with depth_sensor.get_option(rs.option.emitter_enabled)

The link below has example Python code for this function:

https://community.intel.com/t5/Items-with-no-label/How-to-enable-disable-emitter-through-python-wrapper/m-p/547902/highlight/true#M8384

Setting the emitter to '0' = disabled and setting it to '1' = enabled

Another Python script that makes use of it is here:

#4095

[sergorl]

@MartyG-RealSense , you gave me examples how to switch on/off IR emitter. But what about tuning the infrared dot pattern projection? For me feature of switch on/off is not like tuning... What about tuning? Is it possible to change infrared dot pattern projection (not just to switch of/on)? And I tried it, but there is no effect.

[MartyG-RealSense]

You can affect the visibility of the IR dot pattern by changing the Laser Power value. Increasing the value makes the pattern more visible and decreasing the value makes the pattern less visible. This is not an ideal method if you are using a depth stream though, as reducing the Laser Power can make the detail on the depth image more sparse. More information about the pattern is available in the link below.

#3997 (comment)

As far as I know the semi-random dot pattern from the RealSense cameras' projector cannot be customized. There is the option though of using an external projector with the camera instead of the built-in one. An LED projector may also provide less 'noise' on the image than the laser-based projector in the camera, as the laser projector can have a noise phenomenon called 'laser speckle'.

https://dev.intelrealsense.com/docs/projectors

[sergorl]

@MartyG-RealSense , increasing laser power didn't give me significant positive effect: moreover, I don't see changes

[MartyG-RealSense]

Considering the possibility that the tiled ceiling could be causing disruption by creating a repetitive pattern, does the image improve if you tilt the camera down a bit so that the ceiling is not in the camera view?

[sergorl]

I tried it, but didn't see improvements. Beside, I worry about only objects on the floor, because I need stereo camera for object volume estimation. But since object height is always distorted I conclude D455 is not so good for such tasks.

[MartyG-RealSense]

D455 has 2x the depth measuring accuracy over distance of the D435 models, meaning that the D455 has the same depth accuracy at 6 meters distance that the D435 models have at 3 meters distance. The D455 is therefore a very good choice for long-range depth measurement.

Do you have similar disruptions in the Viewer on a depth-only point cloud that does not have RGB mapped to it?

[sergorl]

Yes

[MartyG-RealSense]

If you have auto-exposure enabled, could you test the scene in the Viewer with the auto-exposure option disabled (by left-clicking on the blue box beside it to turn it to the Off-state of black) and let me know the results please?

[MartyG-RealSense]

Hi @sergorl Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.