No difference after On-Chip Self-Calibration

[soheilsbc]

I have an Intel Realsense D415 camera that I don't get a good height. Today I tried On-Chip Self-Calibration on my camera. After the calibration, I have not noticed much difference. On the website, it shows a huge improvement but I have not noticed any. Any suggestion?

You may see the images before and after this Self-Calibration below:

Before:

After:

[agrunnet]

Can you share what the scene looks like as well?
This does look strange.

[soheilsbc]

Here is the scene. The object is placed in a different place but you may see the scene.

[agrunnet]

It does look noisier than I would expect.
Is this a conveyer belt with fast moving items? In that case you may be better off with D435. Also, best if you can move closer.

Since you tried self-calibration using the approach recommended in the white paper, I would be inclined to say that 1. Either the camera calibration has gone completely off or 2. Or the scene is a problem.

In order to determine whether the unit is bad I would recommend pointing at a flat textured target (wall or carpeted floor). Measure the RMS noise with the Image Quality Tool sample app and check whether it is in the range expected for this camera by looking at the white paper describing the expected noise.

Alternatively report back with your distance from target and I can tell you what noise you should expect. In either case pointing at a textured flat wall or white wall with projector and looking at depth map and noise should give clear indication of whether unit is simply bad.

Finally, do you have any other RealSense cameras you can use? Do they all show same bad behavior?

[soheilsbc]

Thanks agrunnet for the reply. Yes it a conveyor belt with fast moving objects. I tried the self-calibration on another D415 but didn't get the results. The distance to the empty conveyor belt is 1044mm.

[agrunnet]

At 1044mm we would expect a depth rms noise of 1.6mm assuming static image and textured surface. This is with no post-processing, like spatial filter or temporal filter.
If you can move to 50cm the RMS noise should he about 0.4mm.
But D415 is not good for high speed motion.

The D435 is much better for that. But like you said you would need to be closer. If you are 30cm for example you would have 90FOV at RMS of about 0.4mm-0.8mm. More importantly you can actually run at 90fps and even at 300fps with reduced vertical window size 848x100 in new FW which is being released soon.

[soheilsbc]

Thanks for the reply. It makes sense. The images that I posted on the top, are taken when the conveyor belt was stopped. I don't expect so much difference with this on-chip self-calibration but at least a minor improvement but I didn't see any difference.

Also, I can not switch to D435. First of all, I have installed the camera and I can not change it. Second, based on the specs, D415 has a higher pixel density, which I believe it’s more accurate for any given area. D415 is more than 2x more accurate than the D435. So I want to stick to D415 camera.

What can be done to make this self-calibration working?

[RealSenseCustomerSupport]

---

Hi,

Based on the info you provided, I suspect the issue is not with the calibration of the camera but more likely the conditions under which it's being used (or possibly some other aspect of the camera). A few comments & questions:

1. It appears that the depth noise is primarily in the background region of the image, which is mostly if not entirely a conveyor belt, which appears to be black. The portion of the image containing the object (a plastic container) appears to be fairly clean and close to expectation. Is this the case?
2. If so, a likely cause of the excess noise is a reduced signal due to absorption of the projector light. Since the background has very little, if any, natural texture, it's completely reliant on the projector to provide a good depth image. If that IR pattern is significantly reduced due to very low reflectivity, the signal is reduced and depth is degraded, similar to an object being very far away.
3. A simple way to confirm or test this would be to temporarily lay a flat but better reflecting material onto the conveyor belt. A piece of cardboard or white/gray sheet of paper large enough to see the effect should work. If you see an image over this ROI more in line with expectations, then would confirm the above hypothesis. I suspect you will see a significant improvement. This would also be a way of doing a more quantitative measurement of the depth metrics using the DQT to determine whether there is any problem with the calibration.
4. If the issue is due to a weak signal, then the things that you can do to potentially improve are:

• Make sure projector power is at max value (360mW).

• Try using a different preset such as high accuracy. It will result in a less dense image but also with less noise.

• Experiment with use of post-processing and filter settings. This should help to improve depth quality but you need to keep in mind the inherent tradeoffs of filtering.

• If possible, reduce the operating distance as much as possible. This will almost always improve performance. You may have already done this or may not be able to for practical reasons, but it's good to keep in mind.

Let me know if you have any questions or issues with these suggestions and let us know if any of it helps.

[soheilsbc]

Thanks for the comments. I tried the experiments that you mentioned by trying the calibration using ambient light or additional light but the result was the same. Then I tried to put a piece of cardboard under the camera and perform the calibration and I have not noticed any difference. I also performed the calibration by placing the object on a white background or checkerboard but again no difference. I have also set the laser power to 360mW in my experiments. Below you can see the screenshots from my experiments.

Cardboard on the background:

Original depth:

New depth:

Checkerboard background with ambient light:

Original depth:

New depth:

Checkerboard background with extra light:

Original depth:

New depth:

White background (White wall):

New depth:

[agrunnet]

@soheilsbc

As was mentioned earlier, this is not likely a calibration issue.
Were you able to try a different RealSense camera so we can rule our a bad camera vs difficult scene? If you can please do so, so we can rule out the camera and get you a replacement.

Thanks for trying the self-calibration. As mentioned in the paper, if you print out the target in the appendix and use that, then you cannot get better results than that. If the problem persists then it is not a calibration issue. This target is an example of a "textured" scene.

Regarding some of your results on calibration, you have very little inherent texture in the scene so if you use with projector on you would need to use "white wall mode". Also, as mentioned in the white paper, a periodic pattern like a checker board is not recommended for this self-calibration.

[jnsweets]

Thanks for the additional info and data. To follow up on previous comments:

Just to clarify, my suggestions were not specifically related to the calibration itself but mainly to test the performance of the camera (assuming its calibration was ok). In any case, it's fine that you tested running the self-cal using the different targets. The health check errors, with one exception, appear to be reasonably small and consistent which does indicate that calibration is not a significant issue or limitation.

Since there is very little texture in your scenes, you should use the white wall mode with D415 when running self-cal. Here is an example using the DQT on a camera about 1m from a white painted wall in a darkened room (so only projector is providing texture for depth).

Here is result after self-cal using white wall mode:

In this case, the camera was fairly well calibrated to begin with and so there is very little change after re-cal.

You can also see the depth metrics which are very slightly improved after re-cal but fairly good (0.3-0.4% RMSE) in either case. This is the type of performance you should see for a D415 over a 40% ROI on a uniform flat surface for a Z1m. It will be better with a well lit and textured surface (and proj off) and over a smaller ROI but this is just for reference.

If you could check the metrics on your camera in a similar manner, then we can confirm whether its calibration is ok or not.

Also, as mentioned, if you use the 'high accuracy' preset, you should see a slight reduction in RMSE but probably more holes in the depth image. Post processing will help even further.

[RealSenseSupport]

Hi @soheilsbc Do you need further help on this issue?
If we don't hear from you in 7 days, this issue will be closed.