How to get point cloud from depth&RGB images captured from D435i?

[Jordy-Li]

Required Info
Camera Model	D435i
Firmware Version	05.12.14.50
Operating System & Version	win10
Platform	PC
SDK Version	2.0
Language	python
Segment	others

Hello, I'm making a program written in Python on Windows 10 using D435i camera.

My goal is to get the 3D coordinate (x, y, z) and RGB to reconstruction a plant. My limitation right now is that I only have depth images and color images also I can't get the external reference.

I can find out the internal reference，and I'm try to use Open3d and PCL to align depth and color to get the point clouds but the result goes wrong，the point cloud has he problem of distortion. That can't mapping out real values from point cloud.

I need some help on 2 things (an example code would be better):

How can I get the camera external reference(may it can help me to get real point cloud)?
Is there other way to get the point cloud (to output .pcd file) ?

Please let me know if there are things that are unclear or some info that I should have included above.
Thank you in advance.

[MartyG-RealSense]

Hi @Jordy-Li If you are creating a pointcloud by performing depth to color alignment and then obtaining the 3D real-world point cloud coordinates with rs2_deproject_pixel_to_point then the use of alignment may result in inaccuracies. Using points.get_vertices() instead to generate the point cloud should provide better accuracy. This subject is discussed in detail in the Python case at #4315

#4210 has a Python PCL script that saves pointcloud data as a pcd.

---

If you are able to use Python only instead of Open3D and PCL then the RealSense SDK's Python example program opencv_pointcloud_viewer.py may be a helpful Python point cloud reference.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/opencv_pointcloud_viewer.py

If you need to use Open3D, a Python point cloud script shared by a RealSense user at isl-org/Open3D#473 (comment) that makes use of Open3D may provide useful insights.

[Jordy-Li]

Hi @Jordy-Li If you are creating a pointcloud by performing depth to color alignment and then obtaining the 3D real-world point cloud coordinates with rs2_deproject_pixel_to_point then the use of alignment may result in inaccuracies. Using points.get_vertices() instead to generate the point cloud should provide better accuracy. This subject is discussed in detail in the Python case at #4315

#4210 has a Python PCL script that saves pointcloud data as a pcd.

If you are able to use Python only instead of Open3D and PCL then the RealSense SDK's Python example program opencv_pointcloud_viewer.py may be a helpful Python point cloud reference.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/opencv_pointcloud_viewer.py

If you need to use Open3D, a Python point cloud script shared by a RealSense user at isl-org/Open3D#473 (comment) that makes use of Open3D may provide useful insights.

I've tried this before, but it seems that this method can only be applied to the running process of data flow, it can't process the depth&rgb images off line. When I use this method I need numpy to work with it then output 'pointcloud.txt', it's too slow for me to get the real point cloud. May any way more faster?

The code I have tried before:

pc = rs.pointcloud()
pc.map_to(color_frame)
points = pc.calculate(aligned_depth_frame)
vtx = np.asanyarray(points.get_vertices())
with open(file_pcd1 + '{:0>6d}'.format(frame_count) + '.txt', 'w') as f:
for i in range(len(vtx)):
f.write(str(np.float64(vtx[i][0]) * 1000) + ' ' + str(np.float64(vtx[i][1]) * -1000) + ' ' + str(
np.float64(vtx[i][2]) * -1000) + ' ' + str(np.float64(colorful1[i][0])) + ' ' + str(
np.float64(colorful1[i][1])) + ' ' + str(np.float64(colorful1[i][2])) + '\n')
print("Point cloud 1 saved.")

[MartyG-RealSense]

If you need to export a point cloud with color data to a file, you could consider creating a .ply point cloud data file instead, which is easily importable into other programs. Information about using the export_to_ply instruction in Python to export a point cloud to ply with color included can be found at #6194 (comment)

[MartyG-RealSense]

Hi @Jordy-Li Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.