i met a problem with d415 creating pointcloud

[MentalBaka1]

---

Required Info
Camera Model	{ D400 }
Firmware Version	(pyrealsense2)
Operating System & Version	{Win10}
Kernel Version (Linux Only)
Platform	PC
SDK Version	2.54.2.5684
Language	{python }
Segment	{others }

Issue Description

I used d415 to generate a depth map and a color map and aligned them.

my saving code like this:

########################################################
depth_image_1 = np.asanyarray(depth_frame_1.get_data())
color_image_1 = np.asanyarray(color_frame_1.get_data())
# Apply colormap on depth image (image must be converted to 8-bit per pixel first)
depth_image_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.5), cv2.COLORMAP_JET)

    # Camera 2
    # Wait for a coherent pair of frames: depth and color
    frames_2 = pipeline_2.wait_for_frames()
    depth_frame_2 = frames_2.get_depth_frame()
    color_frame_2 = frames_2.get_color_frame()
    if not depth_frame_2 or not color_frame_2:
        continue
    # Convert images to numpy arrays
    depth_image_2 = np.asanyarray(depth_frame_2.get_data())
    color_image_2 = np.asanyarray(color_frame_2.get_data())
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.5), cv2.COLORMAP_JET)

    # Stack all images horizontally
    images = np.hstack((color_image_1, depth_image_1,color_image_2, depth_image_2))

    # Show images from both cameras
    cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
    cv2.imshow('RealSense', images)
    # Save images and depth maps from both cameras by pressing 's'
    ch = cv2.waitKey(1)
    if ch==115:
        cv2.imwrite("my_image_1.jpg",color_image_1)
        cv2.imwrite("my_depth_1.jpg",depth_image_1)
        cv2.imwrite("my_image_2.jpg",color_image_2)
        cv2.imwrite("my_depth_2.jpg",depth_image_2)
        print ("Save")

########################################################

and then i want to use open3d to generate pointcloud from this two pictures.
such as
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color_raw, depth_raw, convert_rgb_to_intensity=False)
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd_image, Intrinsic)

the pointcloud i created is completely wrong.

its seems like this

i think the depth is wrong.
can anybody help me?

[MentalBaka1]

depth_image_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.25), cv2.COLORMAP_JET)

and when i change the alpha , i can get different pointcloud

[MartyG-RealSense]

Hi @MentalBaka1 Is your pointcloud correct after changing the alpha, please?

[MentalBaka1]

No, it's still wrong .

[MartyG-RealSense]

There was a Python Open3D case at #12090 where the pointcloud was incorrect. The RealSense user in that particular case found at #12090 (comment) that the problem was with their intrinsics code.

Does testing the script at #10894 (comment) produce a more correct pointcloud, please?

[MentalBaka1]

There was a Python Open3D case at #12090 where the pointcloud was incorrect. The RealSense user in that particular case found at #12090 (comment) that the problem was with their intrinsics code.

Does testing the script at #10894 (comment) produce a more correct pointcloud, please?

In fact, I want to save the depth map and RGB image at that time when I press the 's' key, and then convert the depth map and RGB image into a point cloud offline. I found that if you do not perform the operation "depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.35), cv2.COLORMAP_JET)", the saved depth map will only be in black and white. Also, changing the alpha allows me to get a different depth map, but the point cloud generated with this depth map is still incorrect, can you give me an example of how to save the depth map correctly.Or could you tell me how to create pointcloud from rgb and depth map.
Thanks in advance.

[MentalBaka1]

There was a Python Open3D case at #12090 where the pointcloud was incorrect. The RealSense user in that particular case found at #12090 (comment) that the problem was with their intrinsics code.

Does testing the script at #10894 (comment) produce a more correct pointcloud, please?

This is all my saving code:
import pyrealsense2 as rs
import numpy as np
import cv2

ctx = rs.context()
devices = ctx.query_devices()
dev0 = ctx.query_devices()[0]
dev1 = ctx.query_devices()[1]

device_model0 = str(dev0.get_info(rs.camera_info.name))
device_model1 = str(dev1.get_info(rs.camera_info.name))

print(f'device_model0: {device_model0}')
print(f'device_model1: {device_model1}')

print('Config ... ')
pipeline_1 = rs.pipeline()
config_1 = rs.config()
config_1.enable_device(dev0.get_info(rs.camera_info.serial_number))
config_1.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config_1.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

pipeline_2 = rs.pipeline()
config_2 = rs.config()
config_2.enable_device(dev1.get_info(rs.camera_info.serial_number))
config_2.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config_2.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

Start streaming from both cameras

profile1 = pipeline_1.start(config_1)
profile2 = pipeline_2.start(config_2)

intr = profile1.get_stream(rs.stream.color).as_video_stream_profile().get_intrinsics()
print(intr)
try:
while True:

    # Camera 1
    # Wait for a coherent pair of frames: depth and color
    frames_1 = pipeline_1.wait_for_frames()
    depth_frame_1 = frames_1.get_depth_frame()
    color_frame_1 = frames_1.get_color_frame()
    if not depth_frame_1 or not color_frame_1:
        continue
    # Convert images to numpy arrays
    depth_image_1 = np.asanyarray(depth_frame_1.get_data())
    depth_image_11 = np.asanyarray(depth_frame_1.get_data())
    color_image_1 = np.asanyarray(color_frame_1.get_data())
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_image_1 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_1, alpha=0.35), cv2.COLORMAP_JET)

    # Camera 2
    # Wait for a coherent pair of frames: depth and color
    frames_2 = pipeline_2.wait_for_frames()
    depth_frame_2 = frames_2.get_depth_frame()
    color_frame_2 = frames_2.get_color_frame()
    if not depth_frame_2 or not color_frame_2:
        continue
    # Convert images to numpy arrays
    depth_image_2 = np.asanyarray(depth_frame_2.get_data())
    depth_image_22 = np.asanyarray(depth_frame_2.get_data())
    color_image_2 = np.asanyarray(color_frame_2.get_data())
    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_image_2 = cv2.applyColorMap(cv2.convertScaleAbs(depth_image_2, alpha=0.35), cv2.COLORMAP_JET)

    # Stack all images horizontally
    images = np.hstack((color_image_1, depth_image_1,color_image_2, depth_image_2))

    # Show images from both cameras
    cv2.namedWindow('RealSense', cv2.WINDOW_NORMAL)
    cv2.imshow('RealSense', images)
    # Save images and depth maps from both cameras by pressing 's'
    ch = cv2.waitKey(1)
    if ch==115:
        cv2.imwrite("my_image_1.jpg",color_image_1)
        cv2.imwrite("my_depth_1.jpg",depth_image_1)
        cv2.imwrite("my_image_2.jpg",color_image_2)
        cv2.imwrite("my_depth_2.jpg",depth_image_2)
        print ("Save")
    if ch==27:
        cv2.destroyAllWindows()
        break

finally:

# Stop streaming
pipeline_1.stop()
pipeline_2.stop()

[MentalBaka1]

Oh, I have solved it, saving the depth map in png format is no problem, anyway, thank you very much

[MartyG-RealSense]

You are very welcome. It's great to hear that you achieved a solution. Thanks very much for the update!

[MartyG-RealSense]

Hi @MentalBaka1 Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.