Sdk pointcloud

[Erol444]

TODO:

• docs
• example
• find optimal stereo postprocessing filters, try open3d's voxelization and removing statistical outliers (already played around with it)

Minimal example with rerun visualization:

import cv2
from depthai_sdk import OakCamera
from depthai_sdk.classes.packets import PointcloudPacket, FramePacket
import rerun as rr

import subprocess
subprocess.Popen(["rerun", "--memory-limit", "200MB"])
rr.init("Rerun ", spawn=False)
rr.connect()
def callback(packet: PointcloudPacket):
    colors = cv2.cvtColor(packet.color_frame.getCvFrame(), cv2.COLOR_BGR2RGB)
    rr.log_image('Color Image', colors)
    points = packet.points.reshape(-1, 3)
    rr.log_points("Pointcloud", points, colors=colors.reshape(-1, 3))

with OakCamera() as oak:
    pcl = oak.create_pointcloud()
    oak.callback(pcl, callback=callback)
    oak.start(blocking=True)

[Erol444]

Updated depth visualization - convert to disparity first instead of normalization/histogram. Mouse overlay now looks like this for depth:

[Erol444]

Copying from luxonis/depthai-python#809, depth visualization:
Processing time, code here: https://pastebin.com/HgjsdTwK. For depthai-python examples I'd use proposed approach, as depth-to-disp takes 30LoC. In SDK we will rather use depth-to-disp.

400P:
depth to disp avrg: 2.5ms
depth percentile (my method) avrg: 4 ms
normalize (current method): 1ms
800P:
percentile 15ms
normalize: 2.5ms
depth to disp: 10ms

[Erol444]

0_depth.2023-05-05.16-17-54.mp4

[zrezke]

LGTM, thanks!