Problem with point cloud generation from depth map

[chekhovana]

My task is to generate point cloud from 2d image segmentation mask for a given object.
Following the post: https://www.agisoft.com/forum/index.php?topic=13569.0, I am trying to solve it in the two following ways:

1. using chunk.point_cloud.pickPoint for each mask pixel:

Code: [Select]

   
chunk = doc.chunk
cam = chunk.cameras[0]
pt_img = Metashape.Vector([img_x, img_y])
pt_3d_picked = chunk.point_cloud.pickPoint(cam.center, cam.unproject(pt_img))

2. using depth map of the camera:

Code: [Select]

depth_map = chunk.depth_maps[cam]
undistorted_x, undistorted_y = depth_map.calibration.project(cam.calibration.unproject(pt_img))
depth_val = depth_map.image()[int(undistorted_x), int(undistorted_y)][0]
ray = cam.unproject(pt_img) - cam.center
ray /= ray.norm()
ray *= depth_val / cam.transform.inv().mulv(ray).z
pt_3d_dmap = cam.center + ray

Method 2 is preferable due to performance reasons.
For many points (approximately 50000 of 65000 in total) the following occurs:

- pickPoint in method 1 returns definite (not None) 3d point
- depth value returned in method 2 is zero, so the method becomes inapplicable

As far as I understand, this shoudn't happen, as pickPoint uses depth value to convert from 2d to 3d.
What am I doing wrong / misunderstand?

Points with nonzero depth calculated by method 2 correspond to results of method 1 quite well.

[Alexey Pasumansky]

Hello chekhovana,

If you check the depth map visually, do you see, whether there are any gaps or holes?

pickPoint call is sending the ray from the camera center through the specified pixel and intersects it with the surface - point cloud in your case. Some interpolation is applied for the point cloud case, as 3D points themselves do not represent continuous surface.

[chekhovana]

Hello Alexey,

thanks for the quick response.

I attached the original image and 2 generated depth maps, built with downscale factors 1 and 2. Black pixels on depth maps correspond to zero depth values. Changing downscale factor from 2 to 1 resulted in better quality of depth map (I am interested in generating point cloud for transmission pole bounded by red rectangle). Are there any other parameters of buildDepthMap method that help to reduce the number of zero-valued depth map points? Filter mode, maximum number of neighbor images, anything else?