Identifying the location of a dense cloud point within a raw photo

[dyoung]

Hello -- if I select a specific point in my dense cloud (or any arbitrary point in 3D space), is there any way I might be able to "back-calculate" where that point would be in one of my raw photos (i.e., which pixel of the raw photo is closest to it)? I realize this is a very unusual request, but I wanted to ask just in case someone has a clever solution.

The reason I ask is that I would like to be able to mark the top and bottom of a tree in my point cloud (or simply specifying the 3D coordinates) and then identify where those two points would be (pixel coordinates) in one of my photos.

Thanks for any insight!

[Alexey Pasumansky]

Hello dyoung,

Yes, you can project a 3D point to the cameras and get the 2D coordinates in the image space using Python. Do you need an example?

[dyoung]

Hi Alexey --  Excellent! Yes, I would love an example, if you have time. Thank you!

[fjgarciam]

Hello dyoung,

Yes, you can project a 3D point to the cameras and get the 2D coordinates in the image space using Python. Do you need an example?

can be done for a shape on the orthomap? i can import a shape and have all the frames with 2d coordinates of the shape?

[Paulo]

Yes, just select the shape and with right click menu, select Filter Photos by Shapes and filtered images appear in Photos pane...

[dyoung]

Hello Alexey,

I would still be very interested in some tips or examples for how to project a 3D point to the cameras and get the 2D coordinates in the image space. Is this done through the Metashape Python API, or other Python libraries? I understand that this can be done using geometry, but my understanding of focal length, sensor sizes, etc. is too limited to write an algorithm from scratch. Is there any documentation you could point me toward?

Thank you in advance!

[chrisbin07]

Hello dyoung,

Yes, you can project a 3D point to the cameras and get the 2D coordinates in the image space using Python. Do you need an example?

Hi Alexey,
I'm currently trying to do this. Can you please share the example. Thanks

[Alexey Pasumansky]

Hello,

Can you please check on smaller dense cloud, if the following code provides the desired result and save for each dense cloud point the following information:
point_id, XYZ geographic coordinates and list of cameras and related 2D projections in pixel coordinates:

Code: [Select]

import Metashape

output = Metashape.app.getSaveFileName("Specify the path to output file:", filter = "*.txt")

chunk = Metashape.app.document.chunk
T = chunk.transform.matrix
crs = chunk.crs
point_cloud = chunk.point_cloud
aligned_cameras = [c for c in chunk.cameras if c.transform and c.type == Metashape.Camera.Type.Regular]

pc_reader = Metashape.PointCloud.Reader()
pc_reader.open(point_cloud)

read_step = 1000
n_points = point_cloud.point_count
file = open(output, "wt")
point_id = 0

for step in range(n_points // read_step + 1 * bool (n_points % read_step)):
points = pc_reader.read(read_step)
for point in points:
point_int = point.position
point_geo = crs.project(T.mulp(point_int))
file.write("{:d},{:.6f},{:.6f},{:.6f}\n".format(point_id, point_geo.x, point_geo.y, point_geo.z))
for camera in aligned_cameras:
vect_2D = camera.project(point_int)
if not vect_2D:
continue
if (0 <= vect_2D.x < camera.sensor.width) and (0 <= vect_2D.y < camera.sensor.height):
file.write("{:s},{:.2f},{:.2f}\n".format(camera.label, vect_2D.x, vect_2D.y))

point_id += 1
file.flush()

pc_reader.reset()
file.close()
print("Done")