Replicating GUI Marker Placement on Masked Images via API

[joswil]

I am trying to reproduce the manual marker placement in Metashape (the Create Marker option in the GUI / right‑click → Add Marker) using the Python API.

The point I want to place lies on a masked part of the image (I am using a turntable shooting setup). When I place the marker manually in the GUI, Metashape successfully estimates the 3D XYZ position, even though the pixel is masked.

However, I cannot reproduce this behavior through the API.

I tried: marker.projections[camera] = (imgX, imgY). This correctly places the marker projection on the image, but Metashape does not compute its 3D XYZ position.

I also tried the solution suggested here: https://www.agisoft.com/forum/index.php?topic=7446.0 but it does not work for my case, because chunk.point_cloud.pickPoint() searches for the nearest point in the sparse cloud, which is far away from the actual location (the masked region contains no tie points).

Could you please help me reproduce the GUI marker placement behavior through the API?

Thanks

[Alexey Pasumansky]

Hello joswil,

Here is the example of Python script that works similar to Add Marker command applied to the image in Photo view:

Code: [Select]

import Metashape

chunk = Metashape.app.document.chunk #active chunk

N = 10 #camera id
x, y = 3000, 2000 #pixel coordinates for marker on Nth camera in the chunk

if chunk.point_cloud:
surface = chunk.point_cloud
elif chunk.tiled_model:
surface = chunk.tiled_model
elif chunk.model:
surface = chunk.model
else:
surface = chunk.tie_points

cameras = [c for c in chunk.cameras if c.type == Metashape.Camera.Type.Regular and camera.transform]
camera = chunk.cameras[N]
marker = chunk.addMarker(camera.unproject([x,y]))
point = surface.pickPoint(camera.center, camera.transform.mulp(camera.sensor.calibration.unproject(Metashape.Vector([x,y]))))

for camera in cameras:
if not camera == chunk.cameras[N]:
continue
if marker.projections[camera]: #skipping, if marker has projections on the given image
continue
x, y  = camera.project(point)
if (0 <= x < camera.sensor.width) and (0 <= y < camera.sensor.height):
marker.projections[camera] = Metashape.Marker.Projection(Metashape.Vector([x,y]), False)
print("Script finished")

Scripts sends the ray for Nth camera through x,y coordinates of the image, intersects it with the available 3D surface in the following order, depending on presence in the active chunk: dense point cloud, tiled model, mesh model, tie point cloud. Then re-projects the intersection point back to all the other aligned cameras.

This should work in 2.3 version.