Convert pixel coordinates to real world without dense cloud

[SamuelPATE]

Hello

The conversion of  2D pixel coordinates to 3D real world coordinates using surface.pickPoint works perfectly.
Is there a way to do it without the dense cloud, using only depth maps ?

[Paulo]

Samuel hello,

yes this could be done by scripting....

[Alexey Pasumansky]

Hello Samuel,

Yes, should be possible.

From the depth maps you can get the distance from camera to the point, and also you need to use calibration information to estimate the direction from camera center to the point. This information should be sufficient to get the 3D coordinates of that point.

[matttob]

Hi

We have been working on doing this  2D to 3D projection ourselves outside of agisoft in Python because the API command pickPoint works really,  but when we have to transpose 1000s of points we either find that Agisoft crashes or just that the pickPoint commands take a really long to complete.

We have had some success using this method:

https://math.stackexchange.com/questions/4382437/back-projecting-a-2d-pixel-from-an-image-to-its-corresponding-3d-point

by using the the xml and fbx files to construct both the intrinsic and extrinsic matrices required but! we are still only able to get accurate within about 1m whereas the the pickPoint command seems sub CM accurate.

So I guess we would just really like some advice if this is possible without giving away any commercial secrets as to how we can improve our accuracy or if you can think of why pickPoint might be slow for many points or causing crashes.

thanks so much

Matt

[Alexey Pasumansky]

Hello Matt,

Can you please elaborate what information you have in Metashape project apart from the alignment results? Depth maps, dense cloud, mesh? In case of dense point cloud or mesh, please specify the number of points in the cloud or number of faces in mesh.

Also when you say that Metashape crashes, what code you are using to call pickPoint command in loop?

[matttob]

Hi Alexy,

Thanks very much for gettng back to me, we have a mesh with 37 203 247 faces. The code snippet with the API call is:

sensor = cam.sensor
 x = chunk.model.pickPoint(cam.center, cam.transform.mulp(sensor.calibration.unproject(point)))

I think in general we woudl also really like to be able to do 2d-3d (image coordinate to 3d model coordinates) transformation outside of Agisoft using the fundamenal photogramettry equations if possible, but maybe we are never going to be able to be super accurate using the camera positions and associated values from the .xml and .fbx files? do you have any thoughts about wether this will be possible or is Agisoft doing something completely different internally when it runs pickPoint?  Like using bundle adjustment or something like this?

Thanks so much for any insight or help you might be able to offer

Matt