Pixel coordinates projection from image to 3D mesh model

[jrh87]

I want to project pixel coordinates from an image to the 3D mesh model as per the scripts below. However, no matter the approach or the conversions, it is not returning the results I can see when opening the mesh and selecting a point either with Cloudcompare or Open3D (as per attached screenshot). What am I missing?

import Metashape

mesh_path = "model.obj"

def project_pixel_to_mesh(camera, pixel_coord, chunk):

    mesh = chunk.model  # Get the 3D model (mesh)

    T = chunk.transform.matrix
    pt_2d = Metashape.Vector(pixel_coord)
    pt_3d = mesh.pickPoint(camera.center, camera.unproject(pt_2d))
    pt_3d_world = chunk.crs.project(T.mulp(pt_3d))
    print('Camera_center', camera.center)
    print('pt_3d', pt_3d)
    print('pt_3d_world', pt_3d_world)

doc = Metashape.Document()
doc.open(path="test.psx")
chunk = doc.chunkcamera = chunk.cameras[0]
pixel_coord = (4000, 3000)  # Example pixel coordinates
project_pixel_to_mesh(camera, pixel_coord, chunk)

The console output is:

Camera_center Vector([0.25185996974317976, -9.252179636717418, -2.411546712801472])
pt_3d Vector([1.6831388473510742, -9.926933288574219, -15.181743621826172])
pt_3d_world Vector([113.96841070820898, 22.387860599545277, 89.6002808787151])

Whereas the projected coordinates should be something close to: [-21, -23, 85]

[Paulo]

Hi jrh,

what is your chunk.crs? If it is geographic then you should convert to some cartesian CS (projected or local) before exporting your model, points to CC,,

Hope this can help you,

[jrh87]

Hi Paulo,

The chunk.crs is <CoordinateSystem 'WGS 84 (EPSG::4326)'>

I am exporting my model using chunk.exportModel(path=mesh_path, format=Metashape.ModelFormatOBJ, texture_format=Metashape.ImageFormat.ImageFormatJPEG, save_texture=True).

Thanks!

[jrh87]

Thanks for the suggestion Paulo. Looks like the script below does the trick

chunk.crs = None # Set the local CRS <CoordinateSystem 'Local Coordinates (m)'>
T = chunk.transform.matrix # Get the transformation matrix
origin = Metashape.Vector([T[0, 3], T[1, 3], T[2, 3]])
T = Metashape.Matrix.Translation(-origin)*T
chunk.transform.matrix = T  # Preserve transformations

chunk.exportModel(path=mesh_path, format=Metashape.ModelFormatOBJ, texture_format=Metashape.ImageFormat.ImageFormatJPEG, save_texture=True, crs=chunk.crs)