Orientating mesh as per camera view

[jrh87]

Hello,

I am capturing a building facade using drone images. From the resulting 3D model, I would like to obtain a planar view of the facade as per the drone view. How could I orientate the mesh accordingly?

My current approach is, for one of the (5280x3956) pictures looking straight at the facade, I project 3 points to build a plane. Then I try to create the rotation matrix accordingly as per the script below. But the results are wrong.

import Metashape
import numpy as np

doc = Metashape.Document()
doc.open(path=project_path)
chunk = doc.chunk

chunk.crs = None # Set <CoordinateSystem 'Local Coordinates (m)'>
T = chunk.transform.matrix # Get the transformation matrix
origin = Metashape.Vector([T[0, 3], T[1, 3], T[2, 3]])
T = Metashape.Matrix.Translation(-origin)*T
chunk.transform.matrix = T  # Preserve transformations

for camera in chunk.cameras:

    if camera.label == 'DJI_0268_V':

        T = chunk.transform.matrix

        p1 = camera.unproject(Metashape.Vector([0,0]))
        p2 = camera.unproject(Metashape.Vector([5280,0]))
        p3 = camera.unproject(Metashape.Vector([0,3956]))

        ortho = np.cross(np.array(p2-p1), np.array(p3-p1))

        p1 = p1/np.linalg.norm(p1)
        p2 = p2/np.linalg.norm(p2)
        p3 = Metashape.Vector(ortho/np.linalg.norm(ortho))

        p1 = chunk.crs.project(T.mulp(p1))
        p2 = chunk.crs.project(T.mulp(p2))
        p3 = chunk.crs.project(T.mulp(p3))

        rotation_matrix = np.column_stack((p1, p2, p3))

        break

Any suggestion?

[Alexey Pasumansky]

Hello jrh87,

I think it should be easier to use camera rotation matrix to define the projection plane, if you say that it corresponds to the desired plane.

[jrh87]

This should make it then:

T1 = np.array(camera.transform).reshape(4,4)[:3,:3]
T2 = np.array(chunk.transform.matrix).reshape(4,4)[:3,:3]
rotation_matrix = (T2.dot(T1)).T