11
« on: May 25, 2022, 01:38:24 PM »
Hi,
So I am trying to compute 3D world coordinates for certain pixels within my aligned photo. However, I do not want to intersect the camera ray with the 3D model (as the pickPoint() function does) but I want to intersect it with a custom plane (which I have computed from the sparse point cloud). The reason is that the 3D model is not available or sometimes has holes.
For intersecting the camera ray of a certain pixel (X/Y) with the plane I need to compute the direction vector (cameraCenter to Pixel). As I understand the upper right 3X3 matrix of rotation = np.asarray(segmentCam.transform).reshape(4,4)[0:3,0:3] contains the rotation matrix that allows the transformation of the pixel coordinates in the 3D world coordinates. On this I then apply the chunk transformation matrix to derive the real world coordinates:
M = np.asarray(chunk.transform.matrix).reshape(4,4)[0:3,0:3] # extracting the transformation from internal Metashape to world coordinates
rotation_M = np.matmul(M,rotation)
and this Matrix I finally apply on the internal camera coordinates which I picked (and corrected using the camera calibration parameters):
b = np.matmul(np.transpose(rotation_M), b_loc)
However, the results are not correct and I strongly suspect that I am misunderstanding the two transformation matrices. Could somebody help me with estimating the direction vector (cameraCenter to Pixel)?
Thanks!
Here is the complete code for a better understanding:
segmentCam = chunk.cameras[0]
R, C = getProjectionPlane(chunk) # derive plane from sparse point cloud (works fine)
# R....3x3 matrix containting the plane defining direction vectors
# C...barymetric center of the plane (EPSG:31256)
C_camera = chunk.crs.project(chunk.transform.matrix.mulp(segmentCam.center)) # 3D coordinates of Camera center [m] (EPSG: 31256)
row = np.asarray([3000,1000]) # image coordinates [px]
b_temp1 = np.append(row,0) # homogeneous image coordinates [px]
b_temp2 = np.array([segmentCam.calibration.cx, segmentCam.calibration.cy, # calibration parameters of the camera
segmentCam.calibration.f])
b_loc = b_temp1-b_temp2 # image coordinates in the "local" camera coordinate system
rotation = np.asarray(segmentCam.transform).reshape(4,4)[0:3,0:3] # extractin the camera transformation parameters (3X3 matrix)
M = np.asarray(chunk.transform.matrix).reshape(4,4)[0:3,0:3] # extracting the transformation from internal Metashape to world coordinates
rotation_M = np.matmul(M,rotation) # combining the matrices by multiplying
b = np.matmul(np.transpose(rotation_M), b_loc) # applying the transformations to the picked camera coordinates
b_norm = b / np.linalg.norm(b) # normalizing (not really necessary)
C_plane = Metashape.Vector(C) # barycenter of the plane
a1 = np.asarray(R).reshape(3,3)[0,0:3] # first plane direction vector
a2 = np.asarray(R).reshape(3,3)[1,0:3] # second plane direction vector
A = np.transpose(np.array([a1,a2,-b])) # solve equation (intersect plane with image ray)
param = np.matmul(np.linalg.inv(A),(C_camera-C_plane))
S1 = a1*param[0] + a2*param[1] + C_plane # apply parameters to the equation and retrieve the
# intersection of the image ray with the plane -> S1==S2
S2 = b*param[2] + C_camera
Show Posts
