which coordinates should be exported for dense point cloud to match camera.xml

[shunfu]

Hi, guys

     My goal: Select any point from the dense point cloud reconstructed by metashape, and use the camera internal and external parameters in camera.xml exported from metashape to trace back to multiple two-dimensional pictures containing the selected point.

     My confusion: The image I used to reconstruct the dense point cloud is with GPS information. I do not konw which coordinates should be exported for dense point cloud to match camera.xml. Besides, From the forum, I saw many people mention chunk transform. I don't know what this means?

    I hope I could state my problem clearly and hopefully I have not missed the answer in some other thread of this forum.

Thanks for your help, guys!

[shunfu]

I use the following formula to trace a two-dimensional picture of a three-dimensional point:
[u,v,1]=K*RT*[x,y,z,1]
where K and RT are from camera.xml, [x,y,z,1] is from dense point cloud coordinates.

[Paulo]

Hello shunfu,

the formula you show goes from project internal coordinates (x,y,z) to camera pixel coordinates (u,v).

If you have a point P in world coordinates :

P= Metashape.Vector((X,Y,Z))

then to get the coordinates in chunk internal coordinate system (x,y,z) you do:

chunk.transform.matrix.inv().mulp(chunk.crs.unproject(P))

Hope this is clear,

[shunfu]

Hi, Paulo

    I'm still confused about the coordinate system of metashape.
    The chunk of the dense point cloud I reconstructed is WGS 84 coordinate. When exporting dense point clouds, I chose WGS84/ UTM zone 49N as coordinate system.
    my goal:Select a point P (x,y,z) in WGS84/ UTM zone 49N as coordinate system, use the camera internal and external parameters in camera.xml exported from metashape to trace back to multiple two-dimensional pictures containing the selected point.
    So, i want to use following formula to trace a two-dimensional picture of the point P in WGS84/ UTM zone 49N:
    [u,v,1]=K*RT*chunk.transform.matrix.inv()*P
    Is that right ？
   Thanks for your help！
shunfu




   

[Paulo]

shunfu,

given the point in WGS84/ UTM zone 49N CS, you would need first to apply chunk.unproject(P) to transform to geocentric coordinates and then use

 [u,v,1]=K*RT*chunk.transform.matrix.inv().mulp(chunk.unproject(P)) where RT is the inverse camera transform and K is calibration....

Hope you can get ahead,

[shunfu]

Hi,Paulo

    Thank you for your kind help.
     I tried your code, but it didn't seem to work. Maybe I took some steps wrong. My setting is below:

     My setting :
     1)chunk.world.crs:'WGS 84(EPSG::4978)' geocentric coordinates
     2)chunk.crs:'WGS 84(EPSG::4326)' geographic coordinates
     3)chunk.dense_cloud.crs:none
     4)export dense point cloud to 'local coordinates or WGS 84/UTM zone 49N'

     My code:
     [u,v,1]=K*RT*chunk.transform.matrix.inv().mulp(chunk.unproject(P))
     where P was a point selected from export dense point cloud in 'local coordinates or WGS 84/UTM zone 49N',
     K and RT was obtained from camera.xml exported from metashape and did not change. 

     My understanding towards metashape coordinates system:
   (1) chunk.unproject(P): from 'local coordinates or WGS 84/UTM zone 49N' to WGS 84(EPSG::4326)' geographic coordinates
   (2) chunk.transform.matrix.inv().mulp(): from WGS 84(EPSG::4326) geographic coordinates to 'WGS 84(EPSG::4978)' geocentric coordinates.

     I'm sorry to bother you these days. If you're free, please contact me.

best wishes,
shunfu
     

[Paulo]

Hello shunfu,

let me try again to explain. I have a chunk georeferenced in WGS84 / UTM 10N CS. Given a point P with (683479.174414,5618398.647569,349.670) coordinates then following screen shows how to:

- go to geocentric coordinates ESPG 4978 with chunk.crs.unproject();
- go to internal CS coordinates with chunk.transform.matrix.inv().mulp();
- go to u,v camera pixel coordinates for a given camera where P is imaged with camera.project().

Note that camera.project() method combines the K (calibration) and RT (invrse camera transform) from cameras.xml and is more complete as K calibration matrix does not include radial and tangential distorsion...

Hope this is clearer for you,

[Paulo]

Shunfu,

the following excel file shows the different matrix operations involved in going from Point geocentric coordinates to camera pixel u,v with or without distorsion applied....

This can be insightful,

[shunfu]

Hi,Paulo

     Thank you ! it works!

      A 3D point P has been traced back by camera.project(P) to many qualified images. However, some of these images do not contain P points because of some occlusion.

      Therefore, I want to know whether there are records(something like tracks) stored for each point P in metashape python API. because each point is reconstructed from a pixel of many pictures.

Best Wishes,
Shunfu

[Paulo]

Hello shunfu,

to just show projections from a given point to cameras without obstruction, one could use the pickPoint function to see if inverse projection from camera pixel position back to model corresponds to original model point. In following example, I placed 2 points (1 and 2) on model and found 10 unobstructed projections for point 1 and 19 for point 2. Point 1 has less unobstructed projections as it is near the foot of house wall see attachment...

This aplication could be deveopped... please send me PM for more details