Forum

Author Topic: Back-projection of 3D points into images  (Read 5419 times)

hati

  • Newbie
  • *
  • Posts: 4
    • View Profile
Back-projection of 3D points into images
« on: January 12, 2017, 06:32:00 PM »
Hi,

I have been trying to back-project 3D points (from a dense point cloud) into the images that were used for 3D reconstruction in Photoscan, but I have not been successful so far. Here is a sample data extracted from the XML file:

Sensor data:
<sensor id="0" label="ILCE-5100 (35 mm)" type="frame">
        <resolution width="6000" height="4000"/>
        <property name="pixel_width" value="0.0040384615384615376"/>
        <property name="pixel_height" value="0.0040384615384615376"/>
        <property name="focal_length" value="35"/>
        <property name="fixed" value="false"/>
        <calibration type="frame" class="adjusted">
          <resolution width="6000" height="4000"/>
          <fx>9224.096556427452</fx>
          <fy>9221.4743472578975</fy>
          <cx>2984.1827998108429</cx>
          <cy>1957.1578685750774</cy>
          <skew>-0.8817199952550705</skew>
          <k1>0.07592812059486298</k1>
          <k2>-0.65150688706789472</k2>
          <k3>3.4940427181259963</k3>
          <k4>-9.8701638118834438</k4>
          <p1>-0.001219949431043783</p1>
          <p2>0.0010383172308077325</p2>
        </calibration>
 </sensor>

Camera/Image data:
<camera id="75" label="1473949355_001042_IMG_0170.jpg" sensor_id="0" enabled="true">
        <meta>
          <property name="geoid/ellipsoid_height" value="2080.67541497234"/>
        </meta>
        <transform>9.0348028508963163e-01 -3.9801074495336441e-01 1.5909375021045291e-01 -2.3345357516553039e-01 4.2030105929495776e-01 8.9545274587834001e-01 -1.4666764965212120e-01 4.8508265946113938e-01 -8.4085634979432958e-02 1.9937860166178545e-01 9.7630823984513038e-01 4.5984825426855175e-01 0.0000000000000000e+00 0.0000000000000000e+00 0.0000000000000000e+00 1.0000000000000000e+00</transform>
        <orientation>1</orientation>
        <reference x="-104.73608301693" y="38.918854436339998" z="2097.99301497234" yaw="-94.37929796761" pitch="6.7643150325299999" roll="-30.119596380050002" enabled="true"/>
 </camera>


Transformation data:
<transform>
      <rotation>-3.1027677460040076e-01 8.4421578532726338e-01 4.3706753591160719e-01 7.2242930570434949e-01 -8.9450467921191323e-02 6.8563438657072284e-01 6.1791926770663474e-01 5.2848682253482149e-01 -5.8213182098554328e-01</rotation>
      <translation>-1.2643645917064098e+06 -4.8071305115363048e+06 3.9866305644460404e+06</translation>
      <scale>1.6814570146121601e+00</scale>
 </transform>


I am trying to project a point with the following coordinates (522878.864998, 4307805.3358, 2077.556198) that should project close to pixel coordinates of (2742, 1435).

I would appreciate if you help me to figure out the correct process.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Back-projection of 3D points into images
« Reply #1 on: January 12, 2017, 07:28:00 PM »
Hello hati,

Is the chunk referenced in any geographic/projected coordinate system?

Also it looks to me that you have the cameras checked on in the Reference pane (having WGS84 coordinates), while the coordinates of the point you wish to project back to the photo are clearly not in WGS84.
Best regards,
Alexey Pasumansky,
Agisoft LLC

hati

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Back-projection of 3D points into images
« Reply #2 on: January 12, 2017, 07:42:42 PM »
Hi Alexey,

Thanks for the quick reply. I think by the chunk being referenced in any geographical/projected coordinate system, you are referring to the following:

<reference>GEOGCS["WGS 84",DATUM["World Geodetic System 1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9102"]],AUTHORITY["EPSG","4326"]]</reference>

Is there a way for me convert my point coordinates to WGS84?

Thanks

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Back-projection of 3D points into images
« Reply #3 on: January 12, 2017, 07:46:58 PM »
Hello hati,

And in which coordinate system point (522878.864998, 4307805.3358, 2077.556198) is defined?
Best regards,
Alexey Pasumansky,
Agisoft LLC

hati

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Back-projection of 3D points into images
« Reply #4 on: January 12, 2017, 08:03:07 PM »
This is a point that I have extracted from the .las file. So, I am not sure which coordinate system it belongs to.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Back-projection of 3D points into images
« Reply #5 on: January 12, 2017, 08:18:27 PM »
Hello hati,

LAS file may contain the information about the coordinate system used in the file header. You can try to open it in some GIS application.

Without knowing the coordinate system of this point it cannot be converted or projected to another coordinate system space.

Generally, if the point x is in the same coordinate system the chunk is referenced in, you can use the following Python code to get the coordinates of the points on 2D image:
Code: [Select]
import PhotoScan

chunk = PhotoScan.app.document
T = chunk.transform.matrix
crs = chunk.crs

camera = chunk.cameras[0] #first camera
point = PhotoScan.Vector([x, y, z]) #point of interest

a = T.inv().mulp(crs.unproject(point))
b = camera.transform.inv().mulp(a)

coords_2D = camera.sensor.calibration.project(b)
Best regards,
Alexey Pasumansky,
Agisoft LLC

hati

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Back-projection of 3D points into images
« Reply #6 on: January 12, 2017, 08:38:31 PM »
Alexey,

Thanks for the help. I could extract the following from the header of the las file:

X Scale Factor : 1.0E-4
Y Scale Factor : 1.0E-4
Z Scale Factor : 1.0E-4
X Offset : 522800.0
Y Offset : 4307700.0
Z Offset : 2000.0

Does this mean that I have to apply the scale and offset to my 3D point and then use the math for projection?

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Back-projection of 3D points into images
« Reply #7 on: January 12, 2017, 08:50:50 PM »
I think these values say about the internal representation of the points in LAS file and it's not connected to PhotoScan.

You've got the point (522878.864998, 4307805.3358, 2077.556198) somewhere close to the offset, so I think that from these values LAS subtract the mentioned Offset values and use the scale factor to keep the coordinates of the points. But yet again, without any information about the coordinate system we cannot move forward.

Looks like some UTM projection, according to the values.
Best regards,
Alexey Pasumansky,
Agisoft LLC

jjurado

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: Back-projection of 3D points into images
« Reply #8 on: April 09, 2018, 02:12:17 PM »
Hi,

I have been reading this post and my aim is the same: the projection of 3D point of cloud for each image. I have checked the code lines showed but I think that I must test before if the point of interest is projected by image. Is possible to make this testing?

Best regards,

JJurado

Emanuele1234

  • Newbie
  • *
  • Posts: 14
    • View Profile
Re: Back-projection of 3D points into images
« Reply #9 on: June 04, 2021, 05:05:57 PM »
Hi everyone,

when I obtain the coords2D, that in my case I insert in a list, these have type: Metashape.vector(coord2D). I would like to use these coord for make a polygon, but I don't understand how to extract a list of only int/float coord without this vector type. Do you have any Idea about solve this problem?

Thank you in advance for who will answer!

ps. this is an example of the list that I obtain:

[Vector([4487.467456499467, 4724.472068038938]), Vector([4491.722834552519, 4723.479952075152]), Vector([4496.532831576831, 4721.955053350271]), Vector([4484.4644832271515, 4724.819504895852]), Vector([4488.020088719773, 4723.997431237368]), Vector([4496.612417644212, 4722.017372068412]), Vector([4496.266248244279, 4722.790323937209]), Vector([4490.26616917476, 4721.596411398969]), Vector([4489.307185386602, 4729.175754631007]), Vector([4498.092177685087, 4726.549650712506]), Vector([4492.830413054364, 4728.494083066204]), Vector([4500.063634090015, 4727.286639288784]), Vector([4495.175886426696, 4723.516960399526]), Vector([4491.574518970151, 4724.528124771405]), Vector([4491.375263337253, 4725.067263233492]), Vector([4498.619344435478, 4723.281060767815]), Vector([4498.626978572264, 4723.941696509445]), Vector([4502.134831922718, 4723.0413444295245])]

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Back-projection of 3D points into images
« Reply #10 on: June 04, 2021, 06:27:09 PM »
Hello Emanuele1234,

you can do the following, assuming that your list is list_of_vectors:
Code: [Select]
list_of_tuples = [(v.x, v.y) for v in list_of_vectors]
Best regards,
Alexey Pasumansky,
Agisoft LLC

Emanuele1234

  • Newbie
  • *
  • Posts: 14
    • View Profile
Re: Back-projection of 3D points into images
« Reply #11 on: June 07, 2021, 10:18:54 AM »
Perfect, it's work!

Thank you very much! I didn't know that v.x and v.y could be vectors coordinates!