Forum

Author Topic: Creating virtual cameras  (Read 2764 times)

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Creating virtual cameras
« on: February 15, 2022, 06:59:02 PM »
Hi there,
I'm trying to create some virtual cameras at locations specified by point coordinates over my mesh, in order to use these cameras to export images of the mesh.

To create these cameras I'm trying this:

Code: [Select]
i = 0
for point in points:
    tmp_sensor = chunk.addSensor()
    tmp_sensor.label = str(i)
    tmp_sensor.location = Metashape.Vector(point)
    tmp_sensor.location_enabled = True
    tmp_sensor.rotation = Metashape.Vector([0,0,0])
    tmp_sensor.rotation_enabled = True
    chunk.addCamera(tmp_sensor)
    i+-1

However it seems I cannot assign a location to a sensor in this way:

Code: [Select]
2022-02-15 16:56:26 Error: 'Metashape.Sensor' object attribute 'location' is read-only
Does anyone know what the correct way to create such virtual sensors might be?

Thanks!

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14839
    • View Profile
Re: Creating virtual cameras
« Reply #1 on: February 15, 2022, 09:28:33 PM »
Hello stephan,

I would suggest to consider using chunk.model.renderImage() method, if you just need to get an image of the mesh from the desired locations using the user-defined calibration model parameters.
Best regards,
Alexey Pasumansky,
Agisoft LLC

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #2 on: February 16, 2022, 04:51:50 PM »
Thanks Alexey, I will give that a try!

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #3 on: February 17, 2022, 04:52:39 PM »
Hi Alexey,

I've been working with chunk.model.renderImage() but I have a question: what is the format of the camera matrix that is expected for the first argument?
I mean: I can use the transform matrix of an existing camera, and this works, but I am interested in understanding the format of this matrix in order to be able to create my own parameters procedurally. Is there any documentation on this somewhere?

Cheers,
Stephan

Paulo

  • Hero Member
  • *****
  • Posts: 1318
    • View Profile
Re: Creating virtual cameras
« Reply #4 on: February 17, 2022, 09:44:14 PM »
Hello stephan,

the transform matrix can be constructed in following manner, starting from a given position vector (X, Y, Z) and orientation vector (Yaw, Pitch, Roll):
Code: [Select]
#define point of view (position, orientation)
import Metashape as ps
position = ps.Vector((6.566944698204851,46.59323523897216,735.2981091487283)) # position vector X, Y, Z in chunk.crs 'WGS 84 + EGM96 height (EPSG::9707)'
orientation = ps.Vector((7.997216319444453,55.883424713290765,-4.79245619778298)) # orientation vector Yaw, Pitch, Roll in chunk.crs

#calculate relevant transform matrix t
chunk = ps.app.document.chunk
position = chunk.crs.unproject(position)  # position in ECEF
orientation = chunk.crs.geogcs.localframe(position).rotation().t() * ps.Utils.ypr2mat(orientation) # orientation matrix in ECEF
transform = ps.Matrix.Translation(position) * ps.Matrix.Rotation(orientation)
transform = chunk.transform.matrix.inv() * transform * ps.Matrix.Diag((1, -1, -1, 1))
t = ps.Matrix.Translation(transform.translation()) * ps.Matrix.Rotation(transform.rotation()) # 4x4 transform matrix 3 translations and 3 rotations

Hope this is helpful,
« Last Edit: February 18, 2022, 04:24:02 AM by Paulo »
Best Regards,
Paul Pelletier,
Surveyor

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #5 on: February 18, 2022, 03:25:49 PM »
Hi Paul,
That's very helpful, thanks!


stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #6 on: February 18, 2022, 05:03:46 PM »
I'm still confused and I can't get this to work... I think there is something I fundamentally and getting wrong with the way Photoscan creates rotation angles...

Code: [Select]
orientation = ps.Vector((7.997216319444453,55.883424713290765,-4.79245619778298)) # orientation vector Yaw, Pitch, Roll in chunk.crs

In the line of code above, am I right in assuming that the values should be in degrees? so (0,0,-90) should be straight down, correct? Or am I getting this confused?


Paulo

  • Hero Member
  • *****
  • Posts: 1318
    • View Profile
Re: Creating virtual cameras
« Reply #7 on: February 18, 2022, 08:53:49 PM »
Hi stephan,

the yaw, pitch, roll angles are defined as follows (from User Manual p.87):
Quote
Rotation angles for the camera coordinates in Metashape are defined around the following axes: yaw axis
runs from top to bottom, pitch axis runs from left to right wing of the drone, roll axis runs from tail to nose
of the drone.
So camera looking straight down with drone flying in north direction corresponds to (0,0,0) and not (0,0,-90) which has a roll of -90 degrees i.e. camera turned right looking to horizon.

For example, in screen copy, the second camera (selected) has ypr = (0, 0, -90) . It is looking to horizon to the east... first camera has ypr = (0, 0, 0) and is looking straight down flying north....

Note: in DJI angle convention (0, -90, 0) (-90 deg pitch) corresponds to (0, 0, 0) (0 deg pitch) in Agisoft angle convention,
i.e. pitchDJI = pitchAgisoft - 90
« Last Edit: February 19, 2022, 06:41:21 AM by Paulo »
Best Regards,
Paul Pelletier,
Surveyor

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #8 on: February 19, 2022, 07:26:26 PM »
Thanks, that's very helplful Paulo!

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #9 on: February 19, 2022, 08:52:01 PM »
I've made a lot of progress in understanding the coordinate system.
I'm still getting stuck on camera rotation / position however. I have no problem defining the locations that I want inside my chunk coordinate system (verified by putting markers in the locations)...

I stumbled upon the code from this thread for rendering an image:
https://www.agisoft.com/forum/index.php?topic=12545.msg55685#msg55685


This is the extract:
Code: [Select]
import Metashape

chunk = Metashape.app.document.chunk
T = chunk.crs.localframe(chunk.transform.translation) * chunk.transform.matrix

location = chunk.region.center + chunk.region.rot * chunk.region.size
direction = (chunk.region.center - location).normalized()
vertical = T.inv().mulv(Metashape.Vector([0, 0, 1]))

horizontal = Metashape.Vector.cross(direction, vertical).normalized()
vertical = Metashape.Vector.cross(direction, horizontal).normalized()
R = Metashape.Matrix([horizontal, vertical, direction])

cameraT = Metashape.Matrix().Translation(location) * Metashape.Matrix().Rotation(R.t())

image = chunk.model.renderImage(cameraT, chunk.sensors[0].calibration)
image.save("c://Work//render2.jpg")


The part I don't fully understand is this:

Code: [Select]
horizontal = Metashape.Vector.cross(direction, vertical).normalized()
vertical = Metashape.Vector.cross(direction, horizontal).normalized()
R = Metashape.Matrix([horizontal, vertical, direction])

What is going on here?
From what I understand we are calculating a component vector that corresponds to the components of the orientation in the chunk's internal reference frame? But I don't get how these lines work individually...



Paulo

  • Hero Member
  • *****
  • Posts: 1318
    • View Profile
Re: Creating virtual cameras
« Reply #10 on: February 20, 2022, 02:09:39 AM »
Hello,

this script creates a model render Image from a ViewPoint (location) displaced from region center by  the region diagonal length (region.size.norm()) and  in direction from region top left corner to center as in first attachment.

The resulting rendered image is shown in 2nd attachement

Third attachment shows viewpoint with its CS axes (hor, ver are // to render image plane, dir points in viewpoint direction towards RegionCenter). Axes Xint, Yint, Zint represent internal CS with its Origin. In this case Viewpoint is displaced from center only by 0.75 of region diagonal length.
« Last Edit: February 22, 2022, 09:18:08 AM by Paulo »
Best Regards,
Paul Pelletier,
Surveyor

stephan

  • Full Member
  • ***
  • Posts: 129
    • View Profile
Re: Creating virtual cameras
« Reply #11 on: February 23, 2022, 03:19:03 PM »
Awesome thanks!
I the end I built my own system to do this but your code helped me understand a lot better!