Multicamera setup rgb/thermal: alignement results in two levels of tie points

[BirgittaP]

I have tried to process rgb and thermal images from the DJI XT2 Zenmuse as a multicamera setup and encountered difficulties: The alignment results in two distinct levels of tie point clouds (at different altitudes). This offset results in incorrect DEM and orthophotos.
I have tried the following:
- manually delete one level of the tie points of the thermal images to retrieve a correct DEM for a correct rgb orthomosaic, but the corresponding thermal orthophoto looks blurred and distorted
- check the camera calibration option (master/slave), with fixed f, same outcome with the two levels
I attached screenshots of the model view and camera calibration dialogue and I can also provide sample images.
Does someone understand what causes this offset and how it could be handled?

[Paulo]

Hi Birgitta,

the GSD of thermal is so different from rgb that there are probably no tpts between 2 datasets. However what is the estimated master / slave offset in Camera Calibration dialog?

Please share a few images from both to see what could be going wrong?

[BirgittaP]

Hi Paulo,
Thanks for your thoughts on this. Indeed, the GSD is very different, the thermal images contain only a very small area of what can be seen in the respective rgb images.
I uploaded the slave offset parameters from the camera calibration dialogue. I received:
Omega    -1.41822    0.00272987
Phi            -0.196527   0.0027611
Kappa   3.70276      0.000784025
Again, thanks for looking into this.

[Paulo]

Hi BirgittaP,

what you can do is select an image and context menu select Show matches and you will be able to see if there are any matches between rgb and thermal images...probably not.

So in this case to reliably determine the offset between master and slave is difficult...

Can you share a small sample of rgb and thermal thru PM and I can have a look at it..

[BirgittaP]

Hi Paulo,
Yes, I checked the matches and between thermal and rgb it is usually not more than 10 matches, whereas between rgbs there are around 1000 matches per pair. Sure, I can share a few sample images.

[Paulo]

Hello Birgitta,

it seems definitely that there is some sort of zoom problem or crop problem in either RGB or thermal camera. As when I process both as a multi camera rig then Z offset between master and slave is huge (78 m) while if I process each camera separately I see same shift as the thermal tie points are way above the rgb.... see attached screen shot

Do you somehow used digital zoom on either camera?

[BirgittaP]

Hi,
your comment on the zoom made me check mission settings and question everything else again. There was no zoom set in the mission. BUT: I checked the specs of the XT2 again and it happened that I had used an incorrect camera calibration xml file for the thermal camera! With the correct file, there is no offset any more. The resulting DEM and Ortho look fine as well. Thank you so much for your support!

[Paulo]

Hello Birgitta,

Good to hear you solved the issue. However the f value of 1117 pixels would mean the focal in mm to be:

Code: [Select]

1117 pix  x 0.26 mm/pix = 290 mm
which would suggest a heavy zoom from 19 mm!

Unless the pixel pitch of 0.26 mm is erroneous...

[BirgittaP]

Hello Paulo,
Yes, you are right. Interestingly, 0.26 mm is not the pixel pitch I entered for the new calibration file, but 0.017 mm. I found that 0.26 mm is the assumed value. If I enter the correct 0.017, f changes and can be saved in .xml file. If I then, at later, I import this .xml, only f is still correct (1117.647), but the pixel pitch is again the "default". This behavior is confusing, but seems to have no consequence if f in the .xml is just right, i.e. calculated from the correct pixel pitch before.

[Paulo]

Hello Birgitta,

yes pixel size of 0.017 mm seems correct as this gives:

Code: [Select]

image width = 0.017 * 336 = 5.71 mm which corresponds to small size sensor
The problem of 0.26 mm pixel pitch comes from exif metadata:

Code: [Select]

---- ExifIFD ----
FNumber                         : 1.3
DateTimeOriginal                : 2022:06:18 12:57:28
FocalLength                     : 19.0 mm
ImageNumber                     : 0
SubSecTimeOriginal              : 37
FocalPlaneXResolution           : 3.84
FocalPlaneYResolution           : 3.072
where FocalPlaneX and Y Resolution seem wrong.

[BirgittaP]

Alright, that makes sense. I see this issue solved now. Thanks again!