Forum

Author Topic: combining infrared and rgb camera  (Read 9255 times)

ferdywallinx

  • Newbie
  • *
  • Posts: 28
    • View Profile
combining infrared and rgb camera
« on: December 29, 2015, 06:48:58 PM »
Hi All,

Our company is flying with a 36MP RGB camera and a similar NIR converted camera (infracutfilter removed, RGB-filter added). We use these camera's for for aerial photography but the images are not well aligned, because of large distance and inperfect alignment of cameras on mount.

Because of this I have two datasets, one RGB and one NIR. The images have identical gps positions because they are triggered at the same time and receive GPS data at the same moment. Does any one of you have the same setup and what did you do to align the images? I want a well overlapping orthomosaic but normal alignment does not work since the correlation between infrared and RGB is too low.

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: combining infrared and rgb camera
« Reply #1 on: December 30, 2015, 12:13:16 AM »
Hi Fredy

Can send me 10-30 img ( RGB + the IR  ) for few tests ?
if their are synced it should work

send it to muzeumhb@gmail.com
----------------
www.mhb.sk

ben73

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: combining infrared and rgb camera
« Reply #2 on: May 14, 2017, 02:16:25 AM »
Hello
I know its an old thread, but did you found an answer.

FlyRight-3D

  • Newbie
  • *
  • Posts: 16
    • View Profile
Re: combining infrared and rgb camera
« Reply #3 on: May 14, 2017, 12:45:32 PM »
Did you process in two separate chunks, then merge the layers? 

jessica_tay

  • Newbie
  • *
  • Posts: 24
    • View Profile
Re: combining infrared and rgb camera
« Reply #4 on: May 22, 2017, 06:24:51 PM »
great, i am also interested in this topic.

I am also trying to combine my RGB chunk with my NIR chunk.

I've so far build the spare cloud for both chunk separately and then I merge the chunk according to the points.. it is still processing and I'll update if it works.

I hope to hear from you guys who are trying as well.

Thanks!

2bForgotten

  • Newbie
  • *
  • Posts: 17
    • View Profile
Re: combining infrared and rgb camera
« Reply #5 on: June 19, 2019, 04:24:18 PM »
great, i am also interested in this topic.

I am also trying to combine my RGB chunk with my NIR chunk.

I've so far build the spare cloud for both chunk separately and then I merge the chunk according to the points.. it is still processing and I'll update if it works.

I hope to hear from you guys who are trying as well.

Thanks!

Hi there,
Would like to know about your experience so far. I suppose the processing has finished yet. ;)
Regards,

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: combining infrared and rgb camera
« Reply #6 on: June 19, 2019, 04:46:24 PM »
Hello 2bForgotten,

Since the time the topic has been started in Metashape Pro the multi-camera system approach has been implemented, that allows to process the data captured from multiple sensors that are fixed relative to each other. It could be both RGB cameras looking in different directions and different range sensors (RGB + NIR, for example).

The main requirements for such approach are: properly synchronized sensors and absolutely fixed relative orientation of the sensors in such multi-camera system rig.
Best regards,
Alexey Pasumansky,
Agisoft LLC

2bForgotten

  • Newbie
  • *
  • Posts: 17
    • View Profile
Re: combining infrared and rgb camera
« Reply #7 on: June 25, 2019, 12:08:40 PM »
Hello 2bForgotten,

Since the time the topic has been started in Metashape Pro the multi-camera system approach has been implemented, that allows to process the data captured from multiple sensors that are fixed relative to each other. It could be both RGB cameras looking in different directions and different range sensors (RGB + NIR, for example).

The main requirements for such approach are: properly synchronized sensors and absolutely fixed relative orientation of the sensors in such multi-camera system rig.

Hi Alexey,

Could you please tell me if when using IR camera, the radiometric info will still be available in the ortho when created with Metashape?

Regards,

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: combining infrared and rgb camera
« Reply #8 on: June 28, 2019, 04:59:59 PM »
Hello 2bForgotten,

Each band of the input source data is blended individually, so if you have one of the band related to the IR sensor - the resulting orthomosaic would contain this information. For example, if the source data contain the values that can be transformed to temperature values using any formula, you would be able to apply the same transformation to the resulting orthomosaic.
Best regards,
Alexey Pasumansky,
Agisoft LLC

2bForgotten

  • Newbie
  • *
  • Posts: 17
    • View Profile
Re: combining infrared and rgb camera
« Reply #9 on: June 28, 2019, 08:19:55 PM »
Hi Alexey,

Thank you for your answers.

One final question though about the combined RGB and IR camera.
Given that they will fire at the same time in the exact same direction. There still would be a different field of view for the RGB compared to the IR.
Would Metashape be able to handle that? Or would one have to create two different workflows for both image sets?

I read around on the internet, but haven't found a conclusive answer to that question,

Regards,

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: combining infrared and rgb camera
« Reply #10 on: June 28, 2019, 08:36:02 PM »
Hello 2bForgotten,

When you have separate synchronized sensors it is suggested to use Multi-camera system approach. In you case:
1. Create a new directory and then two subdirictories in it. Call them "1-RGB" and "2-IR", then put the images to each folder coming from the certain sensor - the order of the images in the folder should correspond to each other.
2. In Metashape interface use Workflow Menu -> Add Folder option and point to the "master" directory created in the beginning of the previous step. In the Add Photos dialog shown shortly after the folder contents are analyzed, select Multi-Camera System approach - > "Create sensor from each subfolder" option.
3. After loading images go to Tools Menu -> Camera Calibration dialog and, if necessary, enable "Reference" option for the slave sensor. In case the offset between the sensors cannot be neglected, then enable also "Adjust location" option. Additionally make sure that Layer Index in the Bands tab is different for Master and Slave sensor.
Approach would work if the sensors are synchronized and their relative exterior orientation doesn't change during the capturing process.
Best regards,
Alexey Pasumansky,
Agisoft LLC

Eckemon

  • Newbie
  • *
  • Posts: 5
    • View Profile
Re: combining infrared and rgb camera
« Reply #11 on: September 13, 2019, 12:32:38 PM »
problem solved :)
« Last Edit: September 16, 2019, 12:34:20 PM by Eckemon »

MichalDV

  • Newbie
  • *
  • Posts: 27
    • View Profile
Re: combining infrared and rgb camera
« Reply #12 on: May 18, 2022, 06:04:29 PM »
Hi, unfortunately this approach is not working with Duet-T camera from Sensefly.

Hello 2bForgotten,

When you have separate synchronized sensors it is suggested to use Multi-camera system approach. In you case:
1. Create a new directory and then two subdirictories in it. Call them "1-RGB" and "2-IR", then put the images to each folder coming from the certain sensor - the order of the images in the folder should correspond to each other.
2. In Metashape interface use Workflow Menu -> Add Folder option and point to the "master" directory created in the beginning of the previous step. In the Add Photos dialog shown shortly after the folder contents are analyzed, select Multi-Camera System approach - > "Create sensor from each subfolder" option.
3. After loading images go to Tools Menu -> Camera Calibration dialog and, if necessary, enable "Reference" option for the slave sensor. In case the offset between the sensors cannot be neglected, then enable also "Adjust location" option. Additionally make sure that Layer Index in the Bands tab is different for Master and Slave sensor.
Approach would work if the sensors are synchronized and their relative exterior orientation doesn't change during the capturing process.


Paulo

  • Hero Member
  • *****
  • Posts: 1303
    • View Profile
Re: combining infrared and rgb camera
« Reply #13 on: May 18, 2022, 10:22:33 PM »
Hey Michal,

please refer to following for more info on how to process DuetT data https://www.agisoft.com/forum/index.php?topic=13058.0.

Quote
another option to be able to process the RGB Soda imagery with the thermal absolute temperature band in a Multi Camera setup would be the following:
- export the 2nd plane of thermal Rjpeg as 16 bit single band tif imagery;
- combine the soda RGB and 16 bit imagery exported above in multi camera layout with Soda RGB in plane 1 Master and 16 bit thermal imagery in plane 2 Slave
- continue with align photos, generate depth maps, DEM, Ortho with Raster Transform formula B4*0.01 - 273.15 for thermal display of results
For this process, a script could be developed, resulting in something similar to following screen...

You can always send me a PM for further discussions...
« Last Edit: May 19, 2022, 05:23:16 AM by Paulo »
Best Regards,
Paul Pelletier,
Surveyor