Forum

Author Topic: Preprocessing RGB pictures  (Read 3618 times)

b_sky

  • Newbie
  • *
  • Posts: 2
    • View Profile
Preprocessing RGB pictures
« on: July 15, 2020, 02:49:16 PM »
Hi,

we are flying missions with dji phantom 4 pro v2 for the use of orthos and elevation rasters. Sometimes i have ortho colorization problems when cloud coverage is changing during the fly and besides that i try to improve my workflow for the structure from motion method.
I thought about picture preprocessing by capturing them in raw and the usage of color panels for radiometric calibration (taking pictures from panel with different cloud coverage and adjust white balance of mission pictures before going into metashape).
I already searched for the uav photogrammetry usage of color panels but they are being used mainly for multispectral uav missions (e.g. 2*) or close range photogrammetry (1*) and raw capturing isnt supported by many mission planning apps and not being used by the majority of workflows (e.g. usgs workflows 3*).
So my questions:
 
1. Does working with raw pictures and/or color panels make any sense for rgb pictures or is it not worth the effort (raw memory size)?
   (useing calibrated colors eg for analysis combining uav and satellite images should be usefull?!)

2. Why is it being used with multispectral cameras but not to calibrate the rgb bands?

3. Could preprocessing pictures also effect model accuracy by better finding matching points?

4. Any other benefits you maybe experienced by working with raw / color panels?


Thanks in advance!

Ben


*1
https://unity3d.com/files/solutions/photogrammetry/Unity-Photogrammetry-Workflow_2017-07_v2.pdf

*2
https://www.researchgate.net/publication/318325581_Study_of_radiometric_variations_in_Unmanned_Aerial_Vehicle_remote_sensing_imagery_for_vegetation_mapping

*3
https://uas.usgs.gov/nupo/otherresources.shtml

nprokofyev

  • Newbie
  • *
  • Posts: 22
    • View Profile
Re: Preprocessing RGB pictures
« Reply #1 on: July 15, 2020, 05:43:18 PM »
I used to use RAW images with IR converted cameras. This is the only way you can extract something useful from them.

And if there were clouds, neither RAW nor color panels would improve the situation because ambient light spectrum is completely different on a direct sunlight and under the clouds. To eliminate clouds, you need to use downwelling light sensor and correct every picture with its own parameters. That is how it works on multispecral cameras.

1. I am not sure about P4, but generally, working with raw pictures allows you to achieve better dynamics range, gives you additional ~ 1 EV to correct exposure and a chance to apply any white balance you want. But IRL JPG quality isn't as dramatically bad as JPG processing is easier.
Gray panels are quite good for setting manual WB before flight, especially with JPG.

2. Multispectral data have to be calibrated, because it's main derivative is reflectance values. Think of it not as of photography, but as of spectrometry.

3. Yes, you can make it worse if you don't apply chromatic abberation correction in RAW mode xD  Seriously, you may achive slightly better sharpness by the price of increased noise. Doubtful.

4. RAW allows you to convert it to 16-bit TIFF and then perform raster calculations with significantly lower statistical error. Do you use raster calculations on RGB data?
« Last Edit: July 15, 2020, 05:47:48 PM by nprokofyev »

b_sky

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: Preprocessing RGB pictures
« Reply #2 on: July 16, 2020, 12:38:08 PM »
Thanks for your response!

1. Ok i guess i got it. So color panels are for multispectral data because the data acquired is affected by sensor characteristics, illumination conditions, ... and thats why the DN values has to be converted to be representative of the surface reflectance right?

2. So light sensors are used to get the radiation intensity that comes through clouds and others to adjust colors from different lightning conditions. I just know them mounted on drones but would it be possible to get a separate one like JYP-1000 (PAR) sensor, measure from different light conditions in the field and adjust pictures afterwards?

3. Shouldnt it also be possible to get the difference in light condition with a reference white from color panel? i mean it should give me the different RGB ratio (white balance) and the intensity with absolute RGB values or am i confuse something?

4. I guess to use color panels just for white balance doesnt make so much sense because the afford is to big and the auto whitebalnce of the camera good enough?

Yep we are doing some simple classifications (broadleaf and coniferous) and training a CNN to use it on satellite level. I would like to have better calibrated colorization ... even if we plan to flight on slightly cloudy days - light conditions are changing often :/

 
Best regards,
Ben

nprokofyev

  • Newbie
  • *
  • Posts: 22
    • View Profile
Re: Preprocessing RGB pictures
« Reply #3 on: July 17, 2020, 08:04:15 PM »
1. Basically, yes, reflectance panels are for multispectral data because you need to convert DN to reflectrance.
2. There are two sets of parameters to determine: spectral and directional. Real life reflection is described by a bi-directional function ( https://en.wikipedia.org/wiki/Bidirectional_reflectance_distribution_function ) which means that parameter set depends on aircraft direction. Theoretically, you can measure all the values and calculate the corrections, but it is a hard task.

3. Well, you may use white/gray panel  to determine light conditions at the moment the picture was shot. And this may be applied for pictures with same light conditions. But, once again, due to directional component, IRL you have more than two parameter sets (not only "sunny" and "cloudy").

4. Actually, you don't need color panels to set WB, you need only white/gray card. And in my opinion, some cameras may benefit from custom WB in JPG mode, even expensive ones. Auto-WB works really simple: camera averages whole image and adjust multipliers so that average tone is neutral gray. Sounds coarse, isn't it?

I'd suggest you to try several flights with fixed WB (not necessarily RAW). It may improve overall color situation under constant light conditions. 

dpitman

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Preprocessing RGB pictures
« Reply #4 on: July 18, 2020, 02:28:35 AM »
"I'd suggest you to try several flights with fixed WB (not necessarily RAW). It may improve overall color situation under constant light conditions.  "

+1

I use a P4P to capture for topography and not vegetation. But I always use a fixed WB, generally Sunny, even when clouds are passing through during capture.  Having a uniform balance between all images seems to lead to better results.