Errors of calibrated parameters

[Yoann Courtois]

Hi Karad !

Thanks again for all your papers you linked, really helpful !

But, i'm still a bit puzzled...
On my last UAV survey, I've tried, using all the same parameters / filters, to optimize by two different methods:
- First one, keeping all the distortions enabled (Picture 1)
- Second one, removing K3-K4 and P3-P4 as their correlations were high. (Picture 2)

The results are:
- Global reprojection error become higher on the second workflow (0.712 pix) compare to the first one (0.649 pix)
- Visual image residuals is worst on the second workflow
- Highest errors on GCPs are bigger on the second workflow

It looks that using all the available distortions allow to fit and adjust better, even they present high correlations.

I would lik to have your point of view on that !

Regards

[agrest]

I did two more test:
First dataset: http://www.agisoft.com/datasets/monument.zip


Second dataset: https://s3.amazonaws.com/mics.pix4d.com/example_datasets/example_building.zip

I used: High accuracy, No generic and reference preselection, No mask, No adaptive model fitting, 40000 key point limit and 4000 tie point limit, No gradual selection after alignment.
It seems better to consider the parameters selected by Photoscan than manually make parameter uncorrelated.

[Yoann Courtois]

Hi agrest,

What is the difference between your two tests ?

Some remarks about your tests:
- By not using "adaptive model fitting", you don't adjust your calibration onto your dataset of the day (No idea what is your camera, but calibration is rarely totally fixed)
- By not using gradual selection, you still use some wrong or imprecise tie points in your adjustment.
- By the way, did you adjust/optimize on some know GCPs ? Where do your RMS errors (0.54m/0.53m/0.62m) come from ?
We are actually talking about adjusting photogrammetric model onto centimetric-accuracy GCPs, and the best workflow to fit it as well as possible without over-estimate the camera distortions.

Regards

[Mina]

Hi Mina !

Did you fix your focus during all your pictures ? By each set of pictures ?
Did you use autofocus ?

It looks like only your focal length has changed between your two calibrations...should search about that !

Regards

Hi Yoann,

I used the autofocus but tried to find a larger depth of the field using the smaller aperture. Since my camera has the option of focusing automatically or manually, how can I fix the focus, which probably means that the whole scene is automatically on focus?

Best,
Mina

[Yoann Courtois]

Hi Mina,

That's what I was thinking about. Considering you used autofocus, you really CANNOT:
- Compare your two tests
- Feel really confident on one calibration or the other.

Indeed, changing your focus (which happened at each picture with autofocus, even it's just a little) means changing your focal length...
So, you can actually consider that the focal length written in your calibration reports are the mean focal length of each set of picture.

A better workflow would be:
- Take the first picture with autofocus to set it in order to have clear pictures.
- Then block the focus by switching to manual focus but also by using tape on your camera and so one. NOTHING has to change !
- Take your whole set (or sets) of pictures and take care to keep the average same distance to the object than you did at the first picture in autofocus.
- As you already did before, keep the aperture as small as possible to have a large depth of field.

Be careful not to change the focus between two sets of pictures you want to compare the calibration !

Regards

[karad]

Hi Courtois!

Very interesting results!
Which is your GCP configuration and check points position?
Because 8.8 mm FL is too short, eg super wide angle, you have to use another lens distortion model as suggested by photoscan?

Regards

[agrest]

Hi agrest,

What is the difference between your two tests ?

Some remarks about your tests:
- By not using "adaptive model fitting", you don't adjust your calibration onto your dataset of the day (No idea what is your camera, but calibration is rarely totally fixed)
- By not using gradual selection, you still use some wrong or imprecise tie points in your adjustment.
- By the way, did you adjust/optimize on some know GCPs ? Where do your RMS errors (0.54m/0.53m/0.62m) come from ?
We are actually talking about adjusting photogrammetric model onto centimetric-accuracy GCPs, and the best workflow to fit it as well as possible without over-estimate the camera distortions.

Regards

Two different cameras (Canon EOS 5D and Sony NEX-5) and scenes (ground/aerial photos). It was a simple and quick comparative test without reducing errors.

1. I don't think so. Photoscan always adjust calibration onto your dataset if you didn't choose Fix calibration. Adaptive model fitting from manual is "automatic selection of camera parameters to be included into adjustment based on their reliability estimates". Without it you get "refine only the fixed set of parameters: focal length, principal point position, three radial distortion coefficients and two tangential distortion coefficients." So the most, 8 parameters have been adjusted.
2. I compared only the effect of parameter correlation on general error without removing wrong tie points.
3. I didn't use GCP and didn't optimized based on them (they were not delivered to projects that came from the internet). However, if you use GCPs, they will probably make the camera parameters uncorrelated. So it will not be useful to test.
RMSEs from second test came from geotagged photos.

[Mina]

Hi Mina,

That's what I was thinking about. Considering you used autofocus, you really CANNOT:
- Compare your two tests
- Feel really confident on one calibration or the other.

Indeed, changing your focus (which happened at each picture with autofocus, even it's just a little) means changing your focal length...
So, you can actually consider that the focal length written in your calibration reports are the mean focal length of each set of picture.

A better workflow would be:
- Take the first picture with autofocus to set it in order to have clear pictures.
- Then block the focus by switching to manual focus but also by using tape on your camera and so one. NOTHING has to change !
- Take your whole set (or sets) of pictures and take care to keep the average same distance to the object than you did at the first picture in autofocus.
- As you already did before, keep the aperture as small as possible to have a large depth of field.

Be careful not to change the focus between two sets of pictures you want to compare the calibration !

Regards

Hi Yoann,

Thanks for your reply!

I know that we need to try to capture all images on focus. Sometimes it is impossible to keep the same distance to the object in the whole sets of the pictures. I think if the photo is on focus, it does not matter from which distance you took the photo!
Regarding the workflow, we need to take the first photo and try to take as sharp as possible and then switch to the manual focus and I did not get the rest! Using the tape!!
Could you please explain it a bit more clear.

Best,
Mina

[Yoann Courtois]

Hi all !

Karad:
You can find attached the GCPs distribution around the area and the RMS errors on them. Only one point was disabled (set as check point) as it wasn't a referenced target.
Indeed 8.8mm is really short (DJI Phantom 4 Pro) but it's still a frame format. So I used frame type of camera. I use fisheye for example when dealing with GoPro pictures. Actually The P4Pro Camera is a kind of "mongrel" format that's why image residuals are quite big (short scale), but we can aim enough high precision for our use.

Agrest:
1. My bad, actually I've never noticed big difference between checking or unchecking that... How do you interpret what happened when it's check ?
2. Higher errors (caused by wrong tie points) may make more complex the distortion model. I would say it might be possible to have less correlation if you don't filter your tie point cloud ^^ Using filter will delete wrong points and so you will get the almost real distortion model, which may don't need K2-K3-K4.
3. They is no correlation (sic.) between "using GCPs" and "distortion parameters correlation". Indeed I always use GCPs to accurately set the model. Without any GCPs, your model only depend on tie points and and calibration.

Mina:
That's the deal with photogrammetry, one measurement one focus, either you may group your pictures by focus (in calibration panel, two focal length => two cameras). If you cannot keep the same distance, you have to adapt your survey:
- Be further to have bigger depth of field
- Take a shorter focal length
- Close ever more the aperture
I said you should use tape because on most of camera, there is a ring for manual focusing.
So firstly set the focus correctly by using autofocus
Then switch to manual focus
And use tape to block the ring so that you cannot manually change the focus Hope to be clearer !


Regards

[karad]

Hi Courtois!
I think the links below are very helpful :

http://www.sciencedirect.com/science/article/pii/S0169555X16311291
http://www.itc.nl/library/papers_2016/msc/gfm/nasrullah.pdf

regards

[Yoann Courtois]

Hi Karad !

Thanks a lot for those papers.
Unfortunately I'm almost at the end of my thesis time so I guess I won't have time to really focus of reading those.
Hope to continue of that field later and have a deep look on !

Regards

[karad]