Georeferencing using GCPs: Optimize -vs- Update tool

[BobvdMeij]

Dear all,

In a quest to optimize our workflow I have been 'playing' around lately with the Camera Optimization tool to evaluate how fitting different parameters (or not) influence our model and associated accuracies. This was partially motivated by one of my colleagues asking me what type of transformation Agisoft applied when optimizing the model based on markers/ GCPs, and I found myself not knowing the answer to that question.

This caused me diving into the User Manual and Forums a bit more and fiddling around with a multitude of settings in the software. This also led me to this forum post, wherein it is stated that the Update Tool makes use of an Affine (or linear) Transformation (translation, scale, rotation) for updating the model based on Markers. Strikingly enough, I had never really located, understood or used the Update Tool before. Instead, I always 'optimized' my models based on markers using the Optimize Cameras Tool. Under Optimization of camera alignment in the user manual the following is stated:

During georeferencing the model is linearly transformed using 7 parameter similarity transformation (3
parameters for translation, 3 for rotation and 1 for scaling). Such transformation can compensate only a
linear model misalignment. The non-linear component can not be removed with this approach. This is
usually the main reason for georeferencing errors.

Possible non-linear deformations of the model can be removed by optimizing the estimated point cloud
and camera parameters based on the known reference coordinates. During this optimization PhotoScan
adjusts estimated point coordinates and camera parameters minimizing the sum of reprojection error and
reference coordinate misalignment error.

I am not entirely sure whether the former paragraph refers to either the Alignment step, the Update Tool and/ or any other step (any elaboration thereon is appreciated!), but I find the latter paragraph particularly interesting. Although this is not explicitely mentioned it seems to relate to the Camera Optimization Tool. It furthermore suggests that this tool applies a non-linear transformation, hereby making it a sincerely different tool from the Update Tool.

I subsequently setup a testing dataset. Upon importing all (257) images, running Alignment and marking a total of 17 Ground Control Points and 13 Check Points I duplicated the chunk to ensure the base dataset for all subsequent trials was identical. I then applied the Update Tool (once) and a variety of Camera Optimization Tools (fitting different parameters each time) for each chunk seperately.

As expected the Update Tool hardly altered my model. Apart from changing the initial TiePoint coordinates from WGS84 (from the camera's EXIF data) to the GCP's coordinate system nothing striking happened. Relative camera positions seemingly remained unchanged, as did the relative distribution of TiePoints and the overall height gradient across my model. The implications of applying different Optimization Camera Tool runs were noticably more vast. Not only the relative position and orientation of each camera is slightly altered, the Sparse Cloud demonstrates similarly varied transformations across the various iterations. Although it is very hard to pinpoint this with absolute certainty based on a visual inspection of the SparseCloud/Camera positions alone, it indeed seems that the model is transformed beyond rather simple scaling/rotation/translation axis.

The total CheckPoint error does also indicate that Camera Optimization allows for a more complex/ non-linear fitting of the TiePoint model to the Markers. Whereas the Update Tool produced a massive Total CheckPoint error of 21.1cm, the CheckPoint error varied between 1.3cm and 3.5cm across the different Optimize Cameras iterations. On the contrary, the RMS reprojection error was relatively similar for both the Update and Camera Optimization tool (0.67 pix).

Although the Total Error quiet clearly speaks in favor of applying Camera Optimization over the Update Tool, I am still not convinced. From a traditional photogrammetry perspective I am admittingly somewhat anxious, not to say reluctant, to apply non-linear transformations. Not least because this may invoke model overfitting on the one hand while producing unrealistic model accuracies that may differ across the model as GCP density varies. More specifically, although the model accuracy measured very locally at the position of the CheckPoints suggests very high model accuracy, I am not so sure to straightforwardly believe this is also the case for locations further away from these CheckPoints.

On the other hand, aerial photogrammetry based on nadir oriented imagery alone tends to produce the so-called 'doming-effect'. Or, put differently, models that demonstrate significant bulging or depressing towards the center of the model. This affect can be easily gotten rid of by application of GCPs and then fitting the model towards these GCPs by using Camera Optimization. Obviously, this notorious error is preserved when one only fits the model to the GCPs in a linear/ affine fashion by using the Update Tool.

In short then, I'M LOST. What is the right way to go? What does each of these tools precisely do? What transformations are applied? Which tool should one use in which instance?

[Jeremiah_ROWE]

I too have used the "Optimize Cameras" pretty much exclusively. I would like to know in which cases we would use that vs. the "Update" button.

[James]

It seems like you pretty much have the answer, and know about as much as i do, yet i don't feel confused by it so will try and explain it my way, as concisely as i can!

The update tool is like an 'apply' button which affects the world coordinate system any time you modify your reference data, i.e. gcp or camera coordinates. It doesn't affect the positions of cameras relative to each other, so any dense cloud or mesh is preserved in the new coordinate system that you have defined/modified by the addition/edition of coordinate data.

The optimize tool also takes account of your reference data, and any changes to it, but it also looks at changes to tie points (i.e. from edits to the sparse cloud including gradual selection/deletion) and also treats markers (with or without coordinates) as valid tie points and uses this info to re-estimate the position of cameras relative to each other, and within the coordinate system, and their interior orientation parameters (depending on which you enable when prompted). If you modify your camera calibration groups then you will also see different results after using optimize. Any changes resulting from this optimization mean that the dense cloud and mesh etc are now obsolete and need to be regenerated from the newly optimized sparse cloud. Edits to the sparse cloud which reduce the number of tie points in any image below a certain threshold (i'm not sure what it is) mean that it's position can no longer be estimated and so it will be reset (no longer aligned).

Just hitting optimize alone will have very little effect (some small effect due to stochastic nature of the algorithms, presumably?) unless:

• You edit the sparse cloud
• You add/move/remove any markers
• You changed any GCP or camera coordinates
• You change camera calibration groups/calibration model
• You want to optimize additional lens distortion parameters

Hitting the update button will do nothing unless:

• You add/move/remove any markers
• You changed any GCP or camera coordinates

Note these lists are probably not exhaustive, and I have no inside knowledge so this is all based on experience and observation.

[James]

...continued

So if you align your images with GCPs from the start, and do not need to change any coordinates, and are happy with the sparse cloud and error values, then you don't really need to optimize/update anything.

If you align your images with GCPs from the start, and notice some GCPs are misplaced or have incorrect coordinates then you would probably want to correct that and then optimize.

If you align your images with or without GCPs and there is clearly something dodgy in the sparse cloud then you would want to clean up the sparse cloud and then optimize.

If you align your images without GCPs and it looks fine, then add GCPs then you would hit update and inspect the error values.

If they are all acceptable then you probably don't need to do anything.

If they are not acceptable you might find that you entered one incorrectly, in which case you would correct it and hit update again. Then inspect the error values and if they are now acceptable then nothing more to do.

If they are still not as good as you think they should be then you can assign an accuracy value to each and hit optimize and it will try to re-estimate everything with this additional information. If it turns out that you did still enter the coordinates incorrectly then the 'optimized' alignment may be worse than it was before.

If it was not acceptable and you didn't enter anything incorrectly then you may have misplaced a marker in one or more images, in which case you need to correct this and hit optimize.

If you really made no mistakes with markers, and your images are ok, then photoscan has mismatched something and you need to find the error in the sparse cloud and delete those points before trying optimize again.

When it still doesn't work then that's a whole other thread.

[Jeremiah_ROWE]

Thanks for your well thought response. Seems to align with my general way of thinking about it too.