Optimising lens parameters without optimising camera positions

[bigben]

Not so much of a feature request (unless it works of course), but more thinking out loud about the alignment/optimisation process.

In short: in situations where the number/ position of cameras in a project is not optimal it may be possible to get better results by separating the optimisation of lens distortion parameters from the calculation of camera position/fov.

The longer version:
As a beta tester for a pano stitching application many years ago, one of the more challenging things to work out was a sequence of optimising camera orientation and lens parameters when virtually no data was provided (ie. automatic alignment).  In non-optimal situations (e.g. very narrow overlap in this case) the optimisation of lens distortion parameters could conflict with the optimisation of fov (typically k2 and fx/fy to use Photoscan equivalents), so whilst the overlap areas were well aligned, the overall image was noticeably distorted. The automated process was thus designed to firstly get optimised camera orientations and fov followed by the inclusion of lens distortion parameters.

Now I know there are some significant differences with photogrammetry, but from what I've seen, there do seem to be some similarities with the calculation of lens parameters.  To give an example I tried doing a reconstruction from 4 historical BW aerial photographs (scanned from negatives... ~80% overlap, single flight path).  Running them through with the default settings, the resulting terrain a) showed a typical "bowl" effect and b) the elevation was exaggerated vertically.
https://skfb.ly/DMCY

So based on my experience with pano stitching gone wrong I adapted the optimisation process to be a little closer to the sequence I use for panoramas.
1: Align images
2: Transfer the adjusted lens settings to the initial lens settings, but setting k1,k2,k3 to 0 and fixing the calibration
3: Repeat alignment with new settings
4: Weed out bad points
5: Optimise cameras (fov and offset only)

The result was much closer to what I was expecting
https://skfb.ly/DNUp

This would suggest to me that (in the absence of adequate data) the optimisation of lens parameters and camera positions are "overcompensating" for each other, leading to a mathematically correct result that is technically incorrect.  In such cases it may be possible to get a better result by excluding lens distortion parameters in the alignment phase, and also having the option to disable the optimisation of camera positions when optimising cameras (thus tweaking only the lens settings)

Just a thought, but may be worth exploring one day.