Hi,
Maybe you have made progress on this since your original post? I have been looking at direct georeferencing recently and have found an issue I can't explain - it may (or may not) be relevant to your work.
I find that during optimisation, the relative weighting between camera positions used as control measurements and tie points is not handled as I would expect in a least-squares bundle adjustment. If poor-precision camera position measurements, included as control, are degrading the shape of the surface model, I have not been able to improve the model shape by changing the camera 'accuracy' setting to reflect greater uncertainty in the camera locations. It seems that changing the 'camera accuracy' setting doesn't vary the relative weight between camera data and tie points(?). Of course, one could optimise without using the camera positions, then simply update the georeferencing afterwards, but this shouldn't be necessary. I have also found that the relative weights between cameras do seem to be handled - i.e. you can successfully down-weight one camera with a poor position measurement with respect to the others..., just not all of them with respect to the tie points.
It doesn't immediately answer your problem, but maybe there is a contributing factor somewhere.
Maybe someone could shed some light on the relative weighting of tie points and camera positions please (I haven't found such problems with tie points vs. markers)? Any suggestions greatly appreciated.
Thanks,
Mike