I think it fails for a couple reasons.
First, the image you see is really a combination of several thousand, 1 pixel tall, photos. Those very narrow photos don't contain enough data for the program to compute the distortion parameters and there's no overlap.
Second, during post processing, every single one of those 1pixel tall photos is scaled and translated using the GPS/IMU data to smooth out the plane's movement. RAW imagery out of an airborn linear CCD isn't very pleasing to look at.. too many undulations to be useful without a lot of post processing. That post processing changes every image and makes it worthless for photomodeler reconstructions.
If you took a set of photos that you take with a normal digital camera and open them all in photoshop and scaled just all a slightly different amount.. they wouldn't work in photo modeler anymore either. It's the same thing here.