If I understood well, there are just two points in common between two photos. This is not enough as they behave like a hinge. you'll need more, and I'd recommend 5... this is going to take long, and seems not the way to go with such a big number of photos.
I have downloaded your photo and seen that Photoscan has been able to find 82700 feature points which is beyond very good. This means you must doing something wrong unless ovelap is near 0%. So the problem in my opinion must come from a diferent source.
what can you tell about camera and lens?. Have you set a camera model with numbers that make sense?. Please, send screenshots of the camera calibration dialog showing us initial and estimated parameters.
Best,
GEOBIT