Looks like there may be a disconnect in your image sequence... a sudden reduction in image overlap in one section of the building.
e.g.
https://skfb.ly/CJpR, Alcove on the left hand side appears to have been crushed. Compared that to
https://skfb.ly/CPpn A few ways forward.
1. Calibrate your lens and fix the lens parameters
2. Add GCPs through areas that have not aligned well
3. Take more shots to provide extra overlap, especially around corners of buildings
In the examples I provided above, the difference was repeating the alignment with fixed, "calibrated" lens parameters.