David hi,
if you have an error of 50 cm but it is more or less evenly distributed at the most of a DEM than I think it's just the matter of correction at GIS software. More odd when the delta varies in the range between -2 m and +2 m and absolutely unevenly.
I remember that recently you dwelled on comparision between LIDAR and PS for a castle roof. As far as I remember you reported (finally) good results +- 20 cm.
What did you do there to improve results?
I, personally, do not experience the problem at the sites like e.g. fields but when it comes to a more complex areas, e.g. densely forested areas, my models get distorted mostly at the edges and at different spots inside the model (randomly). To a degree I account it for GCPs placement, which is not an easy task in the forest (vegetation issues etc. - i.e. it's hard to find a bare or true ground level).
On the other hand, I have also noticed a problem that PS sometimes doesn't reconstruct individual (or groups of individual) trees, which is also a pain and may cause additional problems.
Have you resolved your problems with model distortion.
George
P.S. Btw, what a UAV do you have?