Hello Alexey, By any chance, did you have time to look further into this issue ? I can confirm that such behaviour is present every time in 1.7.2 but with fluctuating strength.
And in the full release? (https://www.agisoft.com/downloads/installer/)
Indeed, exactly the same results, with and without masks...There is a real problem with the new depth maps.