OK, so I found my problem and it is simpler than what one would think.
In the cases I looked at the images of the two different sets seem to have a two high angle between them, so that PS is not able to match them. In another case it worked.
Masking only one of the two sets is also OK, but sure, one has to clean the point cloud later on (which for me seems to be less work than masking 25 images).
So, I think that was it with this subject.
Cheers