Manual photo alignment - tips?

[ThomasVD]

Hey everyone,

I've been working on a dataset of 58 pictures, taken with two different analogue cameras in the year 2000. The object of study is an ancient ship, the Belgian "Kogge" (http://kogge.be/). There's no 3D model of the site and it would be very cool to make one 14 years after excavation.
Since the pictures have been scanned from dias and hard copies there's no exif data, and some of the pictures are rather poor quality. Additionally they've been taken on different dates, and so objects around the vessel change from day to day. Unfortunately this is the data we've got to work with so here's my workflow:

- masked the pictures so they include only the vessel (ie avoid changing objects around it)
- automatic photo align at high accuracy, pair preselection generic, 40k point limit, constrain features by mask
=> after this phase 23/58 pictures were aligned, covering the starboard side of the vessel (see attachment)
- manually photo align by manually putting markers on unaligned as well as aligned pictures, and one by one choosing "align selected cameras" for unaligned photos.
=> after adding roughly 120 manual markers in this way I've aligned 42/58 pictures, covering both sides of the vessel (see attachment 2)

Now, while these manually aligned pictures do in fact cover every part of the port side, there's still massive holes in the dense reconstruction. Does anyone have any ideas on how to improve this result? I feel like PhotoScan only uses the matches it's found in the initial automatic "Align Photos" phase to make the sparse point cloud. Is there any way to sort of tell the software "well now that I've manually shown you where this picture goes, look really thoroughly around that area for matches in nearby pictures"?

Anyone have experience with manual picture alignment?

Thanks in advance for any advice you may have!
Cheers,

Tom

[Marcel]

I feel like PhotoScan only uses the matches it's found in the initial automatic "Align Photos" phase to make the sparse point cloud.

That would be easy enough to check: disable the manually aligned cameras and run the dense cloud reconstruction again with the same settings. If you get a similar amount of points then you could be right.

I'm still not sure if the points in the Sparse Cloud are needed for the Dense Cloud reconstruction. If tgat would be the case,  then the lack of Sparse Cloud points might explain the holes.

[David Cockey]

Sparse points are very important to dense point cloud construction. Sparse points inside the reconstruction bounding box are used to select the camera pairs used for dense point cloud. This is why sometimes dense point cloud construction to fail if the box is too small.

[Marcel]

Sparse points are very important to dense point cloud construction. Sparse points inside the reconstruction bounding box are used to select the camera pairs used for dense point cloud. This is why sometimes dense point cloud construction to fail if the box is too small.

Ah, so in Thomas case, his manually aligned cameras won't be selected because they don't have any points in the Sparse Cloud?

Is that all the Sparse Cloud does in relation to the Dense Cloud reconstruction (selecting which cameras are used)?

[David Cockey]

Ah, so in Thomas case, his manually aligned cameras won't be selected because they don't have any points in the Sparse Cloud?

Is that all the Sparse Cloud does in relation to the Dense Cloud reconstruction (selecting which cameras are used)?

That is my understanding. A method is needed to generate the Sparse Cloud without re-aligning the cameras.

Also my understanding is Dense Cloud construction only uses the Sparse Cloud to select the camera pairs it uses.

[Marcel]

Ah, so in Thomas case, his manually aligned cameras won't be selected because they don't have any points in the Sparse Cloud?

Is that all the Sparse Cloud does in relation to the Dense Cloud reconstruction (selecting which cameras are used)?

That is my understanding. A method is needed to generate the Sparse Cloud without re-aligning the cameras.

Also my understanding is Dense Cloud construction only uses the Sparse Cloud to select the camera pairs it uses.

Thanks for the info, have been wondering about that for a while. (second part especially).

[ThomasVD]

Ah, so in Thomas case, his manually aligned cameras won't be selected because they don't have any points in the Sparse Cloud?

Is that all the Sparse Cloud does in relation to the Dense Cloud reconstruction (selecting which cameras are used)?

That is my understanding. A method is needed to generate the Sparse Cloud without re-aligning the cameras.

Also my understanding is Dense Cloud construction only uses the Sparse Cloud to select the camera pairs it uses.

Exactly, it should be a method that re-generates the Sparse Cloud based on the camera's position that is now known, thanks to the manually added markers.
I find it strange for instance that when you re-align all pictures after you've added the markers, the software doesn't take the markers into consideration in order to find matches between pictures (even though the markers obviously indicate: "these and these points are the same in these two pictures, clearly there are other points in between that will match as well").

HOWEVER, a small bit of progress: I discovered that when I disable pair pre-selection, the original automatic align finds a lot more matches between pictures, and therefore after manual align, your Spare Cloud is denser:
-1st picture in attachment: pair pre-selection generic: only the very obvious overlap between two unaligned pictures has a lot of points (strip on the right), the remaining area of the picture hardly has any matches.
-2nd picture in attachment: same image aligned with pair pre-selection disabled: although processing takes a lot longer, clearly a lot more points are found in an area that was previously almost void of matches.

=> so notice: pair pre-selection can reduce time but also reduce results, specifically on qualitatively poor data sets like this one!

[Marcel]

You can also try running an alignment with Pair Selection didabled and with unlimited points (enter  0 in the point limit field). Have the console open at point extraction phase and keep an eye on the numbers,  because if it doesn't extract more than 40k points for each image then it doesn't make sense to run the alignment again.

[ThomasVD]

You can also try running an alignment with Pair Selection didabled and with unlimited points (enter  0 in the point limit field). Have the console open at point extraction phase and keep an eye on the numbers,  because if it doesn't extract more than 40k points for each image then it doesn't make sense to run the alignment again.

Aha, that's interesting! Didn't know entering 0 made for unlimited points, I'm currently in the field but when I get back I'll definitely give this a try and share the results - thanks!