sequence of photo alignment

[RalfH]

Still trying to get the best results out of PhotoScan, and still coming up with questions. This one is about the sequence of photo alignment.

In one of my recent PhotoScan projects, trying to align all photos in a single process resulted in a big mess. The cameras were scattered all over (even though they should have been more or less in one plane as I was photographing a desert surface while walking back and forth), and there were numerous modelled ground surfaces intersecting each other at different angles.

While playing with the project for a while I found that it can help to align a few photos (between 2 and 4 in this case). If they don't align as expected, reset photo alignment and try with another group of photos until a small number of photos aligns well. If I then individually (one by one) align the remaining photos, they are aligned to the existing well-modelled group, and the entire model comes out quite clean.

I find it quite strange that the sequence of photo alignment influences the modelling result.

My question now is what causes this behaviour and why PhotoScan is not able to achieve comparable results by itself and whether any a priori information (e.g., camera positions should be more or less in a plane) could be fed into a future version of PhotoScan to improve the results.

[FoodMan]

nice find

f/

[pixel_Killer]

Hey RalfH,

I'm only guessing on this one but I assume your desert scene is not composed of dunes? eh. It was just the first thing that came to mind. I think it's one of the downside of auto feature detection. Agisoft already does a good job at it, but the scene really needs a good amount of textures/patterns to do some kind of alignment and triangulations.

Sometimes when out shooting, I tend to overestimate Agisoft capabilites. "Yeah there's enough texture and data for detection. It'll work, I'm sure it will." lol.

Laying down a few markers or ping pong balls may help with alignment. I'm not sure if there is an ultimate shooting method that will work for all scenarios. Maybe I'm dead wrong.

I wish the standard version offered markers/ground controls as apart of the basic package. Matchmoving/camera tracking is a part of my profession... and I've come to hate when a programs takes full control without any user input.


"They've done studies, you know. 60% of the time, it works every time." -Anchorman

[RalfH]

Hello pixel_Killer,

it's not dunes but desert pavement, so there is some unique texture. Of course it's not a great target, especially when you can't get near-nadir images but have to use obliques. I was just testing shooting methods and trying to find the limit of what PhotoScan can handle, with "walking along keeping the shutter pressed" as an extreme scenario.

What surprised me (as I said) was that the outcome of the 3D modelling differed a lot depending on the sequence in which the photo alignment was done, and I'd like to know why. Also, I think that knowing this, perhaps the performance of PhotoScan can be further improved (e.g. by aligning the best matches first and perhaps allowing user input for poor matches).

And I completely agree that there should be as much user control as possible (and/or, at the very least, very good documentation so the user can understand what is going on).

[ju523m]

Hi,

as far as I have understood PS follows in general the same principles as Bundler - the SfM package by Noah Snavely - but PS comes with a more sophisticated implementation of the algorithms. It all starts with automatic detection and matching of tiepoints in order to find homologous points in the images. This is normally done by some kind of feature extraction mechanism (SIFT, SURF or other). The potential tiepoint candidates are then filtered for outliers or error-prone points by statistical means and the resulting best matches are fed into the scene reconstruction process which uses the principles of epiolar-geometry to determine the relative and interior orientation of the cameras. In Bundler this is a sequential process that starts with one pair of images and succsively adds one image after the other.

So, when you have bad luck and your choosen start-image-pair leads to a bad paramaters - for what reason soever -, you will not get happy with the rest.

Therefore in Bundler you can define which images should be taken as initial pair for the calculations.

But this must not also  be valid for PS.

The Problem with the PS docu is, that it does not explain what is going on behind the scenes. But when one is concerned with aerial surveying it is absolutely necessary to know what the software is doing behind it's nice GUI.
The reason is simple: You must give a prove to your customer for the accuracy of your calculations. But this seems to be difficult with Photoscan as the other discussion about accuracy show.

Cheers


p.S.  I made a similar "camera in Burst mode Test" with a Nikon D50, 55mm focal length, altitude aGR 560 metres, speed aGr 100km/h. It tuned out, that image base distance was to small, or in other words the images had partially too much endlap.

For classical stereoimages (binocular human vision) there is a formular to calculate the optimal image base distance :

                                           nearest point distance * far point distance * 0.02
image base distance = ------------------------------------------------------------
                                           far point distance - nearest point distance


Or you can alternatively use the rule of thumb: Image base distance = 1/30 of nearest point distance

[RalfH]

OK, sounds logical and explains the issue. And I do agree that better documentation would help a lot - using black-box approaches where you have to blindly trust the results is frowned upon both in business and in science.

I guess it would be great if Alexey could tell us which criteria are used by PhotoScan to select the starting images / image sequence for alignment. To me there seems to be some potential for the improvement of image alignment, be it fully automatic or (preferably) automatic with user-controllable options (e.g., number of matches, quality of matches...). On the other hand, I greatly appreciate Agisoft's efforts to keep the software very easy to use and not to overwhelm the user with hundreds of settings and parameters. From my point of view: keep it simple, but add more optional user controls.

[ajg-cal]

Maybe, by restricting alignment to certain groups of photos, you're changing the pairs that are preselected.

If you choose groups of photos that you know are near each other, you perhaps reduce the chance of incorrect pairings.

thanks for the detailed post ju523m - it was interesting!

[xabierr]

I am facing a similar problem. Have 700+ photos along a transect. If trying to align them all at the same time the results are not good. However, I can align subsets of photos at a time (between 5-20 photos). So I'm trying to align the subsets as different chunks and then align the chunks based on points. 

A this procedure is very time consuming, I was just wondering if this is the best way to go or is there a better approach that could reduce time and/or error introduction? Maybe a script that chooses best pairs and then adds photos until no more alignment is possible at which point a new chunk is created and process re-iterated?

Thanks

Javier

[Marcel]

I have a question that is somewhat related to this: when doing the alignment, does Photoscan try to match all points in all photos to eachother?

Because in situations like this, there are only matches to be found in the 3-4 nearest photos:



Trying to match the points between photos in the first and last row would be a waste of time because there can never be a match.

So does Photoscan make a quick pre-scan to see which photos share the same features? If not, that might be a really nice optimization to speed up the alignment phase.

[James]

So does Photoscan make a quick pre-scan to see which photos share the same features?

I believe this is what the generic 'pair pre-selection' option in the align images stage does if you enable it. Otherwise I understand that it does indeed compare each image with every other

Generic pair pre-selection does work pretty well i have found so now I use it often.

[Alexey Pasumansky]

Hello Marcel,

Using Generic pre-selection will speed up the processing of the big datasets. As James already mentioned, in this case PhotoScan will try to find overlapping images using downscaled images at first and then will be matching only such images and not every pair.

Also it could be useful to have camera position coordinates in the shooting scenario like on your screenshot. In this case you'll be able to choose Ground Control pre-selection option.

[Alexey Pasumansky]

Hello Javier,

If PhotoScan fails to align the whole dataset then point based chunk alignment could be useless since chunk could have insufficient overlap.

Are you sure that overlap is good for the project you have problems with?

[Marcel]

Using Generic pre-selection will speed up the processing of the big datasets. As James already mentioned, in this case PhotoScan will try to find overlapping images using downscaled images at first and then will be matching only such images and not every pair.

Thanks Alexey and James!

I mistakenly thought that the pair selection was only when using camera pairs (2 cameras at a fixed distance). Perhaps this option should be called 'pre-scan' instead, so it is more clear what it does?

My 18 hour alignment is almost done, with the next alignment I'll have it enabled, I am very curious how it will impact the speed!