Underlying Algorithms for PhotoScan

[Olliebrown]

We make use of photoscan to seed our research into light field rendering and digital curation.  As such we often mention photoscan in our research papers as being an incredible tool that puts powerful computer vision algorithms in a simple, easy to use and tune package!

In doing so we've speculated as to the actual algorithms in use under the hood.  Since this is proprietary software that is not open source we don't know for certain what is in use but based on the output generated and the format of the console messages we have been able to make some reasonable deductions.

I'm wondering if you guys (Agisoft) would be willing to officially confirm what general algorithms you use, minus all the details that are trade secrets of course (you wield all the algorithms way better than any other software out there and I don't want to undermine you ability to make money off of that).

Here are some of the things I've assumed up to now:

• Some form of SIFT or SURF must be in use to identify features up front along with descriptors for matching
• Bundle adjust in some form must be used to fit the camera model for each view and get accurate camera poses and the sparse point cloud (The console output also suggests this is an SfM process but I don't know enough here to deduce anything more specific)
• The dense cloud reconstruction must be some form of MVS disparity calculation, again I don't know enough to deduce more than that
• The mesh reconstruction is clearly the free implementation of screened poisson surface reconstruction available here: http://www.cs.jhu.edu/~misha/Code/PoissonRecon/Version6.13/
• Texture generation, I have no idea how this is done, if there is an appropriate research paper about it, especially the color correction (which at least also seems to be utilizing bundle adjust but I have no background here to deduce more)

Anyways, just wondering if you can share more so we can talk about it more precisely in academic circles where they care about these sorts of things.

Thanks!
Seth Berrier

[green]

check here:
http://www.agisoft.com/forum/index.php?topic=89.0

[Olliebrown]

HA!  That's almost uncanny how similar that is to my post.  Thanks for the link.

[green]

We make use of photoscan to seed our research into light field rendering and digital curation.  As such we often mention photoscan in our research papers as being an incredible tool that puts powerful computer vision algorithms in a simple, easy to use and tune package!

In doing so we've speculated as to the actual algorithms in use under the hood.  Since this is proprietary software that is not open source we don't know for certain what is in use but based on the output generated and the format of the console messages we have been able to make some reasonable deductions.

I'm wondering if you guys (Agisoft) would be willing to officially confirm what general algorithms you use, minus all the details that are trade secrets of course (you wield all the algorithms way better than any other software out there and I don't want to undermine you ability to make money off of that).

Here are some of the things I've assumed up to now:

• Some form of SIFT or SURF must be in use to identify features up front along with descriptors for matching
• Bundle adjust in some form must be used to fit the camera model for each view and get accurate camera poses and the sparse point cloud (The console output also suggests this is an SfM process but I don't know enough here to deduce anything more specific)
• The dense cloud reconstruction must be some form of MVS disparity calculation, again I don't know enough to deduce more than that
• The mesh reconstruction is clearly the free implementation of screened poisson surface reconstruction available here: http://www.cs.jhu.edu/~misha/Code/PoissonRecon/Version6.13/
• Texture generation, I have no idea how this is done, if there is an appropriate research paper about it, especially the color correction (which at least also seems to be utilizing bundle adjust but I have no background here to deduce more)

Anyways, just wondering if you can share more so we can talk about it more precisely in academic circles where they care about these sorts of things.

Thanks!
Seth Berrier

It would be cool if you can mention which institute you are working with? and if you provide links to those papers!  You may want to add them to AgiSoft Wiki:
1. http://wiki.agisoft.com/wiki/Main_Page
or
2. http://www.agisoft.ru/w/index.php?title=Main_Page

Note: don't expect Agisoft to provide any further details after 2011 post . Many companies play catch up - mainly Autodesk .

[Olliebrown]

Absolutely!  I will provide links and just generally talk about our work in the forums once we get it published (we hold it close until then).  It's submitted to Digital Heritage 2015 now and we'll hear back pretty soon it looks like.

Finger crossed!

Thanks for the info everyone.

Seth B.

[Vlad]

As I understand from old days nothing changed.
After some comparison with other modern photogrammetry software commercial and opensource I found that only photoscan required masks. And create mesh with known amount of polygons that reproduce dense cloud shape with all it noise.
After read some of research papers with dense, mesh etc. reconstruction I understand that
all photogrammetry work with 3D volume in Photoscan stop after dense cloud generation.
After this step is only "brute force" generation of mesh with other post processing of this data.( poison, decimate, laplassian smooth, etc)
While already exist different algorithms that use depth map data in mesh reconstruction and mesh refinement steps.
Autodesk Remake, Bentley ContextCapture, CapturingReality and OpenMVS use such algorithms. And results in this programs or toolsets is looks clean and have more details (details != noise) than from Photoscan.

I hope Agisoft team made something with this and shift quality of it great app to quality comparable with other modern software by changing current used but autdated algorithms to modern.