Algorithms used in Photoscan

[Arko]

Dear Dmitry,
First of all, I would like to say that you have produced a very impressive bit of software. We are using Photoscan and Photoscan Professional here at the University of Tasmania to derive 3D point clouds from historical aerial photography and aerial photographs acquired from an unmanned aerial vehicle (UAV). I have previously worked with SIFT, libsiftfast, Bundler, PMVS2, and CMVS, and have achieved some very good results with these tools. I have come across some examples where Photoscan outperforms Bundler when it comes to calculating camera position, orientation, and distortion parameters, especially when the focal length is unknown or uncertain (like in underwater photography). I am keen to publish some of our results in scientific journals, however, in order to do that I want to be able to report on the general type of algorithms that are used in Photoscan. As far as I can tell Photoscan follows a very similar approach to the following workflow:

• SIFT
• key matching
• bundle adjustment for camera position, orientation, and distortion parameters + sparse point cloud (based on Bundler?)
• patch-based multi-view stereo for dense point cloud reconstruction (based on PMVS2 and or CMVS?)
• surface reconstruction based on Delauney triangulation or Poisson surface reconstruction
• Texture mapping

I am curious to known if you developed your own algorithms from scratch or if you have built on and improved on existing algorithms. I realise that you might not be able to or not want to provide too much technical detail. However, in order to use the Photoscan results in a publication I want to be able to report on the workflow and methods. Any help would be greatly appreciated.
Thanks in advance and keep up the great work!
Arko

[Dmitry Semyonov]

Hello Arko,

We are happy to get positive feedback regarding PhotoScan software.

PhotoScan workflow is similar to the one you have presented, with an exception that our implementation is not based on the popular Bundler+PMVS2+CMVS assembly.

Here is a more detailed explanation of individual processing steps:

• Feature matching across the photos.
  At the first stage PhotoScan detects points in the source photos which are stable under viewpoint and lighting variations and generates a descriptor for each point based on its local neighborhood. These descriptors are used later to detect correspondences across the photos. This is similar to the well known SIFT approach, but uses different algorithms for a little bit higher alignment quality.
• Solving for camera intrinsic and extrinsic orientation parameters.
  PhotoScan uses a greedy algorithm to find approximate camera locations and refines them later using a bundle-adjustment algorithm. This should have many things in common with Bundler, although we didn't compare our algorithm with Bundler thoroughly.
  
• Dense surface reconstruction.
  At this step several processing algorithms are available. Exact, Smooth and Height-field methods are based on pair-wise depth map computation, while Fast method utilizes a multi-view approach.
  
• Texture mapping.
  At this stage PhotoScan parametrizes a surface possibly cutting it in smaller pieces, and then blends source photos to form a texture atlas.
  

Many of the PhotoScan algorithms are based in part on previously published papers, but are implemented from scratch and are thoroughly optimized for faster processing speeds. It is worth noting that we have favored algorithms with higher accuracy output over faster approaches with less accurate output.

With best regards,
Dmitry Semyonov
AgiSoft LLC

[sjupin]

hi,

you r referencing papers your great soft is based on, would it be possible to get
references of this papers ?
I would need these for biblio for a paper about using photoscan on standing stones a research lab.
by the way, I really loves the 0.9 evols.

thks for the work.

[an198317]

Hi Dmitry,

We are a research team at Kansas State University. We used PhotoScan Pro heavily for various of projects. So do you think it's possible to share the literatures you mentioned in your post? We need do a project report as well based on Algorithms Photoscan uses.

Thanks a lot!!
Nan

Hello Arko,

We are happy to get positive feedback regarding PhotoScan software.

PhotoScan workflow is similar to the one you have presented, with an exception that our implementation is not based on the popular Bundler+PMVS2+CMVS assembly.

Here is a more detailed explanation of individual processing steps:

• Feature matching across the photos.
  At the first stage PhotoScan detects points in the source photos which are stable under viewpoint and lighting variations and generates a descriptor for each point based on its local neighborhood. These descriptors are used later to detect correspondences across the photos. This is similar to the well known SIFT approach, but uses different algorithms for a little bit higher alignment quality.
• Solving for camera intrinsic and extrinsic orientation parameters.
  PhotoScan uses a greedy algorithm to find approximate camera locations and refines them later using a bundle-adjustment algorithm. This should have many things in common with Bundler, although we didn't compare our algorithm with Bundler thoroughly.
  
• Dense surface reconstruction.
  At this step several processing algorithms are available. Exact, Smooth and Height-field methods are based on pair-wise depth map computation, while Fast method utilizes a multi-view approach.
  
• Texture mapping.
  At this stage PhotoScan parametrizes a surface possibly cutting it in smaller pieces, and then blends source photos to form a texture atlas.
  

Many of the PhotoScan algorithms are based in part on previously published papers, but are implemented from scratch and are thoroughly optimized for faster processing speeds. It is worth noting that we have favored algorithms with higher accuracy output over faster approaches with less accurate output.

With best regards,
Dmitry Semyonov
AgiSoft LLC

[Ludwig Hilger]

Hello,

same here! has there been any info concerning the papers yet?

best regards,
Ludwig

[Christina K]

this is very interesting, any paper yet for reference on the algorithms used in photoscan?

[stihl]

Have you really just bumped up a topic from page 29?!

[ThomasVD]

Not sure if this would be too much of a "treat" to competing photogrammetry software companies, but it would indeed be interesting to know on which algorithms each step is based, specifically when writing academic articles.

To answer one of Arko's original questions; I believe PhotoScan's Surface Reconstruction algorithm is based on the Poisson Surface Reconstruction algorithm, but modified to automatically remove excessively large polygons (ie the mesh is not closed). I took a dense point cloud from PhotoScan, processed it in Meshlab with Poisson Reconstruction, and the resulting mesh was a precise copy of the mesh produced in PhotoScan (but PhotoScan is a lot more stable than Meshlab, yey!)

[jtuhtan]

Would also be interested in the papers, since we are often asked which algorithims are implemented and are currently forced to admit that we can only guess..

[ugo]

Hello,

can i get this algorithmus papers?

[hachpai]

Hello!

I would be really interested too. Maybe you cannot provide everything, but a pseudo code, or kind of detailed functions workflow is thinkable?

We're experiencing a lot, and would like to understand correctly what is happening under the hood.

Thanks!

PH

Labokube