Your opinion on USGS Agisoft Processing Workflow

[BobvdMeij]

Dear all,

While randomly scouting across the internet in search of clarifications on certain terminology used in Agisoft I came across this (also see the attachment below) seemingly well structured Agisoft Photoscan Workflow formulated by USGS (United States Geological Survey) in March 2017. USGS being a globally renowed organization I like to believe considerable thought and extensive testing and validation led to this document.

I personally very much like the column named ‘Function’, which supposedly describes what each step does and how it affects the output. Especially because such information is often lacking in the rather technical and somewhat limited explanation provided in Agisoft’s official user manual. This is particularly valid for the various and seemingly important Gradual Selection stages. I believe this ranks among the most frequently discussed themes on this forum, although a comprehensive/ understandable explanation of what it exactly does and how it should be applied is still missing.

I'm eager to learn what you all think of this USGS workflow, how it relates to your own and if you could perhaps comment on why certain steps are executed in this order using particular settings. I’m particularly intrigued by the presented order of the Marking of GCPs, Camera Optimization and the subsequent Optimization Parameters.

I personally always use the following methodology:

1.   Align Photos
2.   Mark Ground Control and Checkpoints
3.   Optimize Camera’s (checking all parameters except p3 and p4)
4.   Gradual Selection Reprojection Error at 0.5 > delete points > Optimize Camera’s (check allexcept p3/p4)
5.   Gradual Selection Reconstruction Uncertainty at 10 > delete points > Optimize Camera’s (check all except p3/p4)
6.   Gradual Selection Projection Accuracy at 2-3 > delete points > Optimize Camera’s (check all except p3/p4)
7.   Dense Cloud > DEM > Orthomosaic

The USGS workflow, however, employs a much more complex procedure. Rather than running ALL Gradual Selection stages AFTER marking the GCPs (as done by me), the USGS workflow applies Reconstruction Uncertainty and Projection Accuracy BEFORE any GCPs are marked. Only the Reprojection Error step is executed AFTER GCPs are included. Also note that the USGS workflow suggests to change the Tie Point Accuracy setting within the Reference Settings from 1.0 to 0.1 along the way. The workflow furthermore suggests to check/uncheck different Camera Parameters for distinct Gradual Selection stages, rather than keeping this the same across the board as is done by me (and I believe by many others). 

Again, the workflow seems to be thought out well but I still cannot wrap my head around certain details. I’m hoping some of you, and Agisoft’s developers in particular, are able to reflect on the matter!

Thanks in advance.

Bob

[GPC]

There are more good files in the directory:

https://rmgsc.cr.usgs.gov/outgoing/UAS/workshop_data/2017_UAS_Federal_Users_Workshop/

Some great info there, even sample image files.

[Hydro_Ydé]

Thanks  BobvdMeij for this very interesting topic.
I found normal to clean the cloud at maximum level before including GCPs. That gives a clean "aero side only" solution before trying to adjust to real world coordinates. But that would not change a lot in my opinion to follow your own process (i.e. include GCP first).
Also the 0.1 parameter for Tie point seems very low for me, as I have often not so crisp images. 0.3 is better in my case.
One thing they don't mention is to have a look on camera calibration values. This is something I really look closely, i.e. values are consistant between various surveys. Big Z error can come from bad F value, even with good GCPs.

[andyroo]

Just wanted to provide a little context on these docs and credit where credit is due -

The material is from a 2017 workshop presented by the USGS National UAS Project Office, and based heavily on training provided by Tom Noble (US BLM, Ret.) and Neffra Matthews (US BLM).

Tom's methodology "...works for fixed cameras, a wide angle lens, decent photos and is based on years of experience, testing and comparing..." and he is the first to tell you that photogrammetry is as much an art as it is a science.

I have been to several workshops Tom has instructed or participated in, and he often cautions that these methods are based around DSLRs with prime lenses or other fixed lens and even metric cameras. The workflow as summarized in a table or a few pages of a PDF is dramatically oversimplified from the use cases and guidance he presents in his workshop.

That said, the referenced USGS materials are pretty useful background, and a great point for discussion. For example, Tom would tell you that the 0.1 pixel parameter for tie point accuracy is only something you'd constrain after you go through iterations to remove the points with bad reconstruction uncertainty and projection accuracy using the gradual selection tools, and it might not be achievable if you're not using a fixed-lens camera with minimal motion blur and a good lens.

I know Tom is pretty busy right now, but hopefully he'll provide more background when he has a chance.

Andy

[Hydro_Ydé]

Thank you Andy for the response.
Much agreed about the art part, mapping is always also a bit of artistic creation.
I hope Tom will find some time to share his experience here. This field is still quite new, and at least we are happy to see that Photoscan is a good choice as the main tool for photogrammetry. Maybe we could also talk about pre and post PS workflow.

[mks_gis]

This is really interesting! How would you update the workflow with the adaptive fitting option that PS now has? In the expanded document they say to use the default values or use the checked parameters for optimisation, which is best?

Cheers
Martin

[karad]

I think that Photoscan developers should explain the whole process in photogrammetric and adjustement terms for better understanding and judging.

[mks_gis]

I'm unclear of one thing, amongst others. Perhaps someone can help.

In the gradual selection process they mention the percentage of points selected in each step. Is that percentage of total tie points you started with, or percentage of tie points left after each step. The latter obviously reduces the tie points considerably, when I tried it with some P4 data by about 90% of tie points we started with.

[macsurveyr]

Hello All,

Learning and executing photogrammetry properly is so much more than following a workflow, a recipe or a checklist. Understanding the physics of light and the workings of a camera/lens system and the identification of sources of error in order to either eliminate or effectively model those errors is more challenging than checking all the boxes of a checklist or following all the steps of a workflow.

It is very difficult to teach and learn in a software forum - there are just too many variables - so many variables.

I will tell you that the reason to do any optimization is to improve the camera calibration. However, if the camera is not stable, with no consistent focus, has rolling shutter effect, has a small sensor, if the focal length is to long, the images are compressed, etc., then improving the calibration - no matter what parameters are checked - might be a futile effort. If the camera calibration can be improved, how far things can or should be optimized are also not always the same - many variables. The workflow cited in this forum topic - that has its origins from me - is for high quality cameras with a relatively large sensor, relatively wide angle lens, high quality photos, and is modeled after being able to achieve high quality results comparable to several other tried and true, more traditional, photogrammetry software suites from days gone by, but are still the gold standard as far as quantifiable results. Much more effort was necessary and no, it wasn’t at all obvious or straightforward as to how to achieve those results. It took a lot of experience. PhotoScan is very good and capable of achieving excellent results and while I sometimes wish good results were possible with no background knowledge and/or no experience, that just is not the case. PhotoScan has made incredible strides to that end. Maybe someday it will be the case.

Tom

[AndyC]

Well said Tom, thanks.

[Dave Martin]

Well said Tom, thanks.

+1
Dave

[jmaeding]

Oh Geeze, the results after using USGS methodology are 100x better than not.
This should be part of the agisoft standard workflow IMO.

I wonder if the gradual selection steps could be automated. I am a .net guy mainly but can adapt to other languages.
I sent my "re-write" of the USGS workflow to agisoft sales. Hope they do something with it.
In the end, the PS results were better than Dronedeploy and pix4d.

[mks_gis]

Oh Geeze, the results after using USGS methodology are 100x better than not.
This should be part of the agisoft standard workflow IMO.

I wonder if the gradual selection steps could be automated. I am a .net guy mainly but can adapt to other languages.
I sent my "re-write" of the USGS workflow to agisoft sales. Hope they do something with it.
In the end, the PS results were better than Dronedeploy and pix4d.

I've automated grad sel here using Python, loosely based on this, but not fully (only do RecUncert/ProcAcc once each):

http://www.agisoft.com/forum/index.php?topic=9578.0

workflow with GCP:
run script
set export folder
set any file prefix for export files
in Custom Menu Align with grad
Do your GCP work
in Custom menu Run all after alignment - Geo with grad

Opinions welcome.

[stihl]

Oh Geeze, the results after using USGS methodology are 100x better than not.
This should be part of the agisoft standard workflow IMO.

I wonder if the gradual selection steps could be automated. I am a .net guy mainly but can adapt to other languages.
I sent my "re-write" of the USGS workflow to agisoft sales. Hope they do something with it.
In the end, the PS results were better than Dronedeploy and pix4d.

Funny, seeing how I followed this workflow and got worse results than following our own developed workflow.

I suppose it all depends on your data set.

[mks_gis]

Funny, seeing how I followed this workflow and got worse results than following our own developed workflow.

I suppose it all depends on your data set.

Could you outline your workflow and describe your type of data, out of interest?