Aerial photogrammetry (DSM + ortho) in 1.0 for "large" projects (~1000 images)

[andyroo]

I am interested in what other people are doing for a 1.0 workflow with "large" aerial projects.

[Edited 12 Mar 2014 to add camera calibration step since this helps sooo much with GCP placement - thankyou Porly for your suggestion]

First, I want to say thank you and cheers to the Agisoft team. I am really enjoying the changes that are taking shape in 1.0 and I love how involved with the user community you are on this forum and in PMs. It's a pleasure to work with folks who do such a great job of supporting and developing their product while keeping in touch with the user community.

I collect aerial imagery via Cessna with a wing-mounted 12MP camera at ~600 m elevation over ~25 linear km shooting images every 3 seconds in four overlapping passes (about 1km wide swath). I am using a Canon D10, but about to do a flight with a EOS M (22mm EF-M lens and big beautiful sensor - for my low budget anyway). I am doing repeat flights at least once/month. Ground pixel resolution is 10-15cm depending on elevation, but generally about 12cm.

With the changes in processing in 1.0, and my ongoing learning, I find that I get best results for the overall dataset as follows:

0 (new step) If you have used your camera(s) on another project, Export the adjusted camera calibration and import it into the new project to improve initial alignment and reduce/eliminate bowl effect - this was a HUGE timesaver for me since the bowl effect made placing GCPS a hunting effort)

1)Align all images with Accuracy=High, Pair pre-selection=generic, point limit=40 000.

2) trim flyers and sinkers - I usually just trim the obvious stuff - though I have experimented with gradual selection - I would be interested in other folks' experiences for 1.0 workflow using gradual selection especially.

3)Set Coordinate system, Import GCPs and manually place 3 to 4 GCPs that are well-distributed over the flight area (2 images each). Then update georeferencing. This generally gets all the other calculated point locations somewhat close to their real location.

4) starting from one end of the project, I sort GCPs by lat or long (depending on orientation of project) and work my way through all GCPs by filtering photos by marker, then placing the GCPs on all images where they are visible. Generally I update georeferencing after doing this for each GCP.

5) After all GCPs are placed, I optimize alignment, unchecking fit aspect, fit skew, and fit k4 - this is based on what I read on earlier forum postings. I am especially interested in feedback on this step too. Seems like skew might be useful for folks trying to do stuff with rolling shutter cameras, but not sure..

6) copy optimized model into multiple chunks and clip each chunk to about a 5th of the model (roughly 5km sections). Trim GCPs and cameras and adjust model extent. Generally I overlap about 500 meters (2 or 3 GCPs) on each side with the adjacent chunk. Note that doing the GCPs first before splitting saves a lot of GCP placement time and seems to provide better continuity.

7) generate dense pointcloud with ultra-high quality/moderate depth filtering ( I wish there was more control over depth filtering - still dealing with bad noise in water). This gives me about 400 million points per chunk.

8 ) build mesh with height field/dense cloud/Interpolation enabled/custom face count = 40 million faces. This gives me good enough quality to produce a DSM with 0.5m xy resolution where I can resolve features with z relief of about the same magnitude, like logs on the ground and slope breaks from coarse to fine sediment in scarps.

9) export color corrected RGB-average image and DSM.

Notes:

Step 8 and 9 are batch-processing. I would love to develop a python script to always export DSMs and models with the same coordinate system and extents (and resolution for the DSM) but I haven't had time to sit down and play with the API yet.

Up until recently (maybe build 1684?) I generally got good orthophoto results (RGB average) by simply constructing a mesh for each chunk from the sparse cloud, then I constructed a dense mesh (custom, 40-50 million faces) for the DSM. I liked the sparse cloud orthos because they didn't cause so many artifacts in forested areas - and the trees looked more natural in the orthophoto. Now it seems like there are blending or projection issues with the sparse cloud that are resolved when using the dense cloud for orthos, but it takes much longer (about 30 hrs to build dense cloud and mesh in my case).

I use RGB averaging because I find that it increases detail from my relatively noisy sensor by essentially working like image stacking to increase the signal to noise ratio.

Hardware:

My system is aDell T7500 with dual Xeon X5647s@2.93GHz with 192GB RAM and either 1 NVidia GeForce 560Ti and one ATI HD 7970 or two ATI 7970s depending on how successful I am at making everything happy together. I just ordered a R9 290, but man those things are hard to get your hands on!

[oxiabit]

Hi andyroo

I am also interested about photogrammetry. We are trying to produce orthophotos.

In a few days, we are about to buy new workstation.

I looked at your system. Your memory is 192Gb. how much size you are able to produce with this memory.

We are planing to work on up to 300 hectar area with uav. But ofcourse not single flight. multiple flights. We need to merge chunks i Think. So, What do you advise?

[andyroo]

oxiabit, I replied in PM to your question, since it's off-topic for the subject (ortho/dsm workflow in 1.0)

[piste]

Hi Andyroo...
I see that your workflow is quite similar to that one used for "small" dataset.
I have some question for you.

• What about gradual selection? I read that someone trim flyers and sinkers with gradual selection in so doing...select point in the sparse cloud with reproduction error higher than 1 by dragging the slider and delete them. Then they optimize the point cloud and make up to 3 iterations untill they are satisfied with accuracy. In so doing they can remove the bowl effect due to the lens before optimizing point cloud starting from GCP coordinates...
  Do you use the same workflow? Any change in 1.0.0 version of Photoscan?
• What do you mean in step 3 and 4? Could you explain it?

Thanks of all
Greatings
David

[andyroo]

Hi David,
 about your questions:

(1) I haven't had luck with gradual selection removing the flyers and sinkers. Sometimes it's good for cleaning an "exploded" area of the pointcloud from one or two misaligned images, but most of my flyers and sinkers are from spectral reflection on water, and I manually edit the worst of those - but it is still a big problem for me in the wetted areas. I haven't really been using the "multiple iterations" method of gradual selection because I get pretty good results with my method of first manual cleaning, then adding GCPs, then optimizing (only once).

(2) In step 3 I first change Ground Control Settings from local coordinates to WGS84 (EPSG:4326), then I import my Ground Control Points (GCPs) using JMR's method posted here (because I am doing multiple flights over the same area). Once the GCPs show up in my marker list, then I manually place markers on two photos each for four GCPs that bracket the corners of my project (it is a 1km wide ~25km long strip). Then I update (not optimimize) the ground control. This puts all of my points in tentative locations - generally pretty close to where they belong where they are near the GCPs I already added, and further as I go further away.

In step four, I was just saying that because I have a strip, I sort the GCPs by lat/long and work my way from one end of the strip to the other. Generally I update every couple of GCPs. I try to avoid optimizing until the end, but sometimes my GCPs are so far away from where they should be (because of the bowl effect) that I optimize once before I am finished.

Andy

[piste]

Thanks Andy!!
I totally agree with you on the method to be followed.
The only point in which I can't be sure yet is that about manual or automatic (gradual selection) cleaning of point cloud...
at this stage we can intervene to improve significantly the quality of the final model (according to me) and eliminate distortions caused by the lens...
can anyone suggest a workflow functional?
you can skip this step if we are in possession of a camera calibration file?
someone managed to correctly calibrate the camera and lens coupled with "Lens" to get a usable file for processing images?
Many Thanks...
David

[andyroo]

I am also wondering if there is another approach for generating final orthophotos. Especially the part where I add GCPs after sparse cloud generation, then optimize and chop into chunks before dense cloud and mesh. It seems like I would make a single point cloud with a couple billion points (or crash my workstation) if I try to do the whole scene with one dense cloud and one mesh (plus I'd have a crazy number of polys and probably couldn't view. Right now I do about 40 mil polys/chunk after breaking up project into multiple chunks. Anyone have a better idea?

[ayounggun]

Hi andy

is there a reason why you don't create an temp low res model at the beginning of stage 4?

When doing this, you can use the guided marker placement and then tweak those guided markers.

[andyroo]

Hi ayounggun.

The main reason that I don't do that is that I would have to generate the low-res model, which still takes quite some time with ~1000 - 1500 images. The second reason is that I think I would still go through and manually place each GCP, or at least most of them, so I don't think it would save me much, or maybe any time.  In the method using only the aligned images, for each GCP I place, after I place the GCP on two images, guided marker placement still aligns the little gray wisp symbol pretty close to the GCP. And once I have the model tied to three of the GCPs, the others fall pretty close to where they belong to start with. As a side note, I find that after I place 2 or 3 markers, then right-clicking and selecting "filter photos by marker" will sometimes pop up a few more images with the GCP on them.

It gets tedious with a couple hundred control points and a couple thousand images, but I think it's arguably the most important step in the whole process. Basically I haven't found a better method (when importing GCPs) than the one described in JMR's "GCP faster" post.

I think the things that would speed up this method significantly would be:

1) an "accept" hotkey that would accept the current position of the "gray wisp" when (1) the marker is the only marker selected from the Ground Control pane, and (2) the image is the only image selected on the photo pane and is visible in the image window (ideally either the photo pane or the window pane could have focus). Right now I click on each photo, then if I like where the marker is, I have to click on it without moving it too much. if I could place a few markers, then go through the other 10 images by clicking (or arrow keys?) to each in the photo pane  and press the "G" key (for example) if I like where the wisp is on that photo.

2) a "target recognition" mode where you could toggle on a shape-finding algorithm that would "auto-pick" the center of a high-contrast target. The challenge would be for the tool to do a good job of finding the target boundaries. I've used a Leica TLS that had that option, but I am pretty sure it was looking at return intensity, and I had to tell the software if it was a cylinder, sphere, etc.

[piste]

Hi Andyroo!!
I have never used multiple chunks in Photoscan, since I use your same workflow but with small dataset (100-150 photos).
I haven't a workstation so it could be usefull processing these few photos in multiple chunks...
Could you explain me step by step your point 6 please?
Another question...Have you ever used a calibrated camera in Photoscan processing? How does workflow change in this case?
Thanks

[andyroo]

...Could you explain me step by step your point 6 please?
Another question...Have you ever used a calibrated camera in Photoscan processing?

Hi piste,

First, I have never used a calibrated camera, so I am not sure how that would affect either my workflow or my end product (does anybody know? I am curious too).

 I have done all of my work so far with Canon cameras because I can control them through firmware modification (CHDK on point/shoot, Magic Lantern or Tragic Lantern on EOS).

Second, below I try to explain step-by-step for #6 above. Note that I have a footnote that I think is a simpler workflow, it just leaves the cameras and GCPS from the overall project, which seems sloppy to me...

First, some context: I am starting from a large project with large extent, aligned from GCPs as described in earlier steps. So picture a big strip with about 1000-1500 images. I can do alignment and optimization with this big strip, but to get the resolution I want (0.5m DSM over tens of km) I would need to have maybe 200 million polys, and if I process on ultra-high, my dense cloud would be maybe 1.5 - 2 billion points. I think even my machine can't do that. BUT if I break the project into chunks before alignment, I have to duplicate GCP placement. That step is the most time consuming (for me, not my computer) so the main reason to do this whole process chain is to save time in that step - I have more than 100 GCPs and I hand-place them on each target in each photo, since this is the main control I have on accuracy.

So, starting from my 1 chunk with 1500 images with GCPs and sparse cloud:

1) duplicate chunk 4 times.
2) rename each chunk (BEACH,LR,ALD,MR,MILLS) (I regularly fly the same path)
3) For each chunk (with that chunk selected/active) and with cameras,markers, and region visible:
  a) select the area(s) in model view that are outside of the extent of the chunk I want to end up with.
  b) right click on selected photos in "photos" pane and Remove Cameras
  c) right click on selected markers in "Ground Control" pane and Remove Markers
  d) in model view, select edit/delete selection
  e) select tools/reset region. This clips the region to the new chunk extent.
  f) adjust the region by hand if desired

That's it!*

*note that I think I can replace 3 with this:

3) For each chunk (with that chunk selected/active) and with cameras,markers, and region visible, select the extent of the chunk I want to end up with and select Edit/crop selection. Then select Tools/reset region, and adjust the region by hand if desired.

 

[MrFilson]

Andy,

Where are you getting GCP?

[andyroo]

I am flying a repeat project (43 flights so far), so I can export GCP from earlier .psz file and import those points to a new flight. The GCPs are mostly targets I have placed, and some are easily recognizable ground features (like the end of a parking stripe or the corner of a stop stripe, on roads.

[MrFilson]

OK, I should rephrase my question. I am looking for a way to accurately collect some ground control points to add to my imagery. Like you I am flying a 172 and am able to collect 1000's of 10mpx images all day long over very large areas.

I am looking to tighten up my imagery to be able produce reasonably high accuracy topo line maps. I do not want (nor can I get) a full blown survey of these very large areas.

Im just wanting to get as accurate GCP's of my own to add to my imagery on some small test projects.

I am looking at what adding the ground control would be able to do on small projects. I need to find a way collecting it and then going from there on dialing it in even further.

[MrFilson]

I get it on picking targets out and them being visible in the imagery.  My issue is getting accurate lat, long, & elevation of those visible targets.