Agisoft Metashape

Agisoft Metashape => General => Topic started by: George on December 20, 2012, 12:12:15 PM

Title: Accuracy (round N)
Post by: George on December 20, 2012, 12:12:15 PM
Hi all who concerned with aerial photography
what do you think would be the precision of the resulting DEM and Mosaic (XYZ) if the GCP arranged at the PS are of the following errors?
Btw, could continious optimization of the point cloud help?
Another question is would it help if I push down the error (in meters and pix) to a 0.0xxx values? Could it in the end spoil DEM?
Cheers
Title: Re: Accuracy (round N)
Post by: Oli63 on December 20, 2012, 01:55:23 PM
Don't know where is the problem. The results look fine, partly even incredibly good. The 3 spikes might have to to with bad overlapping or any other reasons, but not really a problem.
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 02:42:27 PM
... there is no actual problem with that ... I also think that the accuracy displayed is quite good and it appeared from the first try. But this is just at Photoscan GCP window.

The actual problem is that the final result systematically does not match DTM (the official one). The delta is around +- 0,6-0,7 at different places of the model, which seems to be quite weird considering accuracy of GCPs.

Below some examples of XY and XYZ tests (see images names) plus random samples - intersection lines.

Yeah ... one short comment is that the accuracy of XY to a degree dependent on my fingers - just clicking mouse button at ArcGIS
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 02:43:10 PM
...
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 02:45:38 PM
...
Title: Re: Accuracy (round N)
Post by: Alexey Pasumansky on December 20, 2012, 02:51:16 PM
Hello George,

Do you have a scheme of the GCP positions on the reconstructed area?
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 03:07:39 PM
Hi Alex,
please attached. The area is relatively large - approx. 200 ha
May be you could recommend something to that?
Cheers, George
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 03:19:57 PM
In addition the workflow was the following:
1. Alignment based on camera positions. Quality - medium (unfortunately) since High didn't work at a part of the imagery
2. Gradual selection (reprojection error  around 3)
3. Import of GCP and their arrangement at the imagery (a circle of arrangements in order to reduce an error)
4. Optimization of the point cloud. At that stage error became low - somewhat at the level 0.xxx
5. Another round of GCPs adjustment - error became extremely low 0.0xxx
6. Geometry and the rest stuff
7. GIS analysis (XYZ tests and Z samples)
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 04:07:54 PM
just in addition ... I would have appreciated if the error between DEM and DTM would had been fairly regular (+- 0.0x or even 0.00x) but it is irregular and jumps in a range of +- 1.xx meters which doesn't make confident in the model in general. Whereas individual parts may be fine
Title: Re: Accuracy (round N)
Post by: fpbv on December 20, 2012, 04:17:50 PM
George

Which camera are you using?

Is it a full frame camera?
Title: Re: Accuracy (round N)
Post by: Alexey Pasumansky on December 20, 2012, 04:39:58 PM
Hello George,

Seems like the "bowl-effect" is not completely removed during optimization.
Even with unused 4 and 5 points the positions or GCPs are OK (however, it would be better to use more points n the center and on the bottom).

Could you please check the markers with large errors - seems like these are: 10, 11, 8, 9.
After that we recommend to optimize the alignment with marker accuracy set to zero.
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 04:45:48 PM
Lumix LX5 - very simple stuff tho :)

but 24 mm angle, which is quite good
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 04:53:41 PM
Alexey,
could you explain please "bowl-effect". It stands for ...?
Will try to place more points in the center. Tho it is not an easy task. Since there are no clear objects exactly at that place of the site.
Do you mean Markers accuracy at Ground Control Settings?
Title: Re: Accuracy (round N)
Post by: George on December 20, 2012, 04:56:03 PM
Alexey, what do you think about the workflow? Generally it is default but when it comes to Gradual Selection etc. is it reasonable? Several rounds of optimization?
Title: Re: Accuracy (round N)
Post by: Alexey Pasumansky on December 20, 2012, 06:02:32 PM
Hello George,

Under "bowl-effect" I mean obvious and usually huge model distortion, like on the attached image.

The workflow described is fine.
Removing outliers and points with large reprojection error for optimization is required for most cases. Repeating such procedure two or three times may improve the results.
Title: Re: Accuracy (round N)
Post by: fpbv on December 20, 2012, 11:20:23 PM
Lumix LX5 - very simple stuff tho :)

but 24 mm angle, which is quite good

George, this a very very very good camera and your problem is not the lens.
One problem is that not a full frame camera because has the some conversion lens factor.
I would suggest you use the some DSLR like Canon 5D MK II/III or Nikon, and use a fixed focal lens.
Zoom lens didn?t work as good as fixed ones.

If you want to know more pm.

 
Title: Re: Accuracy (round N)
Post by: ju523m on December 21, 2012, 12:07:00 AM
Hi all,

George, what options did you check in the optimization dialog? I had similar problems with various scene reconstructions and in the end I came to the conclusion NOT to check the skew and aspect option in the optimization dialog. For the camera setup I used, the things improved a bit, though the error was not completely eliminated.

Cheers
Title: Re: Accuracy (round N)
Post by: George on December 21, 2012, 09:25:18 AM
I check all 3 available options tho. Do you think it is related to the camera?
Title: Re: Accuracy (round N)
Post by: George on December 21, 2012, 09:40:22 AM
Hi Fabricio,
thanks. Do you use these cams?
G


George, this a very very very good camera and your problem is not the lens.
One problem is that not a full frame camera because has the some conversion lens factor.
I would suggest you use the some DSLR like Canon 5D MK II/III or Nikon, and use a fixed focal lens.
Zoom lens didn?t work as good as fixed ones.

If you want to know more pm.
Title: Re: Accuracy (round N)
Post by: fpbv on December 21, 2012, 03:24:35 PM
Hi George

The camera adds to the problem. The zoom lens is not recommended because their internal construction do not keep the distance focal all the way through.
Beyond that, you have the crop factor this kind lens have, so you do not have the real distance focal.
So, a profissional camera with a full frame sensor - without crop factor - and a fixed lens will give you a better results.
You can use a 24, 28, 35 or 50 mm fixed lens, it depends how higher you will flight and which are your purposes.
I?ve been using a Canon EOS 5D MK II with 21 megapixel with a 28 and 35 mm fixed lens.
I am looking for a new Nikon D800 with 36.3 megapixel or the new 5D MKIII.

I hope to helped you.

Fabricio
Title: Re: Accuracy (round N)
Post by: George on December 21, 2012, 03:32:26 PM
yeah cool. That sounds really cool!
Does it matter if all my zoom options are disabled at the camera settings?
Could you share some imagery of yours and may be a DEM as well if you can, please?
What kind of UAV do you have?
Title: Re: Accuracy (round N)
Post by: George on December 21, 2012, 03:50:12 PM
this is now the third round of optimization
Dark green - delta between DEM and DTM is just about +- 10 cm (perfect)
Light green - delta +- 20 cm (good)
Blue - +- 50 cm
Yellow - +- 1 meter
White and Black a vegetation
Red are strange patterns, since at the aerial imagery these are mostly exposed areas but the resulting delta is much bigger there, however, it should not exist there :(
I will go for more points and 4th round. Will see then
Title: Re: Accuracy (round N)
Post by: fpbv on December 22, 2012, 01:39:57 AM

Does it matter if all my zoom options are disabled at the camera settings?
 

You can do that, use zoom at minimum and do not use auto focus.
But still you can have some focal distance range, unfortunaly the lens geometry is not rigid for zoom lens for use in photogrammetry.
I still do not use UAV, I saw some here in Brazil but we are dealing with some legal issues to use it properly.
I think you could change your camera for this new mirrorless digital like Nikon N1, Samsung NX 1000 or Sony Alpha NEX 7, etc.
Title: Re: Accuracy (round N)
Post by: George on December 22, 2012, 07:14:38 PM
The optimization round number 4 gave an error of 2-5-10 cm. Will share the picture later! Cool!
Title: Re: Accuracy (round N)
Post by: piste on December 23, 2012, 05:15:58 PM
Hi guys...
I have George's same problem!
We use a canon G10 mounted on an UAV.
Photos are taken with minimum zoom and, obviously, with AF activated.
My workflow is the following:

Following this workflow I have obtained results that seem very fine!! (error of 0,06 meters and 0,2 pixels).
Photos have been catched from 100 meters on terrestrial surface...
When I have exported point cloud and compared it with the terrestrial laserscanner one I observed an error on Z-value till about 50cm...
It seems too high according with what I have obtained in Photoscan...
why this?
Thanks for future replies and
Merry Christmas to all.
David
Title: Re: Accuracy (round N)
Post by: George on January 09, 2013, 02:08:49 AM
David hi,
if you have an error of 50 cm but it is more or less evenly distributed at the most of a DEM than I think it's just the matter of correction at GIS software. More odd when the delta varies in the range between -2 m and +2 m and absolutely unevenly.

I remember that recently you dwelled on comparision between LIDAR and PS for a castle roof. As far as I remember  you reported (finally) good results +- 20 cm.
What did you do there to improve results?

I, personally, do not experience the problem at the sites like e.g. fields but when it comes to a more complex areas, e.g. densely forested areas, my models get distorted mostly at the edges and at different spots inside the model (randomly). To a degree I account it for GCPs placement, which is not an easy task in the forest (vegetation issues etc. - i.e. it's hard to find a bare or true ground level).
On the other hand, I have also noticed a problem that PS sometimes doesn't reconstruct individual (or groups of individual) trees, which is also a pain and may cause additional problems.

Have you resolved your problems with model distortion.
George

P.S. Btw, what a UAV do you have?
Title: Re: Accuracy (round N)
Post by: George on January 11, 2013, 05:58:33 PM
Here are the final results ...
1. Aerial image
2. DEM minus DTM (by National Land Survey)
3. Precision is satisfactoryat the most of the model - in the range of + - 10 to 20 cm
4. The colours legend is on the DEM
Title: Re: Accuracy (round N)
Post by: George on January 11, 2013, 06:08:58 PM
DEM
The most important colour is green and its variations, little bit yellow and finally orange - the error is just 5 to 20 cm.
Other colours represent sort of natural situation at the site i.e. forest, other vegetation, objects etc.
Title: Re: Accuracy (round N)
Post by: Oli63 on January 14, 2013, 01:48:18 AM
George, what software did you use to create the jpg from the last post (out of the Geo-tiff I guess)?
Title: Re: Accuracy (round N)
Post by: George on January 14, 2013, 09:23:10 AM
Hi Oli63
it is a screenshot from ArcMap simply processed in Paint
Actually, the raster you see is the result of mathematical operation ("minus") between DSM and DTM.
I just gave some colour (classified the resulting raster) to visualize the difference in error with pace of 5 cm.
GR
Title: Re: Accuracy (round N)
Post by: George on January 15, 2013, 10:22:26 AM
This is now a sort of finalisation
Upper image is the final one after all the optimizations; below are the old comparisions of DSM and DTM (airborn LIDAR of National service)
Obvious improvement. Correct GCPs, lense calibration "Refined", optimization, XYZ errors pushed down to less than 0.00x ... all these give a good result for natural resources inventory as well as just aerial imagery ...
Title: Re: Accuracy (round N)
Post by: George on January 15, 2013, 10:23:08 AM
...