Once again about DEM and 3D accuracy

[George]

Hi all,
I understand that here were already some discussions about comparing Photoscan and LIDAR data i.e. about checking the accuracy. However, my question is more practical and may be someone of you, guys, Dmitry and Alexey, have done some tests checking PS's accuracy regarding surfaces, 3D stuff, DEM etc.
So the question is what are the practical methods to check, measure and prove PS accuracy at reconstruction of surfaces, elevations etc.
Thanks.
George

[Herman]

Hello George
I have done a number of practical surveys and have compared results with Totalstation and GPS surveys. The total station and GPS survey was based on the same control points used to fix the ground control points , thus the surveys are inter related.
A few spot checks was done as follows. The survey was draped over the cadastral (legal) boundaries captured independently but related to the same co-ordinate system and it is a perfect match. This being verified by seeing the boundary lines intersect the cairn marking the the boundary beacon or with walls and fences perfectly fitting the boundary layout.
Regarding test done by surveying a GCP but not using it to control the the PS project, results are as follows. In theory I understand that accuracy should be equal to XY=2*GSD and Z=4*GSD. I find it rather to be in the range XY=1*GSD and Z=2*GSD. When testing in relation to a arbitrary spot shot taken , not a physically identifiable point, results are  better than  XY=2*GSD and Z=4*GSD.
Basic rules of interpolation should be applied and rather too many GCP's.
Also the shape of the project seems to play a roll , long thin project (corridor mapping) is problematic if approached in the wrong way.
Sampling rate has a definite influence on accuracy , but for your standard 25min flight with 500-600 images ,options "smooth" and "high" produce the results above. I fly with an average GSD of 4-5cm.
Regards
Herman

[Santiago]

Hi,

I'm new to Agisoft and tried the trial version of photoscan pro a few days ago. The report said my error was 0.497 pix and my ground resolution was 0.077 m/pix. I asked the support team to explain the results to me but their explanation didn't clarify my doubts. They told me an error <0.5 was considered to be good, but I want to know what's the error in meters for my project. Or compared to a total station or a RTK GNSS how did I do. Do you know based on what do they say <0.5 is good.

Thanks for any help you can give me.

Best regards,

Santiago

[andyroo]

Regarding test done by surveying a GCP but not using it to control the the PS project, results are as follows. In theory I understand that accuracy should be equal to XY=2*GSD and Z=4*GSD. I find it rather to be in the range XY=1*GSD and Z=2*GSD. When testing in relation to a arbitrary spot shot taken , not a physically identifiable point, results are  better than  XY=2*GSD and Z=4*GSD...

I have had basically the same results as Herman. I fly with a GSD of 10 - 15 cm and a network of ~15 - 20 control points placed around a ~3 km reservoir, and I am generating surfaces that are +/- <25 cm difference over ~90% of the raster cells when compared flight-to-flight and with independent RTK GNSS profiles and aerial and terrestrial LiDAR. I am probably pushing the distance between control points a bit - I do see a little higher error in areas with a longer range between control points - but it is working very well for my purposes (calculating sediment volume).

[George]

Andyroo hi,
you mentioned a reservoir. Is it water?
Do you use lidar?
George

[EGuerra]

Hello Herman, What in your opinion would be the right strategies for corridor mapping? regards
Eduardo

Hello George
I have done a number of practical surveys and have compared results with Totalstation and GPS surveys. The total station and GPS survey was based on the same control points used to fix the ground control points , thus the surveys are inter related.
A few spot checks was done as follows. The survey was draped over the cadastral (legal) boundaries captured independently but related to the same co-ordinate system and it is a perfect match. This being verified by seeing the boundary lines intersect the cairn marking the the boundary beacon or with walls and fences perfectly fitting the boundary layout.
Regarding test done by surveying a GCP but not using it to control the the PS project, results are as follows. In theory I understand that accuracy should be equal to XY=2*GSD and Z=4*GSD. I find it rather to be in the range XY=1*GSD and Z=2*GSD. When testing in relation to a arbitrary spot shot taken , not a physically identifiable point, results are  better than  XY=2*GSD and Z=4*GSD.
Basic rules of interpolation should be applied and rather too many GCP's.
Also the shape of the project seems to play a roll , long thin project (corridor mapping) is problematic if approached in the wrong way.
Sampling rate has a definite influence on accuracy , but for your standard 25min flight with 500-600 images ,options "smooth" and "high" produce the results above. I fly with an average GSD of 4-5cm.
Regards
Herman

[piste]

Hi everybody...
once again: 3d accuracy of point cloud...
I did a test on a river here in italy.
Here is my workflow:

• Import photos
• Mask them (river)
• Align Photos
• Edit > Gradual selection filter (reprojection error: 3 times)
• Create geometry (cloud mode)
• Edit marker
• Import GCP (.txt file)
• optimize point cloud
• create geometry
• create texture
• Export

In Photoscan I obtain a total error of 0.014 m and 0.284 pixels using 14 GCP with known coordinates...
very good I say...(pixel size was of about 3cm)
When I have compared the point cloud exported with that one obtained in the same area
with a terrestrial laserscanner I have found an error in height till 50 cm...
Why this?
How can I read these values of accuracy so?
Thank you

[George]

Hi Piste,

could you share an image of a graph (the one for comparision between LIDAR and your data), please?
Do you mask (river) each individual image? What do you think about closing holes?
Why do you use Gradual selection tool?

[George]

Hi Herman,
thanks a lot for your post!
Almost one month gone since I reached your message. Sorry for that.
Did I get you correctly that you have obtained even better precision than the theoretical one (which is XY=2*GSD and Z=4*GSD)?
Why do you think Smooth will be optimal?
Cheers, George

[Herman]

Hello George

Sorry for replying so late, yes I get results in the order of xy= 1 * GSD & z = 2 * GSD
In the example attached the area was flown at 2.5cm GSD and the results are shown.
Points were tested with conventional survey and heights where within 40mm.
Regarding smooth vs sharp , I think is has got to do with what algorithms are use in
the processing , sharp being more suited to to built up areas.


Hello Piste
In Photoscan I obtain a total error of 0.014 m and 0.284 pixels using 14 GCP with known coordinates...
very good I say...(pixel size was of about 3cm)
When I have compared the point cloud exported with that one obtained in the same area
with a terrestrial laserscanner I have found an error in height till 50 cm...
Why this?
Can be you be assured that both surveys are based on the same height control?
I have done a 1000ha survey and the previous baseline survey was done by lidar,
Comparable results where good , I can forward some images, they are to large to attach,
if you are interested email me at    -  herman@sasurv.com
 
Hello Santiago

The report said my error was 0.497 pix and my ground resolution was 0.077 m/pix. I asked the support team to explain the results to me but their explanation didn't clarify my doubts. They told me an error <0.5 was considered to be good, but I want to know what's the error in meters for my project. Or compared to a total station or a RTK GNSS how did I do. Do you know based on what do they say <0.5 is good.

The 0.497 pix basically indicate that in generating the point cloud and in aligning the photographs
image to image the GCP marked in photo 1 agrees to photo 2 at an average of 1/2 a pixel.
The 0.077m is the average combined xyz error , this can be split in the actual x,y & z error per
point "View errors" tab in "Ground control" view
You can accept this accuracy but results are still based on the number and also position of GCP.
To few GCP and to far apart would have the obvious results.

Hello Eduardo

What in your opinion would be the right strategies for corridor mapping

I dont really know , I have just done a 65km pipeline survey. The results were very bad.
I have noticed that a slim rectangular shape does not process well. As soon as the ratio of
the long side related to the short side get to approx. 0,2 the results are poor.
This can firstly be spotted if the plane of the photo/camera's are not parallel to the bounding
box.
In a slim corridor the area rotates very easily along the length of the corridor.
In the 65 km pipeline the area was shaped in a "C" form and I thought this should give it
strong geometry , but no luck.
We had 380 GCP spaced at a 250m interval, and we opted to cut the corridor into 65 lenghts
of approx.  1.5km in length (with some overlap) and this worked ok.
I don't know really what approach to take, maybe tie strips flown perpendicular to the corridor
might help, this would result in a lot of extra work but might be essential , will experiment with
this in the next project.

[piste]

Hello Herman

Can be you be assured that both surveys are based on the same height control?
I have done a 1000ha survey and the previous baseline survey was done by lidar,
Comparable results where good

Terrestrial Laserscanner survey and UAV's one have been made in two different days...
so GCP wasn't the same...
Both in terrestrial laserscanner and in UAV survey we have surveyd GCPs in RTK mode.
Error in height are at maximum of 2cm...
I will send to you same images in the next days...
Thanks and Happy new year!

Hello George!

could you share an image of a graph (the one for comparision between LIDAR and your data), please?
Do you mask (river) each individual image? What do you think about closing holes?
Why do you use Gradual selection tool?

I have masked the river on every photo because I don't consider points matched on water surface as good points for 3D model reconstruction...
Moreover in terrestrial laserscanner 3D model I have no information about water surface (laser beam crosses water or it presents diffraction phenomena) so, for a best fitting I masked it also in photos...
I have used Gradual selection tool (reprojection error) to avoid sparse points in the air or too far from ground surface (till reprojection error was about 1 and with no more iterations of 3 times...)

[Mohabon]

(till reprojection error was about 1 and with no more iterations of 3 times...)

Hello Piste,

what % of points are rejected using such a strict reprojection error = 1? I believe that at least 10% of total points should be discarded or more. Accordingly with your experience, do you have any matching problem for cutting so many points?

Las question: it is said that FIT Aspect and FIT skew should be both DISABLED when doing this optimization. What do you think is best?

Kind regards,
MoH

[ju523m]

Hi all,

for detailed info on deformation of photogrammetric model, please see:

Kraus, K. : Photogammetry: Geometry from Images and Laser Scans, Band 1, chapter 4.3.6.2., "Deformation of the photogrammetric model", pp 215 ff.

Citation from Page 216:
" It should be clearly mentioned, that these deformations are superimposed on the surface of the model. It is, as if the height reference plane, the xy-plane, of a model free of errors in relative orientation is distorted into an inclined plane, a cylinder, paraboloid and so on."

Please see also Chapter 4.3.4, pp 205 ff for influence of relative orientation and effects such as critical / dangerous cylinder or circle.

I conclude from this, that relative orientation of camera stations against each other does play the keyrole.

In classical stereo-photogrammetry we use metric cameras - their inner orientation is measured and known, which is a precondition for an exact determination of the relative orientation. The relative orientation is further determined by the MEASUREMENT of at least six homologous points - the Gruber points - in the overlapping area of the images, which equals the area of the stereoscopic model.

When we are working with images from non-metric cameras and Agisoft PS, we neither know the exact camera intrinsics, nor do we measure the tiepoints. PS calculates/estimates this for us based on automatically determined image tiepoint candidates, which are not free of errors. PS tries to minimize errors by sophisticated algorithms, but to minimize does not mean to eliminate.

So we should not be too disappointed when PS-results are not as exact as results of much more expensive and time-consuming classical photogrammetry. But we should also not make the mistake to tell our potential clients that we can achieve the same results with our kind of equipment.


Cheers

[jmgc]

Hi All,

I'm dealing with this kind of problems right now.
Herman, did you use a on-board GPS during the pipeline (65km) work?