Cut \ fill plan (volume) for a quarry

[Vavania]

Hello everybody! I'm trying to use photogrammetry to support our quarry.
I use Photoscan and Golder Surfer to work with exported DEM.
I put some markers around the quarry (200x400 m), and used them in both Photoscan models.
I tried different methods for creating dense clouds.
Everyhing went well until I tried to create a cut/fill plans for the quarry (200x400 m) for a period of time.

The problem is in slightly shifting the slopes of the quarry so, if they are steep, even a small offset leads to
large vertical difference. Thus, on the cut \ fill plan, almost all areas of the quarry looks modified but it isn't true.

Has anyone encountered this problem and what rules should be recommended to fix it?   

[SAV]

Hi Vavania,

I assume you have acquired the imagery for photogrammetric processing by drone/UAV.
Did you use the same UAV/camera and did you shoot the pictures at the same resolution? Were you flying at the same altitude above ground?

All these factors influence the GSD (ground sampling distance) as well as the accuracy of the model. For example, if you compare a low resolution model to a high resolution model of exactly the same area, you will get a volumetric difference even though nothing has changed. Additionally, the X/Y/Z accuracy is typically a function of your GSD. If highly accurate (cm-accuracy) ground control points are used, then one can expect a horizontal (X/Y) accuracy of 1-2 x GSD and a vertical (Z) accuracy of 2-4 x GSD.

Additional factors that could help to explain the differences between your surveys:
1) Differences in processing settings (e.g., accuracy settings for markers and camera stations in Photoscan, quality of dense point cloud, aggressive or mild filtering on dense point cloud).
2) Reflective areas such as water puddles which cannot be accurately reconstructed
3) '3D noise' due to dust in the air (which is common in quarries)
4) Differences in image quality (e.g., one survey might have more blurry images, which influences the reconstruction quality)

As you can see, there are many things to consider. But here is a workflow that might help you:
1. Export the before & after dense point clouds from Photoscan (as *.las files) and open them in CloudCompare (www.cloudcompare.org, free and open-source).

2. Align both dense point clouds (known as registration in CloudCompare), either by using ICP on the point clouds or by manually picking point pairs in both clouds. See this for details: http://www.cloudcompare.org/doc/wiki/index.php?title=Alignment_and_Registration

3. Compute the volumetric difference (cut/fill). By adjusting the grid size you might be able to mitigate some volumetric errors related to small misalignments of the clouds. All details here: http://www.cloudcompare.org/doc/wiki/index.php?title=2.5D_Volume

All the best.

Regards,
SAV

[Vavania]

Thank you SAV very much for very constructive reply!

Well,  it was the same camera and pictures were made in the same automatic mission (same route, same height).

1) I noted however a bit strange thing - EXIF data from the first flight has 17 m altitude and 180 m for the second while the real  flight height was 100 m in both cases (above ground of course). Photoscan estimated height as 170 and 175 m, but I used markers altitude to calculate dense clouds.
 
2) I used the same processing settings, or more precisely I can use only the range lowest-medium quality as the file very large with better quality. Px size is 10-20 cm in this case. We don't need better acuracy. I don't find any difference between depth filtering options (disable - agressive) for my cases and used the same.

3) Water or dust are not the case. However the surface covered with thin snow. It didn't reflect as it was cloudy, but maybe it was a bit dark (attached).

4) I supplemented vertical pictures with pictures made at some angle as a panorama. I supposed that it helped Photoscan to evaluate the real depth. However this additional pictures were made from different positions in both cases. Maybe I was wrong.

I will try you recommendation about CloudCompare.
It is quite important for us to workout some rules and limits before recommendation using this technology on all our quarries.

[SAV]

Hi Vavania,

ad 1) If you used GCPs as the main reference, then it should be fine. For images where the flight altitude is 17m, you should uncheck all the cameras in the reference pane in order to ignore them. What was the side overlap and forward overlap that you have chosen? What app and UAV did you use?

ad 2) Note that you should choose quality HIGH for the photo alignment step, then insert GCPs and then run OPTIMISATION (magic wand icon in reference pane). I suggest to use quality MEDIUM for your dense point cloud and then directly compute the DEM from it (building a mesh is not necessary). From your DEM you can then extract contours.

ad 3) The thin snow film will definitely be picked up and explains some of your differences.

ad 4) It is beneficial to add a few pictures at an oblique angle. But even if you just shoot nadir (straight down) it should work fine in your case. Just make sure that you have GCPs at different altitudes/levels. They should be horizontally AND vertically distributed over the area of interest. In your case, a few at the bottom of the pit/quarry, a few at the upper edges and a few in-between on benches.

Regarding CloudCompare. It is a really powerful software package. If you have any questions about it, feel free to get in touch with me. Please also consider this post if you decide to use it professionally:
http://www.agisoft.com/forum/index.php?action=post;topic=8402.0;last_msg=40112

Regards,
SAV

[Vavania]

Hi SAV!

1) I did just what you say - unchecked cameras and checked GCP in both cases. Side overlap was  65% and front overlap - 75%. It was DJI Mavic Pro under control of DroneDeploy. I just wondered why EXIF were so differenet in both cases.

2) I don't remember which quality was for photo aligning and will repeat the step. I used optimisation also.

3) It isn't favorable conditions, but it is reality - snow in winter, grass in summer.

4) Unfortunately it is not easy to keep GCP on the active quarry bottom and benches for a long time. That is is why I had five GSP around the pit and three GSP on the bottom.

I have tried CloudCompare, but so far only confirmed that clouds of the quarry are different with main problems on the slopes.

[SAV]

Hi Vavania,

ad 1) I suggest to use a slightly higher side overlap (75%) and forward overlap (80%) next time. Instead of Dronedeploy you could use MapPilo or DJI GS PRO instead.

ad 2) Note that your before/after point clouds will only be accurately aligned if you surveyed your ground control points at high accuracy (i.e., using and RTK GPS). If that's not the case, you could align your point clouds in CloudCompare using ICP before computing the cut/fill volumes. Details here: http://www.cloudcompare.org/doc/wiki/index.php?title=ICP

ad 3) Yep, I understand that this is not a controlled / laboratory environment but the real world  . Maybe reduce the resolution of your DEMs to 10-20cm (which should still be OK for volume estimate) to reduce some of the 'high resolution noise/differences'.

ad 4) Maybe you could get paint some large GCPs on pit faces that are not going to be mined anytime soon, then survey them ones and use them for all your other surveys. Or place some permanent GCPs, for example you could have some DIY concrete GCPs distributed around and inside your quarry/pit

If you can/want to share your point clouds or imagery with me, I can have a look at them to see if there is any obvious issue.

Regards,
SAV

[Vavania]

Hi SAV

1) I'll try 75% overlap. I tried several flight control app. DJI GS PRO can make only 99 pictures in a mission and works only with iPad. Map Pilot is not free and I don't know why I should by it if I satisfied with DroneDeploy.

2) I will try CloudCompare

3) Usually px size in the DEM clouds is 10-20 cm.

4) I'll try to create some points around the quarry next time

Thanks, but I believe it will be waste of your time to look on my clouds so far. I will try to do my best to get better shots first. My be you have a good practice examples - height of flight, quantity of markers per area and etc?

Meanwhile there are some points which I can't explain. For example on the attached picture there are markers 8B and 6B. Resulted points clouds corresponds to 8B height - about 80 m, but doesn't correspond to 6B height - about 100 m instead of 81.5 m. (I did camera optimization and calculated density cloud with checked markers and unchecked cameras - twice)

[SAV]

Hi Vavania,

ad 1) DJI GS Pro is not limited to 99 pictures per mission.

ad 2) :-) You won't regret it.


When it comes to ground control points, not only their number but also their location is important. I suggest to use between 8 and 15 GCPs and distribute them throughout the area. Employing a larger number of GCPs allows you to use some as check points (instead of control points), which can subsequently be used to assess the quality of your model. For example, when using 10 GCPs I would then use 7 of them as control points and 3 of them as check points. Has worked well for me.

You can probably fly somewhere between 40-120m above the ground (120m is the legal limit where I am operating). At 40m your GSD is 1.2cm/pixel, at 100m it is 3.1cm/pixel. As you can see, no problem to generate a 10-20cm resolution DEM from such imagery.

Regarding your 6B and 8B GCP issue. Did you compute a new dense point cloud, new DEM and new contour lines after you optimised your camera alignment? I also cannot see that calculated RMS for you control points in the screenshot... has it been computed?

Regards,
SAV

[Vavania]

Hi SAV, I'm back from the site...

1) DJI GS Pro has several options to get pictures (Hover at point \ Time interval \ Distance interval). If I'm not mistaken in the first case I got message about  limit 99 points. Then I changed regime to distance and everything was ok.

2) I tried my best  to improve quality of shooting for control of quarry volume. I put around and inside quarry some boards as GCP. We don't have RTK GPS so I took coordinates for the GCP from the first flight and used them in other for markers. I took GCP elevation from our surface survey. I have several shootings in month intervals, but I believe that example with day interval is more demonstrative because there weren't  real changes on the next day.

Both shootings were made with the same mission at high of 100 m and overlapping 75%. After aligning pictures with high quality in Agisoft markers were defined with the same static coordinates for both cases, cameras were optimizated and unchecked. Markers were checked and density cloud was constructed with medium quality.

On the attached cut\fill picture:
black contour - quarry limit
black points - GCP
white area - difference between two days less than +\-0.25 m
light pink \ light blue - difference +\- 0.25-0.5 m
pink \ blue - difference +\- 0.5 - 1 m
dark blue - difference  more than 1 m

It is visible that difference at the most area of the quarry is more than 0.25 m and often more than 1 m which is significant. It looks like in two adjacent days quarry "had" differenet shape which means that calculated volumes for each days are not trustfull. It is pity and I don't believe that CloudCompare can help: it is not shifting, it is not scaling.



 

[GPC]

Would you be willing to share the data? As a professional surveyor, we use this same technique every day to perform the same type of projects and would be willing to try our hand at it.

[Vavania]

Sure, as pics - ~3Gb each mission (I have three so far), as las files (~1Gb each). Just let me know how to do that. I'm working out a solution for our quarries and would very appriciate for any help. Next mission is planned by the end of the month.

[GPC]

I need the pics. Can you provide a Dropbox link? PM me the link or just post here. If not I can provide a directory for upload.

[Vavania]

Done

[GPC]

I'll review and be in touch privately. Your request for nondisclosure will be respected.

[GPC]

The simple answer is you don't have sufficiently spaced control points. This is a very large site and the distance between points 2 and 4 is over 600 meters. You need to establish a grid across the site and  have targets that are used first a control points (triangles) and points that are initially used a just check points. Do a first adjustment where you fix the triangles only, and see how the circles are predicted and fit in agisoft.. then further move to fix those points. I think the data looks good, the control just needs more points to really "tie" down  the site.