"Warped" vegetation in Orthomosaic

[BobvdMeij]

Dear all,

The attached screendump displays a section of an Orthomosaic we produced recently, it depicts a section of a vegetated water barrier. Obviously, the vegetated sections in particular are faced with severe levels of 'warping'. Although we have mapped various vegetated regions before, the level of warping has rarely been this severe. I am keen to find out any tips or tricks to reduce such artifacts and enhance our output! The workflow we used was as follows:

Image Acquisition
UAV: DJI Inspire
Camera: Zenmuse X5
Altitude: 40m AGL
Flightspeed: 4.3m/s
Overlap: 75%/75%

Agisoft Image Alignment
Accuracy: High
Preselection: Generic & Reference
Key- / Tiepoint limit: 40.000 / 4.000

Agisoft Dense Cloud
Quality: High
Depth Filtering: Aggressive

Agisoft Build DEM
Source data: Dense cloud
Interpolation: Enabled
Pixelsize: 2cm (default)

Agisoft Build Orthomosaic
Source data: DEM
Hole filling: Enabled
Pixelsize: 1cm (default)

Unfortunately, we are unable to pay another visit to the location and redo our flights. Therefore, I am ideally looking to optimize our output through adjusting the processing workflow only. I am considering various options, such as:

1. Disable Interpolation during DEM generation to reduce weird artifacts in the DEM, particulary around hard to match vegetation, on which the Orthomosaic is eventually based. Hole filling for Orthomosaic generation remains enabled.
2. Increase Pixelsize during DEM generation to 'smooth' any small scale elevation changes around hard to match vegetation. Interpolation (DEM generation) and Hole filling (Orthomosaic generation) remain enabled.
3. Produce a Mesh rather than a DEM and use the Mesh as the surface for the Orthomosaic to be build upon. I am still unsure whether to produce a mesh using the Sparse Cloud (loss of detail) or the Dense Cloud (more plausible artifacts).

I wlll likely give each of the above a try in the days to come, but I nonetheless appreciate any help or advice. Does any of the above 'solutions' make sense? Are there other solutions out there which I might give a try, or should I tweak any of the above workflows? More importantly, any hints on why a particular workflow works better than others in instances as these are much appreciated to we can adjust our procedures accordingly in the future!

Thanks in advance.

[Jeremiah_ROWE]

See if you can create a good ground classification of the points in your dense cloud, then make a DEM or mesh just from the ground class. Then build an orthomosaic from the ground-based DEM or mesh.

[BobvdMeij]

See if you can create a good ground classification of the points in your dense cloud, then make a DEM or mesh just from the ground class. Then build an orthomosaic from the ground-based DEM or mesh.

I sure will give that a try! Although I genuinely doubt whether that approach would work, considering well over 95% of our surface is vegetated.

[Jeremiah_ROWE]

Sounds like a tough cookie!

[SAV]

Hi BobvdMeij,

It is generally difficult to produce good photogrammetric models from vegetation such as bushes & trees. One issue is that they are non-stationary (moving in the wind), hence SfM algorithms have troubles to find high quality tie points. Additionally, bushes and trees have quite complex geometries, which means you would need to use a very high image overlap (80-90%). For best possible results, you would even fly a grid layout (2 surveys that are perpendicular to each other) with the camera pointing down at about 70-75 degrees to shoot oblique imagery.

One problem in your case is the image overlap of 75%. You were flying at only 40m and bushes/trees are sticking out of the ground, hence the image overlap in such areas will be even smaller, which is a problem for the scene reconstruction and leading to the low quality results.

However, here is what you could do to get a nice looking (but probably not super-accurate) orthophotograph:
  1 - Open the orthophotograph in PhotoScan
  2 - Draw a polygon around the area where warping appears (pentagon icon in main toolbar).
  3 - Right click within the polygon and choose ASSIGN IMAGES
  4 - Photoscan will then show you a list of pictures that cover this particular area. Pick the one that 'looks best'.
DONE.

When it comes to the DEM, it is (to say it in Jeremiah_ROWE's words) a tough cookie.

One more thing. Based on the screenshot you shared I can see that you haven't set your White Balance (WB) properly. You should choose SUNNY when the sun is shining and CLOUDY when there are clouds in the sky. The automatic WB leads to these differences in color in your pictures. You can change WB settings in the flight planning app that you use for your DJI Inspire.

All the best.

Regards,
SAV

[Jeremiah_ROWE]

"Assign Images" per shape is a great function! You will find yourself using it to help give a consistent "lean" to the features in your image. Sometimes the automated image stitching/selection will choose less than desirable images.

Keep in mind that the resolution you build the ortho from will factor into things. If your client only wants an ortho with a GSD of 1/2 ft (0.15m) pixels, building it at scale will automatically smooth out some of the inconsistencies as there is less precise image matching needed than if it were a full resolution (more like 1cm pixels) render.

[BobvdMeij]

Hi BobvdMeij,

It is generally difficult to produce good photogrammetric models from vegetation such as bushes & trees. One issue is that they are non-stationary (moving in the wind), hence SfM algorithms have troubles to find high quality tie points. Additionally, bushes and trees have quite complex geometries, which means you would need to use a very high image overlap (80-90%). For best possible results, you would even fly a grid layout (2 surveys that are perpendicular to each other) with the camera pointing down at about 70-75 degrees to shoot oblique imagery.

One problem in your case is the image overlap of 75%. You were flying at only 40m and bushes/trees are sticking out of the ground, hence the image overlap in such areas will be even smaller, which is a problem for the scene reconstruction and leading to the low quality results.

However, here is what you could do to get a nice looking (but probably not super-accurate) orthophotograph:
  1 - Open the orthophotograph in PhotoScan
  2 - Draw a polygon around the area where warping appears (pentagon icon in main toolbar).
  3 - Right click within the polygon and choose ASSIGN IMAGES
  4 - Photoscan will then show you a list of pictures that cover this particular area. Pick the one that 'looks best'.
DONE.

When it comes to the DEM, it is (to say it in Jeremiah_ROWE's words) a tough cookie.

One more thing. Based on the screenshot you shared I can see that you haven't set your White Balance (WB) properly. You should choose SUNNY when the sun is shining and CLOUDY when there are clouds in the sky. The automatic WB leads to these differences in color in your pictures. You can change WB settings in the flight planning app that you use for your DJI Inspire.

All the best.

Regards,
SAV

Thanks for your extensive reply, SAV! Particularly with respect to the white balancing. We had been observing similar coloring patterns in previous projects and kept on wondering how we might solve the issue. We considered lowering of the flight speed, or even stop the aircraft for each image capture, but I reckon your proposed solution may just to the job as well, if not better!

As far as the Assign Images solution is concerned, I was already well aware of that option. Unfortunately the artifacts are so vast that that workaround does not seem to work properly as pretty much every squared decimeter faces sóme artifacts.

That being said, I have just completed an Orthomosaic using the Mesh rather than the DEM for the underlying surface. Guess what, it worked! I would say 99% of the artifacts are now gone. The only exception remains a corn field, although this might be explained by the complex geometry of corn and it being positioned on the area's edge and resultantly lower image overlap in that particular region. Then again, the corn field was not part of our study so we can live with the output. It is worth noting that the Mesh was generated using the Sparse Cloud, a Mesh based on the Dense Cloud (even using Low settings) resulted in most artifacts still remaining in the scene.

All there is to it now is to see how the planimetric accuracy was affected. I'm afraid that may have been impacted significantly by basing the Orthomosaic on the SparseCloud indirectly.

[SAV]

Hi BobvdMeij,

Note that by using a mesh derived from the sparse point cloud you are NOT generating an orthophotograph or orthomosaic anymore, but a 'large aerial photograph'.
Why? In order to create a real orthophotograph/orthomosaic, you need a decent dense point cloud/DEM/mesh because it is used to orthorectify/undistort your images. In your case your pictures have cm resolution but the mesh for elevation information has meter (or more) resolution, hence it is impossible to appropriately correct for distortion. Your 'orthophotograph' might look good now, but cannot/shouldn't be used to measure true distances or accurate locations. Pretty pictures are not always good pictures 

Please be sure to convey this message to your client.

Regards,
SAV

[BobvdMeij]

Hi BobvdMeij,

Note that by using a mesh derived from the sparse point cloud you are NOT generating an orthophotograph or orthomosaic anymore, but a 'large aerial photograph'.
Why? In order to create a real orthophotograph/orthomosaic, you need a decent dense point cloud/DEM/mesh because it is used to orthorectify/undistort your images. In your case your pictures have cm resolution but the mesh for elevation information has meter (or more) resolution, hence it is impossible to appropriately correct for distortion. Your 'orthophotograph' might look good now, but cannot/shouldn't be used to measure true distances or accurate locations. Pretty pictures are not always good pictures 

Please be sure to convey this message to your client.

Regards,
SAV

Thanks for that wake-up call SAV, you are absolutely right! We will likely deliver both the (DEM-based) true Orthomosaic and the (Sparse Mesh-based) 'Orthotograph', each accompanied with a dedicated quality report stating the planimetric accuracies based on RTK checkpoints. The primal goal of the output is not so much to measure or quantify, but I reckon a sensible disclaimer and plausible explanation therefore is the least one could ask for.

Until then we will keep try to optimize the output and minimize artifacts, hopefully resulting in an 'as good as it gets' True Orthomosaic based on the DEM or Dense Cloud Mesh!

[stihl]

Keep in mind that the resolution you build the ortho from will factor into things. If your client only wants an ortho with a GSD of 1/2 ft (0.15m) pixels, building it at scale will automatically smooth out some of the inconsistencies as there is less precise image matching needed than if it were a full resolution (more like 1cm pixels) render.

I too use this technique. If my client specifically needs a True Orthomosaic, I'll offer it at 5 cm and collect the data at 1.5 cm GSD. This will ensure an aesthetically pleasing True Ortho.

[BobvdMeij]

The above does indeed make sense, hence I suggested to create a DEM at 'only' 5cm or 10cm resolution and drape the Ortho thereover.

However, it seems you are suggesting to actually adjust the cell size of the Ortho rather than the DEM, or both? I ideally like to preserve the 1cm GSD of my images for the Ortho due to our client wishing to be able to distill relatively small elements that will likely lose out once I export the Ortho itself at only 5cm or 10cm.

[stihl]

That's correct.
You likely would not have to downsample it from 1.5 cm to 5 cm. Downsampling it to 2 or 3 cm will already make a positive change compared to exporting the True Ortho on the GSD resolution.

In the situation where it's important to keep the resolution I'd suggest creating a  non True Ortho and manually selecting the best nadir images for your objects (buildings, vegetation). This will result in a different scale of the objects compared to the True Ortho but at least it will be artifact free. It's a difficult tr

[SAV]

Hi again BobvdMeij,

Nothing wrong using the data as long as you & your client are both aware of its limitations/shortcomings.

Regarding the quality report. The total error mentioned in it might actually not mean much because it is based on the sparse point cloud/tie points & control points. However, your 'orthophoto' is probably less accurate because it hasn't been properly orthorectified due to data limitations. For relatively flat areas this should not be a big problem, but for 'elevated areas' and objects that 'stick out of the ground' (= areas with lower image overlap and less accurate reconstruction) the error will be higher than mentioned in the processing report.

All the best.

Regards,
SAV

Hi BobvdMeij,

Note that by using a mesh derived from the sparse point cloud you are NOT generating an orthophotograph or orthomosaic anymore, but a 'large aerial photograph'.
Why? In order to create a real orthophotograph/orthomosaic, you need a decent dense point cloud/DEM/mesh because it is used to orthorectify/undistort your images. In your case your pictures have cm resolution but the mesh for elevation information has meter (or more) resolution, hence it is impossible to appropriately correct for distortion. Your 'orthophotograph' might look good now, but cannot/shouldn't be used to measure true distances or accurate locations. Pretty pictures are not always good pictures 

Please be sure to convey this message to your client.

Regards,
SAV

Thanks for that wake-up call SAV, you are absolutely right! We will likely deliver both the (DEM-based) true Orthomosaic and the (Sparse Mesh-based) 'Orthotograph', each accompanied with a dedicated quality report stating the planimetric accuracies based on RTK checkpoints. The primal goal of the output is not so much to measure or quantify, but I reckon a sensible disclaimer and plausible explanation therefore is the least one could ask for.

Until then we will keep try to optimize the output and minimize artifacts, hopefully resulting in an 'as good as it gets' True Orthomosaic based on the DEM or Dense Cloud Mesh!

[BobvdMeij]

Thanks SAV, once again, for your elaboration.

For what it's worth, we hardly ever use the Error figures provided by Agisoft itself. Although we have found it comes pretty close every now and then, we always apply a seperate and standalone verification of our CP accuracy using a different software package.

I'd like to point out that we have just witnessed a substantial improvement of our Orthomosaic, regardless of it still being draped over our 2cm DEM. Initially we ran the full workflow using the images alone, just to quickly see how things played out. We have just now completed the same workflow but this time incorporating our GCPs during Alignment and Camera Optimization. Without changing any of the parameters our Orthomosaic now is as sharp as ever and one has a tricky time to find any artifacts in areas that previously were full of them!

[SAV]

No worries, BobvdMeij.

Seems like you guys are under control :-)

If you have accurate GCPs, then ALWAYS use them for image alignment.
I would have not expected such dramatic improvements though. Good to know. Thanks for sharing.

Regards,
SAV