Holes in point cloud and distortion around trees

[mfrancia]

Hi everyone,
I'm currently working on a dataset of UAV images of avocado trees and I want to generate an orthomosaic, but I'm getting some problems when building it. The main problem is that there're holes in the point cloud around the trees, that leads to some distortion in the DEM, Mesh and orthomosaics. I've already tried some solutions such as classify ground points, build the orthomosaic based on a mesh, use a lower quality in the alignment of the images (that generates more points around the trees, but not enough and reduces significantly the orthomosaic quality) and use more quantity of tie points, but none of them has worked as expected.

I believe the problem might be due to the movement of the trees caused by the wind, the complexity of the branches, and the angle of the camera, but I'm not sure how to solve it.

I've put the images of tie points, point cloud, 3d model and orthomosaic in the attachments. Hope it helps to understand the problem.
Thank you in advance!

By the way, the photos that I'm using have this characteristics:

Type of data:   RGB Bird-view images
Unmanned Aerial Vehicle DJI Phantom Pro 4:
Max H-Speed:72 Km/h
Max V-speed:35 km/h
Weight 1.5 Kg
Action radius 500 m
Autonomy 20-30 Min
Radio control Frequency 2.4 GHz

RGB Camera:
Sensor CMOS
Lens: 8.8 mm/24 mm
FOV: 84°
Resolution: 5472 × 3648
Supported formats: JPG, PNG and RAW
Operating Temperature Range: 0° to 40°C

[dpitman]

What is the desired goal or data from your work?

What is the overlap setting applied to your image capture distance?

[mfrancia]

The orthomosaic that I'm creating is for training a neural network for object segmentation.
The overlap of the images is 75%.

[dpitman]

Try bumping your overlap up a little and try a slightly acute camera angle.  You can also employ a cross grid capture.
These things will help capture more detail in the areas below the edge of the tree canopies.  If it is breezy when you are photographing and the branches are moving. There is no magic setting that will fix that.

It is odd that the open areas between the trees are not rendering well though. Can you share a few of the images from the flight?

[mfrancia]

I agree that adjusting the overlap and camera angle could be a great solution. However, I'm not the creator of the dataset and I have no control over the image acquisition process, so I'm trying to improve tthe result with just the images that I have.
I've put the photos as attachments in this reply, but the full dataset is on this link of ScienceDirect: https://www.sciencedirect.com/science/article/pii/S2352340922009428

[dpitman]

I'm guessing the images are being down-sampled upon upload to the forum, and that they are actually 20mp (5472 × 3648) image resolution? The attached versions look a little soft.  If the actual images look the same,  you might try adding a bit of sharpening before processing in order to give Metashape more "edges" to work with in the bare earth areas.  Use an editor that does not disturb the exif data. 

I'm not certain it will improve the result, but it's worth a try.

Are you using High Quality and Mild Depth Filtering in the point cloud generation?

[dpitman]

I downloaded the images to take a look. 

Firstly, the author seems to have used a reduced resolution (4864 x 3648) for the image capture rather than the recommended native full sensor resolution of 5472 × 3648. This may be because some earlier flight control software defaulted to 4:3 rather than the native 3:2 aspect ratio.

Second.  As suspected, the overlap is insufficient for good results.  I've attached Metashape's overlap report along with a report from another project showing what a good report looks like.

Third, the dark shade under the mature trees, and the reflective poly sheathing under the young trees is causing the problems directly under the trees.  For photographing water, or reflective surfaces, to get the best results, light overcast is preferred.

For the images that you have to work with, I think Metashape is doing a decent job.  If the result is not going to work for your purpose, you may need to seek out a superior image set.

[dpitman]

another image

[mfrancia]

Oh I see, I was hoping that the problem was mine haha
Thank you very much for your help, I've learned a lot! I'll start searching other datasets now, but when I started this project I didn't find any. Hope that there were published more now.