Scanning objects in an uncontrolled environment

[chris741]

For context the object I'm trying to scan is a signpost
The model I'm trying to scan has a number of large thin parts

What is the best way to process objects in an outdoor uncontrolled environment?
For example if there are clouds, cars, people or trees that are moving but not part of the object I'm scanning why does masking them out produce significantly worse results for camera alignment?

Does using a point cloud improve quality?

Does depth filtering have an impact of thin objects?

Also for objects that are large but thin, how can I better get a model generated?

Should I use depth maps or point cloud to generate the model?

How di I do manual alignment, I cant seem to find the option for manual markers

[James]

In general you need to try to fill the frame of each image with the object you're scanning.

For 'thin' objects this means getting closer to them, and therefore taking a lot more photos.

If the thin object doesn't take up much of the image Metashape won't know that it's 'important' and it will just align based on what else it can see.

If you mask out everything in the background and only leave a small part of the frame unmasked (the thin object) then the angles to matching points won't be well enough distributed to give a good 3D position estimation (alignment), or there just might not be enough image unmasked to even find matching points.

If you do fill the frame of each image with your object but it has no texture, then it also won't find many useful matching points.

So you may need to include some background in the images to help it align. Don't worry too much about cars, people or clouds - Metashape won't find many points there anyway, and only worry about masking moving trees once you've got a good alignment and want to make it better.

If you get closer to your object then background objects will start to go out of focus anyway, assuming your camera is not fixed focus like a GoPro etc.

Using an external point cloud as a data source is only applicable to the Pro version, and won't really help photos that can't align by themselves. Markers are also a Pro only feature, and manually aligned photos that wouldn't align by themselves will still not be very useful when you come to the meshing stage.

Trying to force photos to align that just won't align by themselves is a thankless task even with the Pro version and it's almost always necessary to return and get more well focussed photos with plenty of the object in them and lots and lots of overlap.

Yes depth filtering will have more impact on thin objects, but I've never found that changing that setting helped if I was getting bad results with the default.

I would almost always use depth maps method for mesh generation and skip the dense point cloud generation step.

For a ~2m fence 'post' for example, I would take ~30 photos in a circle around it, all facing in towards it, and down to include the bottom half and some of the ground. Take another ring of 30 photos looking more horizontally to get the middle and some background. Then hold the camera high above your head so you can capture the top half without pointing the camera to the sky. The sky is no use, cloudy or clear!

That should give you 90 photos that align ok, but depending on the texture of the post and the distance to it, might still not give you a nice clean model. If it's a shiny or clean plain post then it will be very hard, but if it's a nice old wooden one then you can get better results by just getting closer and taking more photos.

[chris741]

I took video with a 360 camera. I found the more equirectangular images I extracted the better the result
At 700 images the resulting point cloud and mesh looks good, but the texture is slightly off.
If only I could somehow identify which images are causing the bad texture and not use them or re-align them it would be good

Would adding frame images help?

I tried extracting frame images from the 360 video but that didnt work too well

When creating a point cloud I can see a set of points offset.
I can only seem to select and filter using the tie points.
I did that and removed the alignment

What is the next process, optimize or re-align? I can see where there is a group of points incorrectly aligned

[James]

For improving the texture you could try

• estimate image quality - this will give a score to each image according to how sharp it is, which you can use to filter out blurry images. if you perform quality estimation, you can sort by quality in the details view of the photos pane and disable the n% worst images.
• add frame images - if you use a normal camera (rather than extract from your existing 360 images) to get higher quality coverage of the object, then align these images with your current dataset, you can leave them disabled for mesh generation, and the enable them and disable the 360 images for texture generation.

i'm a bit dubious about 360 cameras, because i'm sure it's physically impossible for them to truly have a single 'spherical centre' as they normally work by having two sensors looking opposite ways, just very close together. they might be stitched very nicely, but i don't think the geometry quite works as a perfect spherical image, so i wouldn't expect the alignment to work very well.

once the images are converted to equirectangular there's probably not much you can do about that, but if you are able to access the individual raw fisheye images before they are stitched then you may get a better alignment. if they are provided as a pair of fisheyes in the same image then make a copy of each image and mask out the left in one set and the right in the other.

if you're stuck with equirectangular video frames then you may have luck with identifying where the stitch line is - it should be a vertical seam down the middle, or pair of vertical seams somewhere else (i guess!) and you can take the same approach as above by copying the images and masking out either side of the seam(s) so metashape is able to ignore the incorrect geometry caused by assuming a single spherical sensor when it was really two close together, so long as the image wasn't too warped in the process of stitching.

never tried it and all totally hypothetical, and probably a dangerous rabbit hole to get lost down.

better solution would be to use a normal camera for scanning 'objects' whereas 360 cameras are useful for enclosed spaces, based on the simple rule of thumb that the thing you are scanning should fill the image. in an enclosed or interior space the 360 camera is great as the image is full of what you are trying to scan, because it's everything around you.