Model Generation - Holes Walls Problem - Camera position / Depth Maps (?)

[marcion1408]

Hey!

Can anyone advise me, please?

I have a set of photos of a shipwreck.
The exterior was no problem - the model generated just fine.

Then I started working on the interior, and that’s when things went wrong.
The walls aren’t generating properly.

Visibility was perfect, lighting was good, and the depth maps look fine.
The photos were taken with a GoPro 7 using a Domeport, and the camera calibration is set to Fisheye.

I've tried to clean tie points, etc. no luck.

When I generate a model only using exterior photos, or only using interior photos - it works perfectly.
The problem appears only when I try to build a model using both inside and outside photos.
Metashape seems to have issues with wall thickness - it tries to blend both sides and ends up creating holes, even though the depth maps on both sides look fine.

I also tried the tweak from the guide "How to improve the modeling of thin objects in Metashape", but it didn’t help - the results looked just as bad.

[Mariusz_M]

I have also had this kind of problems. It seems like Metashape has a problem with the thickness of walls even if depth maps on both sides are ok.


Alexey, perhaps this can be solved using some tweaks? Perhaps it is possible to use some kind of combination of max_neighbours and filtering tweaks? Or perhaps there is a tweak that will increase wall thickness somehow or prioritise closing holes in walls over creating them where the software is not sure?

[Mariusz_M]

In the example Marcin has shown there is a boat hull photographed from both sides - outside and inside.

Depth maps from both sides show there is a surface, but the result of meshing has holes. It is probably because there is a small misalignment of the photographs taken inside, where is darker. The curvature of the inside is most likely a bit different than the curvature on the ouside, so depth maps on one side partially intersect with depth maps from the opposing side creating holes in the areas of intersection.

What we need to fix it is the algorithm prioritising creating a surface without holes. So in situation like this we have a group of depth maps from the outside that clearly show a surface and another group of depth maps facing roughly the opposite direction also showing a surface. For the algorithm this should be an indication there must be a surface there, not holes. So the way I see it work is:

1. The algorithm sees a group of depth maps facing roughly the same direction and together creating a surface.
2. It also sees a group of depth maps roughly facing the opposite direction also creating a surface together. (The opposite direction can be defined as a range of angels like 150 - 220 degrees, to make sure the fix is not used everywhere in the model, but only in the right areas of it)
3. It determines there must be a surface without holes if depth maps on both sides show a surface without holes.
4. It establishes which side is more important. I would say the one that has either more depth maps or depth maps with a higher level of confidence.
5. Then in detects the areas where depth maps from the prioritised side intersect with depth maps from the less important side.
6. In those areas only the depth maps from the prioritised side are taken into consideration for creating a surface and the parts of depth maps from less important side that intersect are ignored.
(7. Or better, not completely igronred, but later projected on top of the mesh created based on the prioritised side, to still add some small details similarily to "refine mesh". But they shoud not cancel out the depth maps from the opposite side.)

This way a continuous surface is always created based on more important side and still has plenty of details also on the other side, just maybe not in the areas where less important depth maps intersected with more important depth maps. But in the end it is better to have less details in some areas on one side than holes on both sides. Furthermore, if depth maps do not intersect anywhere, then a surface is created anyway with full details on both sides, because no part of less important side gets ignored.


This kind of algorithm modification could be manually activated only when needed, as an additional option in model generation called something like "fix holes in thin walls" or "prioritise surfaces over holes".

[marcion1408]

bump.

[Alexey Pasumansky]

Hello!

It would be helpful to get an example (even the sub-set of the main project) that we can use to reproduce the issue during our internal tests. So if possible, please share the image set and Metashape project with the alignment results to support@agisoft.com as a download link via any convenient file transfer service.

Most likely such problems are caused by inaccurate alignment of camera sub-sets from both sides (or between interior and exterior cameras), which misplaces the possible surfaces of both sides making some intersections between them.

[marcion1408]

Hi Alexey ! Thank you for your answer !

Sure, I will upload that for you within few days, as I'm processing a different project at the moment.

Most likely on sunday or monday.


But if that is caused by inaccurate alignment, is there anything that can be done? The whole project is about 7k photos, so any manual actions are not really an option - is there any way to handle this automatically?

[Mariusz_M]

Alexey. Do you see the solution I proposed as doable? It would help a lot with underwater photogrammetry. It is one of the common problems, ,because under water there are always some smal misalignments and as long as walls are thick this does not affect the model, but in most wrecks the corroded steel is very thin.