Generate masks from model issue in 2.2

[michaelRRI]

Hey all,
I have noticed some problems with the "generate masks from model" feature. I have been using this for years flawlessly and now I get some issues.

I have been running it on some larger projects and have noticed some key changes. 1. It is a lot slower and slows down on some photos. 2. It tends to create an all black mask on some frames when it shouldn't. 3. On some photos, it looks like "tearing" type shapes that are not constrained by the model are showing up in the mask. These seem to get worse further through the list of photos in the project so I suspect a memory issue. I opened an old large project from before 2.2 release which did have masks made correctly and re-running it now fails.

Also, when running a batch on a project with MANY small chunks (100+ with 30-40 photos each), I also notice some chunks as if they are clipping the object as one would expect with the mask defocus areas. This only happens on some chunks. When I run the generation again just on these chunks alone, they work.

Has something drastically changed under the hood in regards to generating masks? Let me know if I can provide some samples to help diagnose as I am going to have to downgrade back to 2.1 to get a few projects done.

[vineg]

Hi, Michael!

Thank you very much for the detailed description. It helped a lot to investigate this issue.
Clipping issue appeared after addition of model reference support.
Hanging and "tearing" was caused by a mistake in handling invisible faces during a rendering algorithm optimization.

Both issues will be fixed in the next 2.2.2 update.

Sorry for inconvenience, Egor.

[michaelRRI]

Thanks Egor,

I look forward to the update.
One other thing I have been thinking for a while, is it possible to have an option (possibly checkbox) when generating masks from a model to ignore backface? It would certainly help in some of my workflows.

[vineg]

HI!

2.2.2 pre-release is available here:

Download links:
https://download.agisoft.com/metashape-pro_2_2_2_x64.msi
https://download.agisoft.com/metashape-pro_2_2_2.dmg
https://download.agisoft.com/metashape-pro_2_2_2_amd64.tar.gz

And it is currently possible to render masks with back face culling using Python
Here is an example ( based on https://github.com/agisoft-llc/metashape-scripts/blob/master/src/render_photos_for_cameras.py ):

Code: [Select]

import Metashape
import os
import multiprocessing
import concurrent.futures

# Checking compatibility
compatible_major_version = "2.2"
found_major_version = ".".join(Metashape.app.version.split('.')[:2])
if found_major_version != compatible_major_version:
    raise Exception("Incompatible Metashape version: {} != {}".format(found_major_version, compatible_major_version))


def get_cameras(chunk):
    selected_cameras = [camera for camera in chunk.cameras if camera.transform and camera.selected and camera.type == Metashape.Camera.Type.Regular]

    if len(selected_cameras) > 0:
        return selected_cameras
    else:
        return [camera for camera in chunk.cameras if camera.transform and camera.type == Metashape.Camera.Type.Regular]

def process_camera(chunk, camera):
    if not camera.type == Metashape.Camera.Type.Regular: #skip camera track, if any
        return (None, None)
    photo_dir = os.path.dirname(camera.photo.path)

    render = chunk.model.renderMask(camera.transform, camera.sensor.calibration, cull_faces=True)

    photo_filename = os.path.basename(camera.photo.path)
    render_filename = os.path.splitext(photo_filename)[0] + "_mask.png"
    render_path = os.path.join(photo_dir, render_filename)

    render.save(render_path)
    return (camera, render_path)

def render_cameras():
    print("Script started...")

    chunk = Metashape.app.document.chunk
    if not chunk.model:
        raise Exception("No model!")
 
    mesh_faces = chunk.model.faces # keep model loaded

    with concurrent.futures.ThreadPoolExecutor(multiprocessing.cpu_count()) as executor:
        for (camera, render_path) in executor.map(lambda camera: process_camera(chunk, camera), chunk.cameras):
            if (camera is not None):
                mask = Metashape.Mask()
                mask.load(render_path)
                Metashape.app.document.chunk.masks[camera] = mask

    print("Script finished!")


label = "Scripts/Render masks for cameras"
Metashape.app.addMenuItem(label, render_cameras)
print("To execute this script press {}".format(label))

[michaelRRI]

Hi,

I am checking this again now 2.2.2 is released. The main issues of black frames, slow processing and tearing type artefacts are all fixed.

But, this one still remains:
"Also, when running a batch on a project with MANY small chunks (100+ with 30-40 photos each), I also notice some chunks as if they are clipping the object as one would expect with the mask defocus areas. This only happens on some chunks. When I run the generation again just on these chunks alone, they work."

I just noticed it in a batch process on 2.2.2
It usually happens when there is a camera lens difference between some of the chunks it seems.

M

[vineg]

Hi, Michael!

In case clipping occurs from the far side of a model, I can guess there may have remain depth_threshold setting from older versions of Metashape. You can try to remove it from python using following line in console:

Metashape.app.settings.setValue("tasks/GenerateMasks/depth_threshold", 1.0e20)

Or reset all application settings.

Sorry for inconvenience
Egor

[michaelRRI]

Thanks Egor. I tried those things and it is still happening.
The clipping does indeed occur from the far side of the model.
I did happen to make a test project of the bug though and submitted as a ticket if you want to follow up there.
Michael