Forum

Author Topic: Mapping from regions in photos to point cloud  (Read 2555 times)

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Mapping from regions in photos to point cloud
« on: January 29, 2021, 11:13:16 AM »
Hi all,

I am new to Metashape. I am working with drone imagery and creating a point cloud and mesh model using metashape from the drone imagery. Ultimately, I export to "oc3" or "obj" file and view it in the Agisoft viewer.

In addition, I also perform object detection in the captured scenery. For instance, I detect a man-made object such as a storm water drain and I can visualize it using a bounding box in the photo. I would like to use this detected object coordinates within the photo and map it to the point cloud so that one or more such objects can be annotated and visualized within metashape. Is there a way to acheieve this via Python scripting or the GUI.
Another way I am considering is to colour the points the point cloud which correspond to the detected object in a specific photo. Note that the same object can be in many photos since there is a lot of overlap between photos.

Any help or pointers would be useful.

Thanks.

Raj.

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #1 on: January 29, 2021, 12:53:39 PM »
Raj,

you can just open the photo of interest and draw a shape (point, line or polygon) for object you want to outline... In example, I drew a polygon in a photo for an electric registry and this shape automatically is reprojected onto the dense cloud....
Best Regards,

Paul Pelletier,
Surveyor

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #2 on: January 30, 2021, 12:01:12 AM »
Hi Paul,

Thanks for your suggestions. I have a small clarification.

SHould I draw the polygon using Metashape application or Python API or simply feed in photos with the polygon already drawn?

Thanks.

Raj.

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #3 on: January 30, 2021, 01:50:40 AM »
Yes Raj,

i would use the shape drawing tools inside the GUI to restitute the objects of interest...
Best Regards,

Paul Pelletier,
Surveyor

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #4 on: January 30, 2021, 02:01:57 AM »
Hi Paul,

Thanks for that. I figured out how to do it within the GUI and yes as you illustrated, the polygon appears in the model.

Within my scenery, there are 100s of photos which are input to my model. In each photo, there are several objects I detect offline and I have the polygon enclosing the objects.

My question now  - is it possible to automate the process through python scripting. For instance, can I add the polygons to the photos programmatically and save my model. I have a feeling that it should be possible.

Thank you very much for your assistance so far.

Regards,

Raj.

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #5 on: January 30, 2021, 03:30:53 PM »
Hello Raj,

yes it could be done by scripting. For each camera/photo, if you have a corresponding file with the polygons present, then you could open each polygon file and load the coordinates for each polygon and reproject them to your model...It supposes that the coordinates of your polygons are in a pixel format....each vertex of polygon has coordinates (xpix,ypix) xpix is number of pixels to right from top left corner photo and ypix is number of pixels to bottom from top left corner.

So this could be starting point for your solution using scripts
« Last Edit: January 30, 2021, 06:17:36 PM by Paulo »
Best Regards,

Paul Pelletier,
Surveyor

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #6 on: January 30, 2021, 11:58:57 PM »
Hi Paul,

Thanks for you answer again.

I do have the polygon points defined in a file in pixel coordinates corresponding to each photo. What I am envisaging to do is to use the Python API to accomplish projecting the polygons to the model.

I had a look at the Python API manual on Agisoft website and could not figure out how to do it. Are you able to point me to the class or method that is relevant? Thanks.

If there is another way of scripting from the GUI, please let me know.

Thanks once again for your help, it is much appreciated.

Regards,

Raj

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #7 on: January 31, 2021, 12:30:30 AM »
Hi Paul,

I read the section on shapes in the user manual. Although I haven't tried it, it appears that shapes can be imported from a file. Is is what you were referring to earlier? If so, are there examples of how these 2D shapes can be specified in a file?

Regards,

Raj

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #8 on: January 31, 2021, 12:45:07 AM »
Hey Raj,

supposing you have a project with dense cloud generated in a chunk. Then you would open each file corresponding toaphoto  and read each polygon inside the file.

Supposing you read the list of vertex coordinates of first polygon in file as polygon[0]=list((x0,y0), (x1, y1), ....

Code: [Select]
vertices = list()
for  (x,y) in polygon[0]:
            ray_origin = camera.unproject(Metashape.Vector([x, y, 0]))
            ray_target = camera.unproject(Metashape.Vector([x, y, 1]))
            vertices.append(chunk.crs.project(T.mulp(surface.pickPoint(ray_origin, ray_target))))
            shape = chunk.shapes.addShape()
            shape.attributes["Photo"] = camera.label
            shape.type = Metashape.Shape.Type.Polygon
            shape.vertices =vertices
            shape.has_z = True
where T = chunk.transform, camera is photo where polygon appears,
and surface = chunk.dense_cloud

I think this can give you a good base for developping your script and look at attached script as a starting point as it projects onto model surface the 4 corners of each photo as footprint shape...
« Last Edit: January 31, 2021, 12:52:22 AM by Paulo »
Best Regards,

Paul Pelletier,
Surveyor

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #9 on: January 31, 2021, 12:48:58 AM »
Raj,

for importing shapes from a file, you need to have them in 3D in projected or geographic CS. A file with 2D shapes (coordinates in photo) will not do...
Best Regards,

Paul Pelletier,
Surveyor

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #10 on: February 01, 2021, 03:40:27 AM »
Hi Paul,

Thanks for sending the code snippet. It is really helpful. I followed the code and wrote a script to process the polygons for each photo(camera). I run the script using "metashape.sh -r <script.py>".
The project I load was previously created in Metashape GUI and I follow the workflow steps of - align cameras, build dense cloud, build mesh and finally save the project.

However, the script fails on the pickPoint() call which is meant to find the intersection of the ray with the mesh - the return value in None.

Is there anything obvious that I am missing?

Regards,

Raj.

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #11 on: February 01, 2021, 07:44:49 AM »
Hello Raj,

maybe you can give  more detail, like th list of (x,y) from polygon as well as corresponding camera... is the dense cloud present in the chunk^? Is the camera aligned?

Or run the snippet in GUI and try print out (x,y) ray_origin, .... to try and disgnostic better the issue

See the attached example where I have polygon in img_6316.jpg with 5 vertices in pixel coordinates  and running snippet creates a shape in model with 5 vertices in GCS....
« Last Edit: February 01, 2021, 10:00:47 AM by Paulo »
Best Regards,

Paul Pelletier,
Surveyor

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #12 on: February 04, 2021, 01:50:06 AM »
Hi Paul,

Thanks for your suggestions. I had a bug in my code. It is now working perfectly.

Some further questions:

I have different types of objects in my scene - is it possible to use polygons with different colours based on type of object. I don't see any option to set colour anywhere.

After I build the model and project my polygons on to the point cloud, I see there is an option to export to a shapes file. Is there a way to export both the model and shapes into a file which can then be viewed by either the Agisoft viewer or some other third party viewer. What I have in mind is an exportable file which can be distributed to interested parties.

Regards,

Raj.

Paulo

  • Hero Member
  • *****
  • Posts: 1007
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #13 on: February 04, 2021, 02:37:29 AM »
Hey Raj,

you could create a shape group for  each type of object you are outlining and define its color as in example where we create a footprints group or layer with color defined as R,G,B (30, 239, 30):

Code: [Select]
    footprints = chunk.shapes.addGroup()
    footprints.label = "Footprints"
    footprints.color = (30, 239, 30)

and when you create a shape later in the code, define its group as footprints and it will take the color defined above:
Code: [Select]
shape.group = footprints
Then you can export the shapes and dense_cloud and visuaize in a GIS or web browser as SputNik https://www.agisoft.com/forum/index.php?topic=11119.msg50019#msg50019 or Virtual surveyor.....
« Last Edit: February 04, 2021, 03:38:22 AM by Paulo »
Best Regards,

Paul Pelletier,
Surveyor

Raj Calisa

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: Mapping from regions in photos to point cloud
« Reply #14 on: February 04, 2021, 01:32:11 PM »
Hi Paul,

Fantastic. Thank you. I was able to load the shapes stored in KML along with the point cloud (.oc3) format using Sputnik GIS.
The colours also work quite well.

Regards,

Raj.