Import markers from multiple files

[Mrblonde8888]

Hello,
Is there a way to upload more than one xml file of markers into my project? I'm used to adding many markers in my projects (around 1000). The marker projections in photos are not only useful but fundamental in my workflow. But adding too many markers slows the reactivity of the software to commands. I was wondering if there is a method to export groups of markers (sort of 300-500 per time) in different xml files, then delete them in the project to speed it up, and once finished the entire "marking" process, upload not one single group but all xml files containing each a single group. I tried conventionally but Metashape allows only one file to be uploaded.
Also, I want to add that dense clouds are already created and I don't need to use these markers as GCP to allineate the images.
Thanks

[SB]

A few years ago I wrote a python script to do this.  It will need rewriting for the current API but you can see the workflow in the code

Code: [Select]

import PhotoScan
from xml.dom import minidom

path = PhotoScan.app.getOpenFileName("Specify input XML file:")

doc = PhotoScan.app.document
chunk = doc.chunk

xml = minidom.parse(path)
main = xml.childNodes[0]

xml_markers_reference = xml.childNodes[0].childNodes[1].childNodes[5]
xml_markers_proj = xml.childNodes[0].childNodes[1].childNodes[7].childNodes[1].childNodes[1]
xml_camera = xml.childNodes[0].childNodes[1].childNodes[3]

cameras = dict()
markers = dict()
marker_proj = dict()
marker_proj_pins = dict()

for i in range(1, len(xml_camera.childNodes) - 1, 2):
camera_id = xml_camera.childNodes[i].getAttribute("id")
camera_label = xml_camera.childNodes[i].getAttribute("label")
cameras[camera_label] = camera_id

for i in range(1, len(xml_markers_reference.childNodes) - 1, 2):
marker_id = xml_markers_reference.childNodes[i].getAttribute("id")
marker_label = xml_markers_reference.childNodes[i].getAttribute("label")
markers[marker_id] = marker_label

for i in range(1, len(xml_markers_proj.childNodes) - 1, 2):
cam_list = dict()
pinned_list = dict()
marker_id = xml_markers_proj.childNodes[i].getAttribute("marker_id")

for j in range(1, len(xml_markers_proj.childNodes[i].childNodes) - 1, 2):

camera_id = xml_markers_proj.childNodes[i].childNodes[j].getAttribute("camera_id")
x = float(xml_markers_proj.childNodes[i].childNodes[j].getAttribute("x"))
y = float(xml_markers_proj.childNodes[i].childNodes[j].getAttribute("y"))
pinned = xml_markers_proj.childNodes[i].childNodes[j].getAttribute("pinned")

cam_list[camera_id] = PhotoScan.Vector([x, y])
pinned_list[camera_id] = pinned.lower() in ("yes", "true", "1")
       

marker_proj[marker_id] = cam_list

marker_proj_pins[marker_id] = pinned_list



for marker_id in marker_proj.keys():

        dup_found = False
        for mrkr in chunk.markers:
                if mrkr.label == markers[marker_id]:
                        m = mrkr
                        print ("Duplicate Marker found " + mrkr.label + " - using it")
                        dup_found = True
       
        if not dup_found:
                chunk.addMarker()
                m = chunk.markers[-1]
                m.label = markers[marker_id]

        for camera in chunk.cameras:
                if camera.label in cameras.keys():
                        if cameras[camera.label] in marker_proj[marker_id].keys():
                                print ("Adding camera " + camera.label + " to marker " + m.label )
                                m.projections[camera] = marker_proj[marker_id][cameras[camera.label]]
                                if marker_proj_pins[marker_id][cameras[camera.label]] :
                                        print ("Setting pin to True")
                                        m.projections[camera].pinned = marker_proj_pins[marker_id][cameras[camera.label]]

print("Script finished")


[Mrblonde8888]

Thanks, very much appreciated!

[Zim240]

Good morning,

I have the same problem and would like to use your script in metashape version 1.62 but cant get it working.

Did you Mrblonde8888 perhaps update the script to work with the new Metashape?

I am getting the below error when trying to run the script.

2020-04-20 10:54:56     xml_markers_proj = xml.childNodes[0].childNodes[1].childNodes[7].childNodes[1].childNodes[1]
2020-04-20 10:54:56 IndexError: list index out of range
2020-04-20 10:54:56 Error: list index out of range

Thank you

[Alexey Pasumansky]

Hello Zim240,

Do the different files contain the information about the projections for the same markers or whether they are completely independent?

[Zim240]

Thanks for responding.

There are no duplicate markers in the files.

I saved a few copies of the same project and had different people pick control to cut down on time. I deleted sections of the control file to make sure the same marker is not picked by 2 people.

I did not realise I cannot import multiple marker files back into one project afterwards.

So i exported all the markers from the different projects into XML files.

You can download 2 sample files below:

https://wetransfer.com/downloads/4d7a7479e7a0ac8958824b8a573da88a20200420131642/af7b96e26e630a10bfea490b5ba8b23320200420131711/5c9839

Much appreciated

[Alexey Pasumansky]

Hello Zim240,

Can you please also send the CSV file with the camera list and XYZ coordinates?

[Zim240]

Hi Alexey,

Please download the files you need below:

https://wetransfer.com/downloads/028612b7c601eae12c4f0f4448d8a96520200421064211/5a222d2df9587b0a85bd1c8da1ca03b420200421064238/072f06

Thanks

[Zim240]

Hi Alexey,

Could you manage to download and test the data?

I am stuck with my project at the moment and would not want to redo allot of work.

Thank you for the help.

[andyroo]

  A few years ago I wrote a python script to do this.  It will need rewriting for the current API but you can see the workflow in the code

I found this little gem from SB and updated it to work with Metashape 1.6.5. Also added an option (boolean pop-up window) to strip extensions or not. Note that this does NOT load positions - but you can easily do that by importing the markers with positions (like from CSV that you export from another project) in the reference pane. I tested that, and it does not reset pinning for identically-named markers.

Code below, with credit to original author.

[edit] - fixed the strip-extension method to work for labels that aren't 8.3 format by using os.path.splitext instead of camera_label = camera_label[0:8] (forgot I left that in there)...

Code: [Select]

#script to add/load pinned markers from multiple XML files created via File/export Markers.
#Tested with Metashape 1.6.5
#
#modified from original posted by SB on metashape forum
#original here https://www.agisoft.com/forum/index.php?topic=11519.msg51750#msg51750
#
#modified by andyroo 20201124 to use etree instead of minidom because
# (a) minidom hurt my brain and (b) etree seems less likely to break since it uses
# named labels instead of arbitrary node numbers/counts to crawl the xml
#
#I also added a second boolean checkbox that will strip extensions (eg JPG)
# in case you have that enabled in advanced prefs. If nothing happens and
# you think it should, try selecting/unselecting.
#
#thank you @SB for posting your script!!!

import Metashape
import xml.etree.ElementTree as ET
import os

path = Metashape.app.getOpenFileName("Specify input XML file:")
strip_ext = Metashape.app.getBool("Do you need to strip extensions for image labels?")
doc = Metashape.app.document
chunk = doc.chunk

tree = ET.parse(path)
root = tree.getroot()

cameras = dict()
markers = dict()
marker_proj = dict()
marker_proj_pins = dict()

for item in root.findall("./chunk/cameras//camera"):
camera_id = item.get("id")
camera_label = item.get("label")
if strip_ext:
camera_label = str(os.path.splitext(camera_label)[0])
cameras[camera_label] = camera_id

for i in root.findall("./chunk/markers//marker"):
marker_id = i.get("id")
marker_label = i.get("label")
marker_reference = i.get("reference")
markers[marker_id] = marker_label

for i in root.findall("./chunk/frames/frame/markers/marker"):
cam_list = dict()
pinned_list = dict()
marker_id = i.get("marker_id")

for j in i.findall("location"):

camera_id = j.get("camera_id")
x = float(j.get("x"))
y = float(j.get("y"))
pinned = j.get("pinned")

cam_list[camera_id] = Metashape.Vector([x, y])
pinned_list[camera_id] = pinned.lower() in ("yes", "true", "1")
       

marker_proj[marker_id] = cam_list

marker_proj_pins[marker_id] = pinned_list

for marker_id in marker_proj.keys():

        dup_found = False
        for mrkr in chunk.markers:
                if mrkr.label == markers[marker_id]:
                        m = mrkr
                        print ("Duplicate Marker found " + mrkr.label + " - using it")
                        dup_found = True
       
        if not dup_found:
                chunk.addMarker()
                m = chunk.markers[-1]
                m.label = markers[marker_id]

        for camera in chunk.cameras:
                if camera.label in cameras.keys():
                        if cameras[camera.label] in marker_proj[marker_id].keys():
                                print ("Adding camera " + camera.label + " to marker " + m.label )
                                m.projections[camera] = Metashape.Marker.Projection(Metashape.Vector(marker_proj[marker_id][cameras[camera.label]]))
                                if marker_proj_pins[marker_id][cameras[camera.label]] :
                                        print ("Setting pin to True")
                                        m.projections[camera].pinned = marker_proj_pins[marker_id][cameras[camera.label]]
print("Script finished")

[Mrblonde8888]

Hello, after a considerable time I came back to this topic. I managed to merge different xml files using visual studio. The script didn't work, markers were scattered in chaotic order inside the point cloud.