Forum

Author Topic: Extraction of XMP/EXIF data for editing outside Metashape  (Read 6228 times)

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Extraction of XMP/EXIF data for editing outside Metashape
« on: January 04, 2019, 10:34:50 PM »
Dear all,

We've found that the default transformation conversions implemented in Photoscan/Metashape are lacking some features and are therefore not meeting our needs. Hence we are considering conducting the transformation outside of Metashape using dedicated third party software and subsequently overwriting the original WGS84 image coordinates in Metashape by reimporting the transformated coordinates.

The transformation itself isn't expected to be an issue, but we're still seeking a straightforward method to extract the values from the desired fields stored in our Phantom 4 RTK's XMP/EXIF data. We're curious to learn what options are out there.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: Extraction of XMP/EXIF data for editing outside Metashape
« Reply #1 on: January 05, 2019, 09:06:21 PM »
Hello BobvdMeij,

If you want to do it directly in Metashape, then a simple Python script can be used that will save to text file the information from camera.photo.meta (extracted automatically be the application) or (with a bit more complex script) would look into the source images XMP to find the needed tag.
Do you have a list of tags that you would like to output?
Best regards,
Alexey Pasumansky,
Agisoft LLC

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Extraction of XMP/EXIF data for editing outside Metashape
« Reply #2 on: January 07, 2019, 12:00:04 PM »
Hello BobvdMeij,

If you want to do it directly in Metashape, then a simple Python script can be used that will save to text file the information from camera.photo.meta (extracted automatically be the application) or (with a bit more complex script) would look into the source images XMP to find the needed tag.
Do you have a list of tags that you would like to output?

Dear Alexey,

Thanks for the swift response. Regarding the required fields/tags I reckon it matters which sourcefile Metashape normally acquires the image position and altitude from. More specially whether the software extracts this from the EXIF or the XMP file.

Considering I've set my preferences to 'Load camera calibration/orientatation/accuracy from XMP meta data' I assume Metashape also acquires the Longitude/Latitude/Altitude information from the said XMP file. Or is this is an incorrect assumption and does Metashape import this information from somewhere else? If the XMP file is indeed the correct sourcefile I reckon I'm seeking the tags/fields as visualized in the screendump below.

In short I'm only after the Long/Lat/Alt data, which I then plan to transform externally outside of Metashape. I subsequently seek a way to re-import the transformed data (in Northing/Easting/Altitude format) and to replace the original Long/Lat/Alt values. All other information already imported from the XMP file (including XYZ image accuracy, Yaw, Pitch, Roll etc.) should remain intact.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: Extraction of XMP/EXIF data for editing outside Metashape
« Reply #3 on: January 07, 2019, 02:15:59 PM »
Hello BobvdMeij,

Camera locations coordinates (Lat, Long and Altitude) are loaded from image EXIF, so the data from XMP is not loaded to the Reference pane, Relative and Absolute Altitude values are loaded to the camera.photo.meta and are stored in the project file, just for case anyone wants to use them.

The following script example will print out to the console the required tags for the cameras in the active chunk:
Code: [Select]
import Metashape
from xml.dom import minidom

elements = ["drone-dji:AbsoluteAltitude=", "drone-dji:GpsLatitude=", "drone-dji:GpsLongtitude="]

chunk = Metashape.app.document.chunk #active chunk
for camera in chunk.cameras:
path = camera.photo.path
with open(path, "rb") as file:
image = file.read()
string = str(image)
xmp_start = string.find('<x:xmpmeta')
xmp_end = string.find('</x:xmpmeta')
file.close()

if xmp_start != xmp_end:
xmpString = string[xmp_start : xmp_end + 12]

print(camera.label)
for element in elements:

try:
value = string[string.find(element) + len(element) : string.find(element) + len(element) + 10]
value = float(value.split('\"',3)[1])
print(element, value)
except IndexError:
print(element, " not found")

The external coordinate information can be loaded from the text file to the Reference pane or loaded via another script.
Best regards,
Alexey Pasumansky,
Agisoft LLC

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Extraction of XMP/EXIF data for editing outside Metashape
« Reply #4 on: January 07, 2019, 02:41:52 PM »
Hello BobvdMeij,

Camera locations coordinates (Lat, Long and Altitude) are loaded from image EXIF, so the data from XMP is not loaded to the Reference pane, Relative and Absolute Altitude values are loaded to the camera.photo.meta and are stored in the project file, just for case anyone wants to use them.

The following script example will print out to the console the required tags for the cameras in the active chunk:
Code: [Select]
import Metashape
from xml.dom import minidom

elements = ["drone-dji:AbsoluteAltitude=", "drone-dji:GpsLatitude=", "drone-dji:GpsLongtitude="]

chunk = Metashape.app.document.chunk #active chunk
for camera in chunk.cameras:
path = camera.photo.path
with open(path, "rb") as file:
image = file.read()
string = str(image)
xmp_start = string.find('<x:xmpmeta')
xmp_end = string.find('</x:xmpmeta')
file.close()

if xmp_start != xmp_end:
xmpString = string[xmp_start : xmp_end + 12]

print(camera.label)
for element in elements:

try:
value = string[string.find(element) + len(element) : string.find(element) + len(element) + 10]
value = float(value.split('\"',3)[1])
print(element, value)
except IndexError:
print(element, " not found")

The external coordinate information can be loaded from the text file to the Reference pane or loaded via another script.

Thanks Alexey! The script does indeed succesfully write the ID/Long/Lat/Alt to the console. However the script seems to suggest the data is extracted from the XMP file while I thought you mentioned we should be looking at the EXIF instead, right?

It also seems to only write 6 decimals for Long/Lat whereas the Reference Pane actually includes up to 16 decimals for Long/Lat. For precision purposes a minimum of 8 decimals is required. Likewise the Altitude is only printed with 2 decimals whereas the Reference Pane includes up to 3.

At last how do we go about converting the values printed in the Console to a separate file? We basically need a single .txt/.csv file with four colums storing the exported camera-ID, Longitude, Latitude and Altitude respectively for all cameras in the chunk. This we can then put through an Excel-macro in order to transform the values to the desired coordinates.

BobvdMeij

  • Full Member
  • ***
  • Posts: 157
    • View Profile
Re: Extraction of XMP/EXIF data for editing outside Metashape
« Reply #5 on: January 07, 2019, 04:14:28 PM »
In the meantime we've managed to extract the image-ID, GPSlongtitude, GPSlatitude and GPSaltitude values from all images within a folder and writing this to a single .txt or .csv file using ExifTool.

However, when reimporting the .txt file with the transformed coordinates this also causes the Accuracy field in the Reference Pane to be reset to the default 10m. Is there any way to only overwrite the Long/Lat/Alt fields but keep all the other fields untouched?

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: Extraction of XMP/EXIF data for editing outside Metashape
« Reply #6 on: January 09, 2019, 02:01:11 PM »
Hello BobvdMeij,

See updated export script version (it adds a new Custom menu item) that allows to save to the text file the following values from the XMP header: "drone-dji:RelativeAltitude=", "drone-dji:AbsoluteAltitude=", "drone-dji:GpsLatitude=", "drone-dji:GpsLongtitude="
and the following from the project meta information (automatically read by Metashape from XMP and EXIF):
'DJI/RtkStdHgt', 'DJI/RtkStdLat',  'DJI/RtkStdLon', 'Exif/GPSAltitude', 'Exif/GPSLatitude', 'Exif/GPSLongitude'


Code: [Select]
import Metashape
from xml.dom import minidom

def main_save_meta():
xmp_elements = ["drone-dji:RelativeAltitude=", "drone-dji:AbsoluteAltitude=", "drone-dji:GpsLatitude=", "drone-dji:GpsLongtitude="]
meta_elements =  ['DJI/RtkStdHgt', 'DJI/RtkStdLat',  'DJI/RtkStdLon', 'Exif/GPSAltitude', 'Exif/GPSLatitude', 'Exif/GPSLongitude']

if not len(Metashape.app.document.chunks):
print("Empty project, script aborted")
return False
export_path  = Metashape.app.getSaveFileName("Specify the export path to the file:", filter = "Text file (*.txt);;All formats (*.*)")
if not export_path:
print("Invalid path, script aborted.")
return False

export = open(export_path, "wt")
chunk = Metashape.app.document.chunk #active chunk
for camera in chunk.cameras:
path = camera.photo.path
with open(path, "rb") as file:
image = file.read()
string = str(image)
xmp_start = string.find('<x:xmpmeta')
xmp_end = string.find('</x:xmpmeta')
file.close()

if xmp_start != xmp_end:
xmpString = string[xmp_start : xmp_end + 12]

export.write(camera.label + "\n")
for element in xmp_elements:

try:
value = string[string.find(element) + len(element) : string.find(element) + len(element) + 10]
value = value.split('\"',3)[1]
export.write("\t{:s}\t{:s}\n".format(element, value))
except IndexError:
export.write("\t" + element + " not found\n")
for element in meta_elements:
if element in camera.photo.meta:
export.write("\t{:s}\t{:s}\n".format(element, camera.photo.meta[element]))
else:
export.write("\t" + element + " not found\n")


export.close()
print("Script finished, file written to:\n" + export_path)
return True


Metashape.app.addMenuItem("Custom menu/Save DJI meta to file", main_save_meta)
Best regards,
Alexey Pasumansky,
Agisoft LLC