Topics

1

Python and Java API / Unprojecting coordinates (bug?)

« on: July 17, 2024, 11:35:30 AM »

Hi all,

I'm using Agisoft Metashape as part of my workflow studying animal behaviour. I used the software to construct an orthomosaic and subsequently obtained drone videos of animals moving in this area. I then used computer vision algorithms to obtain coordinates of animals in these videos. To then convert these x-y coordinates from pixel space to geographical coordinates, I used the following code:

Code: [Select]

import Metashape
import pandas as pd

# Load the Metashape document and access the first chunk and its surface model
doc = Metashape.app.document
chunk = doc.chunks[0]
surface = chunk.model

# Switch to a different chunk for processing
chunk = doc.chunks[2]
print(f"Processing Chunk: {chunk.label}")

# Read the CSV file into a DataFrame
csv_path = '/Users/vivekhsridhar/Library/Mobile Documents/com~apple~CloudDocs/Documents/Metashape/TalChhapar/output/test_uv.csv'
df = pd.read_csv(csv_path)

# Initialize an empty list to store the data
data = []

# Iterate through cameras in the chunk
for camera in chunk.cameras:

# Filter the DataFrame to only include rows related to a specific camera
df_filtered = df[df['Camera'] == camera.label]

# Iterate through the filtered DataFrame and process each point
for index, row in df_filtered.iterrows():
    idx = row['idx']
    camera_label = row['Camera']
    u = row['u']
    v = row['v']

    # Create a 2D vector from the coordinates
    coords_2D = Metashape.Vector([u, v])
   
    # Pick a point on the model surface using the camera's center and the unprojected 2D coordinates
    point_internal = surface.pickPoint(camera.center, camera.unproject(coords_2D))

    # Transform the internal 3D point to world coordinates
    point3D_world = chunk.crs.project(chunk.transform.matrix.mulp(point_internal))

    # Append the data to the list
    data.append({
        'idx': idx,
        'Camera': camera_label,
        'u': u,
        'v': v,
        'x': point3D_world.x,
        'y': point3D_world.y,
        'z': point3D_world.z
    })

# Convert the list to a DataFrame
df_output = pd.DataFrame(data)

# Save the DataFrame to a CSV file
output_csv_path = '/Users/vivekhsridhar/Library/Mobile Documents/com~apple~CloudDocs/Documents/Metashape/TalChhapar/output/test_3D_world.csv'
df_output.to_csv(output_csv_path, index=False)

print(f"3D world coordinates saved to {output_csv_path}")

I know the code works because I've tested it with data from a single frame. However, when I run entire videos, it fails with the following error message.

2024-07-17 10:21:31 Traceback (most recent call last):
2024-07-17 10:21:31   File "/Users/vivekhsridhar/Library/Mobile Documents/com~apple~CloudDocs/Documents/Metashape/TalChhapar/scripts/test.py", line 40, in <module>
2024-07-17 10:21:31     point3D_world = chunk.crs.project(chunk.transform.matrix.mulp(point_internal))
2024-07-17 10:21:31 TypeError: argument 1 must be Metashape.Vector, not None
2024-07-17 10:21:31 Error: argument 1 must be Metashape.Vector, not None

I understand this occurs because my variable point_internal returns None. However, this shouldn't occur. I manually examined one specific case where the software unprojects the pixel coordinates (5025.5, 1802.5) to None. However, even a small change in the coordinates (5025.50001, 1802.5) or (5025.49999) works well and the software unprojects the coordinate to return reasonable values for the variable point_internal.

Since I'm not looking for that level of precision, I will proceed by jittering the value by small amounts when the software returns None and this will solve my problem. I however wanted to report this in case this is a bug with the software? If it isn't a bug, I would of course also like to understand what's causing it.

Best,
Vivek

2

General / Aligning photos across years

« on: July 01, 2024, 10:45:20 AM »

Dear Metashape Community,

I'm an animal behaviour researcher using metashape to reconstruct the landscape where the animals move. For this, I'm looking to create orthomosaics of the area. As part of the process, I've collected UAV-based images of the same location two years in a row (2023 and 2024). Unfortunately in 2023, I did not use markers or gather images systematically using a flight plan. Hence, the error on this map is much higher than the error I have on the 2024 map (where I used 5 markers and used an automated flight plan to gather images with 75% overlap).

Since I have good animal movement data from 2023, I would like to align the 2023 map to the 2024 map to reduce error. While the landscape has changed considerably in parts, there are roads and tracks that can be used to align one map to the other. Does the following pipeline make sense to do this?

1. Align photos 2023 in chunk 1
2. Align photos 2024 in chunk 2
3. Align chunks with 2024 as reference
4. Create DEM 2023 followed by the orthomosaic

I was wondering if this workflow makes sense. Since all my images are geotagged, both chunks 1 and 2 have [R] next to them. Does this matter when I use the align chunks function? I was curious as in most posts I find, users align chunks and then merge them. This seems like a way to break the computational requirement. In my case however, I would like both these chunks to be kept separate as they represent data from separate years and only align chunks to improve accuracy of older data.

Thank you for your help!

Best,
Vivek

3

Python and Java API / Projecting from image coordinates to the orthomosaic

« on: January 17, 2024, 06:00:15 PM »

Hi all,

I'm a biologist that studies animal movement and behaviour. As part of my work, I acquired UAV images which I used in Agisoft Metashape to reconstruct a georeferenced map of my animals' habitat. As a next step, I have videos I collected within this habitat. I have used computer vision algorithms to track my animals within my video, thus giving me their location in pixel coordinates. I would now like to convert trajectories of these animals from pixel space to global coordinates. To test if this worked, I first marked specific points on my orthomosaic. I then converted these coordinates to image space (from 3D to 2D) using the following code:

Code: [Select]

import numpy as np
import pandas as pd
import Metashape

def list_coordinates(file_path):
    data = []
    with open(file_path, 'r') as file:
        for line in file:
            values = line.strip().split(',')[1:-1]
            # Filter out empty strings before converting to float
            values = [float(val) if val else 0.0 for val in values]
            data.append(tuple(values))
    return data

def process_chunk(chunk, global_coordinates):
    T = chunk.transform.matrix

    data_list = []

    for point_idx, global_coord in enumerate(global_coordinates):
        p = T.inv().mulp(chunk.crs.unproject(Metashape.Vector(global_coord)))
        print(f"Image pixel coordinates of point {point_idx + 1} with global coordinates ({chunk.crs.name}): {global_coord}")

        for i, camera in enumerate(chunk.cameras):
            project_point = camera.project(p)

            if project_point:
                u = project_point.x  # u pixel coordinates in camera
                v = project_point.y  # v pixel coordinates in camera

                if 0 <= u <= camera.sensor.width and 0 <= v <= camera.sensor.height:
                    # Extract video and frame_seq from the camera label
                    camera_label_parts = camera.label.split('_')
                    video = '_'.join(camera_label_parts[:-1])
                    frame_seq = camera_label_parts[-1]

                    data_list.append([point_idx + 1, camera.label, video, frame_seq, u, v])

    columns = ['Point', 'Camera', 'video', 'frame_seq', 'u', 'v']
    df = pd.DataFrame(data_list, columns=columns)
    return df

# Define global coordinates for multiple points
points = list_coordinates('/Users/vivekhsridhar/Library/Mobile Documents/com~apple~CloudDocs/Documents/Metashape/Solitude/output/points_xyz.txt')

# Use active Metashape document
doc = Metashape.app.document

# Iterate through all chunks in the document
df = pd.DataFrame()
for chunk in doc.chunks:
    if chunk.label != 'BaseMap':
        # Set the current chunk to the one being processed
        doc.chunk = chunk

        # Process the current chunk
        tmp = process_chunk(chunk, points)
        df = pd.concat([df, tmp], ignore_index=True)

# Save the results to a CSV file
df.to_csv('/Users/vivekhsridhar/Library/Mobile Documents/com~apple~CloudDocs/Documents/Metashape/Solitude/output/points_uv.csv', index=False)

I know this code works because I plotted the 2D coordinates onto the respective images and they appear at the right location on the image. So next, I decided to convert these 2D coordinates back to 3D and check if I obtained coordinates of my original points. I used the following code to do this:

Code: [Select]

import Metashape
import datetime


doc = Metashape.app.document
chunk = doc.chunk
surface = chunk.point_cloud

chunk = doc.chunks[1]
print(chunk.label)

camera = chunk.cameras[0]
print(camera)

x=337.67729912777475
y=903.5153164982171

coords_2D = Metashape.Vector([x, y])
point_internal = surface.pickPoint(camera.center, camera.unproject(coords_2D))
point3D_world = chunk.crs.project(chunk.transform.matrix.mulp(point_internal))

print(point_internal)
print(point3D_world)

This however does not work and the newly obtained 3D coordinates do not match the location of the original point on the orthomosaic. Where have I gone wrong?

Thanks in advance for your help! I really appreciate any pointers in this direction.

Cheers!
Vivek