How to get makers errors

[CindyFr]

Dear all,

I am quite new in Photoscan scripting, and I try to export markers errors.
I have tried this code in order to have the mean error foreach marker :

Code: [Select]


def computeMeanErrorForeachMarker():
listErrors = []
run = 0
for marker in chunk.markers:
sumError = 0
num = 0
cam = chunk.cameras
for camera in cam:
print(marker)
print(camera)
v_proj = marker.projections[camera].coord
v_reproj = camera.project(marker.position)
diff = (v_proj - v_reproj).norm()
sumError += diff
num += 1
if(num>0):
meanError = sumError / num
else:
meanError = 0
listErrors.append(meanError)
return listErrors

 
listE = computeMeanErrorForeachMarker()
print(listE)


but I have the following error :

"Traceback (most recent call last):
File "C:\ToolsPerso\scripts-photoscan\script_compute_error_by_marker.py", line 31, in <module>
listE = computeMeanErrorForeachMarker()
File "C:\ToolsPerso\scripts-photoscan\script_compute_error_by_marker.py", line 18, in computeMeanErrorForeachMarker
v_proj = marker.projections[camera].coord
AttributeError: 'NoneType' object has no attribute 'coord' "

Can you please help me ?
Ps : I have searched in many topics, but nobody seems to have the same error as me ...

Thanks

Cindy.

[CindyFr]

Ok, I found some errors in my script :

Code: [Select]

def computeMeanErrorForeachMarker(p_chunk):
listErrors = []
run = 0
for marker in p_chunk.markers:
listTmp = [marker]
sumError = 0
num = 0
cam = p_chunk.cameras
for camera in cam:
print(marker)
print(camera)
v_proj = marker.projections[camera]
if v_proj is None:
print('V_proj est de type None')
continue
v_projc = v_proj.coord
v_reproj = camera.project(marker.position)
diff = (v_projc - v_reproj).norm()
sumError += diff
num += 1
if(num>0):
meanError = sumError / num
else:
meanError = 0
listTmp.append(meanError)

listErrors.append(listTmp)
return listErrors


doc = PhotoScan.app.document
chunk = doc.chunk
listE = computeMeanErrorForeachMarker(chunk)
print(listE)


I get my markers errors, but they are not the same that those displayed in the Reference pane ....

Python script gives :
[[<Marker 'target 1'>, 2.4433671504923136],
[<Marker 'target 2'>, 5.009520853120668],
[<Marker 'target 3'>, 9.195848226433014],
[<Marker 'target 4'>, 36.59943582802248],
[<Marker 'target 5'>, 15.89091610402427],
[<Marker 'target 6'>, 37.653693469253454],
[<Marker 'target 7'>, 18.194628972436924],
[<Marker 'target 8'>, 31.967252685314055],
[<Marker 'target 9'>, 192.8648202514543],
[<Marker 'target 10'>, 4.613443389363529]]


Instead of :
marker 1 : 2.953
marker 2 : 5.643
marker 3 : 15.249
marker 4 : 58.868
marker 5 : 21.155
marker 6 : 41.257
marker 7 : 26.193
marker 8 : 36.196
marker 9 : 279.845
marker 10 : 5.111

Any help please ?
marker 1

[Alexey Pasumansky]

Hello CindyFr,

Do you need to get total error in meters, total error in pixels or individual reprojection errors for each marker/camera pair?

[CindyFr]

Hello Alexey,

First, thank you for your response

In fact, I need to get the accuracy of my 3D model :
I know markers coordinates in real world.
I've put them into Photoscan, and update my model.
Now, i want to know when I pick a point in my model, what is the max error possible ( the interval into my real point can be)

[CindyFr]

Anybody ?

[Erik Holmlund]

If you want the total marker error:

Code: [Select]

doc = PhotoScan.app.document
chunk = doc.chunk

for marker in chunk.markers:
    est = chunk.crs.project(chunk.transform.matrix.mulp(marker.position))  # Gets estimated marker coordinate
    ref = marker.reference.location

    if est and ref:
        error = (est - ref).norm()  # The .norm() method gives the total error. Removing it gives X/Y/Z error
        print(marker.label, error)

That code gives the same error values for me.

[CindyFr]

If you want the total marker error:

Code: [Select]

doc = PhotoScan.app.document
chunk = doc.chunk

for marker in chunk.markers:
    est = chunk.crs.project(chunk.transform.matrix.mulp(marker.position))  # Gets estimated marker coordinate
    ref = marker.reference.location

    if est and ref:
        error = (est - ref).norm()  # The .norm() method gives the total error. Removing it gives X/Y/Z error
        print(marker.label, error)


That code gives the same error values for me.

Thank you so much eriksh, i'll test that next week !

[CindyFr]

Thank you Eriksh for your code, but it gives me only the errors for the markers wich I know real coordinates.
I want to retrieve the error of my estimated markers positions. I haven't their real position, just their coordinates in the local system and their estimation in the new system.

[Erik Holmlund]

Hi,
I modified an old script to work with what I think you're after, since I'm quite interested in it myself. It does only half of the work since I'm not sure that numpy works out of the box in 1.4.0 (an input on this would be lovely, I installed it manually so I don't know if it's 'supposed' to work or not).

This code takes two random images at a time and notes the resulting estimated coordinate of a marker, and then takes another pair of images and so on. It does so for all the markers, for half as many iterations as there are marker projections, and doesn't do anything if there are less than three projections. The output is a csv-table with the different estimated positions of each marker. The standard deviation of the values is a good measure of its precision.

With numpy this could be given as an output directly, e.g. through just np.std(marker_specific_result)

Code: [Select]

import PhotoScan
import random

doc = PhotoScan.app.document
chunk = doc.chunk


result = []
for marker in chunk.markers:
    num_projections = len(marker.projections)
    print(num_projections)

    if num_projections > 2:  # Marker needs more than two projections to evaluate error
        cam_list = list(marker.projections.keys())

        for x in range(int(round(num_projections / 2, 0))):  # Do half as many iterations as there are projections

            random.shuffle(cam_list)
            selected_cameras = cam_list[:2]  # Two random cameras

            # Note pinned pixel coordinates
            px_coords = {camera: marker.projections[camera].coord for camera in cam_list}

            # Unpinning the non-selected cameras
            for camera in cam_list:
                if camera not in selected_cameras:
                    marker.projections[camera] = None

            # Save the estimated position and marker label
            output = (marker.label,) + tuple(chunk.crs.project(chunk.transform.matrix.mulp(marker.position)))
            result.append(output)

            # Revert pinned coordinates
            for camera in cam_list:
                coord = PhotoScan.Marker.Projection(px_coords[camera])
                marker.projections[camera] = coord
                marker.projections[camera].pinned = True

# Write a CSV at desired position
file_name = PhotoScan.app.getSaveFileName("Save output file", filter="*.csv")
with open(file_name, "w") as file:
    for line in result:
       
        entry = ""
        for value in line:
            entry += str(value).replace("'", "") + ","
       
        file.write(entry + "\n")


Regards

Erik

[Erik Holmlund]

Hi again,
This concept turned out to be really useful for me, to evaluate marker quality in a particularly problematic dataset of mine. Therefore, I made some improvements with the code.

First off, it iterates through every possible combination of image pairs to project it, instead of choosing pairs at random, and then saves the standard deviations of X, Y, Z and Total, respectively. The output csv also notes how many iterations it handled, which should be close to n! of how many projections there are (up to the maximum limit).

Code: [Select]

import PhotoScan
import numpy as np
import itertools
import random

doc = PhotoScan.app.document
chunk = doc.chunk

max_iterations = 200 # Max allowed iterations for one marker

result = []
for marker in chunk.markers:
    num_projections = len(marker.projections)

    positions = []
    if num_projections > 2 and marker.type == PhotoScan.Marker.Type.Regular:  # Marker needs more than two projections to evaluate error, and not be a fiducial
        cam_list = [cam for cam in marker.projections.keys() if cam.center]  # Every aligned camera with projections
        random.shuffle(cam_list)  # Needed if the max_iterations is exceeded
       
        count = 0
        for a, b in itertools.combinations(cam_list, 2):  # Testing pairs of every possible combination

            if a.group and b.group and a.group == b.group and a.group.type == PhotoScan.CameraGroup.Type.Station:  # Skip if the cameras share station group
                continue

            if count >= max_iterations:  # Break if it reaches the iteration limit
                break
            count += 1

            selected_cameras = [a, b]

            # Note pinned pixel coordinates and if pinned or not (green or blue)
            px_coords = {camera: (marker.projections[camera].coord, marker.projections[camera].pinned) for camera in cam_list}

            # Unpinning the non-selected cameras
            for camera in cam_list:
                if camera not in selected_cameras:
                    marker.projections[camera] = None

            # Save the estimated position
            positions.append(list(chunk.crs.project(chunk.transform.matrix.mulp(marker.position))))

            # Revert pinned coordinates
            for camera in cam_list:
                coord, pinned = px_coords[camera]
                marker.projections[camera] = PhotoScan.Marker.Projection(coord)
                marker.projections[camera].pinned = pinned

        iterations = len(positions)  # Amount of tested positions
        positions = np.array(positions)
        std = np.std(positions, axis=0)  # Standard deviation
        rms = (np.sqrt(np.mean(std**2)))  # RMS of standard deviation

        result.append((marker.label,) + tuple(std) + (rms, iterations))

# Write a CSV at desired position
file_name = PhotoScan.app.getSaveFileName("Save output file", filter="*.csv")
if file_name:  # If an input was given
    with open(file_name, "w") as file:
        file.write("Label, X, Y, Z, Total, Iterations\n")
        for line in result:

            entry = ""
            for value in line:
                entry += str(value).replace("'", "") + ","

            file.write(entry + "\n")


It turned out that some of my markers were really poorly placed, which with this tool is incredibly apparent. So thanks, in a way!

Regards,

Erik

EDIT: The script makes PhotoScan freeze for me sometimes, yet it works perfectly after a restart... Don't know what that's about.

[CindyFr]

Hi Erik,
Thank you for your answer !
I must admit that for lack of solution I put this problem aside.
So I'm going to think about with your solution and test it on my side.

Cindy.

[SAV]

Hi Erik,

This is a great script. Very useful. Thanks for sharing.   

Regards,
SAV

Hi again,
This concept turned out to be really useful for me, to evaluate marker quality in a particularly problematic dataset of mine. Therefore, I made some improvements with the code.

First off, it iterates through every possible combination of image pairs to project it, instead of choosing pairs at random, and then saves the standard deviations of X, Y, Z and Total, respectively. The output csv also notes how many iterations it handled, which should be close to n! of how many projections there are (up to the maximum limit).

Code: [Select]

import PhotoScan
import numpy as np
import itertools
import random

doc = PhotoScan.app.document
chunk = doc.chunk

max_iterations = 200 # Max allowed iterations for one marker

result = []
for marker in chunk.markers:
    num_projections = len(marker.projections)

    positions = []
    if num_projections > 2 and marker.type == PhotoScan.Marker.Type.Regular:  # Marker needs more than two projections to evaluate error, and not be a fiducial
        cam_list = [cam for cam in marker.projections.keys() if cam.center]  # Every aligned camera with projections
        random.shuffle(cam_list)  # Needed if the max_iterations is exceeded
       
        count = 0
        for a, b in itertools.combinations(cam_list, 2):  # Testing pairs of every possible combination

            if a.group and b.group and a.group == b.group and a.group.type == PhotoScan.CameraGroup.Type.Station:  # Skip if the cameras share station group
                continue

            if count >= max_iterations:  # Break if it reaches the iteration limit
                break
            count += 1

            selected_cameras = [a, b]

            # Note pinned pixel coordinates and if pinned or not (green or blue)
            px_coords = {camera: (marker.projections[camera].coord, marker.projections[camera].pinned) for camera in cam_list}

            # Unpinning the non-selected cameras
            for camera in cam_list:
                if camera not in selected_cameras:
                    marker.projections[camera] = None

            # Save the estimated position
            positions.append(list(chunk.crs.project(chunk.transform.matrix.mulp(marker.position))))

            # Revert pinned coordinates
            for camera in cam_list:
                coord, pinned = px_coords[camera]
                marker.projections[camera] = PhotoScan.Marker.Projection(coord)
                marker.projections[camera].pinned = pinned

        iterations = len(positions)  # Amount of tested positions
        positions = np.array(positions)
        std = np.std(positions, axis=0)  # Standard deviation
        rms = (np.sqrt(np.mean(std**2)))  # RMS of standard deviation

        result.append((marker.label,) + tuple(std) + (rms, iterations))

# Write a CSV at desired position
file_name = PhotoScan.app.getSaveFileName("Save output file", filter="*.csv")
if file_name:  # If an input was given
    with open(file_name, "w") as file:
        file.write("Label, X, Y, Z, Total, Iterations\n")
        for line in result:

            entry = ""
            for value in line:
                entry += str(value).replace("'", "") + ","

            file.write(entry + "\n")


It turned out that some of my markers were really poorly placed, which with this tool is incredibly apparent. So thanks, in a way!

Regards,

Erik

EDIT: The script makes PhotoScan freeze for me sometimes, yet it works perfectly after a restart... Don't know what that's about.