Creating Markers from a single 2D Photo Coordinate on a processed model

[AndyMatthews]

Hi Guys,

We are trying to use 2D point coordinates from the scene photos to create a marker on the processed model. We are detecting the markers in an external script directly from the images used to create the chunk.

If we import a marker via an xml file, the point does not propagate to the model, unless there is at least one other corresponding points on another image. We need to do this from a single image and a single reference.

Within the documents Photos pane Agisoft can Right+Click>Create Marker and a marker will 'propagates' to the model and relevant images.

What is the Python Method or process to propagate the point to the other models.

Thanks

Andy M

[Alexey Pasumansky]

Hello Andy,

Here the script sample for the simulation of the Create Marker operation (in the sample the projection coordinates and the camera is built-in to the script body):

Code: [Select]

import PhotoScan

def cross(a, b):
result = PhotoScan.Vector([a.y*b.z - a.z*b.y, a.z*b.x - a.x*b.z, a.x*b.y - a.y *b.x])
return result

doc = PhotoScan.app.document
chunk = doc.chunk
model = chunk.model
vertices = chunk.model.vertices

T0 = chunk.transform.matrix

camera = chunk.cameras[0] #camera index
marker_2D = (2000, 1000) #projections of marker on the given photo

marker = chunk.addMarker()
marker.projections[camera] = marker_2D

point_2D = marker.projections[camera].coord
vect = camera.sensor.calibration.unproject(point_2D)
vect = camera.transform.mulv(vect)
center = camera.center

#estimating ray and surface intersection
for face in model.faces:

v = face.vertices

E1 = PhotoScan.Vector(vertices[v[1]].coord - vertices[v[0]].coord)
E2 = PhotoScan.Vector(vertices[v[2]].coord - vertices[v[0]].coord)
D = PhotoScan.Vector(vect)
T = PhotoScan.Vector(center - vertices[v[0]].coord)
P = cross(D, E2)
Q = cross(T, E1)
result = PhotoScan.Vector([Q * E2, P * T, Q * D]) / (P * E1)

if (0 < result[1]) and (0 < result[2]) and (result[1] + result[2] <= 1):
t = (1 - result[1] - result[2]) * vertices[v[0]].coord
u = result[1] * vertices[v[1]].coord
v_ = result[2] * vertices[v[2]].coord

point_3D = T0.mulp(u + v_ + t)
if chunk.crs:
point_3D = chunk.crs.project(point_3D)
break

if chunk.crs:
point = chunk.crs.unproject(point_3D)
point = T0.inv().mulp(point)

for cur_camera in chunk.cameras:

if (cur_camera == camera) or not cur_camera.transform:
continue
cur_proj = cur_camera.project(point)

if  (0 <= cur_proj[0] < camera.sensor.width) and (0 <= cur_proj[1] < camera.sensor.height):
marker.projections[cur_camera] = cur_proj

print("Script finished")
Hope it helps. If there are any issues, please let me know, as the script was adapted from the older version.

[AndyMatthews]

Wow that was a fast replay.

We tried a version of this script earlier today but iit was finding the back faces on the mode , but we will try this asap and let you know how it goes.



Thanks Alexey.

[Alexey Pasumansky]

Hello Andy,

Probably, you need also to take into consideration the normals of the mesh faces, however, at the moment there's no access to them via Python API, so you might need to write your own function for that.

[albert.feliu]

Hi,

I've done some subtle tweaks that adds the points we need on the right faces. However it's quite slow as it needs to loop through all the cameras and all the points.

Code: [Select]

import PhotoScan
import csv
import os
import math

resultfolder = "\TemplatesPics"



def cross(a, b):
    result = PhotoScan.Vector([a.y*b.z - a.z*b.y, a.z*b.x - a.x*b.z, a.x*b.y - a.y *b.x])
    return result

def distance3d(p1,p2):
    distance = abs(math.sqrt((math.pow(p2[0]-p1[0],2))+(math.pow(p2[1]-p1[1],2))+(math.pow(p2[2]-p1[2],2))))
    return distance



def createMarker(marker_2D,nameiter):

    #marker_2D = (1666, 2690) #projections of marker on the given photo
    marker = chunk.addMarker()
    marker.projections[camera] = marker_2D

    point_2D = marker.projections[camera].coord
    vect = camera.sensor.calibration.unproject(point_2D)
    vect = camera.transform.mulv(vect)
    center = camera.center
    print(center)

    arr_points3D = []

    #estimating ray and surface intersection
    for face in model.faces:

        v = face.vertices

        E1 = PhotoScan.Vector(vertices[v[1]].coord - vertices[v[0]].coord)
        E2 = PhotoScan.Vector(vertices[v[2]].coord - vertices[v[0]].coord)
        D = PhotoScan.Vector(vect)
        T = PhotoScan.Vector(center - vertices[v[0]].coord)
        P = cross(D, E2)
        Q = cross(T, E1)
        result = PhotoScan.Vector([Q * E2, P * T, Q * D]) / (P * E1)

        if (0 < result[1]) and (0 < result[2]) and (result[1] + result[2] <= 1):
            t = (1 - result[1] - result[2]) * vertices[v[0]].coord
            u = result[1] * vertices[v[1]].coord
            v_ = result[2] * vertices[v[2]].coord

            point_3D = T0.mulp(u + v_ + t)

            if chunk.crs:
                point_3D = chunk.crs.project(point_3D)

            arr_points3D.append(point_3D)

            #break

    print (arr_points3D)

    arr_points = []
    for p in arr_points3D:
        if chunk.crs:
            point = chunk.crs.unproject(p)
        point = T0.inv().mulp(point)
        arr_points.append(point)

    # find nearest point
    finaldst = 99999
    index = 9999
    for i,pnt in enumerate(arr_points):
        dst = distance3d(center,pnt)
        if dst < finaldst:
            finaldst = dst
            index = i
    try:
        correct_pnt = arr_points[i]
    except UnboundLocalError:
        nameiter+=1
        return nameiter

    #print (arr_points)
    done = False


    markerlist = [marker]
    chunk.remove(markerlist)

    for c,cur_camera in enumerate(chunk.cameras):
        while not done:

            marker = chunk.addMarker()
            marker.label = "trackPoint_{0}".format(str(n))
            marker.projections[camera] = marker_2D

            cur_proj = cur_camera.project(correct_pnt)
            marker.projections[cur_camera] = cur_proj
            done = True

            nameiter += 1

    return nameiter

#read markers from file
def readMarkers(file):
    point_arr = []
    with open(file, 'r') as csvfile:
        spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
        for row in spamreader:
            point_arr.append((float(row[0]),float(row[1])))

    return point_arr

doc = PhotoScan.app.document
chunk = doc.chunk
model = chunk.model
vertices = chunk.model.vertices
T0 = chunk.transform.matrix
camera = chunk.cameras[38] #camera index
n = 1



markers = readMarkers(blah)

for m in markers:
    n = createMarker(m,n)


print("Script finished")
The issue is on this bit, if I just point directly at the camera that we are interested, the marker doesn't appear. If we loop through all the cameras, it's very slow. Can you let us know if there's a way to optimise this?

Code: [Select]

for c,cur_camera in enumerate(chunk.cameras):
        while not done:

            marker = chunk.addMarker()
            marker.label = "trackPoint_{0}".format(str(n))
            marker.projections[camera] = marker_2D

            cur_proj = cur_camera.project(correct_pnt)
            marker.projections[cur_camera] = cur_proj
            done = True

            nameiter += 1
Many thanks!

[albert.feliu]

Hi,

I was wondering if someone had the chance to look at this further. Thanks

Albert

[Paul Lee]

Hello,
    I have tried this scrip and just add two lines in the python script to show the color in the camera images, but the result shows that the pixel colors in different image are different, the point in different cameras' projection should have the same pixel color. And why this happens?

Code: [Select]

if  (0 <= cur_proj[0] < camera.sensor.width) and (0 <= cur_proj[1] < camera.sensor.height):
marker.projections[cur_camera] = cur_proj
image = cur_camera.photo.image()
print(image[cur_proj[0],cur_proj[1]]) 

color result:

Code: [Select]

2017-03-10 17:01:12 (85, 80, 58)
2017-03-10 17:01:12 (94, 81, 62)
2017-03-10 17:01:12 (159, 154, 151)
2017-03-10 17:01:12 (96, 92, 65)
2017-03-10 17:01:12 (157, 140, 146)
2017-03-10 17:01:12 (164, 155, 150)
2017-03-10 17:01:12 (178, 161, 167)
2017-03-10 17:01:12 (209, 186, 196)
2017-03-10 17:01:12 (204, 180, 193)
2017-03-10 17:01:12 (185, 166, 168)
2017-03-10 17:01:12 (209, 184, 187)
2017-03-10 17:01:12 (206, 184, 187)
2017-03-10 17:01:12 (68, 60, 58)
2017-03-10 17:01:12 (75, 74, 69)

[Alexey Pasumansky]

Hello Paul,

Why you assume that color of the point, corresponding to the marker projection should be the same?

The images could be taken in the different conditions, especially if speaking of aerial datasets, also automatic camera parameters, like shutter speed, aperture, ISO may lead to different colors, even not speaking about the JPG compression and noise.

[Paul Lee]

Thank you Alexey, I think you are right. I want to compare the color because I want to confirm I get the same 3D point in different camera images.
Thank you for your reply.