Creating and Managing Multiple Chunks

[LFSantosgeo]

Hello!

I'm finding difficulty to add a large number of chunks in loop.
 

Code: [Select]

    from_range = 1000
    to_range = 351000
    interval = 1000

    list_labels = list(range(from_range, to_range, interval))
    lenght = len(list_labels)
    print("Will be created {} chunks for this process".format(lenght))

    chunk = PS.app.document.chunks
    for chunk in range(lenght):
        doc.addChunk()
 UPDATED: managed to create multiple chunks...

[LFSantosgeo]

Now I'm getting trouble with assigning each chunk a new label based on a list:

Code: [Select]

    for chunk in doc.chunks:
        for values in range(len(list_labels)):
            for label in chunk.label:
                labels = "Chunk: " + str(int(values / 1000)) + " / 0"
            chunk.label = labels
Any help would be appreciated!

[Alexey Pasumansky]

Hello lfsantosgeo,

What is in the list_labels?

[Alexey Pasumansky]

Maybe you need to try that:

doc = PhotoScan.app.document

Code: [Select]

for i in range(len(doc.chunks)):
    doc.chunks[i].label =  "Chunk: " + str(int(list_labels[i]/1000)) + " / 0"

[LFSantosgeo]

doc = PhotoScan.app.document
for i in range(len(doc.chunks)):
    doc.chunks.label =  "Chunk: " + str(int(list_labels/1000)) + " / 0"

The code gives this error: "Error: 'list' object has no attribute 'label'"

[Alexey Pasumansky]

Hello lfsantosgeo,

Should be

Code: [Select]

doc.chunks[i].label

[LFSantosgeo]

Thank you, Alexey! That works now!

And that's my code:

Code: [Select]

    try:
        nchunks = (len(doc.chunks))  # returns number of chunks
        range_chunks = range(0, nchunks)  # gives range of a number of chunks
        for i in range_chunks:  # adding the labels to each chunk
            lbl_formatting = "Chunk: {} / 0".format(int(list_labels[i] / 1000))
            doc.chunks[i].label = lbl_formatting
        print("Chunk labels modified!")
    except Exception as exc2:
        print("Error:{}".format(exc2))

UPDATE: added the code for labeling every chunk in workspace based on a list

[LFSantosgeo]

Based on the information collected in another forum post by Alexey for the Exif tags: (sorry didn't get the address)

PhotoScan reads the following EXIF tags:
FocalLength - that is put to the corresponding field of the Camera Calibration window. Whereas for the Sensor Pixel Size uses either a pair of the following tags:
FocalPlaneXResolution and FocalPlaneYResolution, or if they are missing, uses the next one: FocalLengthIn35mmFilm.
The parameters mentioned above are used to define the initial values for the autocalibration procedure.

Also GPS tags are read:
GPSAltitude, GPSLatitude and GPSLongitude.

Made the following code to implement preselection based on photo's Exif information existence. And the from the PhotoScan 1.3 manual:

Pair preselection
The alignment process of large photo sets can take a long time.
A significant portion of this time period is spent on matching of detected features across the photos. Image pair preselection option may speed up this process due to selection of a subset of image pairs to be matched.

--> In the Generic preselection mode the overlapping pairs of photos are selected by matching photos using lower accuracy setting first.

--> In the Reference preselection mode the overlapping pairs of photos are selected based on the measured camera locations (if present).
[...]

Code: [Select]

    # check if images in every chunk added have Exif reference to use
    # Reference Preselection as a plus to optimize image alignment
    for i in range_chunks:
        camera = chunk.cameras[i]
        if camera.photo.meta["Exif/GPSLatitude"]:  # check for Lat info on Exif
            # generic preselection [True, False]
            gnr_pslc = True
            # reference prelection [True, False]
            ref_pslc = True
        else:  # if there's no exif information then use No Preselection enabled
            gnr_pslc = False
            ref_pslc = False

The code was based on a test that's shown bellow for the best performance regarding time and errors.

[LFSantosgeo]

I made some tests using geotagged photos (from the drone's onboard GPS system) regarding preselection.

With these parameters:
High Accuracy (keeps 100% of each image, not upscales or dowscales)
Total cameras (photos): 155 geotagged
Key points = 10,000 (not that much to optimize time in the processing)
Tie points = 0

-----------------------------------------------------
No preselection
-----------------------------------------------------
148 aligned cameras of 155
Points 50,476 of 55,628

RMS reprojection error: 0.115293 (0.602434 pix)
Max reprojection error: 0.348031 (19.6074 pix)
Mean key point size: 4.19666 pix

Effective overlap 2.58956

Match time: 00:23:56
Align time: 00:01:46
Total time: 00:25:42 -- compatible time with the Reference preselection only

-----------------------------------------------------
Generic preselection only
-----------------------------------------------------
148 of 155 aligned cameras
Points 50,780 of a total of 54,274 possible

RMS reprojection error: 0.115442 (0.60689 pix)
Max reprojection error: 0.348413 (19.5868 pix)
Mean key point size: 4.19952 pix

Effective overlap: 2.59327

Matching time: 00:37:49
Alignment time: 00:01:49
Total time: 00:39:38

-----------------------------------------------------
Reference preselection only
-----------------------------------------------------
148 of 155 aligned
Points 50,670 of 54,789

RMS reprojection error: 0.115329 (0.610028 pix)
Max reprojection error: 0.348384 (20.2262 pix)
Mean key point size: 4.19839 pix

Effective overlap: 2.59209

Matching time: 00:23:48
Alignment time: 00:01:49
Total time: 00:25:37

-----------------------------------------------------
Generic + Reference both selected!
-----------------------------------------------------
148 of 155 aligned
Points 50,790 of 54,269

RMS reprojection error: 0.115142 (0.605183 pix)
Max reprojection error: 0.347645 (19.5746 pix)
Mean key point size: 4.19816 pix

Effective overlap: 2.59146

Matching time: 00:18:39
Alignment time: 00:02:07
Total time: 00:20:46

That justifies the code to use generic + reference if there's geotagged photos. And if not, no preselection.

[LFSantosgeo]

Now I'm stuck with exporting the Show Info for every chunk on a single txt file:

Code: [Select]

    # path to write file
    path_report = path_export
    path_report += "\Showinfo_chunks_{}.txt".format(prjrun)
    print("\nCreating export file and analyzing chunks...\n")
    # open file
    try:
        # getting chunk labels
        lis_label = []
        for i in range_chunks:
            lis_label.append(doc.chunks[i].label)
        print(lis_label)

        # returning aligned cameras for each chunk
        list_align = []
        for i in range_chunks:
            chunk = doc.chunks[i]
            for cameras in chunk.cameras:
                if camera.transform:
                    align_cams = len(chunk.cameras)
            list_align.append(align_cams)

        # returning total cameras for each chunk
        list_cams = []
        for i in range_chunks:
            chunk = doc.chunks[i]
            cam_total = len(chunk.cameras)
            list_cams.append(cam_total)

        # alignment time for each chunk
        list_align_dur = []
        for i in range_chunks:
            chunk = doc.chunks[i]
            align_dur = chunk.point_cloud.meta["align/duration"]
            list_align_dur.append(align_dur)

        # matching time for each chunk
        list_match_dur = []
        for i in range_chunks:
            chunk = doc.chunks[i]
            match_dur = chunk.point_cloud.meta["match/duration"]
            list_match_dur.append(match_dur)

        # list comprehension to avoid None values (substitutes for 0)
        # and to convert its strings into float to sum values
        align_time = [0 if i is None else i for i in list_align_dur]
        align_time = [float(i) for i in align_time]  # converting to float
        match_time = [0 if i is None else i for i in list_match_dur]
        match_time = [float(i) for i in match_time]  # convert to float

        # sum lists to get total time for each chunk
        total_time = [sum(i) for i in zip(align_time, match_time)]

        # number of tie-points (sparse cloud) for each chunk
        list_sp_points = []
        for i in range_chunks:
            chunk = doc.chunks[i]
            point_cloud = chunk.point_cloud
            points = point_cloud.points
            sp_points = len(points)
            list_sp_points.append(sp_points)

    except Exception:
        print("Error: unable to get data from chunks!")
        raise

    try:
        with open(path_report, "wt", newline="") as showinfo:
            lists = [lis_label, list_align, list_cams, align_time,
                     match_time, total_time, list_sp_points]
            for i in zip(*lists):
                showinfo.write("\n{}\t{}\t{}\t{}\t{}\t{}\t{}\n\n".format(*i))
        print("Successfully generated and exported to TXT report!")
    except Exception:
        raise


UPDATE: managed to get:
- number of aligned cameras
- number of cameras
- alignment time
- matching time
- align + match = total time
- sparse cloud number of points


I'm still stuck with:
- RMS reprojection error
- Max reprojection error
- Mean key point size
- effective overlap

[LFSantosgeo]

For reprojection error, got the following code from: http://www.agisoft.com/forum/index.php?topic=5267.msg26153#msg26153

Code: [Select]

        # open empty lists to get exported when done
        list_npoints = []
        list_index = []
        list_x = []
        list_y = []
        list_z = []
        list_error = []


        for i in range_chunks:  # for every chunk in workspace
        # number of tie-points (sparse cloud) for each chunk
            chunk = doc.chunks[i]
            point_cloud = chunk.point_cloud
            points = point_cloud.points
            npoints = len(points)
            list_npoints.append(npoints)

        # Reprojection error
            M = chunk.transform.matrix
            crs = chunk.crs
            projections = point_cloud.projections
            tracks = point_cloud.tracks

            points_coords = {}
            points_errors = {}

            for photo in chunk.cameras:
                if not photo.transform:
                    continue

                T = photo.transform.inv()
                calib = photo.sensor.calibration

                point_index = 0
                for proj in projections[photo]:
                    track_id = proj.track_id
                    while point_index < npoints and points[point_index].track_id < track_id:
                        point_index += 1
                    if point_index < npoints and points[point_index].track_id == track_id:
                        if not points[point_index].valid:
                            continue

                        coord = T * points[point_index].coord
                        coord.size = 3
                        dist = calib.error(coord, proj.coord).norm() ** 2
                        v = M * points[point_index].coord
                        v.size = 3

                        if point_index in points_errors.keys():
                            point_index = int(point_index)
                            points_errors[point_index].x += dist
                            points_errors[point_index].y += 1
                        else:
                            points_errors[point_index] = PS.Vector([dist, 1])

                list_index.append(point_index)

                for point_index in range(npoints):
                    if not points[point_index].valid:
                        continue

                    if chunk.crs:
                        w = M * points[point_index].coord
                        w.size = 3
                        X, Y, Z = chunk.crs.project(w)
                    else:
                        X, Y, Z, w = M * points[point_index].coord

            error = math.sqrt(points_errors[point_index].x / points_errors[point_index].y)

            list_x.append(X)
            list_y.append(Y)
            list_z.append(Z)
            list_error.append(error)


For chunk test gives error = 1.6664504017324355
And testing again, a different value = 1.2779145775553513
Actually, everytime I run the code I get a different value for the "error".

In the Show Info pane comes as RMS = 0.145085 (4.41861 pix) and Max = 0.437942 (54.9417 pix)

What is wrong? Or the code returns in different format?

[LFSantosgeo]

Any comments on the matter presented above? ?

[LFSantosgeo]

When importing markers XML file into PhotoScan I get this error:

Error: Missing sensor id

Have I lost the file and have to start over?

[Alexey Pasumansky]

Hello lfsantosgeo,

Was the XML exported from the same PhotoScan version? Which version (including the build number) you are currently using?

[LFSantosgeo]

Hello Alexey!

It was created with version 1.3.2 and now I'm using 1.4.0 b5076.x64.