Show Posts
1
Python and Java API / Cannot achieve multi-threading with concurrent.futures.ThreadPoolExecutor
« on: April 05, 2022, 05:51:30 PM »
Hi,
I noticed that in the official Python scripts collection (https://github.com/agisoft-llc/metashape-scripts), footprints-to-shapes and automatic_masking use concurrent.futures.ThreadPoolExecutor to achieve multi-threading processing.
However, I use the same method in my script but no matter I run it from either Tool>Run script or from the Python console directly, it seems to always use only one core for processing. Could you please enlighten me what might go wrong?
Below is my code
I noticed that in the official Python scripts collection (https://github.com/agisoft-llc/metashape-scripts), footprints-to-shapes and automatic_masking use concurrent.futures.ThreadPoolExecutor to achieve multi-threading processing.
However, I use the same method in my script but no matter I run it from either Tool>Run script or from the Python console directly, it seems to always use only one core for processing. Could you please enlighten me what might go wrong?
Below is my code
Code: [Select]
def GetPointMatchList(chunk, *band, collate=True):
point_cloud = chunk.point_cloud
points = point_cloud.points
point_proj = point_cloud.projections
npoints = len(points)
camera_matches = dict()
point_matches = dict()
for camera in chunk.cameras:
total = dict()
point_index = 0
# If no band number input, only process the master channel
try:
proj = point_proj[camera.planes[band[0]]]
except IndexError:
proj = point_proj[camera]
except KeyError:
# Skip the camera if there is no projection information
continue
for cur_point in proj:
track_id = cur_point.track_id
# Match the point track ID
while point_index < npoints and points[point_index].track_id < track_id:
point_index += 1
if point_index < npoints and points[point_index].track_id == track_id:
# Add point matches and save their pixel coordinates to list
total[point_index] = cur_point.coord
try:
point_matches[point_index][camera.planes[band[0]]] = cur_point.coord
except KeyError:
point_matches[point_index] = dict()
try:
point_matches[point_index][camera.planes[band[0]]] = cur_point.coord
except IndexError:
point_matches[point_index][camera] = cur_point.coord
except IndexError:
point_matches[point_index][camera] = cur_point.coord
try:
camera_matches[camera.planes[band[0]]] = total
except IndexError:
camera_matches[camera] = total
if collate:
# Keep tie points which have at least 3 observation in same band if collate is True
point_to_keep = set()
new_camera_matches = dict()
new_point_matches = dict()
for point_index, value in point_matches.items():
if len(value) >= 3:
new_point_matches[point_index] = value
point_to_keep.add(point_index)
for camera, points in camera_matches.items():
new_camera_matches[camera] = {point: coord for point, coord in iter(points.items()) if
point in point_to_keep}
# camera_matches describes point indice and their projected pixel coordinates for each camera
# point_matches describes point's pixel coordinates in different cameras for each point
return camera_matches, point_matches
with concurrent.futures.ThreadPoolExecutor() as exe:
results = exe.map(lambda _index: GetPointMatchList(chunk, _index),
[_index for _index, _band_name in enumerate(bands)])
for result in results:
cameras, points = result
camera_matches = {**camera_matches, **cameras}
point_matches = {**point_matches, **points}I also tried to put multiprocessing.cpu_count() as an argument of the executor, but didn't work either.
