Mixing of network and non-network processes in the same script

[eastonmbio]

I'm looking to develop a script for network processing whereby tasks that are able to be done using network processing are mixed with those that do not fit in to the "tasks" classification.

For example, Photo alignment (network) then dense cloud generation (network), and dense cloud filtering based on point color (non-network), and so on. I'm unsure if I am able to do this. I think I would need to send the cleaning tasks to the network as queued jobs, but am unsure how I would go about this. If I was to simply insert code with the goal of point cloud cleaning, it would appear to run immediately, and execute on a point cloud that did not yet exist. The problem would persist I think, if I loaded several scripts into the batch processing window, to be executed one after the other.

The only solution I could think of is embedding any non-network tasks into a loop routine that checks whether the necessary point cloud exists before executing the code, waiting an arbitrary amount of time before running again?

An example of two sections of code I'd like to integrate is:

Code: [Select]


import Metashape

path = Metashape.app.getOpenFileName("Specify path to the PSX document:")
root = Metashape.app.getExistingDirectory("Specify network root path:")

doc = Metashape.Document()
doc.open(path)
chunk = doc.chunk

client=Metashape.NetworkClient()

network_tasks = list()

task = Metashape.Tasks.MatchPhotos()
task.network_distribute = True
task.downscale = 1
task.keypoint_limit = 40000
task.tiepoint_limit = 4000

n_task = Metashape.NetworkTask()
n_task.name = task.name
n_task.params = task.encode()
n_task.frames.append((chunk.key, 0))
network_tasks.append(n_task)

task = Metashape.Tasks.AlignCameras()
task.adaptive_fitting = False
task.network_distribute = True

n_task = Metashape.NetworkTask()
n_task.name = task.name
n_task.params = task.encode()
n_task.frames.append((chunk.key,0))
network_tasks.append(n_task)

task = Metashape.Tasks.BuildDepthMaps()
task.downscale = 4
task.filter_mode = Metashape.FilterMode.MildFiltering
task.network_distribute = True

n_task = Metashape.NetworkTask()
n_task.name = task.name
n_task.params = task.encode()
n_task.frames.append((chunk.key, 0))
network_tasks.append(n_task)

task = Metashape.Tasks.BuildDenseCloud()
task.network_distribute = True
n_task = Metashape.NetworkTask()
n_task.name = task.name
n_task.params = task.encode()
n_task.frames.append((chunk.key, 0))
network_tasks.append(n_task)


client.connect('metashape-qmgr.aims.gov.au')
batch_id = client.createBatch(path[len(root):], network_tasks)
client.resumeBatch(batch_id)
print("Job Started...")


I would like to integrate this with:

Code: [Select]


dense_cloud = chunk.dense_cloud

dense_cloud.selectPointsByColor(color=[85,170,255], tolerance=35, channels='RGB')
dense_cloud.removeSelectedPoints()


I would like this to be followed by mesh building.

I have seen this question asked before, but no answers were given.

Thanks.

[Alexey Pasumansky]

Hello eastonmbio,

If you need to apply some custom operation, which cannot be defined via Tasks/NetworkTasks classes, you should use RunScript task and integrate it to the workflow.

I have just posted an example of such pipeline, when it is necessary to open the project and configure the raster transformation, prior to the export operation, please check the code example here:
https://www.agisoft.com/forum/index.php?topic=13390.msg59677#msg59677

In the RunScript task you should open the project, perform the required operations and save the project.

[eastonmbio]

Thank you Alexey. I have attempted to integrate the RunScript task into a test version of my code, shown here:

Code: [Select]

import Metashape, sys

root = "Z:/"
path = sys.argv[1]

doc = Metashape.Document()
doc.open(path) #loading existing project using relative path from the root
chunk = doc.chunk #active chunk of the project

client=Metashape.NetworkClient()

tasks = []


task = Metashape.Tasks.RunScript() #Reducing overlap as network task
task.code = 'import Metashape\ndoc = Metashape.Document()\ndoc.open("Z:/projects/pilot_testing/PythonTesting.psx", ignore_lock = True)\nchunk = doc.chunk\nchunk.reduceOverlap(overlap=4)\ndoc.save()\n'
tasks.append(task)

#converting tasks to network tasks
network_tasks = []
for task in tasks:
if task.target == Metashape.Tasks.DocumentTarget:
network_tasks.append(task.toNetworkTask(doc))
else:
network_tasks.append(task.toNetworkTask(chunk))



client.connect('my.server.here')
batch_id = client.createBatch(path[len(root):], network_tasks)
client.resumeBatch(batch_id)
print("Reducing overlap ... Check the network monitor now")


However, the document fails to open and I am presented with the error message :

2021-07-06 21:23:17   File "<string>", line 3, in <module>
2021-07-06 21:23:17 OSError: Can't open file: No such file or directory (2): Z:/projects/pilot_testing/PythonTesting.psx
2021-07-06 21:23:17 Error: Can't open file: No such file or directory (2): Z:/projects/pilot_testing/PythonTesting.psx

I know that the path to the file in the RunScript command is correct, since I copied it directly from the Metashape GUI.

I am aware that during your mentioned example, the user was directed to a slightly different code structure in the RunScript task ,involving using the command

doc.open(Metashape.app.settings.network_path + "' + project_path + '", ignore_lock = True)

to open the document. However, when I alter my code structure to be reflective of this, I am presented with a similar error message, and the document also fails to open.

I am unsure why and any help would be appreciated, as this would significantly reduce the time a user needs to spend at a desk for processing. I see in your previous post, you mention that the code is intended to work with version 1.7.4 pre-release. I am using 1.7.2 . 12070, so I am certain that there are just some issues to do with specifying the file paths correctly. Within the current structure, it is no issue for me to send the "normal" network processing type jobs off (analyze photos, align photos etc.). I just cannot make the runscript process open the file correctly, whether I specify the file location directly, or define the path earlier on in the script.