Forum

Author Topic: script runs in network mode in console but not batch/run script - 1.6.5  (Read 2155 times)

andyroo

  • Sr. Member
  • ****
  • Posts: 438
    • View Profile
I wrote the below code to build a 25cm ortho (rather than full-res) with integer cell bounds, and it works great if I copy/paste into the console, but if I run it from the <>run script command and send to network processing, or if I launch it as a batch/run script (and send the batch to network processing) it fails to generate any network tasks (only shows as "running script" and the node reports "no network tasks to do!" from my else statement)

- note that I've only tested this on my local workstation configured as host/monitor/node/client, but I haven't seen this behavior on other scripts I tested.

Code: [Select]
import Metashape
import math
import os
from os import path
#-------------------------------------------------------#
#define user-set variables
raster_rounding_multiple = 40   # Default = 40 - This will be the multiple that the raster resolution is multiplied by to define the units the min/max extents are rounded to (DEM is 10 so this keeps unit rounding consistent)
raster_resolution = 0.25           # Default = 0.25 - cell size of exported ortho
raster_crop = True              # Default = True - True means Bounding Box in rounded IN - minimum extent is rounded up and maximum extent is rounded down from raster edges. False is reversed
                                # TODO - make it so metashape checks to see if this is an interpolated raster (shrink) or uninterpolated (grow?)
                                # ALSO - maybe we want to project the xy coordinates of the 3D dense cloud region and use those instead? this will result in no/minimal collar though...
ortho_subdir = 'ortho'              # this is a subdir that will be created under the document (PSX) path
ortho_suffix = '_NAD83_2011_UTM18'  #suffix to append to ortho (future - modify this to grab from WKT)
raster_rounding_interval = raster_rounding_multiple * raster_resolution
app = Metashape.app
doc = app.document
network_tasks = list()
for chunk in doc.chunks:
    if chunk.elevation:
        print(chunk.label)
        out_projection = chunk.elevation.projection
        compression = Metashape.ImageCompression()
        compression.tiff_compression = Metashape.ImageCompression.TiffCompressionLZW
        compression.tiff_big = True
        compression.tiff_overviews = True
        compression.tiff_tiled = True
           
        def round_down(x):
            return int(raster_rounding_interval * math.floor(float(x)/raster_rounding_interval))

        def round_up(x):
            return int(raster_rounding_interval * math.ceil(float(x)/raster_rounding_interval))


        #chunk.elevation.crs(wkt) #returns CRS for DEM
        testbox = Metashape.BBox() #create a bounding box for the raster
        print('')
        print('original DEM BBox coordinates:')
        print('min: ', Metashape.Vector((min(chunk.elevation.left, chunk.elevation.right), min(chunk.elevation.bottom, chunk.elevation.top))))
        print('max: ', Metashape.Vector((max(chunk.elevation.left, chunk.elevation.right), max(chunk.elevation.bottom, chunk.elevation.top))))

        if raster_crop:
            testbox.min = Metashape.Vector((round_up(min(chunk.elevation.left, chunk.elevation.right)), round_up(min(chunk.elevation.bottom, chunk.elevation.top))))
            testbox.max = Metashape.Vector((round_down(max(chunk.elevation.left, chunk.elevation.right)), round_down(max(chunk.elevation.bottom, chunk.elevation.top))))
        else:
            testbox.min = Metashape.Vector((round_down(min(chunk.elevation.left, chunk.elevation.right)), round_down(min(chunk.elevation.bottom, chunk.elevation.top))))
            testbox.max = Metashape.Vector((round_up(max(chunk.elevation.left, chunk.elevation.right)), round_up(max(chunk.elevation.bottom, chunk.elevation.top))))

        if raster_crop:
            print('extent was SHRUNK to: ')
            print('min: ',testbox.min)
            print('max: ',testbox.max)
        else:
            print('extent was GROWN to: ')
            print('min: ',testbox.min)
            print('max: ',testbox.max)
       
        print('building ortho in network mode')
        task = Metashape.Tasks.BuildOrthomosaic()
        task.blending_mode = Metashape.BlendingMode.AverageBlending
        task.cull_faces = False
        task.fill_holes = True
        task.projection = out_projection
        task.region = testbox
        task.resolution = raster_resolution
        task.resolution_x = raster_resolution
        task.resolution_y = raster_resolution
        task.refine_seamlines = False
        task.subdivide_task = True
        task.surface_data = Metashape.DataSource.ElevationData

        n_task = Metashape.NetworkTask()
        n_task.name = task.name
        n_task.params = task.encode()
        n_task.frames.append((chunk.key, 0))
        network_tasks.append(n_task)


if network_tasks:
    print('sending ', len(network_tasks), 'for processing')
    client = Metashape.NetworkClient()
    client.connect(app.settings.network_host) #server ip
    batch_id = client.createBatch(doc.path, network_tasks)
    client.resumeBatch(batch_id)
else:
    print('no network tasks to do!')
print('script complete')

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Hello Andy,

Does it help, if you change the way of doc definition and use the following:
Code: [Select]
doc = Metashape.Document()
doc.open(path_to_psx)

Also you may need to define the path relatively to "root" in createBatch command.
Best regards,
Alexey Pasumansky,
Agisoft LLC

andyroo

  • Sr. Member
  • ****
  • Posts: 438
    • View Profile
Hi Alexey,

Sorry my original description wasn't totally accurate - I modified the script to remove some network-checking functions (wrote a version that I wanted to work with network and non-network config) and got my outputs confused when reporting behavior. If I run the above script from the console and DO NOT send (the script) for network processing, then it builds the ortho as a network task (as intended).

If I run the script (or put it in a batch step as run script) and send it for network processing from the GUI client, then the node runs the script and I get these error messages from the node:

2021-07-21 13:40:48 RunScript: path = C:/Users/FloSup/Documents/Python_Scripts/HPC_metashape_scripts/alexeytest.py
2021-07-21 13:40:48 GCS
2021-07-21 13:40:48
2021-07-21 13:40:48 original DEM BBox coordinates:
2021-07-21 13:40:48 min:  Vector([-119.89130795946117, 34.401359588674026])
2021-07-21 13:40:48 max:  Vector([-119.84912047308137, 34.426608133905866])
2021-07-21 13:40:48 extent was SHRUNK to:
2021-07-21 13:40:48 min:  Vector([-110.0, 40.0])
2021-07-21 13:40:48 max:  Vector([-120.0, 30.0])
2021-07-21 13:40:48 building ortho in network mode
2021-07-21 13:40:48 PCS
2021-07-21 13:40:48
2021-07-21 13:40:48 original DEM BBox coordinates:
2021-07-21 13:40:48 min:  Vector([785712.068273728, 3811055.956547237])
2021-07-21 13:40:48 max:  Vector([789630.9063509195, 3813949.131303574])
2021-07-21 13:40:48 extent was SHRUNK to:
2021-07-21 13:40:48 min:  Vector([785720.0, 3811060.0])
2021-07-21 13:40:48 max:  Vector([789630.0, 3813940.0])
2021-07-21 13:40:48 building ortho in network mode
2021-07-21 13:40:48 sending  2 for processing
2021-07-21 13:40:48 disconnected from server
2021-07-21 13:40:48 connected to 127.0.0.1:5840
2021-07-21 13:40:48 Traceback (most recent call last):
2021-07-21 13:40:48   File "C:/Users/FloSup/Documents/Python_Scripts/HPC_metashape_scripts/alexeytest.py", line 83, in <module>
2021-07-21 13:40:48     batch_id = client.createBatch(doc.path, network_tasks)
2021-07-21 13:40:48 RuntimeError: Request failed with error: Duplicate batch: D:/metatest/psx/simple_test_project.psx
2021-07-21 13:40:48 Error: Request failed with error: Duplicate batch: D:/metatest/psx/simple_test_project.psx
2021-07-21 13:40:48 processing failed in 0.206 sec

When I send the script to network processing, the node also reports that app.settings.network_enable = False (thus it was resorting to non-network processing before I took that if-statement out).

I tried changing the way of doc definition with your method like this, but still got the same results:

Code: [Select]
app = Metashape.app
docpath = app.document.path
doc = Metashape.Document()
doc.open(docpath, read_only=False, ignore_lock=True)

It seems like the problem is when the script is run from the node, rather than from the client - and I need to do this with my current workflow because it is a combination of batch steps and scripts. If I don't run as a network process then all steps of the batch are done on the single GUI machine. Otherwise I guess my only option is to write a script for the whole workflow... but the batch steps allow me to select only certain chunks, then run scripts only on the chunks with specific products that exist from the batch operations (ie dense cloud, elevation model).