Forum

Author Topic: BuildDEM NetworkTask and the OrthoProjection parameter  (Read 3624 times)

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
BuildDEM NetworkTask and the OrthoProjection parameter
« on: July 17, 2018, 06:40:22 PM »
I am attempting to use Network Processing to run a number of tasks in a distributed environment. The BuildDEM task however, fails for all attempts when constructing the task from the Console window. When constructing the NetworkTask from the UI, I have no problem.

The 'createBatch' result when creating the task via the UI produces the following JSON:

Code: [Select]
{
"id": 152,
"method": "createBatch",
"params": {
"path": "some_photoscan_project.psx",
"tasks": [{
"frames": [
[0, 0]
],
"name": "BuildDem",
"params": {
"projection": {
"crs": "GEOGCS[\"WGS 84\",DATUM[\"World Geodetic System 1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9102\"]],AUTHORITY[\"EPSG\",\"4326\"]]",
"radius": 1,
"surface": 0,
"transform": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
}
}
}],
"username": "someuser"
}
}

When creating the batch via the Python console, the following JSON is produced:

Code: [Select]
{
"id": 4,
"method": "createBatch",
"params": {
"path": "some_photoscan_project.psx",
"tasks": [{
"name": "BuildDem",
"params": {
"projection": {
"crs": "[\"\",[\"\"],UNIT[\"\",0]]",
"radius": 1,
"surface": 0,
"transform": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
}
},
"supports_gpu": false
}],
"username": "someuser"
}
}

The principle difference between the two appears to be the CRS member of the "projection" parameter. For whatever reason, the "PhotoScan.OrthoProjection" type does not appear to serialize the "CRS" member of the projection parameter correctly. Can you provide any examples to create a projection parameter that is crafted correctly for the BuildDEM task?

Thanks,

Ian

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #1 on: July 17, 2018, 08:25:06 PM »
Hello Ian,

How you are creating the network task?

You can encode it from the PhotoScan.Tasks.BuildDem, something like the following:

Code: [Select]
task  = PhotoScan.Tasks.BuildDem()
task.source_data = PhotoScan.DataSource.PointCloudData
task.projection.crs = PhotoScan.CoordinateSystem("EPSG::4326")

network_task = PhotoScan.NetworkTask()
network_task.name = task.name
network_task.params = task.encode()
network_task.frames.append((chunk.key, 0))
path = 'projects/project.psx'
client = PhotoScan.NetworkClient()
client.connect("127.0.0.1")
batch_id = client.createBatch(path, [network_task])
client.resumeBatch(batch_id)
Best regards,
Alexey Pasumansky,
Agisoft LLC

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #2 on: July 17, 2018, 10:09:39 PM »
Interestingly enough, I used the code as provided:

Code: [Select]
task  = PhotoScan.Tasks.BuildDem()
task.source_data = PhotoScan.DataSource.PointCloudData
task.projection.crs = PhotoScan.CoordinateSystem("EPSG::4326")

network_task = PhotoScan.NetworkTask()
network_task.name = task.name
network_task.params = task.encode()
network_task.frames.append((chunk.key, 0))
path = 'projects/project.psx'
client = PhotoScan.NetworkClient()
client.connect("127.0.0.1")
batch_id = client.createBatch(path, [network_task])
client.resumeBatch(batch_id)

And received the following error:

Code: [Select]
AttributeError: 'PhotoScan.OrthoProjection' object attribute 'crs' is read-only

Not quite sure what is wrong at the moment.
« Last Edit: July 17, 2018, 10:53:28 PM by ierickson »

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #3 on: July 18, 2018, 12:30:57 AM »
Hello Ian,

Which version of PhotoScan you are currently using?
Best regards,
Alexey Pasumansky,
Agisoft LLC

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #4 on: July 18, 2018, 01:42:17 AM »
Currently using PhotoScan 1.4.2.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #5 on: July 18, 2018, 01:20:34 PM »
Hello Ian,

Seems to be my fault, try this way instead:
Code: [Select]
task  = PhotoScan.Tasks.BuildDem()
proj = PhotoScan.OrthoProjection()
proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")
task.projection = proj
Best regards,
Alexey Pasumansky,
Agisoft LLC

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #6 on: July 18, 2018, 04:03:50 PM »
Nope.

Code: [Select]
task  = PhotoScan.Tasks.BuildDem()
proj = PhotoScan.OrthoProjection()
proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")
task.projection = proj

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-186-d1e16277323f> in <module>()
      1 task  = PhotoScan.Tasks.BuildDem()
      2 proj = PhotoScan.OrthoProjection()
----> 3 proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")
      4 task.projection = proj
      5

AttributeError: 'PhotoScan.OrthoProjection' object attribute 'crs' is read-only

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #7 on: July 18, 2018, 04:07:52 PM »
The really bizarre part of this is that the 'crs' parameter has a default value of Lat/Long WGS84:

Code: [Select]
orthoProj = PhotoScan.OrthoProjection()

orthoProj.crs
Out[188]: 2018-07-18 07:04:30 <CoordinateSystem 'WGS 84 (EPSG::4326)'>

orthoProj.crs.wkt
Out[189]: 2018-07-18 07:04:33 'GEOGCS["WGS 84",DATUM["World Geodetic System 1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9102"]],AUTHORITY["EPSG","4326"]]'

But, when creating the BuildDem task, and then encoding the value, the result is an empty dictionary:

Code: [Select]
task = PhotoScan.Tasks.BuildDem()

task.projection = PhotoScan.OrthoProjection()

task.source_data = PhotoScan.DataSource.DenseCloudData

task.encode()
Out[193]: 2018-07-18 07:06:35 {}

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #8 on: July 18, 2018, 04:46:57 PM »
I should also add, that I have tried to encode the NetworkTask.params member with a dictionary that I created that matches that of the successful encoding when using the UI:

Code: [Select]
network_task = PhotoScan.NetworkTask()

network_task.name = "BuildDem"

params
Out[243]: 2018-07-18 07:37:50
2018-07-18 07:37:50 {'projection': {'crs': 'GEOGCS["WGS 84",DATUM["World Geodetic System 1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9102"]],AUTHORITY["EPSG","4326"]]',
2018-07-18 07:37:50   'radius': 1,
2018-07-18 07:37:50   'surface': 0,
2018-07-18 07:37:50   'transform': [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]}}

network_task.params = params

network_task.frames.append((PhotoScan.app.document.chunks[0].key, 0))

client = PhotoScan.NetworkClient()

client.connect("127.0.0.1")
2018-07-18 07:38:37 connected to 127.0.0.1:5840

batch_id = client.createBatch(path, [network_task])

network_task.encode()
Out[249]: 2018-07-18 07:39:17
2018-07-18 07:39:17 {'projection': {'crs': 'GEOGCS["WGS 84",DATUM["World Geodetic System 1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9102"]],AUTHORITY["EPSG","4326"]]',
2018-07-18 07:39:17   'radius': 1,
2018-07-18 07:39:17   'surface': 0,
2018-07-18 07:39:17   'transform': [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]}}

However, the call to 'createBatch' produces the following encoding (notice the projection params are encoded as an array as opposed to a dictionary):

Code: [Select]
{
"id": 3,
"method": "createBatch",
"params": {
"path": "some_project.psx",
"tasks": [{
"frames": [
[0, 0]
],
"name": "BuildDem",
"params": {
"projection": ["radius", "surface", "transform", "crs"]
},
"supports_gpu": false
}],
"username": "someuser"
}
}
« Last Edit: July 18, 2018, 05:38:00 PM by ierickson »

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #9 on: July 19, 2018, 04:04:25 PM »
Hello Ian,

It seems that you are doing something wrong.

The parameters (params) of the NetworkTask should be defined either via .encode method of PhotoScan.Task() or using dictionary assignment:
Code: [Select]
proj = PhotoScan.OrthoProjection()
network_task.params["projection"] = proj

Also note that default parameters are not passed when the task is sent to the server.
Best regards,
Alexey Pasumansky,
Agisoft LLC

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #10 on: July 19, 2018, 04:39:32 PM »
Alexey,

Thanks for your patience - and I agree - I'm doing something wrong.

I've tried the following script (entered into the PhotoScan console window):

Code: [Select]
proj = PhotoScan.OrthoProjection()

network_task = PhotoScan.NetworkTask()

network_task.name = "BuildDem"

network_task.params["projection"] = proj

network_task.params["source_data"] = PhotoScan.DataSource.DenseCloudData

network_task.frames.append((PhotoScan.app.document.chunks[0].key, 0))

batch_id = client.createBatch(path, [network_task])

When serialized, over the network (as I'm using Wireshark to examine the TCP content), the createBatch request produces the following JSON:

Code: [Select]
{
"id": 7,
"method": "createBatch",
"params": {
"path": "some_photoscan_project.psx",
"tasks": [{
"frames": [
[0, 0]
],
"name": "BuildDem",
"params": {
"projection": {
"crs": "[\"\",[\"\"],UNIT[\"\",0]]",
"radius": 1,
"surface": 0,
"transform": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
},
"source_data": 1
},
"supports_gpu": false
}],
"username": "someuser"
}
}

As far as I can tell, everything about this JSON message is correct with the exception of the crs member of the projection parameter (see the original post in this thread to inspect the message as sent from the UI). That said, I don't know how to properly set the crs member of the PhotoScan.OrthoProjection class to create a properly formatted JSON message when using createBatch.

For the record I've tried:

Code: [Select]
proj = PhotoScan.OrthoProjection(crs=PhotoScan.CoordinateSystem("EPSG::4326"))

and

Code: [Select]
proj = PhotoScan.OrthoProjection(PhotoScan.CoordinateSystem("EPSG::4326"))

and

Code: [Select]
proj = PhotoScan.OrthoProjection()
proj.crs.init("EPSG::4326")

Which all produce the same JSON message as above (i.e. without a properly formatted crs parameter). Try as I might, under no circumstances can I get the BuildDem network task to properly serialize the createBatch JSON message as it does in the UI. Any help or assistance in that regard would be a life-saver at this point.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #11 on: July 19, 2018, 05:22:05 PM »
Hello Ian,

Have you tried this one?
Code: [Select]
proj = PhotoScan.OrthoProjection()
proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")

Best regards,
Alexey Pasumansky,
Agisoft LLC

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #12 on: July 19, 2018, 05:30:24 PM »
Alexey,

Yes, but that particular statement is invalid:

Code: [Select]
proj = PhotoScan.OrthoProjection()

proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-291-442c63dcb33d> in <module>()
----> 1 proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")

AttributeError: 'PhotoScan.OrthoProjection' object attribute 'crs' is read-only

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14816
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #13 on: July 19, 2018, 05:50:18 PM »
Hello Ian,

I cannot reproduce this issue in PhotoScan Pro 1.4.3 (should be the same in 1.4.2).

Does it happen if you open a new PhotoScan Pro instance and just input these two lines to the Console?
Code: [Select]
proj = PhotoScan.OrthoProjection()
proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")
Best regards,
Alexey Pasumansky,
Agisoft LLC

ierickson

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: BuildDEM NetworkTask and the OrthoProjection parameter
« Reply #14 on: July 19, 2018, 06:26:21 PM »
Alexey,

Success! So apparently, I had to shut down the server, node and PhotoScan app. Restart them all, and then try again. That worked. The Python API is now creating the exact same JSON batch message as the UI. I'm not entirely certain why a restart resolved the issue, but I'm happy it did.

For future reference the Python code was:

Code: [Select]
proj = PhotoScan.OrthoProjection()
proj.crs = PhotoScan.CoordinateSystem("EPSG::4326")

network_task = PhotoScan.NetworkTask()
network_task.name = "BuildDem"
network_task.frames.append((PhotoScan.app.document.chunks[0].key, 0))
network_task.params["source_data"] = PhotoScan.DataSource.DenseCloudData
network_task.params["projection"] = proj
path = "some_photoscan_project.psx"

client = PhotoScan.NetworkClient()
client.connect("127.0.0.1")
batch_id = client.createBatch(path, [network_task])

The resulting createBatch JSON message was:

Code: [Select]
{
"id": 1,
"method": "createBatch",
"params": {
"path": "some_photoscan_project.psx",
"tasks": [{
"frames": [
[0, 0]
],
"name": "BuildDem",
"params": {
"projection": {
"crs": "GEOGCS[\"WGS 84\",DATUM[\"World Geodetic System 1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9102\"]],AUTHORITY[\"EPSG\",\"4326\"]]",
"radius": 1,
"surface": 0,
"transform": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
},
"source_data": 1
},
"supports_gpu": false
}],
"username": "someuser"
}
}

Thank you!

Ian