Agisoft Metashape
Agisoft Metashape => Bug Reports => Topic started by: feiko.lai on February 13, 2017, 06:28:34 AM
-
Environment:
AgiSoft 1.2.6
Xeon dual CPU
128GB ram
Win 10 64bit
AMD R390x
I have a dataset with >5k images so I split it into 2 chunks
The first chunk with 3.7k images and it had no problem all the way through alignment-> dense cloud generation -> mesh generation
and then we turned to the second chunk with 2.8k images and which were taken by same camera and settings as the 1st chunk, but we always failed at "build dense cloud" step (High quality, 24Mpixel multi-spectral, tie points=12Millions), with such log:
2017-02-12 01:49:45 filtering depth maps... done in 76.53 sec
2017-02-12 01:51:02 preloading data... done in 63.463 sec
2017-02-12 01:52:06 accumulating data... done in 19.836 sec
2017-02-12 01:52:30 building point cloud... done in 3.238 sec
2017-02-12 01:53:17 Finished processing in 57848.2 sec (exit code 0)
2017-02-12 01:53:17 Error: bad allocation
and then we upgraded AgiSoft to v1.3.0 and tried again, but we got same failure and log:
2017-02-13 09:47:47 filtering depth maps... done in 42.662 sec
2017-02-13 09:48:30 preloading data... done in 23.542 sec
2017-02-13 09:48:53 accumulating data... done in 9.791 sec
2017-02-13 09:49:05 building point cloud... done in 2.018 sec
2017-02-13 09:52:02 Finished processing in 75365.2 sec (exit code 0)
2017-02-13 09:52:02 Error: bad allocation
It looks like dense cloud generation was finished but failed at the after steps.
-
Hello feiko.lai,
Have you got the project file saved with the depth maps stored in the project data even after failed operation?
We can try to reproduce the issue on our side, if you can provide the project with the depth maps for our internal investigation.
-
Sorry Alexey, we didn't save project with depth map, do you recommend us doing so? We can enable it. While at the same time, any possible solution for us to try to deal with this problem?
Thanks
-
Hello feiko.lai,
It would be helpful, if we have a dataset or the project with the depth maps that we can use to reproduce the issue, without that it's hard to understand why the processing fails.
-
Alexes,
We need to provide to project files with depth map enabled?
how to share it with your side as the size is quite large.
-
For very, very large files (unlimited size), you could use https://infinit.io/
Of course you would compress the files to a ZIP or RAR file first.
Infinit.io is closing down on 31 March though. I just checked and the program is still available for download.
-
Hello feiko.lai,
We either need the project with alignment results and image set or project with the depth maps, so that we can use them for the dense cloud generation, in the second case the original images are not required.
To reduce the project size you can remove the thumbnails, mesh model, DEM and orthomosaic, if any of these products are present in the project data.
If the upload option suggested by rossnixon doesn't work for you, let me know and we'll provide you an account on our ftp server.
-
Alexey, I PM you with a download link of project files with depth map, please check it
-
download link
- removed
-
Hello feiko.lai,
Thank you, we are downloading it now.
-
Hello feiko.lai,
We have managed to generate the dense cloud from the provided depth maps, the memory consumption peak was about 60% on 256 GB RAM machine, however, we've used single node in a network processing mode.
-
Thank you Alexey,
So it means we really don't have sufficient memory. But it is weird that this chunk has much less images than the other chunk which we handled it without any problem on the same machine. Can I safely conclude that: the number of images doesn't necessarily mean more memory required and the contend of images matters as well?
-
Hello feiko.lai,
The memory consumption also depends on the model complexity - shape of the geometry. So not only number of images matters.
To reduce the memory consumption, I can suggest to run the processing in the network mode with fine level task distribution, even on one node, in this case the complete dense cloud wouldn't be stored in the memory at once (in our tests we've disabled the fine level task distribution, so it should be similar to local processing).
-
Thank you Alexey,
So network mode (even with only local node) is a recommended way to reduce memory consumption.
I have one more question about network mode, if I add one networked machine for acceleration, for example, one with 3 powerful graphic cards, then do I still need to install large amount of ram on it? Or less ram (e.g. 64GB) is enough ?
-
Hello feiko.lai,
For some steps that support fine-level distribution, the memory consumption in network mode would be lower, when it's not require to keep all the data in memory.
As for the GPU node, in the version 1.3, you can set the "Capability" for the nodes: CPU, GPU or Any. If you set GPU capability to the node, it will mean that the node would be used only for GPU-supported tasks (image matching and depth maps calculation), while other tasks should be performed on other nodes.