PhotoScan swapping to disk with plenty of RAM available

[KevinCain]

I've tried several times to generate a mesh from a dense point cloud derived from 172 6MP photos -- each attempt, even with 'low' quality specified, halts midway with an 'out of memory' error.

I see the PhotoScan mesh generation process is using only 25GB (at peak) of 128GB RAM available of the system, but more than half the swap space of 4GB is consumed. 

My questions:

- Is the 'out of memory' error really swap related?
- Why is PhotoScan swapping to disk at all when there is so much RAM available?

I'm running v. 1.1.6 build 2038 (64 bit) under Ubuntu linux 15.04.

Thanks for any advice or reports of similar experiences.

[Marcel]

I've tried several times to generate a mesh from a dense point cloud derived from 172 6MP photos -- each attempt, even with 'low' quality specified, halts midway with an 'out of memory' error.

The Mesh quality setting has no influence on the memory use. For arbitrary Mesh construction, Photoscan always builds the mesh at the maximum number of faces. Only after the mesh has completely been build, will it decimate to the number of faces of your target setting (Low, Medium,etc). That means that if you have a Dense Cloud with a lot of points, it will always take a lot of memory to mesh because it first build the mesh at maximum resolution. (I am not sure if it works the same for a Heighfield mesh, I think so).

It's currently not possible to reduce the number of points in the Dense Cloud, so if your Dense Cloud has too many points you are stuck (only solution is to rebuild the DC at a lower setting). I've suggested Dense Cloud filtering as a solution: http://www.agisoft.com/forum/index.php?topic=1813.0


I have no clue why you are running out of memory. Could it be that Linux restricts the amount of memory for a process? On a Windows computer Photoscan has no trouble allocating all memory in the system.

Are you sure that it is Photoscan is using the swap space and not some other program or the OS itself?

[James]

Another option is to use the 'PS110_split_in_chunks_dialog.py' script from http://wiki.agisoft.com/wiki/Python to divide your chunk into multiple chunks with smaller bounding boxes to mesh each one individually before merging into one chunk.

I believe this should help because reducing the size of the bounding box should reduce memory requirement even if the number of points remains the same (points outside the box are just ignored)... i think...

Then the problem is that you will be left with several meshes with seam lines between them that you have to either live with or find some way to 'close' in external software.

Also your project file will be huge if each chunk contains the full point cloud with differently sized bounding boxes.

I haven't tried the script myself, but have had experience where reducing the bounding box size enabled me to mesh a pointcloud that i couldn't previously due to memory constraints.

Before messing with the script maybe worth just manually resizing the bounding box to a smaller area to see if you get any joy.

Sorry i don't have any suggestion about what photoscan is doing or not doing with your other 100GB memory!

[KevinCain]

Thanks James and Marcel for your helpful posts and your insight.

Marcel, thanks for clarifying that the output mesh process' memory footpring does not relate to input cloud density.  I am certain that PhotoScan is responsible for the swap file -- I have monitored swap on this machine closely.

James, I will have a look at splitting the scene into smaller chunks and then merging.  Am I right to be concerned about the boundary between separately process chunks showing artifacts in the final merged output?

[KevinCain]

The Mesh quality setting has no influence on the memory use. For arbitrary Mesh construction, Photoscan always builds the mesh at the maximum number of faces. Only after the mesh has completely been build, will it decimate to the number of faces of your target setting (Low, Medium,etc). That means that if you have a Dense Cloud with a lot of points, it will always take a lot of memory to mesh because it first build the mesh at maximum resolution. (I am not sure if it works the same for a Heighfield mesh, I think so).

To ground the point Marcel is making, my experience is below.  As in my originating thread, when I compute a point cloud in PS at maximum density, any quality mesh extraction attempt fails.  Starting from a less dense point cloud, I am able to successfully output a mesh:

Medium density point cloud -> high density mesh:  peak ram ~9.7GB, resulting in ~3.5M faces

This seems to indicate that the problem is that the system is actually running out of RAM.

[wojtek]

Marcel, thanks for clarifying that the output mesh process' memory footpring does not relate to input cloud density.  I am certain that PhotoScan is responsible for the swap file -- I have monitored swap on this machine closely.

No, it is exactly the other way around. Mesh generation setting has no effect on processing time/memory requirements, it is simply the value of decimation that is applied afterwards. Time and Memory requirements of mesh generation process are fully dependant on your cloud density (the denser it is the high the memory use).

[James]

James, I will have a look at splitting the scene into smaller chunks and then merging.  Am I right to be concerned about the boundary between separately process chunks showing artifacts in the final merged output?

Yes, i believe that when you merge the chunks the meshes will remain as disconnected objects so there may be gaps between them. If any gap it should at least be very fine since the mesh will be bulit right up to the edge of the bounding box, and the script should ensure that there are no gaps between adjacent bounding boxes.

The only way to properly close the gaps though would be in an external program. I use 3dreshaper for that sort of thing but it costs more than photoscan pro!

[Alexey Pasumansky]

To avoid the gaps between meshes (that could appear with 1 pixel thickness when script is used), the bounding box can be extended to provide small overlap between the regions.