Multiple gpus

[terry]

I see a lot of people saying they have pcs with 1 or 2 gpus installed, but does anyone have a multi gpu setup as in 4 or 8 gtx 580’s or something similar, and if so is there a good drop in processing time that justified the cost?

[Wishgranter]

For every GPU need 1 CPU core, but hyperthreaded CPUs need disable 2 cores in Photoscan.

Scaling is fast liner becasue of good programing routines of Photoscan.

It is possible to use "UNLIMITED" amount of GPUs under LINUX.......

[Matt]

The GPU's provide an increase in processing the depth reconstruction phase only and are not used in the mesh generation. Mesh generation is the sticking point as photoscan needs lots of ram.

[r0xx]

Is this information still up to date? Is more GPU only needed for dense cloud generation? Or also for image alignment/texture generation?

[Wishgranter]

The problem with GPUs to be used for other parts of process is that PCI-E is a limiting factor (latency) and GPU memory... so its mostly faster to process it on CPU side..... and there is problem with alogs  that they can be use massive multicore (CPU - GPU ) systems

[igor73]

Just installed 2x Titan X 12gb cards.  Bought them for rendering really but it sure is sweet to speed up dense cloud generation to.  Should i disable SLI while using Agisoft or does it not matter?  Never used dual cards before. 
Recomendation is to disable one core per GPU.  Does that mean hyper thread cores or actuaö physical cores?  I guess hyper thread cores right? 

[Marcel]

SLI is not needed for Photoscan (SLI is more for rendering realtime computer graphics, Photoscan is using OpenCL).

I have disabled all my cores and let the GPU do all the work. Compared to the GPU, the performance of the CPU is really low, especially for your dual Titan setup.

With all CPU cores disabled you can also run multiple scans at the same time without much slowdown. When one scan enters the Depth Map Filtering phase (which is CPU only) you can start the Dense Cloud build for a second scan.

[igor73]

Thanks Marcel.  Only for the depth map generation the GPU is used even if i disable all cores right?

[Wishgranter]

Hi Igor can post some benchmark data ?

[igor73]

I could if i had the sample files.

Both download links to the sample files are broken  http://www.agisoft.com/forum/index.php?topic=651.0

[igor73]

Did a benchmark now.  Device 1 performance: 1004.91 million samples/sec (GeForce GTX TITAN X).   Is that in line with expectations for this card?  Its the EVGA superclocked version so running at about 1280mhz on boost.  I noticed GPU load was only around 60% during processing though on both cards. 

Nvidia Titan X Benchmark



The building sample data was used http://www.agisoft.com/downloads/sample-data/

I uses the following settings: (Same as Anand Tech but tie points set to 0 and CPU 0/12)

## Align Photos: High, disabled, 40000.  0 tie points
## Build Dense Cloud: Medium, Aggressive
## Build Model: Arbitrary, Dense, Interpolation=Enabled, 0 face count
## Build Texture: Generic, Mosaic, 4096, 1


System info
Microsoft Windows 7 Professional
Processor   Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
64GB RAM
2x Titan X 12gb cards.  SLI enabled


Align photos

Finished processing in 294.814 sec (exit code 1)

Dense cloud
All CPU cores disabled in agisoft prefrences.  0/12

finished depth reconstruction in 80.37 seconds
Device 1 performance: 1004.91 million samples/sec (GeForce GTX TITAN X)
Device 2 performance: 994.448 million samples/sec (GeForce GTX TITAN X)
Total performance: 1999.36 million samples/sec
Generating dense point cloud...
selected 50 cameras in 0.22 sec
working volume: 1755x2763x1220
tiles: 1x1x1
selected 50 cameras
preloading data... done in 0.325 sec
filtering depth maps... done in 43.934 sec
preloading data... done in 7.956 sec
accumulating data... done in 1.201 sec
building point cloud... done in 0.608 sec
5706624 points extracted
Finished processing in 137.235 sec (exit code 1)


Mesh

Finished processing in 144.503 sec (exit code 1)


Build Texture
9056814 faces extracted in 32.709s
Calculating vertex colors...
processing nodes... done in 0.016 sec
calculating colors... done in 20.289 sec
Finished processing in 144.503 sec (exit code 1)
Parameterizing texture atlas...
Packing 2117 charts...
Blending textures...
blending textures... ************************************************** done in 49.94 sec
postprocessing atlas... done in 0.117 sec
Finished processing in 154.71 sec (exit code 1)