8000 EUR computer build recommended hardware

[igor73]

Hi

Could i get some advice on a 8000 EUR computer build please?    Will also be used for rendering with Octane render so Nvidia CUDA cards a must.  Minimum 6GB memory on the card so i guess at the moment best option is Titan or Titan Z? 

Would like to fit 256GB of RAM if possible  and plenty of CPU power.  So what parts would you buy for this budget?

[dtmcnamara]

Supermicro 7048GR-TR $1800
256GB DDR4 2133 ECC (16x16GB) $3096
Xeon E5-2640V3 (2) $2000 (CPU is up for debate really depends on if you want fewer faster cores, or more cores at a lower speed)

GPU all depends on what your needing/your programs are needing
Quadro K5000 $1700
EVGA TItan Black $1000-1200
Xeon Phi $300-$$$

Hope this is a good start

[driftertravel]

I can tell you what I'm building for rendering, design, video editing and photoscan. Processors and motherboard are in the mail, yay! The idea was to get as good a deal as possible on parts, even if they weren't totally cutting edge, as I'll probably build a dual skylake xeon next year when the good stuff has been released including PCIE 4.0 graphics cards. You'll have to scour the interwebs for good pricing though. I believe you pay a premium in Europe for parts, my build is less expensive than yours, and I'm harvesting parts from an old build (video cards, RAM, case, storage) so I'm taking that into consideration in planning my new build in order to save some cash.

Price: ~6k USD

Motherboard: ASUS z9pe-d8 (dual xeon, 7x16 pcie slots, 80x pcie 3.0 lanes, ssd caching, up to 256gb ram) - $400 (ebay)
Processor: 2x Xeon e5-2687W 3.4ghz 8 core - $2000 for both (ebay)
RAM: 64gb compatible non-ecc ram (will probably be the first upgrade, but it was salvaged from current build, max compatible for the board unless you go ecc) - ~$600
Video Cards: 4x Nvidia GTX 780TI, these cards rock the titan for cuda processing and are half the price, apparently still better in some aspects for CUDA than the new 980, so grab 'em used (though if you're buying new I believe the 980, even with much fewer cuda cores, is probably slightly better for photoscan). - ~$1600 for four (ebay)
PSU: 1500 watt corsair psu - ~350
Storage: 4x3gb 7200rpm platter drives, 1x 120gb ssd for ssd caching off main work drive, 1x 1tb ssd for system drive (sad, no m.2 on this mobo) - ~$1200
case and 7 noctua fans (2x cpu, 5x case) - ~$300
xbox 360 controller: (for dark souls, 'cause that's how I roll) - $20

Some thoughts.

I went with a higher clock speed, lower core processor as some tasks I do aren't multithreaded. If you are only planning on photoscan you might be better served by more cores and less clock speed.

Go with high end gamer graphics cards, not the tesla cuda cards. The performance isn't that different, but the price certainly is, spend that money on processor as only one stage of photoscan uses CUDA right now...

RAM requirements are directly related to the number of images and the quality of the dense cloud you want to build as far as I can tell. Figure out in advance what you're really going to need. I'm processing ~144 18mp images per scan, even at ultra high 64gb is enough for me. If you're doing massive reconstructions you'll need to push it. Not many motherboards allow dual processors and 4x16 pcie 3.0 graphics, which is pretty important for me. A lot of my rendering is CUDA optimized. Octane is CUDA based, if you use keyshot or some other cpu based renderer you could drop a few of the graphics cards in favor of even more cpu clock speed and cores, but the 16 core xeon v3 is $3000+ and 2.3ghz, which is quite a trade off if you're not doing something multithreaded.

That's my two cents anyway. Someone who knows photoscan better than I can tell me I'm wrong somewhere here.

[igor73]

Thank you both for your advice. 

The new 2011-3 standard requires DDR 4 memory right?   Very hard to get the DDR5  16gb sticks here and they are expensive.  I have not found 32GB sticks so if i want 256GB i need to order from US or maybe UK. 
How important is memory speed?  Is it worth spending money on 2400 mhz men? 

What about the new i7 8 core extreme edition or the 6 core one.  Also using 2011-3 .  Can you use 2 i7 CPU or only one? 

I already have a decent computer built  last year with a single i7 6 core and 64 GB RAM and Titan  but i am doing massive 5000 images processing now and it runs 24/7 so i can't do other work while it runs.    I am using chunks so getting by with 64GB. Not sure if its worth going 256gb ?   So i want to build a second computer.  Hard to salvage part as it had DDR3 memory 8x8GB. 

I need 6GB for textures so it has to be a Titan.  Quadro to expensive. 

Also not decided if I'm gonna switch to Vray instead of Octane.  Interior renders are crazy slow to cook on octane.  So that is a decision to.  If i switch to Vray i think i should max out CPU performance instead of GPU. 

How many GPU can you fit on the ASUS card you recommended? 

[dtmcnamara]

One thing to note, some processes require a Quadro/Firepro card, Quad buffer stereo for one . Might be worth getting a single Quadro and then multiple Nvidia cards if other programs require this.

[dtmcnamara]

I thought DDR4 was going to be more than DDR3 but when you are pricing ECC memory you will find that they are exactly the same price, at least here in the US, and in some packs DDR4 is even cheaper than 3.

From what I have found amount > speed...The slowest DDR3 and DDR4 will write faster than any SSD out there right now. When RAM fills up it has to write to a temp file on your system, if you can limit the amount of times the RAM needs to dump to this temp file you will see faster renders. The only way to reduce this is to have more RAM.

In order to run 2 CPUs you will have to run Xeon CPUs.

If you go with Vray look into the Xeon Phi cards. I have seen some amazingly fast renders on Vray using the Phi coprocessors.

[igor73]

Will Agisoft benefit from  Xeon Phi ?  You have to use Linux? 

[driftertravel]

I've heard that photoscan will not benefit from xeon phi, though that's just what I've heard. Programs need to be specially written to utilize it as far as I know. It would be super cool if it did though, that extra processing power would be worth the price tag of those things, though they are limited by the onboard RAM which is only 8gb I believe, so that's a consideration. That asus board and the newer version (if you're going dual xeon v3 you need the new one) allow for 4 pcie x16 dual slot graphics cards. I think that's the most you'll find nearly anywhere... Most CUDA optimized programs (like the dense cloud stage of photoscan and octane) scale very nicely with multiple GPUs, I noticed a marked increase when I went from 2-3 gpus and then 3-4 was even more impressive, and they were running on 8x pcie lanes on my old machine. That was the i7 6 core (older one) and I think you'll appreciate the power of xeon, one 16 core or two 8 or 10 core processors will chew right through most tasks in a timely manner. The new v3 I believe requires ddr4, which isn't really all that much faster as far as I know, but the power consumption is lower.

[igor73]

Are there any benchmarks available for Agisoft?  Roughly if you would guess how much faster would a system with 2x 8 core xenons and 3x GPU be than my current 6 core i7 with a single titan.  Are we talking 3-4x speed increase?

[dtmcnamara]

I've heard that photoscan will not benefit from xeon phi, though that's just what I've heard.

As of version 1.0.0 Agisoft will run on a Xeon Phi coprocessor....how well nobody knows yet. I have one coming in soon and will be testing this out.

[driftertravel]

Great news on the Xeon Phi, curious if it uses the co-processor for all stages or only the first stage of the dense cloud computation like CUDA GPUs. I will very much look forward to your tests.

Edit: Oh, and I'd wait for the new maxwell based Titan, it supposedly comes out in march, over 3k cuda cores, more efficient cuda architecture and lower power consumption, though the price is supposedly going to be greater than the last generation of titans. Two or three of those would be monstrous. And cost as much as my car. Luckily I don't drive much

[Alexey Pasumansky]

Hi,

Even the support has been added, the output performance (based on the user reports) of Xeon Phi as OpenCL device wasn't significant.

[dtmcnamara]

Hi,

Even the support has been added, the output performance (based on the user reports) of Xeon Phi as OpenCL device wasn't significant.

Do you have numbers from the Phi card, just wondering what it benchmarked

[igor73]

Yeah i think i will just through in a cheap GPU for now and wait for the next gen Titans. 

Prpbably will go for total of 128 GB as 256 would get to expensive given the price and availability of 32 GB sticks.  Are three any mother boards with 16 memory slots btw?

[igor73]

I see now dtmcnamara  has recommended at 16 slot MOBO above.  Any other options that may be cheaper?