Messages

31

General / Re: 8000 EUR computer build recommended hardware

« on: February 16, 2015, 10:09:28 PM »

Completely agree with marcel.

In my tests, clock speed seems to be a more important aspect for photoscan performance than anything else, those results included overclocking the multiplier and raising the bclk, which gave me much better performance with just a couple hundred extra mhz in turbo.

If you're going for a dual system you shouldn't sacrifice clock speed for cores or anything really. If you can't get a dual system with the fastest possible clock speed (and preferably the unlocked ones for overclocking) then it's probably not worth the extra cost to go dual. That's why I got the clock optimized chips, they were the best clock I could find with more than four cores.

That being said, the dual xeon rig is a little more than 4x faster than my old i7-3930k rig at the same clock speed, with the same RAM and video cards, so it was worth it for me. If you can find a good deal on E5 V2 ES or QS unlocked xeon chips (which should be way cheaper than the production ones) it doesn't seem like a bad way to go.

There's also the fact that OP and I will be using our rigs for CUDA/CPU rendering, which is something one should take into consideration as well, this is not a dedicated photoscan rig, and as such some compromises/extra cost was acceptable to me.

32

General / Re: 8000 EUR computer build recommended hardware

« on: February 16, 2015, 05:30:55 AM »

Whoops, I was looking at the old benchmark instead of the new one with the ram clocked properly. The 8 core benchmark is correct, at 8.25min, but the newest 16 core benchmark was actually the one below @ 4.18min, so very nearly double the speed. This is with everything else being the same except the enabled core count.

33

General / Re: 8000 EUR computer build recommended hardware

« on: February 16, 2015, 04:55:34 AM »

I should be running the same benchmark as anandtech, as they posted what their settings are in the benchmarking thread.

So, these results are slanted towards the single processor run for a number of reasons, I didn't want to physically pull a processor, so all I did was disable half the cores in my bios. This had a number of effects that you wouldn't see in a single processor setup since two processors were still active:

1. The system still had access to the 80pci lanes from two processors instead of half that from a single processor rig. This means the 4 graphics cards can still run at PCIe x16 instead of the x8 speed they would have to run on a single processor system, in fact, less cores on the same pci lanes let the graphics cards run a little faster it seems.

2. The system still had access to 2 banks of ram with two memory controllers, again, wouldn't happen in a single processor system.

3. Two 25mb L3 Caches.

4. Probably some stuff I'm forgetting.

I think the size of the dataset is a limitation here too, it doesn't even push my system to full turbo speeds, so, functionally, the cpus were clocking 300-400mhz higher when only 8 cores were enabled, as agisoft pushed harder when it was lacking the cores it wanted.

Still, more cores = more better, as the image shows.

TL;DR

8 Cores - 8.25min
16 Cores - 5.98min

34

General / Re: Best F-stop?

« on: February 16, 2015, 01:44:37 AM »

Yeah, if you have the light and a tripod then go max DOF before refraction. You can look up this data for your lens.

35

General / Re: Sparse Cloud: very few Points

« on: February 16, 2015, 01:43:20 AM »

I don't actually know if this is correct, but in my experience based on your three options:

1. Medium settings - do this! Especially if your target is game ready level assets, unless your target has very small details that you need to pick up.
2. Chunks - not going to help overly much
3. Jpegs - I've found that 8bit tiffs work just as well as 16bit quality wise... not sure about jpegs b/c of the compression artifacts.

Just my .02

36

General / Re: 8000 EUR computer build recommended hardware

« on: February 16, 2015, 12:32:09 AM »

@ igor73:

Well, I never ran the tests with 1 CPU but the Anadtech results from one of my cpus was something like 12min. I suppose I could disable one CPU and re-run the tests with all else being equal if you're interested. Anadtech's results seem to stand up, with photoscan seeing nearly double the performance from one chip to two is not unreasonable, as the program loves the extra cores. I would caution against the 16core v3s though, the clock is like 2.3ghz, which would suck on photoscan for the money.

@stihl:

The dual xeon v2s clock higher than the v3s at the same price point, plus it's easy to get the unlocked ES chips used and cheap, both of which were important to me, as spending $5k on cpus wasn't an option, and overclocking these to 3.5/4.0Ghz (turbo) is a real possibility. The board supports faster ram than 1600 actually, I'm going up to 2133 this week to test it out, and the board supports overclocking the ram over that with the newest bios... so that could help.

I am also looking to use this box for CUDA and CPU rendering, and high poly modeling (zbrush is 64bit now, yay! a functionally unlimited polycount) hence the dual xeons and the 4x GPUs. The 780ti has the most cuda cores for the money right now AFAIK. The 980 has far fewer cores, but is supposedly more efficient (certainly more power efficient). I had 3 of the four GPUs already, so I just got another one off ebay. Spending ~$1600 (4x$400 off ebay) for that much CUDA power is a pretty good deal when you start looking at TITAN or QUADRO specs.

I needed a computer now to do some stuff, and couldn't really wait for the good skylake xeons to be released sometime next year probably, plus even then I'd want to wait to pick 'em up used. I have no idea if clock speed will even improve in the Skylake chips, as it seemed to actually fall with Haswell (though IPS improved, so there's that).

As to RAM. the board only supports 64gb of non-ecc ram (to 768 or some such business if using ECC), and I couldn't really afford a crap-ton of fast ECC ram at this point, hence the 64gb 2133, which I'll try to overclock to 2400 or so. My use case is doing a lot of smaller data sets (144 16mp images on ultra high) so I never touch the full 64gb. DDR4 isn't that much faster (yet) by the time DDR4 3000mhz sticks are cheap I'll be building a skylake rig...

The main downside is that the system uses a lot of power when run full out, at 100% utilization I'm pushing 1500w (in normal usage its nowhere near that, even with photoscan which is a pretty intensive program). Just to be careful I picked up a 1600w sine wave battery backup power regulator and installed a new 30amp circuit in the office.

Those test results are the standardized test that Anadtech is doing, just so I could compare the build to others. It's twice as fast overall as the top i7-5xxx even with the slow ram, and twice as fast is a pretty big deal if you have 300 data sets to run. If I hadn't been careful with my buys and gotten all the parts used/recycle from my last computer that I popped the mobo on, it would have been a very expensive rig, but overall the price/performance ratio is pretty good IMHO.

Plus, I can play Dark Souls 2 @ 8k downsampled to 4k at way over 60fps

37

General / Re: 8000 EUR computer build recommended hardware

« on: February 15, 2015, 07:34:31 AM »

Built the system I mentioned in the prior post today, I posted this in the performance thread, so sorry for the cross-post, but I thought it might interest you.

Just finished the build tonight, so optimizations are still a ways off, but:

System:
ASUS Z9PE-D8 WS
2x  Xeon 2687W ES V2 (3.2ghz base, 3.8 turbo)
RAM 64GB Corsair Vengeance 1600MHZ
4x GTX 780 TI 3GB Video
2x Samsung 850 EVO SSD in RAID0
2x Seagate 3TB 7200RPM in RAID0
2x Seagate 3TB 7200RPM in RAID1
2x WD Black 1TB 7200RPM in RAID1

Building Test #1 (0 CPU Cores enabled):
Stage 1 Alignment:
Matching - 2 min 22 sec (142 sec)
Alignment - 2 sec
Stage 2 Dense Point Cloud:
Time - 25 sec
Stage 3 Model:
Processing Time - 3 min 33 sec (213 sec)
Stage 4 Texture:
UV Mapping - 23 sec
Blending - 19 sec

Total Time: 7.06 Min (424 sec)

Building Test #2 (24/32 CPU Cores enabled):
Stage 1 Alignment:
Matching - 2 min 25 sec (145 sec)
Alignment - 2 sec
Stage 2 Dense Point Cloud:
Time - 29 sec
Stage 3 Model:
Processing Time - 3 min 26 sec (206 sec)
Stage 4 Texture:
UV Mapping - 28 sec
Blending - 21 sec

Total Time: 7.18 Min (431 sec)


So, it looks like the 1600mhz ram is killing me in stage 3, it's only running triple channel, but I'm going to try some quad channel 2133 CAS 11 next week if I can get it to OC properly on this motherboard. I'll post an update if these scores improve. The chips are unlocked, so I may overclock them a mite bit to 3.4/4.0(turbo) or so and see how much it helps. They seem to run about 3.6 when Photoscan is cranking acording to CPU-Z. Not quite double the performance of 1 2687W, but from 12.74min to 7.06 is nothing to scoff at. Also doubles the performance of an i7 4960X, and this rig did not cost twice what that one would (I canabalized some parts from an old build, some things would have changed were I buying this stuff new), but price/performance I'm pretty happy. Plus cinebench R15 scores are 2176 CPU and 60.38fps in OpenGL, so that's nice.

If you (or anyone really) has a sample dataset they want to run through this system to gauge system speeds and see if it's worth it for them to upgrade, let me know. I'd love to run some larger sets through it to see how things hold up. Just let me know your config and how fast it processes the same set so I can get an idea too.

38

General / Re: CPU and GPU benchmarks

« on: February 15, 2015, 07:30:14 AM »

Just finished the build tonight, so optimizations are still a ways off, but:

System:
ASUS Z9PE-D8 WS
2x  Xeon 2687W ES V2 (3.2ghz base, 3.8 turbo)
RAM 64GB Corsair Vengeance 1600MHZ
4x GTX 780 TI 3GB Video
2x Samsung 850 EVO SSD in RAID0
2x Seagate 3TB 7200RPM in RAID0
2x Seagate 3TB 7200RPM in RAID1
2x WD Black 1TB 7200RPM in RAID1

Building Test #1 (0 CPU Cores enabled):
Stage 1 Alignment:
Matching - 2 min 22 sec (142 sec)
Alignment - 2 sec
Stage 2 Dense Point Cloud:
Time - 25 sec
Stage 3 Model:
Processing Time - 3 min 33 sec (213 sec)
Stage 4 Texture:
UV Mapping - 23 sec
Blending - 19 sec

Total Time: 7.06 Min (424 sec)

Building Test #2 (24/32 CPU Cores enabled):
Stage 1 Alignment:
Matching - 2 min 25 sec (145 sec)
Alignment - 2 sec
Stage 2 Dense Point Cloud:
Time - 29 sec
Stage 3 Model:
Processing Time - 3 min 26 sec (206 sec)
Stage 4 Texture:
UV Mapping - 28 sec
Blending - 21 sec

Total Time: 7.18 Min (431 sec)


So, it looks like the 1600mhz ram is killing me in stage 3, it's only running triple channel, but I'm going to try some quad channel 2133 CAS 11 next week if I can get it to OC properly on this motherboard. I'll post an update if these scores improve. The chips are unlocked, so I may overclock them a mite bit to 3.4/4.0(turbo) or so and see how much it helps. They seem to run about 3.6 when Photoscan is cranking acording to CPU-Z. Not quite double the performance of 1 2687W, but from 12.74min to 7.06 is nothing to scoff at. Also doubles the performance of an i7 4960X, and this rig did not cost twice what that one would (I canabalized some parts from an old build, some things would have changed were I buying this stuff new), but price/performance I'm pretty happy. Plus cinebench R15 scores are 2176 CPU and 60.38fps in OpenGL, so that's nice.


Edit: Tweaked some things, including the memory timing which was off. Pulled 4.18min on the test.

39

General / Re: 8000 EUR computer build recommended hardware

« on: February 07, 2015, 11:09:54 PM »

Great news on the Xeon Phi, curious if it uses the co-processor for all stages or only the first stage of the dense cloud computation like CUDA GPUs. I will very much look forward to your tests.

Edit: Oh, and I'd wait for the new maxwell based Titan, it supposedly comes out in march, over 3k cuda cores, more efficient cuda architecture and lower power consumption, though the price is supposedly going to be greater than the last generation of titans. Two or three of those would be monstrous. And cost as much as my car. Luckily I don't drive much

40

General / Re: 8000 EUR computer build recommended hardware

« on: February 07, 2015, 12:20:15 AM »

I've heard that photoscan will not benefit from xeon phi, though that's just what I've heard. Programs need to be specially written to utilize it as far as I know. It would be super cool if it did though, that extra processing power would be worth the price tag of those things, though they are limited by the onboard RAM which is only 8gb I believe, so that's a consideration. That asus board and the newer version (if you're going dual xeon v3 you need the new one) allow for 4 pcie x16 dual slot graphics cards. I think that's the most you'll find nearly anywhere... Most CUDA optimized programs (like the dense cloud stage of photoscan and octane) scale very nicely with multiple GPUs, I noticed a marked increase when I went from 2-3 gpus and then 3-4 was even more impressive, and they were running on 8x pcie lanes on my old machine. That was the i7 6 core (older one) and I think you'll appreciate the power of xeon, one 16 core or two 8 or 10 core processors will chew right through most tasks in a timely manner. The new v3 I believe requires ddr4, which isn't really all that much faster as far as I know, but the power consumption is lower.

41

General / Re: 8000 EUR computer build recommended hardware

« on: February 06, 2015, 10:54:43 PM »

I can tell you what I'm building for rendering, design, video editing and photoscan. Processors and motherboard are in the mail, yay! The idea was to get as good a deal as possible on parts, even if they weren't totally cutting edge, as I'll probably build a dual skylake xeon next year when the good stuff has been released including PCIE 4.0 graphics cards. You'll have to scour the interwebs for good pricing though. I believe you pay a premium in Europe for parts, my build is less expensive than yours, and I'm harvesting parts from an old build (video cards, RAM, case, storage) so I'm taking that into consideration in planning my new build in order to save some cash.

Price: ~6k USD

Motherboard: ASUS z9pe-d8 (dual xeon, 7x16 pcie slots, 80x pcie 3.0 lanes, ssd caching, up to 256gb ram) - $400 (ebay)
Processor: 2x Xeon e5-2687W 3.4ghz 8 core - $2000 for both (ebay)
RAM: 64gb compatible non-ecc ram (will probably be the first upgrade, but it was salvaged from current build, max compatible for the board unless you go ecc) - ~$600
Video Cards: 4x Nvidia GTX 780TI, these cards rock the titan for cuda processing and are half the price, apparently still better in some aspects for CUDA than the new 980, so grab 'em used (though if you're buying new I believe the 980, even with much fewer cuda cores, is probably slightly better for photoscan). - ~$1600 for four (ebay)
PSU: 1500 watt corsair psu - ~350
Storage: 4x3gb 7200rpm platter drives, 1x 120gb ssd for ssd caching off main work drive, 1x 1tb ssd for system drive (sad, no m.2 on this mobo) - ~$1200
case and 7 noctua fans (2x cpu, 5x case) - ~$300
xbox 360 controller: (for dark souls, 'cause that's how I roll) - $20

Some thoughts.

I went with a higher clock speed, lower core processor as some tasks I do aren't multithreaded. If you are only planning on photoscan you might be better served by more cores and less clock speed.

Go with high end gamer graphics cards, not the tesla cuda cards. The performance isn't that different, but the price certainly is, spend that money on processor as only one stage of photoscan uses CUDA right now...

RAM requirements are directly related to the number of images and the quality of the dense cloud you want to build as far as I can tell. Figure out in advance what you're really going to need. I'm processing ~144 18mp images per scan, even at ultra high 64gb is enough for me. If you're doing massive reconstructions you'll need to push it. Not many motherboards allow dual processors and 4x16 pcie 3.0 graphics, which is pretty important for me. A lot of my rendering is CUDA optimized. Octane is CUDA based, if you use keyshot or some other cpu based renderer you could drop a few of the graphics cards in favor of even more cpu clock speed and cores, but the 16 core xeon v3 is $3000+ and 2.3ghz, which is quite a trade off if you're not doing something multithreaded.

That's my two cents anyway. Someone who knows photoscan better than I can tell me I'm wrong somewhere here.

42

General / 35mm focal length

« on: January 24, 2015, 08:48:34 PM »

Hello all, I've noticed that the 35mm focal exif slot is empty for my images. Should photoscan be calculating this and showing it there? Is 35mm focal a exif tag? I'm copying exif data to these tiffs and I'm afraid something is being lost in the conversion. Should I put the crop factor adjusted focal length in the focal length field under Tools>Camera Calibration?

43

Feature Requests / Manually input EXIF data into fields

« on: January 23, 2015, 12:07:52 AM »

I'm using exr files and they get stripped of exif data. It would be great if I could just input focal length/f-stop/iso/35mm focal into the appropriate fields for multiple selected photos. Doesn't seem too hard, but exiftool won't write to .exr and I'm having trouble figuring out a way to batch change the exif data of these files. I'd rather not convert the files, but I suppose I can if necessary. Let me know if there is any other way of accomplishing this. Thanks.

44

General / Re: CPU and GPU benchmarks

« on: January 16, 2015, 05:42:10 PM »

I'm curious, and I thought this thread might be the one to post in. I had a gtx 680 sitting around and popped it in thinking I would get some extra processing power, and I did, but the results were strange. It seems that while dense point cloud reconstruction was indeed slightly quicker (.12x quicker), alignment was slower. I may have mixed something up in the testing, so I thought I'd ask if disabling a CPU core in order to utilize another CUDA card results in such poor performance improvements or if my results seem wonky... Thanks.

45

General / First Scan, suggestions for improvement?

« on: January 13, 2015, 11:33:19 AM »

Hello all, I just began experimenting with photoscan, this is my first attempt. I didn't have much around to scan, so I took off my shoe and did that. 1 man, one camera, das boot

I'm looking for suggestions to improve here. I can see the potential, just want to keep improving. A few things went wrong that I'm aware of, the laces moved as I turned the shoe to get the shots, the photos used were only 10MP, I didn't spend any time cleaning the mesh in zbrush, and only took a few min on the low poly. Nonetheless, here it is for your consideration.

Here's how the low poly turned out:

http://www.verold.com/projects/54b4c8bfda4836f739006c04

Some pics (right click 'view image' or 'show in new window' to see full size) of the High Poly:






And some vanity renders, low poly again:




Thanks!