Strange workstations testing results.

[gEEvEE]

Windows 7 ultimate 64-bit

If it would be a Windows issue, I think that the benchmark tests wouldn't show the true potential of the Workstation's CPUs. In most software applications, the Xeons blow the i7 away...

Geert

[Triplegangers]

Ahhh, do I love this thread? So much weird and fascinating stuff going on. Everyone is getting strange results. Now maybe, just maybe! Our own observation of tests, affects the observed reality  Quantum physics theory at computer software testing    Ok back to our strange stuff.

Spent some time installing and setting up Windows 7 just to run this tests and see if OS makes much difference. And here are some peculiar results:



Switching to Windows 7 did show some positive changes in speed. But, difference between 402 and 394 seconds is very low and I will neglect it, saying that there was no difference between two OS on this stage. However when I started playing with HP, weirdness came up. Not only it worked faster with HP off, it also performed 10% better under Windows 7. Although I think its only due to software issue

My conclusion is that OS doesn't matter that much, and I will stop my tests on this stage before I loose my mind. Clearly, and I hope you all will agree, we have a problem that affects Photoscan performance and its quite complex. Starting with OS and BIOS settings, ending with complex workstation hardware.

I hope in the future, we will see Photoscan that can optimize itself for best performance, based on the system hardware and OS. Feature request maybe!? 

Also wish AgiTeam implements parallel chunk processing some time soon.

[Wishgranter]

No, Agisoft team done excelent work with paralelism in pscan,  for everyone, its better to switch OFF the HyperThreading, as its not something that we can use for our datasources.....

for the ones who understand little bit more, is this link..... im think its clear why its better to disable it.... 

http://bitsum.com/pl_when_hyperthreading_hurts.php

and this one

http://www.agner.org/optimize/blog/read.php?i=6

[Triplegangers]

Hey Milos,

Thank you for that link! Was wondering if there was a way to switch off HP for some programs from windows. Hate turning it off from BIOS, it hurts other software and processes which do benefit from HP.

Instead of completely disabling HyperThreading, you can use programs like Process Lasso (free) to set default CPU affinities for critical processes, so that their threads never get allocated to logical cores. We call this feature HyperThreaded Core Avoidance. It is better than completely disabling Hyper-Threading because it leaves the rest of the system free to take advantage of this otherwise useful feature.

http://bitsum.com/processlasso its free and totally works, just tested. Also has many more interesting tools.

No, Agisoft team done excelent work with paralelism in pscan

I was speaking about parallel chunk processing, feature requested here:
http://www.agisoft.ru/forum/index.php?topic=1337.0

[jedfrechette]

My Xeon E5-2687W is significantly faster with Hyper Threading Enabled, regardless of OS.

The following tests were done using the "sample01" data on the same machine under Debian Jessie and Windows 7 Pro. In both cases PhotoScan Pro 0.9.0 build 1586 was used. The times below are the best of three runs for alignment only.

Hyper-Threading On
====================
Debian: 65.7 s
Win 7: 80.1 s

Hyper-Threading Off
====================
Debian: 84.3 s
Win 7: 92.9 s

It would be interesting to have an automated benchmark built in to PhotoScan. Ideally with the option to upload the results to a public database so that it would be easier to compare performance on a variety of hardware platforms. I had a go at hacking something together in Python a few months ago but ran in to various road blocks and ultimately put it aside.

[Wishgranter]

Jed, thanx for the results....

im want not confuse people, but results are OK, one of the best results could/should be under WinXP and Liunx, then W7 and Win 8. its about how the OS share resources and etc. back in90s we have tested Win 3.11, W95 and Win NT under pshop, and results were so that Win 3.11 was the fastest OS back then.

im wil do tests as have mentioned so we have clear understanding how it work and where it run best....... 
   

[airmap3d]

Hello all,

I have just run a test on my machine and have come up with the following...

I used a large project of mine and tested align photos with hyper threading both on and off.  I am running windows 7 pro with dual xeon E5620's, Photoscan Pro 0.9.0 (1586) and it was 20% faster with hyper threading enabled.

I will continue to use PS with HT enabled but from what I can gather from all of this is that every machine and config is going to be different so I would suggest testing your system and running with what works best for you??  Seems a simple solution to me.

Cheers!!

[Wishgranter]

Wooow, really weird results.... my question is have you  tested it on same dataset ? just to be sure :-)

[airmap3d]

Yep, same data. It was about 500 images so was a reasonable sized job.

This thread seems to be getting some very mixed results! All very confusing!!

[Wishgranter]

yes, litle confusing, but every win version ( linux too ) acess the CPU resources little other way, explaining it here is realy problematic for people that have no extra knowlege on this stuff.....

 

[RalfH]

Also, it shows that there could be a good potential for software optimisation - Photoscan could read hardware and OS data and use them to optimise hardware usage.

[jedfrechette]

Also, it shows that there could be a good potential for software optimisation

Not to mention meat-space optimization. Give users a simple way to reliably compile useful benchmarks and I have little doubt that optimum configurations would be identified quite rapidly.

[Wishgranter]

Hi All, another interesting article on manycore system, this time a 4P system with Win2012 server. read carefully

http://www.anandtech.com/show/7121/trials-of-an-intel-quad-processor-system-4x-e54650l-from-supermicro

[Wishgranter]

Hi All, im digging intro the benchmarks, something its not OK under windows, but will post results afther AGI team say what they think about it.....

[Wishgranter]

Who could help us with few benchmarks ? i need little more benches to be sure about results.
Best if 2P or even 4P system is tested on same, little bigger dataset, with aprox 50 images. We need run tests under WinXP, Win7, Win8, W2012Server, UBUNTU and OSX. must be not run on every OS, but just to have some overview on it too... Single CPUs too,but best if something even with HIGH overclock.

The difference on ALIGN stage is over 390% - same hw setup, so from this could benefit everyone.....