Agisoft Metashape

Agisoft Metashape => General => Topic started by: FoodMan on August 24, 2012, 10:05:26 AM

Title: ... about Networking Pscan
Post by: FoodMan on August 24, 2012, 10:05:26 AM
Hello... a co-worker asked me why Pscan can't reconstruct on several PC's... I don't know why exactly myself...

what's the reason...?

Best
f/ 8)
Title: Re: ... about Networking Pscan
Post by: Wishgranter on August 24, 2012, 03:34:13 PM
propably a solition exist, but need to test it :-) will inform in few weeks..... even PpenCL van work :-)
Title: Re: ... about Networking Pscan
Post by: FoodMan on August 24, 2012, 06:02:54 PM
hello Wishgranter..

Thanks.
So you're saying it  might be possible..? that would be awesome.... !..

.... really looking froward for your test..  8)

best
f/
Title: Re: ... about Networking Pscan
Post by: Wishgranter on August 24, 2012, 07:50:17 PM
need two identical PCs, mostly the CPU speed SHOULD be identical, if someone have let me know,

is possible even use two PC not on same place ( eventuely on 2continents - NET SPEED is a limit probably )

If is someone interested and this could be used in our comunity ( more than 2 PCs ) let me know...

Have found a solution but need a hard tests to be 100% sure....
Title: Re: ... about Networking Pscan
Post by: kris3d on August 25, 2012, 04:44:57 AM
I have, 9 computers with the same parameters
.
I will gladly test it
Title: Re: ... about Networking Pscan
Post by: Wishgranter on August 25, 2012, 09:53:32 AM
Kris, will put things together and let you know.......
Title: Re: ... about Networking Pscan
Post by: Wishgranter on August 26, 2012, 02:15:10 PM
Hello all, will test it with help of Kris, if it work as it should we will inform you......

Title: Re: ... about Networking Pscan
Post by: Infinite on August 26, 2012, 02:43:32 PM
I believe Agisoft LLC are aware of this and are thinking about implementation.

I've got 3 identical PC builds here and am VERY willing to test this idea.
Title: Re: ... about Networking Pscan
Post by: Wishgranter on August 26, 2012, 02:59:54 PM
is not easy for net reconstruct on sw level, but have found solution that can be "easy" implemented.
 Even with this version of Pscan is it possible and without a lot of programing but need test it.......

if everything go smooth, in next 2 weeks can show the solution......



 
Title: Re: ... about Networking Pscan
Post by: CovaH on August 26, 2012, 06:16:56 PM
Would it not be possible to distribute the images used for processing on different machines and have each machine work on that part of the picture so to speak? Then combine the results for quicker processing.
Title: Re: ... about Networking Pscan
Post by: Wishgranter on August 26, 2012, 06:26:20 PM
Guys, wait a few days, max 2 weeks, have found a solution, but need comunicate with agisoft team how solve few things..... puting it all together in mail that can give answers.....

Title: Re: ... about Networking Pscan
Post by: FoodMan on August 27, 2012, 08:32:15 AM
awesome Wishgranter !!...  8) :) 
Title: Re: ... about Networking Pscan
Post by: Wishgranter on September 01, 2012, 09:41:30 PM
OK, the 2nd mentioned "solution" work not, have assumed if you create just ALIGNED file, save 3 times and reconstruct eveytime just few images ( create mesh ) 4 images in one run then save. Afther that just copy from psz files the DEPHT maps and afther that open in pscan and create mesh ( DISABLING the REBUILD DEPHT MAPS ( the newest build ). Bu it will not work. See https://dl.dropbox.com/u/15047343/reconstruction.psd and enable layers, its clearly to see that depht maps are not the same, even the last one could be not processed (Layer three, last image ). so this way is not working....

The firs one mentioned solution need to be tested, IF we get some progress we will inform you ( have some speech with AGISOFT team about possible implementing )
Title: Re: ... about Networking Pscan
Post by: Wishgranter on September 07, 2012, 05:00:49 PM
Hello all, this weekend we wil try our "magic" on the net version, is possible that for proper work it will be a extra NET version of pscan because of the nature of net reconstruction that cannot be simply divided in smaler pieces as showed earlier in this thread...... Agisoft give us some experimental version on what we wil try it... 
Title: Re: ... about Networking Pscan
Post by: Mr_Curious on September 07, 2012, 06:28:04 PM
Hello all, this weekend we wil try our "magic" on the net version, is possible that for proper work it will be a extra NET version of pscan because of the nature of net reconstruction that cannot be simply divided in smaler pieces as showed earlier in this thread...... Agisoft give us some experimental version on what we wil try it...

Fingers crossed for a good result!

Greetings,

MC
Title: Re: ... about Networking Pscan
Post by: Infinite on September 07, 2012, 06:44:12 PM
Hello all, this weekend we wil try our "magic" on the net version, is possible that for proper work it will be a extra NET version of pscan because of the nature of net reconstruction that cannot be simply divided in smaler pieces as showed earlier in this thread...... Agisoft give us some experimental version on what we wil try it...

Fingers crossed for a good result!

Greetings,

MC

For sure it sounds interesting.
Title: Re: ... about Networking Pscan
Post by: Wishgranter on September 21, 2012, 11:16:46 AM
Hello All...

Have someone 2 "identical" machines with GPUs inside ? - need further testing
and let me know what CPUs are there - INTEL/AMD + core count
GPUs - AMD/NVIDIA
Title: Re: ... about Networking Pscan
Post by: Infinite on September 21, 2012, 12:45:26 PM
Hello All...

Have someone 2 "identical" machines with GPUs inside ? - need further testing
and let me know what CPUs are there - INTEL/AMD + core count
GPUs - AMD/NVIDIA

I have 3 but 2 are away at the moment being repaired, back this Monday, they are:
   
Gigabyte GA-X79-UD5 Motherboard   
Intel Core i7-3930K 6 core @ 3.2GHz Processor
64GB DDR3 2000MHz memory
2x nVidia GTX 680 2GB video card (1536 Cuda cores) (SLI)   
120GB Sata III Solid State Sata hard drive (OS)
1TB 7200rpm sata hard drive (Data)
100/1000 gigabit Intel Network LAN RJ45   
Windows 7 pro x64 installed
   

Title: Re: ... about Networking Pscan
Post by: Wishgranter on September 21, 2012, 02:47:28 PM
OK, Infinite cannot help ( time shortage ), someone other possible ?
Title: Re: ... about Networking Pscan
Post by: Wishgranter on September 24, 2012, 12:01:11 PM
Hello all Pscan users, for now exist one free solution for "net" reconstruction, it work under LINUX can find more here - http://www.mosix.org/txt_vcl.html - it Virtualize OpenCL Devices - GPUs, CPUs, APUs - But can be used for Depht Map generation only, is a significant speedup, but not fully net reconstruction process...


The other, is much pricey, is a sw based, like the VirtualCL, but licenses starting from 3.500,- USD for one dual socket PC and 256 GB ram, but is possible to create Workstation up to 256 TB RAM. Im in discusion about license model for use with Pscan, will inform you afther have cleared things with them..... This one can help in other sw applications - ram and CPUs allocation.. 
Im forget to mention that with this solution can be used even the http://www.intel.com/content/www/us/en/high-performance-computing/high-performance-xeon-phi-coprocessor-brief.html


the third solution depends on Agisoft team, how implement the "NET" reconstruction process... is done something in this way........
Title: Re: ... about Networking Pscan
Post by: Mr_Curious on September 24, 2012, 02:40:27 PM
Hello all Pscan users, for now exist one free solution for "net" reconstruction, it work under LINUX can find more here - http://www.mosix.org/txt_vcl.html - it Virtualize OpenCL Devices - GPUs, CPUs, APUs - But can be used for Depht Map generation only, is a significant speedup, but not fully net reconstruction process...


The other, is much pricey, is a sw based, like the VirtualCL, but licenses starting from 3.500,- USD for one dual socket PC and 256 GB ram, but is possible to create Workstation up to 256 TB RAM. Im in discusion about license model for use with Pscan, will inform you afther have cleared things with them..... This one can help in other sw applications - ram and CPUs allocation.. 
Im forget to mention that with this solution can be used even the http://www.intel.com/content/www/us/en/high-performance-computing/high-performance-xeon-phi-coprocessor-brief.html


the third solution depends on Agisoft team, how implement the "NET" reconstruction process... is done something in this way........

Interesting developments!

As for myself, I'm still quietly hoping that the Agisoft team will be able to come up with a solution.  But this is welcome news, nevertheless and I look forward to learning more!

Greetings,

Mr. Curious
Title: Re: ... about Networking Pscan
Post by: fpbv on September 24, 2012, 04:46:45 PM
Hi Wishgranter!

Very nice to hear this, I really hope this net reconstrution will work for PScan.
I have some huge demands here and I think this solution will help me a lot.
Title: Re: ... about Networking Pscan
Post by: mala on January 04, 2013, 01:04:20 PM
Just a quick general question to Alexey and the team.

Is it in theory possible for Agisoft to work across a network of PCs/Mac ?

I know that there is no solution at this moment and sure it would take a lot of development.

I just want to know if maybe sometime in the future this may be possible or is there some part of the process in Agisoft that makes networking impossible ?

Thank you.
mala
Title: Re: ... about Networking Pscan
Post by: Alexey Pasumansky on January 04, 2013, 02:42:51 PM
Hello mala,

In theory it is possible. Maybe in future it will be also possible in practice.
Title: Re: ... about Networking Pscan
Post by: Infinite on January 04, 2013, 02:54:22 PM
I agree with Mala. I think as sensors get better (more MP!) and consumer Cameras get cheaper Photoscan will need to process more and more data :)

I would be happy to contribute to help pay for such development in network processing, as I think it takes huge priority. Being able to process one chunk using the power of 4+ machines would be very powerful. Or being able to distribute 1x multi-chunk 4D processing over many PC's, could also be super powerful.
Title: Re: ... about Networking Pscan
Post by: mala on January 04, 2013, 03:55:04 PM
Thank you for the reply Alexey :)

As Lee says, I think this is something very important for the future.

 If you are processing terrain data or human scans (as Lee and myself do), always using more and more photos at higher resolutions and requiring a high resolution final mesh, faster is always better.
Time is Money ::)

Currently it often makes more financial sense to have more lesser powered machines than one "top-end" monster.

As many people who work with 3D rendering have small-medium sized render farms anyway, it would be very helpful to utilize these with Agisoft.

Cheers,
Merry
Title: Re: ... about Networking Pscan
Post by: Mr_Curious on January 04, 2013, 04:50:43 PM
+1

This is so important.  I also process huge amounts of 3d data and hardware requirement in regard to Agisoft is currently my main bottleneck.  I hope a network solution can be found because if it can then the sky is the limit for what would be possible using Agisoft.

Greetings

Mr. Curious
Title: Re: ... about Networking Pscan
Post by: fpbv on January 07, 2013, 03:30:14 AM
+2!!

Unfortunaly I have troubles with larger projects because the huge data we have.
I hope networking solution will come as soon as possible.
Title: Re: ... about Networking Pscan
Post by: piste on February 05, 2014, 04:26:01 PM
Hi everybody!!
I resume this old post...
A year has passed since the last response and I want to know if you have discovered a few stable solution in the meantime.
I would like to take advantage of these two computers to process my projects:

PC 1
Intel Core i5 - 2320 CPU @ 3.00 GHz (4CPUs)
NVIDIA GeForce GT 520 (4047 MB)
16 GB RAM
Windows 7 Home Premium 64bit

PC 2
Intel Core i5 - 2400 CPU @ 3.10 GHz (4CPUs)
NVIDIA GeForce GT 520 (4047 MB)
RAM 8 GB
Windows 7 Home Premium 64bit

Is it possible?
Many Thanks to everybody
Title: Re: ... about Networking Pscan
Post by: Wishgranter on February 05, 2014, 07:09:29 PM
Hi Piste, still not possible under normal conditions, there exist one solution, sw based ( special LINUX ), that can do specialized Cluster, but just the sw license for the solution cost approx 15.000+ EUR per 2x dual xeon 2,7 Ghz 128 GB ram, and for that setup you need additional hw that cost another approx 10.000 EUR...... even after that its still not tested, if it can AGI work under this sw solution ( teoretically yes )

Its very specialised solution, have searched for it few weeks to be sure its the right one.......
Title: Re: ... about Networking Pscan
Post by: robeastham on June 09, 2014, 03:11:33 PM
Hi Wishgranter,

What was the clustering software you mentioned? Presumably no way to use it with an amazon EC2 instance if hardware is required?

I'm a long time lurker here, just wanted to say thanks for all your posts. You are one of the most knowledgeable members of the community here and your time is appreciated.
Title: Re: ... about Networking Pscan
Post by: Wishgranter on June 09, 2014, 04:20:43 PM
Hi Robeasthem, thanx for appreciation  8) 
Please read PM and reply it.....