Forum

Author Topic: Parallel computing using a network cluster  (Read 12314 times)

7eicher

  • Newbie
  • *
  • Posts: 42
    • View Profile
Parallel computing using a network cluster
« on: August 03, 2013, 07:03:36 PM »
Dear Agisoft Team,

first let me thank you for your great product!

I would like to know about efforts and roadmap for the phrase on your site (about us section):

- parallel computing using a network cluster

As there anything going on and do you have information about it, that you can share, timings, specifics?

We run several times on our  RAM limitations and thats pretty annoying. I know its not about your software, its tough work to process. Especially i wonder if you can overcome the RAM limitation by using virtual memory from SSD? Probably thats not so easy or not possible, as you work on physical memory addresses directly using the MMU. Also the GPU access data from the RAM, therefore the swap isnt used anyway, as it's a kernel/cpu feature. Do see any chance, to reduce the needed amount of RAM, by any kind of technique?

Take care,

7eicher

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Parallel computing using a network cluster
« Reply #1 on: August 03, 2013, 07:13:18 PM »
How much RAM you need for your projects ?
----------------
www.mhb.sk

7eicher

  • Newbie
  • *
  • Posts: 42
    • View Profile
Re: Parallel computing using a network cluster
« Reply #2 on: August 03, 2013, 08:06:06 PM »
256 would be nice, as i work with D800 (7.3K images) but i can't effort such a machine right now. Other point is that performance in a shared network with many GPU's could improve the running time, right?

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Parallel computing using a network cluster
« Reply #3 on: August 03, 2013, 08:14:38 PM »
256 GB is not so much these days :-) how much images are there ? can help eventually so can see processing times and RAM allocation = to see how much really need for processing. The GPUs are used "just" for the DEPHT map generation, if have say 2-4 GPUs (7970s) in PC not need networking
 and from my point of view its take most time with MESH generation on larger projects..... If want upload JPGs to some sharing site ( Quality 10 - in photoshop or 95 in other app ) and can try do few benches....
----------------
www.mhb.sk

Zom-B

  • Newbie
  • *
  • Posts: 1
    • View Profile
distributed calculations over LAN
« Reply #4 on: January 10, 2014, 07:54:07 PM »
Hey guys,

I just discovered PhotoScan and am very amazed by the results you can realize with that neat piece of software.

A great possibility to reduce computation times for the some task in PhotoScan would be the ability to add slaves in local Network to support calculations.

I'm from 3D viz area, where distributed rendering is common along software,  and own for example 2 extra slave computers for render support, every  with 32GB of RAM.
Would be great if I could use them to support speeding up (some) of PhotoScans calculations

I'm sure the are multiple situations where users could combine the power off more then one computer.

Cheers and thanks for this great software!

7eicher

  • Newbie
  • *
  • Posts: 42
    • View Profile
Re: Parallel computing using a network cluster
« Reply #5 on: January 12, 2014, 06:58:31 PM »
Thanks for sharing and supporting the idea of shared computation! I hope PS can get into this asap, in order to reach more peoples environments. Not every can buy a 8k $ machine having 500GB memory or so.

Well, I understand that the computation of heavy data driven project in a distributed environment isnt that easy, but I believe there are ways to achieve it. The biggest challenge is probably how a single node/slave can work on an isolated task, where the necessary image and point cloud data gets isolated too. In my personal projects I prefer quality over speed. So memory usage is a big issue for me.

So can we isolate processes, that only a particular part of the data needs to be in memory for nodes/slaves which have limited memory available?