Big Data Project

[jjllovera]

Hello, I have on the horizon a relatively big job.

A 5500 images and have available 2 computers for you project.

What would be the most efficient way?

I understand that I can do batch processing on a PC, but you can process files on separate machines and finally unite into one big file.

regards

[stihl]

Shouldn't be an issue if you do some effective pre-processing and make sure to slice the project into chunks for processing. With one computer with hardware worth about 4 thousand dollars I've processed a single aerial project that consisted of over 80.000 images. If you chop it up and manage it well all it takes is some time.

I wouldn't unite or merge chunks in Photoscan. Instead export them individually and merge into a sophisticated GIS program like GlobalMapper.

[r0xx]

What setup did you have and how long did it take for those 80.000 images?

[stihl]

Images were 6000x4000 resolution, JPG.

Computer that was used was a

Intel Xeon E5-2630 v2
128GB DDR3 RAM
Win7 X64
nVidia GTX780

Total estimated time from start to finish for the processing is about 2-3 weeks.

[Alexey Pasumansky]

I see several possibilities of processing the data, if it cannot be performed on the single computer:


- just split project according to the reference data into two parts (probably, with overlap) and process the projects separately,
- align photos on the single computer, then by varying the bounding box size create two chunks based on the same alignment and bounding boxes next to each other, then you can generate dense cloud on two computers and merge the results back after both parts are generated,
- use network processing feature and both computers as processing nodes.

[r0xx]

Images were 6000x4000 resolution, JPG.

Computer that was used was a

Intel Xeon E5-2630 v2
128GB DDR3 RAM
Win7 X64
nVidia GTX780

Total estimated time from start to finish for the processing is about 2-3 weeks.

Wow okay thanks!

I have 2-3 weeks for 1'200 images with an I7 4930K, 64 GB DDR3 RAM and a GTX 760...

[Alexey Pasumansky]

Hello r0xx,

I believe that this task can be finished in a couple of days, unless you are planning to process in High quality.

[r0xx]

Yes I have aligned High and Pixelcloud is also processed in High Quality...