Forum

Author Topic: Cache directory in Photoscan  (Read 25205 times)

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Cache directory in Photoscan
« on: April 05, 2012, 01:08:44 PM »
When working on larger projects where more RAM is needed towards the end of the Meshing would it be possible to have a caching directory in Photoscan to cache to instead of using the Windows Swap file, most software is set up this way.

Reason I am asking is I have assigned swap file in Windows but Photoscan keeps on bluescreening when it starts to swap.

I know it will be slow, but processing can then be done on larger sets, there are fast SSD solutions out there that can be used.

So could you please add internal Cache/Swap to Photoscan.

Does swapping from the GPU cause any problems maybe, is that why I get bluescreen?

P.s. I have 64GB of RAM but I am still running easily out of RAM. And there is not really any indication how much more RAM I need.
« Last Edit: April 05, 2012, 01:12:05 PM by olihar »

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #1 on: April 05, 2012, 01:40:38 PM »
I used WinDbg to take a look at the BlueScreen dump file, in the bottom it states.

Code: [Select]
Probably caused by : ntoskrnl.exe ( nt+3ceb8 )

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #2 on: April 05, 2012, 04:38:10 PM »
Well maybe there is a faulty memory in my system.. been running a test on the RAM.



Might be the reason for the error, I still would like to see a cache option in Photoscan.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Cache directory in Photoscan
« Reply #3 on: April 05, 2012, 05:00:52 PM »
Hello olihar,

Thank you for the suggestion, even it's quite hard to be implemented to all PhotoScan algorithms correctly.

And as or the memory problem I think it's better to remove bad memory module (hopefully its warranty period is not over and it could exchanged on a working one) and check if the problem no longer appears.
Best regards,
Alexey Pasumansky,
Agisoft LLC

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #4 on: April 05, 2012, 06:43:27 PM »
So you are saying it is better to have Windows take care of the Swapping as virtual memory? Then something that could be built into Photoscan.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Cache directory in Photoscan
« Reply #5 on: April 05, 2012, 06:48:32 PM »
Quote
So you are saying it is better to have Windows take care of the Swapping as virtual memory? Then something that could be built into Photoscan.
No, I've meant only replacing bad memory module. Using Windows swap is not recommended.
The possible caching solution could take a lot of development and testing time, so I do not think it should be expected soon.
Best regards,
Alexey Pasumansky,
Agisoft LLC

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #6 on: April 05, 2012, 08:34:34 PM »
Yes I am in the process of finding the 1 bad one, I have narrowed down to 4 modules and soon narrow down to 2 modules.

But What I meant is when I need more then 64GB to process dataset I can still use Windows virtual memory to process the dataset. I know it will be slow but it can be done.

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Cache directory in Photoscan
« Reply #7 on: April 05, 2012, 09:51:30 PM »
For the swap, yes you could use some SSD PCI-E solutions but think on that they wear an by something like this it can be "depleted" in few months or even weeks :-(

Best Solution is to grab a good Motherboard with a lot RAM slots, best as i mentioned earlier with 24 RAM slots, because you can use "inexpensive" 4 or 8 GB ram modules, because if use MB with 8-12-18 slots you get quicker to point that expanding meme is very costly http://buy.fudzilla.com/?cat=ramddr3regecc&xf=1454_8192&sort=p search for self with price for 8GB and 16 GB modules, price is twice per GB in 16 GB modules or 2000 EUR fro a 32 GB module... and price for RAM modules will go UP in few weeks, moths - that another story.....

So a for 128 GB is around 600-700 EUR, with 8 GB modules...

what precise Mobo you use ???? probably can be used workaround for you...... let me know...

 
----------------
www.mhb.sk

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Cache directory in Photoscan
« Reply #8 on: April 05, 2012, 10:20:45 PM »
Oli, this is project with 35 x18Mpix photos and it consume around 55 GB ram as i remember.... so a bigger amount of photos will bring system on knees...... http://www.youtube.com/watch?v=RIT50G9MVDI&feature=g-upl&context=G27fef79AUAAAAAAABAA
----------------
www.mhb.sk

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #9 on: April 05, 2012, 11:13:42 PM »
I am using This Motherboard...

http://www.msi.com/product/mb/X79A-GD65--8D-.html

And I have 8x8GB in the motherboard. It does support 128GB but it is very pricey due to cost of 16GB chips.

Right now I am processing a set with 38 x 195Mpix photos.

Buying a dual socked motherboard is not an option right now.

I am fine if the process takes a week, as long as it is able to finish.


Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Cache directory in Photoscan
« Reply #10 on: April 06, 2012, 01:07:57 AM »
Yes, understand.... but to use the 16 GB modules is problematic, it is possible that is mean the new LR-DIMM, but they are still not on market......

when you have not the ram, then you have problem, its all about the mem here, because you cannot finish it..... probably larger SWAP file could help. how much memory it takes in the mesh generation step ? it can be seen in TASK manager, if set the  MAXIMUM WORKING SET, from that you can see how much mem is used..... will PM you.....
----------------
www.mhb.sk

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #11 on: April 06, 2012, 01:13:50 AM »
I am not sure how much RAM you need to be able to work with such a set I am working on right now, I don't think it is wise to try and put it all in RAM, you might as well cache some of it, yes it takes time to process but again, it will finish.

I have now about 1TB in virtual memory + the 64GB in RAM.

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #12 on: April 06, 2012, 01:21:56 AM »
And I guess its worth asking, the Generating Mesh step, is that a single thread process... If so, using the full potential of RAM speed would not be necessary anyways.

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Cache directory in Photoscan
« Reply #13 on: April 06, 2012, 03:24:43 AM »
Windows 7 mem limits:

Click Custom size, type a new size in megabytes in the Initial size (MB) or Maximum size (MB) box, click Set, and then click OK. There is a formula for calculating the correct pagefile size. Minimum pagefile size is one and a half (1.5) x amount of memory. Maximum pagefile size is three (3) x minimum pagefile size. Say you have 4 Gb (4,096 Mb) of memory. 1.5 x 4,096 = 6,144 Mb would be the min. pagefile size and 3 x 6,144 = 18,432 Mb would be the max. pagefile size.
----------------
www.mhb.sk

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Cache directory in Photoscan
« Reply #14 on: April 07, 2012, 03:22:16 PM »
I am still getting out of Memory error...

Photoscan has used all the RAM and has cached just over double the RAM as well, see here in the Screenshot it stopped at 111GB.

I have around 600GB in Virtual memory + RAM but it does not seem to be used. What can be done about this?



I get this in the console.

Error: bad allocation
Finished processing in 82892.9 sec (exit code -1)


Here is from the Console, it didn't allow me to post the whole text from the console due to 2000 character limit.

Code: [Select]
Device 1 performance: 209.55 million samples/sec
Device 2 performance: 242.677 million samples/sec
Generating mesh...
Grid size: 28983 x 28983 x 28983
Tree depth: 15
1520903050 points extracted
Tree set in 8634.19 s
Tree size 106828 MB (1750277166 leaves, 2000316761 nodes)
Tree refined in 15102.3s
Tree size 106828 MB (1750277166 leaves, 2000316761 nodes)
Error: bad allocation
Finished processing in 82892.9 sec (exit code -1)
>>>
« Last Edit: April 07, 2012, 03:30:14 PM by olihar »