Forum

Author Topic: Could MS use MORE RAM???  (Read 4247 times)

CheeseAndJamSandwich

  • Full Member
  • ***
  • Posts: 188
    • View Profile
    • Sketchfab Models
Could MS use MORE RAM???
« on: October 15, 2021, 04:16:35 PM »
This is probably upside-down compared with all other RAM usage questions!!!  :D 8)

I have 64GB of RAM in my P51 Thinkpad, and only deal with 12MP GoPro images (but 1,000s of them)...  When processing my models, with the settings i use, MS just doesn't seem to use much RAM...  It's only in the teens... sometimes in the 20s...  But i'm just never going into the 30s, 40s, let alone the 50s, or 60s!

Could any of the processing be sped up by actually making more use of the RAM if it is plentiful?  Or is the bottleneck completely with the GPU/CPU processing?

Cheers.
My 'little' scan of our dive site, 'Manta Point'.  Mantas & divers photoshopped in for scale!
https://postimg.cc/K1sXypzs
Sketchfab Models:
https://sketchfab.com/cheeseandjamsandwich/models

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Could MS use MORE RAM???
« Reply #1 on: October 15, 2021, 04:58:00 PM »
Windows is automatically keeping last used files in RAM memory until memory is not needed for other heavy tasks. So in scenario you are copying photos from GoPro somewhere on disk, these files are automatically in RAM memory untill restart computer. And if you right after copying run MS and start processing, those files are reading from RAM. That is how windows automatic caching works and programming languages when are asking for some files from disk(first there is still lookup in RAM, and if not there, then read from disk).
If you come to processing in MS after restart PC, you still need to read photo files from disk first time, but any other following work will be using copies of those files in RAM.
Even when MS is creating some files during process and storing them in zip files on disk, those files are immediately in RAM.

You can check in task manager different memory values, total, available, cached, free. RAM memory is used more than just actual usage value.
« Last Edit: October 15, 2021, 05:00:14 PM by Bzuco »

CheeseAndJamSandwich

  • Full Member
  • ***
  • Posts: 188
    • View Profile
    • Sketchfab Models
Re: Could MS use MORE RAM???
« Reply #2 on: October 15, 2021, 06:45:29 PM »
Yes, that is true.
My Task Manager is reporting that it's holding 45GB Cached right now...

But a lot of the work done on software, like metashape, rightly goes into reducing it's RAM requirements... And there are lots of MS users posting on here that deal with very high resolution cameras images, who frequently run out of ram... 
So i was wondering if MS has been optimised perhaps with respect to reducing ram usage, but at the expense of some performance? So perhaps MS could have a RAM hungry mode when there was enough ram?  Where it could keep stuff in ram other than the usual data that's written out to disk, and cached...

it's probably not the case, the bottlenecks are all in the GPU/CPU, but i thought i'd ask.
My 'little' scan of our dive site, 'Manta Point'.  Mantas & divers photoshopped in for scale!
https://postimg.cc/K1sXypzs
Sketchfab Models:
https://sketchfab.com/cheeseandjamsandwich/models

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Could MS use MORE RAM???
« Reply #3 on: October 15, 2021, 07:54:58 PM »
Alignment phase: 2/3 is done on gpu, so the limit is only VRAM. And overal alignment phase is not so RAM/VRAM hungry, so I guess no reductions in memory consumption were applied here at the expense of performance.

The depth maps generation phase is done on gpu and you need to feed GPU through PCIE slot(which is fast enough, so no bottleneck).
You can check during this phase if there are some reads from disk and if, what are the values in spikes, if just few MB/s or tens or hundreds of MB/s. If no significant reads, everything was in RAM  :P .

Next phase is filtering depth maps. In this phase RAM usage depends on few factors. How many depth maps you will filter together(BuildDenseCloud/max_neighbors parameter in tweaks) and of course what is the resolution quality.
Next more important factor for RAM requirements is what is the shape of the object you were photographing and where you were taking photos.
In scenario with one long wall and camera positions one next to each other pointing in the same direction...this is not RAM hungry, because there will be just few(3-4?) depth maps needed together in filtering process at the same time.
In scenario when you were orbiting around object and you take several hundreds of photos pointing in the ~same direction...this will be very hungry, because lot of depthmaps need to be filtered together at the same time.
It would be difficult to predict what scenario might occur and make MS smart enough in terms of RAM decision...if keep more data in RAM or temporarily put data on disk.

This is how I see RAM requirements...more in depth analysis only with MS programmers :D

The last phase of dense cloud generation is joining and storing final data on disk. Here you can speed up things if your project is on SSD instead of classic HDD. I am using HDD and often see, that MS is waiting on disk to write all dense data, but in the size of my projects it still doesn not matter. Your projects are different story :D.