Ultimate PC Metashape Processing 20k+ images 2024?

[AIRWORKS]

I have an ongoing job i have been outsourcing but i would like to see if this is something i can start processing myself.  My PC is about 4-5years old and couldn't handle these last year when i took on the project.  Is anyone able to tell me from experience what the best system i can have built at the moment would be purely for processing 20,000 images into a metashape photogrammetry model from a drone capture?

[jenkinsm]

My PC was built in 2019 and I added a 3090 when I could get my hands on one, and I process that many photos all the time with no problem. 128 GB of RAM with a 10900K.

Try this which might help on your existing PC: https://www.youtube.com/watch?v=BYRIC-qkZJ8

 Where is your computer currently failing?

[AIRWORKS]

It just takes too long, a few weeks and I need to get this down to a 2-4 days if possible.  I reached out to MetaShape but their recommendation seemed like old tech and I was hoping someone here has built something recently with this sort of workflow in mind and can recommend something

[AIRWORKS]

This is a computer built at a shop they recommended, do you think it would be a good one to start with?

Intel Core i9 14900K (6.0GHz max clock speed),
24 Cores, 32 Thread CPU, Socket LGA 1700

Asus ROG MAXIMUS Z790 FORMULA LGA1700
DDR5 ATX

ASUS - ROG-THOR-1200P2-GAMING
ASUS ROG RYUJIN III 360 ARGB White Edition
Liquid CPU Cooler

Corsair DOMINATOR TITANIUM RGB White 96GB
(2x48GB) 6600MHz DDR5

8 x Corsair iCUE LINK QX120 RGB PWM 120mm
White Fan

Crucial T700 4TB M.2 2280 NVMe PCle 5.0 SSD

Microsoft Windows 11 Home, 64 Bit

Samsung 990 PRO 4TB M.2 2280 NVMe PCle 4.0
SSD

ASUS ROG Strix GeForce RTX 4090 24GB
GDDR6X White Edition

[RobertMarshall]

This is a computer built at a shop they recommended, do you think it would be a good one to start with?

Intel Core i9 14900K (6.0GHz max clock speed),
24 Cores, 32 Thread CPU, Socket LGA 1700

Asus ROG MAXIMUS Z790 FORMULA LGA1700
DDR5 ATX

ASUS - ROG-THOR-1200P2-GAMING
ASUS ROG RYUJIN III 360 ARGB White Edition
Liquid CPU Cooler

Corsair DOMINATOR TITANIUM RGB White 96GB
(2x48GB) 6600MHz DDR5

8 x Corsair iCUE LINK QX120 RGB PWM 120mm
White Fan

Crucial T700 4TB M.2 2280 NVMe PCle 5.0 SSD

Microsoft Windows 11 Home, 64 Bit

Samsung 990 PRO 4TB M.2 2280 NVMe PCle 4.0
SSD

ASUS ROG Strix GeForce RTX 4090 24GB
GDDR6X White Edition

I prefer Windows 10, solid as a rock, stable and efficient, Windows 11 is being referred to as the "Windows Vista" of releases, folk are actually waiting for Windows 12 to see what it is like.

A 4090 with 24GB is what I have, tears through the GPU load.

The CPU is good for what you need. A lot of processing is done on the CPU.

The memory is enough but if you can get it to 192GB, and your mainboard will support this, it'll really help finish this render engine off.

Also add in more disks, SSD's or M.2's, the mainboard supports more being added. Last thing you want to do is file system house keeping after 2 or 3 projects, and a cold-storage disk to archive stuff off too is wise.

Specs for your mainboard are here: https://rog.asus.com/motherboards/rog-maximus/rog-maximus-z790-hero-model/spec/

[Bzuco]

Which part of the whole process is the slowest one?

[AIRWORKS]

Thank you, he said it can take a max 192gb Ramm.

They just sent me this and said  this is available but it’s three times the price, do you think it would be much better or the other one with more Ram and SSD is ok?

Case: Fractal Design Define 7 XL
CPU: Threadripper Pro 5965x
CPU cooling : Fractal Design Celsius S36
Motherboard: Asrock WRX80 Creator R2
PSU: Be Quiet Dark Power 13 1600w
RAM: DDR4 256gb(3200)
GPU: 2 x Asus TUF RTX 4090
M.2: 2TB M.2 Kingston x2 (4TB total)
SSD: 2TB x4 (8TB total)

[AIRWORKS]

Which part of the whole process is the slowest one?

I’m working on an older machine about 5+ years and we have upgraded as much as we can but it’s really slow and I’m currently outsourcing anything over about 5000 photos.  Below are two machines I can get a good deal on and wondered if they might be suitable?  I am doing 20,000+ photos using the P1 and looking for a machine that can chew through this in decent times. 

[RobertMarshall]

They are giving you a choice of an Intel or AMD processor.

I would continue using Intel for now but its a personal choice. If I was to make a jump it'd be to the Xeon so I can spec up a server-class platform, different league and far more costly.

Have a read of this to give you an idea of AMD's Threadripper chip versus intel's i9: https://versus.com/en/amd-ryzen-threadripper-pro-5965wx-vs-intel-core-i9-13900k

The 2nd spec machine:

* Has DDR4 not DDR5 - If you're investing now get the latest tech not a generation behind, they probably did this to amp out the memory with larger memory modules?
* You won't get much mileage from having x2 GPU's, Agisoft could comment on how their GPU load works so ask them, they might put threads onto the two GPU's or it might just end up being done on one. I honestly don't think the bottleneck is going to be the GPU, you've already maxed that out with a 4090 RTX and 24GB of GPU memory. The bottleneck is going to be the CPU and Memory. My 4090 RTK eats up the GPU load, the GPU isn't the bottleneck it just has to chomp down quickly on any workload it receives and if you watch Metashape you'll see it isn't 100% maxing out the GPU constantly, it feeds it workloads, they are processed lighting fast (this is why you up-spec the GPU) and the rest of the time is spent in CPU and Memory phases.

Explore a RAID setup for the disks, RAID 10 will halve the available disk space but give you 1) Redundancy 2) faster disk IOP's especially with SSD's or M.2's (practically the same thing) - Do some research on RAID and specifically RAID 10. Find a decent PCI RAID card or use the mainboards built-in RAID support. I would 100% recommend that you get a backup power supply when using RAID, their not that expensive, if you lose power while writing to a RAID disk it can cause data loss or corruption especially if you turn off buffering to gain extra performance. As I said mitigate this by picking up a backup power supply, they plug into the USB port of the computer and tell the computer to power down cleanly after the battery is used up.

Hope that helps!

Thank you, he said it can take a max 192gb Ramm.

They just sent me this and said  this is available but it’s three times the price, do you think it would be much better or the other one with more Ram and SSD is ok?

Case: Fractal Design Define 7 XL
CPU: Threadripper Pro 5965x
CPU cooling : Fractal Design Celsius S36
Motherboard: Asrock WRX80 Creator R2
PSU: Be Quiet Dark Power 13 1600w
RAM: DDR4 256gb(3200)
GPU: 2 x Asus TUF RTX 4090
M.2: 2TB M.2 Kingston x2 (4TB total)
SSD: 2TB x4 (8TB total)

[RobertMarshall]

On those two spec's you'd get 20K of photographs processed in a day, two at the worst.

Like I said the main bottleneck is the CPU and how fast the memory is, disk IOP's are another smaller bottleneck due to the photographs being touched during the life-cycle of a render, as well as all the temporary files that are created (cache) and read from and written to.

About 5k of photographs for a low-end spec'd machine, two or three days end-to-end, and that's alignment, mesh, texture, sounds about right.

Which part of the whole process is the slowest one?

I’m working on an older machine about 5+ years and we have upgraded as much as we can but it’s really slow and I’m currently outsourcing anything over about 5000 photos.  Below are two machines I can get a good deal on and wondered if they might be suitable?  I am doing 20,000+ photos using the P1 and looking for a machine that can chew through this in decent times.

[AIRWORKS]

Thanks so much for all this information its really helpful, i think ill go with the I9 and i can get more Ram and hopefully it will save me a lot of time. 

[Bzuco]

@AIRWORKS
I am reading your older post from July 2023 again, where you mentioned just aligning 14000 photos on medium took you about a day. I think you are doing something wrong with metashape settings or something is wrong in your workflow.
I have slightly weaker PC then your old one mentioned in this post https://www.agisoft.com/forum/index.php?topic=15725.msg68006#msg68006  .
My largest project is 1472 photos and on High(which is full original photo resolution) accuracy it took 15min.(detecting points, selecting pairs, matching points) and another 10min. 20sec. estimating camera locations.
So whole alignment in 25min. ...and with metashape PRO license even less. You have 10x more photos, it should take max. 5 hours.

Buying just new PC will not cut your time to your desired level as you are expecting.

[AIRWORKS]

Hi Bzuco,  Would you mind elaborating on settings, ive tried medium on all processes and it takes weeks, i have a project with 10,000 photos at the moment and building the point cloud has taken 3.5 days already with 1day to go.  You dont think the i9 with a much better graphics card and more ram will make this process faster?   

[Bzuco]

No need to go with just medium quality in alignment settings, always use high(if photo quality is good). It is good to take smaller amount of photos from your large project and test how low parameters(key/tie point limit) still work for you. Check Metashape PDF manual what is saying about each parameter in options.

Quality of depth maps depends on your photo resolution, distance from object and how much precision you need in final result. Ultra high gives you maximum from your photos. On High you are using just half of the photo resolution. On Medium just 1/4.

i9 is just name of CPU series, what really matters is the number of cores(performance + efficient) and the resulted frequency on which they will be running at full utilization. So new CPU/GPU/RAM will finnish your big project faster, but not as much as you are expecting(from few weeks to just few days). If you want to see massive time savings, you need to be 100% sure that your parameters numbers are right chosen(not unnecessarily high and still sufficient for expected final result) and then it makes a lot of sense invest into new HW components hundreds or thousands $€.

Also you can let us know more about your project, what kind of photogrammetry it is, how big the photos are, what you need to produce as a final result(point cloud, mesh with textures, orthophoto maps., ....)

[jakobma]

I process 50k aerial 20MP images regularly and have the following setup with two computers which I always use individually (when one is processing I am working on the other one for other projects)

RTX4090 or RTX3090 (two machines, works good on both)
i9-14900K or i9-13900K (two machines, works very good but they tend to overheat! So I am maxing the Watt usage to 215W to not get 100C or crashes or thermal throttling which is extreme!)
360 aio for the 14900k and 240 aio for 13900k cooler (please get the biggest cooler you can get and some good thermal paste)
192GB DDR5 RAM (Corsair Vengeance) or 128GB DDR5 RAM (Kingston Fury Beast) on the second machine (I have a lot of RAM because I generate 500 - 8K Textures for those types of projects)
Fractal North XL and Fractal North Case (Glass and mesh, don't see a difference, glass looks cooler)
ASUS ProArt Z790 Creator and gigabyte z790 aorus elite ax motherboard on second PC (Mainly use them for the plenty M2 SSD slots)
Lots of Samsung 990PRO M2 SSDs with 2TB or 4TB since a project can easily become up to 1000GB large in my case (without photos, just the metashape folder!)
You could also go with other similar SSDs but I usually only use the samsung ones.


Alignment on Highest takes max 2 days and Building 3d model on ultra takes around 4 days. For a 50k photo RTK-corrected drone set with 20MP each photo and complex geometry. Fair time for me.

By the way, the difference between computer 1 (with the better stuff) is substantial, but not worth the bigger price. I only use it for generation if I need something as quickly as possible. If there are 1-2 extra days I use the "worse" one for processing 24/7 and the better one for regular work.

  Let me know if you have further questions. Please also let me know if you need help on processing such dataset because I do it all the time.