Forum

Author Topic: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!  (Read 8482 times)

andyroo

  • Sr. Member
  • ****
  • Posts: 458
    • View Profile
Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« on: December 07, 2024, 12:11:22 AM »
Hello and best wishes fellow Metashapers!

Wondering if anyone has recently done any direct comparisons between Windows and Linux OS on the same hardware.

I just benchmarked fresh installs of Windows 11 and Rocky Linux 9 on an AMD Ryzen 9 9900X and an Intel i9-9900K and was surprised to see that Metashape 2.1.3 ran about 30-40% faster under Linux than Windows on both machines. Time in seconds for the benchmark is below (attached a graph too):

Build     Rocky Linux 9.5     Windows 11
Intel i9 9900k/RTX 2080 Super8999     11815
AMD Ryzen 9 9900x/RTX 40904380     6016

-OS and benchmark details-
I installed Win 11 Pro from the latest Microsoft Install tool and Rocky Linux 9.5 from the latest distro. For Windows downloaded the latest NVidia driver from their website and for rocky installed with sudo dnf module install nvidia-driver:open-dkms.  The benchmark was run from the OS SSD on both workstations right after OS and NVidia driver install (NTFS for Win 11 and XFS for RL9) and I think I have RAM at 3200MHz on the intel and 5200MHz on the AMD.

This is the first time that I've done a direct comparison of operating systems on the same hardware, and it's possible that I missed something, but I set up both machines the same way I've been setting up my whole cluster so if anyone finds something different please let me know!

My benchmarks are based on the Puget Systems Extended Benchmark for Metashape Pro (download files from their page or this link), with a benchmark script (attached) that I've periodically updated to work on newer versions of Metashape over the last ~5 years.

To run the benchmark the same way I do, download the Puget Systems Benchmark files into a directory, copy my modified benchmark script (attached to this message) into that folder, open the "Puget Systems Benchmark.psx" project in metashape, then run my script from the console. It should prompt you to start the benchmark, then run for a few hours and give you results on-screen and in the results text file.

The totals above are the totals from all three models run with the script - basically the summed totals from all lines with the text "Total Processing Time:" in the sample script output below (The script writes/appends each run to a file called "Benchmark Results.txt" in the benchmark directory:

Code: [Select]
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 16:44 on December 05, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 2080 SUPER
Project: Park Map
Align Photos: 631.2
Build Depth Maps: 438.7
Build Dense Cloud: 730.3
Build Tiled Model: 3641.3
Build DEM: 31.4
Build Orthomosaic: 435.6
Total Processing Time: 5908.5
Project: School Model
Align Photos: 409.8
Build Depth Maps: 229.2
Build Dense Cloud: 684.2
Build Mesh from Dense Cloud: 223.0
Decimate Mesh: 38.2
Build Texture: 266.6
Total Processing Time: 1851.0
Project: School Model using Depth Maps
Align Photos: 409.8
Build Depth Maps: 229.2
Build Mesh from Depth Maps: 319.4
Decimate Mesh: 55.2
Build Texture: 225.8
Total Processing Time: 1239.4
Benchmark Completed at 19:04

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 15282
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #1 on: December 09, 2024, 11:14:07 AM »
Hello andyroo,

Can you please also share the comparison table per task for each configuration/OS? I mean time spent for each major step: match photos, align cameras, build depth maps, build point cloud, build model, etc.

In case you can also share the logs for each run (corresponding to every combination of OS/hardware tested) please either attach them here or send to support@agisoft.com.
Best regards,
Alexey Pasumansky,
Agisoft LLC

andyroo

  • Sr. Member
  • ****
  • Posts: 458
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #2 on: December 10, 2024, 02:46:33 AM »
Here is the per-step benchmark output for all model runs I did on the two workstations. I did not have logging enabled/didn't save logs.

I'll re-run the benchmark on the intel - "Phil" with logging enabled. Probably 24-48h before I have those results since I have to reinstall windoze too.
Code: [Select]
*************************************
******** MACHINE 1 AMD CPU **********
*************************************
-----------------------------------
aritchie new build - Rocky linux 9.5 SMT ON --NUMA AUTO
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 16:08 on October 29, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 262.5
Build Depth Maps: 195.4
Build Dense Cloud: 347.5
Build Tiled Model: 1832.6
Build DEM: 18.4
Build Orthomosaic: 224.2
Total Processing Time: 2880.6
Project: School Model
Align Photos: 173.1
Build Depth Maps: 99.1
Build Dense Cloud: 306.8
Build Mesh from Dense Cloud: 110.3
Decimate Mesh: 18.8
Build Texture: 150.1
Total Processing Time: 858.2
Project: School Model using Depth Maps
Align Photos: 173.1
Build Depth Maps: 99.1
Build Mesh from Depth Maps: 219.7
Decimate Mesh: 26.6
Build Texture: 118.7
Total Processing Time: 637.2
Benchmark Completed at 17:16
-----------------------------------
LINUX - now turn off SMT in BIOS also flipped ACPI L3 cache as NUMA ENABLED
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 18:12 on October 29, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 276.5
Build Depth Maps: 201.0
Build Dense Cloud: 455.5
Build Tiled Model: 2205.0
Build DEM: 19.6
Build Orthomosaic: 230.8
Total Processing Time: 3388.4
Project: School Model
Align Photos: 187.7
Build Depth Maps: 103.1
Build Dense Cloud: 431.8
Build Mesh from Dense Cloud: 122.7
Decimate Mesh: 21.8
Build Texture: 155.5
Total Processing Time: 1022.6
Project: School Model using Depth Maps
Align Photos: 187.7
Build Depth Maps: 103.1
Build Mesh from Depth Maps: 225.0
Decimate Mesh: 31.8
Build Texture: 134.7
Total Processing Time: 682.3
Benchmark Completed at 19:33
-----------------------------------
LINUX - next SMT on and still NUMA on (since NUMA takes long time to train I guess)
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 10:39 on October 30, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 264.9
Build Depth Maps: 193.6
Build Dense Cloud: 349.5
Build Tiled Model: 1842.8
Build DEM: 18.8
Build Orthomosaic: 228.6
Total Processing Time: 2898.2
Project: School Model
Align Photos: 176.1
Build Depth Maps: 99.1
Build Dense Cloud: 307.6
Build Mesh from Dense Cloud: 111.5
Decimate Mesh: 19.8
Build Texture: 149.5
Total Processing Time: 863.6
Project: School Model using Depth Maps
Align Photos: 176.1
Build Depth Maps: 99.1
Build Mesh from Depth Maps: 217.7
Decimate Mesh: 27.6
Build Texture: 126.7
Total Processing Time: 647.2
Benchmark Completed at 11:48
-----------------------------------
LINUX --NUMA AUTO, SMT OFF--
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 12:44 on October 30, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 276.7
Build Depth Maps: 202.6
Build Dense Cloud: 451.3
Build Tiled Model: 2198.9
Build DEM: 19.2
Build Orthomosaic: 233.6
Total Processing Time: 3382.3
Project: School Model
Align Photos: 186.1
Build Depth Maps: 101.7
Build Dense Cloud: 428.2
Build Mesh from Dense Cloud: 123.3
Decimate Mesh: 21.6
Build Texture: 154.5
Total Processing Time: 1015.4
Project: School Model using Depth Maps
Align Photos: 186.1
Build Depth Maps: 101.7
Build Mesh from Depth Maps: 217.9
Decimate Mesh: 31.6
Build Texture: 128.9
Total Processing Time: 666.2
Benchmark Completed at 14:04
-----------------------------------

LINUX --BACK TO SMT ON NUMA AUTO -- whoops I think SMT was still off - 12 cores
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 14:47 on October 30, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 271.9
Build Depth Maps: 198.4
Build Dense Cloud: 451.7
Build Tiled Model: 2201.0
Build DEM: 19.2
Build Orthomosaic: 229.6
Total Processing Time: 3371.8
Project: School Model
Align Photos: 184.7
Build Depth Maps: 100.7
Build Dense Cloud: 426.8
Build Mesh from Dense Cloud: 123.3
Decimate Mesh: 20.8
Build Texture: 152.7
Total Processing Time: 1009.0
Project: School Model using Depth Maps
Align Photos: 184.7
Build Depth Maps: 100.7
Build Mesh from Depth Maps: 225.3
Decimate Mesh: 30.6
Build Texture: 132.9
Total Processing Time: 674.2
Benchmark Completed at 16:07

-----------------------------------
LINUX - SMT on for real again, and NUMA on Auto
-------no more bios changes after this-------
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 16:30 on October 30, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 261.1
Build Depth Maps: 193.6
Build Dense Cloud: 347.4
Build Tiled Model: 1830.2
Build DEM: 18.6
Build Orthomosaic: 223.0
Total Processing Time: 2873.9
Project: School Model
Align Photos: 170.9
Build Depth Maps: 98.5
Build Dense Cloud: 307.0
Build Mesh from Dense Cloud: 110.7
Decimate Mesh: 19.8
Build Texture: 150.3
Total Processing Time: 857.2
Project: School Model using Depth Maps
Align Photos: 170.9
Build Depth Maps: 98.5
Build Mesh from Depth Maps: 218.1
Decimate Mesh: 27.8
Build Texture: 121.9
Total Processing Time: 637.2
Benchmark Completed at 17:39
-----------------------------------
LINUX---------enable nvidia persistenceD (didn't seem to make a diff)
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 12:03 on November 01, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 259.7
Build Depth Maps: 195.2
Build Dense Cloud: 345.6
Build Tiled Model: 1834.2
Build DEM: 19.0
Build Orthomosaic: 227.4
Total Processing Time: 2881.1
Project: School Model
Align Photos: 173.3
Build Depth Maps: 99.5
Build Dense Cloud: 307.2
Build Mesh from Dense Cloud: 110.5
Decimate Mesh: 19.4
Build Texture: 148.1
Total Processing Time: 858.0
Project: School Model using Depth Maps
Align Photos: 173.3
Build Depth Maps: 99.5
Build Mesh from Depth Maps: 219.1
Decimate Mesh: 27.8
Build Texture: 119.3
Total Processing Time: 639.0
Benchmark Completed at 13:12
-----------------------------------
LINUX--re-run after update kernel
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 12:49 on November 14, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 262.3
Build Depth Maps: 195.8
Build Dense Cloud: 348.6
Build Tiled Model: 1831.4
Build DEM: 18.4
Build Orthomosaic: 222.4
Total Processing Time: 2878.9
Project: School Model
Align Photos: 171.7
Build Depth Maps: 99.9
Build Dense Cloud: 307.4
Build Mesh from Dense Cloud: 110.7
Decimate Mesh: 19.4
Build Texture: 152.9
Total Processing Time: 862.0
Project: School Model using Depth Maps
Align Photos: 171.7
Build Depth Maps: 99.9
Build Mesh from Depth Maps: 215.7
Decimate Mesh: 27.2
Build Texture: 124.5
Total Processing Time: 639.0
Benchmark Completed at 13:58
----------------------------------------
LINUX on AD - logged in via SSH need to keep SSH opn
-----------------------------------------

Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 14:53 on November 22, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 266.4
Build Depth Maps: 200.0
Build Dense Cloud: 360.9
Build Tiled Model: 2025.0
Build DEM: 19.6
Build Orthomosaic: 247.1
Total Processing Time: 3119.0
Project: School Model
Align Photos: 168.1
Build Depth Maps: 102.9
Build Dense Cloud: 315.8
Build Mesh from Dense Cloud: 125.1
Decimate Mesh: 20.2
Build Texture: 159.7
Total Processing Time: 891.8
Project: School Model using Depth Maps
Align Photos: 168.1
Build Depth Maps: 102.9
Build Mesh from Depth Maps: 229.4
Decimate Mesh: 28.8
Build Texture: 127.7
Total Processing Time: 656.9
Benchmark Completed at 16:07
----------------------------------------
windows 11 non-AD - balanced power profile
----------------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 14:12 on November 29, 2024
CPU: AMD64 Family 26 Model 68 Stepping 0, AuthenticAMD
Number of GPUs Found: 2
GPU Model(s): AMD Radeon(TM) Graphics (gfx1036), NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 338.5
Build Depth Maps: 341.7
Build Dense Cloud: 791.3
Build Tiled Model: 2867.2
Build DEM: 23.1
Build Orthomosaic: 329.4
Total Processing Time: 4691.2
Project: School Model
Align Photos: 225.5
Build Depth Maps: 160.1
Build Dense Cloud: 642.7
Build Mesh from Dense Cloud: 165.0
Decimate Mesh: 26.6
Build Texture: 161.5
Total Processing Time: 1381.4
Project: School Model using Depth Maps
Align Photos: 225.5
Build Depth Maps: 160.1
Build Mesh from Depth Maps: 309.9
Decimate Mesh: 42.5
Build Texture: 154.5
Total Processing Time: 892.5
Benchmark Completed at 16:02
-----------------------------------
windows 11 non-ad performance power profile
-----------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 17:52 on November 30, 2024
CPU: AMD64 Family 26 Model 68 Stepping 0, AuthenticAMD
Number of GPUs Found: 2
GPU Model(s): AMD Radeon(TM) Graphics (gfx1036), NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 316.6
Build Depth Maps: 293.5
Build Dense Cloud: 551.3
Build Tiled Model: 2430.1
Build DEM: 22.4
Build Orthomosaic: 313.0
Total Processing Time: 3926.9
Project: School Model
Align Photos: 211.2
Build Depth Maps: 151.8
Build Dense Cloud: 572.8
Build Mesh from Dense Cloud: 155.4
Decimate Mesh: 24.4
Build Texture: 142.7
Total Processing Time: 1258.3
Project: School Model using Depth Maps
Align Photos: 211.2
Build Depth Maps: 151.8
Build Mesh from Depth Maps: 294.1
Decimate Mesh: 37.3
Build Texture: 136.6
Total Processing Time: 831.0
Benchmark Completed at 19:26
--------------------------------
installed all drivers (some were missing before in device manager
set game mode for metashape (untitled)
-----------------------------------------------------------------

Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 11:43 on December 1, 2024
CPU: AMD64 Family 26 Model 68 Stepping 0, AuthenticAMD
Number of GPUs Found: 2
GPU Model(s): AMD Radeon(TM) Graphics (gfx1036), NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 322.3
Build Depth Maps: 323.6
Build Dense Cloud: 548.8
Build Tiled Model: 2441.5
Build DEM: 22.6
Build Orthomosaic: 308.6
Total Processing Time: 3967.4
Project: School Model
Align Photos: 219.1
Build Depth Maps: 167.3
Build Dense Cloud: 556.2
Build Mesh from Dense Cloud: 158.2
Decimate Mesh: 25.6
Build Texture: 144.4
Total Processing Time: 1270.8
Project: School Model using Depth Maps
Align Photos: 219.1
Build Depth Maps: 167.3
Build Mesh from Depth Maps: 298.0
Decimate Mesh: 39.5
Build Texture: 138.0
Total Processing Time: 861.9
Benchmark Completed at 13:19

-----------------------------------------
new aritchie build 9900x on AD windows
-------------------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 16:34 on December 05, 2024
CPU: AMD64 Family 26 Model 68 Stepping 0, AuthenticAMD
Number of GPUs Found: 2
GPU Model(s): AMD Radeon(TM) Graphics (gfx1036), NVIDIA GeForce RTX 4090
Project: Park Map
Align Photos: 404.9
Build Depth Maps: 296.2
Build Dense Cloud: 563.0
Build Tiled Model: 2803.2
Build DEM: 24.1
Build Orthomosaic: 356.3
Total Processing Time: 4447.7
Project: School Model
Align Photos: 241.2
Build Depth Maps: 154.5
Build Dense Cloud: 548.8
Build Mesh from Dense Cloud: 172.6
Decimate Mesh: 27.9
Build Texture: 163.4
Total Processing Time: 1308.4
Project: School Model using Depth Maps
Align Photos: 241.2
Build Depth Maps: 154.5
Build Mesh from Depth Maps: 310.9
Decimate Mesh: 42.2
Build Texture: 162.7
Total Processing Time: 911.5
Benchmark Completed at 18:18

*************************************
******** MACHINE 2 INTEL CPU ********
********         "PHIL"      ********
*************************************
---------------------------------------
---------------------------------------
benchmark new win11 install with nvidia latest drivers on Phil -SSD - hyperthread off
grrrrrrrrrrrr turn off powersaving and put on performance settings
--------------------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 15:36 on December 04, 2024
CPU: Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 2080 SUPER
Project: Park Map
Align Photos: 671.2
Build Depth Maps: 602.9
Build Dense Cloud: 1156.9
Build Tiled Model: 5441.9
Build DEM: 38.1
Build Orthomosaic: 569.7
Total Processing Time: 8480.7
Project: School Model
Align Photos: 451.9
Build Depth Maps: 290.2
Build Dense Cloud: 1250.1
Build Mesh from Dense Cloud: 317.1
Decimate Mesh: 53.5
Build Texture: 325.1
Total Processing Time: 2687.9
Project: School Model using Depth Maps
Align Photos: 451.9
Build Depth Maps: 290.2
Build Mesh from Depth Maps: 510.3
Decimate Mesh: 79.6
Build Texture: 298.3
Total Processing Time: 1630.3
Benchmark Completed at 18:57

--------------------------------------
phil windows 11 turned on hyperthreading
--------------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 19:03 on December 04, 2024
CPU: Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 2080 SUPER
Project: Park Map
Align Photos: 645.7
Build Depth Maps: 609.4
Build Dense Cloud: 1004.1
Build Tiled Model: 4817.5
Build DEM: 37.6
Build Orthomosaic: 576.4
Total Processing Time: 7690.7
Project: School Model
Align Photos: 447.9
Build Depth Maps: 310.1
Build Dense Cloud: 1113.8
Build Mesh from Dense Cloud: 291.0
Decimate Mesh: 47.4
Build Texture: 289.1
Total Processing Time: 2499.3
Project: School Model using Depth Maps
Align Photos: 447.9
Build Depth Maps: 310.1
Build Mesh from Depth Maps: 513.7
Decimate Mesh: 71.5
Build Texture: 281.7
Total Processing Time: 1624.9
Benchmark Completed at 22:07
--------------------------------------
phil linux  cpu off  hyperthreading on
--------------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 16:44 on December 05, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 2080 SUPER
Project: Park Map
Align Photos: 631.2
Build Depth Maps: 438.7
Build Dense Cloud: 730.3
Build Tiled Model: 3641.3
Build DEM: 31.4
Build Orthomosaic: 435.6
Total Processing Time: 5908.5
Project: School Model
Align Photos: 409.8
Build Depth Maps: 229.2
Build Dense Cloud: 684.2
Build Mesh from Dense Cloud: 223.0
Decimate Mesh: 38.2
Build Texture: 266.6
Total Processing Time: 1851.0
Project: School Model using Depth Maps
Align Photos: 409.8
Build Depth Maps: 229.2
Build Mesh from Depth Maps: 319.4
Decimate Mesh: 55.2
Build Texture: 225.8
Total Processing Time: 1239.4
Benchmark Completed at 19:04



andyroo

  • Sr. Member
  • ****
  • Posts: 458
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #3 on: December 11, 2024, 09:55:57 PM »
I attached a 7z archive with the complete metashape logfiles from both runs. Let me know if you need any software configs or want me to try any tweaks/metashape versions. I have Win11 and RL95 installed on separate Intel 2TB NVMe drives now (same model) on this workstation, so benching is pretty easy on either OS.

Here are two more runs on "Phil" - one on Rocky Linux 9.5 and one on a fresh install of Win 11.
Rocky Linux 9.5 / XFS was about 33% faster than Windows 11 / NTFS in this case.

Code: [Select]
--------------------------------------
Rocky Linux 9.5 Intel i9-9900K workstation "Phil" Dec 09
--------------------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 18:52 on December 09, 2024
CPU: x86_64
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 2080 SUPER
Project: Park Map
Align Photos: 603.2
Build Depth Maps: 436.9
Build Dense Cloud: 726.5
Build Tiled Model: 3650.1
Build DEM: 32.0
Build Orthomosaic: 435.2
Total Processing Time: 5883.9
Project: School Model
Align Photos: 409.7
Build Depth Maps: 229.6
Build Dense Cloud: 678.6
Build Mesh from Dense Cloud: 222.4
Decimate Mesh: 38.1
Build Texture: 261.8
Total Processing Time: 1840.2
Project: School Model using Depth Maps
Align Photos: 409.7
Build Depth Maps: 229.6
Build Mesh from Depth Maps: 336.4
Decimate Mesh: 56.1
Build Texture: 224.8
Total Processing Time: 1256.6
Benchmark Completed at 21:12
-------------------------
Win 11 Intel i9-9900k workstation "Phil" Dec 10
-------------------------
Agisoft Metashape Professional Version: 2.1.3
Benchmark Started at 15:52 on December 10, 2024
CPU: Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
Number of GPUs Found: 1
GPU Model(s): NVIDIA GeForce RTX 2080 SUPER
Project: Park Map
Align Photos: 698.1
Build Depth Maps: 625.9
Build Dense Cloud: 1049.1
Build Tiled Model: 4832.7
Build DEM: 38.0
Build Orthomosaic: 583.7
Total Processing Time: 7827.5
Project: School Model
Align Photos: 461.3
Build Depth Maps: 308.9
Build Dense Cloud: 1067.2
Build Mesh from Dense Cloud: 292.7
Decimate Mesh: 47.2
Build Texture: 291.4
Total Processing Time: 2468.7
Project: School Model using Depth Maps
Align Photos: 461.3
Build Depth Maps: 308.9
Build Mesh from Depth Maps: 505.2
Decimate Mesh: 70.5
Build Texture: 280.7
Total Processing Time: 1626.6
Benchmark Completed at 18:58
(Note that the previous Windows run on Phil might've been on a Win10 install since I had to enable TPM and SecureBoot in the BIOS to install Win11 this go-'round - sorry doing too many things to too many machines at the moment).

Props to William George of Puget Systems for developing and open-sourcing this benchmark. I likely wouldn't have figured this out if I hadn't been running his bench for the last ~5 years on various hardware!

PolarNick

  • Full Member
  • ***
  • Posts: 107
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #4 on: December 12, 2024, 12:54:56 PM »
Hi, andyroo.

Have you tried to run some kind of CPU benchmarks (f.e. Geekbench) to compare platforms' generic performance?

Because it seems that each step of processing has pretty significant speedup on Linux. So probably this can be due to OS's memory allocator performance or due to difference in CPU-cores scheduling.

andyroo

  • Sr. Member
  • ****
  • Posts: 458
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #5 on: December 13, 2024, 05:34:38 AM »
Hi PolarNick, I haven't tried Geekbench because I haven't heard a huge uproar about Linux being 30-40% faster than Windows across-the-board, and because I ran the Puget Systems benchmark comparing windows vs linux on two very different systems (AMD current generation CPU and Intel ~5y old CPU). I ran both benches on systems the way I build them and spin them up, which is standard OS install, then optimize a few things for performance profile, then get to work.

I don't want to spend a bunch of time identifying whether it's an OS, filesystem, or driver issue, but I welcome any results anyone else wants to share. Last I heard, and I think even tried about 8y ago, Windows and Linux were pretty much the same performance in Metashape, but that was when it was PhotoScan and before the Puget Systems benchmark, so I don't have any data to share. Regardless, at this point I'm planning to migrate our whole cluster to Linux, which I expect to be both painful and fun.  ;D

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #6 on: December 14, 2024, 11:49:04 PM »
I installed Arch linux and tomorrow I will try to bench on one of my project.

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #7 on: December 16, 2024, 04:26:10 PM »
So here are my results:
OS:
Win10 LTSC(21H2, build 19044.1348), updates disabled, nvidia driver 566.14 DCH, NTFS file system
Arch linux, latest kernel 6.12.4, XFCE4 desktop env., proprietary nvidia driver 565.77-5, EXT4 file system
Project: 1472 18Mpix JPEG
Metashape: Standard 2.1.4 build 19370

I locked frequencies for CPU(Intel 11700F) to 4.4GHz and for GPU(RTX 2060 super) to 1700MHz.
I also precached JPEG files using command "copy /b *.jpg nul" on Win and "cat *.jpg >/dev/null" on Linux ...otherwise there were huge time differences in detecting points phase because JPEG files on win are stored on 3/4 full SSD drive(1 partition) and on linux they are stored on other SSD(on almost empty EXT4 partition).

win | linux times:
257s   | 290s   ...detecting points (without JPEG precaching, times were 762s | 300s)
363s   | 361s   ...selecting pairs
941s   | 936s   ...matching points
609s   | 551s   ...estimating camera locations
1039s | 928s   ...gen. depth maps
2000s | 1827s ...gen. point cloud
...when I have free time, I will compare mesh and texture phase.

I am not sure why detect. points on linux is slower(I checked 3 times, to be sure).
Pure GPU tasks times are almost the same. When it comes to CPU side, there is 9-10% speed up in favor of linux... 5% are caused by fact I am running apps on win(like MSI afterburner, HWiNFO, desktop widgets, cusotom firewall, throttle stop, FAN control, email checker), the rest 5% can be caused by OS kernel.

andyroo

  • Sr. Member
  • ****
  • Posts: 458
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #8 on: December 16, 2024, 09:59:03 PM »
Hi Bzuco,

Thanks for testing. looks like you had more like a 10% difference. Makes me wonder (1) how much difference I'd see with Win10 LTSC vs full win10/win11 install/bloat, (2) and (2(less curious)) how much difference I'd see between XFS and ext4 and Rocky Linux and other flavors. Will see if we can get a LTSC channel install somehow

olihar

  • Sr. Member
  • ****
  • Posts: 282
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #9 on: December 16, 2024, 10:02:45 PM »
Install Windows 11 with a tool like this, it gets rid of all the bloatware that comes with standard Windows 11 installation.

https://github.com/memstechtips/WIMUtil

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #10 on: December 16, 2024, 11:55:04 PM »
LTCS comes with less features. I chose it, because those features were not important for me.
In terms of performance I never compared them directly side by side. The important is disable services you don't need(updates, cortana, MS store, defender, superfetch, telemetry, ...). Then you will be closer to linux performance, but it's level is unreachable, because number of active services/processes/threads is way lower than on win... except special unofficial versions like "Phoenix LiteOS 10 LTSC (19044.1381) FBConan" 1.3GB ISO, which are designed to be minimal.
So do not expect significant diferences between untouched LTSC and full PRO version in terms of performance.
I am still on win10 LTSC. Win 11 LTSC is probably a little worse .

I am not expert on linux file systems, but what I searched, XFS is a bit better optimized for large file transfer and parallel read/write scenarios. I don't think Metashape could benefit much from XFS. Metashape is reading and writing in batches, everything happens mostly in memory.

Rocky linux vs other distros, it depends only on used kernel version and also on running services. I chose Arch linux, because it allows me to install everything from scratch, so I can end up with very minimal custom version.
Try to avoid large distributions and chose some lite editions.

andyroo

  • Sr. Member
  • ****
  • Posts: 458
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #11 on: December 17, 2024, 10:32:18 AM »
Hi Bzuco,

Thanks for your suggestions, and it's reassuring to see that some folks are getting more similar #s with Windows/Linux. Hopefully I can find and disarm the culprit. I am already trying to see if I can install LTSC. I read the same stuff re XFS and ext4 but "large" and "small" files aren't well defined, so I'm still uncertain.. I have another identical NVME drive in "Phil" so I *could* try RL9.5+ext4... Don't have a lot of choices unfortunately - it's basically Windoze Server/Pro (unless I can get LTSC which apparently nobody in our org ever asks for), or Rocky/RedHat.

Up to now I didn't spend much time making/using customized debloated windows distros because I hadn't seen that level of impacts in other benches, but it's been like 5 years since I paid attention. Checking if I have access to LTSC, and looking into open source debloat install scripts, and might roll up my sleeves and start removing installed services, but I'm not touching unofficial versions because security. Probably going to migrate our cluster to Rocky Linux if I can't get LTSC to bench close to it. Up to now I do a standard Win10 (now 11) Pro install with oobe\bypassnro and disable all optional crap/telemetry, select performance profile, disable automatic restart, then start cranking. I haven't tried too much the debloating and turning off services because I didn't realize I was taking such a performance hit with the standard "informed expert install", but wow it's a lot. definitely re-evaluating that approach.

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #12 on: December 17, 2024, 12:27:17 PM »
I can also test XFS, for me it means only shrink system partition and create another partition, so easy task. I will let you know.

For debloating win, there are more open source utils, I used Sophia script https://github.com/farag2/Sophia-Script-for-Windows  in combination with "O&O ShutUp10.exe" utility.
Some video tutorials how to debloat you can find on https://www.youtube.com/@ChrisTitusTech  and https://www.youtube.com/@CyberCPU

If you want to create custom windows installation ISO with already removed services, preconfigured options, ... (everything is done from origianl windows ISO), then it is best to use Win Toolkit utility. You will also find tutorials on those YT channels.

For you it would be maybe option to install Linux as a secondary system side by side with win(if it is still somehow important for you). In that case win can time to time after updates disable quietly  ;D multi boot option, which can be fixed.

EDIT: I tried XFS and the times were the same, except gen. point cloud, which was for some reason 2% slower.
« Last Edit: December 17, 2024, 05:54:58 PM by Bzuco »

CheeseAndJamSandwich

  • Full Member
  • ***
  • Posts: 186
    • View Profile
    • Sketchfab Models
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #13 on: December 18, 2024, 01:42:04 AM »
I locked frequencies for CPU(Intel 11700F) to 4.4GHz and for GPU(RTX 2060 super) to 1700MHz.

It would be good to also run this benchmark without any kneecapping of hardware.

A lot of the recent hoo-har wrt Windows 11 vs. Linux is in the way Linux seems to utilise the hardware better, especially the newer CPUs.
Locking stuff down only tests Metashape compiled for Windows vs. Metashape compiled for Linux.  Not so much Windows vs. Linux.
So perhaps just let it rip as fast as possible on both OS'

Afterall, we just want/need our projects to run as fast as possible!

I just ran a quick test with a small drone scan and got totals of 701 seconds for RL vs. 848 seconds for W11... (20%), doing a saved batch of align, model, texture, & dem.
I'm gonna throw one of my bigger underwater scans at them to see what that gives.


And Bzuco, obviously it'd be very interesting to see how your method of running multiple local nodes is also affected.
Andyroo, you should look into what Bzuco has detailed in one of his previous threads.  It's depressing seeing you hardware sitting there only 20% untilsed for hours, when you can sneakily run 6-8 worker nodes and have it melting the hardware for most of the time, giving much faster runs!  This needs to be core functionality asap.

My 'little' scan of our dive site, 'Manta Point'.  Mantas & divers photoshopped in for scale!
https://postimg.cc/K1sXypzs
Sketchfab Models:
https://sketchfab.com/cheeseandjamsandwich/models

Bzuco

  • Full Member
  • ***
  • Posts: 246
    • View Profile
Re: Metashape 2 Benchmark - Windows 30-40% slower than Linux?!
« Reply #14 on: December 18, 2024, 11:52:41 AM »
Quote
Locking stuff down only tests Metashape compiled for Windows vs. Metashape compiled for Linux.  Not so much Windows vs. Linux.
Use the same frequencies is mandatory when it comes to benchmarking software quality like OS kernel algorithms, which is the most important in case linux vs. win and according to my findings and measurements it is under ~5%. The rest % differences which I noticed are not related to linux kernel and are caused simply by running many background processes/services not needed during Metashape processing and SSD behaviours in case of free/full disk space I have(therefore I precached all JPEG files).

I could grab heavy Linux distro and compare it with very light Win distro and the results could be suddenly in favor of Win.

In case of running many local workers, there could be advantage in linux when using XFS file system, but only in subtasks where data are written to disk, which is only a tiny % time fraction of the the whole processing. The rest of the time data are read from and write to RAM, where file system has no affect.