Strange test results on different GPU

[IvanSobol]

I found articles comparing different GPUs in Metashape and results very strange.
https://www.pugetsystems.com/labs/articles/Agisoft-Metashape-1-6-4---NVIDIA-GeForce-RTX-3080-Performance-1884/
https://www.pugetsystems.com/labs/articles/Agisoft-Metashape-1-6-5---NVIDIA-GeForce-RTX-30-Series-Multi-GPU-Performance-2009/

By combining data from these two articles dual 3080 only 10% faster than single 1080ti - can't believe that. By floating point tests results difference about +400%, I think difference in Metashape should be about that, definitely not 10%.

Tests were done using Metashape 1.6, but that version already supported acceleration for most features. Quote from 1.6 manual - "Metashape supports accelerated image matching; depth maps reconstruction; depth maps based mesh, DEM and tiled model generation; texture blending; photoconsistent mesh refinement operation due to the graphics hardware (GPU) exploiting"

Can anyone confirm that these articles absolutely wrong? How they get so wrong results?

[Bzuco]

Hi Ivan,

those tests are mostly correct, but with older agisoft version.
But what is more important total processing time can not reflect correctly performance differences between different GPUs. There are several subtasks, not all are fully multithreaded, there are delays caused by transfering and allocating data in RAM and VRAM, reading photos from disk, ... and during that moments CPU or GPU are not 100% utilized.
But you can be sure, that if some GPU has 2x more floating point performance, it also computes some subtask almost 2x faster when is 100% utilized.

I attached just graph of depth maps generation (puget school map benchmark project) with monitoring CPU/GPU usage and gpu memory allocation. As you can see, when loading/unloading happens on GPU, there is drop in GPU load. On the background CPU load is also fluctuating, because is preparing data for GPU. GPU it self is limited by its memory bandwidth. It also depends on project type, number of photos, processing quality, other parameters... Higher quality settings can utilize GPU longer @100%. Lower settings are causing more often loading/unloading. The graph of alignment phases would look different.

Maybe we should start our community benchmark on our photosets and our HW configurations with latest Metashape version.

[IvanSobol]

I would participate in user benchmark. Particularly I am interested if difference between 1080ti and 3080 still so small - used 3080 cost more than in 2 times higher than used 1080ti and 7% difference don't worth it, hope it different now, because this situation looks absolutely strange for cards that have in 2.6 times different FP32 performance even taking in account data transfer time loss. Even more disappointing situation with twin cards, because 5% gain don't worth double price. I see that many people on this forum have twin cards and all of them have only 5% gain? If situation still so, then twin cards absolutely wrong idea and right idea to buy second PC and install second card there.

[Bzuco]

It is not just 5%. Check the numbers in colors.

Align photos consists from 4 subtask:
1) is heavily penalized by only single threaded decoding jpeg and extracting data, otherwise it would be utilized by GPU @max.
2,3) is pretty well utilized by GPU
4) is not utilized by GPU at all

Depth maps generation results in puget bench. are affected by fact, that medium quality does not create huge task for GPU with 30TFLOPS and batch systen Metashape is using.

I agree, that instead of buying second 3080, is better second PC or run two projects at once in two instances of Metashape, or buying better CPU(and improove times on subtasks which are not GPU accelerated).
I do not know how many users have 2 GPUs rigs, but several users have projects with several thousands photos and when they computing depth maps or matching points phase, 2 GPUs will be still usefull.

[IvanSobol]

Why that results look too strange for me. In that tests almost any card and even twin top card give almost the same performance - difference very very small. But I remember time a few years ago when I've changed GTX1060 to GTX1070 and I measured difference and it was around difference in games - about in 1.5 times (not saved precise results). Then I've changed Intel 4 core CPU (8 threads) 4.4Ghz to Intel 10 cores (20 threads) 4.8Ghz and to GTX1080ti and got difference in 2 times (total - align, mesh, texture) and I am sure that role of CPU was not really important in total time, because I've measured performance of CPU compared to GPU and it was about in 10 times. So I am absolutely don't understand how that was possible to get so strange results in that tests. Maybe corrupted PCIE bus of motherboard?

[Bzuco]

CPU is important in total time, because estimating camera location(lot of time), dense cloud generation(lot of time) are not GPU accelerated and mesh generation and textures are only partly GPU accelerated....+ first detecting points phase also take significant amount of time, because depends on CPU data preparation, which is single threaded process.
Total time is bad metric, if we are changing only GPUs and CPU is the same.
PCIE bus is barely used during whole process in terms of bandwidth.

In attachment old and new GPU, but same CPU. You can see, that Alignment time 2min31s does not change, because is not gpu accelerated. Big speed up was on depth maps generation from 19min26s to 6min15s. But dense cloud generation(not GPU accelerated) took 1hour5min(did not measure with RTX2060, because is not gpu accelerated). Total time would by ~92min vs. ~76min. which does not reflect FP32 difference of GPUs.

[DayGeckoArt]

I recently got an RTX 4060 which is a pretty low end card, but does very well in floating point benchmarks because it has the new CUDA cores that can do both. It seems to do very well with Metashape though I haven't done a direct comparison with my older 2080 Ti card. Anybody know if Metashape is mostly floating point or integer math? I would think FP...

[Bzuco]

@DayGeckoArt
It is mostly FP and thanks to double FP units since RTX 3000, RTX 4060 could achieves the same performance as 2080 ti. It has slightly higher single precision performance, but much weaker memory bandwidth(272 vs. 616)...so if L2 cache does not help, it could be even slower...worth to test  .