"Found X GPUs" message repeatedly during detect points in new versions - slower?

[andyroo]

I remember back in 1.4 or 1.5 it seemed like the point detection was continuous and very fast on 2-3 GPUs. Now I get a "found 2 GPUs" message every 20 images with a 1.5s pause, then a fraction of a second pause after every pair of photos for 10 pairs of photos, then the sequence repeats.

This seems like a significantly slower performance than before, but I'm not sure if it's actually faster because I don't remember what version to compare it with. Also if there are other performance benefits, I understand, I'm just curious what change.

Finally if there's a tweak to use an old method, or to just assume that I have the same video cards and they're behaving properly so it avoids the 1.5s pause ever 20 photos that would be cool because when I'm doing projects with 100,000 photos that equates to just over 2h of pause time, and it's pretty clear looking at my CUDA graphs that it's not continuously using the CUDA.

I'm willing to experiment with different tweaks if they're available, and happy to report my results.

[CheeseAndJamSandwich]

Perhaps the GPUs are coming up for air, grabbing a mars bar and a swig of water, before getting back down to work! 

[Bzuco]

@andyroo
One fact, that right after you drag and drop photo files into project, metashape is checking all files(quite quick process), but on background all files are read from disk and automatically stored in RAM as cached files.
So my point is, that if you start align process right after checking process in dialog is done, you have few photos in RAM and rest still on disk.
You have 3 GPUs, which is enormous pressure on reads from disk.
Also I dont know how much RAM your system have, how big are your photos(megabytes) and if your amount of RAM is able to absorb all 100k files for quick feeding your 3 GPUs.
Photos stored on HDD instead of SSD can cause also significant slowness if photos are not already cached in RAM.

What you describing "1.5s delay every 20 photos" could be cause by facts I mentioned, but also some changes between 1.4/1.5/1.7 versions...can't tell more.
You can check how much your disk is reading during point detection and what is the value of cached RAM in task manager.
You also should be able to see fluctuating PCIE bus usage on each GPU in gpuz utility during delays.

I made quick test on small 200 photo set. When I started align process after all reads from disk ended, the speed of point detection was constant and quick. When I started align process right after drag and drop, point det. on half photos was quick and then the process was slower until the end, because my HDD was stil lreading photos.

[Alexey Pasumansky]

Hello Andy,

Which GPU models do you have installed, what driver version is used and which OS you are working on?