CUDA problem

[j.seinturier]

Hello,

I'm using Agisoft Photoscan 1.4.0 and for now many version, it is impossible for me to get CUDA optimization activated within the software. This problem occurs with 3 machines (that are SuperMicro based) with 3 differents GPU adapters (one GTX 980 for the first computer, two GTX 1080Ti for the two other computers).

The problem is always the same: When i go to Photoscan preferences and when i choose GPU pane, the photoscan console displays "cudaGetDeviceCount failed: unknown error (30)" and i cannot see any GPU detected within the frame.

I'm using on all the machines the latest NVIDIA drivers with updated Windows 10 and i've tested with other drivers, a system reinstall but no solution has worked.

Moreover, if i use the OpenCL version of Photoscan, its detect the whole GPU and use it on all my computers.

On the other hand, when i use other CUDA softwares, they're works perfectly on the computers.

You can find in attachment some screenshots of photoscan on one of my computers when the GPU problem occurs and a screenshot of my NVIDIA configuration panel.

I've got this problem since the apparition of CUDA capabilities within photoscan, support ask me to make many tests but no solution has arised. Is somebody have the same problem and is somebody can help me to solve it ?

Thanks a lot,

Julien

For more information here is the configuration of one of the involved computer:

Motherboard: SuperMicro
CPU: 2 x intel Xeon
GPU: 2 x GeForce 1080 Ti
RAM: 512Gb
System: Windows 10 x64 Fall Creators update

[Kenneniah]

We are having the same exact problem.
Windows 10 64bit, have tried Anniversary, first Creator update and Fall Creators update
Dual Xeon E-5-2680
Dual Quadro P5000s

Tried all available versions of the Nvidia driver.
All other Cuda programs have used both cards with no issue.
Photoscan comes up with the same cudaGetDeviceCount failed message.

Works if we disable one of the GPUs in the Nvidia Control Panel.
Works with both cards in OpenCL version of Photoscan

Nvidia tech checked over our system and everything points to GPUs and Cuda working correctly, just not in Photoscan

[j.seinturier]

Yes, problem seems only to be linked with the count of graphics adapters when there is more than one adapter... But this problem is quite old for me.

[Alexey Pasumansky]

Hello Julien,

We are not still able to reproduce the problem on our side, so it can be hardly said, what is the reason of the issue.

I don't think it is directly connected with the number of GPUs installed on the system, since we are also testing multi-GPU setups for GTX and Quadro configurations.

[j.seinturier]

During the time that we investigate, it would be interesting for me to access to OpenCL version of the last released photoscan as i use it on 3 machines (i've 10 Photoscan Pro licences) that have this problem.

It sounds like the problem occurs many on Supermicro superserver configurations.

[Kenneniah]

Our is happening on a Dell Precision T7910.
Out of curiosity, what chipset are your Supermicro boards using?

The 7910 is using the Intel C610/X99 chipset

[ruyi7952]

I have working double 1080ti and double 1080 and double M40 and double M60 and double P100。

I have using version 1.3.* and 1.4.0，Work normally。

My experience is to install DirectX web setup and recommend the drive version used by enterprise users. It's not very new, but it's more stable.

[j.seinturier]

The problem for us is that any other CUDA or OpenCL program works perfectly... So i don't know why i have to reinstall DirectX. But maybe i will try.

[Hamish]

I have had the same problem, when I first encountered it back in early March I just rolled the graphics driver back to an earlier version as it was installing the latest graphics driver that had caused the problem. I've recently installed the latest build of PhotoScan and upon installing the latest graphics driver the issue has re-emerged - and sort of solved by going back to the earlier graphics driver.
After each install I restart the computer before running PhotScan.

CPU: AMD FX-8350 8 core 4ghz x64
Motherboard: MSI 990FXA-GD65
Ram: 32GB
Graphics Nvidia GTX 760 4GB x2
Windows 8.1 Pro

Photoscan Standard x64 1.4.1 (5925)
Photoscan Standard x64 1.4.2 (6205)

cudaGetDeviceCount failed: unknown error (30)

using Nvidia driver Versions
391.01-desktop-win8-win7-64bit-international-whql
397.64-desktop-win8-win7-64bit-international-whql

---
The only way I have got around the problem is using an earlier Nvidia driver, everything seems to work fine.

388.71-desktop-win8-win7-64bit-international-whql (downloaded 6 January 2018)

---
Update 01/2019, I've explored the problem a little further - For me this is an NVIDIA driver problem not a photoscan/metashape issue.
Other software isn't able to access cuda either. I'm using a couple of GTX 760 cards - reading across various forums others are also having driver problems with nvidia 700 series cards and recent drivers.
As long as Metashape doesn't require Cuda above v9.0, hopefully I should be fine.