Agisoft Metashape

Agisoft Metashape => General => Topic started by: olihar on July 04, 2012, 01:42:13 AM

Title: GPU fail changing to CPU
Post by: olihar on July 04, 2012, 01:42:13 AM
Question 1

I am wondering when I get this error, I guess its due to the images being to large for the GPU to handle, can that be.. ? I am doing 21MP images on ultra and I have GTX 580 with 1.5GB RAM.

NOTE:
Code: [Select]
ocl_engine.cpp line 320: clWaitForEvents failed, CL_MEM_OBJECT_ALLOCATION_FAILURE
GPU processing failed, swithing to CPU mode.

Question 2

So when it does indeed change to CPU from the GPU, it does not seem to utilize 100% CPU, I guess that is due to Photoscan being set up not using all the cores, I have it set to feed info to the GPU on 2 cores (Hyper Threading CPU) so can it be right that Photoscan still has these cores disabled even when it fails to use GPU? It should in optimal world turn back on these 2 cores if they are unused becuase the GPU has been disabled.
Title: Re: GPU fail changing to CPU
Post by: FoodMan on July 04, 2012, 09:42:37 AM
mmm.. not sure... but maybe the GPU is switching back ON at the next image...? or does it go OFF 'til the construction is complete...?

f/
Title: Re: GPU fail changing to CPU
Post by: olihar on July 04, 2012, 01:52:52 PM
mmm.. not sure... but maybe the GPU is switching back ON at the next image...? or does it go OFF 'til the construction is complete...?

f/

It seems to get

ocl_engine.cpp line 320: clWaitForEvents failed, CL_MEM_OBJECT_ALLOCATION_FAILURE
GPU processing failed, swithing to CPU mode.

on every image.
Title: Re: GPU fail changing to CPU
Post by: Alexey Pasumansky on July 04, 2012, 03:22:12 PM
Hello olihar,

This problem may occur if there are some processes run on the GPU (for example, another PhotoScan geometry reconstruction process).

And regarding the second question, it is recommended to enable all CPU cores in case there are no OpenCL devices enabled.
Title: Re: GPU fail changing to CPU
Post by: duststorm on August 07, 2014, 03:56:17 PM
I've just experienced the same issues with a GTX660ti card when upgrading to the 340.52 build of Nvidia drivers.
I downgraded the drivers to an old version 305.68 and everything works fine again.
Title: Re: GPU fail changing to CPU
Post by: Limpopo_River on August 09, 2014, 07:25:56 PM
I've had a similar problem generating a dense point cloud with an ATI Radeon 5870 GPU, 12-core (2 x X5650) proceessor, and 32 Gb of RAM on a Mac OS X 10.8.5.  I didn't get a memory allocation error, the log simply states
Quote
...Reconstructing depth...
[CPU] estimating 1549x2340x160 disparity using 775x1170x32f tiles, offset 0
[GPU] estimating 1626x2512x96 disparity using 542x628x32f tiles, offset 0
GPU processing failed, switching to CPU mode
(I attached a slightly longer excerpt from the log file, but the entire log is too big to attach here.)

I've seen others report similar problems with different GPUs, with or without memory allocation errors, but no clear explanation seems to have emerged, except to test the GPU.  I ran the suite of Geek3D GPU tests, and the GPU appears to be performing normally.  However, the entire dense point cloud generation averaged only 120 million samples per second, which seems very slow.   It's a used Mac Pro with a new installation of OS X 10.8.5, and Apple Care is now installing OS X 10.9 and testing the hardware.  I'll report back if they find any problems or if this cures it, but any other suggestions or explanations for this problem would be much appreciated. 
Title: Re: GPU fail changing to CPU
Post by: Limpopo_River on September 05, 2014, 09:53:27 PM
I just wanted to report that the Radeon ATI 5870 GPU wasn't at fault.  The problem apparently had to do with the way the operating system was installed by the seller of the used Mac Pro.  I suspected this to be the problem because I ran separate tests of the GPU and it checked out fine.  Although I couldn't identify the specific source of the problem, it was solved by having Apple reinstall the OS.  Fortunately, the recently purchased used Mac Pro came with Apple Care and it was easily resolved at no additional cost.  Photoscan is now using the GPU normally.
Title: Re: GPU fail changing to CPU
Post by: nils on September 11, 2014, 12:43:48 PM
Had the same problem but reverting from NVIDIA driver 340.52 to 337.88 fixed it
Title: Re: GPU fail changing to CPU
Post by: andyroo on September 17, 2014, 03:32:25 AM
I had this problem with drivers installed and updated drivers. 2 cards

GTX 560 Ti running 337.88 driver
Radeon HD 7900 running 9.14.10.01029 driver

both are stable with these versions.
Title: Re: GPU fail changing to CPU
Post by: DFin on September 17, 2014, 10:34:35 PM
I can confirm having issues with nvidia 340.xx too on a GTX 760 and a GTX 880m. Rollback to 337.88 solved the issue.

As there are numeral reports of issues with 340.xx driver i think agisoft should provide an official recommendation not to use 340 drivers for now. This might save some people time figuring out them self.
Title: Re: GPU fail changing to CPU
Post by: ThomasVD on September 21, 2014, 06:16:14 PM
Had the same issue on my NVIDIA GTX 765M, rolling back the drivers solved the issue.
Title: Dell XPS 8700 w/ NVIDIA GeForce GTX 745 GPU fails?
Post by: an198317 on September 30, 2014, 10:16:37 PM
Hi,

I just got a new Dell XPS 8700 desktop with a NVIDIA GeForce GTX 745 OEM graphic card, and I have upgraded to the latest graphic card driver.

But when I run my python code to run the model, the console showed that the GPU fails and switched to CPU. I think this NVIDIA GeForce GTX 745 OEM card supports OpenCL 1.1. So I am curious that if someone had the similar problem?

Thanks,
Title: Re: GPU fail changing to CPU
Post by: an198317 on September 30, 2014, 10:34:49 PM
Thanks for putting my original post under this topic. It seems to be a driver issue. I will see if I can reinstall the older driver to solve the problem.
Title: Re: GPU fail changing to CPU
Post by: Alexey Pasumansky on September 30, 2014, 10:37:19 PM
Hello Nan,

Please check driver 337 or older.

PhotoScan version 1.1 should work correctly with latest nVidia drivers.
Title: Re: GPU fail changing to CPU
Post by: an198317 on October 01, 2014, 08:46:33 PM
Hi Alexey,

Thank you very much for your information. I switched back to 337.88 version, and it worked perfectly with GPU! I will look forward to trying the PS 1.1.

Thanks,


Hello Nan,

Please check driver 337 or older.

PhotoScan version 1.1 should work correctly with latest nVidia drivers.
Title: Re: GPU fail changing to CPU
Post by: DCK on October 03, 2014, 08:49:12 PM
Quote
And regarding the second question, it is recommended to enable all CPU cores in case there are no OpenCL devices enabled.

Alexey,

I thought it was recommended to disable one CPU core per GPU enabled, and double this amount if you have hyper-threading. So, with one GPU, 8 CPU cores, and hyperthreading, one should have 6 CPU cores active. Am I mistaken?