Forum

Author Topic: GPU fail changing to CPU  (Read 18055 times)

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
GPU fail changing to CPU
« on: July 04, 2012, 01:42:13 AM »
Question 1

I am wondering when I get this error, I guess its due to the images being to large for the GPU to handle, can that be.. ? I am doing 21MP images on ultra and I have GTX 580 with 1.5GB RAM.

NOTE:
Code: [Select]
ocl_engine.cpp line 320: clWaitForEvents failed, CL_MEM_OBJECT_ALLOCATION_FAILURE
GPU processing failed, swithing to CPU mode.

Question 2

So when it does indeed change to CPU from the GPU, it does not seem to utilize 100% CPU, I guess that is due to Photoscan being set up not using all the cores, I have it set to feed info to the GPU on 2 cores (Hyper Threading CPU) so can it be right that Photoscan still has these cores disabled even when it fails to use GPU? It should in optimal world turn back on these 2 cores if they are unused becuase the GPU has been disabled.
« Last Edit: July 04, 2012, 01:44:45 AM by olihar »

FoodMan

  • Sr. Member
  • ****
  • Posts: 477
    • View Profile
Re: GPU fail changing to CPU
« Reply #1 on: July 04, 2012, 09:42:37 AM »
mmm.. not sure... but maybe the GPU is switching back ON at the next image...? or does it go OFF 'til the construction is complete...?

f/

olihar

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: GPU fail changing to CPU
« Reply #2 on: July 04, 2012, 01:52:52 PM »
mmm.. not sure... but maybe the GPU is switching back ON at the next image...? or does it go OFF 'til the construction is complete...?

f/

It seems to get

ocl_engine.cpp line 320: clWaitForEvents failed, CL_MEM_OBJECT_ALLOCATION_FAILURE
GPU processing failed, swithing to CPU mode.

on every image.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: GPU fail changing to CPU
« Reply #3 on: July 04, 2012, 03:22:12 PM »
Hello olihar,

This problem may occur if there are some processes run on the GPU (for example, another PhotoScan geometry reconstruction process).

And regarding the second question, it is recommended to enable all CPU cores in case there are no OpenCL devices enabled.
Best regards,
Alexey Pasumansky,
Agisoft LLC

duststorm

  • Newbie
  • *
  • Posts: 3
    • View Profile
Re: GPU fail changing to CPU
« Reply #4 on: August 07, 2014, 03:56:17 PM »
I've just experienced the same issues with a GTX660ti card when upgrading to the 340.52 build of Nvidia drivers.
I downgraded the drivers to an old version 305.68 and everything works fine again.

Limpopo_River

  • Newbie
  • *
  • Posts: 25
    • View Profile
Re: GPU fail changing to CPU
« Reply #5 on: August 09, 2014, 07:25:56 PM »
I've had a similar problem generating a dense point cloud with an ATI Radeon 5870 GPU, 12-core (2 x X5650) proceessor, and 32 Gb of RAM on a Mac OS X 10.8.5.  I didn't get a memory allocation error, the log simply states
Quote
...Reconstructing depth...
[CPU] estimating 1549x2340x160 disparity using 775x1170x32f tiles, offset 0
[GPU] estimating 1626x2512x96 disparity using 542x628x32f tiles, offset 0
GPU processing failed, switching to CPU mode
(I attached a slightly longer excerpt from the log file, but the entire log is too big to attach here.)

I've seen others report similar problems with different GPUs, with or without memory allocation errors, but no clear explanation seems to have emerged, except to test the GPU.  I ran the suite of Geek3D GPU tests, and the GPU appears to be performing normally.  However, the entire dense point cloud generation averaged only 120 million samples per second, which seems very slow.   It's a used Mac Pro with a new installation of OS X 10.8.5, and Apple Care is now installing OS X 10.9 and testing the hardware.  I'll report back if they find any problems or if this cures it, but any other suggestions or explanations for this problem would be much appreciated. 

Limpopo_River

  • Newbie
  • *
  • Posts: 25
    • View Profile
Re: GPU fail changing to CPU
« Reply #6 on: September 05, 2014, 09:53:27 PM »
I just wanted to report that the Radeon ATI 5870 GPU wasn't at fault.  The problem apparently had to do with the way the operating system was installed by the seller of the used Mac Pro.  I suspected this to be the problem because I ran separate tests of the GPU and it checked out fine.  Although I couldn't identify the specific source of the problem, it was solved by having Apple reinstall the OS.  Fortunately, the recently purchased used Mac Pro came with Apple Care and it was easily resolved at no additional cost.  Photoscan is now using the GPU normally.

nils

  • Newbie
  • *
  • Posts: 5
    • View Profile
Re: GPU fail changing to CPU
« Reply #7 on: September 11, 2014, 12:43:48 PM »
Had the same problem but reverting from NVIDIA driver 340.52 to 337.88 fixed it

andyroo

  • Sr. Member
  • ****
  • Posts: 438
    • View Profile
Re: GPU fail changing to CPU
« Reply #8 on: September 17, 2014, 03:32:25 AM »
I had this problem with drivers installed and updated drivers. 2 cards

GTX 560 Ti running 337.88 driver
Radeon HD 7900 running 9.14.10.01029 driver

both are stable with these versions.

DFin

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: GPU fail changing to CPU
« Reply #9 on: September 17, 2014, 10:34:35 PM »
I can confirm having issues with nvidia 340.xx too on a GTX 760 and a GTX 880m. Rollback to 337.88 solved the issue.

As there are numeral reports of issues with 340.xx driver i think agisoft should provide an official recommendation not to use 340 drivers for now. This might save some people time figuring out them self.

ThomasVD

  • Full Member
  • ***
  • Posts: 107
    • View Profile
Re: GPU fail changing to CPU
« Reply #10 on: September 21, 2014, 06:16:14 PM »
Had the same issue on my NVIDIA GTX 765M, rolling back the drivers solved the issue.

an198317

  • Newbie
  • *
  • Posts: 42
    • View Profile
Dell XPS 8700 w/ NVIDIA GeForce GTX 745 GPU fails?
« Reply #11 on: September 30, 2014, 10:16:37 PM »
Hi,

I just got a new Dell XPS 8700 desktop with a NVIDIA GeForce GTX 745 OEM graphic card, and I have upgraded to the latest graphic card driver.

But when I run my python code to run the model, the console showed that the GPU fails and switched to CPU. I think this NVIDIA GeForce GTX 745 OEM card supports OpenCL 1.1. So I am curious that if someone had the similar problem?

Thanks,

an198317

  • Newbie
  • *
  • Posts: 42
    • View Profile
Re: GPU fail changing to CPU
« Reply #12 on: September 30, 2014, 10:34:49 PM »
Thanks for putting my original post under this topic. It seems to be a driver issue. I will see if I can reinstall the older driver to solve the problem.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: GPU fail changing to CPU
« Reply #13 on: September 30, 2014, 10:37:19 PM »
Hello Nan,

Please check driver 337 or older.

PhotoScan version 1.1 should work correctly with latest nVidia drivers.
Best regards,
Alexey Pasumansky,
Agisoft LLC

an198317

  • Newbie
  • *
  • Posts: 42
    • View Profile
Re: GPU fail changing to CPU
« Reply #14 on: October 01, 2014, 08:46:33 PM »
Hi Alexey,

Thank you very much for your information. I switched back to 337.88 version, and it worked perfectly with GPU! I will look forward to trying the PS 1.1.

Thanks,


Hello Nan,

Please check driver 337 or older.

PhotoScan version 1.1 should work correctly with latest nVidia drivers.