Forum

Author Topic: CUDA Crash during dense point cloud processing  (Read 8727 times)

edtriplett

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: CUDA Crash during dense point cloud processing
« Reply #15 on: March 27, 2018, 07:14:37 PM »
Thanks for showing me this. I am still getting the same error in Standard. I attached the console text here.
I assume the DWORD item was 64bit, correct?

edtriplett

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: CUDA Crash during dense point cloud processing
« Reply #16 on: March 30, 2018, 11:26:55 PM »
Sorry to bother again. I am posting this here because I can not add an attachment in a personal message.
We just purchased a Agisoft Pro license to work around this issue because this code worked the first time with the demo of Pro:

Quote
PhotoScan.app.settings.setValue("main/depth_max_gpu_multiplier", 1)

Unfortunately, I am getting the same crash again in Pro with this code pasted into the console before running the dense cloud process. I attached the console text here.

Again the error code is this:
Quote
Error: Kernel failed: the launch timed out and was terminated (6) at line 821

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14004
    • View Profile
Re: CUDA Crash during dense point cloud processing
« Reply #17 on: March 30, 2018, 11:44:39 PM »
Hello edtriplett,

To understand, if it is the hardware-related issue, I can suggest several tests:
- try to physically remove all GPUs but one and check, if the processing works or a similar issue occurs, you can also try putting GPU to another slot,
- if you have such an option - after the first test, try to put the GPUs (one by one) to another computer and also check, if the similar issue is observed on the second machine.

It may be also worth trying to run some CUDA-based stress tests on your configuration.
Best regards,
Alexey Pasumansky,
Agisoft LLC

edtriplett

  • Newbie
  • *
  • Posts: 15
    • View Profile
Re: CUDA Crash during dense point cloud processing
« Reply #18 on: May 24, 2018, 11:03:47 PM »
Hi Alexey,
I have gone back to the manufacturer of the PC and they suggested turning off a few CPUs in windows with the "Processor Affinity" option box. This did not have any effect. I am still getting the CUDA crashes.

I took your advice and detached all but one of the GPUS and everything worked fine. Attaching two or all three causes the crash. I am attaching the reports from Cuda-Z software showing the performance of each GPU. I hope that information is useful to you, because I can't interpret it.