Forum

Author Topic: Cuda stream destroy failed unknown error (win10 pro)  (Read 6036 times)

Pkroks

  • Newbie
  • *
  • Posts: 39
    • View Profile
Cuda stream destroy failed unknown error (win10 pro)
« on: March 17, 2019, 12:47:12 AM »
Hi,

I have just updated my windows 64 bit to

windows pro version 1809 (build 17763.379)
nvidia gtx 1070 with driver version 419.35

agisoft metashape version 1.5.1 build 7618

I try to build the depth maps using CUDA enabled card and it gives me the error:

2019-03-16 23:33:38 GPU depth map warping failed: unknown error (30) at line 179
2019-03-16 23:33:38 using CPU implementation...
2019-03-16 23:33:41 Error: Kernel failed: unknown error (30) at line 115
2019-03-16 23:33:41 GPU processing failed, switching to CPU mode
2019-03-16 23:33:41 [CPU] estimating 994x1321x128 disparity using 994x1321x8u tiles
2019-03-16 23:33:42
2019-03-16 23:33:42 Depth reconstruction devices performance:
2019-03-16 23:33:42  - 5%    done by CPU
2019-03-16 23:33:42  - 95%    done by GeForce GTX 1070
2019-03-16 23:33:42 Total time: 85.324 seconds
2019-03-16 23:33:42
2019-03-16 23:33:42 Warning: cudaStreamDestroy failed: unknown error (30)
2019-03-16 23:33:42 Warning: cudaStreamDestroy failed: unknown error (30)
2019-03-16 23:33:42 Error: unknown error (30) at line 211
2019-03-16 23:33:42 Saving project...
2019-03-16 23:33:42 saved project in 0 sec
2019-03-16 23:33:42 Finished batch processing in 327.745 sec (exit code 1)
>>>




I have tried to roll back the nvidia drivers to versions 397.64 and even back to 385 where I had no issues, but then I get a different error message - I try to enable to CUDA graphics in preferences and it brings up the following entry:

the procedure entry point fatBinaryCTl_creathandle could not be located in the dynamic link library c:\\windows\system32\driverstore\filerepository\nvcvi.inf_amd64_3c5b42cdad4fff4f\nvopencl64.dll




any advice or work arounds to get it working would be appreciated. thanks

Pkroks

  • Newbie
  • *
  • Posts: 39
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #1 on: March 17, 2019, 02:10:02 AM »
I have tried to disable 'Use CPU when performing GPU accelerated processing' and i begin the depth maps processing and get the following:

2019-03-17 01:07:57 BuildModel: source data = Depth maps, surface type = Height field, quality = High, depth filtering = Mild, face count = High, smoothness = Low, vertex colors = 1
2019-03-17 01:07:57 Generating depth maps...
2019-03-17 01:07:57 Initializing...
2019-03-17 01:07:57 sorting point cloud... done in 0.083 sec
2019-03-17 01:07:58 processing matches... done in 0.06 sec
2019-03-17 01:07:58 initializing...
2019-03-17 01:07:58 selected 281 cameras from 281 in 0.742 sec
2019-03-17 01:07:58 groups: 18 19 19 18 19 19 19 18 19 19 19 18 19 19 19
2019-03-17 01:07:58 845 of 281 used (300.712%)
2019-03-17 01:07:58 scheduled 15 depth map groups
2019-03-17 01:07:58 saved depth map partition in 0.006 sec
2019-03-17 01:07:58 loaded depth map partition in 0 sec
2019-03-17 01:07:58 Initializing...
2019-03-17 01:07:58 Error: unknown error (30) at line 33
2019-03-17 01:07:58 Finished batch processing in 0.9 sec (exit code 1)
>>>

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #2 on: March 18, 2019, 05:16:20 PM »
Hello Pkroks,

Does the image matching process (first phase of the Align Photos operation) performs well using GPU?
Best regards,
Alexey Pasumansky,
Agisoft LLC

Pkroks

  • Newbie
  • *
  • Posts: 39
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #3 on: March 18, 2019, 07:46:47 PM »
Hello Pkroks,

Does the image matching process (first phase of the Align Photos operation) performs well using GPU?

Hi Alexey, Yes I believe so. Image Alignment on High completes without issue and is quick. Maybe 10 minutes (281 photos).


I have just formatted the laptop, and reinstalled windows 10 pro. Installed latest drivers from manufacturer and the latest nVidia graphics driver from nVidia.

As per Agisoft support I tried to use the latest nvidia driver, and then process depth maps and monitor the RAM usage. PC ram usage never went above 4gb (out of 32gb). And then dedicated GPU Ram usage never went above 2.3gb out of 8gb (it didn't even use any shared GPU memory)


I then attempted to use a tweak and activate OPEN CL instead of CUDA.

Began processing - and it gave me a new error - out of resources.

2019-03-18 16:49:23 using CPU implementation...
2019-03-18 16:49:23 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:23 using CPU implementation...
2019-03-18 16:49:23 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:23 using CPU implementation...
2019-03-18 16:49:23 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:23 using CPU implementation...
2019-03-18 16:49:23 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:23 using CPU implementation...
2019-03-18 16:49:23 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:23 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:24 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:24 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:25 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:25 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:26 GPU depth map warping failed: CL_OUT_OF_RESOURCES (-5) at line 355
2019-03-18 16:49:26 using CPU implementation...
2019-03-18 16:49:29
2019-03-18 16:49:29 Depth reconstruction devices performance:
2019-03-18 16:49:29  - 10%    done by CPU
2019-03-18 16:49:29  - 90%    done by GeForce GTX 1070
2019-03-18 16:49:29 Total time: 260.116 seconds
2019-03-18 16:49:29 CPU fallbacks: 43
2019-03-18 16:49:29
2019-03-18 16:49:29 19 depth maps generated in 262.908 sec
2019-03-18 16:49:29 saved depth maps block in 0.161 sec
2019-03-18 16:49:29 loaded depth map partition in 0 sec
2019-03-18 16:49:29 Initializing...
2019-03-18 16:49:29 Using device: GeForce GTX 1070, 16 compute units, 8192 MB global memory, OpenCL 1.2
2019-03-18 16:49:29   driver version: 419.35, platform version: OpenCL 1.2 CUDA 10.1.120
2019-03-18 16:49:29   max work group size 1024
2019-03-18 16:49:29   max work item sizes [1024, 1024, 64]
2019-03-18 16:49:29   max mem alloc size 2048 MB
2019-03-18 16:49:29   warp size 32
2019-03-18 16:49:29 Error: CL_OUT_OF_RESOURCES (-5) at line 211
2019-03-18 16:49:29 Saving project...
2019-03-18 16:49:29 saved project in 0 sec
2019-03-18 16:49:29 Finished batch processing in 506.592 sec (exit code 1)
>>>
« Last Edit: March 18, 2019, 07:50:56 PM by Pkroks »

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #4 on: March 18, 2019, 09:02:41 PM »
Hello Pkroks,

I see that you have issues both with OpenCL and CUDA implementation.

I'll see, if we have some ideas regarding this report.
Best regards,
Alexey Pasumansky,
Agisoft LLC

Pkroks

  • Newbie
  • *
  • Posts: 39
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #5 on: March 19, 2019, 12:25:33 AM »
Hello Pkroks,

I see that you have issues both with OpenCL and CUDA implementation.

I'll see, if we have some ideas regarding this report.

Please do. I have previously processed this entire data set with no issues on both AGISOFT and PIX4D. Now it seems something has gone wrong and my system is unstable as both PIX4D and AGISOFT are crashing now since this latest update.

I also had an epiphany that perhaps it was to do with voltage supply - I live in Africa and sometimes we have issues with low voltage; I put a voltage regulator on and plugged the laptop into that; seems it helped and it was a lot more stable. Got to 94% of the mesh and depth map creation before it gave me this error:

2019-03-18 20:15:36 [CPU 2]   Updating result...
2019-03-18 20:15:37 [CPU 2] Part done in 62.379 sec
2019-03-18 20:15:37 [CPU 2] 1 Parts 295-295/295...
2019-03-18 20:15:37 [CPU 2]   Initializing octree...
2019-03-18 20:15:37 [CPU 2]   Initializing histograms...
2019-03-18 20:15:41 [CPU 2]   Initializing...
2019-03-18 20:15:42 [CPU 2]   Iterating...
2019-03-18 20:15:52 [CPU 2]   Updating result...
2019-03-18 20:15:53 [CPU 2] Part done in 15.996 sec
2019-03-18 20:15:53 Finished processing in 4262.08 sec (exit code 0)
2019-03-18 20:15:53 Error: Kernel failed: unknown error (30) at line 106
2019-03-18 20:19:44 SaveProject: path = C:/PROJECTS/Embassy HIGH.psx
2019-03-18 20:19:44 Saving project...
2019-03-18 20:19:44 saved project in 0.09 sec
2019-03-18 20:19:44 Finished processing in 0.09 sec (exit code 1)
2019-03-18 20:19:54 Estimating resolution...
2019-03-18 20:19:54 Estimating point cloud resolution...
2019-03-18 20:19:54 Finished processing in 0.186 sec (exit code 1)


I then tried again on the usual HIGH settings and it got to 75% and gave me


2019-03-18 20:43:12 timings: rectify: 0.032 disparity: 0.79 borders: 0.046 filter: 0.145 fill: 0
2019-03-18 20:43:12 [GPU] estimating 2175x1720x480 disparity using 1088x896x8u tiles
2019-03-18 20:43:15 Error: Kernel failed: unknown error (30) at line 115
2019-03-18 20:43:15 GPU processing failed, switching to CPU mode
2019-03-18 20:43:15 [CPU] estimating 2175x1720x480 disparity using 1088x860x8u tiles
2019-03-18 20:43:18 timings: rectify: 0.204 disparity: 6.263 borders: 0.232 filter: 0.353 fill: 0
2019-03-18 20:43:21
2019-03-18 20:43:21 Depth reconstruction devices performance:
2019-03-18 20:43:21  - 5%    done by CPU
2019-03-18 20:43:21  - 95%    done by GeForce GTX 1070
2019-03-18 20:43:21 Total time: 133.407 seconds
2019-03-18 20:43:21
2019-03-18 20:43:21 Warning: cudaStreamDestroy failed: unknown error (30)
2019-03-18 20:43:21 Warning: cudaStreamDestroy failed: unknown error (30)
2019-03-18 20:43:21 Finished processing in 608.628 sec (exit code 0)
2019-03-18 20:43:21 Error: unknown error (30) at line 211




I previously succeeded in processing this data set using point cloud - but wanted to do it using the depth maps and see if what the result was. Now I can't even complete the point cloud processing again?



Could it be that my hardware is somehow faulty now? I have been trying to run UNIGINE HEAVEN to benchmark and test the GPU and CPU and see if it is stable; along with also trying MSI KOMBUSTOR. They both appear stable but the laptop is running hot?

Perhaps it is overheating? I'm really lost as to what would have changed between last week and this week and a new windows upgrade.

Pkroks

  • Newbie
  • *
  • Posts: 39
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #6 on: March 19, 2019, 01:37:29 PM »
Is there a setting or console input in agisoft to limit the resources it uses? Pix4d you can select to use only a portion of available resources.

Perhaps there is a similar command or setting in agisoft? I ask because pix4 has completed the cloud generation with only 80% if computer resources. So perhaps I can do the same with agisoft


Pkroks

  • Newbie
  • *
  • Posts: 39
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #7 on: March 20, 2019, 09:19:32 AM »
So here's an update what I have done.

Tried to install latest drivers. Failed cudastream
Install older driver. Failed
Installed older windows with older drivers. Failed
Installed older windows with new drivers. Failed
Installed old photoscan version. Failed
Tried opencl instead if cuda. Failed.

Then tried some test on my hardware.
Unigine heaven extreme. Passed.
Pass mark burn in. Passed
Pass mark burn in on 100% passed
Memtest86 passed

I took the back off the laptop and replaced thermal pads and thermal paste. Hardly any difference. Still failing on processing.

Tried to process using pix4d on normal.settings. Cuda failed.
Tried to process in pix4d but limited resources to 24gb of ram and 10 CPU threads. Passed

But I don't have a full pix4d license to be able to export any data.

Any ideas? Could it just be that it has been overheating and changing the thermal pads and paste has reduced it by maybe 2 degrees and so not overheating? Would limiting resources to less than max then reduce load and overall temps?

I have also just had a BSOD - kernel mode trap... The dump file indicates the cause of the problem to be the nVidia driver - which is the latest as per nVidias website.


andyroo

  • Sr. Member
  • ****
  • Posts: 438
    • View Profile
Re: Cuda stream destroy failed unknown error (win10 pro)
« Reply #8 on: June 19, 2019, 10:17:27 PM »
I FINALLY was able to get a machine running CUDA instead of openCL again by updating windows 10 to build 1903 and Metashape to 1.5.3. Just on my first alignment (5700 100mpix images) but it seems to be working fine so far (I'm 18 hours in with all GPUs working fine) prior to that I was getting "Stream destroy failed" and Error -999 messages for almost a year and that machine was running OpenCL.

Not sure it will fix your problem, but might be worth a try. Personally I'd try updating to Win 10 v1903 first (use the update assistant).