Hello Agisoft and the community,
We are experiencing the same issue on our side.
We have 6 Gb nVidia TitanX cards and 128 GB SDRAM in each of our PCs we use for computing. Both the Windows Task manager' memory monitoring section and GPUZ tool are reporting to us that this is not a (lack) of memory issue.
Closing Agisoft and reopening seems to be a workaround which works for us most of the time.
So far we used the driver nVidia, which is 358.50. We tried on one PC to upgrade to 378.66, we have the exact same issue.
I added below some logs, in case it will help.
In advance, thank you!
2017-02-21 09:40:27 BuildDenseCloud: quality = High, depth filtering = Moderate
2017-02-21 09:40:27 Initializing...
2017-02-21 09:40:27 Using device: GeForce GTX TITAN Black, 15 compute units, 6144 MB global memory, CUDA 3.5
2017-02-21 09:40:27 max work group size 1024
2017-02-21 09:40:27 max work item sizes [1024, 1024, 64]
2017-02-21 09:40:27 Using CUDA device 'GeForce GTX TITAN Black' in concurrent. (2 times)
2017-02-21 09:40:27 sorting point cloud... done in 0 sec
2017-02-21 09:40:27 processing matches... done in 0.016 sec
2017-02-21 09:40:27 initializing...
2017-02-21 09:40:27 selected 34 cameras from 34 in 0.234 sec
2017-02-21 09:40:27 Loading photos...
2017-02-21 09:40:34 loaded photos in 6.625 seconds
2017-02-21 09:40:34 Reconstructing depth...
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:37 [CPU] estimating 4438x3007x736 disparity using 1110x1003x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:56 timings: rectify: 2.032 disparity: 19.328 borders: 0.265 filter: 0.423 fill: 0
2017-02-21 09:40:58 [CPU] estimating 4365x2929x512 disparity using 1092x977x16u tiles
2017-02-21 09:41:16 timings: rectify: 0.562 disparity: 17.782 borders: 0.359 filter: 0.469 fill: 0
2017-02-21 09:41:18 [CPU] estimating 4298x2969x544 disparity using 1075x990x16u tiles
2017-02-21 09:41:35 timings: rectify: 0.578 disparity: 17.235 borders: 0.374 filter: 0.454 fill: 0
2017-02-21 09:41:38 [CPU] estimating 4720x3027x1120 disparity using 944x1009x16u tiles
2017-02-21 09:41:57 timings: rectify: 0.64 disparity: 20.157 borders: 0.25 filter: 0.407 fill: 0
2017-02-21 09:42:00 [CPU] estimating 2869x4988x384 disparity using 957x998x16u tiles
2017-02-21 09:42:20 timings: rectify: 0.688 disparity: 21.016 borders: 0.391 filter: 0.531 fill: 0
2017-02-21 09:42:25 finished depth reconstruction in 110.957 seconds
2017-02-21 09:42:25 Device 1: 100% work done with performance: 37.7101 million samples/sec (CPU), device used for 95.518 seconds
2017-02-21 09:42:25 Device 2: 0% work done with performance: 0 million samples/sec (GeForce GTX TITAN Black), device used for 0 seconds
2017-02-21 09:42:25 Total performance: 37.7101 million samples/sec
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Finished processing in 118.082 sec (exit code 0)
2017-02-21 09:42:25 Error: unspecified launch failure at line 67
-----------------
[2017-02-21 09:44] Richard Côté:
No Title
2017-02-21 09:40:27 BuildDenseCloud: quality = High, depth filtering = Moderate
2017-02-21 09:40:27 Initializing...
2017-02-21 09:40:27 Using device: GeForce GTX TITAN Black, 15 compute units, 6144 MB global memory, CUDA 3.5
2017-02-21 09:40:27 max work group size 1024
2017-02-21 09:40:27 max work item sizes [1024, 1024, 64]
2017-02-21 09:40:27 Using CUDA device 'GeForce GTX TITAN Black' in concurrent. (2 times)
2017-02-21 09:40:27 sorting point cloud... done in 0 sec
2017-02-21 09:40:27 processing matches... done in 0.016 sec
2017-02-21 09:40:27 initializing...
2017-02-21 09:40:27 selected 34 cameras from 34 in 0.234 sec
2017-02-21 09:40:27 Loading photos...
2017-02-21 09:40:34 loaded photos in 6.625 seconds
2017-02-21 09:40:34 Reconstructing depth...
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:37 [CPU] estimating 4438x3007x736 disparity using 1110x1003x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:56 timings: rectify: 2.032 disparity: 19.328 borders: 0.265 filter: 0.423 fill: 0
2017-02-21 09:40:58 [CPU] estimating 4365x2929x512 disparity using 1092x977x16u tiles
2017-02-21 09:41:16 timings: rectify: 0.562 disparity: 17.782 borders: 0.359 filter: 0.469 fill: 0
2017-02-21 09:41:18 [CPU] estimating 4298x2969x544 disparity using 1075x990x16u tiles
2017-02-21 09:41:35 timings: rectify: 0.578 disparity: 17.235 borders: 0.374 filter: 0.454 fill: 0
2017-02-21 09:41:38 [CPU] estimating 4720x3027x1120 disparity using 944x1009x16u tiles
2017-02-21 09:41:57 timings: rectify: 0.64 disparity: 20.157 borders: 0.25 filter: 0.407 fill: 0
2017-02-21 09:42:00 [CPU] estimating 2869x4988x384 disparity using 957x998x16u tiles
2017-02-21 09:42:20 timings: rectify: 0.688 disparity: 21.016 borders: 0.391 filter: 0.531 fill: 0
2017-02-21 09:42:25 finished depth reconstruction in 110.957 seconds
2017-02-21 09:42:25 Device 1: 100% work done with performance: 37.7101 million samples/sec (CPU), device used for 95.518 seconds
2017-02-21 09:42:25 Device 2: 0% work done with performance: 0 million samples/sec (GeForce GTX TITAN Black), device used for 0 seconds
2017-02-21 09:42:25 Total performance: 37.7101 million samples/sec
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Finished processing in 118.082 sec (exit code 0)
2017-02-21 09:42:25 Error: unspecified launch failure at line 67
------------------
2017-02-21 09:45:00 Hierarchical SGM failed: 2017-02-21 09:45:00 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:45:00 Hierarchical SGM failed: 2017-02-21 09:45:00 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67