Agisoft Metashape

Agisoft Metashape => Bug Reports => Topic started by: FlyRight-3D on February 18, 2017, 07:34:48 PM

Title: launch error at line 67
Post by: FlyRight-3D on February 18, 2017, 07:34:48 PM
I have been getting the failure "launch error at line 67" when performing a dense cloud generation.  it always happens during the restructuring depth phase.   I fiiiiiiiiinally got it to finish processing by selecting the lowest setting possible.  I have the necessary computing power to do 15 of these at one time, so I don't understand why I am having this trouble.   I am running dual xeon e5 2670s, 128GB RAM, and a nvidia geforce titan black.   any ideas?
Title: Re: launch error at line 67
Post by: Alexey Pasumansky on February 18, 2017, 09:09:18 PM
Hello FlyRight-3D,

Are you using latest GPU drivers, also is there only NVIDIA card checked on in the GPU tab of the Preferenes window?
Title: Re: launch error at line 67
Post by: muzykanci on February 18, 2017, 10:42:50 PM
Hi,
I had similar problem when performing a dense cloud generation:

"2017-02-16 19:04:12 CPU fallbacks: 1
2017-02-16 19:04:12 Warning: unspecified launch failure
2017-02-16 19:04:12 Warning: invalid resource handle
2017-02-16 19:04:12 Warning: unspecified launch failure
2017-02-16 19:04:12 Warning: invalid resource handle
2017-02-16 19:04:12 Warning: unspecified launch failure
2017-02-16 19:04:12 Warning: invalid resource handle
2017-02-16 19:04:12 Warning: unspecified launch failure
2017-02-16 19:04:12 Saving project...
2017-02-16 19:04:12 Warning: invalid resource handle
2017-02-16 19:04:12 Error: unspecified launch failure at line 67"

When I turned the NVIDIA card off in the GPU tab it worked fine using CPU only (much slower though)
Title: Re: launch error at line 67
Post by: FlyRight-3D on February 19, 2017, 10:34:59 AM
I had the nvidia card selected in the preferences; when I deselected it, I am able to do the processing.  All drivers are up to date.  There were a couple of times I had this same error during photo alignment when I was using GPU acceleration but other times I didn't.  Same with the point cloud, sometimes it would go ahead and process on a lower quality setting but it works on all settings with GPU acceleration off.  I assume it's the size of the dataset that is messing with me.  It's not particularly big and would be great if someone from agisoft could find a solution without me purchasing yet another graphics card.
Title: Re: launch error at line 67
Post by: Poutinemaster on February 21, 2017, 07:01:19 PM
Hello Agisoft and the community,

We are experiencing the same issue on our side.

We have 6 Gb nVidia TitanX cards and 128 GB SDRAM in each of our PCs we use for computing.  Both the Windows Task manager' memory monitoring section and GPUZ tool are reporting to us that this is not a (lack) of memory issue.

Closing Agisoft and reopening seems to be a workaround which works for us most of the time.

So far we used the driver nVidia, which is 358.50.  We tried on one PC to upgrade to 378.66, we have the exact same issue.

I added below some logs, in case it will help.

In advance, thank you!

2017-02-21 09:40:27 BuildDenseCloud: quality = High, depth filtering = Moderate
2017-02-21 09:40:27 Initializing...
2017-02-21 09:40:27 Using device: GeForce GTX TITAN Black, 15 compute units, 6144 MB global memory, CUDA 3.5
2017-02-21 09:40:27   max work group size 1024
2017-02-21 09:40:27   max work item sizes [1024, 1024, 64]
2017-02-21 09:40:27 Using CUDA device 'GeForce GTX TITAN Black' in concurrent. (2 times)
2017-02-21 09:40:27 sorting point cloud... done in 0 sec
2017-02-21 09:40:27 processing matches... done in 0.016 sec
2017-02-21 09:40:27 initializing...
2017-02-21 09:40:27 selected 34 cameras from 34 in 0.234 sec
2017-02-21 09:40:27 Loading photos...
2017-02-21 09:40:34 loaded photos in 6.625 seconds
2017-02-21 09:40:34 Reconstructing depth...
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:37 [CPU] estimating 4438x3007x736 disparity using 1110x1003x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:56 timings: rectify: 2.032 disparity: 19.328 borders: 0.265 filter: 0.423 fill: 0
2017-02-21 09:40:58 [CPU] estimating 4365x2929x512 disparity using 1092x977x16u tiles
2017-02-21 09:41:16 timings: rectify: 0.562 disparity: 17.782 borders: 0.359 filter: 0.469 fill: 0
2017-02-21 09:41:18 [CPU] estimating 4298x2969x544 disparity using 1075x990x16u tiles
2017-02-21 09:41:35 timings: rectify: 0.578 disparity: 17.235 borders: 0.374 filter: 0.454 fill: 0
2017-02-21 09:41:38 [CPU] estimating 4720x3027x1120 disparity using 944x1009x16u tiles
2017-02-21 09:41:57 timings: rectify: 0.64 disparity: 20.157 borders: 0.25 filter: 0.407 fill: 0
2017-02-21 09:42:00 [CPU] estimating 2869x4988x384 disparity using 957x998x16u tiles
2017-02-21 09:42:20 timings: rectify: 0.688 disparity: 21.016 borders: 0.391 filter: 0.531 fill: 0
2017-02-21 09:42:25 finished depth reconstruction in 110.957 seconds
2017-02-21 09:42:25 Device 1: 100% work done with performance: 37.7101 million samples/sec (CPU), device used for 95.518 seconds
2017-02-21 09:42:25 Device 2: 0% work done with performance: 0 million samples/sec (GeForce GTX TITAN Black), device used for 0 seconds
2017-02-21 09:42:25 Total performance: 37.7101 million samples/sec
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Finished processing in 118.082 sec (exit code 0)
2017-02-21 09:42:25 Error: unspecified launch failure at line 67

-----------------

[‎2017-‎02-‎21 09:44] Richard Côté:
No Title
2017-02-21 09:40:27 BuildDenseCloud: quality = High, depth filtering = Moderate
2017-02-21 09:40:27 Initializing...
2017-02-21 09:40:27 Using device: GeForce GTX TITAN Black, 15 compute units, 6144 MB global memory, CUDA 3.5
2017-02-21 09:40:27   max work group size 1024
2017-02-21 09:40:27   max work item sizes [1024, 1024, 64]
2017-02-21 09:40:27 Using CUDA device 'GeForce GTX TITAN Black' in concurrent. (2 times)
2017-02-21 09:40:27 sorting point cloud... done in 0 sec
2017-02-21 09:40:27 processing matches... done in 0.016 sec
2017-02-21 09:40:27 initializing...
2017-02-21 09:40:27 selected 34 cameras from 34 in 0.234 sec
2017-02-21 09:40:27 Loading photos...
2017-02-21 09:40:34 loaded photos in 6.625 seconds
2017-02-21 09:40:34 Reconstructing depth...
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:36 Hierarchical SGM failed: 2017-02-21 09:40:36 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:40:37 [CPU] estimating 4438x3007x736 disparity using 1110x1003x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:39 [CPU] estimating 4553x4737x320 disparity using 1139x948x16u tiles
2017-02-21 09:40:56 timings: rectify: 2.032 disparity: 19.328 borders: 0.265 filter: 0.423 fill: 0
2017-02-21 09:40:58 [CPU] estimating 4365x2929x512 disparity using 1092x977x16u tiles
2017-02-21 09:41:16 timings: rectify: 0.562 disparity: 17.782 borders: 0.359 filter: 0.469 fill: 0
2017-02-21 09:41:18 [CPU] estimating 4298x2969x544 disparity using 1075x990x16u tiles
2017-02-21 09:41:35 timings: rectify: 0.578 disparity: 17.235 borders: 0.374 filter: 0.454 fill: 0
2017-02-21 09:41:38 [CPU] estimating 4720x3027x1120 disparity using 944x1009x16u tiles
2017-02-21 09:41:57 timings: rectify: 0.64 disparity: 20.157 borders: 0.25 filter: 0.407 fill: 0
2017-02-21 09:42:00 [CPU] estimating 2869x4988x384 disparity using 957x998x16u tiles
2017-02-21 09:42:20 timings: rectify: 0.688 disparity: 21.016 borders: 0.391 filter: 0.531 fill: 0
2017-02-21 09:42:25 finished depth reconstruction in 110.957 seconds
2017-02-21 09:42:25 Device 1: 100% work done with performance: 37.7101 million samples/sec (CPU), device used for 95.518 seconds
2017-02-21 09:42:25 Device 2: 0% work done with performance: 0 million samples/sec (GeForce GTX TITAN Black), device used for 0 seconds
2017-02-21 09:42:25 Total performance: 37.7101 million samples/sec
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Warning: invalid resource handle
2017-02-21 09:42:25 Finished processing in 118.082 sec (exit code 0)
2017-02-21 09:42:25 Error: unspecified launch failure at line 67

------------------

2017-02-21 09:45:00 Hierarchical SGM failed: 2017-02-21 09:45:00 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
2017-02-21 09:45:00 Hierarchical SGM failed: 2017-02-21 09:45:00 GPU processing failed, switching to CPU mode
unspecified launch failure at line 67
Title: Re: launch error at line 67
Post by: FlyRight-3D on February 24, 2017, 09:54:01 AM
ah, thanks for the workaround.  I suppose that's why it intermittently worked for me.  I rolled back the drivers a dozen versions or so and it didn't seem to help.

If need be, I will also supply log information to help with this issue.
Title: Re: launch error at line 67
Post by: bortoman on February 28, 2017, 04:14:45 PM
Hi, i fear this issue is not taken serious enough!

By doing some tests on 5 different computers (all with different Nvidia cards) i can reproduce the issue every time i want...  :-\

It seems to me, that you just have to go high enough with the settings, or increase the dataset for the problem to occur!
(or low enough for it not to happen - depends on your point of view)

I just did some tests with this dataset: http://www.agisoft.com/datasets/building.zip

When generating the dense cloud with ultra high settings and moderate filtering it consistently fails:

Code: [Select]
2017-02-28 12:19:42 Agisoft PhotoScan Professional Version: 1.3.0 build 3772 (64 bit)
2017-02-28 12:19:42 Platform: Windows
2017-02-28 12:19:43 OpenGL Vendor: NVIDIA Corporation
2017-02-28 12:19:43 OpenGL Renderer: GeForce GTX 580/PCIe/SSE2
2017-02-28 12:19:43 OpenGL Version: 4.5.0 NVIDIA 376.53
2017-02-28 12:19:43 Maximum Texture Size: 16384
2017-02-28 12:19:43 Quad Buffered Stereo: not enabled
2017-02-28 12:19:43 ARB_vertex_buffer_object: supported
2017-02-28 12:19:43 ARB_texture_non_power_of_two: supported
2017-02-28 12:19:43 LoadProject
2017-02-28 12:19:43 Loading project...
2017-02-28 12:19:45 loaded project in 2.124 sec
2017-02-28 12:19:45 Finished processing in 2.124 sec (exit code 1)
2017-02-28 12:20:16 Checking for missing images...
2017-02-28 12:20:16 checking for missing images... done in 0.016 sec
2017-02-28 12:20:16 Finished processing in 0.016 sec (exit code 1)
2017-02-28 12:20:16 BuildDenseCloud: quality = Ultra high, depth filtering = Moderate
2017-02-28 12:20:16 Initializing...
2017-02-28 12:20:16 Using device: GeForce GTX 580, 16 compute units, 1536 MB global memory, CUDA 2.0
2017-02-28 12:20:16   max work group size 1024
2017-02-28 12:20:16   max work item sizes [1024, 1024, 64]
2017-02-28 12:20:16 sorting point cloud... done in 0 sec
2017-02-28 12:20:16 processing matches... done in 0 sec
2017-02-28 12:20:16 initializing...
2017-02-28 12:20:16 selected 50 cameras from 50 in 0.125 sec
2017-02-28 12:20:16 Loading photos...
2017-02-28 12:20:20 loaded photos in 4.312 seconds
2017-02-28 12:20:20 Reconstructing depth...
2017-02-28 12:20:22 [GPU] estimating 3498x2857x1728 disparity using 1166x1429x8u tiles
2017-02-28 12:20:25 [CPU] estimating 4002x2665x1120 disparity using 1001x889x8u tiles
2017-02-28 12:20:32 timings: rectify: 1.891 disparity: 8.733 borders: 0.579 filter: 0.655 fill: 0
2017-02-28 12:20:35 [GPU] estimating 2795x2800x224 disparity using 466x400x8u tiles
2017-02-28 12:20:43 timings: rectify: 1.157 disparity: 6.5 borders: 0.546 filter: 0.548 fill: 0
2017-02-28 12:20:45 [GPU] estimating 3561x2717x1024 disparity using 1187x906x8u tiles
2017-02-28 12:20:55 timings: rectify: 1.234 disparity: 8.126 borders: 0.562 filter: 0.672 fill: 0
2017-02-28 12:21:01 timings: rectify: 1.953 disparity: 36.733 borders: 0.702 filter: 0.766 fill: 0
2017-02-28 12:21:01 [GPU] estimating 3961x2493x416 disparity using 793x831x8u tiles
2017-02-28 12:21:05 [CPU] estimating 3498x2857x1728 disparity using 1166x953x8u tiles
2017-02-28 12:21:10 timings: rectify: 1.437 disparity: 8.006 borders: 0.781 filter: 0.812 fill: 0
2017-02-28 12:21:14 [GPU] estimating 4002x2665x1120 disparity using 1001x889x8u tiles
2017-02-28 12:21:24 timings: rectify: 1.797 disparity: 9.003 borders: 0.515 filter: 0.579 fill: 0
2017-02-28 12:21:32 [GPU] estimating 2412x12895x512 disparity using 804x1173x8u tiles
2017-02-28 12:21:41 timings: rectify: 0.969 disparity: 38.007 borders: 0.344 filter: 0.447 fill: 0
2017-02-28 12:21:47 [CPU] estimating 4207x1867x512 disparity using 1052x934x8u tiles
2017-02-28 12:21:58 timings: rectify: 5.922 disparity: 20.824 borders: 2.357 filter: 3.076 fill: 0
2017-02-28 12:22:04 [GPU] estimating 3980x1867x448 disparity using 796x934x8u tiles
2017-02-28 12:22:12 timings: rectify: 1.193 disparity: 6.464 borders: 0.743 filter: 0.767 fill: 0
2017-02-28 12:22:15 [GPU] estimating 4116x1809x800 disparity using 824x905x8u tiles
2017-02-28 12:22:16 timings: rectify: 0.881 disparity: 29.64 borders: 0.489 filter: 0.58 fill: 0
2017-02-28 12:22:20 [CPU] estimating 4143x2414x640 disparity using 1036x1207x8u tiles
2017-02-28 12:22:23 timings: rectify: 1.038 disparity: 6.278 borders: 0.709 filter: 0.667 fill: 0
2017-02-28 12:22:25 [GPU] estimating 4040x1907x832 disparity using 808x954x8u tiles
2017-02-28 12:22:34 timings: rectify: 1.186 disparity: 7.203 borders: 0.643 filter: 0.533 fill: 0
2017-02-28 12:22:37 [GPU] estimating 4091x2108x960 disparity using 1023x703x8u tiles
2017-02-28 12:22:45 timings: rectify: 1.037 disparity: 7.22 borders: 0.624 filter: 0.765 fill: 0
2017-02-28 12:22:48 [GPU] estimating 4028x1981x736 disparity using 806x991x8u tiles
2017-02-28 12:22:55 timings: rectify: 1.049 disparity: 35.688 borders: 0.664 filter: 0.706 fill: 0
2017-02-28 12:22:56 timings: rectify: 1.219 disparity: 6.545 borders: 0.825 filter: 0.873 fill: 0
2017-02-28 12:22:59 [GPU] estimating 3948x1898x288 disparity using 790x949x8u tiles
2017-02-28 12:23:01 [CPU] estimating 3962x2493x416 disparity using 991x1247x8u tiles
2017-02-28 12:23:06 timings: rectify: 0.921 disparity: 5.268 borders: 0.828 filter: 0.754 fill: 0
2017-02-28 12:23:15 [GPU] estimating 4116x1809x800 disparity using 824x905x8u tiles
2017-02-28 12:23:23 timings: rectify: 1.707 disparity: 6.782 borders: 0.671 filter: 0.678 fill: 0
2017-02-28 12:23:26 [GPU] estimating 4090x1982x480 disparity using 818x991x8u tiles
2017-02-28 12:23:34 timings: rectify: 1.409 disparity: 6.47 borders: 0.81 filter: 0.794 fill: 0
2017-02-28 12:23:37 timings: rectify: 2.085 disparity: 36.377 borders: 0.946 filter: 0.942 fill: 0
2017-02-28 12:23:37 [GPU] estimating 4143x2414x640 disparity using 1036x805x8u tiles
2017-02-28 12:23:40 [CPU] estimating 3502x2089x192 disparity using 1168x1045x8u tiles
2017-02-28 12:23:49 timings: rectify: 1.538 disparity: 10.187 borders: 0.973 filter: 1.083 fill: 0
2017-02-28 12:23:52 [GPU] estimating 4201x1868x864 disparity using 841x934x8u tiles
2017-02-28 12:24:01 timings: rectify: 1.237 disparity: 7.529 borders: 0.5 filter: 0.596 fill: 0
2017-02-28 12:24:04 [GPU] estimating 4124x1900x800 disparity using 825x950x8u tiles
2017-02-28 12:24:08 timings: rectify: 0.842 disparity: 28.139 borders: 0.53 filter: 0.564 fill: 0
2017-02-28 12:24:13 timings: rectify: 1.431 disparity: 7.653 borders: 0.696 filter: 0.611 fill: 0
2017-02-28 12:24:15 [GPU] estimating 3581x1597x384 disparity using 1194x799x8u tiles
2017-02-28 12:24:16 [CPU] estimating 2795x2800x224 disparity using 932x934x8u tiles
2017-02-28 12:24:22 timings: rectify: 0.728 disparity: 5.365 borders: 0.58 filter: 0.569 fill: 0
2017-02-28 12:24:30 [GPU] estimating 3730x2433x1216 disparity using 933x811x8u tiles
2017-02-28 12:24:40 timings: rectify: 1.169 disparity: 8.583 borders: 0.5 filter: 0.593 fill: 0
2017-02-28 12:24:43 [GPU] estimating 4028x2848x800 disparity using 806x950x8u tiles
2017-02-28 12:24:46 timings: rectify: 1.415 disparity: 30.671 borders: 0.498 filter: 0.649 fill: 0
2017-02-28 12:24:51 [CPU] estimating 3816x3128x576 disparity using 954x1043x8u tiles
2017-02-28 12:24:57 timings: rectify: 1.963 disparity: 11.251 borders: 0.792 filter: 1.165 fill: 0
2017-02-28 12:25:00 [GPU] estimating 4038x2330x640 disparity using 1010x777x8u tiles
2017-02-28 12:25:11 timings: rectify: 1.58 disparity: 8.472 borders: 0.699 filter: 0.992 fill: 0
2017-02-28 12:25:14 [GPU] estimating 3883x2433x768 disparity using 971x811x8u tiles
2017-02-28 12:25:24 timings: rectify: 1.379 disparity: 8.451 borders: 0.679 filter: 0.779 fill: 0
2017-02-28 12:25:33 [GPU] estimating 3561x2717x1024 disparity using 891x906x8u tiles
2017-02-28 12:25:38 timings: rectify: 1.52 disparity: 48.304 borders: 0.687 filter: 0.833 fill: 0
2017-02-28 12:25:43 [CPU] estimating 3730x2433x1216 disparity using 933x1217x8u tiles
2017-02-28 12:25:43 timings: rectify: 1.659 disparity: 9.446 borders: 0.548 filter: 0.612 fill: 0
2017-02-28 12:25:47 [GPU] estimating 3816x3128x576 disparity using 954x782x8u tiles
2017-02-28 12:25:59 timings: rectify: 1.868 disparity: 10.832 borders: 0.711 filter: 0.834 fill: 0
2017-02-28 12:26:02 [GPU] estimating 4028x2848x800 disparity using 1007x950x8u tiles
2017-02-28 12:26:14 timings: rectify: 1.383 disparity: 10.184 borders: 0.832 filter: 1.024 fill: 0
2017-02-28 12:26:18 [GPU] estimating 4084x2745x1088 disparity using 817x687x8u tiles
2017-02-28 12:26:18 timings: rectify: 1.129 disparity: 36.451 borders: 0.469 filter: 0.632 fill: 0
2017-02-28 12:26:26 [CPU] estimating 4083x2745x1088 disparity using 1021x915x8u tiles
2017-02-28 12:26:32 timings: rectify: 1.767 disparity: 12.628 borders: 0.494 filter: 0.689 fill: 0
2017-02-28 12:26:40 [GPU] estimating 3884x2433x768 disparity using 971x811x8u tiles
2017-02-28 12:26:51 timings: rectify: 1.733 disparity: 8.598 borders: 0.602 filter: 0.874 fill: 0
2017-02-28 12:26:54 [GPU] estimating 3885x2857x448 disparity using 972x953x8u tiles
2017-02-28 12:27:05 timings: rectify: 1.681 disparity: 9.282 borders: 0.97 filter: 1.075 fill: 0
2017-02-28 12:27:08 timings: rectify: 1.608 disparity: 42.767 borders: 0.494 filter: 1.025 fill: 0
2017-02-28 12:27:09 [GPU] estimating 4041x2742x1184 disparity using 809x914x8u tiles
2017-02-28 12:27:12 [CPU] estimating 4038x2330x640 disparity using 1010x1165x8u tiles
2017-02-28 12:27:21 timings: rectify: 2.009 disparity: 10.629 borders: 0.734 filter: 0.863 fill: 0
2017-02-28 12:27:24 [GPU] estimating 3938x2525x1984 disparity using 985x842x8u tiles
2017-02-28 12:27:35 timings: rectify: 1.736 disparity: 9.481 borders: 0.454 filter: 0.596 fill: 0
2017-02-28 12:27:42 [GPU] estimating 3980x1867x448 disparity using 796x934x8u tiles
2017-02-28 12:27:45 timings: rectify: 1.03 disparity: 34.155 borders: 0.55 filter: 0.595 fill: 0
2017-02-28 12:27:49 timings: rectify: 1.151 disparity: 5.721 borders: 0.582 filter: 0.647 fill: 0
2017-02-28 12:27:51 [GPU] estimating 4090x1982x480 disparity using 818x991x8u tiles
2017-02-28 12:27:52 [CPU] estimating 3884x2857x448 disparity using 971x953x8u tiles
2017-02-28 12:28:01 timings: rectify: 0.883 disparity: 8.134 borders: 0.807 filter: 0.799 fill: 0
2017-02-28 12:28:04 [GPU] estimating 4207x1867x512 disparity using 842x623x8u tiles
2017-02-28 12:28:09 Hierarchical SGM failed: 2017-02-28 12:28:09 GPU processing failed, switching to CPU mode
unspecified launch failure at line 208
2017-02-28 12:28:10 [CPU] estimating 4207x1867x512 disparity using 1052x934x8u tiles
2017-02-28 12:28:28 timings: rectify: 1.242 disparity: 38.327 borders: 0.637 filter: 0.788 fill: 0
2017-02-28 12:28:31 [CPU] estimating 3461x2536x768 disparity using 1154x1268x8u tiles
2017-02-28 12:28:58 timings: rectify: 0.899 disparity: 27.291 borders: 0.48 filter: 0.545 fill: 0
2017-02-28 12:29:03 finished depth reconstruction in 522.192 seconds
2017-02-28 12:29:03 Device 1: 19% work done with performance: 16.3823 million samples/sec (CPU), device used for 462.55 seconds
2017-02-28 12:29:03 Device 2: 80% work done with performance: 68.4154 million samples/sec (GeForce GTX 580), device used for 275.848 seconds
2017-02-28 12:29:03 Total performance: 84.7977 million samples/sec
2017-02-28 12:29:03 Warning: unspecified launch failure
2017-02-28 12:29:03 Warning: invalid resource handle
2017-02-28 12:29:03 Finished processing in 527.177 sec (exit code 0)
2017-02-28 12:29:03 Error: unspecified launch failure at line 67

System specs for this test:
Intel I7 920
GTX 580
12GB RAM

I know this is a low-spec-pc, but i'm out of time testing on our production machines...
I assure you even our dual-xeon, nvidia quadro workhorse has the same issue!

Needless to say it works fine once i reinstalled the previous version 1.2.6

I strongly suggest a rollback or at least a switch in the settings to use OpenCL again instead of CUDA!
Title: Re: launch error at line 67
Post by: Chernabog on March 01, 2017, 02:12:51 PM
Hi All,

Just to say that I too am experiencing this same issue trying to stich about 200 36 MP images using the Photoscan version 1.3.0.

Our system spec is as below:
18 virtual processor Xeon E5-2660 v3 (2.6 Ghz)
112 GB RAM
Nvidia GRID K2 (8 cores @ 745 Mhz, 4096 MB vRAM)

Works fine using CPU only, just takes an age to complete.

Regards,

S.
Title: Re: launch error at line 67
Post by: bortoman on March 01, 2017, 08:40:09 PM
Hi,
just to let you know about the progress:

Agisoft is apparently addressing the issue...
Alexey has sent me (and probably some other users) a temporary build 3884 which seems to have multiple different issues...
(I will not discuss them here, as a temporary build is not an official release, i discuss this via private message with Alexey)

All i can say, is that now i don't get an error message, but instead my computer crashes and reboots...  :o

Can someone please share his experience?
Title: Re: launch error at line 67
Post by: Hans Keesom on March 04, 2017, 12:12:34 PM
Isn't this the same as http://www.agisoft.com/forum/index.php?topic=6565.0

I had to go back to previous version because of this. It's a pity as 1.3.0. seems to use the GPU much more and 1080 cards are dropping in price the last few days.
Title: Re: launch error at line 67
Post by: Hans Keesom on March 06, 2017, 12:45:51 AM
I think I have it solved by doing a rollback of the GPU driver as suggested by someone here. See http://www.nvidia.com/object/driver_rollback.html for how to

Now running with GPU average of 58% version 1.3.0 instead of 41% with version 1.2.6. !
Title: Re: launch error at line 67
Post by: Alexey Pasumansky on March 06, 2017, 01:11:55 AM
Hello Hans,

Which driver version you have rolled back to?
Title: Re: launch error at line 67
Post by: FlyRight-3D on March 06, 2017, 07:05:51 PM
I didn't have luck rolling back the drivers but I only tried a few. 

So here is where I am now with these large datasets 200-500 photographs at 12-20MP; I have had luck throttling back the clock speed of my GPU.  I was getting performance caps based on available power and voltage reliability on my Titan black.  After clocked it back a bit, I didn't have nearly as much trouble as before but the problem still perisisted.  I added a second, very small card out of another computer (gtx 745) to run my displays.  This also worked with returning the clock speed on my Titan black to stock settings. I am currently having success with running both GPUs during photo alignment and dense cloud generation.

From these experiences it seems that if your GPU throttles back during a heavy workload, these power fluctuations are causing some issue at "line 67".  I suggest slowing your GPU clock speed and memory clock a bit and see if this helps you.  Also try adding a monitor display card to use your larger GPU (array) to lower the demand on a single graphics card.

I hope this helps and I'll continue to update you all as I find "fixes"
Title: Re: launch error at line 67
Post by: FlyRight-3D on March 06, 2017, 07:46:47 PM
Also, I liquid cooled everything and dropped the temps from about 80C to 50C.  I'm not sure if heat throttling had anything to do with this or not but I figured it's worth mentioning.
Title: Re: launch error at line 67
Post by: Hans Keesom on March 07, 2017, 01:15:31 AM
Hello Hans,

Which driver version you have rolled back to?

Sorry, the error did show up again in align photos. I was too happy too soon :-(
GPU crashes on Allign photos still

I now run/test allign photos with GPU processing off and the rest with it on, see how that goes
Title: Re: launch error at line 67
Post by: Hans Keesom on March 09, 2017, 11:15:50 PM
Also, I liquid cooled everything and dropped the temps from about 80C to 50C.  I'm not sure if heat throttling had anything to do with this or not but I figured it's worth mentioning.

I think you found the workaround. I moved the GPU and Memory offset back to 0 and no error ever since.

Title: ps 1_3 pro "unknown error at line 67" when GPU cards are enabled. Ubuntu 16.10
Post by: bigbirdtommy on March 16, 2017, 03:52:45 PM
When I run "AlignPhotos" on my dev machine I get the "line 67" error when my GPU cards (either of them) are activated.

When I deactivate them it runs as expected (at CPU speeds)

Same with either card (GTX 1050 and GTX 1080TI) on ubuntu 16.10,   both cards work in 1.2.6 as well...

(Log only has the 1080TI enabled)
LOG: 2017-03-16 07:42:16 AlignPhotos: accuracy = Highest, preselection = generic, keypoint limit = 40000, tiepoint limit = 4000, constrain features by mask = 0, adaptive fitting = 0
2017-03-16 07:42:16 Detecting points...
2017-03-16 07:42:19 photo 0: 39939 points
2017-03-16 07:42:23 photo 1: 39783 points
2017-03-16 07:42:25 photo 2: 39799 points
2017-03-16 07:42:27 photo 3: 39785 points
2017-03-16 07:42:30 photo 4: 39672 points
2017-03-16 07:42:30 Finished processing in 14.6119 sec (exit code 0)
2017-03-16 07:42:30 Error: Aborted by user
2017-03-16 07:42:50 AlignPhotos: accuracy = Highest, preselection = generic, keypoint limit = 40000, tiepoint limit = 4000, constrain features by mask = 0, adaptive fitting = 0
2017-03-16 07:42:50 Detecting points...
2017-03-16 07:42:50 Using device: Graphics Device, 28 compute units, 11171 MB global memory, CUDA 6.1
2017-03-16 07:42:50   max work group size 1024
2017-03-16 07:42:50   max work item sizes [1024, 1024, 64]
2017-03-16 07:42:50 Finished processing in 0.508237 sec (exit code 0)
2017-03-16 07:42:50 Error: unknown error at line 67
Title: Re: ps 1_3 pro "unknown error at line 67" when GPU cards are enabled. Ubuntu 16.10
Post by: bigbirdtommy on March 16, 2017, 04:06:06 PM
Slightly more information when running BuildDenseCloud:

Log: 2017-03-16 08:04:20 BuildDenseCloud: quality = Ultra high, depth filtering = Aggressive
2017-03-16 08:04:20 Initializing...
2017-03-16 08:04:20 Using device: GeForce GTX 1050, 5 compute units, 1999 MB global memory, CUDA 6.1
2017-03-16 08:04:20   max work group size 1024
2017-03-16 08:04:20   max work item sizes [1024, 1024, 64]
2017-03-16 08:04:20 Using device: Graphics Device, 28 compute units, 11171 MB global memory, CUDA 6.1
2017-03-16 08:04:20   max work group size 1024
2017-03-16 08:04:20   max work item sizes [1024, 1024, 64]
2017-03-16 08:04:20 Using CUDA device 'Graphics Device' in concurrent. (4 times)
2017-03-16 08:04:20 sorting point cloud... done in 0.000146 sec
2017-03-16 08:04:20 processing matches... done in 0.003077 sec
2017-03-16 08:04:20 initializing...
2017-03-16 08:04:20 selected 14 cameras from 14 in 0.272369 sec
2017-03-16 08:04:20 Loading photos...
2017-03-16 08:04:20 loaded photos in 0.339007 seconds
2017-03-16 08:04:20 Reconstructing depth...
2017-03-16 08:04:25 Hierarchical SGM failed: unknown error at line 672017-03-16 08:04:25 GPU processing failed, switching to CPU mode


2017-03-16 08:04:25 Hierarchical SGM failed: unknown error at line 672017-03-16 08:04:25 GPU processing failed, switching to CPU mode


2017-03-16 08:04:25 Hierarchical SGM failed: unknown error at line 672017-03-16 08:04:25 GPU processing failed, switching to CPU mode


2017-03-16 08:04:25 Hierarchical SGM failed: unknown error at line 672017-03-16 08:04:25 GPU processing failed, switching to CPU mode


2017-03-16 08:04:26 Hierarchical SGM failed: unknown error at line 672017-03-16 08:04:26 GPU processing failed, switching to CPU mode


2017-03-16 08:04:26 [CPU] estimating 1920x3840x160 disparity using 960x960x8u tiles
2017-03-16 08:04:27 [CPU] estimating 1920x3840x352 disparity using 960x960x8u tiles
2017-03-16 08:04:27 [CPU] estimating 1920x3840x288 disparity using 960x960x8u tiles
2017-03-16 08:04:27 [CPU] estimating 1920x3840x128 disparity using 960x960x8u tiles
2017-03-16 08:04:27 [CPU] estimating 1920x3840x288 disparity using 960x960x8u tiles
2017-03-16 08:04:27 [CPU] estimating 1920x3840x128 disparity using 960x960x8u tiles
2017-03-16 08:04:40 Finished processing in 19.9073 sec (exit code 0)
2017-03-16 08:04:40 Error: Aborted by user
Title: Re: launch error at line 67
Post by: Alexey Pasumansky on March 16, 2017, 04:12:02 PM
Hello bigbirdtommy,

I've merged your topic with the related thread. The issue should be solved in the next update. If you wish, I can send you a beta-version for testing purposes.
Title: Re: launch error at line 67
Post by: bigbirdtommy on March 16, 2017, 04:25:10 PM
Thank you Alexey!  :)

I just noticed it was a dup. Sorry about that!

I'd love to kick the tires on a dev build for testing if possible  bigbirdtommy (at) gmail dot com
Title: Re: launch error at line 67
Post by: bigbirdtommy on March 16, 2017, 05:16:20 PM
Hmm...   Alexey : I installed the beta version on my machine:

And now it shows "error at line 86"... So it at least made it 9 lines farther... ;)  J/K...

Let me know if you'd like me to test something else...

2017-03-16 09:11:52 Agisoft PhotoScan Professional Version: 1.3.0 build 3943 (64 bit)
2017-03-16 09:11:52 Platform: Linux
2017-03-16 09:11:52 OpenGL Vendor: NVIDIA Corporation
2017-03-16 09:11:52 OpenGL Renderer: Graphics Device/PCIe/SSE2
2017-03-16 09:11:52 OpenGL Version: 4.5.0 NVIDIA 378.13
2017-03-16 09:11:52 Maximum Texture Size: 32768
2017-03-16 09:11:52 Quad Buffered Stereo: not enabled
2017-03-16 09:11:52 ARB_vertex_buffer_object: supported
2017-03-16 09:11:52 ARB_texture_non_power_of_two: supported
2017-03-16 09:12:13 LoadProject
2017-03-16 09:12:13 Loading project...
2017-03-16 09:12:14 loaded project in 0.686608 sec
2017-03-16 09:12:14 Finished processing in 0.686708 sec (exit code 1)
2017-03-16 09:12:24 AlignPhotos: accuracy = Highest, preselection = generic, keypoint limit = 40000, tiepoint limit = 4000, constrain features by mask = 0, adaptive fitting = 1
2017-03-16 09:12:24 Detecting points...
2017-03-16 09:12:24 Using device: GeForce GTX 1050, 5 compute units, 1999 MB global memory, Device Compute Capability 6.1
2017-03-16 09:12:24   max work group size 1024
2017-03-16 09:12:24   max work item sizes [1024, 1024, 64]
2017-03-16 09:12:24 Using device: Graphics Device, 28 compute units, 11171 MB global memory, Device Compute Capability 6.1
2017-03-16 09:12:24   max work group size 1024
2017-03-16 09:12:24   max work item sizes [1024, 1024, 64]
2017-03-16 09:12:27 photo 0: 40000 points
2017-03-16 09:12:27 Finished processing in 3.31603 sec (exit code 0)
2017-03-16 09:12:27 Error: unknown error at line 86
Title: Re: launch error at line 67
Post by: Alexey Pasumansky on March 16, 2017, 05:43:42 PM
Hello bigbirdtommy,

According to the provided log your second graphic card (GTX 1080Ti) was not properly recognized and is marked as "Graphics Device". Is it posible that the latest NVIDIA drivers were not properly installed? Version 1.3 of PhotoScan is using CUDA for NVIDIA cards, whereas in the older versions OpenCL has been utilized.

I can suggest to run the tests (image matching and dense cloud generation) on each GPU individually, turning off another one in the Preferences and also disabling the CPU.
Title: Re: launch error at line 67
Post by: bortoman on March 18, 2017, 12:04:52 AM
Same issue on my test-machine with the temporary build 3921 !
I will try to make another test with temporary build 3943 but i'm running out of patience...

I can't afford to loose any more time on this, i have set up an old unused machine to run this tests, but i feel like this is taking way too long to fix... why didn't you extensively test this before official release?
The "line67" bug is known for a month now!  :o

Sorry guys, as much as i admire your work so far, you really messed up this time!

Code: [Select]
2017-03-10 10:11:24 Agisoft PhotoScan Professional Version: 1.3.0 build 3921 (64 bit)
2017-03-10 10:11:24 Platform: Windows
2017-03-10 10:11:25 OpenGL Vendor: NVIDIA Corporation
2017-03-10 10:11:25 OpenGL Renderer: GeForce GTX 580/PCIe/SSE2
2017-03-10 10:11:25 OpenGL Version: 4.5.0 NVIDIA 376.53
2017-03-10 10:11:25 Maximum Texture Size: 16384
2017-03-10 10:11:25 Quad Buffered Stereo: not enabled
2017-03-10 10:11:25 ARB_vertex_buffer_object: supported
2017-03-10 10:11:25 ARB_texture_non_power_of_two: supported
2017-03-10 10:11:29 LoadProject
2017-03-10 10:11:29 Loading project...
2017-03-10 10:11:37 loaded project in 7.422 sec
2017-03-10 10:11:37 Finished processing in 7.422 sec (exit code 1)
2017-03-10 10:11:43 BuildDenseCloud: quality = Ultra high, depth filtering = Moderate
2017-03-10 10:11:44 Initializing...
2017-03-10 10:11:44 Using device: GeForce GTX 580, 16 compute units, 1536 MB global memory, Device Compute Capability 2.0
2017-03-10 10:11:44   max work group size 1024
2017-03-10 10:11:44   max work item sizes [1024, 1024, 64]
2017-03-10 10:11:44 sorting point cloud... done in 0 sec
2017-03-10 10:11:44 processing matches... done in 0 sec
2017-03-10 10:11:44 initializing...
2017-03-10 10:11:44 selected 50 cameras from 50 in 0.125 sec
2017-03-10 10:11:44 Loading photos...
2017-03-10 10:11:48 loaded photos in 4.109 seconds
2017-03-10 10:11:48 Reconstructing depth...
2017-03-10 10:11:49 [GPU] estimating 4207x1867x512 disparity using 1403x934x8u tiles
2017-03-10 10:11:52 [CPU] estimating 4002x2665x1120 disparity using 1001x889x8u tiles
2017-03-10 10:11:55 timings: rectify: 0.32 disparity: 5.671 borders: 0.281 filter: 0.375 fill: 0
2017-03-10 10:11:55 [GPU] estimating 4143x2414x640 disparity using 592x483x8u tiles
2017-03-10 10:12:05 timings: rectify: 0.172 disparity: 8.952 borders: 0.265 filter: 0.485 fill: 0
2017-03-10 10:12:06 [GPU] estimating 3962x2493x416 disparity using 991x831x8u tiles
2017-03-10 10:12:14 timings: rectify: 0.203 disparity: 7.719 borders: 0.359 filter: 0.5 fill: 0
2017-03-10 10:12:14 [GPU] estimating 3502x2089x192 disparity using 876x697x8u tiles
2017-03-10 10:12:20 timings: rectify: 0.093 disparity: 5.282 borders: 0.282 filter: 0.343 fill: 0
2017-03-10 10:12:24 timings: rectify: 1.101 disparity: 33.641 borders: 0.468 filter: 0.563 fill: 0
2017-03-10 10:12:24 [GPU] estimating 3961x2493x416 disparity using 991x831x8u tiles
2017-03-10 10:12:28 [CPU] estimating 3498x2857x1728 disparity using 1166x953x8u tiles
2017-03-10 10:12:33 timings: rectify: 0.219 disparity: 7.469 borders: 0.328 filter: 0.485 fill: 0
2017-03-10 10:12:33 [GPU] estimating 4002x2665x1120 disparity using 801x667x8u tiles
2017-03-10 10:12:44 timings: rectify: 0.171 disparity: 9.969 borders: 0.282 filter: 0.406 fill: 0
2017-03-10 10:12:44 [GPU] estimating 2412x12895x512 disparity using 804x1173x8u tiles
2017-03-10 10:12:58 timings: rectify: 0.937 disparity: 30.828 borders: 0.343 filter: 0.423 fill: 0
2017-03-10 10:13:03 [CPU] estimating 3498x2857x1728 disparity using 1166x953x8u tiles
2017-03-10 10:13:07 timings: rectify: 0.375 disparity: 20.062 borders: 0.687 filter: 1.47 fill: 0
2017-03-10 10:13:11 [GPU] estimating 4116x1809x800 disparity using 824x603x8u tiles
2017-03-10 10:13:18 timings: rectify: 0.188 disparity: 6.546 borders: 0.218 filter: 0.297 fill: 0
2017-03-10 10:13:19 [GPU] estimating 4090x1982x480 disparity using 818x991x8u tiles
2017-03-10 10:13:25 timings: rectify: 0.141 disparity: 6.093 borders: 0.313 filter: 0.374 fill: 0
2017-03-10 10:13:26 [GPU] estimating 4143x2414x640 disparity using 691x805x8u tiles
2017-03-10 10:13:35 timings: rectify: 0.937 disparity: 32.157 borders: 0.329 filter: 0.468 fill: 0
2017-03-10 10:13:37 timings: rectify: 0.172 disparity: 10.579 borders: 0.249 filter: 0.391 fill: 0
2017-03-10 10:13:37 [GPU] estimating 4201x1868x864 disparity using 841x934x8u tiles
2017-03-10 10:13:38 [CPU] estimating 2795x2800x224 disparity using 932x934x8u tiles
2017-03-10 10:13:45 timings: rectify: 0.125 disparity: 7.203 borders: 0.296 filter: 0.329 fill: 0
2017-03-10 10:13:45 [GPU] estimating 4124x1900x800 disparity using 825x634x8u tiles
2017-03-10 10:13:53 timings: rectify: 0.14 disparity: 6.813 borders: 0.296 filter: 0.407 fill: 0
2017-03-10 10:13:53 [GPU] estimating 3581x1597x384 disparity using 1194x799x8u tiles
2017-03-10 10:13:59 timings: rectify: 0.141 disparity: 5.25 borders: 0.296 filter: 0.298 fill: 0
2017-03-10 10:14:02 timings: rectify: 0.782 disparity: 24.234 borders: 0.578 filter: 0.703 fill: 0
2017-03-10 10:14:04 [GPU] estimating 3980x1867x448 disparity using 796x623x8u tiles
2017-03-10 10:14:06 [CPU] estimating 3561x2717x1024 disparity using 1187x906x8u tiles
2017-03-10 10:14:10 timings: rectify: 0.172 disparity: 5.561 borders: 0.282 filter: 0.375 fill: 0
2017-03-10 10:14:11 [GPU] estimating 4090x1982x480 disparity using 1023x991x8u tiles
2017-03-10 10:14:19 timings: rectify: 0.187 disparity: 7.781 borders: 0.265 filter: 0.329 fill: 0
2017-03-10 10:14:19 [GPU] estimating 4207x1867x512 disparity using 842x623x8u tiles
2017-03-10 10:14:28 timings: rectify: 0.141 disparity: 8.188 borders: 0.313 filter: 0.405 fill: 0
2017-03-10 10:14:29 [GPU] estimating 4119x1934x864 disparity using 1030x967x8u tiles
2017-03-10 10:14:35 timings: rectify: 1 disparity: 30.75 borders: 0.407 filter: 0.499 fill: 0
2017-03-10 10:14:36 timings: rectify: 0.172 disparity: 6.843 borders: 0.281 filter: 0.375 fill: 0
2017-03-10 10:14:36 [GPU] estimating 4004x1838x736 disparity using 801x613x8u tiles
2017-03-10 10:14:41 [CPU] estimating 3980x1867x448 disparity using 995x934x8u tiles
2017-03-10 10:14:43 timings: rectify: 0.156 disparity: 6.36 borders: 0.234 filter: 0.391 fill: 0
2017-03-10 10:14:44 [GPU] estimating 4011x1947x640 disparity using 1003x974x8u tiles
2017-03-10 10:14:51 timings: rectify: 0.172 disparity: 6.111 borders: 0.297 filter: 0.391 fill: 0
2017-03-10 10:14:55 [GPU] estimating 2795x2800x224 disparity using 932x700x8u tiles
2017-03-10 10:15:02 timings: rectify: 0.172 disparity: 6.14 borders: 0.281 filter: 0.391 fill: 0
2017-03-10 10:15:03 [GPU] estimating 3816x3128x576 disparity using 954x1043x8u tiles
2017-03-10 10:15:05 timings: rectify: 0.766 disparity: 23.718 borders: 0.562 filter: 0.548 fill: 0
2017-03-10 10:15:08 [CPU] estimating 4116x1809x800 disparity using 1029x905x8u tiles
2017-03-10 10:15:13 timings: rectify: 0.234 disparity: 9.517 borders: 0.343 filter: 0.657 fill: 0
2017-03-10 10:15:13 [GPU] estimating 3730x2433x1216 disparity using 746x811x8u tiles
2017-03-10 10:15:24 timings: rectify: 0.156 disparity: 9.469 borders: 0.328 filter: 0.453 fill: 0
2017-03-10 10:15:28 [GPU] estimating 3730x2433x1216 disparity using 933x811x8u tiles
2017-03-10 10:15:31 timings: rectify: 0.782 disparity: 23.687 borders: 0.454 filter: 0.5 fill: 0
2017-03-10 10:15:34 [CPU] estimating 4040x1907x832 disparity using 1010x954x8u tiles
2017-03-10 10:15:37 timings: rectify: 0.219 disparity: 8.313 borders: 0.298 filter: 0.437 fill: 0
2017-03-10 10:15:37 [GPU] estimating 4028x2848x800 disparity using 806x950x8u tiles
2017-03-10 10:15:48 timings: rectify: 0.156 disparity: 10.516 borders: 0.328 filter: 0.469 fill: 0
2017-03-10 10:15:49 [GPU] estimating 4038x2330x640 disparity using 1010x777x8u tiles
2017-03-10 10:15:58 timings: rectify: 0.141 disparity: 7.954 borders: 0.406 filter: 0.516 fill: 0
2017-03-10 10:15:58 [GPU] estimating 3883x2433x768 disparity using 971x811x8u tiles
2017-03-10 10:15:59 timings: rectify: 0.797 disparity: 24.828 borders: 0.422 filter: 0.532 fill: 0
2017-03-10 10:16:02 [CPU] estimating 4091x2108x960 disparity using 1023x1054x8u tiles
2017-03-10 10:16:07 timings: rectify: 0.172 disparity: 8.516 borders: 0.345 filter: 0.483 fill: 0
2017-03-10 10:16:12 [GPU] estimating 4083x2745x1088 disparity using 817x915x8u tiles
2017-03-10 10:16:25 timings: rectify: 0.203 disparity: 11.625 borders: 0.328 filter: 0.547 fill: 0
2017-03-10 10:16:25 [GPU] estimating 4038x2330x640 disparity using 1010x777x8u tiles
2017-03-10 10:16:29 timings: rectify: 0.875 disparity: 26.907 borders: 0.485 filter: 0.546 fill: 0
2017-03-10 10:16:32 [CPU] estimating 4028x1981x736 disparity using 1007x991x8u tiles
2017-03-10 10:16:34 timings: rectify: 0.156 disparity: 8.093 borders: 0.359 filter: 0.516 fill: 0
2017-03-10 10:16:34 [GPU] estimating 3884x2857x448 disparity using 777x953x8u tiles
2017-03-10 10:16:44 timings: rectify: 0.172 disparity: 9.094 borders: 0.281 filter: 0.453 fill: 0
2017-03-10 10:16:45 [GPU] estimating 3461x2536x768 disparity using 866x846x8u tiles
2017-03-10 10:16:53 timings: rectify: 0.094 disparity: 7.937 borders: 0.36 filter: 0.484 fill: 0
2017-03-10 10:16:57 timings: rectify: 0.797 disparity: 25.11 borders: 0.719 filter: 0.578 fill: 0
2017-03-10 10:16:58 [GPU] estimating 3561x2717x1024 disparity using 891x906x8u tiles
2017-03-10 10:17:00 [CPU] estimating 3948x1898x288 disparity using 987x949x8u tiles
2017-03-10 10:17:08 timings: rectify: 0.187 disparity: 8.954 borders: 0.266 filter: 0.453 fill: 0
2017-03-10 10:17:08 [GPU] estimating 3816x3128x576 disparity using 954x782x8u tiles
2017-03-10 10:17:20 timings: rectify: 0.188 disparity: 10.641 borders: 0.359 filter: 0.625 fill: 0
2017-03-10 10:17:20 [GPU] estimating 4028x2848x800 disparity using 806x950x8u tiles
2017-03-10 10:17:22 timings: rectify: 0.797 disparity: 22.344 borders: 0.61 filter: 0.593 fill: 0
2017-03-10 10:17:30 [CPU] estimating 3884x2433x768 disparity using 971x1217x8u tiles
2017-03-10 10:17:32 timings: rectify: 0.125 disparity: 10.396 borders: 0.36 filter: 0.609 fill: 0
2017-03-10 10:17:32 [GPU] estimating 4084x2745x1088 disparity using 817x915x8u tiles
2017-03-10 10:17:45 timings: rectify: 0.125 disparity: 11.703 borders: 0.249 filter: 0.454 fill: 0
2017-03-10 10:17:49 [GPU] estimating 3462x2536x768 disparity using 866x846x8u tiles
2017-03-10 10:17:58 timings: rectify: 0.266 disparity: 8.062 borders: 0.312 filter: 0.469 fill: 0
2017-03-10 10:17:59 [GPU] estimating 4042x2742x1184 disparity using 809x914x8u tiles
2017-03-10 10:18:01 timings: rectify: 0.953 disparity: 31.116 borders: 0.438 filter: 0.562 fill: 0
2017-03-10 10:18:05 [CPU] estimating 3885x2857x448 disparity using 972x953x8u tiles
2017-03-10 10:18:10 timings: rectify: 0.203 disparity: 10.61 borders: 0.359 filter: 0.563 fill: 0
2017-03-10 10:18:11 [GPU] estimating 4017x2456x1056 disparity using 1005x819x8u tiles
2017-03-10 10:18:21 timings: rectify: 0.172 disparity: 9.376 borders: 0.249 filter: 0.407 fill: 0
2017-03-10 10:18:21 [GPU] estimating 4119x2427x1856 disparity using 1030x809x8u tiles
2017-03-10 10:18:33 timings: rectify: 0.094 disparity: 10.796 borders: 0.343 filter: 0.485 fill: 0
2017-03-10 10:18:33 [GPU] estimating 4012x2370x2048 disparity using 1003x790x8u tiles
2017-03-10 10:18:39 timings: rectify: 1.125 disparity: 34.797 borders: 0.673 filter: 0.765 fill: 0
2017-03-10 10:18:43 [CPU] estimating 4041x2742x1184 disparity using 1011x914x8u tiles
2017-03-10 10:18:45 timings: rectify: 0.156 disparity: 11.281 borders: 0.328 filter: 0.422 fill: 0
2017-03-10 10:18:50 [GPU] estimating 3938x2525x1984 disparity using 985x842x8u tiles
2017-03-10 10:19:01 timings: rectify: 0.235 disparity: 9.954 borders: 0.249 filter: 0.376 fill: 0
2017-03-10 10:19:01 [GPU] estimating 4017x2456x1056 disparity using 1005x819x8u tiles
2017-03-10 10:19:11 timings: rectify: 0.125 disparity: 9.641 borders: 0.343 filter: 0.469 fill: 0
2017-03-10 10:19:12 [GPU] estimating 4018x2658x704 disparity using 804x886x8u tiles
2017-03-10 10:19:19 timings: rectify: 1.14 disparity: 36.141 borders: 0.547 filter: 0.688 fill: 0
2017-03-10 10:19:22 timings: rectify: 0.203 disparity: 9.953 borders: 0.281 filter: 0.437 fill: 0
2017-03-10 10:19:23 [CPU] estimating 3938x2525x1984 disparity using 985x1263x8u tiles
2017-03-10 10:19:23 [GPU] estimating 4174x2620x1376 disparity using 835x874x8u tiles
2017-03-10 10:19:34 timings: rectify: 0.266 disparity: 10.421 borders: 0.343 filter: 0.532 fill: 0
2017-03-10 10:19:34 [GPU] estimating 3966x2622x1984 disparity using 794x874x8u tiles
2017-03-10 10:19:46 timings: rectify: 0.11 disparity: 10.328 borders: 0.343 filter: 0.532 fill: 0
2017-03-10 10:19:46 [GPU] estimating 3787x2499x2048 disparity using 947x833x8u tiles
2017-03-10 10:19:52 timings: rectify: 0.937 disparity: 30.578 borders: 0.313 filter: 0.437 fill: 0
2017-03-10 10:19:55 timings: rectify: 0.172 disparity: 8.25 borders: 0.235 filter: 0.343 fill: 0
2017-03-10 10:19:55 [GPU] estimating 1751x16213x320 disparity using 876x1248x8u tiles
2017-03-10 10:19:59 [CPU] estimating 4013x2370x2048 disparity using 1004x1185x8u tiles
2017-03-10 10:20:17 timings: rectify: 0.312 disparity: 19.298 borders: 0.625 filter: 1.281 fill: 0
2017-03-10 10:20:17 [GPU] estimating 1597x13064x256 disparity using 799x1188x8u tiles
2017-03-10 10:20:28 timings: rectify: 1.094 disparity: 29.578 borders: 0.281 filter: 0.344 fill: 0
2017-03-10 10:20:32 [CPU] estimating 4174x2620x1376 disparity using 1044x874x8u tiles
2017-03-10 10:20:34 timings: rectify: 0.25 disparity: 15.064 borders: 0.547 filter: 1 fill: 0
2017-03-10 10:20:40 [GPU] estimating 3966x2622x1984 disparity using 992x874x8u tiles
2017-03-10 10:20:51 timings: rectify: 0.219 disparity: 9.548 borders: 0.249 filter: 0.407 fill: 0
2017-03-10 10:20:51 [GPU] estimating 3953x2538x1664 disparity using 791x846x8u tiles
2017-03-10 10:21:02 timings: rectify: 0.094 disparity: 9.781 borders: 0.328 filter: 0.5 fill: 0
2017-03-10 10:21:02 [GPU] estimating 4069x2468x896 disparity using 1018x823x8u tiles
2017-03-10 10:21:06 timings: rectify: 1.093 disparity: 34.579 borders: 0.641 filter: 0.609 fill: 0
2017-03-10 10:21:09 [CPU] estimating 3939x2495x768 disparity using 985x1248x8u tiles
2017-03-10 10:21:12 timings: rectify: 0.172 disparity: 8.843 borders: 0.36 filter: 0.484 fill: 0
2017-03-10 10:21:12 [GPU] estimating 4069x2181x704 disparity using 1018x727x8u tiles
2017-03-10 10:21:21 timings: rectify: 0.141 disparity: 7.719 borders: 0.312 filter: 0.469 fill: 0
2017-03-10 10:21:21 [GPU] estimating 4317x2009x1568 disparity using 864x670x8u tiles
2017-03-10 10:21:30 timings: rectify: 0.14 disparity: 8.063 borders: 0.312 filter: 0.469 fill: 0
2017-03-10 10:21:30 [GPU] estimating 4352x2021x2048 disparity using 871x1011x8u tiles
2017-03-10 10:21:39 timings: rectify: 0.938 disparity: 30.249 borders: 0.624 filter: 0.751 fill: 0
2017-03-10 10:21:40 timings: rectify: 0.172 disparity: 9.126 borders: 0.296 filter: 0.438 fill: 0
2017-03-10 10:21:40 [GPU] estimating 4075x2025x2048 disparity using 815x675x8u tiles
2017-03-10 10:21:43 [CPU] estimating 4069x2468x896 disparity using 1018x1234x8u tiles
2017-03-10 10:21:51 timings: rectify: 0.156 disparity: 9.579 borders: 0.312 filter: 0.438 fill: 0
2017-03-10 10:21:51 [GPU] estimating 2126x15004x352 disparity using 1063x1155x8u tiles
2017-03-10 10:22:15 timings: rectify: 1.031 disparity: 32.86 borders: 0.625 filter: 0.703 fill: 0
2017-03-10 10:22:17 timings: rectify: 0.437 disparity: 23.017 borders: 0.656 filter: 1.422 fill: 0
2017-03-10 10:22:17 [GPU] estimating 2171x12943x640 disparity using 724x1079x8u tiles
2017-03-10 10:22:19 [CPU] estimating 4008x2338x1344 disparity using 1002x1169x8u tiles
2017-03-10 10:22:21 Error: 2017-03-10 10:22:21 GPU processing failed, switching to CPU mode
Kernel failed: unspecified launch failure at line 114
2017-03-10 10:22:29 [CPU] estimating 2171x12943x640 disparity using 1086x996x8u tiles
2017-03-10 10:22:47 timings: rectify: 0.985 disparity: 28.359 borders: 0.578 filter: 0.594 fill: 0
2017-03-10 10:22:51 [CPU] estimating 3966x2169x672 disparity using 992x1085x8u tiles
2017-03-10 10:23:16 timings: rectify: 0.844 disparity: 26.25 borders: 0.375 filter: 0.484 fill: 0
2017-03-10 10:23:19 [CPU] estimating 4106x2164x768 disparity using 1027x1082x8u tiles
2017-03-10 10:23:46 timings: rectify: 0.891 disparity: 27.406 borders: 0.329 filter: 0.421 fill: 0
2017-03-10 10:23:56 [CPU] estimating 2630x11638x544 disparity using 877x1058x8u tiles
2017-03-10 10:25:06 timings: rectify: 3.469 disparity: 73.188 borders: 1.188 filter: 1.765 fill: 0
2017-03-10 10:25:13 [CPU] estimating 1650x11478x384 disparity using 825x1044x8u tiles
2017-03-10 10:26:03 timings: rectify: 2.313 disparity: 51.297 borders: 0.891 filter: 1.203 fill: 0
2017-03-10 10:26:12 [CPU] estimating 2007x13245x256 disparity using 1004x1019x8u tiles
2017-03-10 10:27:18 timings: rectify: 3.109 disparity: 67.891 borders: 1.234 filter: 1.672 fill: 0
2017-03-10 10:27:24 finished depth reconstruction in 935.22 seconds
2017-03-10 10:27:24 Device 1: 21% work done with performance: 19.258 million samples/sec (CPU), device used for 832.493 seconds
2017-03-10 10:27:24 Device 2: 78% work done with performance: 69.7752 million samples/sec (GeForce GTX 580), device used for 516.03 seconds
2017-03-10 10:27:24 Total performance: 89.0332 million samples/sec
2017-03-10 10:27:24 Warning: unspecified launch failure
2017-03-10 10:27:24 Warning: invalid resource handle
2017-03-10 10:27:24 Warning: unspecified launch failure
2017-03-10 10:27:24 Warning: unspecified launch failure
2017-03-10 10:27:24 Finished processing in 940.595 sec (exit code 0)
2017-03-10 10:27:24 Error: unspecified launch failure at line 86
Title: Re: launch error at line 67
Post by: bortoman on March 18, 2017, 09:54:05 AM
G O O D      N E W S !

Temporary Build 3943 finally seems to resolve the "line 67" and line "line 86" issues on my test machine!
Also at first look the additional noise issue of build 3884 seems to be gone...
And last but not least i'm finally able to see a performance increase thanks to CUDA of aprox. 30% to OPENCL (v1.2.6)...

I hope I don't find any more surprises!
Title: Re: launch error at line 67
Post by: Heinrich on March 21, 2017, 01:02:52 AM
Alexey,
I now - after runnin many models without issue on Build 3772 - also face this issue. May I test Build 3943?


Running an NVIDIA Tesla M2070-Q on a XEON X5670 (12 cores).
Title: Re: launch error at line 67
Post by: Chernabog on March 21, 2017, 05:23:15 PM
I concur with bortoman, build 3943 seems to have fixed the "line 67" error message I was getting on dense point cloud generation.

I have more work to process so if anything else weird turns up I will report it here.

Regards,

S.
Title: Re: launch error at line 67
Post by: F_Lopo on April 02, 2017, 12:35:18 AM
Hi.
I have stumbled onto the same problem, consistently.
I am using build 3772.
Could I please get build 3943 too?
Thank you in advance,
Regards,
Frederico
Title: Re: launch error at line 67
Post by: Hans Keesom on April 08, 2017, 11:52:49 AM
Testing 1.3.1 at the moment. Runs my nvdia titan GPU at 100%.. 89 hours to go, no line 67 error so far.