Hello solosails,
Can you please provide the processing log from the node where the "out of memory" message occurred?
OK, so it completed using CPU only (however, I used lower settings, so maybe this info isnt helpfull - but here is the log for the failed GPU/CPU run....
019-01-02 15:33:03 BuildDenseCloud: quality = High, depth filtering = Mild, reuse depth maps, point colors = 1
2019-01-02 15:33:03 Generating dense point cloud...
2019-01-02 15:33:03 Generating dense point cloud...
2019-01-02 15:33:03 initializing...
2019-01-02 15:33:09 selected 186 cameras in 6.218 sec
2019-01-02 15:33:09 working volume: 2543x2402x1729
2019-01-02 15:33:09 tiles: 1x1x1
2019-01-02 15:33:09 saved dense cloud data in 0.004 sec
2019-01-02 15:33:09 saved regions in 0.008 sec
2019-01-02 15:33:09 saved camera partition in 0.007 sec
2019-01-02 15:33:09 scheduled 10 depth filtering groups
2019-01-02 15:33:09 scheduled 1 dense cloud regions
2019-01-02 15:33:09 loaded camera partition in 0 sec
2019-01-02 15:33:09 loaded dense cloud data in 0.001 sec
2019-01-02 15:33:10 preloading data... done in 8.702 sec
2019-01-02 15:33:19 filtering depth maps... Finished processing in 1224.39 sec (exit code 0)
2019-01-02 15:53:27 Error: Aborted by user
2019-01-02 15:54:10 Analyzing dense cloud...
2019-01-02 15:54:11 Finished processing in 1.236 sec (exit code 1)
2019-01-02 15:54:34 BuildModel: source data = Depth maps, surface type = Arbitrary, quality = High, depth filtering = Mild, face count = Medium, smoothness = Low, vertex colors = 1
2019-01-02 15:54:35 Generating mesh...
2019-01-02 15:54:35 [GPU 1] Using device: GeForce GTX 680, 8 compute units, free memory: 1627/2048 MB, compute capability 3.0
2019-01-02 15:54:35 driver version: 417.35, driver/runtime CUDA: 10000/5050
2019-01-02 15:54:35 max work group size 1024
2019-01-02 15:54:35 max work item sizes [1024, 1024, 64]
2019-01-02 15:54:35 Preparing depth maps...
2019-01-02 15:54:35 Using 238 cameras
2019-01-02 15:57:43 Depth maps: 186
2019-01-02 15:57:43 done in 188.507 sec
2019-01-02 15:57:43 Subdividing space...
2019-01-02 15:59:04 Nodes: 255546177
2019-01-02 15:59:04 done in 80.374 sec
2019-01-02 15:59:04 Balancing space...
2019-01-02 16:03:19 Nodes: 296923625
2019-01-02 16:03:19 done in 255.537 sec
2019-01-02 16:03:19 Estimating density...
2019-01-02 16:04:43 done in 83.573 sec
2019-01-02 16:04:43 Estimating surface...
2019-01-02 16:04:45 Level #5/13
2019-01-02 16:04:45 Subdividing...
2019-01-02 16:04:45 done in 0.514 sec
2019-01-02 16:04:46 [GPU 1] 1 Parts 1-1/1...
2019-01-02 16:04:46 [GPU 1] Initializing octree...
2019-01-02 16:04:46 [GPU 1] Initializing histograms...
2019-01-02 16:05:17 [GPU 1] Initializing to most probable...
2019-01-02 16:05:17 [GPU 1] Iterating...
2019-01-02 16:05:17 [GPU 1] Updating result...
2019-01-02 16:05:19 [GPU 1] Part done in 33.591 sec
2019-01-02 16:05:19 Devices performance:
2019-01-02 16:05:19 - 100% done by [GPU 1] with timings: 33 s = 0% fetch + 2% init + 91% hist (0% GPU) + 0% init + 0% iters (65% + 34%) + 6% updating
2019-01-02 16:05:19 IO utilization: 5%
2019-01-02 16:05:19 Level done in 34.304 sec
2019-01-02 16:05:19 Level #7/13
2019-01-02 16:05:19 Subdividing...
2019-01-02 16:05:20 done in 0.479 sec
2019-01-02 16:05:20 [GPU 1] 1 Parts 1-1/1...
2019-01-02 16:05:20 [GPU 1] Initializing octree...
2019-01-02 16:05:21 [GPU 1] Initializing histograms...
2019-01-02 16:05:49 [GPU 1] Initializing...
2019-01-02 16:05:51 [GPU 1] Iterating...
2019-01-02 16:05:54 [GPU 1] Updating result...
2019-01-02 16:05:56 [GPU 1] Part done in 35.832 sec
2019-01-02 16:05:56 Devices performance:
2019-01-02 16:05:56 - 100% done by [GPU 1] with timings: 35 s = 0% fetch + 2% init + 81% hist (5% GPU) + 3% init + 8% iters (82% + 18%) + 6% updating
2019-01-02 16:05:56 IO utilization: 5%
2019-01-02 16:05:56 Level done in 36.501 sec
2019-01-02 16:05:56 Level #9/13
2019-01-02 16:05:56 Subdividing...
2019-01-02 16:05:56 done in 0.631 sec
2019-01-02 16:05:56 [GPU 1] 6 Parts 1-6/8...
2019-01-02 16:05:56 [GPU 1] Initializing octree...
2019-01-02 16:06:00 [GPU 1] Initializing histograms...
2019-01-02 16:06:36 [GPU 1] Initializing...
2019-01-02 16:06:38 [GPU 1] Iterating...
2019-01-02 16:07:34 [GPU 1] Updating result...
2019-01-02 16:07:36 [GPU 1] Part done in 99.912 sec
2019-01-02 16:07:36 [GPU 1] 2 Parts 7-8/8...
2019-01-02 16:07:36 [GPU 1] Initializing octree...
2019-01-02 16:07:37 [GPU 1] Initializing histograms...
2019-01-02 16:08:05 [GPU 1] Initializing...
2019-01-02 16:08:06 [GPU 1] Iterating...
2019-01-02 16:08:10 [GPU 1] Updating result...
2019-01-02 16:08:10 [GPU 1] Part done in 33.731 sec
2019-01-02 16:08:10 Devices performance:
2019-01-02 16:08:10 - 100% done by [GPU 1] with timings: 133 s = 0% fetch + 3% init + 47% hist (49% GPU) + 2% init + 44% iters (83% + 17%) + 3% updating
2019-01-02 16:08:10 IO utilization: 3%
2019-01-02 16:08:10 Level done in 134.487 sec
2019-01-02 16:08:10 Level #11/13
2019-01-02 16:08:10 Subdividing...
2019-01-02 16:08:12 done in 1.57 sec
2019-01-02 16:08:12 [GPU 1] 5 Parts 1-5/148...
2019-01-02 16:08:12 [GPU 1] Initializing octree...
2019-01-02 16:08:17 [GPU 1] Initializing histograms...
2019-01-02 16:12:00 [GPU 1] Initializing...
2019-01-02 16:12:03 [GPU 1] Iterating...
2019-01-02 16:19:49 [GPU 1] Updating result...
2019-01-02 16:19:50 [GPU 1] Part done in 697.904 sec
2019-01-02 16:19:50 [GPU 1] 9 Parts 6-14/148...
2019-01-02 16:19:50 [GPU 1] Initializing octree...
2019-01-02 16:19:55 [GPU 1] Initializing histograms...
2019-01-02 16:23:42 [GPU 1] Initializing...
2019-01-02 16:23:45 [GPU 1] Iterating...
2019-01-02 16:31:21 [GPU 1] Updating result...
2019-01-02 16:31:23 [GPU 1] Part done in 692.748 sec
2019-01-02 16:31:23 [GPU 1] 5 Parts 15-19/148...
2019-01-02 16:31:23 [GPU 1] Initializing octree...
2019-01-02 16:31:28 [GPU 1] Initializing histograms...
2019-01-02 16:33:57 [GPU 1] Initializing...
2019-01-02 16:33:59 [GPU 1] Iterating...
2019-01-02 16:36:45 [GPU 1] Updating result...
2019-01-02 16:36:47 [GPU 1] Part done in 324.016 sec
2019-01-02 16:36:47 [GPU 1] 5 Parts 20-24/148...
2019-01-02 16:36:47 [GPU 1] Initializing octree...
2019-01-02 16:36:53 [GPU 1] Initializing histograms...
2019-01-02 16:39:04 [GPU 1] Initializing...
2019-01-02 16:39:06 [GPU 1] Iterating...
2019-01-02 16:43:33 [GPU 1] Updating result...
2019-01-02 16:43:37 [GPU 1] Part done in 410.484 sec
2019-01-02 16:43:37 [GPU 1] 7 Parts 25-31/148...
2019-01-02 16:43:37 [GPU 1] Initializing octree...
2019-01-02 16:43:45 [GPU 1] Initializing histograms...
2019-01-02 16:45:29 [GPU 1] Initializing...
2019-01-02 16:45:31 [GPU 1] Iterating...
2019-01-02 16:46:37 [GPU 1] Updating result...
2019-01-02 16:46:38 [GPU 1] Part done in 181.268 sec
2019-01-02 16:46:38 [GPU 1] 6 Parts 32-37/148...
2019-01-02 16:46:38 [GPU 1] Initializing octree...
2019-01-02 16:46:43 [GPU 1] Initializing histograms...
2019-01-02 16:49:03 [GPU 1] Initializing...
2019-01-02 16:49:05 [GPU 1] Iterating...
2019-01-02 16:53:36 [GPU 1] Updating result...
2019-01-02 16:53:38 [GPU 1] Part done in 419.143 sec
2019-01-02 16:53:38 [GPU 1] 8 Parts 38-45/148...
2019-01-02 16:53:38 [GPU 1] Initializing octree...
2019-01-02 16:53:43 [GPU 1] Initializing histograms...
2019-01-02 16:55:45 [GPU 1] Initializing...
2019-01-02 16:55:48 [GPU 1] Iterating...
2019-01-02 16:59:41 [GPU 1] Updating result...
2019-01-02 16:59:42 [GPU 1] Part done in 364.535 sec
2019-01-02 16:59:42 [GPU 1] 8 Parts 46-53/148...
2019-01-02 16:59:42 [GPU 1] Initializing octree...
2019-01-02 16:59:48 [GPU 1] Initializing histograms...
2019-01-02 17:01:20 [GPU 1] Initializing...
2019-01-02 17:01:23 [GPU 1] Iterating...
2019-01-02 17:04:11 [GPU 1] Updating result...
2019-01-02 17:04:13 [GPU 1] Part done in 270.713 sec
2019-01-02 17:04:13 [GPU 1] 8 Parts 54-61/148...
2019-01-02 17:04:13 [GPU 1] Initializing octree...
2019-01-02 17:04:18 [GPU 1] Initializing histograms...
2019-01-02 17:06:38 [GPU 1] Initializing...
2019-01-02 17:06:40 [GPU 1] Iterating...
2019-01-02 17:11:07 [GPU 1] Updating result...
2019-01-02 17:11:08 [GPU 1] Part done in 415.087 sec
2019-01-02 17:11:08 [GPU 1] 9 Parts 62-70/148...
2019-01-02 17:11:08 [GPU 1] Initializing octree...
2019-01-02 17:11:13 [GPU 1] Initializing histograms...
2019-01-02 17:12:45 [GPU 1] Initializing...
2019-01-02 17:12:47 [GPU 1] Iterating...
2019-01-02 17:15:33 [GPU 1] Updating result...
2019-01-02 17:15:34 [GPU 1] Part done in 266.279 sec
2019-01-02 17:15:34 [GPU 1] 11 Parts 71-81/148...
2019-01-02 17:15:34 [GPU 1] Initializing octree...
2019-01-02 17:15:39 [GPU 1] Initializing histograms...
2019-01-02 17:16:18 [GPU 1] Initializing...
2019-01-02 17:16:20 [GPU 1] Iterating...
2019-01-02 17:17:25 [GPU 1] Updating result...
2019-01-02 17:17:26 [GPU 1] Part done in 111.885 sec
2019-01-02 17:17:26 [GPU 1] 12 Parts 82-93/148...
2019-01-02 17:17:26 [GPU 1] Initializing octree...
2019-01-02 17:17:31 [GPU 1] Initializing histograms...
2019-01-02 17:18:15 [GPU 1] Initializing...
2019-01-02 17:18:17 [GPU 1] Iterating...
2019-01-02 17:19:21 [GPU 1] Updating result...
2019-01-02 17:19:22 [GPU 1] Part done in 116.118 sec
2019-01-02 17:19:22 [GPU 1] 17 Parts 94-110/148...
2019-01-02 17:19:22 [GPU 1] Initializing octree...
2019-01-02 17:19:28 [GPU 1] Initializing histograms...
2019-01-02 17:20:09 [GPU 1] Initializing...
2019-01-02 17:20:11 [GPU 1] Iterating...
2019-01-02 17:21:15 [GPU 1] Updating result...
2019-01-02 17:21:16 [GPU 1] Part done in 113.81 sec
2019-01-02 17:21:16 [GPU 1] 38 Parts 111-148/148...
2019-01-02 17:21:16 [GPU 1] Initializing octree...
2019-01-02 17:21:20 [GPU 1] Initializing histograms...
2019-01-02 17:21:49 [GPU 1] Initializing...
2019-01-02 17:21:51 [GPU 1] Iterating...
2019-01-02 17:22:22 [GPU 1] Updating result...
2019-01-02 17:22:23 [GPU 1] Part done in 66.767 sec
2019-01-02 17:22:23 Devices performance:
2019-01-02 17:22:23 - 100% done by [GPU 1] with timings: 4451 s = 0% fetch + 1% init + 35% hist (95% GPU) + 1% init + 62% iters (75% + 25%) + 0% updating
2019-01-02 17:22:23 IO utilization: 2%
2019-01-02 17:22:23 Level done in 4452.93 sec
2019-01-02 17:22:23 Level #13/13
2019-01-02 17:22:23 Subdividing...
2019-01-02 17:22:25 done in 2.234 sec
2019-01-02 17:22:25 [GPU 1] 5 Parts 1-5/316...
2019-01-02 17:22:25 [GPU 1] Initializing octree...
2019-01-02 17:22:31 [GPU 1] Initializing histograms...
2019-01-02 17:22:32 Finished processing in 5277.28 sec (exit code 0)
2019-01-02 17:22:32 Error: out of memory (2) at line 179