Pink screen freeze when running PhotoScan

[maddin]

Today, my system started freezing up while PhotoScan Pro was building geometry, coloring the entire screen in a uniform pink color and locking up completely (had to power it down).

This is on Win7 x64, 12GB RAM, Geforce GTX 580.

I have updated the Nvidia drivers now but would like to ask whether anyone else here has seen this before? Should I order a new GPU?

Thanks!
Martin

[Alexey Pasumansky]

Hello Martin,

In case the problem will appear again please use some tests for memory modules and graphic card. Probably hardware is overheated.

[maddin]

In case the problem will appear again please use some tests for memory modules and graphic card. Probably hardware is overheated.

Yep, problem persists after driver update. Always in the middle of a Medium Quality reconstruction of 26 aligned images, each 6732*8984 pixels.

What kind of tests do you recommend?

Thanks
Martin

[Alexey Pasumansky]

Hello Martin,

Could you please save the log file (path should be specified in the PhotoScan Preferences)?

Please also let us know if the problem could be reproduced on the different datasets.
And another test could be performed if you uncheck GPU in the OpenCL tab and restart the calculation (even it will take longer time).

For memory checks we recommend Memtest86 utility.

[maddin]

Could you please save the log file (path should be specified in the PhotoScan Preferences)?

log file attached (up to the point where it crashed)

It crashes when I try to build Geometry (using Height Field / Sharp / Medium settings) during 'Reconstructing Depth' at around 17%.

I also tried other GPU-intensive software (3ds Max iray) as well as OpenCL benchmarks and none crashed my machine, despite the GPU getting hotter (64 degrees celsius vs. 62 for PhotoScan) and the GPU memory being filled more (up to 1.4GB vs. 900MB for PhotoScan).

FWIW, my two screens are no longer pink now after the crash but 'petrol' and 'dark magenta'

Please also let us know if the problem could be reproduced on the different datasets.
And another test could be performed if you uncheck GPU in the OpenCL tab and restart the calculation (even it will take longer time).

Will do and report back.

For memory checks we recommend Memtest86 utility.

Will that test GPU memory, too?

Thanks
Martin

[maddin]

Some more bits and pieces of information:

- processing the identical scene using CPU only takes quite long but works flawless otherwise

- enabling GPU support after the CPU has computed the depth maps works fine, too, with the 'Resue Depth Map' option

- processing a different scene (using more but smaller images from a different camera, though) works fine with OpenCL GPU at the same geometry setting

- monitoring the GPU during the calculations (using GPU-Z) shows nothing special (peak temp is 61 deg) and when compared to the luxmark OpenCL benchmark, for example, I see very similar figures. Also, filling the GPU memory with geometry and textures using 3ds Max iray does not produce any errors.

Any more suggestions?

martin

[Alexey Pasumansky]

Hello Martin,

Disabled OpenCL and re-use of depth maps doesn't utilize GPU, so it looks like there could be some problems with graphic cards.
I recommend to run some stress tests on your GPU, I'll also try to find some if you do not have any by the hand. For example, GPU Caps, but I'm not sure if it loads the GPU hard.

[maddin]

I recommend to run some stress tests on your GPU, I'll also try to find some if you do not have any by the hand. For example, GPU Caps, but I'm not sure if it loads the GPU hard.

I have done that yesterday without any crashes. I tried 3D Mark, I tried luxmark (which targets specifically OpenCL), I tried CompuBench CL, I tried 3ds Max iray, each one utilizing the GPU at 100% for more than ten minutes. No crashes anywhere.

I am happy to try GPU Caps right now but do not expect any crashes either. It is somehow related to what PhotoScan is doing at Medium Quality with these large photos. :-(

Martin

[Wishgranter]

Maddin, pink color point to some problem with GPU ram ( you need to stress test entire ram of GPU for mor ethen 10minutes) or its something with the drivers..... try uninstall drivers, disconnect from NET, restart, install new drivers, restart and then run some benches...... what amount of triangle you generate in that project ?? its Win or OSX system ??

[maddin]

pink color point to some problem with GPU ram ( you need to stress test entire ram of GPU for mor ethen 10minutes)

OK, thanks for the confirmation, I could not find any pointer on the web about these solid color screen crashes. I have 1.5GB VRAM, so the normal benchmarks do not fill this. Which is why I rendered a very heavy scene (both geometry and textures) using iray, which is a CUDA-enabled GPU raytracer that comes with 3ds Max. With that, I could fill my VRAM entirely without any crashes.

or its something with the drivers..... try uninstall drivers, disconnect from NET, restart, install new drivers, restart and then run some benches......

I did a normal driver update (not uninstall, reboot, install) right after the second crash without any differences.

what amount of triangle you generate in that project ?? its Win or OSX system ??

The freeze happens long before PhotoScan gets to the triangulation stage, right in the middle of the disparity/depth calculation. On the same system, I have successfully processed scenes with over 70 mio faces and 40 mio vertices in PhotoScan without issues. Also keep in mind that no freeze occurs when I compute the depth maps using CPUs only and then reuse those for Geometry building.

System specs: Win7 x64, 12 GB RAM, GTX 580 with 1.5GB VRAM, Nvidia ForceWare 327.23 (no overclocking), PhotoScan Professional 0.9.1 build 1714

Thanks!
Martin

[maddin]

Here's an update:

Running the 'GL 2.1 - Furry Cube' test at high resolution and FSAA for ten minutes, I was able to trigger the same solid color screen freeze, so it's not directly related to PhotoScan!

I swapped the GPU for a GTX 650 Ti (nothing else available ATM) and now PhotoScan works nicely again, with OpenCL enabled.

Funnily enough, the graphics card is still on waranty for two more days, so I should be able to get a replacement.

BTW, can anyone recommend a good graphics card for use with PhotoScan?

Thanks again for the support!
Martin

[Wishgranter]

7970 from AMD......

[maddin]

7970 from AMD......

Thanks! Need Nvidia for other software, though. I consider a GTX 780 now.

Martin

[maddin]

Just a follow-up for anyone interested: Yesterday, I damaged my second graphics card while running Photoscan. Not sure it is a coincidence ;-)

Martin

[Exhale]

As we disscussed your PSU needs to be checked.. Even upgrade should be done...
Sad but true, now you can go for AMD Radeon™ HD 7970 Graphics   
I will go for it as well....  Hawaii(7970) shows remarkable performance for sure..