Forum

Author Topic: Azure NC24 & Nvidia K80 not working with Photoscan (edit: DOES work)  (Read 5203 times)

TXPE

  • Jr. Member
  • **
  • Posts: 54
    • View Profile
My IT group set up new NC24 running Server 2012 R2 Datacenter w/ 2 Xeon E5-2690 and 4 Nvidia Tesla K80 cards.

When they originally set it up, it didn't have any Nvidia drivers.  I installed latest (377.35) drivers.

I am connecting via VNC.

I am still getting the "text labels will not be rendered with current OpenGL." message. I can hit OK and hit Alt-T to get to Preferences to change Stereoscopic Display to Hardware and screen will appear (along with another error).

Log file shows:
2017-06-09 22:47:09 OpenGL Vendor: Microsoft Corporation
2017-06-09 22:47:09 OpenGL Renderer: GDI Generic
2017-06-09 22:47:09 OpenGL Version: 1.1.0
2017-06-09 22:47:09 Maximum Texture Size: 1024
2017-06-09 22:47:09 Quad Buffered Stereo: not enabled
2017-06-09 22:47:09 ARB_vertex_buffer_object: not supported
2017-06-09 22:47:09 ARB_texture_non_power_of_two: not supported

I don't know why the Nvidia card(s) are not showing here.

The Nvidia Control Panel appears to show be very limited info.

Any ideas on how to make Agisoft use Nvidia cards?

I have a 10,448 photos I need to work on.

Thanks,
Doug
« Last Edit: June 13, 2017, 09:47:44 PM by TXPE »

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14107
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan
« Reply #1 on: June 10, 2017, 09:30:07 AM »
Hello Doug,

Do you see Tesla cards in the GPU tab of the Preferences window? I don't think that Tesla can be used for the OpenGL rendering, as their main intention is to be a compute device.
Best regards,
Alexey Pasumansky,
Agisoft LLC

TXPE

  • Jr. Member
  • **
  • Posts: 54
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan
« Reply #2 on: June 10, 2017, 06:19:57 PM »
Yes, all 4 Tesla K80 cards show up.  Initially, they were all unchecked. I checked them.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14107
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan
« Reply #3 on: June 10, 2017, 09:31:29 PM »
Hello Doug,

I also can suggest to switch the default view mode to  Point Cloud in preferences, to avoid automatic switching to mesh view. High poly models may be problematic for visualization on software emulation of OpenGL renderer.
Best regards,
Alexey Pasumansky,
Agisoft LLC

TXPE

  • Jr. Member
  • **
  • Posts: 54
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan
« Reply #4 on: June 13, 2017, 06:43:43 AM »
Well, it (NC24) seems to be working.

It aligned 10409 out of 10412 photos in 4h, 40min.  This was with 3 of the 4 Tesla cards.  For some reason, one of the cards wasn't being recognized in Device Manager.  I rebooted server and now it recognizes the 4th card.

I've started the dense point cloud step with Med and Moderate settings.  Machine has 224GB of RAM, so I hope it runs this step OK.  I'm about 25 minutes into this step (reconstructing depth) and at 43GB used.  I also have the "use CPU" option unchecked.


TXPE

  • Jr. Member
  • **
  • Posts: 54
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan
« Reply #5 on: June 13, 2017, 07:58:30 PM »
The dense point cloud is complete after 13h 25min.  I actually thought it might go a little faster than that, but I'm happy with it and I'm very glad PS is able to handle the ~10k photos at once (no chunks). 

I thought the console used to show how fast the GPUs processed data.  I don't see that info.  I do see where it shows a breakdown by percentage of how each card was used (24-25% per card).

As you stated, the NC24 is set up for for "GPU computing" not "GPU visualization".  I found this:
https://azure.microsoft.com/en-us/blog/azure-n-series-general-availability-on-december-1/ that explains the basic differences, but how these differences apply here are over my head.

Do you think the GPU visualization (NV) setup would perform better than the GPU computing (NC) setup?

Thanks,
Doug

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14107
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan
« Reply #6 on: June 13, 2017, 08:42:52 PM »
Hello Doug,

K80 seems to have more CUDA cores compared to M60 from NV configurations, so in terms of PhotoScan performance NC24 should be faster than NV24.

If working with NC configurations I can suggest to switch the default view mode to point cloud the Preferences window and avoid manual switching to the mesh view modes (shaded/solid/wireframe/textured) for high-poly models.
Best regards,
Alexey Pasumansky,
Agisoft LLC

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14107
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan (edit: DOES work)
« Reply #7 on: June 14, 2017, 07:20:59 PM »
I think that the following instruction would be helpful for those who are experiencing similar OpenGL issues with Tesla cards:
https://stackoverflow.com/a/30143380/2107734
Best regards,
Alexey Pasumansky,
Agisoft LLC

jinjamu

  • Newbie
  • *
  • Posts: 49
    • View Profile
Re: Azure NC24 & Nvidia K80 not working with Photoscan (edit: DOES work)
« Reply #8 on: July 09, 2017, 02:58:05 PM »
Hi
Rather than starting a new thread on Azure GPU machines I hope you don't mind continuing on this one.
I've started to carry out some benchmarking on Azure VMs in anticipation of of a project where I form part of a team which will be using photogrammetry to document deep historic wrecks in the seas around Malta.  I am working with a sample set of 607 stills captured from HD video (so 2 Megapixel each still), attempting to compare the performance of the NV24 and NC24 VMs.  (More info on the VMs here https://azure.microsoft.com/en-us/blog/azure-n-series-preview-availability/ )
I aligned the images, optimised the cameras and generated the dense cloud.  In both cases I reset the Photoscan parameters to default to ensure the same baseline.  I ran the alignment and the dense cloud both on "High", otherwise leaving the settings as per default.
What has really confused me is that the different machines, running the same software version, with the same settings, on the same set of photos, generated different outcomes!  For starters, the NV alignment was more successful than the NC - after the initial run, on the NV just some 6-7 adjacent photos were not aligned and had to be aligned using the Right Click..Align method.  On the NC there was an additional set of some 15 adjacent photos which needed "forced" alignment.  The final outcome was the same though - 603 out of 607 aligned
When I ran the dense cloud, the NC was a lot faster, however on examining the result, the NC only generated some 7 million points vs the NV's 17 million - which I guess accounts at least in part for the NC being faster.
I am attaching the 2 different "info"s....there are a number of differences also in other areas (way above my head) for example the RMS reprojection errors are totally different, etc
Would be really curious to know if anyone can provide any explanation.
The models can be made available if anyone wants a closer look
Thanks,
John