Forum

Author Topic: Performance questions and possibly a bug...?  (Read 4017 times)

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Performance questions and possibly a bug...?
« on: August 16, 2016, 03:44:43 AM »
Just curious, but how many of the processes are multithreaded; like switching from vertex color mesh to standard solid mesh takes a very long time on 40mil poly meshes, and I notice it is using just 25% CPU... are there plans to make Photoscan more multi-threaded?

I made a scan of a large section of a wall, and when I generated the mesh, it said High wanted about 34mil polys I think.. so I set it to custom and went to 70.  Looked soft, so I chunked it out an redid the dense cloud (at high) and mesh at custom of 40 mil (this time high said 24mil).  not any sharper in that area, but my question is.. did I need to redo the dense cloud?  What are the steps to get more resolution in details areas you break off to their own chunks?  Did I do it correctly?  Should i have re-aligned and maybe gotten a better tie-point value or something?  Is there any overhead lost by not doing this, or by not deleting the cameras that don't contribute to this new section?  The reason I know this is not a limitation of the source images is because the dense point cloud looks better than the vert-colored mesh... There is still far more detail there.

Possible bug:
Finally (for now), when generating large meshes like this, my machine with ~260GB of memory gets taxed pretty heavily, as expected, but I am noticing visual issues.. where some of my mesh is missing in the solid view immediately after the mesh generation finished, until I turn the colored mesh on..  You can see this in the two screenshots I am attaching.  I have seen this numerous times before, and the mesh is indeed missing for that lower section until I switch.  Trying to switch back to solid mesh, ALWAYS freezes at 25% CPU maxed for a very long time. 

I can go back and forth between dense point cloud and vertex colored mesh in less than a few seconds... Sounds backwards?  Is there no shading and specular or something with the colored mesh?  All I know is that it pegs 6gigs of ram (went from 100 to generate to 20 after, and trying to go to solid dumps the ram and jumps to 6 and just sits at 25% cpu.. I have not been able to get a solid mesh view at these resolutions easily.  Curious if it's the memory on the GPU limiting the system?  I have two MSI 970GTXs and one Quadro K4000 (which is set to be my display card in nvidia settings).  Any help in figuring this out would be appreciated.

Thanks!
« Last Edit: August 16, 2016, 03:51:10 AM by MeHoo »

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #1 on: August 16, 2016, 04:04:09 AM »
Oh, one other note.. when I re-built the dense cloud, I re-used the depth maps.. wondering if this is what caused identical output quality between scans?

I know it is hard to judge a dense cloud to a vert-colored mesh, but this seems pretty far off for doubling Photoscan's high estimate... I just feel like I am doing something wrong.  When I re-built the dense cloud after trimming down the chunk, it went from 91mil points to 108mil on the trimmed run.  So trimming it down did seem to add more detail to the area, but the mesh is still lacking in detail IMHO.  What's interesting to notice is that the previous chunk which was trimmed down from a much larger scan, ended up at 22mil polys when I generated it at double what Photoscan wanted originally for the whole wall.  The new estimate on the chunk when rebuilt went to 24mil.. so I have to believe it improved slightly at least.  it's just not perceivable.

And especially when projecting textures.. I can use 15 x 16k maps and still not get the same res as the dense cloud in areas, which makes me wonder how it can create such detailed point clouds and meshes, but the texture projection doesn't seem to use the same information?  This also leads me to believe that I am doing something wrong in my process.

Anyhoo.. I am wondering of the mesh is static for the most part because the diffuse details don't contribute enough to the shape.  I guess I just feel that the meshes are a bit softer than they should be overall.

Dense cloud vs 2x mesh density:
« Last Edit: August 16, 2016, 04:06:36 AM by MeHoo »

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #2 on: August 16, 2016, 04:26:06 AM »
..just loading them up tonight..

Ok.. so.. when I break these "detail chunks" off, I go in and prune the sparse cloud and shrink the bounding box.

Does "pruning" the sparse cloud matter at all?  Is this used to generate the dense cloud at all?

I am not getting any more sharpness or detail out of these scans no matter what I do.  Bounding box seems pointless unless memory is an issue.  Photoscan seems to generate a pretty damn good dense cloud from the get-go and no matter what I do after the fact, it seems that what I get from step one is what I get.  Is there anything I can do after generating a cloud, then moving up to a dense cloud, that might make "ultra high" even remotely worth the time involved?  It seems to take exponentially longer and produce very similar results to high for me.

Just curious how much manual labor might be wasted.  Thanks!
« Last Edit: August 16, 2016, 05:22:59 AM by MeHoo »

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #3 on: August 16, 2016, 05:06:48 AM »
Ok. further development.. It will switch between vertex color and solid rather quickly if I force the openGL renderer in the nvidia manage3d settings to the second 970GTX card.  Guessing that extra ~2 gigs of onboard memory helps..? (3 with reserved 1 gig.. ugh)

Any way to make Photoscan detect this sort of thing automatically?  This has been uber frustrating.

Also wondering if this is why my vertex color calculation step takes FOREVER at 100% for hours.. is it just switching to the solid mode and paging because the GPU can't handle the added memory?

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14104
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #4 on: August 16, 2016, 08:03:20 PM »
Hello MeHoo,

About the calculating vertex color step. It shouldn't take long time, so if it does, please send us the processing log from the Console pane related to the operation where we can see the timing and some auxiliary information. I think that the most likely thing that has happened is the visualization issue related to the Quadro card that was trying to display 40 million polygons at once (the fragmented model like in your first post seems to be indicating the same), so the application was not responding while the graphic card was struggling with the high-poly model. Usually professional class GPUs like Quadro are not optimized for polygonal model display and are mostly related to CAD graphics, so games class cards with plenty of VRAM are preferred for millions of polygons display. PhotoScan uses default OpenGL rendering device and cannot switch it automatically, but at least in the Console pane on the application start you can see which GPU is used for the visualization.
I can suggest to switch the default view to the Point Cloud mode in the Preferences window,  in this case PhotoScan wouldn't be  switching to the mesh when it is generated.
Best regards,
Alexey Pasumansky,
Agisoft LLC

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #5 on: August 16, 2016, 08:12:03 PM »
What's the email again?

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14104
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #6 on: August 16, 2016, 08:14:09 PM »
Best regards,
Alexey Pasumansky,
Agisoft LLC

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14104
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #7 on: August 16, 2016, 08:15:53 PM »
And as for the texture of the lower resolution, are you checking the details on the exported texture atlas or directly in PhotoScan window? In the latter case you should keep in mind that depending on the GPU capabilities PhotoScan may downscale the texture for the visualization purposes (see Maximum Texture Size line in the Console pane).
Best regards,
Alexey Pasumansky,
Agisoft LLC

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #8 on: August 16, 2016, 08:20:05 PM »
Emailed.  No these were previous scans, and the output texture compared to the in-view dense cloud detail was incredibly different.  No matter what I tried I could not get it to project properly.  I ended up just painting over it.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14104
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #9 on: August 16, 2016, 08:22:34 PM »
Hello MeHoo,

Thank you for sending the log. However, I can see only the part related to the texture generation there not to the mesh generation (calculating vertex colors is a step of Build Mesh process). However, I see that the color correction option is turned on and took about 1.5 hour (and about 20 minutes for the texture generation itself).
Color correction can really take a long time depending on the scene complexity and the number of images.
Best regards,
Alexey Pasumansky,
Agisoft LLC

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #10 on: August 16, 2016, 08:31:39 PM »
Yeah, I wasn't sure what was in the log because I am submitting all of this remotely.

I will try to remember to save one next time I generate a model.  And yes, color correction takes a while, but on these meshes it seems to help.

I'll send you a log next time I run a model

MeHoo

  • Full Member
  • ***
  • Posts: 136
    • View Profile
Re: Performance questions and possibly a bug...?
« Reply #11 on: August 18, 2016, 03:17:28 AM »
OK, Alexey, the log file is sent.  Thanks.