Processed thumbnails load too slowly

[cbnewham]

Thumbnails load incredibly slowly once the depth maps have been generated.

What's worse is that only a small number are loaded - if you scroll down, it forces more to load and the earlier ones disappear. It doesn't behave like this when the photos are initially loaded - everything is loaded super fast and always visible. It's been like this on every version up to the one I'm using (1.7.0) and I've always assumed it was a bug.

If this is not a bug, please provide a way to switch this behaviour off (so that only the thumbnail is loaded, and not any other background information). With many hundreds or thousands of photos it makes the thumbnails unusable.

[Alexey Pasumansky]

Hello cbnewham,

Thank you for reporting.

We'll try to reconsider the process of thumbnail loading (with the presence of the depth maps), but it may take some time.

[cbnewham]

I've only just seen you reply.

Thank you very much for considering a possible fix.

cbn

[CheeseAndJamSandwich]

I might be wildly wrong... But is this what's causing my 15+ minutes of 100% CPU when i load my big model???
It has 15,000 12MP GoPro photos...
Though Photos is not turned on in the GUI.  Only console & workspace.

[wojtek]

+1

I never realized it was the depth maps that slowed things down.

[wojtek]

Looks like this is fixed in 1.7.4?

[Alexey Pasumansky]

Hello wojtek,

There were some optimizations related to the depth maps thumbnail data.

[CheeseAndJamSandwich]

Hello wojtek,

There were some optimizations related to the depth maps thumbnail data.

I can see the difference between 1.7.3 and 1.7.5, whereby it's not getting stuck the thumbnails.zip file...
But it still remains reading every single depth map zip file, which for my project, means 20 minutes of 100% CPU.

I've bought this problem up here:
https://www.agisoft.com/forum/index.php?topic=13507.0

[JRM]

does having a single zip for thumbnails by chunk is a real space saver ? The decompression had a noticeable delay at opening with +100k files in a PSX project.

[CheeseAndJamSandwich]

does having a single zip for thumbnails by chunk is a real space saver ? The decompression had a noticeable delay at opening with +100k files in a PSX project.

If you open up any of the zips, you'll see there's zero compression, they just use compression set to 'store' only, which probably helps performance a bit as you're writing one big file to disk instead of thousands of little ones... where the HDD (less so for an SSD) will spend more time talking about reading, writing files, than actually reading or writing the files...
Using Store also does save disk space, especially with small files, as you eliminate slack space in the drive's clusters...  As you can only have one file per cluster... so tiny files, or just a lot of files, will have that slack space adding up.  One big zip file will only have the slack space from the very last cluster allocated to the file.

Looking at Procmon, it read the thumbnails.zip file right after load, is finished with that quickly, and then gets on with reading the 300 depth map zip...  Which takes aaaaaaaaaaaaaagggggggggggeeeeeeeeessss for me.
Does it really need to read them after load?  Could it read them later, when doing so is a prerequisite for a task?
Gonna continue deleting the depth maps, i guess...

[Alexey Pasumansky]

Hello CheeseAndJamSandwich,

As a workaround you can just uncheck "Set as active" option for the Depth Maps asset in the Workspace pane and check it again only when you need to re-use the depth maps data for the processing.

[CheeseAndJamSandwich]

Hello CheeseAndJamSandwich,

As a workaround you can just uncheck "Set as active" option for the Depth Maps asset in the Workspace pane and check it again only when you need to re-use the depth maps data for the processing.

Ah, that will be great!

As i'm only on Standard, and i'm pretty much done when the meshing has finished, and texturing doesn't actually show the topography as well as shaded, for a dive map like mine, i'm just deleting the depth maps, as it saves me a tonne of space too!

Thanks!