Forum

Poll

How can I get around this memory leak?

Agisoft need to fix the bug!
1 (100%)
Try lower resolution (with poor image texture this is not really an option!)
0 (0%)

Total Members Voted: 1

Author Topic: Memory leak in point matching  (Read 6421 times)

dcm39

  • Newbie
  • *
  • Posts: 34
    • View Profile
Memory leak in point matching
« on: November 21, 2013, 12:28:44 PM »
I am using a 16core system with 384GB RAM - Running out of RAM should not be an issue!

When I run a large photoset (1500-2500 photos) I expect the matching to run for somewhere between 5-8 days, but there appears to be a slow memory leak. As a result, once the matching starts, the amount of memory in use slowly increases without halt, until all 384GB is in use... but it doesn't crash immediately, it only crashes once the matching is basically finished!

I have tried this on both 0.9.1 and 1.0.0. The memory appears to leak more slowly on the beta version 1.0.0, but it is still a problem for anything longer than about 2 days.

Any help much appreciated.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13913
    • View Profile
Re: Memory leak in point matching
« Reply #1 on: November 21, 2013, 12:43:54 PM »
Hello dcm39,

Could you please send us the log file for this operation? Please also specify the image resolution and alignment parameters selected (accuracy, point limit, pre-selection).
Best regards,
Alexey Pasumansky,
Agisoft LLC

Wishgranter

  • Hero Member
  • *****
  • Posts: 1202
    • View Profile
    • Museum of Historic Buildings
Re: Memory leak in point matching
« Reply #2 on: November 21, 2013, 04:34:30 PM »
What OS you use ?
----------------
www.mhb.sk

dcm39

  • Newbie
  • *
  • Posts: 34
    • View Profile
Re: Memory leak in point matching
« Reply #3 on: November 21, 2013, 06:37:25 PM »
I'm using a Mandriva Linux system on a server, running agisoft remotely via ssh.

The photos are 3648 x 2736px, high accuracy, with no pair pre-selection. Anything above around 15000 point limit results in run times so long that the memory leak will fill all 384GB of RAM.

Where is the logfile written to on the linux based version?

Thanks

EDIT:
I have been using a few different numbers of photos, between about 1700-2250 photos.
The photos are pretty poorly textured, hence running at a large number of points.
« Last Edit: November 21, 2013, 06:41:57 PM by dcm39 »

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13913
    • View Profile
Re: Memory leak in point matching
« Reply #4 on: November 22, 2013, 02:08:30 PM »
Hello dcm39,

Thank you for the information, we will check if there is any memory leak.

However, for huge datasets we recommend to use pre-selection - Ground Control for aerial surveys if camera coordinates are present, or Generic for general tasks, if will reduce the Align Photos stage significantly.
Best regards,
Alexey Pasumansky,
Agisoft LLC

dcm39

  • Newbie
  • *
  • Posts: 34
    • View Profile
Re: Memory leak in point matching
« Reply #5 on: November 22, 2013, 02:52:30 PM »
Hi Alexey,

Unfortunately I don't have any camera coordinates, but I have tried using the generic pair selection ( I presume this uses an initial low resolution matching to select the pairs?). Sadly though, my data set is pretty poorly textured, so the generic pair selection is insufficient to align my photos. Hence I chose to bite the bullet and wait for the much longer processing for matching all pairs. Sadly though, this crashes for runs longer than about 2.5 days - even with such a large dataset , it really shouldn't need more than 300GB of RAM!

How does logging work in the linux version of agisoft? Does it write a log file automatically? If so, where to?

Many thanks.

dcm39

  • Newbie
  • *
  • Posts: 34
    • View Profile
Re: Memory leak in point matching
« Reply #6 on: November 22, 2013, 03:09:59 PM »
Also, If I have a partial list of camera coordinates (for example those generated in a lower res run for those parts of the model which are constructed ok) how will photoscan treat this if I select ground control preselection?

In other words do you need a complete list of camera positions in order to use ground control preselection? And if not, how does it treat cameras without location?

Thanks.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 13913
    • View Profile
Re: Memory leak in point matching
« Reply #7 on: December 13, 2013, 07:20:19 PM »
Hello dcm39,

Please try the latest pre-release update (build 1789). The problem should be fixed.
Best regards,
Alexey Pasumansky,
Agisoft LLC

dcm39

  • Newbie
  • *
  • Posts: 34
    • View Profile
Re: Memory leak in point matching
« Reply #8 on: February 20, 2014, 09:16:02 PM »
Thanks Alexey, the problem certainly seems to be solved now.

The time cost dependency on the number of features per photo seems to have changed now though... The time required used to scale really badly with the set maximum points limit, whereas now doesn't have much effect.

Has the algorithm changed somewhat? Have you introduced some filtering to reduce the number of points which actually get matched?

Cheers