Forum

Author Topic: Network process size problem  (Read 5071 times)

bmc130

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Network process size problem
« on: January 22, 2016, 01:52:28 PM »
Hi, I have some issues running scans on our render farm and get really large files compared to what I guess I should have.

My original scan in psx format is 10gigs. In order to process the dense cloud on our farms I need to split up my dense clouds in smaller chunks. So I split the 10 gig chunk into two chunks and remove all points I dont need for those chunks (half points are removed in each chunk)

I delete my original chunk (full 10 gig chunk) and have two chunks (should be 5 gig each kind of)

I save my psx and notice now I have a 20gig file, it looks like the points I have deleted are not deleted? Is this expected, I assume my file should still be 10 gig even though it has 2 chunks with removed data?


Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Network process size problem
« Reply #1 on: January 22, 2016, 02:15:10 PM »
Hello bmc130,

When you remove points from the sparse cloud you just mark the tie points as invalid (not used) but their projections are still kept for the corresponding images.
Best regards,
Alexey Pasumansky,
Agisoft LLC

bmc130

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Network process size problem
« Reply #2 on: January 22, 2016, 02:18:48 PM »
Not the sparse cloud I remove but the dense cloud, I assume those are the ones that are the 10gigs? I have already deleted my spares cloud.

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Network process size problem
« Reply #3 on: January 22, 2016, 02:20:40 PM »
Hello bmc130,

Have you tried re-saving the project under the different name with the chunks that have remove dense cloud points?
Best regards,
Alexey Pasumansky,
Agisoft LLC

bmc130

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Network process size problem
« Reply #4 on: January 22, 2016, 02:27:50 PM »
Did a resave now and removed all chunks except one, and that one alone is as large as the full original file (full dense pointcloud, 10gig)

bmc130

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Network process size problem
« Reply #5 on: January 22, 2016, 02:30:22 PM »
tried deleting half of the dense points from that chunk and still 10 gig

bmc130

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Network process size problem
« Reply #6 on: January 22, 2016, 05:47:06 PM »
Is there a reason that deleted dense points are not deleted when you delete them? If I choose to delete points I most likely do it for quality and scene size optimization, but if nothing gets deleted might as well keep them?

In my real scene example I went from 10 gigs size of scene to 80 gigs!, moving that amount of data on the network is time consuming and not what you would like in a work environment..

Any workaround for this issue would really help..

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Network process size problem
« Reply #7 on: January 22, 2016, 05:49:04 PM »
Hello bmc130,

We are currently trying to reproduce the problem.

The points are not deleted instantly from the project data, as otherwise you wouldn't be able to undo the point removal operation.
Best regards,
Alexey Pasumansky,
Agisoft LLC

Alexey Pasumansky

  • Agisoft Technical Support
  • Hero Member
  • *****
  • Posts: 14813
    • View Profile
Re: Network process size problem
« Reply #8 on: January 22, 2016, 08:11:00 PM »
Hello bmc130,

The problem in case of PSX format is related to the following: the dense cloud is not loaded fully in the memory, so updating it performed partially, but in fly and the points are marked as deleted (allowing the Undo operation) but are actually kept in the cloud. Removing points completely after each point editing operation would mean re-writing of the dense cloud, that can take tens of minutes. I think we'll implement the option in the next update, allowing to remove the deleted points from the dense cloud completely by the user request.
Best regards,
Alexey Pasumansky,
Agisoft LLC

bmc130

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Network process size problem
« Reply #9 on: January 24, 2016, 12:21:17 AM »
It would be nice to choose both options :)