Heavy dense cloud not showing when duplicating chunks

[Thibaud Capra]

Hello,

I'm working with heavy point clouds (~300 million points) where I need to locate areas of interest to then work on these tiny areas.
Here's my workflow:

• Align Photos
  
• Densify Cloud
  
• Duplicate chunk
  
• Define AOI on dense cloud
  
• Mesh
  
• Texture
  
• Export

The sparse cloud is way too sparse to locate the AOIs that's why I'm working on the dense cloud. As I'm working on an average 40 AOI per cloud, I have a Python script going through steps 1/2/3 then letting me do step 4 before running another script for steps 5/6/7.

Problem is, when running my script successfully, I have no dense cloud in my chunks. I checked the console, and it says it created 307.000.000 points. However, my chunks, even though being created, only contain the sparse clouds (1.117.000 points) !

Is it because of hardware issues?
Current workstation:

• CPU: Intel Xeon E5 2650 v4
• GPU: nVidia Quadro M4000
• RAM: 64Gb

RAM's far from being overflowed: 13Gb are used... Maybe the GPU being unable to render it?
If it is a hardware issue, I'll change my workflow accordingly. If it isn't, where does it come from?

Thanks in advance.

[Alexey Pasumansky]

Hello Thibaud,

Do you see the dense cloud in the chunk contents in the Workspace pane?

[Thibaud Capra]

Hello Alexey,

No, I can't see it. The only thing is the chunk is "Tie Points".
I can do the Meshing step though.
By the way, PhotoScan 1.2.6 here.

[Alexey Pasumansky]

Hello Thibaud,

And how you are duplicating the chunks via scripting?

[Thibaud Capra]

Hello Alexey,

I'm using this piece of code:

Code: [Select]

chunkIter = 1
chunk = doc.chunk                 # active chunk, if there's any in the project
while chunkIter <= nbTrous:
      chunk.label = "Trou" + str(chunkIter) + "-Green"
      chunk =  doc.chunk.copy()
      chunk.label = "Trou" + str(chunkIter) + "-Approche"
      chunk =  doc.chunk.copy()
      chunkIter += 1

[Alexey Pasumansky]

Hello Thibaud,

Is it correct that active chunk contains the dense cloud before it is duplicated for the first time?

Can you run just a single line copy from the Console, simulating the operation to check if it is working properly.

[Thibaud Capra]

Hello Alexey,

Yes, the full code includes the point cloud densification before duplicatinig chunks. According to the console, there's a line saying "XXXX points extracted".

I did the point cloud manually in one chunk, it have it now displayed properly. I'll try to run the chunk duplication with a separate script if that's what you mean?

Here's the full code:

Code: [Select]

import os, sys, PhotoScan

doc = PhotoScan.app.document
chunk = doc.addChunk()
nbTrous = int(sys.argv[1])
path_photos = PhotoScan.app.getExistingDirectory("""Spécifiez le dossier
 contenant les photos""")
path_photos += "\\"

# Checking save filename
project_path = PhotoScan.app.getSaveFileName("""Spécifiez le nom du projet
 à enregistrer : """)
if not project_path:
    print("""Annulation du script : pas de chemin d'enregistrement fourni""")

if project_path[-4:].lower() != ".psx":
    project_path += ".psx"

# Saving for G3D Reco
project_path_reco = PhotoScan.app.getSaveFileName("""Spécifiez le nom du projet
 à enregistrer pour Green 3D Reco : """)
if not project_path_reco:
    print("""Annulation du script : pas de chemin d'enregistrement fourni""")

if project_path_reco[-4:].lower() != ".psx":
    project_path_reco += ".psx"

image_list = os.listdir(path_photos)
photo_list = list()
for photo in image_list:
    if photo.rsplit(".",1)[1].upper() in ["JPG", "JPEG", "TIF", "PNG"]:
           photo_list.append(path_photos + photo)
           print(photo)
    else:
           print("""Pas de photo exploitable dans le dossier.""")
print(photo_list)
chunk.addPhotos(photo_list)
PhotoScan.app.update()

chunk.matchPhotos(accuracy = PhotoScan.MediumAccuracy,
    preselection = PhotoScan.ReferencePreselection, filter_mask = False,
    keypoint_limit = 40000, tiepoint_limit = 4000)
chunk.alignCameras()
doc.save(project_path_reco)

chunk.buildDenseCloud(quality = PhotoScan.MediumQuality,
filter = PhotoScan.AggressiveFiltering)
doc.save(project_path_reco)

chunkIter = 1
chunk = doc.chunk                 # active chunk, if there's any in the project
while chunkIter <= nbTrous:
      chunk.label = "Trou" + str(chunkIter) + "-Green"
      chunk =  doc.chunk.copy()
      chunk.label = "Trou" + str(chunkIter) + "-Approche"
      chunk =  doc.chunk.copy()
      chunkIter += 1

doc.save(project_path)
Bonus question: how can I use the PhotoScan.estimateImageQuality to align only photos with a quality score > 0,5?

[Jeremiah_ROWE]

When I can't determine an AOI to snip out from either cloud, I make a quick lo-res mesh and texturize it. I then use that as my visual guide for locating control points and other features of interest. Keep in mind, I'm not doing any scripting.

[Thibaud Capra]

Hello Jeremiah,

I would do that but I'm also using the whole point cloud for other purposes.
Basically the whole point cloud is processed (saved in a different file) using another script, and the AOI are processed individually using a third script for finer resolution. Try to imagine a process branching out, and this first part is the common part to both, so I need this to work!

[Alexey Pasumansky]

Hello Thibaud,

When you are saving the document in PSX format for the first time, it works as "Save As" function, so PSX project is saved and immediately re-opened. And in Python you need to re-assign chunk as chunk = doc.chunk. Otherwise the chunk that you have processed is not in the active document anymore.

If you do not need to think about that, you can save document in the very beginning of the script and after that use chunks from it.

[Thibaud Capra]

Hello Alexey,

I did not know that! Thanks for helping. I'll run the new code right away and see... Tomorrow, 24 hours computing time.

As a heads up, I ran my chunk duplicating part only on the single dense cloud I had manually computed and I had my 40 chunks with the dense cloud each time, so I can confirm it's not my hardware limiting whatsoever.

Thanks again.

[Thibaud Capra]

Hello Alexey,

Here's an update: it didn't work.

I added a chunk = doc.chunk between the Align Photos where I'm saving the project and the densification but it does not work. I guess it's not the right place!

Here's the code:

Code: [Select]

chunk.matchPhotos(accuracy = PhotoScan.MediumAccuracy,
    preselection = PhotoScan.ReferencePreselection, filter_mask = False,
    keypoint_limit = 40000, tiepoint_limit = 4000)
chunk.alignCameras()
doc.save(project_path_reco)

chunk = doc.chunk

chunk.buildDenseCloud(quality = PhotoScan.MediumQuality,
filter = PhotoScan.AggressiveFiltering)
doc.save(project_path_reco)

chunkIter = 1
chunk = doc.chunk                 # active chunk, if there's any in the project
while chunkIter <= nbTrous:
      chunk.label = "Trou" + str(chunkIter) + "-Green"
      chunk =  doc.chunk.copy()
      chunk.label = "Trou" + str(chunkIter) + "-Approche"
      chunk =  doc.chunk.copy()
      chunkIter += 1

[Alexey Pasumansky]

Hello Thibaud,

I've made a quick test in 1.2.6 removing unnecessary lines and it worked properly.

I can suggest to make a quick test run (no need to run the script for hours to get no result) based on the already existing alignment with just a few cameras:

Code: [Select]

path = "D:/test.psx"
chunk = PhotoScan.app.document.chunk
doc.save(path)
chunk = doc.chunk

chunk.buildDenseCloud(PhotoScan.LowestQuality)
if chunk.dense_cloud:
      print(chunk.dense_cloud.point_count)
else:
      print("no points)
doc.save(path)
chunk = doc.chunk
if chunk.dense_cloud:
      print(chunk.dense_cloud.point_count)
else:
      print("no points)

chunk = doc.chunk.copy()
if chunk.dense_cloud:
      print(chunk.dense_cloud.point_count)
else:
      print("no points)



[Thibaud Capra]

Hello Alexey,

After running a few tests with my script on smaller samples as you advised, I fixed the code. The fact that I was saving two different projects while processing messed up the whole thing!

Thanks for the support, as always.