Parallelizing large scale computations

[pts_and_clouds]

Hi everyone, I have an application that I am trying to make more computationally efficient.

I would like to project ALL dense cloud points to ALL cameras in my project to see which 3D points have valid projections.  So, I loop over all cameras and run a camera.project on all the 3D points.

The loop over all cameras is a simple for loop, but I have been able to speed up the loop over all 3D points by using a map operation over camera.project.

For each camera, on my M1 Max Mac with 64 GB RAM and 10 cores, the projection of all 3D points for each camera takes about 5 seconds.  A typical project has ~10 million points and ~300 cameras, so this operation takes a few minutes, which is something I would like to improve.

Some of the things I do now:
1. Remove redundant 3D points written out in the dense cloud, cuts down up to a third of 3D points (I don't know why the redundant points get written in the first place).
2. Instead of looping over every 3D point, I use a map operation to execute camera.project over all the 3D points.

Question:
1. How can I parallelize the camera.project step over all 3D points?  I am assuming that it is currently running only on 1 core
2. How can I parallelize the loop over all the cameras?

Thanks in advance, any help is greatly appreciated.

[Alexey Pasumansky]

Hello pts_and_clouds,

For the large amounts of point clouds and big number of points Python may be not so good, as it is slow in general. You may consider porting the code to Java API and compare if it works considerably faster compared to Python on the same data.

[pts_and_clouds]

Thank you, Alexey.