Pointcloud: Tackling Noise

[BobvdMeij]

Dear all,

We've recently started testing processing image sets with more complex geometries, i.e. containing both nadir and oblique shots using different camera angles. Unfortunately we're somewhat disappointed thus far with respect to both the processing time AND the outputted (dense) pointcloud. The disappointment grows when we compare both the processing time and subsequent output with another photogrammetry suite, which I will not name here for obvious reasons.

Attached you'll find a two screendump of a pointcloud depicting a bridge, the first being processed using Agisoft and the other using a different software. The differences are obvious, with the pointcloud in Agisoft depicting significant amounts of noise around the beams whereas the pointcloud created using the other software depicts rather sharp edges with no or very little noise. This likely follows from background noise in the original images, depicting the ground and/ or the sky in the background, and Agisoft for some reason (and unsuccesfully) trying to align these pixels. Furthermore Agisoft is hardly capable of reconstructing the metal pipes on the either side of the bridge, whereas the other package does not indicate such troubles at all!

How would I be able to get rid of this in an efficient manner? I'm aware of the option to mask images in Agisoft, but considering the rather complex geometry of the object under study and the image set containing well over 1.000 images this is hardly option as it will likely take several weeks (not so say months) to do so! It is striking to see that the other software suite seems to be very capable of removing/ filtering such background noise automatically, as no image masking was done to produce the output in the second image.

It's also worth mentioning that we were forced to downsize the original image set (1.100 images) to about 800 images for processing in Agisoft as the full set would not process at all OR would take well over a week (regardless of our decent PC setup). The other software package, however, had no problems at all with aligning all the images and was even able to do so within less than 24 hours!

[SAV]

Hi BobvdMeij,

Thanks for sharing these results. I assume you were comparing Agisoft PhotoScan Pro to Pix4D (or Reality Capture?)

Anyway, this new feature which is available since version 1.4.1 might help you to speed up masking and therefore help to remove noise around the beams:
http://www.agisoft.com/index.php?id=49

Regarding processing time. The point cloud produced with Agisoft looks more dense in your screenshots (more and smaller 'pixels' compared to a smaller amount of bigger 'pixels' in the second point cloud). Have you used comparable quality settings?

I've experienced the same when trialing Pix4D some time ago. It is capable of processing large projects without having to split them up (I guess the software does that automatically in the background). Agisoft PhotoScan does not do that automatically, but there is a simple fix using this Python script:
https://github.com/agisoft-llc/photoscan-scripts/blob/master/src/split_in_chunks_dialog.py

Would be interesting to see if you get better results with adjusted processing settings. Keep us updated.

All the best.

Regards,
SAV

[JMR]

Obviously is not P4D. Your images show undeniable advantage in this scenario for the competitor but should be noted that they show diferent stages in processing. while the first is the result of dense stereo matching to produce the dense cloud, the second is a mesh model rendered as points, ant that explains partially the cleaniness. That means more processes have been carried out, and not just brute force matching algorithms.
Instead of being an excuse, this fact makes the comparison more dramatic when it comes to speed.
Agi team must take note

[BobvdMeij]

SAV & JMR, thanks for both of your reponses!

The other package applied was indeed Reality Capture, not P4D. The pointcloud was processed in RC using 'Normal' quality (it only has Normal or High), as processing on High would cause me running into memory issues. Although, admittingly, the point density in RC's pointcloud is seemingly lower compared to the pointcloud processed in Agisoft (set to High quality), I still believe the pointcloud by RC is significantly more usable than the one outputted by Agisoft. Regardless of the slightly lower point density it managed to capture notably more details and individual components, whereas Agisoft did only marginally or was incapable of rendering such details all together. The pointcloud from RC may be succesfully translated into a BIM model of some sort, whereas the one from Agisoft is rendered largely useless in this respect. Add to that the significantly shorter processing time of RC and I must agree with you, JMR, that the results is especially dramatic.

I will give both of your suggestions a try, SAV, and will come back to you as soon as I get some new results. I am wondering the following about the python script though, what specific stages of the Agisoft workflow does it encompasses? Or, put differently, which steps of the workflow are to be completed before I run the script? Does it run from Align Photos onwards or does it only run the Densecloud generation? In case of the former I'd like to point out that I still require various manual steps in between specific processing steps and marking GCPs, applying Gradual Selection filters and Optimizing Camera's in particular.

Cheers!

[Alexey Pasumansky]

Hello BobvdMeij,

If there's any sample image set for part of this project or any similar project with the same issues, so that we could test it on our side for internal purposes.

Also is the dense cloud the final product that you are interested in? (or at least one of the products).

As for the long processing time for the dense cloud filtering stage, it depends on the image overlap - for the datasets with the excessive overlap the processing time will be considerably longer. In the version 1.4.2 you can use a "hidden setting" to limit the number of evaluated pairs for each camera during the filtering to reduce the processing time.
To do that go to Tweaks dialog from Advanced Preferences tab and input the parameter with the name "main/dense_cloud_max_neighbors" and set its value to 60, for example. (To switch back to default behavior use -1).

[BobvdMeij]

Thanks for your feedback Alexey, much appreciated!

I'd be very willing to share the image set to see how you will get on, but the dataset is pretty large. What would be the best way to get it out to you?

As for your suggested hidding setting I will give that a try ASAP and let you know if it helps. I was only capable of downloading 1.4.2 as of today so I'll be checking some other new features in the meantime first. I am wondering though, how should I interpreted this suggested value of '60' for the said new parameter? Does this relate to a percentage or absolute number and, in case of the latter, on what kind of scale? Also how does it compare to the default value of -1? Also, am I correct to assume that limiting the number of evaluated image pairs will possible introduce lower model accuracy and/ or poorer image alignment?

[Alexey Pasumansky]

Hello BobvdMeij,

As for the hidden setting - it's an absolute number defining the number of paired images per camera to be evaluated to filter the depth (in the version 1.3.0, for example, the setting has been implemented as default with the 50 value. it worked faster, but for some projects this threshold was too low and additional noise appeared, so we removed the threshold. -1 means that there's no limit).
I think values ranging from 50 to 80 should give considerable speedup for the projects with the excessive overlap, but as for the quality of the output, likely it wouldn't be better than for default settings - unlimited threshold. So setting this parameter may lower the quality of the output (meaning more noise), but wouldn't affect the image alignment - the option is only for the dense cloud filtering, not for any other stages.

For the data sharing I can send you the access parameters to our FTP server if it is a convenient method for you.

[BobvdMeij]

Hello BobvdMeij,

As for the hidden setting - it's an absolute number defining the number of paired images per camera to be evaluated to filter the depth (in the version 1.3.0, for example, the setting has been implemented as default with the 50 value. it worked faster, but for some projects this threshold was too low and additional noise appeared, so we removed the threshold. -1 means that there's no limit).
I think values ranging from 50 to 80 should give considerable speedup for the projects with the excessive overlap, but as for the quality of the output, likely it wouldn't be better than for default settings - unlimited threshold. So setting this parameter may lower the quality of the output (meaning more noise), but wouldn't affect the image alignment - the option is only for the dense cloud filtering, not for any other stages.

For the data sharing I can send you the access parameters to our FTP server if it is a convenient method for you.

Looks like I may be able to upload the image set to our WeTransfer account and send you a link for download. Would that work? If so, what would be the best way to provide you with the download link? Through a PM, perhaps?

[Alexey Pasumansky]

Hello BobvdMeij,

The download link would be fine. You can send it to me via PM or to support@agisoft.com with the reference to this thread.

[BobvdMeij]

Hello BobvdMeij,

The download link would be fine. You can send it to me via PM or to support@agisoft.com with the reference to this thread.

I just sent you a PM. Please let me know if you did or did not receive the message.

Cheers!

[Alexey Pasumansky]

Hello BobvdMeij,

I've got the link with the additional info. Starting the download now.

[BobvdMeij]

Hello BobvdMeij,

I've got the link with the additional info. Starting the download now.

I understand you are likely very busy, but any news on this perhaps?

Cheers!

[Alexey Pasumansky]

Hello BobvdMeij,

We have just finished reprocessing the dataset on our side and are now observing similar issues in the dense cloud that you have mentioned in the first post.

[BobvdMeij]

Hello BobvdMeij,

We have just finished reprocessing the dataset on our side and are now observing similar issues in the dense cloud that you have mentioned in the first post.

That's good to hear in a sense that I'm supposedly doing nothing wrong. On the other hand, any clues or suggestions on how such artifacts can be prevented in the future?

[Mak11]

Sorry for bumping this topic up. But has anything new been done in MetaShape (PS 1.5xx)  to solve those kind of issues?

Cheers

Mak