focus stacking for photogrammetry

[Gianni]

Hi guys,

I'm trying to find a good workflow for focus stacking to use the stacked images in agisoft.

Right now the process is to stack all the photos in photoshop (stack by stack) and saving them out manually.
This takes a ton of time, so I'm wondering if there is a better way or another software, that can kind of batch this process.

Something like say it makes groups of 5 or 10 or whatever many images by itself, stacks them and saves the focus stacked images out?

All ideas are much appreciated

Cheers

[wizprod]

Just out of curiosity, have you tried loading all images into PhotoScan?
Maybe the sharpness zones will be detected by Photoscan (out of focus areas have less weight than sharp areas), saving you a lot of manual stacking.

[Gianni]

I've done that before, sometimes it works and sometimes it doesn't. I think that depends on how much of the image is actually sharp.
I also think that agisoft sometimes has trouble with aligning if two cameras have the same position.

Also it takes a lot longer to align and build the dense cloud when you have something like 5 times the images (or even more) if they are not stacked before loading them into agisoft.

I somehow just can't believe that there is no automated process for this, I hope somebody has already found a way to do that.

[Twister]

Indeed, there are a number of focus stacking programs available, both commercial, as 'Helicon Focus' and 'Zerene Stacker', and free, as 'Picolay' und 'Combine ZP'.

The sticking point, however, when you use focus stacked images for photogrammetry, is perspective: The photogrammetric process requires a unique perspective for all parts of the image, which usually is not present in a focus stack, whether you capture the images with a macro rail or by adjusting the focal plane with the focal ring. In the "stacked" image with extended focal depth, the slight perspective errors are not noticable to the human eye, but they deteriorate the quality of a 3d model calculated with PhotoScan or any other photogrammetric software. The models may be acceptable for compact subjects, but for delicate subjects as insects, the results are mostly disappointing.

A way to overcome the persprcective problem, is a calibration of the camera motion used for the image capturing, together with an algorithm for the correction of the perspective change, see for this our publication zookeys.pensoft.net/articles.php?id=24584 on insect 3d modelling with PhotoScan.

Best regards,

Twister

[James]

just an idea i haven't tried, but perhaps if you used the 'station type' camera group feature of photoscan (metashape) to tell it that images from each stack are absolutely in the same location, then some of the processing inefficiency and uncertainty could be overcome.

also split any camera calibration groups so that it is able to attribute individual distortion parameters to each differently focussed image.

[morganW]

What kind of camera are you using and how small is the object? If you use a macro lens you can shoot at F36 which helps to increase focal range. OR, using a small point and shoot (small sensor) which will give you a better focal range (but without the DSLR quality.) The smallest object I have digitized is 1.5 inches across. I took 190+ photos. None of the photos were fully in focus.

I have Helicon focus. It is great but it will distort the image as it is literally "rendering" a new image out of the photo stack. It's not accurate for photogrammetry. 

[Gianni]

Indeed, there are a number of focus stacking programs available, both commercial, as 'Helicon Focus' and 'Zerene Stacker', and free, as 'Picolay' und 'Combine ZP'.

The sticking point, however, when you use focus stacked images for photogrammetry, is perspective: The photogrammetric process requires a unique perspective for all parts of the image, which usually is not present in a focus stack, whether you capture the images with a macro rail or by adjusting the focal plane with the focal ring. In the "stacked" image with extended focal depth, the slight perspective errors are not noticable to the human eye, but they deteriorate the quality of a 3d model calculated with PhotoScan or any other photogrammetric software. The models may be acceptable for compact subjects, but for delicate subjects as insects, the results are mostly disappointing.

A way to overcome the persprcective problem, is a calibration of the camera motion used for the image capturing, together with an algorithm for the correction of the perspective change, see for this our publication zookeys.pensoft.net/articles.php?id=24584 on insect 3d modelling with PhotoScan.

Best regards,

Twister

Thanks for that in detail explanation, that makes totally sense. I haven't made it yet to read the full paper but I will denitely go through it as soon as I can. I have tried helicon on an almond and it worked well, but it's just one model I will have to see if it works on other stuff too or if it's just been a lucky one.

[Gianni]

just an idea i haven't tried, but perhaps if you used the 'station type' camera group feature of photoscan (metashape) to tell it that images from each stack are absolutely in the same location, then some of the processing inefficiency and uncertainty could be overcome.

also split any camera calibration groups so that it is able to attribute individual distortion parameters to each differently focussed image.

Nice, didn't know about that feature. Looks promising, I will test it and let you know if it worked

[Gianni]

What kind of camera are you using and how small is the object? If you use a macro lens you can shoot at F36 which helps to increase focal range. OR, using a small point and shoot (small sensor) which will give you a better focal range (but without the DSLR quality.) The smallest object I have digitized is 1.5 inches across. I took 190+ photos. None of the photos were fully in focus.

I have Helicon focus. It is great but it will distort the image as it is literally "rendering" a new image out of the photo stack. It's not accurate for photogrammetry.

I'm currently using an mirrorless sony with a macro lense. F36 gives me to much of diffraction and I really need that texture detail, so I'm trying not to go over F16.

[Mike25660]

If you are still developing your workflow for focus stacking, then you might glance over this article:

COUPLING FOCUS STACKING WITH PHOTOGRAMMETRY TO ILLUSTRATE SMALL FOSSIL TEETH, https://www.jpaleontologicaltechniques.org/pasta3/JPT%20N18/Bulletin.html

[Alexey Pasumansky]

Some additional links to materials related to focus stacking and reconstruction in PhotoScan:

https://blog.sketchfab.com/focus-stacked-reverse-lens-macro-photogrammetry/
https://www.researchgate.net/publication/304002864_SFM_TECHNIQUE_AND_FOCUS_STACKING_FOR_DIGITAL_DOCUMENTATION_OF_ARCHAEOLOGICAL_ARTIFACTS
https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W5/421/2017/isprs-archives-XLII-2-W5-421-2017.pdf