Specular surface detection (or pixel intensity maps?)

[andyroo]

This is half-question/half-request, because I only somewhat grasp the process of structure-from-motion.

Most of my work is around water, and specular reflections from water are always requiring time-consuming editing or masking. In LiDAR data the dramatic difference in return intensity of water allows easy semi-automated masking. I was wondering if the pixel intensities (and uniformity of color) could be used to automate specular surface detection and mask it from processing in SfM - producing a sort of "intensity map" from the orthoimagery.

I don't know if it would make more sense to mask it in the image processing step or in a pre-processing step (like manual masking) but it would be nice to be able to exclude returns from the water surface from surface processing, while maintaining those pixels in orthoimagery.

Of course an even better solution would be to detect and model the specular surface, rather than masking it. I've been looking for work done on that topic, and this is the best reference I've found:

http://www.cs.columbia.edu/CAVE/projects/spec_stereo/

Of course, I recognize that water is not just a specular reflector, but also at times a transparent refractor, and like I said, there's a lot about SfM I don't understand, but I would appreciate any insight into the problem of mapping water areas. And of course I would REALLY appreciate any tools that were developed to deal with the issue (or anyone's insight into how they do it).

Andy

[FoodMan]

maybe another help from this paper...?

http://research.edm.uhasselt.be/tmertens/papers/meso_cvpr2008.pdf

f/

[andyroo]

Very interesting paper. It was unclear to me from reading it whether you would need different light angles on the same subject or only different viewing angles.

Also unclear how well it would work on running water, which is changing from image to image. But if it did work, it would be cool - it could be the "SAUS" (structure acquisition using specularities) that makes SfM even tastier!

I am working right now on postprocessing artifacts from specular surfaces on a DSM, and so far ruggedness and curvature look like they'll be big winners in helping to mask and reprocess the specular surface artifacts.

[RalfH]

Foodman, thanks for the interesting link! This looks like something inbetween shape-from-shading and structured light scanning. Finally someone is dealing sucessfully with specular surfaces. Would be very interesting to see this as a ready-to-use software!