Forum

Author Topic: Illumination artifact when texturing shiny surfaces  (Read 2871 times)

Twister

  • Newbie
  • *
  • Posts: 24
    • View Profile
Illumination artifact when texturing shiny surfaces
« on: July 06, 2018, 04:42:49 PM »
In an insect digitizing project (see: zookeys.pensoft.net/article/24584/), we observe illumination artifacts when texturing the smooth and shiny, but otherwise dark surface of some beetles, e. g. black scarabaeidae. As an example, the attached figure shows three photos of a beetle, taken from viewing angles differing by about 10°. On each of them, a mirror image of the dark circular “spyhole” for the camera in the center of the illumination dome is visible, marked with A, B, C. These structures (and many more from the other photos) appear also on the texturized model (right) and cause false and misleading surface texture. For explanation, we assume that PhotoScan, when texturing a piece of surface, typically uses the photo of the camera closest to the normal viewing direction on that surface. This leeds to a multiple appearance of spyhole images or other illumination structures connected with the camera, as ringlights. N.B., the artifact only affects texture, not the model.

Does anybody have experience with such texture artifacts? Can they be reduced with optical means (polarization filters)? Is there a way to preclude the “normal viewing direction” from being used for texturing?

Thank you!

Dave Martin

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: Illumination artifact when texturing shiny surfaces
« Reply #1 on: July 07, 2018, 12:42:28 PM »
Twister,

Looking at the photographs of your splendid rig, it looks like the camera aperture in your illuminating sub-hemisphere is potentially occluding some tens of degrees without illumination.

A few thoughts, without knowing the practicality in terms of your rig:
  • Can you move the illumination surface further back towards the camera, thus reducing the occluded angle?
  • If the dome has to stay where it is, can you fire in some extra light from back near the camera? - e.g. a ringflash-type light close around the lens? - or lights 'over the camera shoulder' but several around the camera/lens?
  • Can you dispense with the front dome and put the whole rig in a light tent?
  • In addition to the images you've posted (ABC) it might help if you could post both full-frame and crops from a couple of source images - preferably with the subject at different / extreme orientations; but I wonder if it might be possible to mask the area in question, as whilst the projection will change, I guess it will always be centre-frame. If that affects the model generation, you could generate the model with the existing images and then substitute the masked ones when generating the texture.
Dave
Edit - expand reference to ABC images
« Last Edit: July 07, 2018, 12:58:35 PM by Dave Martin »

mala

  • Full Member
  • ***
  • Posts: 109
    • View Profile
Re: Illumination artifact when texturing shiny surfaces
« Reply #2 on: July 07, 2018, 04:28:03 PM »
Great little rig  :)
Good to see someone do this, I played around with macro scanning a few years ago but never had the free time to put into it.

I would have thought that polarisation would help your project quite a bit, as a good polarised lighting rig combined with a polariser on the camera would give the ability to capture both diffuse and specular images.

Depending on the quality of the polarisers and position of the lights & camera you should in theory be able to elminate the specular reflection entirely by using cross polarisation.

If you have the abilty to "toggle" the polarisation angle between 0º and 90º, which as your subject does not move (not alive) you would not have to do very fast, perhaps just one polariser in front of the camera that can be rotated from 0º to 90º.
Then you lights only need one polarisation state ( i.e fixed linear polarising filters).

The issue I can see with all the above is that I suspect that your lighting dome and the position of the LED strips will change the angle of polarisation coming from the lights.
Ideally you would need a sphere of evenly spaced lights around the subject all pointing directly at it.

Having said all that I'm sure it's worth just putting a polarising filter in front of your camera and seeing if it helps fix the issue of the spyhole reflection.

Cheers,
mala

Twister

  • Newbie
  • *
  • Posts: 24
    • View Profile
Re: Illumination artifact when texturing shiny surfaces
« Reply #3 on: July 07, 2018, 05:51:40 PM »
Dear Dave and mala,

Thank you for your helpful and inspiring replies on my post.

@Dave

- "...  it looks like the camera aperture in your illuminating sub-hemisphere is potentially occluding some tens of degrees without illumination."
The minimum diameter of the spyhole results from the insect diameter (here: 8 mm), the distance of the spyhole from the insect (60 mm = radius of the illumination dome) and the distance of the spyhole from the lens entrance pupil in the far position of the camera (32 mm). In the case shown, the spyhole diameter was 3 mm. Therefore, an angle of (3/60) rad or 2.9° is occluded from illumination, whereas the angular distance of neighboring photos is 10°.

- "Can you move the illumination surface further back towards the camera, thus reducing the occluded angle?"
That would indeed reduce the size of the artifacts, but not eliminate the artifacts completely. In addition, the radius of the illumination dome is constrained by geometric and optical considerations.

- "If the dome has to stay where it is, can you fire in some extra light from back near the camera? - e.g. a ringflash-type light close around the lens? - or lights 'over the camera shoulder' but several around the camera/lens?"
Indead, I have also considered to "fill" the hole with some extra light, perhaps by use of a beamsplitter between the lens and the spyhole. But that looked to me as a challenging modification with a questionable result. Any light sources around the spyhole (as a ringlight) would cause additional reflections / artifacts.

- "Can you dispense with the front dome and put the whole rig in a light tent?
I fear that this would cause mirror images of the camera and parts of the whole rig, with even worse artifacts.

- "... I wonder if it might be possible to mask the area in question, as whilst the projection will change, I guess it will always be centre-frame. If that affects the model generation, you could generate the model with the existing images and then substitute the masked ones when generating the texture."
That is an excellent idea! Why did I not hit on this before? Indeed, it is possible to use masked photos for texturing, thus excluding the masked regions from being used for texturing, see the PhotoScan Manual:
"During texture atlas generation (for single mesh model and tiled model), masked areas on the photos
are not used for texturing. Masking areas on the photos that are occluded by outliers or obstacles helps
to prevent the 'ghosting' effect on the resulting texture atlas." The masking needs to be done manually, but as only about 30 of a total of 398 photos show the spyhole reflection, the effort is quite tolerable. I will try that next week and report on
my results.

@mala

- "I would have thought that polarisation would help your project quite a bit, as a good polarised lighting rig combined with a polariser on the camera would give the ability to capture both diffuse and specular images.
... Depending on the quality of the polarisers and position of the lights & camera you should in theory be able to elminate the specular reflection entirely by using cross polarisation.
... If you have the abilty to "toggle" the polarisation angle between 0º and 90º, which as your subject does not move (not alive) you would not have to do very fast, perhaps just one polariser in front of the camera that can be rotated from 0º to 90º."
Indeed, the need to measure with two orthogonal polarisations is a problem. Although our subjects are dead and pinned, that would duplicate the scanning time. As we are aimed ad mass digitization of insects, "time is money". Nevertheless, I will test the use of different polarization filters (also circular).

Thank you again, best regards and a nice weekend

Twister

Dave Martin

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: Illumination artifact when texturing shiny surfaces
« Reply #4 on: July 08, 2018, 11:57:10 AM »
Twister - thanks very much for such detailed and informative feedback, and look forward to hearing how you get on. With respect to masking, if it works with masking individual images, I would wonder if in fact you might get away with defining one mask and using that throughout the project, in fact for all images taken of similar sized objects (might need several to cover different diameter artefacts as the size of object being imaged changes).
Dave

Twister

  • Newbie
  • *
  • Posts: 24
    • View Profile
Re: Illumination artifact when texturing shiny surfaces - problem solved
« Reply #5 on: July 10, 2018, 11:39:29 AM »
Hi,
masking the spyhole reflections in the photos and using the masked photos for texturing completely removes the artifact - at the expense of only a few minutes extra time. See the attached images.
Thank you again, Dave and mala, for your helpful comments
Twister

Dave Martin

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: Illumination artifact when texturing shiny surfaces
« Reply #6 on: July 11, 2018, 06:30:17 PM »
Twister,

That's great, and thanks for reporting back. Your R&D with such a rig is advancing work in this area - great to see.

To save time manually masking individual images (even if only a few dozen - you still need to identify which ones to mask) I still wonder if you could define one mask to cover the largest version of the spyhole artefact - and then just apply that to all images taken?

Dave

Twister

  • Newbie
  • *
  • Posts: 24
    • View Profile
Re: Illumination artifact when texturing shiny surfaces
« Reply #7 on: July 11, 2018, 10:34:32 PM »
Dave,

thank you for your appreciating words, it does good to read that.

Unfortunatedly, the site of the artifact on the photos is not necessarily the center of the images. The artifact appears whereever the camera looks normally on a smooth and shining peace of surface. It is even possible that several mirror images of the spyhole appear on the same photo - somewhat ressembling a house of mirrors. We hope to gain some experience now with different types of insect bodies, perhaps some day one could cope with the spyhole artifact using some kind of AI - anyway, an exciting problem!

Twister