Function createDifferenceMask not working correctly?

[bmartin]

Hello,

I tried using the function createDifferenceMask to mask the WHITE parts of the attached test image named TestImage.png.  I get weird results where the mask also covers blue regions and doesn't mask all the white regions! 

To reproduce this bug, just edit testMaskSimple.py to change the location of TestImage.png to where you saved it on your system (line 13) and run the script.  You should get the same results I got (see MaskPartiallyWorkingBug.png.)

Can somebody confirm if it's a known bug, something new or expected (strange) behavior???  Is there a workaround for this?  I really need to generate good masks automatically for lots of images with white mobile structures and I'm afraid this can be a problem...

Thanks,

Bruno

[bmartin]

Hello,

I did another test using the same script but with concentric black and white ellipses.  I also got very strange (and wrong) results where the two innermost white ellipses are NOT masked...  See the attached images.

I don't understand as the createDifferenceMask function is not using a seed and should not rely on the connectivity of the pixel components as long as they meet the expected color within tolerance?

It seems a simple linear image traversal should not miss any pixel meeting the criteria.  Could it be caused of some post processing used in order to get a polygonal representation of the mask border instead of a more memory hungry binary image?

I tried using OpenCV to get the equivalent result (see the testOpenCV.py attached script) but the Python interpreter kept on telling me the image I gave to the OpenCV function was not a numpy array? 

   2018-01-11 10:54:16 Traceback (most recent call last):
   2018-01-11 10:54:16 File "D:/Corriveau/Python/testOpenCV.py", line 14, in <module>
   2018-01-11 10:54:16 gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
   2018-01-11 10:54:16 TypeError: src is not a numpy array, neither a scalar

What am I doing wrong?  How can I access the real pixel array to do some processing with OpenCV to bypass the problem with the createDifferenceMask function?

Regards,

Bruno Martin

[Alexey Pasumansky]

Hello Bruno,

Masking from the background is not using per-pixel comparison. It is also trying to form some areas during the masking process. So in some cases small unmasked areas may appear.

[bmartin]

"So in some cases small unmasked areas may appear."

In that case, I will probably need to code my own masking with OpenCV.  Could someone tell me the right way to pass the image data contained in a PhotoScan.Photo object to OpenCV?  In my last post, it seems OpenCV expects a numpy array but I don't know how to access it?

Regards,

Bruno

[Alexey Pasumansky]

Hello Bruno,

Please check the related post with the example of converting PhotoScan.Image() to numpy array:
http://www.agisoft.com/forum/index.php?topic=6345.msg30866#msg30866

[bmartin]

Hello Alexey,

I looked at your example from:

http://www.agisoft.com/forum/index.php?topic=6345.msg30866#msg30866

Code: [Select]

import PhotoScan, numpy

    camera = PhotoScan.app.document.chunk.cameras[0]
    mask = camera.mask.image()
    bytes = mask.tostring()
    pixels = numpy.fromstring(values, dtype=numpy.uint8)
    (unmasked,) = numpy.nonzero(a == 255)

In my case, I want to pass the original image to OpenCV which is RGB and not simply an unsigned char.  How can I do to get a numpy array of RGB pixels to pass to OpenCV?

Code: [Select]

cam = chunk.cameras[0]
src = cam.photo.image()
bytes = src.tostring()
pixelsRGB = numpy.fromstring(bytes, dtype=numpy.uint8) # WHAT dtype TO USE TO GET RGB PIXEL ARRAY?
gray = cv2.cvtColor(pixelsRGB , cv2.COLOR_BGR2GRAY)

Finally, once I'm finished with image processing, how do I put back my own mask from the numpy array of 0/255 I will get?

Thanks,

Bruno

[PolarNick]

You need to reshape numpy array to (height X width X numberOfChannels):

Code: [Select]

import cv2
import numpy as np

chunk = PhotoScan.app.document.chunk
camera = chunk.cameras[0]
image = camera.photo.image()

img = np.fromstring(image.tostring(), dtype=np.uint8)
assert (len(img) == image.height * image.width * image.cn)
img = img.reshape(image.height, image.width, image.cn)

# Please, note that OpenCV assume that image is BGR, while PhotoScan provides RGB, you can fix this with numpy:
#     img = img[:, :, ::-1]
# Or with OpenCV:
#     img = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)

# You can show image in window with OpenCV:
#     cv2.imshow("window with img", img)
#     cv2.destroyWindow("window with img")


[bmartin]

Thanks PolarNick!

That was exactly what I needed!  No need to use the disk as a scratchpad to get the representation right between Photoscan and OpenCV.

This trick, and the other way around, should be part of the Python API documentation!

[tkwasnitschka]

Maybe this is really stuped (not for me):
could you please show "the other way around" = how to write a numpy array back to a photoscan image? I just dont get it.
Thanks!
Tom