Scene linear color spaces are handled incorrectly

[jedfrechette]

I apologize in advance for the long post, but color space issues are always confusing so I'm trying to be as detailed as possible. All testing was on done Windows 7 with version 0.9.1 build 1714.

PhotoScan currently applies some undocumented, lossy, colorspace transformation to OpenEXR images. This makes it almost useless for generating HDR textures. To illustrate the problem I've provided several sample files at:

http://preview.tinyurl.com/n66sgfa

DSC_0721_lnh.exr (http://preview.tinyurl.com/l8zl78l) is the original scene linear source file that has been normalized following the standard convention where an 18% gray card has a luminance value of 0.18. Note that the brightest pixels in the image occur in the specular reflections on the upper left corner of the ColorChecker's frame and have luminances of ~1.4.

ps_color_test.psz (http://preview.tinyurl.com/lxo2e47) is a minimal PhotoScan project where all that has been done is loading DSC_0721_lnh.exr.

export_image.py (http://preview.tinyurl.com/luwp5ov) is a python script that, when run from within the above project, takes the source image and immediately saves it to DSC_7021_copied.exr then undistorts it and saves it to DSC_7021_undistorted.exr. As expected these two output images are very similar with the mean difference between pixels being ~0.002. However, They are vastly different from the source image.

The image at the bottom of this post shows the difference between the source and output images after conversion to sRGB for display using oiiotool:

Code: [Select]

oiiotool DSC_7021_lnh.exr --tocolorspace sRGB -o DSC_7021_srgb8.jpg
oiiotool DSC_7021_copied.exr --tocolorspace sRGB -o DSC_7021_copied_srgb8.jpg
Note how closely the pixel values of the Neutral 5 patch in the source image match the nominal sRGB values provided by x-rite (http://preview.tinyurl.com/kh5snvc) and how much different they are in PhotoScan's version.

There are at least two major problems with the EXR files generate by PhotoScan.

• The highlights have been clipped. The output file is clipped to the range 0-1, however, even pixels that were well below 1 in the original image, for example the white strap behind the ColorChecker, have also been clipped.
• An undesired gamma correction of ~2.2 seems to have been applied at some point during PhotoScan's color pipeline as I can get closer to the correct colors by applying the inverse (0.455). This would be less of an issue if it weren't for #1. As long as the transformation was known it could be losslessly reversed if the data hadn't been clipped.

There are also a couple of more minor issues with the EXR output. First the original 16-bit half float image was promoted to a 32-bit float, which is probably not needed for most image data. If so much data had not been destroyed during the color conversions the size of the output image (61 MB) would have nearly doubled relative to the source image (37 MB). Finally, all of the metadata in the original EXR was destroyed, although this is probably less important in the typical workflow where PhotoScan is creating an image from scratch.

It would be great if Agisoft can fix these issues, ideally by letting a well tested third party library such as OpenColorIO handle color conversions.

[Alexey Pasumansky]

Hello Jed,

Thank you for reporting.

The problem you have found is caused by the fact that camera.image() returns 8-bit image - you can check it in Python console, when just type cam.image().
To open HDR image format properly you should use the following (using your code for cam variable):
image = PhotoScan.Image()
image.load(cam.path, 0, "F32")

Now when you use image.save(path) you'll see that colors are the same. Gamma correction is applied only for visualization purposes.
For image.undisort you also need to use F32 and not U8 image.

As for the output file size it could be larger than original file size, since compression level currently can not be controlled manually inside PhotoScan.

[jedfrechette]

Hi Alexey,

Thanks for the quick response and instantiating the Image class manually as you show does allow me to correctly copy the image. Hopefully the API documentation can make it more clear that this is the intended usage.

Unfortunately, this exposes another problem.

Code: [Select]

image.load(cam.path, 0, 'F32')
produces a 4 channel image, even though the source image only has three channels. I can save this image to disk as an EXR and strip the alpha channel manually resulting in an image that is identical to the source image. Attempting to call image.undistort, however, throws an exception because image doesn't have 3 channels.

I've updated the script to show this behavior.

Thanks,

[Alexey Pasumansky]

Hello Jed,

We'll try to fix image.undistort feature in the next update, I forgot that it works now only with 8-bit images. Sorry for the inconvenience.