Forum

Author Topic: RAW, JPG and different types of noise  (Read 27593 times)

RalfH

  • Sr. Member
  • ****
  • Posts: 344
    • View Profile
RAW, JPG and different types of noise
« on: January 22, 2013, 05:55:10 PM »
As I occasionally see the opinion (in this forum and elsewhere) that RAW images are always superior to JPG images, I want to clarify a few issues here. Generally, digital photographs are affected by different types of noise which ultimately have relevance for using them for 3D modelling.

Sensor noise is caused by random fluctuations in the reaction of each sensor pixel to a given amount of photons. Thermal noise is responsible for a good part of the total sensor noise, so in theory cooling the sensor would help. Besides this usually not being feasible, my guess would be that we could at best reduce sensor noise by a few per cent within the range of temperatures in which our cameras are working. Increasing the sensitivity of the sensor (i.e., higher ISO number) means that sensor output for a given number of photons will be higher. Background noise remaining more or less equal, this of course means that this will reduce the signal to noise ratio (SNR).

Quantisation noise is caused by the mapping of an analog signal onto a digital range of values or by the mapping of a larger onto a smaller digital range of values. Primary quantisation in digital consumer cameras is usually 10 or 12 bit which means that the minimum brightness difference which can be detected (the least significant bit, LSB) is 1/1024 or 1/4096 of the recorded brighness range. The noise we see in a raw image is a combination of sensor noise and primary quantisation noise. Primary quantisation is usually not a big source of noise; it is much smaller then sensor noise. However, when the raw 10 or 12 bit data are re-mapped onto the 8 bit (values 0 to 255) scale used in JPG, there are again quanitisation errors. Their relative importance also depends on whether you are looking at darker or brighter areas of an image (or underexposed/well-exposed images): a change in the LSB in a dark part of an image (let’s say, average value of 10) is equivalent to a 10% change in brightness while a change in the LSB in a medium-bright area of an image (let’s say, average value of 127) is equivalent to less then 1% change in brigthness. Being able to detect small changes in brightness is of course important to be able to detect subtle features in an image (be it visually or by an algorithm).

When the resulting image with 8 bit colour depth is saved as a JPG by the camera, additional noise (or better, artefacts) are introduced, because JPG uses a lossy data compression. This can be best appreciated when splitting a colour JPG into hue, saturation and lightness (HSL) or hue, saturation and value (HSV) channels: the lightness or value channels look quite intact unless you use really strong JPG compression settings, but artefacts can often easily be seen in the hue and saturation channels. [At this point, it would be interesting to find out whether PhotoScan works with the RGB channels, with lightness or something else.]

In consumer cameras, sensor noise is often very noticable when looking at a raw image at full zoom. And because most consumers want to see nice clear pictures without a lot of noise, in-camera noise reduction is performed before converting the raw images to 8 bit colour depth JPG files. Depending on how much noise is produced by a given sensor, noise reduction will usually be more or less severe. I compared the low-cost Canon A3000IS with the more pricey Canon G12, and found that the raw images saved by the A3000IS had much more sensor noise than the raw images saved by the G12 and that the noise reduction in the A3000IS was much more severe than in the G12. Of course, noise reduction also removes image detail. Therefore, even with the same JPG compression strength a A3000IS JPG of the same object would be a smaller file than a G12 JPG (both are 10 MPIX cameras).

Now, what does all this tell us about using RAW or JPG in PhotoScan?

(a) The lowest possible ISO value should be used to reduce sensor noise.
(b) Underexposure and dark areas in images should be avoided because sensitivity to subtle brightness differences (and, as a result, feature detection) in dark areas is poor.
(c) RAW images suffer much less from quantisation noise and not at all from loss of image detail due to noise removal  and compression artefacts, but they contain all sensor noise (which can be severe). JPG images, on the other hand, often have less (visible) noise than raw images because of the in-camera noise removal and are much smaller files which are processed much faster. If your camera has very low sensor noise, RAW will be the better choice. For many consumer cameras, JPG may be the better choice unless you are able to apply a better sensor noise removal than the camera itself.
(d) In the end, it all comes down to SNR. You will always have some noise, and you should not only try to reduce noise but also to enhance the signal: a good lens, perfect focus, and an illumination which brings out as much fine detail as possible while not producing underexposed areas. In many cases, more photos taken closer to the subject will also help a lot.

Working on these points improves PhotoScan results much more than simply using RAW instead of JPG.
« Last Edit: January 22, 2013, 08:05:20 PM by RalfH »

Infinite

  • Sr. Member
  • ****
  • Posts: 366
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #1 on: January 22, 2013, 08:35:13 PM »
Very useful post! and mimics the tests I've made with RAW and JPG using Canon 550D and 600D's anyway. I have seen no gain with using RAW what so ever. Only in the final color output but not geometry build. As you mentioned under exposing and over exposing are both bad, you need that sweet spot. Dark regions will come out noisy, so will over bright.
_______________________________________________
I N F I N I T E
www.ir-ltd.net

RalfH

  • Sr. Member
  • ****
  • Posts: 344
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #2 on: January 22, 2013, 08:44:31 PM »
Thanks. Overexposure is different from underexposure in that the problem is not quantisation error but values exceeding the range of values (i.e., everything that is brighter than whatever is equivalent to a value of 255 will also have the value 255).

andy_s

  • Full Member
  • ***
  • Posts: 147
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #3 on: March 03, 2013, 09:31:42 PM »
RalfH - that is excellent information, thanks for the post.

gregoconn

  • Newbie
  • *
  • Posts: 5
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #4 on: March 04, 2013, 12:16:43 PM »
You just saved me hours if not weeks of research. Thankyou so much!

Patribus

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #5 on: June 10, 2014, 03:00:38 PM »
Thumb up! Very good information! Thanks.

Marcel

  • Sr. Member
  • ****
  • Posts: 309
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #6 on: June 10, 2014, 03:43:26 PM »
In a studio setup shooting JPG is probably more practical ( because you can control the lighting), but for outdoor work I would be too afraid to end up with badly exposed JPGs. Being able to rescue highlights and boost shadows is a real lifesaver (next to setting the whitebalance). Working with RAW files is an extra step that takes up more time, but the advantages are worth it in my opinion.

Mfranquelo

  • Full Member
  • ***
  • Posts: 171
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #7 on: June 10, 2014, 05:31:59 PM »
Great post, *claps*  :)

ozbigben

  • Jr. Member
  • **
  • Posts: 64
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #8 on: June 11, 2014, 07:12:06 AM »
On the exposure side of things, has anyone done any experiments making multiple point clouds with different exposures and then merging them? I'm toying with a couple of ideas for a building and don't necessarily want to go down the full HDR/tone mapping path which would require precise image alignment of exposure groups during shooting.

Marcel

  • Sr. Member
  • ****
  • Posts: 309
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #9 on: June 11, 2014, 02:53:52 PM »
On the exposure side of things, has anyone done any experiments making multiple point clouds with different exposures and then merging them? I'm toying with a couple of ideas for a building and don't necessarily want to go down the full HDR/tone mapping path which would require precise image alignment of exposure groups during shooting.

I know what you mean with full HDR being a problem, I tried it and the results were worse than single shots (taken from the same set). Photoscan just craves really sharp pixels.

I don't think multiple point clouds shot at different exposures will work, though it's an interesting idea. What I suspect is that the overexposed and underexposed parts will generate a lot of noise in the scan, and there is no way to filter out noise before merging chunks. A properly exposed RAW photo is still the best option in my opinion (although I also run into situations where I don't have enough dynamic range).

frank.stremke

  • Full Member
  • ***
  • Posts: 206
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #10 on: July 16, 2014, 03:48:42 PM »
stupid question on the side how do i introduce raw images into photoscan?
i have NEF files and usualy i use rawtherapee to develop them but then i have jpgs again so which format do i need to make use of the 16bit features
is it only tiff ?
because my 20mb nef file turns into a 128mb tiff file which is very large
so what would a good procedure be?
frank

David Cockey

  • Full Member
  • ***
  • Posts: 209
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #11 on: July 16, 2014, 11:13:38 PM »
Frank, try developing the raw files as JPEGS with 100% quality.

I used to always use 16 bit TIFF images as input to PhotoScan. The images were obtained by "developing" RAW files. But after some experimentation and comparisons I switched to using JPEG images with "quality" of 100%. These images are also obtained by "developing" RAW files, and are not the JPEG images output by the camera. My comparisons showed that using 100% quality JPEG images results in a decrease of a few percent in the number of sparse points, and does not have a noticeable impact on the PhotoScan results. JPEG images with lower quality settings are smaller but result in fewer sparse points and sufficiently low settings of quality resulted in noticeable impact on PhotoScan results.

All JPEG images are compressed, and as RalfH noted above the compression can result in artifacts and loss of data. But not all JPEG images are the same. The algorithm used for generating JPEGs has an input parameter called "quality" which can range up to 100%. Higher quality values result in fewer artifacts, less loss of detail but larger files. Generally the quality setting used internally in a camera to generate JPEGs is not available but is selected to keep file sizes relatively small.

16 bit TIFFs from raw files from my Canon T1i/500D are 86.3 MB. JPEGs with quality set to 100% range from 10 MB to 21 MB depending on image quality. (The corresponding JPEGs direct from the camera with unknown "quality" setting range from 4 MB to 6 MB.)

Creating the JPEGs by "developing" the raw files allows me to correct for chromatic aberration, adjust the exposure and contrast if needed while avoiding clipping in the areas of interest, and select the JPEG quality setting.

JohnyJoe

  • Jr. Member
  • **
  • Posts: 62
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #12 on: March 02, 2015, 04:29:31 AM »
BTW its an older topic but still... did the smaller size of jpgs also results in faster processing in photoscan? (like quicker photo allign or dense generation)?

And what about lossless compresion of tiffs (lzw) does it contrary slow down processing in photoscan (due to the need to "uncompress" the files when (before) reading them?

bigben

  • Sr. Member
  • ****
  • Posts: 406
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #13 on: March 02, 2015, 07:00:31 AM »
Compression doesn't affect the processing speed other than reading/decompressing the images.  It's still the same number of pixels to process.   One thing not mentioned in this is the relative size of noise to the detail in the image.  Increase the sensor resolution and you decrease the impact of noise.

JaneT

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: RAW, JPG and different types of noise
« Reply #14 on: April 01, 2017, 02:55:36 PM »
I use Photo Mechanic to import and cull my photos before pulling them into Lightroom. Having the JPEG [ https://www.cleverfiles.com/howto/what-is-jpg.html ]means that things go at a blistering speed and I can be done culling 500 RAW+JPEG photos in the time that Lightroom takes to load and generate 50 RAW photos. I think that PM will be just as fast using the JPEG previews in the RAW files, but I haven't tried it, and I suspect the quality won't be there, so it won't help in choosing the best of multiple similar photographs.
I also like the option of having the simulations SOOC while still keeping the RAW file if I feel the need to make adjustments (usually WB and exposure, to rescue blown out photo or one taken in terrible light).
« Last Edit: April 01, 2017, 02:57:07 PM by JaneT »