Noise Reduction in Digital Images
Lana J. Jobes

Digital cameras have become very popular over the last several years for both professional and personal use.⁽¹⁾   Under most conditions, digital cameras are an efficient, convenient way to take pictures.⁽¹⁾  If, however, an exposure time longer than 1/4 second is desired, the sensor array in the digital camera introduces undesirable noise into the image.⁽²⁾  Figures 1 and 2 illustrate the additive noise seen in an image acquired using an exposure time of 30 seconds.

This research project has resulted in the development of a noise reduction technique which can be used to decrease the visible noise in images taken using exposure times greater than 1/4 second.  Using this noise reduction technique, people who have taken images in low-light situations are able to take advantage of the many benefits of digital photography.  Digital photography allows nearly instantaneous access to the images;  image dissemination is accomplished quickly across the Internet; and image modifications can be done quickly.  In addition to saving money by eliminating the need for film and film developing, some digital cameras have an LCD display panel for instant image review.⁽³⁾  This feature eliminates the need to take multiple pictures of an event to ensure its capture.
 
 
 
 

Zamora and Mitra used color space transformations in their image compression research involving lossless coding techniques.⁽⁴⁾    In their research they transformed test images into CIE L*a*b* color space and applied a different compression ratio to each channel independently.  Their research concluded that the compression ratio achievable while maintaining acceptable image quality was significantly higher in CIE L*a*b* color space than in RGB space.

In research performed by Zheng, Valavanis, and Gauch, artificial noise was randomly added to RGB images.⁽⁵⁾    Images were transformed  from RGB space to both HIS and CIE L*a*b* color space.  Noise filters were applied to the original RGB images and independently to the channels of the transformed images.  A mean squared error measurement was used for comparison of the results.  They concluded that no significant image quality improvements were obtained by using the color space transformation before filtering.

In the current research project the noise of CCDs was characterized under real conditions.  The additive noise found in images acquired using long exposure times is caused by thermal excitation in the sensor array.  This type of noise may be different than noise added randomly to the image as in the research performed by Zheng, et al. Additionally, the analysis done in that study used the mean squared error measurement for determination of results; the current research project results will be determined based on the perception of the noise by the human visual system.

CCD sensors consist of thousands of elements grouped in either a linear or matrix array.  The elements on the CCD array are coated alternately with red, green, and blue filter material.⁽⁶⁾   In these cameras, typically 25% of the elements are coated with red filter material, 25% with blue, and 50% with green.⁽⁷⁾   Having twice as many green elements as red and blue gives the image the appearance of being more detailed because of the human eyeís enhanced sensitivity to green. ⁽⁶⁾   When light hits these elements, the CCD elements build up a charge proportional to the amount of light hitting them.  The charge is transferred to the output portion of the CCD and converted to a signal which is digitized and stored in memory. ⁽³⁾

CCDs are more responsive to the red portion of the spectrum than the blue.  To compensate for this, differential gain is included for each channel.  Unfortunately, this leads to more  noise in the channels with the higher gain.  The CCD also responds strongly to wavelengths in the infrared portion of the spectrum.  An infrared filter covers the CCD to avoid detection of these values. ⁽⁷⁾
 

When light hits the elements of the CCD each element builds up a charge proportional to the amount of light reaching it.  Using long exposure times, thermal excitation of the CCD array causes an additional charge to build in some elements.  While the additional signal is random, it generally increases with exposure time, while the location of noisy pixels remains relatively stable.

Figures 3 and 4 show two different 'dark' images taken four months apart using a 30 second exposure time.  In each of these images, taken using a Kodak DCS 315 digital camera, the lens cap was left on the camera, isolating the additive noise.  It is clear that the noise occurs in the same pixels of each image. It is also apparent that the level of noise differs among the color channels of the image.  There is a band-to-band difference in the total number of noise affected pixels and in the size of groups of noisy pixels.  The noise in the blue band appears the worst, followed by the red band.
 

Figure 3:  30 Second Image Acquired 12/98         Figure 4:  30 Second Image Acquired 4/99

The fact that the location of the noise doesnít change allowed the identification of pixels that are typically noisy for a particular CCD array.  To do this, dark images were obtained as shown in Figures 3 and 4.  A threshold for each band of the dark image was determined.  Any pixel that had a digital count above the threshold was considered a ëhotí pixel.  This was done independently for each color band of the image because the amount and intensity of the noise differs for each band.

The trade-off between noise reduction and loss of detail in the image needed to be considered in determining thresholds.  While the objective is to filter out all of the noise, if too large of a percentage of the pixels are filtered, image artifacts would be evident.  The histogram function in Adobe Photoshop was chosen to determine the threshold for each band.  The thresholds chosen are listed in Table I.
 

             Table I:  Thresholds

Noise Reduction Methods

Pixel Substitution
The first method utilized to reduce the noise consisted of substituting one pixel value for another.  A program was written to scan through a dark image looking for pixels with values above the threshold input to the program.  These pixels will be referred to as 'hot' pixels.  When a hot pixel was found, the program located the lowest pixel value within two pixels from the hot pixel.  This pixel was assumed to not typically contain a large noise value and will be referred to as a ëcoldí pixel in this thesis.   The locations of each of the hot and cold pixels were recorded in a data file used when processing images.

An image was taken with the digital camera characterized earlier.  A program was run which opened the image file and the data file containing the hot and cold pixel locations.  For each of the hot pixel locations, the digital count in that location was replaced by the value in the corresponding cold pixel.

Figures 5 and 6 show the results of the substitution method.  A reduction in the amount of noise is evident in the processed image.  However, upon close examination of the image, artifacts are seen in the dark areas of the image.
 
 

Additionally, an artifact was discovered at many of the edges in the image.  This artifact, shown in Figure 8,  was caused by having a hot pixel in the dark part of the image and the corresponding cold pixel in the lighter area.
 
 

While some reduction in the noise has been achieved, the presence of this edge artifact was unacceptable, so another approach had to found.
 
 

Median Filtering in RGB Space
Another approach to noise removal in digital images is median filtering.  The disadvantage of median filtering an entire image is loss of image sharpness.  Figure 7 is an image acquired using an exposure time of thirty seconds. Figure 10 shows the effect of median filtering the entire image using a filter radius of five pixels.  The loss of sharpness is apparent.  Note the difference in the ability to resolve the text on the books.  Even the details of the camera, bag, and the grain of the wood are noticeably blurred.
 
 

Making use of the thresholds obtained during Noise Characterization allowed us to apply a median filter to the pixels which are typically noisy.  Once the threshold for each band of the image was determined, a binary mask image was created.  The mask image contained a value of one for each pixel that was above the threshold in the original dark image and zero elsewhere.  The blue channel of the mask image contained the hot pixel locations found in the blue channel of the dark image.  The same procedure was followed for the red and green channels. This mask was an important component in the filtering process.  The Adobe Photoshop Adjust Threshold command was used for creating the mask images.

The filter radius was determined by observation of the size of the noise clusters in a dark image like those in Figures 3 and 4.  The radius for each band needed to be large enough to provide a median value that was within acceptable limits.  Like the threshold discussed in Noise Characterization, the filter radius needed to be determined separately for each color band.

Adobe Photoshop was used to filter each color band of the image.  A band was selected, and using the Load Selection command, the corresponding band of the mask image was used to select the pixels which were noisy in the original dark image.  The Adobe Photoshop median filter was then used to filter the band.  This process was performed for each band of the image.

Figure 11 contains the same image seen in Figure 5.   In Figure 12, the noise has been removed by selective median filtering.  Only the noisy pixels identified by the mask image were filtered using the filter radius appropriate for each color band.
 
 

A substantial improvement is seen in the filtered image.  Figure 14 shows the edge artifact found using the pixel substitution method.  In Figure 15, the same area of the image is shown, this time after selective median filtering.  The artifact is not present.
 
 

While selective median filtering produced images with a decreased amount of noise, some artifacts were seen.  It was decided to investigate using CIE L*a*b* space for the filtering.
 

Median Filtering in CIE L*a*b* Space
Both the substitution method and the preliminary median filtering for this research were conducted using RGB color space.  A color space transformation to CIE L*a*b* space prior to filtering was used to further optimize our technique. CIE L*a*b* is a uniform color space which is modeled after the human visual system.  The L* channel contains the luminance information for the image, while the a* and b* channels contain the chrominance information. ⁽⁸⁾     The opponent red-green information is contained in the a* channel and the b* channel contains the opponent yellow-blue information.

The perception of sharpness for the human visual system is based mostly on the information contained in the luminance (L*) channel.  The amount of filtering for that channel was minimized. The L* channel had a much lower noise level than the a* and b* channels and the noise present was mostly in isolated pixels.

Most of the additive noise was observed in the chrominance (a* and b*) channels.  Both channels exhibited noise clusters; these clusters were larger and more frequent in the b* channel.  Because these channels donít contribute significantly to the perceived sharpness, they were filtered aggressively.  The noise in the b* channel was so severe that the entire channel was filtered.  The a* and b* channels are opponent channels; in a histogram of the channel, the peak of the distribution is found in the middle with tails on each side.  Thresholds were found for each side of the peak for the a* channel (no thresholds were used for the b* channel since the entire channel was filtered).  A mask image was created for each band and median filtering performed, as was done with the RGB median filtering.  The parameters used in the median filtering for CIE L*a*b* space are listed in Table II.  Once the filtering had been performed on each band, the image was converted back to RGB space.
 

            Table II:  Thresholds

Selectively median filtering the image in Figure 16 produced the image shown in Figure 17.  The appearance of the noise in the filtered image has diminished substantially.   This can be easily seen in the dark areas of the image.
 
 

Figures 18 and 19 offer a comparison between the image filtered in RGB space and the image filtered in CIE L*a*b* space.    The image filtered in RGB space appears to be less sharp, but at the magnification of these two images, it is difficult to see much difference.  Magnified versions of the images are available by clicking on them.
 
 

Figures 20 and 21, however, show the difference well.  The text in Figure 21 is sharper that in Figure 20.  The CIE L*a*b* filtered image also appears to have less artifacts and remaining noise.  The artifacts in the RGB filtered image are very apparent in the text and the dark line of the book.
 
 

Figure 22 shows an image acquired using an exposure time of 4 seconds.  No noise filtering has been done to the image.  Figure 24 contains an image acquired using an exposure time of 30 seconds and selectively median filtering the noise in CIE L*a*b* space.  Again, since the difference is difficult to see at this resolution, magnified versions of the images are available by clicking the images.  A comparison of the two images shows that by selectively median filtering the 30 second image, it is comparable to the image taken using a 4 second exposure.
 
 

This senior project has investigated techniques to reduce the amount of noise in digital images acquired using long exposure times.  Pixel substitution was tried, but caused artifacts in the edge and dark areas which were unacceptable.  Selective median filtering in RGB color space produced much better results.  Some artifacts and loss of sharpness were seen, however.  Filtering in CIE L*a*b* space produced the best results without an objectionable amount of image degradation.

This can be done by completing the follows steps:

1. Capture a dark image using the digital camera.
2. Convert the dark image to CIE L*a*b* space.
3. Determine thresholds and a filter radius for each band of the image.
4. Create a mask image for the camera.  For each pixel that was above the threshold in the dark image, set the corresponding mask pixel value to one.  For pixels below the threshold in the dark image, the mask value is set to zero.
5. Use Adobe Photoshop to open the image to be filtered.
6. For each band of the image, select the pixels that are set to one in the mask image (using the Load Selection command).
7. Apply the Adobe Photoshop median filter to the selected pixels.
8. Convert the image back to RGB space.

This research project has shown that the amount of noise present in images acquired using a long exposure time can be significantly decreased by selectively median filtering the images in CIE L*a*b* space.

This technique has been found to decrease the amount of additive noise without apparent loss of sharpness.  This will enable not only the practical use of longer exposure times, but higher equivalent ISO ratings, as well.