Space of Spectral Sensitivity Functions for Digital Color
Cameras |
 | |
Camera spectral sensitivity functions relate scene radiance with captured RGB
triplets. They are important for many computer vision tasks that use color
information, such as spectral imaging, color rendering, and color constancy.
In this paper, we aim to explore the space of spectral sensitivity
functions for digital color cameras. After collecting a database of 28
cameras covering a variety of types, we find this space convex and
two-dimensional. Based on this statistical model, we propose two methods to
recover camera spectral sensitivities using regular reflective color targets
(e.g., color checker) from a single image with and without knowing the
illumination. We show the proposed model is more accurate and robust for
estimating camera spectral sensitivities than other basis functions. We also
show two applications for the recovery of camera spectral sensitivities --
simulation of color rendering for cameras and computational color
constancy. |
Publications
Jun Jiang, Dengyu Liu, Jinwei Gu and Sabine Süsstrunk.
What is the Space of Spectral Sensitivity Functions for
Digital Color Cameras?. Workshop on Applications of Computer Vision (WACV) 2013.
Jun Jiang, Dengyu Liu, Jinwei Gu and Sabine
Süsstrunk.Supplementary
Document (with proof and other experimental details).
|
Images
 |
|
Experimental setup to obtain the ground truth of camera spectral
sensitivity:
We have measured the spectral sensitivity functions for 28
cameras, including professional DSLRs, point-and-shoot, industrial and
mobile cameras (i.e.Nokia N900), using a monochromator and a spectrometer
PR655. At each wavelength, the camera spectral sensitivity in RGB channels
is calculated by c(λ) = d(λ)/(r(λ)⋅t(λ)), where d is the raw data
recorded by the camera, r is the illuminant radiance measured by the
spectrometer, and t is the exposure time of the camera. All other settings
(i.e., ISO and aperture) remained the same during the measurement for each
camera. The procedure is repeated across the whole visible wavelength from
400 to 720nm with an interval of 10nm.
|
 |
|
Normalized camera spectral sensitivity:
The spectral sensitivity of 28
cameras are measured in our database, including professional DSLRs,
point-and-shoot, industrial and mobile cameras. Statistical analysis of
these measurements shows the space of camera spectral sensitivities is
two-dimensional. This statistical model is useful to recover camera
spectral sensitivities from a single image with regular broadband
reflective color targets.
|
 |
|
Noise in direct inversion:
The need for statistics prior when estimating the camera spectral
sensitivities. Direct inversion suffers even with a small amount of noise
(1%) due to the low dimensionality of spectral reflectance of real-world
objects. The subscripts m and e stand for the measured and estimated camera
spectral sensitivities.
|
 |
|
Luther condition evaluation:
A camera satisfies the Luther condition if
its spectral sensitivity function is a linear transformation of the
CIE-1931 2-degree color matching function. The Luther condition can be
evaluated by the RMS error between C2deg and T⋅C, C2deg are the CIE-1931
2-degree color matching functions, and C are the measured camera spectral
sensitivities. Color difference (CIEDE00) is calculated between C2deg and T⋅C under CIE D65 illuminant and the 1269 Munsell color chips. Ideally,
spectral RMS and color differences are zero if a camera perfectly satisfies
the Luther condition. Overall, most cameras have a deviation from the
Luther condition, especially for the two industrial cameras.
|
 |
|
Principal
components of camera spectral sensitivities:
The principal components of
camera spectral sensitivities. The three columns represent the R/G/B
channels, respectively. We performed PCA on 28 cameras including Canon
Nikon, SONY etc. The 1st principal component accounts for over 95% of total
variance for all three channels, and the first two principal components
accounts for over 97% of total variance. Thus, we model camera spectral
sensitivity functions as two-dimensional functions.
|
 |
|
The recovery of camera spectral sensitivies of Canon 60D:
(a) The measured spectrum of a daylight. (b) The spectral reflectance of a
color checker DC. (c) The captured image (glossy and duplicate patches are
removed to avoid overweighting certain colors). (d) The recovered spectral
sensitivities with known daylight spectrum. By using a daylight model, we
can recover both the daylight spectrum (e) and the camera spectral
sensitivities (f). The subscripts m and e in (d) and (f) stand for the
measured and estimated camera spectral sensitivities, respectively.
|
 |
|
Comparison of recovered camera spectral sensitivities using 3 basis
functions:
(a) Fourier basis, (b) polynomial basis, and (c) radial basis.
The results are worse than that of using the PCA model. The subscripts m
and e stand for the measured and estimated camera spectral sensitivities,
respectively.
|
 |
|
Comparison of four types of basis functions for modeling camera spectral
sensitivity functions:
A-PCA model, B-Fourier basis, C-radial basis and
D-polynomial basis with the ground truth (E). A color checker is rendered
under D65 with camera spectral sensitivities recovered using these basis
functions, and converted to sRGB. The average color difference between the
renderings (from A to D) and the ground truth (E) are 1.59, 3.54, 2.43 and
7. The gain of the imaging system remains the same for all four basis
functions.
|
 |
|
Simulation of color rendering for cameras:
The images are rendered to
sRGB based on the measured (top row) and estimated (bottom row) camera
spectral sensitivities of Canon 60D. (a) face, (b) beads, and (c) peppers
are from the multispectral image database [25]. The values in the
parentheses are the average color difference (CIEDE00) between the
bottom and top images in each column. For all three examples, the color
difference is close to one, indicating a close color match.
|
 |
|
Correction of images by Canon5D Mark II:
CC is put in the scene to
locate the white point. The estimated camera spectral sensitivity of
Canon5D Mark II is used to calculate T. Left column: The captured image;
Middle column: the corrected image based on T, and Right column: the corrected
image by dividing the white point (without using T). The images are
rendered in sRGB color space.
|
|
Slides
Workshop on Applications of Computer Vision (WACV) 2013 Presentation
Workshop on Applications of Computer Vision (WACV) 2013 Poster
|
Software
Code for recovering camera spectral sensitivity from a single image This demo MATLAB code shows the recovery of camera spectral sensitivity with a regular color checker from a single picture under unknown daylight. An example image captured by a Canon 60D (CR2 RAW format) is included. The measured camera spectral sensitivity for Canon 60D and measured daylight are also included for comparison.
|
Database
Database of camera spectral sensitivity The database includes the spectral sensitivity functions for 28 cameras,
including professional DSLRs, point-and-shoot, industrial and mobile
cameras. The measurement starts from 400nm to 720nm in an interval of 10nm. The database is in the
form of a text file. Each entry starts with camera name and follows by
measured spectral sensitivities in red, green and blue channel.
|
Spectral Sensitivity Measurement, CVL, University of Tokyo
|
|