This research will concentrate on furthering the manual image registration and simple ratio image fusion portions of the research. The results obtained previously did not take into account a very diverse set of image resolutions. The resolution factors examined were 1x, 2x, 3x, 4x, 5x, 8x, 10x, 16x, and 30x. This research will examine a set of image resolution factors that were skipped over in the previous study. The factors to be examined will be 6x, 12x, 20x, and 24x. These resolution factors will be manually registered and then fused using the simple ratio fusion method. The results will then be examined for registration accuracy, using ground control point RMS, and image fusion accuracy, using image RMS. It is hoped that this study will help fill in some of the gaps left by the previous study and thus lead to a clearer understanding of the usefulness of these image exploitation algorithms on low resolution images.

Abstract

List of Figures

1.0 Significance

2.0 Experimental Design and Methods

3.0 Conclusion

4.0 Timeline

References

Exploitation tasks on remotely sensed imagery, both in military and in civilian applications, have historically relied on high spatial resolution requirements to insure reasonable success in accomplishing a task. This requirement is due in part to the fact that a majority of the image exploitation has been performed by human analysts relying heavily on the classical remote sensing cues of texture shape, context, etc. Many of these cues contain features which are best captured and conveyed by imaging systems with high spatial modulation transfer functions (MTF). This high spatial MTF is achieved through the design of physically smaller detectors with a relatively wide broad-band spectral range. This results in panchromatic images characterized by very high spatial fidelity. The human visual system is predisposed to analyzing this high frequency content imagery and performs remarkably well despite the subjectivity of the process and the lack of understanding of the underlying psychophysical and cognitive mechanisms.

Concurrent improvements to imaging sensors and computer technology has provided an ability to generate unprecedented amounts of image data – particularly those obtained using electro-optical means. The shift of remote sensing toward commercialization has also opened another market in the civilian sector which has fueled the demand for image data and pushed for further improvements to the technology. As exciting as the prospects are for the generation of massive volumes of image data sets, the technology for adequately analyzing the libraries of incoming and archived data has not maintained pace with the image acquisition end of the imaging chain. The limited resource of human-based image exploitation is inadequate for rigorously analyzing the bulk of the incoming image data. In addition, increasing spatial resolution to fulfill human-based exploitation requirements (logical and attractive as it may be even if the sensor technology is capable of achieving this requirement) only compounds the data volume problem and severely limits the coverage of a sensor (i.e. coverage must often be traded for resolution). There are also tasks in exploitation which can be performed only with marginal success using high resolution monochrome imagery, particularly for automated exploitation. Tasks involving material identification and composition analysis fall in this category of exploitation tasks. (Farnung et. al. 1997)

The purpose of this study is to establish a foundation for image registration and image fusion remote sensing exploitation tools to be characterized in terms of their performance from a spatial/spectral resolution standpoint. Metrics that quantify the performance will also be investigated (namely ground control point RMS and image RMS) to validate their merit and offer potential improvement strategies. The following technical discussion section (Experimental Design and Methods) will detail the specifics of each exploitation task mentioned and the rationale for investigating each one for this study.

b)

The baseline imagery (which was used for the previous study) and that will be used for this study is a subset of the Southern Rainbow M7 imagery, taken on 8/24/95. This particular imagery was selected because it has many well defined targets in it such as tanks, trucks, two 21 meter by 7 meter grayscale panels, and a 12 meter by 5 meter resolution target. All of these objects have sharp corners and well defined features which aid in the image registration process. The imagery consists of two images, a multispectral image and a digitized air photo image. The size of the baseline multispectral image is 1272 pixel columns by 792 pixel rows by 15 bands. The size of the air photo image from which the panchromatic image was extracted is 1426 pixel columns by 883 pixel rows by 3 bands. Band 2, the green band, of the high resolution air photo image was chosen arbitrarily as the simulated panchromatic image to be registered. (cf. Figure 2.1) This green high resolution band will be used as the sharpening band in the fusion algorithm analysis, explained in more detail later in the proposal. (Farnung et. al. 1997)

This exploitation technique is of critical importance because the accuracy of registering imagery from various imaging platforms will greatly influence the performance of spatial resolution compensating algorithms such as image fusion. Precise image-to-image registration is necessary to form image mosaics, map temporal changes accurately, compare images from two different sensors, or combine multispectral images in a color composite. (Schowengerdt, 1983) Image registration involves selecting two images of the same scene, choosing small ground features, called ground control points (GCP’s), and warping one of them such that recognizable features from both (tracks on the ground, edges of vehicles or buildings, etc.) correspond to the same pixel locations. The resulting registered images appear to have been captured from the exact same detector location because all of the distinguishable features on the two coincide. (cf. Figure 2.2)

a) b)

c) d)

As seen in Figure 2.2, registration is important in my analysis because the two registered images can be combined by image fusion to gain information about a particular scene that each of the individual images could not provide on their own. For instance, panchromatic images usually have high spatial resolution because the broad spectral band allows smaller detectors to be used while maintaining high signal to noise. Multispectral (including thermal) images typically have lower spatial resolution than their panchromatic counterparts because the detectors are collecting less energy per unit area due to the narrower spectral bands. This means that in order to detect a sufficient signal the size of the detector must be increased. Better target detection can often be obtained by analyzing a high resolution panchromatic image, and better identification of target materials may be obtained by analyzing a multispectral image. Both types, panchromatic and multispectral images, have their advantages and disadvantages but by combining the two, one can potentially produce an image which has the properties of both in one image. Image registration is a crucial step in image fusion and image analysis because if the images do not coincide precisely, erroneous conclusions could be drawn from the resulting fused image. (Farnung et. al. 1997)

I want to study the effects of spatial resolutions on image registration to gain a solid understanding of how registration is affected by decreased resolution. This will help give electro-optical sensor designers an optimal target point for their system specifications. Likewise, an analyst’s understanding of the limitations of certain exploitation algorithms will provide a realistic degree of confidence associated with conclusions drawn from an analysis.

The registration method that will be examined in more detail for this study will be the manual GCP’s selection method. The manual method is performed by selecting points on a base image, using a computer program with a registration module, and then selecting the corresponding points on the image to be registered. These points are then used as inputs to a warping algorithm which shifts the image to be registered so that the GCP’s act as tie points and the portions of the image that were "tied" are in approximately the same location on the two images. (cf. Figure 2.2)

Registration performance will then be measured by both visual and statistical methods. The visual method can be performed, using a particular software package, by displaying the base and the registered images in separate windows on the display screen and then "linking" the two windows such that the image of one is superimposed over the other. This allows for a direct visual comparison of the registration performance, any difference in the two images will be seen immediately. (cf. Figure 2.2 c and d)

The quantitative measure of registration performance that will be used is carried out by selecting a new set of points on the registered and base images then calculating the Euclidean distance between the base and registered image points. The average of all the distances can be computed so that an estimation of the total image registration error can be obtained as a GCP RMS error. Theoretically, the RMS error should be zero if the GCP’s are selected correctly because the objects in the scene should be in the same pixel locations thus having zero distance between them. This is the same measure that was used in the previous study. (Farnung et. al. 1997)

For the image registration portion of this study the software package ENVI, (Environment for Visualizing Images) ,Version 5.0, developed by Research Systems inc., will be used. This was the same program used in the previous study. (Farnung et. al. 1997) The baseline images will be degraded to 6x, 12x, 20x, and 24x resolution factors using ENVI and utilizing the same procedure that was used before. (Farnung et. al. 1997) Refer to Figure 2.3 for examples of degrade images. These degraded images will then be registered and the analysis on the results will be performed. These registered images will then be used as inputs to the image fusion section of this study.

a) b)

Image fusion involves combining different images into a single new hybrid image. The original images may be products of different remote sensing platforms, and may have different spectral and spatial resolutions. For example, one might wish to merge data from Landsat Thematic Mapper (TM) with that obtained from French Systeme Pour l’ Observation del la Terre (SPOT). The TM has seven spectral bands ranging from 0.45 to 2.35 microns. Six of the bands (1-5 and 7) have 30 meter spatial resolution. The seventh band (band 6) provides thermal information and has 120 meter spatial resolution. SPOT has 3 spectral bands in the visible and near infrared region with 20 meter spatial resolution. It also has a panchromatic band with 10 meter spatial resolution. The most efficient method for an analyst to examine imagery from these two platforms would be to combine the useful information from both into a single image. Image fusion is important because it allows an analyst a technique to spatially enhance image quality without requiring high spatial resolution from the multispectral sensor by using a high resolution panchromatic data set. (Farnung et. al. 1997)

In this study low resolution multispectral images will be fused with higher resolution panchromatic images to form high resolution multispectral images. The image results obtained from the registration performance study (cf. Image Registration) will be used as input to the fusion algorithms. Inputs to the simple ratio fusion algorithm used in this study requires that the images be registered. (cf. Figure 2.4)

The fusion algorithm used for this study was developed by the Digital Imaging and Remote Sensing lab (DIRS). The simple ratio method will be used and it falls into the category called "Ratio Methods" for image fusion.

The ratio methods are image fusion techniques designed to maintain the radiometry of the original image. They require that the panchromatic sharpening image be highly correlated with the multispectral image. The procedure begins by dividing the pixels of the multispectral image into subpixels which are equal in size to the pixels of the high resolution panchromatic image. A superpixel corresponds to a collection of subpixels equivalent in size to the low resolution pixels.

The Simple Ratio method is designed to preserve as much radiometric accuracy as possible for highly correlated bands, and forms the basis for other ratio methods which are discussed in more detail in the previous study. (cf. Farnung et. al. 1997) The Simple Ratio method begins by pixel replicating and blurring the high resolution panchromatic image so that its subpixels are the same size as the pixels of the low resolution multispectral image. The panchromatic image is registered to the multispectral image to preserve the radiometry of the multispectral image.

Fusion performance can be measured quantitatively by performing an image RMS evaluation. This is performed by comparing the fused product with the multispectral input at a resolution factor higher than the factor that has been fused. The resulting fused image will have the same pixel spatial resolution, throughout all of the bands, as the original multispectral image. It is between this fused product and the original multispectral image that an RMS comparison is applied. For example, in this study the 5x multispectral image from the previous study will be compared to the 20x fused product to be created. The spatial resolution for the 5x multispectral image equals that of the 20x fused product, both are 5 meters per pixel. The following chart shows the images that will be compared and their respective ground resolutions. (Farnung et. al. 1997)

3x multi (3m) = 12x fused (3m)

5x multi (5m) = 20x fused (5m)

6x multi (6m) = 24x fused (6m)

These three comparisons will then be used to fill in the gaps left from the previous study (only two comparisons were made, 4x to 16x and 2x to 8x). These results will hopefully lead to a better understanding of the effects of resolution on image fusion.

In conclusion, by creating a new set of degraded image resolution factors and registering and fusing them it is hoped that a better understanding of the effects of resolution on these exploitation techniques will be gained. By gaining a better understanding of the errors witnessed, future users will have a clearer understanding of the usefulness of these techniques and be able to adjust their studies accordingly.

He following is a proposed timeline for this study.

Week ending December 6: Create 6x, 12x, 20x, and 24x resolution factors.

Week ending December 13: Register and fuse the images.

Week ending December 20: Perform RMS calculations.

Week ending December 27: Summarize results and write up the report.

Monday January 5: Hand in final report.

Farnung, C. E., D., Konno, J.R. Allen, R.V. Raqueño, G. Robinson, J.R. Schott, "Influence of MTF on Exploitation Accuracy (Task 2)", RIT/DIRS Report 97-72-155, 1997.

Schowengerdt, R. A., "Techniques for Image Processing and Classification in Remote Sensing", Academic Press Inc., London, 1983.