Correction of Geometric Distortions in

Line Scanner Imagery

Peter A. Kopacz

1. Introduction 

The Digital Image and Remote Sensing laboratory, located at RIT’s Center for Imaging Science, has developed an airborne imaging system also known as the Modular Imaging Spectrometer Instrument (MISI). This sensor is capable of imaging in the visible (VIS, 0.4-0.7 mm), mid-wave infrared (MWIR, 3-5 mm), and long-wave infrared (LWIR, 8-14 mm) regions of the electromagnetic spectrum. Possible applications of this high resolution data collecting system are, among other things, earth observation research and image acquisition for development of atmospheric removal algorithms. The downfall of this and any remote system is its inherent geometric distortions. It would be necessary to remove the distortions before the imagery can be analyzed. The objective of this research is to incorporate corrections based on the geometric characteristics of the sensor under study, as well as the flight parameters involved in the acquisition process. There is confidence that this can be accomplished while preserving the mean radiometry (i.e. original brightness values).


2.  Background 

2.1 Line Scanner

Line scanners are electro-optical imaging systems that collect data along a single line on the ground, perpendicular to the direction of motion (Figure 2.1).  Each line is composed of individual ground pixels. A single ground pixel is known as the Ground Instantaneous Field of View (GIFOV). A scanning mirror then projects the sampled ground pixels onto the detector(s) in equal angular increments, sweeping across the direction of the aircraft’s flight path. This angular sampling is also known as the Instantaneous Field of View (IFOV). The total angular extent of this cross-track rotation, known as the Field of View (FOV), can often be 90-120 degrees. With each rotation of the scanning mirror, the airborne platform (detection system) moves along track to sample consecutive lines on the ground, which form an image when taken together.[Schott, 1997]

 
 
Figure 2.1 Airborne line scanner collection scheme (Schott, 1997)
 
 

2.2  Geometric Distortions

There are two types of geometric distortions: those associated with the attitude (orientation) of the platform, and those reflecting the optical characteristics of the sensor.  These distortions will be reflected in the output image. Another words, platform motion and attitude instabilities will cause the sampled data to be projected in the ‘wrong’ places (ex: shifted) in the reconstructed image. The three geometric distortions responsible are roll, pitch, and yaw.  Since the scope of this research involved only the roll aspect, the pitch and yaw distortions will not be discussed.

 
 
Figure 2.2  Geometric Distortions due to aircraft orientation (Schott, 1997)

The other distortions are a result of how the sensor collects ground data are related to its optical and scanning properties. These are tangent and v/H respectively.
 

2.2.1 Roll Distortion

The platform may undergo rotation about its direction of flight vector, causing it to roll. The result of the roll distortion is that the individual scan lines are shifted.  This is illustrated in Figure 2.2-1, along with the hypothetical output image. [Salacain, 1995]

Figure 2.2-1  Roll Distortion

 

 2.2.2 Tangential Distortion

As mentioned before, line scanners sample each ground pixel at equal angular increments (IFOV). This implies that ground pixels located away from Nadir will represent larger ground areas. An ‘edge’ pixel can typically be 2-3 times the area of the pixel at Nadir. This would cause the output image to appear ‘compressed’ near the edges.

 
 
Figure 2.2-2  Tangent distortion, due to the focal plane geometry of the sensor, causes an unequal ground representation between the  collected pixels ( Dedge > Dnadir ).
 

 

2.2.3 v/H Distortion

The ground velocity v and altitude H of the aircraft play an important role in the image formation, along with the scan rate at which lines are being acquired. This rate must be in sync with the v/H ratio. Failure to appropriately adjust the scan rate can lead to one of the following cases: oversampling or undersampling.

Oversampling occurs when the scan rate is faster than it should be. This causes the Line Advance, distance traveled by the platform before the next scan line is acquired, to be smaller than the ground spot of the sensor (GIFOV). The result of oversampling is an abundance of scan lines, causing the image to be stretched along track.

Figure 2.2-3 V/H oversampling.
 

Undersampling is a result of the scan rate being too slow. In this case, the Line Advance is greater than the GIFOV along track.  Scanning  too slowly will leave gaps between consecutive lines, causing the image to be compressed along track.

 
 
Figure 2.2-4  V/H undersampling.

Based on the above evaluation of the geometric distortions present in line scanner imagery, it is obvious that the output image may appear far different from what it is expected to be. Such distortions may obscure certain image features, making them undetectable to the image analyst. For these reasons it is desirable to remove geometric distortions from the acquired imagery prior to qualitative or quantitative interpretation.
 


3. Experimental / Design 

3.1 Correction of Roll and Tangent Distortions

The following section describes the theory behind the designed roll and tangent correction algorithms.

3.1.1 Description of the Involved Flight Parameters

Figure 3.1-1 illustrates the sensor flight parameters involved in the process of data acquisition, excluding roll effects for simplicity. In this particular example, the sensor platform is centered with respect to the area being imaged (i.e. qs = -qe).

 Figure 3.1-1 Sensor Flight Parameters
The aircraft is positioned at an altitude H, which is measured at the shortest distance between  the platform and the ground. This distance is often referred to as Nadir. The sensor’s field of view FOV is determined as follows:
Eq.1

where N is the number of samples (or pixels) per scan line, and IFOV is again the instantaneous field of view angle at which the sensor acquires single ground samples. In this particular example data has been acquired from left to right, as indicated by the direction of IFOV. This angle takes on a positive value due to the direction of the scan. By contrast, collecting data from right to left would make the IFOV angle negative.

Allowing the Nadir line to serve as the “zero point” (origin), leads to another important  sign convention:

 
Figure 3.1-2  Sign Convention
 
Based on Figure 3.1-2, the starting angle for each collected scan line, assuming symmetry, is:
 
Eq.2
However, according to the above sign convention, this angle is actually negative. This means that the angle at which each collected scan line ends is found to be:
 
Eq.3

Letting qs=-0.384 radians, IFOV=0.00153398 radians, and N=512, the end of collection angle would then be 0.384 radians, which again corresponds with the above sign convention. Based on the previous definitions of Nadir, aircraft altitude, and qs, the starting and ending ground distance coordinates for a single scan line are found to be:
 

 Eq.4
Eq.5

These parameters are illustrated in Figure 3.1-2. Previously established sign convention also applies to all ground parameters.

As has already been explained in the previous section, the sensor platform undergoes attitude instabilities, leading to significant geometric distortions. The most significant of these is the roll distortion. It is an unsystematic type of error (i.e. variable), and it must therefore be treated on a per line basis. According to the previously mentioned sign convention, a scan line shifted to the left of Nadir would have a negative roll angle. Similarly, a scan line shifted to the right would have a positive roll angle.

 
Figure 3.1-3 Roll effects on a scan line

Figure 3.1-3 illustrates a scan line that was shifted to the left, by exactly one Nadir ground pixel (or IFOV). This is not typically what happens, since the amount of shift will most likely be a fraction and not a whole quantity. The new sample locations are depicted with the dotted lines. It is important to realize that the scan line is now located even farther to the left of Nadir, due to the roll distortion.  The samples near the start of the scan line (ex: at the left edge) cover a larger ground area than those in a scan line without the roll effects. This further increases the tangential distortions near left edge of the picture. The starting and ending ground coordinates can once again be calculated for a single line, with equations 4 and 5 now accounting for the roll:
 

Eq.6
Eq.7

These two calculations are repeated for all scan lines in the image, with the goal of finding the largest ground width. Based on Figure 3.1-4, this metric would be:
 

Eq.8

where |   | indicate absolute value to permit scans which go from right to left. In this example, xstart1 was located farthest to the left of Nadir (negative), while xend2 was farthest off to the right of Nadir (positive). Their absolute difference yields the largest ground separation. This calculation ensures that when the input scan lines will be roll-corrected (i.e. shifted), there will be enough freedom to do so.

 
Figure 3.1-4  Illustration of maximum ground swath

The next question is what size should the corrected ground pixels be? In order to answer that one should consider the ground pixel at Nadir, which is the only region not exposed to tangent distortion. To correct this error, all image pixels have to be resampled so that they represent equal ground areas. Figure 3.1-5 illustrates the Nadir pixel along with the appropriate equation used to calculate its ground representation.

 
Figure 3.1-5  Nadir pixel geometry
 
 Eq.9

Dividing the previously determined maximum ground width (Eq.8) by the Nadir pixel (Eq.9) yields the new width of  the roll and tangent corrected image. This is a unitless quantity since both maximum ground width and D Nadir  are in units of distance. From a resolution perspective, it is also worth noting that D Nadir is essentially the sensor’s GIFOV parameter.
 

  [pixels]  Eq.10
 

3.1.2 Calculation of Transform Coordinates

This is the very backbone of the entire roll and tangent correction process. In order to shift the input data, a matrix of transform coordinates must first be determined. Recall Figure 3.1-1 which illustrated a center of a given ground pixel. Mathematically, this ground distance parameter can be expressed as:

   Eq.11

The maximum xstart parameter, calculated with Eq.6, now serves as a Ground Control Point (GCP). This GCP is used to calculate the number of Nadir-size pixels separating it from the ground center.

 
Figure 3.1-6  Illustrates the separation between a GCP and a given ground pixel center
 
 Eq.12

The output coordinate is an index location of a corrected image pixel in the output array. In the example above this would be an integer value, which typically is not the case.  In any case, Eq.12 determines an output image coordinate as a function of an input image coordinate at non-integer increments. It is possible to obtain the transform coordinates in this matter, but with severe consequences. At large viewing angles (i.e. large FOV), this “forward-transformation” technique will overlook certain output indices, leaving “empty” pixels. It would be much better, however, to determine a (new) coordinate in the input image, based on a given output coordinate. This geometric “back-transformation” process steps through the output array indeces instead, which happen to be integers [ Jensen, 1986].  In order to do this, Eq.12 is rearranged to yield:
 

Eq.13

The left hand side of this expression can now be replaced by Eq.11 and rearranged to solve for x, which indicates the location of a given output pixel in the input image.
 

    Eq.14
 

3.2 Correction of v/H Distortion

The following section describes the theory behind the designed v/H correction algorithm.

3.2.1 Description of the Involved Flight Parameters

When the sensor oversamples or undersamples the target, there is a mismatch between the GIFOV and Line Advance parameters. This results in either an overlap or a gap between consecutive scan lines. Based on the illustration below, this difference between GIFOV and Line Advance for M scan lines is:

Eq. 15
 
Figure 3.2-1 Illustration of Line Advance and GIFOV mismatch

It was previously mentioned that this difference (v/H distortion) is caused by an incorrect scan rate. The ideal rate, at which consecutive scan lines should be collected, is governed by the following expression:

    Eq.16

where v is the aircraft velocity. From this expression, it is quite apparent that the scan rate has a strong dependence on the aircraft velocity and altitude, which explains why this problem is referred to as v/H. The IFOV parameter remains constant throughout the time of collection, and is therefore neglected.  The actual scan rate at which the lines are acquired is described by Eq.17, where t scan is the time required to obtain a single line:
 

  Eq.17

Equations 16 and 17 can be rearranged for GIFOV and Line Advance terms, and substituted into Eq.15 to yield:
 

Eq. 18

which indicates how much oversampling or undersampling occurred for M scan lines. The actual ground area that was supposed to be covered by the collection  (i.e. without v/H) is:
 

 Eq. 19

which is then used to determine the new height of the roll, tangent, and v/H corrected output image.
 

 Eq. 20

Note that GIFOV and D Nadir are equivalent expressions for the sensor’s maximum ground resolution, and that the new image height has to be an integer value.

3.2.2 Calculation of Transform Coordinates

In order to eliminate the v/H errors, a second transform coordinate matrix was determined. Each transform coordinate represents a new column index in the output array for a given input scan line:
 

   Eq.21

where line center represents the center of any given scan line on the ground, and is calculated to be:

 Eq.22

where ystart serves as a GCP with respect to any input scan line center. This parameter can be eliminated, assuming that the first line was acquired at position 0.  Figure 3.2-2 illustrates the overall concept.

 
Figure 3.2-2  Separation between a GCP and a given line center

These y transform coordinates, along with the x coordinates calculated in section 3.1.2,  are used to extract brightness values from the input image and assign them to the appropriate locations in the output image. This process, also known as interpolation, will now be discussed in detail.
 

3.3 Intensity Interpolation of Geometrically Corrected Data

3.3.1 Nearest Neighbor

This resampling technique, also known as zero-order interpolation, looks for the brightness value of the input pixel closest to the calculated x and y coordinate, and assigns it to a corresponding output array coordinate [Jensen, 1986]. The algorithm increments through each of the output array coordinates, looking for the closest match. This process ensures all output pixels are included in the interpolation process. The advantage of using nearest neighbor is that it is computationally fast and it preserves the original brightness values in the output image. Its downfall is that the resampled output will have certain visual artifacts, thereby degrading the image.
 

Figure 3.3-1 Two dimensional Nearest Neighbor Resampling
 

3.3.2 Bilinear Interpolation

This method, also known as first-order interpolation, finds four pixels nearest the desired position (i.e. new x and y coordinate) in the input image, and computes a new brightness value based on the weighted distances to these points [Jensen, 1986]. The closer an input pixel to the desired transform coordinate location, the more weight it will have in the calculation of the brightness value for the output pixel. This brightness value BV is determined by Equation 23:

  Eq.23

where Zk are the brightness values of the four nearest pixels, and Dk2 are the squared distances from the output pixel (being 'determined') to these four pixels. Figure 3.3-2 illustrates this process.

 
Figure 3.3-2 Two dimensional Bilinear Interpolation

The bilinear method requires more computation time, and it permanently alters the original brightness values. Its main advantage is the visual quality of the output image.
 


4. Results  

The following sections compare the input images used to test the described algorithms to the corrected output images. A sensitivity analysis has also been performed on the involved flight parameters to identify potential sources of error.

4.1 Error Sensitivity Analysis

The pixel position at which a given ground object is collected by the sensor depends on the orientation of the platform, aircraft velocity, and the scan rate of the mirror. Beer’s Law of Error Propagation is a useful approach in determining which of the contributing flight parameters are the largest sources of error, and which have negligible effects on the collected data. The error associated with accurately determining a ground pixel’s location can be treated in terms of across track (error in X) and along track (error in Y) directions. Separable treatment of X and Y is possible only if the effects of pitch and yaw distortions are negligible, and the sensor platform is also assumed to be traveling along a straight line.

 Figure 4.0-1 Indication of the Along Track (Y) and Across Track (X) directions.
 

4.1.1 Across Track Errors

Governing Equations

As it was previously stated in Section 3.1.1, any given ground pixel center is determined according to the following equation:

Eq.24

Parameters qs, qr, H, are recorded during the flight. The minimal IFOV parameter can be determined by the following equation:
 

Eq.25

where H is again the flying altitude and GIFOV  is the minimum Ground Instantaneous Field of View of the sensor, also denoted as D Nadir. Schowengerdt (1996) claims that:
 

   Eq.26

where w is the width of a single detector, and f is the focal length of the optical system. The GIFOV in turn strongly depends on H, w, and f parameters. Rearranging and solving Eq.25 for GIFOV yields:
 

    Eq.27

Beer’s Law of Error Propagation allows us to determine the total error in the ground pixel center, due to all contributing terms mentioned above. Based solely on equation 24, assuming pitch and yaw effects to be negligible, the error across track (X) direction is:
 

    Eq.28

where SX is the total error associated in determining the ground pixel center. The right hand side of this equation represents partial derivatives of each contributing term, along with their sources of error (SH, Sqs, Sqr, Sx, SIFOV).

Based on Eq.26 and Beer’s Law of Error Propagation, the total error in IFOV is dependent on the detector width w and the focal length f or :
 

Eq.29

where Sw and Sf are errors associated with the detector width and focal length respectively.
 The partial derivatives with respect to each of the above parameters are:
 
 

 
 
For the total error in IFOV, the partial derivatives with respect to the detector width and focal length are:
 
During any given scan, parameters H, qr, x are subject to change.  The other parameters, qs and IFOV, remain fixed during this time. The analysis was performed by independently varying the involved parameters according to the following criteria:
 
 
Table 1. List of Initial Test Parameters

The source of error for IFOV partial derivative (Eq.28) has only a minute contribution from the focal length :

Table 2. Source of error to IFOV partial (Eq.28)
 
 

Error Plots
 

 
Figure 4.1-1 Error Contribution from Altitude

According to the above results, the across track error is only a concern for pixels far away from Nadir. Even a modest variation of altitude by 32 feet leads to a displacement error of 23 pixels in the image. Just as it was expected, there is no error associated with the pixel directly at Nadir.
 

 
Figure 4.1-2 Error Contribution from Roll Angle

The largest source of across track error came from the start and roll angles, since both parameters had the same partial derivatives and error measurements. Only one plot was sufficient to illustrate (Figure 4.1-2). Even at Nadir, a mild roll variation of only 2 degrees resulted in a 23 pixel error. At a start of collection angle of 45 degrees, that same roll variation increased the error to 48 pixels.
 

 
Figure 4.1-3 Error Contribution from IFOV Angle

Without a surprise, this was the smallest source of error across track. This parameter is fixed during a given scan, so any variation in its measurement is minimal. Even at 45 degrees from Nadir, the total error was only 2.4 pixels. Again, the Nadir pixel was not affected.

 
 4.1.2 Along Track Errors

Governing Equations

The positional accuracy at which a ground pixel is collected also depends on the along track (Y) conditions. Proper conditions must be ensured that consecutive scan lines are acquired at the right time, to prevent oversampling or undersampling. In other words, the GIFOV and Line Advance parameters should always match. Conditions other than that result in v/H distortion, and are again described by the following equation:
 

 Eq.30

The ideal rate sr ideal, at which consecutive scan lines “should be” collected, is again governed by the following equation:
 

  Eq.31

Since obtaining scan lines at ideal time intervals is virtually impossible, the actual rate at which a single scan line is acquired can again be expressed as:
 

   Eq.32

Beer’s Law of Error Propagation then leads to the following expression for measuring the error along track :

 

Eq.33
 
 
where SH , Sv are errors associated with aircraft altitude and velocity, SIFOV  is the error due to the IFOV, and St scan is the error associated with the time per single scan. The other equation terms are just again partial derivatives of Equation 30 with respect to the contributing terms. Based on the governing equation (Eq.30) and Beer’s Law of Error Propagation expression, the partial derivatives with respect to M, H, vIFOV, and t scan are:
 
 
 
Recall that partial derivatives with respect to the detector width w and focal length f are:
 
 
During any given scan, the following parameters are subject to change: H, v.  The other parameters (M, t scan, and IFOV) remain fixed during this time. Intuitively parameters Mt scan, and IFOV will have negligible error contributions. The analysis was performed with the following initial parameters:
 
Table 3. List of Initial Test Parameters

Again, the source of error for the IFOV partial derivative (Eq.28) has a minimal contribution from the focal length :

Table 4. Source of error to IFOV
 
 
Error Plots
 
Figure 4.1-5  Error Contribution from Altitude

According to these results, the along track error increases linearly with the error in flying altitude. Again the Nadir pixel is unaffected, but near the edges the displacement error is almost 17 pixels in the image.

 
Figure 4.1-6  Error Contribution from Velocity

The largest source of error along track was caused by the aircraft velocity. Recall that this parameter, along with the altitude, is a major factor in the v/H distortion. For example, an aircraft flying at a very high velocity and scanning too slow, will not collect enough data. Based on this plot, there is a linear relationship between the velocity and the along track error. After collecting just 32 lines the error reaches 24 pixels as the velocity increases. Note that the specified speed increase is quite moderate too, and the scan rate is rather generous (high). Throughout any given scan, the velocity parameter may fluctuate up and down since it is virtually impossible for the pilot to maintain a constant speed. This fluctuation would also be reflected in the resulting image.

As it was previously mentioned, the IFOV and the tscan parameters have negligible impact on the along track error. Essentially, they remain constant throughout any given flight. Overall, the sensor shows its weakness when imaging at high viewing angles. The errors could be somewhat reduced by trying to stabilize the flying conditions (i.e. roll gyroscope, a more stable aircraft).
 

4.2 Application of the Correction Algorithms

The implemented correction algorithms were tested on three different scene simulations, each one modeling a real life line scanner. Recall that these synthetic images were created with DIRSIG, which stands for Digital Imaging and Remote Sensing Image Generation. This section qualitatively compares the corrected images with their corresponding inputs.

Test Image 1

Table 5 lists the flight parameters associated with Figure 4.2-1. These values were used to calculate the new image size and transform coordinates needed to remove the distortions. Refer to Section 3 for a detailed description of the design stage.
 

Table 5. Flight Profile for Test Image 1
 
 
Figure 4.2-1 Input Test Image 1

This particular input image contains a mild degree of roll (~ 1o), which can be seen along the runway and in the highlighted region. Since tangent effects are usually most significant near the edges, they are not as obvious in this particular example. The apparent vertical stretching of scene objects (along track) represents the v/H distortion artifacts. The arrow indicates the direction of flight as well as the platform position with with respect to the in-scene objects.

 
 Figure 4.2-2 Corrected Test Image 1

The correction algorithms successfully removed the discussed distortions. The new image dimensions are 334 lines by 551 pixels per line. This increase in image width, a result of the roll and tangent corrections, ensures that there is enough room to manipulate its rows. The new height dimension is a result of the v/H correction algorithm, which simply eliminated the oversampled information. The image appears completely rectified (i.e. free of roll), with all its pixels representing equal ground areas. The tangent and v/H corrections have ensured equal ground representation both across track and along track  (i.e. helicopter). This particular test used the nearest neighbor resampling technique. The histograms of both images, displayed below, illustrate that the brightness value information was preserved very well.
 

Figure 4.2-3 Histogram Comparison for Test Image 1: (a) Input,  (b) Output
 

Test Image 2

The flight parameters listed in Table 5 are also valid for this test image (Figure 4.2-4), with two exceptions. First, the scene was generated without roll effects, to focus the analysis on the tangent and v/H corrections. Second, the flying altitude was reduced to 250 feet, as an attempt to enhance the tangent errors. The scan rate for such a low altitude was too slow, leading to significant undersampling along track. This explains why the image appears so compressed along the vertical.
 

 
Figure 4.2-4 Input Test Image 2

Figure 4.2-5 is the corrected test image 2. It is very clear that the v/H correction was successful. The height of the new image increased to 1336 lines, which indicates a significant amount of data being “added” via resampling to eliminate undersampling. Again, the nearest neighbor method proved quite effective in preserving the original brightness values. The width of the new image increased (slightly) to 541 pixels, due to the tangent correction. The highlighted region illustrates that this feature appears to have the proper across track and along track dimensions (i.e. square pixels).
 

 
Figure 4.2-5 Corrected Test Image 2
 

Test Image 3

The previous flight parameters remained valid for this case as well (Figure 4.2-6). The scene was again generated without roll effects, to focus the analysis on the tangent and v/H corrections. The platform remained at 250 feet. What distinguishes this particular scene from the others is that the sensor ‘flew’ over the scene at a 45 degree angle, as indicated by the arrow. It was anticipated that this ‘procedure’ would enhance the tangent errors, which are somewhat visible in the slight curvature of the runway. Again, the scan rate for such a low altitude was too slow, leading to an apparent compression along the vertical.
 

 
Figure 4.2-6 Input Test Image 3

The correction algorithms completely eliminated the v/H errors - the height of this image is again 1336 lines. The ‘tangent’ curvature in the runway has been somewhat reduced, but not eliminated as expected. The increase in width (541 pixels) confirms that the pixels have been resampled to represent equal ground areas. The dimensions along track and across track are once again ‘square’.
 

Figure 4.2-7 Corrected Test Image 3
 
 
 

5. Conclusion and Future Work 

Based on the obtained evaluation results, it is clear that the correction algorithms are performing as expected. Factors that make their validation more difficult are the imprecise determination of the aircraft point of observation (i.e. flight parameters). This was demonstrated with the error analysis approach. Future efforts should be aimed at testing the current algorithms on real imagery containing more evident tangent errors. The last test image (#3) used for validation did not prove to be sufficient in this aspect. The curvature along the runway is still visible. It would be ideal to generate an image at a 45 degree angle, for a ± 60 degree FOV system. The image obtained after tangent correction could then be compared to the results in Figure 5.0-1. Future efforts should also include implementation of pitch and yaw correction algorithms.  Lastly, now that MISI is operational, its performance can be validated against the currently developed algorithms.

 
 
Figure 5.0-1  Input image (left) with apparent tangent effects along the road (diagonal). The corrected image (right) should have straight diagonal roads. Also, notice the difference between the tanks located in the lower left hand corner. The along track compression has been eliminated, ensuring square pixel representation.  (Image courtesy of the Rochester Institute of Technology, DIRS lab.)

Table of Contents