|
Good Pipe |
Cracked Pipe |
 |
 |
Proof-of-Concept: Three Dimensional Pipe Surface Reconstruction from 2D Laser Scanned Image
Matthew Altman
This project began in 1997 as an extension of a three-dimensional laser scanning science research project. The idea was posed by members of the oil community at a meeting of the ARCO Engineering Club (Anchorage, AK) that an easily portable scanning device would be extremely beneficial for use on the inside of larger pipes which can become easily clogged or suffer from corrosive damage. It was later discovered that a similar idea was posed in the April 1996 issue of NASA Tech Briefs. There, one brief gave an initial design for the creation of what was termed a “cavity inspection device”. The purpose of the design was for follow-up by interested parties to create the actual device. To date, no parties have undertaken the project.
All literature searches to date have required looking in numerous fields for applicable information. Because there are numerous aspects to this project, research has been conducted in the following areas: machine vision, remote sensing, photography (particularly panoramic imaging), computer science, algorithm development, and robotics. Numerous methods have been utilized for the creation of three-dimensional images using a laser projection of sorts; of particular interest has been projection of “structured light” and the use of projected fringe and beam patterns. Because of the nature and intent of the project, it was determined that an annular projection of a laser beam should be created for use on the pipe scanning system.
In the winter of 1997/1998, as part of the initial design work being done on this project, numerous methods for crawling through a pipe were examined. Because of prohibitive costs and the desire to keep the overall design simple and inexpensive, a new mechanical design was sought. By March, a design was completed and presented. This design made use of a tri-axial system that depends on falling force and driving torque for complete balance. It also provides several methods for recovery and from becoming jammed within the pipe. This design won an award from the Department of the Army (Army Engineering), but has yet to be tested. The internal design of the scanner system will be redesigned.
In the areas of algorithmic development, several digital image processing books, algorithm references, and experts were inquired for algorithms that perform similar operations as those that I wish to use. There was no success; therefore, it was determined that algorithms which could be combined would need to be found. During this process, two specific types of algorithms were determined: those of panoramic image unwrapping and those of object location (and recognition). While panoramic image unwrapping involves a direct look-around image being unfolded into a two dimensional, “flat” image, the image at hand is a look-ahead image that contains an annular area of illumination. Because both systems require an unwrapping and refolding, these systems were considered similar enough for analysis. From machine vision systems, object recognition (and shape/pattern recognition) was found to be a necessary component in order to register the image. This will also be applied in algorithms that register between multiple frames of the captured video sequence. In an overall sense, a similar system is a derivation of stereometric ranging through a monocular imaging system. A system such as this entails a single image capture device, capturing images at different positions (using specified motion control), and the image compilation procedures are fairly similar, as they require two image compilation for a three dimensional result, just with compensation across different axes. Final display algorithms vary largely, and because of this fact, the imaging system at hand can make use of many of them. Options for this include an x,y pipe fly-through, xyz half pipe analysis (cut away model), and xyz full pipe analysis (full model). Each of these require different reconstruction methods, particularly making use of different convolution methods, as will be required in the xyz models according to the expansion methods used in the employed algorithms.
To date, there have been no known successful attempts to build such a device. One research group has attempted to build an algorithm for a device that has no motion axis but simply takes a picture from within the explicit center of the pipe. In talking with Marty Anderson of MIAC, a corrosion engineering and awareness company in Sterling, Alaska, no known devices are in use for said purposes; however, a device which could be used for these purposes could spare companies millions of dollars in major repair efforts, prevent leaks and environmental damages, and provide large amounts of data that have previously not been collected but could prove to be highly valuable (coating analysis, buildup analysis, stress and weakened points). Of greatest value will be its ability to analyze the entirety of the pipe from the inside – this means it could be used on pipes that go from the deck of an oil rig to the sea floor and below.
Theory
In order for the algorithms to
determine the proper 3D transformation of the video sequence, several
characteristics must be known about the pipe and the system relative to the
pipe. These properties include the diameter of the pipe, the different
scan angles in order to determine the size of the annular projection, and, if
multiple frames are to be analyzed, what the velocity of the system is as it
moves through the pipe. Since the system is working in pixel units, the
image must be translated into a pseudo-polar look up table, which can then be
related to a conical system, traced back into cylindrical coordinates. The
system assumes that all data given is constant and bases the multi-frame
analysis on the initial view of the pipe (i.e. frame #1).
The methods employed to develop the final three dimensional representation were as follows. First, the decision was made pertaining to what would be illuminated; namely, a specific interior area of the pipe. Visualizations were done in Strata 3D, which allowed for the ability to make assumptions as to what the camera would be viewing. With this portion of the modeling complete, simulated camera views were created using gradient tools in Adobe Photoshop. The models were specifically made such that the annular image could be tested with the camera "slightly off angle" - in actuality the area at center of the annulus is slightly skewed and is a few pixels off of the exact center. Then methods were derived allowing for the detection of registration points on the frame, which would allow for the determination of a center point. Transformations relative to the data provided by the user and values determined by the algorithms (as described in Theory) were then compiled and completed, creating a multidimensional array complete with voxel values. In order to visualize the data set, several methods were developed (or used from the IDL library). These included the extensive and powerful XVolume, provided by Research Systems, and XObject View (XObjview), a customizable and less-memory intensive version. Also developed was a frame output program which allows the user to segment the array into its respective Z axis components and view each ring sequentially (for the purposes of creating fly-through movies). Other backup software components that were written include a frame segmenter which takes the complete frame set of a video sequence and splices together a requested frame set. For example, if the frame set had 100 frames, and we wanted to view 1/5 of the sequence, the program would take every fifth frame and rename it in sequential order. Sequential order is required for the master program.
Results
The results obtained were the images seen in
the figures below. As can be seen, two images, a simulated
"good" pipe and a simulated "cracked" pipe, were used (as
created in Photoshop).
|
Good Pipe |
Cracked Pipe |
|
|
Running these images through the IDL (Interactive Data Language) program created in IDL 5.4, the data cube was created. Visualization, as explained above in methods, was done through Xvolume and through Xobjview.
|
Good Pipe - Xobjview |
Cracked Pipe - Xobjview |
 |
 |
As can be seen above, Xobjview provides a simple model layout of the datacube. However, the Xvolume widget allows for the application of grids, look-up-tables, and color segmentation (seen below).
At the onset of testing the images, the
transformations took upwards of thirty minutes; however, the code has been
optimized and allows for single frame or multi-frame analysis. While both
methods require large amounts of memory in order to do the processing, the
datacube for a single 500x500 frame is ~26million voxels, whereas a set of three
frames is upwards of 121 million voxels. The time to process the single
frame is about 3 minutes, and an increase of only 4 minutes has been observed
for the three-frame analysis; however, the time to display each remains
approximately the same at 2 minutes, 30 seconds for either the Xobjview or
Xvolume methods. In order to process and visualize the pipe, a swap file
of 389 MB was required.
From these results, the program was modified to
be more versatile and allow for different sized frames, and have better point
registration. The limiting factor to the system is the extensive amount of
memory and processing capabilities that would be required; the system could
easily take 30 minutes of video (54,000 frames), while the software is limited
to 30 frames of input per datacube constructed.
Discussion
The project was successful in more ways
than simply displaying a three dimensional recreation of the pipe. As a
result of having the majority of the programming done before the finalizing
date, designs and a simple model for the pipe scanner itself were created.
Also, a complete optical layout was designed, developed, and tested. While
the actual mechanics were not tested, nor was the system put into a pipe for
testing, all simulations were successful. Each of the three dimensional
representations give the expected pipe shape and color rendering; and it was
shown that by averaging across several frames with a velocity component, the
pipe has more detail, less noise, and a larger area. Fly-through
animations can be developed by segmenting the datacube, and, provided that the
image has enough contrast, cracks and flaws within the pipe can easily be
located. The application of color tables to the models was also
helpful in providing numerous ways to visualize the data and to determine
certain characteristics of the pipe.
Conclusion
All aspects of the research project except for
in-situ testing of the final product in a pipe scenario were fulfilled. Of
greatest importance was the creation of the transformation algorithm. Upon
presentation of this project to industry, it was determined that several other
applications could be made for this type of equipment and algorithmic
processing; namely, endoscopic medical imaging. As many people realize, pipe
systems play a major role in our everyday lives - the water that we drink, the
gas that we use in our cars, petroleum used for so many household goods - all
these things are transported by pipes, and even down to the veins, arteries, and
other "pipes" within our own bodies. The project was successful
in determining an adequate method of display for transformed frames, and it is
my hope and desire that such equipment be used for inspections as our pipes
begin to corrode, deteriorate, develop blockage, and generally age.