J an. 2011, Volume 5, No.1 (Serial No. 38), pp.

39-47
J ournal of Civil Engineering and Architecture, ISSN 1934-7359, USA

Digital Image Processing Methods for Bridge Coating
Management and Their Limitations
Sangwook Lee
Department of Construction Engineering and Engineering Technology, Texas Tech University, Lubbock, TX 79424, USA

Abstract: Accurate and objective rust defect assessment is required to maintain good quality steel bridge coating surfaces and make a
decision whether a bridge shall completely or partially be repainted. For more objective rust defect recognition, digital image
recognition methods have been developed for the past few years and they are expected to replace or complement conventional painting
inspection methods. Efficient image processing methods are also essential for the successful implementation of steel bridge coating
warranty contracting where the owner, usually a state agency, and the contractor inspect steel bridge coating conditions regularly and
decide whether additional maintenance actions are needed based on the processed data. There are two approaches to develop automated
rust defect recognition methods: applying a statistical method or an artificial intelligence technique. This paper presents the application
of previously developed image processing methods for defect evaluations on a bridge coating surface and discusses their limitations
under three environmental conditions which are often encountered while acquiring digital images.

Key words: Bridges, deterioration, image processing, condition assessment.

1. Introduction


Digital image processing has been applied to
diverse industry disciplines. Recently, in the civil
engineering domain, digital image recognition
methods have been utilized in, but not limited to, steel
bridge coating inspection, underground pipeline and
pavement condition assessments, and construction
material inspections. The wide application of digital
image processing can be attributed to the following
advantages: accuracy, objectivity, speed, and
consistency. These distinct advantages will facilitate
existing inspection methods to be replaced or
supplemented by advanced infrastructure inspection
methods.
For instance, the conditions of steel bridge painting
surfaces can be evaluated accurately and quickly by

Sangwook Lee, PhD, assistant professor, research fields:
applications of emerging technologies for more efficient
infrastructure management, bidding strategy and bid-markup
decision, development of construction warranties for better
maintenance of built environments, greener construction
methods and materials for sustainable development, Enterprise
Resource Planning (ERP) systems in construction industry.
E-mail: goswlee@yahoo.com.
applying digital image processing. Also, machine
vision-dependent inspections can provide more
consistent inspection results than human visual
inspections. Conventional inspection results can be
highly dependent on personal preferences, familiarity
with the work, or the workload of the inspectors.
Reliable coating condition assessment methods are
necessary so that bridge managers can develop
long-term cost-effective maintenance programs and
make decisions as to whether a bridge shall be
completely or partially repainted immediately or later.
With these goals in mind, digital image recognition
methods have been developed for objective rust defect
recognition in the past few years. Rust defects are one
of the most commonly observed defects on coating
surfaces and can severely affect the structural integrity
of bridges.
Efficient rust defect recognition methods are also
essential for the successful implementation of steel
bridge coating warranty contracting where the owner,
usually state agencies, and the contractor inspect steel
bridge coating conditions on a regular basis and
decide whether additional maintenance actions are
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

40
needed. Indiana Department of Transportation
(INDOT) has tentatively set up the maximum
warranty period of five years and the maximum
allowable rust percentage of 0.3% within a total steel
structure. If the painting rust defect percentages on
steel bridge surfaces are estimated as less than or
equal to 0.3% within a five year warranty period, the
painting work will be accepted; otherwise, the
contractor must conduct repair work before
terminating warranty contracts. However, it is
extremely difficult, if not impossible, for the naked
eye to determine if the rust percentage is above or
below 0.3%. Under the conventional visual inspection
method, even two experienced paint inspectors may
come up with different defect percentages and rate the
condition differently. Thus, efficient digital image
recognition methods need to be developed to
complement existing inspection methods, which can
provide an effective and convincing means to assess
the condition of steel bridge coating.
There are two approaches to develop automated rust
defect recognition methods: applying a statistical
method or an artificial intelligence technique. This
paper selected two image recognition methods under
each category: the NFRA (Neuro-Fuzzy Recognition
Approach) method and the SKMA (Simplified
K-Means Algorithm) method (1 & 2). The NFRA
method uses artificial intelligence techniques to
separate rust pixels from background pixels. The
SKMA method segments object pixels and
background pixels in a digitized image using a
statistical method, called the K-means algorithm. Even
if both methods pass through different processing
procedures, one common thing is that they first
convert original color images to grayscale images and
further process the grayscale images.
This paper presents the application of previously
developed image processing methods for painting rust
defect evaluations and discusses their limitations under
several specific environmental conditions which are
often encountered while acquiring digitized images.
These environmental conditions include non-uniform
illuminations, low-contrast digital images, and noises
on coating surfaces.
2. Rust Defect Recognition Methods
2.1 Digital Image Processing Procedures
To identify rust defects in a bridge coating image
using digital image processing techniques, the images
must pass through a linear procedure consisting of
several linked steps. Fig. 1 shows the framework of
digital image processing procedures for bridge paint
condition assessment.
The first step is data acquisition, where digital
images of steel bridge painting components that need
to be examined are obtained using a digital camera.
Then, the acquired images are transferred to the


Fig. 1 Digital image processing procedures.
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

41
computer on-site or in an office for data storage and
processing. The next step is preprocessing the
transferred images. Images can be cropped to capture
target areas as planned. The size of images can be
adjusted because a personal computer has processing
limitations for multiple large-size images, and
acquired images consume a huge amount of storage
space. Image enhancement techniques may be
required for improving the quality of digital images,
such as the contrast and/or brightness control.
Preprocessed digital images are converted to grayscale
images before moving to a main processing stage.
Grayscale is expressing a digital image using light
intensities on a 0 to 255 scale, where usually 0 means
black and 255 represents white. The next stage is a
main processor, which consists of applying the
developed recognition methods, such as the NFRA or
the SKMA method. Throughout this stage, given
painting images are expressed as binary images that
contain only two colors, black and white. Bridge
coating image processors divide a whole image into
two groups or clusters, the background and the rust
defects. Object (rust) pixels are expressed as black in
color, and background pixels are expressed as white.
The rust percentage can be calculated by counting the
black pixels among all pixels. Finally, the processed
results are stored in a computer and analyzed and
monitored for the bridge painting condition
assessment. Fig. 2 describes the schematic procedures
of bridge coating image processing.
2.2 NFRA Method
The NFRA method integrates the artificial neural
networks and the fuzzy adjustment system to resolve
the recognition problems arising from non-uniformly
illuminated images (1). After acquiring a digital image
and converting it to grayscale, the NFRA segments the
grayscale image into three areas according to the
illumination values of the pixels in that image.
Illumination values are expressed as a number
between 0 and 1. The darkest pixel becomes 0 and the
brightest becomes 1. The average illumination values
of three areas are computed and serve as the input to
pre-trained artificial neural networks (ANN). Once
input values are provided to the artificial neural
networks, three threshold values are generated for the
three areas as shown in Fig. 3.
The fuzzy adjustment system is utilized to adjust
the gray level values of the image pixels along the
boundaries according to fuzzy if-then rules. Finally,
image thresholding is performed to produce a binary
image containing only the object and the background
pixels. Each area is thresholded according to its
corresponding threshold value computed from a
previous stage. Pixels with gray level values smaller
than the threshold values are considered as rust defects


Fig. 2 Schematic diagram of bridge coating image processing ((a) original coating image; (b) computerized image processor;
(c) binary processed image; (d) processed rust percentage).
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

42
Three Illumination
Values
Three Threshold
Values
Pre-Trained Neural
Network
99
106
112
0.44
0.51
0.59
1
4
5
3
2 7
6
X
1
X
2
Y
6
Y
7

Fig. 3 Artificial neural network operations.

Fig. 4 Architecture of NFRA method.

and pixel values higher than the threshold values are
considered as the background. The rust percentage can
be calculated by the ratio of black (defect) pixels to all
pixels. If image pixels are expressed as F(i, j) and
defect pixels are D(i, j), then rust percentage (RP)
becomes,
( , )
(%) 100
( , )
D i j
RP
F i j
= , for all i, j (1)
Fig. 4 shows the architecture of the NFRA method
and Fig. 5 illustrates a processed result of the NFRA.
2.3 SKMA Method
The SKMA method segments a grayscale image
into two groups or clusters in a statistical method
using so-called K-means algorithm (2). The K-means
algorithm self-organizes data to create predetermined
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

43
clusters or classes. In the case of bridge coating
surface assessment, two classes are required: defective
areas and non-defective areas. Clustering requires
iterative processes to effectively separate rust pixels
from background pixels. First, the gray levels of the
first two pixels in an image are assigned to be the
centers of two clusters. The K-means algorithm then
assigns each of the remaining pixels to one of the two
clusters based on the Euclidean distances from each
point to the cluster centers. Then, the sample vectors
in each cluster are averaged to produce new cluster
centers. Next, each of the sample vectors is reassigned
to the class to which the new representative center is
closest. The iterative process is continued until the
sum of the squared distances from all points in a
cluster domain to the cluster center is minimized (3).
Fig. 6 shows the clustering process by the K-means
algorithm.
Light intensity values or the gray level values of an
image are distributed on a 0 to 255 scale as shown in
Fig. 6(a). Rust defects usually feature darker values
than background areas, which makes grayscale
distributions bimodal. A threshold value is selected
between two modes, rust and background, so that
processing errors are minimized (Fig. 6(b)).
Once the process terminates, the SKMA method
generates a thresholded binary image which yields the
processed results. From the binary image, rust
percentages can be calculated by computing the defect
pixels out of all pixels in the image. Fig. 7 shows the
architecture of the SKMA method and Fig. 8 shows the
processed results.


Fig. 5 NFRA processed results ((a) original coating image;
(b) binary processed image).

Fig. 6 Clustering process: (a) frequency of intensity values;
(b) image thresholding.


Fig. 7 Architecture of SKMA method.
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

44

Fig. 8 SKMA processed results: (a) original coating image;
(b) binary processed image.
3. Limitations for Bridge Coating
Inspections
3.1 Problematic Environmental Conditions
Although several rust defect assessment methods
were developed in the past few years to evaluate
bridge painting surfaces more objectively, they still
have limitations when processing digitized images
taken under several environmental conditions, which
include: non-uniform illuminations, low-contrast
digital images, and noises on painting surfaces. These
situations are often experienced during bridge painting
inspection or image acquisition and dealing with them
is not an easy task when developing computerized
programs. The following three examples show coating
images related to these situations and the processed
results by the NFRA and the SKMA methods.
Fig. 9 illustrates non-uniform illumination conditions,
which may happen due to the fact that pavement,
surrounding trees, and passing vehicles reflect
sunlight irregularly. Fig. 9(a) is an original color rust
defect image acquired under non-uniform illumination
conditions. Several spots of rust defects are located in
the lower left corner. If the reason is unclear as to the
non-uniform illumination of the image, it may be
better to observe Fig. 9(b) which is a grayscale image
of the color image. It is noticeable that light intensities
on the right side are darker than those on the left side
of the image.


Fig. 9 Non-uniform illumination: (a) original coating image; (b) grayscale image; (c) processed image by NFRA method; (d)
processed image by SKMA method.

Digital Image Processing Methods for Bridge Coating Management and Their Limitations

45

This image was processed using two image
processing methods: NFRA and SKMA methods. Fig.
9(c) shows the processed results by the NFRA method
and Fig. 9(d) is the processed results by the SKMA
method. As can be seen from Figs. 9(c) and 9(d), both
methods failed to produce satisfactory results when
the results are compared to the original color image.
Fig. 10 is an example of low-contrast rust images
where rust pixels are not very distinct against the
background. Figs. 10(c) and 10(d) show the processed
results by the NFRA method and the SKMA method,
respectively. The processed image by the SKMA
method recognized the rust areas too intensely and the
processed image by the NFRA method recognized
almost nothing. This example implies that dealing
with low-contrast images is not a simple matter. Chen
and Chang [1] also indicated that the contrast between
rust defects and a background significantly affects the
accuracy of the processed results and sharp contrast
normally generates better recognition results.
Fig. 11 shows an example of noises on painting
surfaces. Steel bridge painting surfaces are often
stained with foreign materials such as accumulated
dirt or the remains of small worms. Thus, developing
noises from a background is an important issue. Fig.
11(a) is a coating image containing noises on the
surfaces where noises can be easily observed from the
color image. Figs. 11(c) and 11(d) shows the
processed result by the NFRA and SKMA methods.
Both methods recognized rust defects and noises on
the surfaces at the same time with a similar pattern,
but the degree of recognition areas is somewhat
different. The SKMA method recognized more noises
than the NFRA method.
3.2 Findings and Discussions
Previous image processing examples demonstrated
that the NFRA and the SKMA methods failed to
generate reliable results under specific environmental
conditions. Even if the NFRA and the SKMA
programs pass through different processing procedures,
one common thing is that they first convert original
color images to grayscale images and further process
the grayscale ones. Unsatisfactory processed results
may be related to grayscale image processing.
Grayscale images do not contain any information on
color and express light intensities on a 0 to 255 scale.
Such grayscale images contain inherent limitations to
separate rust defects on steel bridge surfaces under the
problematic environmental conditions. For example,
in the case of noises on painting surfaces, once
original color images are converted to grayscale
images, light intensities of rust defects become similar
with those of noises. Those light intensities then are
mixed, which makes the developed image processing
methods have more difficulty to separate them
efficiently. Likewise, under the non-uniform
illuminations, after original color images are
transformed into grayscale images, rust defects and
low-illuminated areas do not make a big difference in
terms of light intensities, while the rust defects are
visibly distinct in the color images. These facts
hindered the NFRA and the SKMA methods from
differentiating rust defects reliably.
These findings formulated research motivations on
the application of digital color image processing for
more advanced rust defect assessment methods. Rust
defects can be easily identified when looking at color
images rather than grayscale images. Viewing and
identifying color are originally a natural and powerful
human sense. Since humans can perceive color, they
can drive a car, watch a movie, distinguish similar
objects by color, and enjoy four seasons. Returning to
a steel bridge coating inspection, a bridge coating
inspector can recognize the existence of rust defects
by color as the defects on steel surfaces often appear
reddish or brownish, which characteristically are
distinctive against the paint background.
Color images, generally speaking, contain more
information than grayscale images. Color images are
represented by three-dimensional spaces. For instance,
the RGB color space, the most fundamental color
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

46


Fig. 10 Low-contrast images: (a) original coating image; (b) grayscale image; (c) processed image by NFRA method; (d)
processed image by SKMA method.


Fig. 11 Noises on painting surfaces: (a) original coating image; (b) grayscale image; (c) processed image by NFRA method;
(d) processed image by SKMA method.
Digital Image Processing Methods for Bridge Coating Management and Their Limitations

47

space, is expressed as a cube as shown in Fig. 12. The
origin of the cube corresponds to black, and the point
with (1,1,1) represents a white color. The line
connecting black and white is called a gray-scale line,
on which every point represents gray colors with
different intensities. The three primary colors of red,
green, and blue are located in each primary axis.
Many different colors can be created depending on the
proportion of these three color channels. For instance,
the addition of red and green in the same proportion
generates yellow and the addition of blue and green
creates cyan. Thus, it can be possible to develop more
efficient defect assessment methods to deal with such
problematic environmental conditions by investigating
digital color spaces.


Fig. 12 RGB color space.
4. Conclusions
An accurate and objective rust defect assessment is
required to maintain a good-quality steel bridge
painting surfaces and make a decision whether a
bridge shall completely or partially be repainted. For
more objective rust defect recognition, digital image
assessment methods have been developed for the past
few years. Efficient image processing methods are
also essential for the successful implementation of
steel bridge coating warranty contracting where the
owner, usually a state agency, and the contractor
inspect steel bridge coating conditions regularly and
decide whether additional maintenance actions are
needed based on the processed data.
This paper introduced two rust defect assessment
methods and explained their theoretical backgrounds
and application examples. The image recognition
methods explained here are the NFRA and the SKMA
methods. The NFRA method uses artificial intelligence
techniques to separate rust pixels from background
pixels. The SKMA method segments object pixels and
background pixels in a digitized image using a
statistical method, called the K-means algorithm.
This paper also showed that the developed rust
defect assessment methods have some limitations when
processing digitized images taken under specific
environmental conditions. These environmental
conditions include: non-uniform illuminations,
low-contrast digital images, and noises on painting
surfaces. Those situations are often experienced during
bridge painting inspections and have to be overcome to
facilitate the automation of steel bridge coating
inspection. To deal with those environmental
conditions effectively, defect recognition methods
using digital color imaging system need to be
considered. Since color images basically provide more
information than grayscale images, it may be possible
to develop more efficient defect recognition methods
by investigating digital color information.
References
[1] P. H. Chen and L. M. Chang, Intelligent steel bridge
coating assessment using neuro-fuzzy recognition
approach, Computer-Aided Civil and Infrastructure
Engineering 17 (5) (2002) 307-319.
[2] L. M. Chang, P. H. Chen and S. Lee, Implementation of
steel bridge protection policy, FHWA/IN/J TRP-2001/29,
J oint Transportation Research Program, Purdue University,
West Lafayette, Indiana, Final Report, Oct. 2003.
[3] C. G. Looney, Pattern Recognition Using Neural
Networks, Oxford University Press, Inc., New York, 1997