IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO.
3, MAY 1999
Object Detection Using Pulse Coupled Neural Networks
H. S. Ranganath and G. Kuntimad
Abstract This paper describes an object detection system
based on pulse coupled neural networks. The system is de-
signed and implemented to illustrate the power, flexibility and
potential the pulse coupled neural networks have in real-time
image processing. In the preprocessing stage, a pulse coupled
neural network suppresses noise by smoothing the input image.
In the segmentation stage, a second pulse coupled neural-network
iteratively segments the input image. During each iteration,
with the help of a control module, the segmentation network
deletes regions that do not satisfy the retention criteria from
further processing and produces an improved segmentation of the
retained image. In the final stage each group of connected regions
that satisfies the detection criteria is identified as an instance of
the object of interest.
Index TermsImage smoothing, image segmentation, iterative
segmentation, object detection, pulse coupled neural network,
pulse coupled neuron.
I. INTRODUCTION
I F a digital image is applied as input to a two dimensional
Eckhorns neural network, the network groups image pix-
els based on spatial proximity and brightness similarity [1].
During grouping, the network bridges small spatial gaps and
minor local intensity variations. This is a desirable property for
many window based image processing applications and can be
exploited for image processing applications.
When Eckhorns model was analyzed from image process-
ing perspective we noticed a few practical limitations [2].
Therefore, we modified Eckhorns neuron model and proposed
a new pulse coupled neuron (PCN) model for image processing
applications [1]. We eliminated the feeding leaky integrators,
assumed that all linking leaky integrators are identical, altered
the operation of the threshold signal generator and described a
method for determining the lower limit for the maximum value
for the threshold signal. In this model, every time a neuron
pulses its threshold signal is changed to a predetermined
maximum value regardless of its value just prior to the pulsing.
During the last two years, we have used pulse coupled neural
networks (PCNNs) for image smoothing, segmentation and
even feature extraction. A PCNN is a single layered, two-
dimensional, laterally connected network of pulse coupled
neurons. There exists a one-to-one correspondence between
the image pixels and network neurons. That means, each
pixel is associated with a unique neuron and vice versa.
Interested readers may find detailed description of PCNN and
its operation in our previously published papers [1].
Manuscript received January 28, 1997; revised January 3, 1999.
H. S. Ranganath is with the Computer Science Department, University of
Alabama, Huntsville, AL 35899 USA.
G. Kuntimad is with the Rocketdyne Division, Boeing North American,
Huntsville, AL 35806 USA.
Publisher Item Identifier S 1045-9227(99)03181-1.
10459227/99$10.00 1999 IEEE
615
The PCNN-based iterative image smoothing technique
is shown to smooth images without blurring, eroding, or
dilating edges. The thin lines and curves present in the
image that are often completely removed by the median
filter degrade gracefully in the new approach [3]. The
PCNN based segmentation method, under certain conditions,
guarantees perfect segmentation of the input image even
when the intensity ranges of adjacent regions overlap.
Perfect segmentation conditions are derived in [1]. The
inclusion of inhibition receptive field in the neuron model
significantly improves the segmentation capabilities of the
PCNN by compressing the intensity ranges of the individual
regions in the image and by reducing the extent of overlap
of the intensity ranges of adjacent regions. Simulation
results indicate that the PCNN yields better segmentation
results than most of the commonly used methods [2].
II. OBJECT DETECTION SYSTEM
A typical object or target recognition system consists of
a detection system and a recognition system. The detection
system finds potential objects in the input image and extracts
a subimage from each potential object area. These subimages
are further processed by the recognition system. This paper
describes an object detection system which utilizes PCNNs
for smoothing and segmentation of digital images.
A. System Organization
The object detection system, shown in Fig. 1, consists of
five major components: 1) smoothing module; 2) segmentation
module; 3) control module; 4) detection module; and 5)
knowledge base.
Any a priori information about the objects to be detected
can be stored in the knowledge base. However, it is desirable
to store the attributes that are easily computed from the
output of the PCNN. Moment based features such as area,
centroid, radius of gyration and Hus invariant moments can
easily be computed for each segment identified by the PCNN
[4]. The smoothing module, a PCNN, reduces the random
noise present in the image before presenting the image to
the segmentation module. The segmentation module is also
a PCNN. During each pulsing cycle, the segmentation module
forward the resulting segments to the control module [2]. The
control module, based on the computed attributes and the
attributes stored in the knowledge base, checks each segment
to determine whether the segment is acceptable as a part of
the desired object. The segments that are not acceptable are
eliminated from further consideration. This is accomplished by
disabling the group of neurons in the PCNN that correspond
616 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999

Fig. 2. Input image to the object detection system.

Fig. 3. Smoothed image of the tank. The tank in Fig. 2 was smoothed with
Fig. 1. Object detection system.
to the rejected segment. A neuron, once disabled, ceases to
pulse during the subsequent pulsing cycles. This completes the
current iteration. For the next iteration, the value of the linking
coefficient is updated and the image is resegmented.
The process of segmentation and elimination of segments
continues until the termination condition is attained. The
rules for updating and the termination condition are image
dependent. Note that the control module has the ability to
enable the previously disabled neurons. This feature provides
an elegant way to backtrack to recover from errors when
needed. The segments obtained during the final iteration are
presented to the detection module. The detection module
identifies each grouping of the input segments which satisfies
the constraints specified in the knowledge base as a potential
object of interest.
B. System Operation
In order to understand the operation of each module and the
overall system in detail, consider the task of detecting tanks
in input images. Several 128 128 gray scale images (256
levels) obtained from television camera were used as inputs to
the detection system. One such image is shown in Fig. 1. The
detection approach is described in steps below.
Step 1: The input image is smoothed by a PCNN to reduce
the effects of random noise.
Image smoothing using PCNN is accomplished by mod-
ifying the intensities of noisy pixels based on the neuron
firing patterns. In general, the intensity of a noisy pixel is
expected to be significantly different from the intensities of
the surrounding pixels. Therefore, neurons corresponding to
 = 0:02, r = 1:5 and C = 2 over 4 pulsing cycles.
most of the noisy pixels do not pulse with their neighbors.
In other words, neurons corresponding to noisy pixels neither
capture neighboring neurons nor are captured by them. Image
smoothing can be accomplished by adjusting the intensity of
each pixel so that the corresponding neuron either captures
its neighbors or is captured by them [2]. The intensity of each
neuron is adjusted when it pulses and the adjustment procedure
is described below.
When a neuron pulses, if a majority of the neurons
linked to it have not yet pulsed during the current pulsing
cycle, the intensity is adjusted down by subtracting
, where a predetermined constant.
When pulses and a majority of the neurons linked to
it have already pulsed during the current pulsing cycle,
the intensity is adjusted up by adding .
If there is no clear majority then the intensity is not
altered.
At the end of each pulsing cycle, the PCNN is reset, the
modified image is applied to the PCNN and the smoothing
process is repeated. The smoothing process continues until the
termination condition is attained. The Unidirectional termina-
tion condition is used [2]. The image in Fig. 3 is obtained
by smoothing the image in Fig. 2 with linking coefficient
, linking receptive field radius and .
The smoothed image of the tank is applied as input to the
segmentation module which is also a PCNN.
Step 2: The image is segmented into several regions using
a PCNN.
The general approach to segment images using PCNN is
to adjust the parameters of the network so that the neurons
corresponding to the pixels of a given region pulse together
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999
and the neurons corresponding to the pixels of adjacent regions
do not pulse together.
Assume that the image to be segmented consists of
regions and is applied as input to a PCNN. The network
neurons pulse based on their feeding and linking inputs. Note
that the feeding input to a neuron is equal to the intensity of
its corresponding pixel. Due to the capture phenomenon the
neurons associated with each group of spatially connected pix-
els with similar intensities tend to pulse together. Thus, each
contiguous set of synchronously pulsing neurons identifies a
segment of the image. The value of the linking coefficient
has significant effect on the segmentation process. If the
value of is low, the pixels that belong to a single region are
partitioned by the PCNN into several segments. This is known
as over-segmentation. On the other hand, if the value of is
high, the PCNN groups pixels that belong to two or more
regions to form a single segment. This is known as under-
segmentation. It may be possible to find the optimal value
of based on the intensity probability density function of
the image and the geometry of objects present in the image.
However, the determination and the use of the optimal value
of does not guarantee perfect results.
Since the determination of the optimal values of parameters
is not always possible, an iterative approach is used to segment
the image. In this approach, values of all parameters except
remain constant throughout the object detection process.
The PCNN begins the first iteration with a low value of
to ensure over-segmentation. Its value is increased slightly by
the control module for each subsequent iteration. The rationale
for choosing this approach is described later. At the end of a
pulsing cycle all the segments are presented to the control
module.
Step 3: Image segments to be discarded are determined.
The knowledge base includes information to facilitate the
evaluation of individual segments and the merging of adjacent
segments. The actual size and shape of the tank in the image
varies depending on the orientation, the imaging system and
the imaging geometry. Model based algorithms and special
purpose hardware are available for the real-time estimation of
the values of these attributes for any desired object. Therefore,
it is decided to store in the knowledge base 1) the maximum
and minimum sizes of the tank; 2) the maximum value of the
radius of gyration of segments; and 3) the maximum distance
between the centroids of adjacent segments that are candidates
for merging. The control module examines each segment and
discards segments that violate any of the following constraints.
Area constraint: The upper and lower limits of the area
of the tank in the image are determined by the imaging
system parameters and the distance between the tank and
the camera. If the area of a segment exceeds the upper
limit, it is unlikely to represent a tank. Therefore, such
large segments are eliminated from further consideration.
For the set of images used in simulation, the lower and
upper limits for the area of the tank were determined to
be 480 and 700 pixels, respectively.
Elongation constraint: In addition to its area, the elonga-
tion (spatial extent) of a segment is an important property.
The radius of gyration of a segment is a good measure
617
of the segments elongation. For a given area, radius of
gyration assumes its minimum value when the segment
is circular. Its value is maximum when all pixels are
collinear. A segment whose radius of gyration exceeds
the expected maximum value of the radius of gyration of
the tank cannot represent the tank or a part of the tank.
The maximum value of the radius of gyration of a single
segment (which is same as the radius of gyration of the
tank) was determined to be 14 pixels in length.
In order to eliminate a segment from further consideration
the feeding inputs (pixel intensities) of the corresponding
neurons are set to a unique negative value. This will also allow
the control module to enable the previously disabled segments
selectively.
If the number of active segments at the end of the current
iteration is less than a prespecified value (25 segments), the
segmentation process is terminated. The resulting segments
are input to the detection module for further processing. Oth-
erwise, the value of is increased by a constant amount and
the process of segmentation (Step 2) and segment elimination
(Step 3) is repeated.
The justification for gradually increasing the value of
starting from a low initial value is given in this paragraph.
It is assumed that there is some contrast between the regions
of tank and the surrounding background regions. If this is not
true the tank cannot be distinguished from the background by
any method. When the contrast between the tank and its sur-
rounding regions is low the risk of the tank or a part of the tank
pulsing with the background increases. Therefore, a low initial
value of is chosen to ensure that the tank segments do not
merge with the background segments due to increased linking
activity. During the early iterations, any segment that is larger
than a tank represents homogeneous background regions such
as sky and meadow. Similarly, large and elongated regions
represent features such as roads and rivers. Thus, they may be
eliminated from further consideration. After eliminating large
homogeneous regions, the image is fragmented and consists of
many less homogeneous regions. The value of is increased
to reduce fragmentation.
In the first iteration, the segmentation of the image in
Fig. 3 with and yielded 61 segments
as shown in Fig. 4(a). Six regions that did not satisfy the
size or elongation constraints were deleted. The black regions
in Fig. 4(b) represent the deleted regions. Note that more
than 50 percent of the pixels were eliminated from further
consideration. Since the number of segments was greater than
25 the image in Fig. 4(b) was segmented again. The value
of was increased by 0.03 at the end of each iteration. The
results of five iterations are summarized in Table I. Note that
the number of segments decreases as the processing continues
from one iteration to the next. During the fifth iteration only
one segment was deleted to yield a total of 21 segments as
shown in Fig. 4(c). Therefore, the segmentation process was
terminated.
Step 4: Detection of a group of connected regions that could
form an object of interest.
Each segment received by the detection module may be
a tank or a part of a tank. The function of the detection
618 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999

(a) (a) (b)

(b) (c) (d)

Fig. 5. Cluster seeking. The clusters that result from the segments in
Fig. 4(c). Number of segments (in addition to the root segment) in the clusters
are (a) 3, (b) 0, (c) 3, and (d) 5.

(based on the attributes in the knowledge base). After the

(c)
Fig. 4. Iterative segmentation. (a) Segments at the end of the first iteration.
(b) Segments remaining at the end of the first iteration after elimination by
the control module. (c) Segments remaining at the end of the final iteration.
TABLE I
RESULTS OF ITERATIVE SEGMENTATION
module is to select groups of segments that could constitute a
potential tank. This task is accomplished in two stepscluster
seeking and cluster analysis. First, in cluster seeking, the input
segments are grouped into several clusters so that segments
that belong to different clusters, together, cannot form a tank
clusters are formed, for each cluster, all combinations of
segments that satisfy the constraints given in the knowledge
base are detected as potential tanks (cluster analysis).
1) Cluster Seeking: There are ( ) nonempty subsets
that can be formed with input segments. In real-time or
near real-time image processing systems it is not feasible
to evaluate each subset to determine whether the subset can
constitute a tank. Therefore, it is necessary to reduce the
number of segment combinations that need to be analyzed.
This is accomplished by grouping the segments into several
clusters so that only the segments that belong to the same
cluster can be combined to form a tank. The method used to
form a cluster is described below.
The set of input segments are sorted in the nonincreasing
order of their areas.
The segment that has the largest area in the input set
forms the root segment of a new cluster.
A segment in the input set is included in the cluster of the
current root segment if the union of the segment and the
root segment does not violate the area, elongation and
separation constraints. The separation constraint states
that the distance between the centroids of the root and any
segment in the cluster must be less than the prespecified
value (40 pixels). The process continues till all the
segments in the input set are processed. This completes
the formation of a new cluster. Now, the root segment of
the cluster is deleted from the set of input segments.
The largest possible object that can be assembled from
the segments of a cluster consists of all the segments in
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999
(a) (b)
(c) (d)
(e) (f)
Fig. 6. Regions detected as potential tanks. (a)(d) Potential tanks in the
first cluster. (e) Potential tank in the second cluster. (f) Potential tank in the
third cluster.
the cluster. Therefore, if the sum of the areas of all the
segments in a cluster is less than the minimum area of
the tank, the cluster is eliminated.
If the sum of the areas of all the segments remaining in
the input set is less than the minimum area of the tank, the
clustering algorithm is terminated. Otherwise, the control
is passed to Step 2 to identify the next cluster.
It may be noted that the clusters do not form mutually
exclusive subsets of the input segment set. It is also possible
that a cluster may consist of only the root segment.
The cluster seeking algorithm processed the 21 segments of
the image in Fig. 4(c) and identified the four clusters shown
in Fig. 5(a)(d). In each figure the bright region represents the
root segment of the cluster. The boundary of each member
segment of the cluster is highlighted. There are three member
segments in the first cluster [Fig. 5(a)], zero member segments
in the second cluster [Fig. 5(b)], three member segments in
the third cluster (Fig. 5(c)) and five member segments in the
fourth cluster [Fig. 5(d)].
619
(a) (b)
(c) (d)
(e) (f)
Fig. 7. Regions that contain the tank. (a)(f) Potential tanks in the fourth
cluster.
2) Cluster Analysis: The union of every subset of segments
in a cluster constitutes a potential tank if the union satisfies
the following conditions.
The union must include the root segment. This eliminates
duplication by ensuring that any subset of segments that
can be formed from the segments of a cluster cannot be
formed from the segments of any other cluster.
The union of segments must satisfy area and elongation
constraints.
The union of segments must form a connected region.
The application of the above constraints to the four clusters
in Fig. 5 identified only 12 of 49 possible unions of segments
as potential tanks as shown in Figs. 6 and 7.
III. CONCLUSIONS AND RECOMMENDATIONS
The PCNN-based object detection system described in this
paper illustrates that the PCNNs can be easily integrated
into image processing systems. The iterative segmentation
approach, to a great extent, has alleviated the problem of
620
determining optimal value for the linking coefficient. The
use of very simple knowledge has succeeded in reducing the
number of regions that need to be processed by the recognition
system. The use of additional knowledge is likely to improve
the performance further.
REFERENCES
[1] H. S. Ranganath and G. Kuntimad, Image segmentation using pulse
coupled neural networks, in Proc. Int. Conf. Neural Networks, Orlando,
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999
FL, JuneJuly 1994, pp. 12851290.
[2] G. Kuntimad, Pulse coupled neural network for image processing,
Ph.D. dissertation, Dept. Comput. Sci., Univ. Alabama, Huntsville,
1995.
[3] H. S. Ranganath, G. Kuntimad, and J. L. Johnson, Pulse coupled neural
networks for image processing, in Proc. IEEE Southeastcon, Raleigh,
NC, Mar. 1995.
[4] M. K. Hu, Visual pattern recognition by moment invariants, IRE Trans.
Inform. Theory, vol. IT-8, pp. 179187, 1962.
[5] H. S. Ranganath and G. Kuntimad, Iterative segmentation using pulse
coupled neural networks, Applicat. Sci. Artificial Neural Networks II,
SPIE Aerosense, Orlando, FL, vol. 2760, pp. 543554, Apr. 1994.