A tracking technology of feature points fuzzy clustering algorithm

based on image stabilization

Qu Dan-dan*
a
, Zhao Yue-jin
a
,Wang Bin
a
, Hui Mei
a


a
Department of Optical Engineering, School of Information Science and Technology, Beijing Institute of
Technology, Beijing 100081, China

ABSTRACT
Object tracking technology combined with image stabilization is called tracking technology based on image
stabilization. Moving objects affect stabilization compensation in tracking algorithm based on image stabilization. The
middle value method, former background method and dynamic clustering method are not useful for rotate or non-rigid
objects. Fuzzy Clustering of feature points is proposed to solve these problems. First background and objectives
membership value of the pixel are calculated, and then pixels are classified to background and target categories
accurately by defining membership threshold of the background and targets. Experimental results show that fuzzy
clustering algorithm resolves the moving target interference problem in image sequence and realizes steady tracking of
object. It is proved to be more robust and less sensitive to the numbers of initial clustering.

Keywords: Information processingFuzzy clusteringImage stabilizationImage trackingNon-rigid object

1. INTRODUCTION
The requirement of observation and monitoring system in real-time is much higher in modern wars. Image and
video for observation in high quality is the necessary condition of improving the weapon power. The camera system on
working platform is difficult to stay stable because of the application environment. It causes vibration of image sequence.
This image sequence may cause bad effect of observation and the following processing such as image tracking,
recognition and analyst
[1][2]
. It needs image stabilization processing to avoid the disadvantage of camera system.
Digital image stabilization is a new technology. It detects the movement between frames using electronic
equipments and realizes image stabilization based on image process
[3]
. Clear and stable image sequences are obtained by
this way. The electrical stabilization system has several advantages. For example, size of digital image stabilization
system is small, the system design is simple and image stabilization system can be applied with tracking and recognition.
Object tracking technology combined with image stabilization is called tracking technology based on image stabilization
[4][5]
.
*s.qdd123bit@bit.edu.cn; phone 010-68912559; fax 010-68913658;

2008 International Conference on Optical Instruments and Technology: Advanced Sensor Technologies and Applications,
edited by Anbo Wang, YanBiao Liao, AiGuo Song, Yukihiro Ishii, Xudong Fan, Proceedings of SPIE Vol. 7157, 715718
2009 SPIE CCC code: 0277-786X/09/$18 doi: 10.1117/12.807611
Proc. of SPIE Vol. 7157 715718-1
2. METHODS TO SOLVING THE DISTURBING OF MOVING OBJECT
Moving objects usually appear in the background such as running persons, moving tanker, flicking leaves and so on.
The changes in the image affect the moving estimation in the computation of overall motion vector estimated. The
moving objects change the structure of the following frames. Motion vector estimation will be imprecise without process
of moving object. If the moving object comes into the searching area of partial motion vector estimation, stabilization
becomes object tracking. When moving objects leave from the image, matching error may appear because match points
can not be found. So the affection of moving objects must be eliminated to ensure the quality of stabilization. Now the
main solving methods include middle value method, former background method, dynamic clustering method, and so on
[6]
.
In the middle value method, moving objects sometimes exist in one or more sub-region of images, local motion
vectors obtained from this or these sub-regions should be excluded. Because the speed of the camera's movement should
be much lower than the frame rate, motion vectors between two frames caused by camera jitter should be similar. Based
on the assumption of camera system motion, the following methods are adopted by the local motion vector to determine
the overall motion vector and rule out the interference of objects movement to overall motion vector at the same time.
The entire image is divided into four regions, campaign of the local vector is calculated in each region. Global motion
vector is determined by local motion vector of sub-region and middle value of overall motion vectors of former image.
That is
{ }
1
4 3 2 1
, , , ,

=
t
g
t t t t t
g
V V V V V mean V (1)
Where
t t t t
V V V V
4 3 2 1
, , , is the local motion vectors of current frame,
1 t
g
V is overall motion vectors of former
image.
Although the above methods calculate with high reliability, the complexity increases by several times. Four
local motion vector estimations should be calculated even in the absence of moving objects situation,. So the efficiency
of the system is low. In former background method, image is divided into two parts, the background and the front ground.
Then they are divided into four regions. First calculate motion estimation in front ground. If the match degree does not
satisfy the matching degree R, calculate motion estimation in four regions of background in turn. Estimate the motion
vector every time and compare with R until find the appropriate motion vector. Definition of matching degree R is as
follows, see equation (2) where
SAD
S is the average value of all SAD

SAD
MAD
S
S
R =1 (2)
.Matching degree R is the reliability of the reference standard of each campaign estimate. Find the best match
position 0 =
MAD
S , then 1 = R .If
SAD MAD
S S = , 0 = R , this means that it is not matched completely,. Therefore,
when a moving object comes into the screen center, the structure of the matching template is changed at the same time. If
the matching degree is lower to threshold, it is regard as a failed match. Turn to the background area to make motion
estimates again.
Proc. of SPIE Vol. 7157 715718-2
Assess every estimate results, calculate matching degree R. If the search is finished in all regions and R does not
meet the threshold, this estimation is regarded as a failure. Motion compensation is not made, the calculation is stopped,
and the current frame is re-selected as a reference frame. This approach avoids that motion estimation is made repeatedly
in several regions, reduces the complexity and ensure the high credibility of the results at the same time.
Clustering is in accordance with certain requirements and rules of conduct to distinguish between things and
classification process. This process does not have any prior knowledge of the sub-category and the similarity between
things is a category-specific criteria. In dynamic clustering method clustering analysis is to study and deal with the
classification of a given object using mathematical methods. A sample set without classification is divided into a number
of sub-sets. Similar samples are classified as a class as far as possible. Different samples are divided into different
categories. If the interference rules out moving objects from this point of view, it belongs to the scope of cluster analysis.
Cluster analysis is a mathematical statistical method with study of how to classify things. The three key points are: a
selected sample as a measurement of the distance between the similarity measure; Identify criteria function of evaluation
which evaluates the cluster quality; Set the initial classification, calculate clustering iteratively and make criteria function
achieve extreme value. In the subject of this application, first calculate the average motion vector v of all motion
vectors and get the average value
b
v of all motion vectors which are smaller than v .Make it as the clustering center
of background feature points. Then get the average value
t
v of all motion vectors which are bigger than v and make it
as the clustering center of abnormal points. Take the sum of square error as the clustering criterion, that is

= =
+ =
n
j
t j t
m
i
b i b
v v v v S
1
2
1
2
(3)
Where m is the number of background feature points, n is the number of abnormal points, } {
i b
v is each sample of
background feature points, } {
j t
v is each sample of abnormal points. The clustering makes S reach minimum which is
the best result under the sum of square error criterion.
Assume move a sample
k
v from } {
i b
v to } {
j t
v , every clustering changes into }' {
i b
v and }' {
j t
v . The average
changes into
) (
1
1
'
) (
1
1
'
t k t t
k b b b
v v
n
v v
v v
m
v v

+
+ =

+ =
(4)
Calculate the sum of square error of two clustering separately
Proc. of SPIE Vol. 7157 715718-3
2
1
2
2
1
2
1
1
1
t k
n
j
t j t t
b k
m
i
b i b b
v v
n
v v S
v v
m
m
v v S

+
+ =

=
=
(5)
The new sum of square error is
2 2
1 1
1
b k t k t b
v v
m
m
v v
n
S S S S

+
+ = + = (6)
If
2 2
1 1
1
b k t k
v v
m
m
v v
n

<
+
(7)
Total sum of square error will become smaller. That is to say the distance between sample
k
v and } {
j t
v is
smaller than the distance between sample
k
v and } {
i b
v . Sample
k
v should be classed into } {
j t
v .
All above-mentioned three methods may be applied in image stabilization technology. Choice of threshold is very
important and to be resolved in latter two methods. These three methods are useful for parallel moving but not for rotate
or non-rigid object. So other methods need to be proposed in this situation. Fuzzy Clustering of feature points is
proposed to solve these problems. Fuzzy system may describe the changing, uncertainty and inexact information. It is
appropriate to decide the area edge through differentiation of membership. The study of Fuzzy Clustering is more and
more in-depth and is applied to many fields. It receives satisfying affection and is broadly applied in image processing
field such as image division, edge detecting, image enhancement, image compress, image smoothing, image matching,
and so on
[7][8]
. Fuzzy Clustering method is introduced as follow. It solves the disturbance of non-rigid object and achieves
overall motion vector estimation, and then the object feature points are obtained to realize tracking
[9]
.

3. USING THIS TEMPLATE AND ITS AUTOMATIC FORMATTING
Membership value which describes the degree of an object belonging to a set is defined in the paper. For example, if
an object x belongs to a set A, it is written as ) (x
A
and 1 ) ( 0 x
A
. 1 ) ( = x
A
means the object x belongs to the
set A and it is equal to A x . 0 ) ( = x
A
means the object x doesnt belong to the set A and it is equal to A x . The set
of the objects A whose membership is more than 0 in the space } {x X = is called the supporting set S(A) of fuzzy set A,
} 0 ) ( , , { ) ( > = x X x x A S
A
.The members of the supporting set are called supporting points of fuzzy set A. The certain
set is a special set of a fuzzy set because the values of membership function are either 0 or 1.
According to the figure of the dynamic clustering method, m + n feature points are written as
} , , 2 , 1 , { n m l v
l k
+ = L , the clustering center of base image is written as
b
v , clustering center of unusual points is
Proc. of SPIE Vol. 7157 715718-4
written as
t
v , their membership functions are written as ) (
l k b
v ) (
l k t
v then the loss function defined by the
membership functions is

+
=
+ =
n m
l
t l k l k t b l k l k b f
v v v v v v S
1
2 2
} )] ( [ )] ( {[

(8)
Where 1 > is a controllable constant of clustering fuzzy degree. The loss function defined by different membership
functions represents different fuzzy clustering methods. For example, in the condition of equation (9), the minimum of
the loss function is computed by differential.
n m l v v
l k t l k b
+ = = + , , 2 , 1 , 1 ) ( ) ( L (9)
Make the partial differential coefficients of
f
S about
b
v
t
v and ) (
l k t
v zeros. The conditions are

+
=
+
=
=
n m
l
l k b
n m
l
l k l k b
b
v
v v
v
1
1
)] ( [
)] ( [

(10)

+
=
+
=
=
n m
l
l k t
n m
l
l k l k t
t
v
v v
v
1
1
)] ( [
)] ( [

(11)
n m l v v
v v v v v
t l k
b l k b l k l k b
+ =
+ =

, , 2 , 1 ], ) / 1 (
) / 1 [( ) / 1 ( ) (
) 1 /( 1
2
) 1 /( 1
2
) 1 /( 1
2
L

(12)
n m l v v
v v v v v
t l k
b l k t l k l k t
+ =
+ =

, , 2 , 1 , ) / 1 (
) / 1 ( [ ) / 1 ( ) (
) 1 /( 1
2
) 1 /( 1
2
) 1 /( 1
2
L

(13)
According to the analysis, the calculating steps of the fuzzy clustering are as follows:
Set a parameter
Initialize the classification and calculating the clustering center of the base
b
v and the clustering center of unusual
points
t
v
Calculating the membership function using the current
b
v and
t
v according to equation (12)(13)
Calculating the clustering centers of base and unusual points according to equation(10)(11) , update
b
v and
t
v
at the same time.
Proc. of SPIE Vol. 7157 715718-5
Carry out the steps and iteratively until the change of membership function value is less than the threshold
( is a small positive number).
The iterative calculation is convergent and the membership function values of base clustering center, the
membership function values of unusual points and the feature points to base and unusual points are got. Some noise may
affect the results because the normalized condition (9). If a noise point is far from every kind of region, the membership
function values to each kind are small but constrict by equation (9), both the membership function values are 0.5. It may
affect the final results, so the condition is changed to:

+
=
+ = +
n m
l
l k t l k b
n m v v
1
) ( ) ( (14)
n m l
v v v v v v n m v
n m
l
t l k b l k t l k l k t
+ =
+ + =

+
=

, , 2 , 1
} ] ) / 1 ( ) / 1 [( /{ ) / 1 )( ( ) (
1
) 1 /( 1
2
) 1 /( 1
2
) 1 /( 1
2
L

(15)
Calculate according to the steps of the fuzzy clustering but the calculation of membership function according to
equation (14) and (15). The improved method has two advantages. If there are points of noise, the clustering results are
still very satisfying. On the other hands, the clustering results are not sensitive to the initial clustering number but
sensitive to the initial clustering center. If the clustering center is very near to a feature point, the clustering result
possibly contains only one feature point. At this condition a nonlinear process should be added to equation (14) and (15)
to ensure the distance is no less than a constant.

4. EXPERIMENTAL RESULTS AND ANALYSIS
Image sequence which contains 700 images is used in the experiment. The image size is 320240. The camera is
fixed on a platform and shaking on the way because the way is rocky. The image sequence shakes at random. The
tracking target is a walker who is walking on the way. In the stabilization steps the motion of non-rigid body should be
worked off. In the experiment fuzzy clustering is used to process and analyze the image, here 2 = . There is no obvious
difference between background motion vectors and target motion vectors. In order to avoid the disturbing of noise points
and improve the success rate of distinguishing, the threshold of membership function is 0.6, which is analyzed below.
The pixels whose membership function is between 0.4 and 0.6 are deleted. After finding the target area, set the gray
value of other area to 0. The gray value of the pixels whose gray value is 0 is set to 255 in order to emphasize the target.
Find the centroid of the target. Make a rectangular tracking window whose center is the centroid of the target. The
distance between the centroid and the farthest pix is the axis of the tracking window.
Figure 1 is two frames of an image sequence. The edge area of the image is black. It stands for the area which needs
to be compensated. The area contains 30 pixels in the experiment. Calculate the membership function of the reference
frame according to equations (10)~(12). Figure 2 is the membership function distribution. The difference of the
membership function is not obvious because the target is non-rigid body. The area whose membership value is more than
0.6 is defined as moving target. The area whose membership value is less than 0.4 is defined as background and is
deleted. Estimate the global moving parameter using the background feature points and track using the target feature
points at the same time. In this way tracking based on image stabilization is realized, see Figure 3
Proc. of SPIE Vol. 7157 715718-6
' I ' I I I
i b
I I I I
. i i
k I
- .
2 0
i s
2 3 0


Fig. 1 Original images

.
Fig. 2 Distribution of membership value


Fig. 3 Processed images
There are 120 effective feature points in each picture. About 15 feature points on moving object are found. Calculate
the centroid of the moving object and then draw the tracking windows. If the upper limit of the membership value is 0.8,
there are fewer than 5 feature points on moving object will be found. It is not enough to identify the centroid of the
moving object exactly. Dynamic clustering method is not applicable to calculate tracking parameters in this situation.
Membership values of all image sequence are calculated and the numbers of feature points on moving object are
obtained. The results of the frames are listed in Table 1. With the limit of the membership value increasing, the number
of feature points on the moving object becomes small and the correct rate of feature points becomes high. But the
precision of the moving object centroid becomes high firstly and then becomes low. The precision of the moving object
Proc. of SPIE Vol. 7157 715718-7
centroid is determined by the distance between calculated centroid and real centroid. When the limit of the membership
value is 0.6, the precision of the moving object centroid is highest. So the limit of the membership value is set to 0.6.
Table. 1 The numbers of feature points on moving object based on different limit of the membership value
limit of the
membership value
The numbers of feature
points on moving object
Correct rate (%)
Precision of the
moving object
centroid (pixels)
0.4 30 66 5
0.5 19 82 2
0.6 15 93 1
0.7 12 100 2
0.8 5 100 6

5 CONCLUSIONS
Feature points fuzzy clustering method is proposed in this paper to solve the disturbance of the moving object in the
tracking based on image stabilization. Different membership value is chosen according to vectors of non-rigid bodys
different part and degree of object and background rotation. Find out the needed feature points of moving object and
work off the disturbance of moving object to realize steadily tracking of object stabilization. The experimental results
show that the fuzzy clustering method solves the disturbance of non-rigid bodys moving in the image sequence. This
method is proved to be more robust and less sensitive to the numbers of initial clustering.
REFERENCES
[1] Han S K, Zhao Y J, Liu M Q, "Electronic image stabilization techniques and development," Optical Technique,
Papers 27(1), 71-73(2001).
[2] Shen H H, Liu J P, Jia P, "Overview of imaging stabilization ," Optics and Precision Engineering, Papers 9(2),
115-120(2001).
[3] Yang C Q, Wang X T, Xu X G,, "Novel image stabilization technology based on soft-platform ," ACTA
ARMAMENTARII, Papers 27(3) 471-474(2006).
[4] Wang B, Zhao Y J, YIN D S, "Image stabilization algorithm for digital image tracking," Proc. SPIE 6623,
66230Y-1-8 (2007).
[5] J. Dale, D. Scott, D. Dwyer, J.Thornton, "Target tracking, moving target detection, stabilization and enhancement of
airborne video ,"Proc. SPIE 5787, 154-165(2005).
[6] Zhong P, Feng J L, "Between frames of dynamic image sequences," Optical Technique, Papers 29(4)
441-444(2003).
[7] Yang R L, Gao X B, Jie J, "A fast image segmentation algorithm based on fuzzy clustering," Xian Univ. Arch. &
Tech, Papers 39(2), 280-285(2007).
[8] Cheng T, He Z S, "Novel multiple targets tracking algorithm based on fuzzy clustering ," Systems Engineering
&Electronics, Papers 28(9), 1332-1334, (2006),.
[9] Wang H W, Li C H, Shi H, "Moving object detection algorithm in video sequences," ACTA ARMAMENTARII,
Papers 27(3) , 446-450 (2006).
Proc. of SPIE Vol. 7157 715718-8