See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/221001479

FGKA: a Fast Genetic K-means Clustering

Algorithm.

Conference Paper January 2004

DOI: 10.1145/967900.968029 Source: DBLP

CITATIONS READS

97 238

5 authors, including:

Shiyong Lu Susan J Brown

Wayne State University Kansas State University
108 PUBLICATIONS 1,280 CITATIONS 388 PUBLICATIONS 6,857 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Shiyong Lu on 18 March 2017.

The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document
and are linked to publications on ResearchGate, letting you access and read them immediately.
 POSTER ABSTRACT
FGKA: A Fast Genetic K-means Clustering Algorithm
Yi Lu, Shiyong Lu, Farshad Fotouhi
Department of Computer Science
Wayne State University
Detroit, MI 48202, USA
{luyi, shiyong, fotouhi}@cs.wayne.edu
ABSTRACT
In this paper, we propose a new clustering algorithm called Fast
Genetic K-means Algorithm (FGKA). FGKA is inspired by the
Genetic K-means Algorithm (GKA) proposed by Krishna and
Murty in 1999 but features several improvements over GKA. Our
experiments indicate that, while K-means algorithm might
converge to a local optimum, both FGKA and GKA always
converge to the global optimum eventually but FGKA runs much
faster than GKA.
Categories and Subject Descriptors
H.2.8 [Database Management]: Database Applications Data
mining.
General Terms
Algorithms, Measurement, Performance, Experimentation.
Keywords
Clustering, Genetic algorithm, K-means algorithm
1. INTRODUCTION
In recent years, clustering algorithms have been effectively
applied in molecular biology for gene expression data analysis.
By clustering algorithms such as K-means, hierarchical
clustering, SOM, genes are partitioned into groups based on the
similarity between their expression profiles. In this way,
functionally related genes are identified. As the amount of
laboratory data in molecular biology grows exponentially each
year due to advanced technologies such as Microarray, new
efficient and effective clustering methods must be developed to
process this growing amount of biological data.
In this paper, we propose a new clustering algorithm called
Fast Genetic K-means Algorithm (FGKA). FGKA is inspired by
the Genetic K-means Algorithm (GKA) but features several
improvements over GKA. One salient feature of FGKA (as well
as GKA) is that it will always converge to the global optimum,
whereas the K-means algorithm might converge to a local
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
SAC04, March 14-17, 2004, Nicosia, Cyprus.
Copyright 2004 ACM 1-58113-812-1/03/04$5.00.
Youping Deng, Susan J. Brown
Division of Biology
Kansas State University
Manhattan, KS 66506, USA
{ydeng, sjbrown}@ksu.edu
optimum. To demonstrate the effectiveness and efficiency of
FGKA, we performed a comparative study on gene expression
data analysis using FGKA, GKA and K-means respectively. Our
experiments indicate that both FGKA and GKA have better
optimum convergence than K-means algorithm, but FGKA runs
much faster than GKA. We proposed FGKA in the context of
clustering gene expression data. However, the algorithm itself is a
generic clustering method that can be applied to other domains as
well.
2. THE OBJECTIVE FUNCTION
The gene expression data for clustering consists of N genes and
their corresponding N patterns. Each pattern is a vector of D
dimensions recording the expression levels of the genes under
each of the D monitored conditions or at each of the D time
points. The goal of FGKA algorithm is to partition the N patterns
into user-defined K groups, such that this partition minimizes the
Total Within-Cluster Variation (TWCV, also called square-error
in the literature), which is defined as follows.
Let X1, X2,, XN be the N patterns, and Xnd denotes the dth
feature of pattern Xn (n=1N). Each partitioning is represented
by a string, a sequence of numbers a1aN, where an takes a value
from {1, 2,, K} representing the cluster number that pattern Xn
belongs to. Let Gk denote the kth cluster and Zk denote the number
of patterns in Gk. The Total Within-Cluster Variation (TWCV) is
N D K 1 D
defined as TWCV = X nd 2 SF kd 2 where
n = 1 d =1 k =1 Z k d =1
SFkd is the sum of the dth features of all the patterns in Gk. Due to
space limit, we omit the induction and readers are referred to [3]
for details.
3. FAST GENETIC K-MEANS ALGORITHM
FGKA maintains a population (set) of Z coded solutions
(partitions), where Z is a parameter specified by the user. Each
solution is coded by a string a1aN of length N. Given a solution
Sz = a1aN, we define the legality ratio of Sz, e(Sz), as the number
of non-empty clusters in Sz divided by K. Sz is legal if e(Sz)=1,
and illegal otherwise.
FGKA starts with the initialization phase, which generates
the initial population P0. The population in the next generation
Pi+1 is obtained by applying the following genetic operators
sequentially: the selection, the mutation and the K-means operator
on the current population Pi. The evolution takes place until the
termination condition is reached.
The initialization phase randomly generates the initial
population P0 of Z solutions which might end up with illegal
strings. At first sight, illegal strings are undesirable. For this
 reason, the GKA algorithm [2] makes significant effort to
eliminate illegal strings. Illegal strings, however, are permitted in
FGKA, but are considered as the most undesirable solutions by
defining their TWCVs as + and assigning them with lower
fitness values. Our flexibility of allowing illegal strings in the
evolution process avoids the overhead of illegal string
elimination, and thus improves the time performance of the
algorithm. In the following, we give a brief description of the
three genetic operators.
3.1 The Selection Operator
We use the so-called proportional selection for the selection
operator in which, the population of the next generation is
determined by Z independent random experiments. Each
experiment randomly selects a solution from the current
population {S1, S2,, Sz} according to the probability distribution
F ( Sz )
{p1,p2,,pZ} defined by pz = ( z = 1, L , Z ), where
Z
z =1 F ( Sz )
F(Sz) denotes the fitness value of solution Sz. In our context, the
objective is to minimize the Total Within-Cluster Variation
(TWCV). Therefore, solutions with smaller TWCVs should have
higher probabilities for survival and should be assigned with
greater fitness values. In addition, illegal strings are less desirable
and should have lower probabilities for survival, and thus should
be assigned with lower fitness values. We define F(Sz) as follows,
1.5 * TWCV max TWCV ( Sz ), if Sz is leagal
F ( Sz ) =
e( Sz ) * F min, otherwise
where TWCVmax is the maximum TWCV that has been
encountered till the present generation, Fmin is the smallest fitness
value of the legal strings in the current population if they exist,
otherwise Fmin is defined as 1.
3.2 The Mutation Operator
The mutation operator performs the function of shaking the
algorithm out of a local optimum, and moving it towards the
global optimum. During mutation, we replace an by an for
n=1,,N simultaneously. an is a cluster number randomly
selected from {1,,K} with the probability distribution
{p1,p2,,pK} defined by
1 .5 * d max ( X n ) d ( X n , c k ) + 0 .5
pk = K Shalon, D. Botstein, and P. Brown. The transcriptional
k =1 (1 . 5 *d max ( X n ) d ( X n , c k ) + 0 .5)
where d(Xn,ck) is the Euclidean distance between pattern Xn and
the centroid ck of the kth cluster, and
d max( Xn) = max {d ( Xn, ck )} . If the kth cluster is empty, then
k Cybernetics, 29(3):433439, June 1999.
d(Xn,ck) is defined as 0. The bias 0.5 is introduced to avoid divide-
by-zero error in the case that all patterns are equal and are
assigned to the same cluster in the given solution.
First of all, the above mutation operator ensures that an
arbitrary solution, including the global optimum, might be
generated by the mutation from the current solution with a
positive probability. Second, it encourages that each Xn is moving
towards a closer cluster with a higher probability. Third, it
promotes the probability of converting an illegal solution to a
legal one.
View publication stats
3.3 The K-means Operator
In order to speed up the convergence process, one step of the
classical K-means algorithm, which we call K-means operator
(KMO) is introduced. Given a solution that is encoded by a1aN,
we replace an by an for n=1,,N simultaneously, where an is the
number of the cluster whose centroid is closest to Xn in Euclidean
distance.
To account for illegal strings, we define d(Xn,ck) = + if the
kth cluster is empty. This definition is different from section 3.2,
in which we defined d(Xn,ck) = 0 if the kth cluster is empty. The
motivation for this new definition here is that we want to avoid
reassigning all patterns to empty clusters. Therefore, illegal string
will remain illegal after the application of KMO.
4. EXPERIMENTAL RESULTS
Our experiments were conducted on a Dell Dimension 8200 PC
machine with 2.4G Hz CPU and 512M RAM. Three algorithms,
the K-means algorithm, the original GKA algorithm and our
proposed FGKA algorithm were implemented in C language with
Microsoft Visual Studio tool. Two data sets are used to conduct
our experiments, the fig2data [1] and chodata [4].
The experiments show that K-means algorithm might
converge to a local optimum, both FGKA and GKA always
converge to the global optimum but FGKA runs almost 20 times
faster than GKA. It also shows that three improvements: efficient
calculation of TWCVs, avoiding illegal string elimination
overhead, and the simplification of the mutation operator have
different improvement impact factor over GKA. More details are
available in [3].
5. CONCLUSIONS
In this paper, we propose a new clustering algorithm called Fast
Genetic K-means Algorithm (FGKA). FGKA is inspired by the
Genetic K-means Algorithm (GKA) [2] but features several
improvements over it, including efficient calculation of TWCVs,
avoiding illegal string elimination overhead, and the
simplification of the mutation operator. The initialization phase
and the three operators are redefined to achieve these
improvements.
6. REFERENCES
[1] V. Iyer, M. Eisen, D. Ross, G. Schuler, T. Moore, J. Lee, J.
Trent, L. Staudt, J. Hudson, M. Boguski, D. Lashkari, D.
program in the response of human fibroblasts to serum.
SCIENCE, 283, January 1999.
[2] K. Krishna and M. Murty. Genetic K-means algorithm. IEEE
Transactions on Systems, Man and Cybernetics Part B:
[3] Y. Lu, S. Lu, F. Fotouhi, Y. Deng, and S. Brown. Fast
genetic K-means algorithm and its application in gene
expression data analysis. Technical Report TR-DB-06-2003,
http://www.cs.wayne.edu/~luyi/publication/tr0603.pdf,
2003.
[4] S. Tavazoie, J.D. Hughes, M.J. Campbell, R.J. Cho, and
G.M. Church. Systematic determination of genetic network
architecture. Nature Genetics, 22:281285, 1999.