Professional Documents
Culture Documents
Short Communication
A MATLAB toolbox for Self Organizing Maps and supervised neural network
learning strategies
Davide Ballabio a,, Mahdi Vasighi b
a
b
Milano Chemometrics and QSAR Research Group, Department of Environmental Sciences, University of Milano Bicocca, Milano, Italy
Department of Computer Science and Information Technology, Institute for Advanced Studies in Basic Sciences (IASBS), Zanjan, Iran
a r t i c l e
i n f o
Article history:
Received 17 April 2012
Received in revised form 5 July 2012
Accepted 14 July 2012
Available online 22 July 2012
Keywords:
Self Organizing Maps
Supervised pattern recognition
Articial Neural Networks
MATLAB
Kohonen maps
a b s t r a c t
Kohonen maps and Counterpropagation Neural Networks are two of the most popular learning strategies based
on Articial Neural Networks. Kohonen Maps (or Self Organizing Maps) are basically self-organizing systems
which are capable to solve the unsupervised rather than the supervised problems, while Counterpropagation
Articial Neural Networks are very similar to Kohonen maps, but an output layer is added to the Kohonen
layer in order to handle supervised modelling. Recently, the modications of Counterpropagation Articial
Neural Networks allowed introducing new supervised neural network strategies, such as Supervised Kohonen
Networks and XY-fused Networks.
In this paper, the Kohonen and CP-ANN toolbox for MATLAB is described. This is a collection of modules for calculating Kohonen maps and derived methods for supervised classication, such as Counterpropagation Articial
Neural Networks, Supervised Kohonen Networks and XY-fused Networks. The toolbox comprises a graphical
user interface (GUI), which allows the calculation in an easy-to-use graphical environment. It aims to be useful
for both beginners and advanced users of MATLAB. The use of the toolbox is discussed here with an appropriate
practical example.
2012 Elsevier B.V. All rights reserved.
1. Introduction
Kohonen maps (or Self Organizing Maps, SOMs) are one of the most
popular learning strategies among the several Articial Neural Networks
algorithms proposed in literature [1]. Their uses are increasing related to
several different tasks and nowadays they can be considered as an
important tool in multivariate statistics [2]. Kohonen maps are selforganizing systems able to solve unsupervised rather than supervised
problems. As a consequence, methods based on the Kohonen approach
but combining characteristics from both supervised and unsupervised
learning have been introduced. Counterpropagation Articial Neural
Networks (CP-ANNs) are very similar to Kohonen maps, since an
output layer is added to the Kohonen layer [3]. When dealing with
classication issues, CP-ANNs are generally efcacious methods for
modelling classes separated with non-linear boundaries. Recently, modications to CP-ANNs led introducing new supervised neural network
strategies, such as Supervised Kohonen Networks (SKNs) and XY-fused
Networks (XY-Fs) [4].
As a consequence of the increasing success of Self Organizing
Maps, some toolboxes for calculating supervised and unsupervised
SOMs were proposed in literature [58]. The Kohonen and CP-ANN
Corresponding author at: Dept. of Environmental Sciences, University of MilanoBicocca, P.zza della Scienza, 120126 Milano, Italy. Tel.: +39 02 6448 2801; fax: +39
02 6448 2839.
E-mail address: davide.ballabio@unimib.it (D. Ballabio).
0169-7439/$ see front matter 2012 Elsevier B.V. All rights reserved.
doi:10.1016/j.chemolab.2012.07.005
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
dri
old
xi wr
d max 1
where is the learning rate and dmax the size of the considered
neighbourhood, that decreases during the training phase. The topological
distance dri is dened as the number of neurons between the considered
neuron r and the winning neuron. The learning rate changes during the
training phase, as follows:
t
start
final
final
1
t tot
25
where t is the number of the current training epoch, ttot is the total number of training epochs, start and nal are the learning rate at the beginning and at the end of the training, respectively.
When the batch training is used, the whole set of samples is
presented to the network and winner neurons are found; after this,
the weights are calculated on the basis of the effect of all the samples,
at the same time:
I
P
uir xi
wr i1I
P
uir
i1
where wr are the updated weights of the r-th neuron, xi is the i-th
sample, and uir is the weighting factor of the winning neuron related
to the i-th sample with respect to neuron r:
uir 1
dri
d max 1
where, , dmax and dri are dened as before (see Eq. (1)).
At the end of the network training, samples are placed in the most
similar neurons of the Kohonen map; in this way, the data structure
can be visualized and the role of experimental variables in dening
the data structure can be elucidated by looking at the Kohonen
weights.
Fig. 1. Structures of Kohonen maps and related methods (CP-ANNs, SKNs, and XY-Fs) for a generic dataset constituted by J variables and G classes. Notation in the g. refers to notation
used in the text: xij represents the value of the jth variable for the i-th sample, wrj represents the value of the j-th Kohonen weight for the r-th neuron, cig represents the membership of the
i-th sample to the gth class expressed with a binary code, and yrg represents the value of the g-th output weight for the r-th neuron.
26
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
dri
old
ci yr
d max 1
Table 1
Network settings available in the toolbox.
Settings
Description
Possible values
Default
net_type
nsize
epochs
topol
bound
training
init
a_max
a_min
scaling
absolute_range
ass_meth
scalar
NaN
NaN
NaN
square
toroidal
batch
random
0.5
0.01
none
classical
1
1
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
27
Table 2
MATLAB routines of the toolbox related to the calculation of Kohonen maps and their main outputs. For each routine, outputs are collected as elds of a unique MATLAB structure.
MATLAB routine
model_kohonen
pred_kohonen
Outputs
Description
settings
scal
top_map
top_map
Table 3
MATLAB routines of the toolbox related to the calculation of CP-ANNs, SKNs and XY-Fs and their main outputs. For each routine, outputs are collected as elds of a unique MATLAB
structure.
MATLAB routine
Description
Outputs
Description
model_cpann, model_skn,
model_xyf
W_out
neuron_ass
settings
scal
class_true
class_calc
class_weights
top_map
class_param
cv_cpann, cv_skn, cv_xyf
settings
class_true
class_pred
class_weights
class_param
class_pred
class_weights
top_map
28
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
of each sample placed in the neuron. Finally, the confusion matrix is provided. This is a squared matrix with dimensions GG where each entry
ngk represents the number of samples belonging to class g and assigned
to class k. The most known classication indices, such as error rate,
non-error rate specicity, sensitivity, precision and ratio of not assigned
samples are derived from the confusion matrix [16].
Cross-validation can be performed by means of the functions
listed in Table 3, by choosing the number of cancellation groups and
the cross-validation method for separating the samples into cancellation groups (venetian blinds or contiguous blocks). The output of this
routine is a MATLAB structure containing the confusion matrix and
the derived classication indices calculated in cross-validation.
Unknown or test samples can be predicted by using an existing
model: new samples are compared with the trained Kohonen weights,
placed in the closest neuron and assigned to the corresponding class.
This calculation can be made in the toolbox by means of the functions
Fig. 3. Kohonen and CP-ANN toolbox: interactive graphical interface for visualizing the Kohonen top map.
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
29
Fig. 4. Kohonen and CP-ANN toolbox: interactive graphical interface for visualizing the optimization results.
The user can load data, sample and variable labels, and the class vector when dealing with supervised classication, both from the MATLAB
workspace or MATLAB les. Then, in addition to basic operations, such
as looking at the data, plotting variable means and sample proles, all
the calculation steps described in the previous paragraphs can be easily
performed in the graphical interface.
Optimization of the network structure to choose the optimal number of epochs and neurons can be performed directly in a proper window. Once the user has decided how to set the network, settings and
parameters for cross-validation can be dened in a proper window,
where basic and advanced settings are divided in order to facilitate
practitioners who are not skilled with SOMs.
Once a model has been calculated, results of the optimization step,
models, settings and cross validation results can be exported in the
MATLAB workspace. Saved models can be easily loaded in the toolbox
for future analyses, as well as new samples can be loaded and
predictions can be calculated on the basis of previously calculated
models.
When dealing with supervised classication, the user can graphically evaluate indices for classication diagnostic (confusion matrix,
error rate, non error rate, specicity, sensitivity, purity) and analyze
ROC curves (Receiver Operating Characteristics). These are graphical
tools for the analysis of classication results and describe the degree
of separation of classes. ROC curves are graphical plots of 1 Specicity
(also known as False Positive Rate, FPR) and Sensitivity (also known as
True Positive Rate, TPR) as x and y axes, respectively, for a binary
classication system as its discrimination threshold is changed. In this
toolbox, ROC curves are separately calculated for each class, by changing
the threshold of assignation over the output weights from 0 to 1.
Table 4
Example of analysis: some of the indices calculated by the toolbox and used for classication diagnostic. Error rate, non-error rate, specicity, sensitivity and precision
obtained in tting, cross-validation (10 cancellation groups) and on the external test
set of samples are shown.
Classication parameter
Fitting
Cross-validation
Non-error rate
Error rate
Precision of class 1
Precision of class 2
Sensitivity of class 1
Sensitivity of class 2
Specicity of class 1
Specicity of class 2
0.97
0.03
0.99
0.94
0.97
0.98
0.98
0.97
0.97
0.03
0.99
0.94
0.97
0.98
0.98
0.97
0.96
0.04
0.98
0.91
0.95
0.97
0.97
0.95
30
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
Fig. 5. Example of analysis: a) variable prole for each class produced by the toolbox. In this plot, the average of the Kohonen weights of each variable calculated on the neurons
assigned to each class is shown; b) plot of ROC curves produced by the toolbox.
are common; in these cases, it is not easy to solve the data interpretation with a simple visual approach.
For this reason, the toolbox allows the calculation of Principal
Component Analysis (PCA) on the Kohonen weights, in order to investigate the relationships between variables and classes in a global
way and not one variable at a time [17]. A GUI for calculating PCA
on the Kohonen weights is provided in the toolbox. Details on its
use are given in the section describing the illustrative example of
analysis.
Fig. 6. Example of analysis: a) Kohonen top map produced by the toolbox. In the top map, each sample is labelled on the basis of its class. Each neuron is colored with a gray scale on
the basis of Kohonen weights of variable 2 (uniformity of cell size): white corresponds to Kohonen weight equal to 0, black to Kohonen weights equal to 1; b) prole of Kohonen
weights of one of the neurons where samples of class 2 (malignant) were placed.
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
31
Fig. 7. Example of analysis: a) score plot of the rst two principal components calculated on the Kohonen weights. Each neuron is colored with a gray scale on the basis of the
output weight corresponding to class 2 (malignant): white corresponds to output
weight equal to 0, black to output weights equal to 1; b) loading plot of the rst two
principal components calculated on the Kohonen weights. Each variable is labelled
with its identication number.
References
[1] T. Kohonen, Self-Organization and Associative Memory, Springer Verlag, Berlin,
1988.
[2] F. Marini, Analytica Chimica Acta 635 (2009) 121131.
[3] J. Zupan, M. Novic, J. Gasteiger, Chemometrics and Intelligent Laboratory Systems
27 (1995) 175187.
[4] W. Melssen, R. Wehrens, L. Buydens, Chemometrics and Intelligent Laboratory
Systems 83 (2006) 99113.
[5] J. Vesanto, J. Himberg, E. Alhoniemi, J. Parhankangas, SOM Toolbox for Matlab 5,
Technical Report A57, Helsinki University of Technology, 2000.
[6] M. Schmuker, F. Schwarte, A. Brck, E. Proschak, E. Tanrikulu, A. Givehchi, K.
Scheiffele, G. Schneider, Journal of Molecular Modeling 13 (2007) 225228.
[7] I. Kuzmanovski, M. Novic, Chemometrics and Intelligent Laboratory Systems 90
(2008) 8491.
32
D. Ballabio, M. Vasighi / Chemometrics and Intelligent Laboratory Systems 118 (2012) 2432
[14] D. Polani, Kohonen Maps, In: On the optimisation of self-organising maps by genetic
algorithms, Elsevier, Amsterdam, 1999.
[15] I. Kuzmanovski, M. Novic, M. Trpkovska, Analytica Chimica Acta 642 (2009)
142147.
[16] D. Ballabio, R. Todeschini, Infrared Spectroscopy for Food Quality Analysis and
Control, In: Multivariate Classication for Qualitative Analysis, Elsevier, 2008.
[17] D. Ballabio, R. Kokkinofta, R. Todeschini, C.R. Theocharis, Chemometrics and Intelligent
Laboratory Systems 87 (2007) 7884.
[18] W.H. Wolberg, O.L. Mangasarin, Proceedings of the National Academy of Sciences
of the United States of America 87 (1990) 91939196.