Professional Documents
Culture Documents
e
y
N
i i i
x w w
1 0
Comparison of Algorithms (cont)
Binary decision tree:
Binary and Linear decision trees also tend to
explicitly construct the decision-region
boundaries.
Advantages: Easy implementation, easy
explanation of input-output relationship
Disadvantages: Limited complexity on the
constructed boundary, the tree structure may not
be global optimal.
Root
x
i
>=c
1
x
i
<c
1
x
j
>=c
2
x
j<
c
2
x
k
>=c
3
x
k
<c
3
Comparison of Algorithms (cont)
Neural Network:
Feedforward neural network and radial-basis
function network both tend to implicitly construct
the decision-region boundaries.
Advantages: They can both approximate any
complex decision boundaries provided that enough
nodes are used.
Disadvantages: Long training time
Comparison of Algorithms (cont)
Supporting vector machine
Supporting vector machine also tends to implicitly
construct the decision-region boundaries.
Advantages: This type of classifier has been shown to
have good generalization capability.
Comparison of Algorithms (cont)
Bays Rule:
Unimodal Gaussian:
Unimodal Gaussian explicitly construct the PDF,
compute the prior probability P(C
j
) and posterior
probability P(C
j
|X).
Advantages: Easy implementation, confidence
level can be obtained from the posterior
probabilities.
Disadvantages: Sample distributions may not be
Gaussian.
) ( / ) ( ) | ( ) | ( X P C P C X P X C P
j j j
) ( ) ( 2 / 1
) 2 (
1
1
2 / 1
2 /
) | (
j j
T
j
j
N
M X V M X
V
j
e C X P
Comparison of Algorithms (cont)
Gaussian mixture modify unimodal Gaussian in
the way that the PDF is estimated by a weighted
average of multiple Gaussian.
Similar to Gaussian mixture Parzens windows
approximate PDF using weighted average of
radial Gaussian.
Advantage: Given enough Gaussian components,
the above architectures can approximate arbitrary
complex distributions
Comparison of Algorithms (cont)
K nearest neighbor classifier
K nearest neighbor tends to construct posterior
probabilities P(C
j
|X)
Advantage: No training is required, confidence
level can be obtained
Disadvantage: classification accuracy is low is
complex decision-region boundary exists, large
storage required.
Other Useful Classifiers
Projection Pursuit: aims to decomposing
the task of high-dimensional modeling into
a sequence of low-dimensional modeling.
This algorithm consists of two stage: the
first stage projects the input data onto a
one-dimensional space while the second
stage construct the mapping from projected
space to the output space.
Other Useful Classifiers (cont)
Multivariate adaptive regression splines (MARS)
tends to approximate the decision-region
boundaries in two stages.
At the first stage, the algorithm partitions the state
space into small portions.
At the second stage, the algorithm construct a
low-order polynomial to approximate the
decision-region boundary within each partition.
Disadvantage: This algorithm is intractable for
problem with high (> 10) dimensional inputs
Other Useful Classifiers (cont)
Group method of data handling (GMDH)
also aims to approximate the decision-
region boundaries using high-order
polynomial functions.
The modeling process begins with a low
order polynomial, and then iteratively
combines terms to produce a higher order
polynomial until the modeling accuracy
saturates.
Keep The Following In Mind
Use multiple algorithms without bias and
let your specific data help determine which
model is best suited for your problem.
Occams Razor: Entities should not be
multiplied unnecessarily -- "when you have
two competing models which make exactly
the same predictions to the data, the one
that is simpler is the better."
A New Member In Our Group