SUPPORT VECTOR MACHINES AND

ASSOCIATIVE CLASSIFIERS
SVMSupport Vector Machines
A new classification method for both linear and
nonlinear data
It uses a nonlinear mapping to transform the original
training data into a higher dimension
With the new dimension, it searches for the linear
optimal separating hyperplane (i.e., decision
boundary)
With an appropriate nonlinear mapping to a
sufficiently high dimension, data from two classes
can always be separated by a hyperplane
SVM finds this hyperplane using support vectors
(essential training tuples) and margins (defined by
the support vectors)
SVMHistory and Applications
Vapnik and colleagues (1992)groundwork from
Vapnik & Chervonenkis statistical learning theory in
1960s
Features: training can be slow but accuracy is high
owing to their ability to model complex nonlinear
decision boundaries (margin maximization)
Used both for classification and prediction
Applications:
handwritten digit recognition, object recognition,
speaker identification, benchmarking time-series
prediction tests

SVMGeneral Philosophy

Data Set D (X1, y1), (X2, y2)(Xn, yn)

Xi Training tuples

yi Class labels
Linearly Separable

Best separating line minimum classification

error

Plane / Hyperplane
Maximal Margin Hyperplane (MMH)

Larger margin more accurate at classifying

future samples

Gives largest separation among classes

Sides of margin are parallel to hyperplane

Distance from MMH to closest training tuple of

either class

SVMLinearly Separable High Dimensional Data

Classification using SVM

Maximal Margin Hyperplane: d(XT)= i=1 l
yiiXiXT + b0

Xi Support vectors (l numbers) and XTTest data

Class depends on sign of result

SVM vs. Neural Network

SVM
Relatively new concept
Deterministic algorithm
Nice Generalization properties
Hard to learn learned in batch mode using
quadratic programming techniques

Using kernels can learn very complex functions

Neural Network
Relatively old
Nondeterministic algorithm
Generalizes well but doesnt have strong
mathematical foundation
Can easily be learned in incremental fashion
To learn complex functionsuse multilayer
perceptron

Associative Classification

Associative classification

Association rules are generated and analyzed for

use in classification

Search for strong associations between frequent

patterns (conjunctions of attribute-value pairs)
and class labels

Classification: Based on evaluating a set of rules

in the form of
p1 ^ p2 ^ pl Aclass = C (conf, sup)

Effectiveness

It explores highly confident associations among

multiple attributes

Overcome some constraints introduced by

decision-tree induction which considers only one
attribute at a time

Associative classification has been found to be

more accurate than some traditional classification
methods, such as C4.5

Typical Associative Classification Methods

CBA (Classification By Association)

Mine possible rules in the form of

Cond-set (a set of attribute-value pairs)

class label

Iterative Approach similar to Apriori

Build classifier: Organize rules according to

decreasing precedence based on confidence and
then support

CMAR (Classification based on Multiple Association

Rules)

Uses variant of FP-Growth

Mined rules are stored and retrieved using tree

structure

Rule Pruning is carried out Specialized rules

with low confidence are pruned

Classification: Statistical analysis on multiple

rules

CPAR (Classification based on Predictive

Association Rules)

Generation of predictive rules (FOIL-like

analysis)

High efficiency, accuracy similar to CMAR

Uses best-k rules of each group to predict class

label