1

Pattern Recognition

Speaker: Wen-Fu Wang
Advisor: Jian-Jiun Ding
E-mail: r96942061@ntu.edu.tw
Graduate Institute of Communication Engineering
National Taiwan University, Taipei, Taiwan, ROC
2
Outline

Introduction
Minimum Distance Classifier
Matching by Correlation
Optimum statistical classifiers
Matching Shape Numbers
String Matching

3
Outline

Syntactic Recognition of Strings
String Grammars
Syntactic recognition of Tree
Grammars
Conclusions
4
Introduction
Basic pattern recognition flowchart

Sensor
Feature
generation

Feature
selection
Classifier
design
System
evaluation
5
Introduction
The approaches to pattern recognition
developed are divided into two principal areas:
decision-theoretic and structural

The first category deals with patterns described
using quantitative descriptors, such as length,
area, and texture

The second category deals with patterns best
described by qualitative descriptors, such as
the relational descriptors.

6
Minimum Distance Classifier
Suppose that we define the prototype of each
pattern class to be the mean vector of the
patterns of that class:


Using the Euclidean distance to determine
closeness reduces the problem to computing
the distance measures
1
j
j j
x w
j
m x
N
e
=

j=1,2,,W (1)
( )
j j
D x x m =
j=1,2,,W (2)
7
Minimum Distance Classifier

The smallest distance is equivalent to
evaluating the functions


The decision boundary between classes and for
a minimum distance classifier is
j=1,2,,W (3)
j=1,2,,W (4)
1
( )
2
T T
j j j j
d x x m m m =
( ) ( ) ( )
ij i j
d x d x d x =
1
( ) ( ) ( ) 0
2
T T
i j i j i j
x m m m m m m = + =
8
Minimum Distance Classifier
Decision boundary of minimum distance classifier

-0.5 0 0.5 1 1.5 2 2.5
0
0.5
1
1.5
2
x
1
x
2
3.aGG--d
D
Class C
1
Class C
2
9
Minimum Distance Classifier
Advantages:
1. Unusual direct-viewing
2. Can solve rotation the question
3. Intensity
4. Chooses the suitable characteristic,
then solves mirror problem
5. We may choose the color are one kind
of characteristic, the color question
then solve.
10
Minimum Distance Classifier
Disadvantages:
1. It costs time for counting samples,
but we must have a lot of
samples for high accuracy, so it is
more samples more accuracy!
2. Displacement
3. It is only two features, so that the
accuracy is lower than other methods.
4. Scaling
11
Matching by Correlation
We consider it as the basis for finding
matches of a sub-image of size
within an image of size ,
where we assume that and
( , ) ( , ) ( , )
s t
c x y f s t w x s y t = + +

J K
( , ) f x y M N
J M s K N s
for x=0,1,2,,M-1,y=0,1,2,,N-1 (5)
12
Matching by Correlation
Arrangement for obtaining the correlation of and at
point
M
K
J
Origin
o
f
w
0 0
( , ) w x s y t = + +
0 0
( , ) x y
( , ) f x y
0 0
( , ) x y
13
Matching by Correlation
The correlation function has the disadvantage of being
sensitive to changes in the amplitude of and
For example, doubling all values of doubles the value
of
An approach frequently used to overcome this difficulty
is to perform matching via the correlation coefficient





The correlation coefficient is scaled in the range-1 to 1,
independent of scale changes in the amplitude of and
f
w
f
( , ) c x y
1
2
2 2
[ ( , ) ( , )][ ( , ) ]
( , )
[ ( , ) ( , )] [ ( , ) ]
s t
s t s t
f s t f s t w x s y t w
x y
f s t f s t w x s y t w

+ +
=

+ +
`
)

f
w
14
Matching by Correlation
Advantages:
1.Fast
2.Convenient
3.Displacement
Disadvantages:
1.Scaling
2.Rotation
3.Shape similarity
4.Intensity
5.Mirror problem
6.Color can not recognition
15
Optimum statistical classifiers
The probability that a particular
pattern x comes from class is
denoted
If the pattern classifier decides that x
came from when it actually came
from , it incurs a loss, denoted
1
( ) ( )
W
j kj k
k
r x L p w x
=
=

i
w
( )
i
p w x
j
w
i
w
ij
L
16
Optimum statistical classifiers
From basic probability theory, we
know that
1
1
( ) ( ) ( )
( )
W
j kj k k
k
r x L p x w P w
p x
=
=

| |
( ) ( ) ( ) ( ) p A B p A p B A p B =
1
( ) ( ) ( )
W
j kj k k
k
r x L p x w P w
=
=

17
Optimum statistical classifiers
Thus the Bayes classifier assigns an
unknown pattern x to class
1 1
( ) ( ) ( ) ( )
W W
ki k k qj q q
k q
L p x w P w L p x w P w
= =
<

i
w
1
ij ij
L o =
1
( ) (1 ) ( ) ( )
W
j kj k k
k
r x p x w P w o
=
=

( ) ( ) ( )
j j
p x p x w p w =
18
Optimum statistical classifiers
The Bayes classifier then assigns a
pattern x to class if,


or, equivalently, if


i
w
( ) ( ) ( ) ( ) ( ) ( )
i i j j
p x p x w P w p x p x w P w <
( ) ( ) ( ) ( )
i i j j
p x w P w p x w P w >
( ) ( ) ( )
j j j
d x p x w P w =
19
Optimum statistical classifiers
Bayes Classifier for Gaussian Pattern
Classes
Let us consider a 1-D problem (n=1)
involving two pattern classes (W=2)
governed by Gaussian densities




( ) ( ) ( )
j j j
d x p x w P w =
2
2
( )
2
1
( )
2
j
j
x m
j
j
e P w
o
to

= 1, 2 j =
20
Optimum statistical classifiers
In the n-dimensional case, the
Gaussian density of the vectors in the
jth pattern class has the form




1
1
( ) ( )
2
1 2
2
1
( )
(2 )
T
j j j
x m C x m
j
n
j
p x w e
C t

=
21
Optimum statistical classifiers
Advantages:
1. The way always combine with other
methods, then it got high accuracy

Disadvantages:
1.It costs time for counting samples
2.It has to combine other methods
22
Matching Shape Numbers
Direction numbers for 4-directional
chain code, and 8-directional chain code
0
1
2
3
0
1
2
3
4
5
6
7
23
Matching Shape Numbers
Digital boundary with resampling grid superimposed
24
Matching Shape Numbers
All shapes of order 4, 6,and 8
Order6
Order8
Chain code: 0321
Difference : 3333
Shape no. : 3333
Chain code: 003221
Difference : 303303
Shape no. : 033033
Chain code: 00332211
Difference : 30303030
Shape no. : 03030303
Chain code:03032211
Difference :33133030
Shape no. :03033133
Chain code: 00032221
Difference : 30033003
Shape no. : 00330033
Order4
25
Matching Shape Numbers
Advantages:
1. Matching Shape Numbers suits the processing
structure simple graph, specially becomes by the
line combination
2. Can solve rotation the question
3. Matching Shape Numbers most emphatically to the
graph outline, Shape similarity also may completely
overcome
4. The Displacement question definitely may
overcome, because of this method emphatically to
the relative position but is not to the position
26
Matching Shape Numbers
Disadvantages :
1. It can not uses for a hollow structure
2. Scaling is a shortcoming which
needs to change, perhaps coordinates
the alternative means
3. Intensity
4. Mirror problem
5. The color is unable to recognize
27
String Matching
Suppose that two region boundaries,
a and b, are coded into strings
denoted and ,respectively
Let represent the number of
matches between the two strings,
where a match occurs in the kth
position if
max( , ) a b | o =
1 2
...
n
a a a
1 2
..
m
b b b
o
k k
a b =
28
String Matching
A simple measure of similarity
between and is the ratio



Hence R is infinite for a perfect match
and 0 when none of the
corresponding symbols in and match
( in this case)
max( , )
R
a b
o o
| o
= =

a
b
29
String Matching
Simple staircase structure.
Coded structure.
b
b
b
b
b
b
30
String Matching
Advantages:
1.Matching Shape Numbers suits the
processing structure simple graph, specially
becomes by the line combination
2.Can solve rotation the question
3.Intensity
4.Mirror problem
5. Matching Shape Numbers most emphatically to
the graph outline, Shape similarity also may
completely overcome
6. The Displacement question definitely may
overcome, because of this method emphatically to
the relative position but is not to the position
31
String Matching
Disadvantages:
1.It can not uses for a hollow structure
2.Scaling
3.The color is unable to recognize

32
Syntactic Recognition of Strings
String Grammars
When dealing with strings, we define a
grammar as the 4-tuple
is a finite set of variables called non-
terminals,
is a finite set of constants called
terminals,
is a set of rewriting rules called
productions,
in is called the starting symbol.
( , , , ) G N P S = E
N
E
P
S N
33
Syntactic Recognition of Strings
String Grammars
Object represented by its skeleton
primitives.
structure generated by using a
regular string grammar
a c
b
34
Syntactic Recognition of Strings
String Grammars
Advantages:
1.This method may use to a more
complex structure
2.It is a good method for character set
Disadvantages:
1.Scaling
2.Rotation
3.The color is unable to recognize
4.Intensity
5.Mirror problem
35
Syntactic Recognition of Tree
Grammars
A tree grammar is defined as the 5-tuple
and are sets of non-terminals and
terminals, respectively
is the start symbol, which in general can be
a tree
is a set of productions of the form ,
where and are trees
is a ranking function that denotes the
number of direct descendants(offspring) of a
node whose label is a terminal in the grammar
( , , , , ) G N P r S = E
N
E
S
P
i j
T T
i
T
j
T
r
36
Syntactic Recognition of Tree
Grammars
Of particular relevance to our discussion are
expansive tree grammars having productions
of the form



where are not terminals and k is a
terminal
1 2,
, ...,
n
X X X
37
An object
Primitives used for representing the
skeleton by means of a tree grammar
Syntactic Recognition of Tree
Grammars
a b c d e




38
Syntactic Recognition of Tree
Grammars
For example
a b c d e
1
(1)


S a
X

1
1
(2)


X b
X

1
2 3
(3)


X c
X X

2
2
(4)


X d
X

2
(5) X a
3
3
(6)


X e
X

3
(7) X a





39
Syntactic Recognition of Tree
Grammars
Advantages:
1. This method may use to a more
complex structure
2. It is a good method for character set
3. The Displacement question definitely
may overcome, because of this method
emphatically to the relative position but
is not to the position

40
Syntactic Recognition of Tree
Grammars
Disadvantages :
1. Scaling is a shortcoming which
needs to change, perhaps
coordinates the alternative
means
2. Rotation
3. The color is unable to recognize
4. Intensity

41
Conclusions
The graph recognizes is covers the domain very
widespread science, in the past dozens of years, all kinds
of method is unceasingly excavated, also acts according
to all kinds of probability statistical model and the
practical application model but unceasingly improves.

The graph recognizes applies to each different
application domain, actually often also simultaneously
entrusts with the entire wrap to recognize the system
different appearance, which methods thus we certainly
are unable to define to are "best" the graph recognize
the method.
42
Conclusions
Summary the seven approach to pattern recognition,
each methods has advantages and disadvantages
respectively. Therefore, we have to understand each
method preciously. Then we choose the adaptable
method for efficiency and accuracy.

The A method has obtained extremely good recognizing
rate in some application and is unable to express the
similar method applies mechanically in another
application also can similarly obtain extremely good
recognizing rate.

43
Conclusions
Below provides several possibilities
solutions the method
1. Scaling problem we may the reference area solve.
2. Neural networks solves for rotation problem.
3.The color question besides uses RBG to solve also may
use the spectrum to recognize differently.
4. Doing correlation with the reverse match filter for
Intensity mirror problem
5. We can use the measure of area for a hollow structure


44
References
[1] R. C. Gonzolez, R. E. Woods, "Digital Image
Processing, Second Edition", Prentice Hall 2002
[2] , "Matlab", 2005
[3] S. Theodoridis, K. koutroumbas, "Pattern
Recognition", Academic Press 1999
[4] W. K. Pratt ,"Digital Image Processing, Third Edition",
John Wiley & Sons 2001
[5] R. C. Gonzolez, R. E. Woods, S. L. Eddins, "Digital
Image Processing Using MATLAB", Prentice Hall 2005
[6] , -Matlab, 2000
[7] J. Schurmann, " A Unified View of Statistical and
Neural Approaches" Pattern Classification, Chap4, John
Wiley & Sons, Inc., 1996
45
References
[8]K. Fukunaga, Introduction to Statistical Pattern
Recognition, Second Edition, Academic Press, Inc.,1990
[9] E. Gose, R. Johnsonbaugh, and Steve Jost, "Pattern
recognition and Image Analysis", Prentice Hall Inc., New
Jersey, 1996
[10] Robert J. Schalkoff, "Pattern Recognition: Statical,
Structural and Neural Approaches", Chap5, John Wiley &
Sons, Inc., 1992
[11] J. S. Pan, F. R. Mclnnes, and M. A. Jack, "Fast
Clustering Algorithm for Vector Quantization", Pattern
Recognition 29, 511-518, 1996