I HC QUC GIA H NI

TRNG I HC CNG NGH

L Tin Mi




MNG NEURAL RBF
V
NG DNG NHN DNG CH VIT TAY








KHO LUN TT NGHIP I HC H CHNH QUY



Ngnh: Cng ngh thng tin














H NI - 2009

I HC QUC GIA H NI
TRNG I HC CNG NGH


L Tin Mi




MNG NEURAL RBF
V
NG DNG NHN DNG CH VIT TAY








KHO LUN TT NGHIP I HC H CHNH QUY



Ngnh: Cng ngh thng tin
Cn b hng dn: PGS.TS Hong Xun Hun













H NI - 2009

LI CM N

Ti mun by t s cm n su sc ca mnh ti thy Hong Xun Hun, thuc
b mn Khoa hc my tnh, khoa Cng ngh thng tin, trng i hc Cng ngh,
HQGHN. Trong thi gian thc hin kha lun, thy nhit tnh hng dn v gip
ti rt nhiu. Ngoi thi gian tm hiu v cung cp ti liu, thy cng ch ra nhng
vng mc trong qua trnh lm, gip ti khc phc t hiu qu cao hn. Thy
cng tn tnh gip ti c mt ch lm vic yn tnh trong sut qu trnh lm
kha lun.

Ti cng mun by t s cm n ca mnh ti cc cc thy, cc c trong b
mn, cng nh cc thy, cc c trong khoa, trng ht sc to iu kin tt v gip
cho ti hon thnh kha lun ca mnh.

TM TT NI DUNG

Mc d c nghin cu t rt lu, nhng n nay bi ton ni suy v xp
x hm nhiu bin vn cn c rt t cng c ton hc gii quyt. Mng Neural nhn
to l mt phng php hay gii quyt bi ton ni suy, xp x hm nhiu bin.
Nm 1987 M.J.D. Powell a ra mt cch tip cn mi gii quyt bi ton ni
suy hm nhiu bin s dng k thut hm c s bn knh (Radial Basis Function -
RBF), nm 1988 D.S. Bromhead v D. Lowe xut kin trc mng Neural RBF v
tr mt cng c hu hiu gii quyt bi ton ni suy v xp x hm nhiu
bin(xem [11]).

Ni dung chnh ca kha lun l trnh by kho cu v mng Neural RBF
gii quyt bi ton ni suy, xp x hm nhiu bin sau ng dng c s l thuyt trn
xy dng phn mm nhn dng ch s vit tay.
















MC LC

M U................................................................................................................... 1
Chng 1 BI TON NI SUY, XP X HM S V MNG NEURAL RBF1
1.1 PHT BIU BI TON NI SUY V XP X HM S............................ 1
1.1.1 Bi ton ni suy.......................................................................................... 1
1.1.1.1 Ni suy hm mt bin s ...................................................................... 1
1.1.1.2 Bi ton ni suy hm nhiu bin .......................................................... 2
1.1.2 Bi ton xp x ........................................................................................... 2
1.1.3 Cc phng php gii quyt bi ton ni suy v xp x hm s .................. 2
1.2 MNG NEURAL NHN TO....................................................................... 3
1.2.1 Gii thiu mng Neural nhn to................................................................ 3
1.2.1.1 Mng Neural sinh hc.......................................................................... 4
1.2.1.2 Mng Neural nhn to......................................................................... 5
1.3 MNG NEURAL RBF..................................................................................... 8
1.3.1 Gii thiu mng Neural RBF...................................................................... 8
1.3.1.1 Bi ton ni suy nhiu bin v k thut hm c s bn knh................. 8
1.3.1.2 Kin trc mng Neural RBF............................................................... 10
1.3.1.3 ng dng ca mng Neural RBF....................................................... 10
1.4 CC PHNG PHP HUN LUYN MNG NEURAL RBF ................... 11
1.4.1 Phng php hun luyn mt pha............................................................. 11
1.4.2 Phng php hun luyn hai pha.............................................................. 12
1.4.3 Phng php hun luyn 2 pha HDH ....................................................... 13
1.4.4 Phng php hun luyn ba pha y .................................................... 16
1.5 KT QU THC NGHIM.......................................................................... 16
1.5.1 Kt qu..................................................................................................... 16
1.5.2 Nhn xt................................................................................................... 19
Chng 2 NHN DNG CH VIT TAY........................................................... 20
2.1 NHN DNG MU...................................................................................... 20
2.1.1 Nhn dng mu ........................................................................................ 20
2.1.1.1 Mu l g ? ........................................................................................ 20

2.1.1.2 Nhn dng mu l g ?...................................................................... 20
2.1.1.3 Lch s ca lnh vc nhn dng mu.................................................. 21
2.1.1.4 ng dng ca nhn dng mu ........................................................... 21
2.1.1.5 Cc bi ton nhn dng mu ............................................................. 22
2.1.1.6 Cc bc x l trong h thng nhn dng mu ................................. 22
2.2 BI TON NHN DNG CH VIT TAY................................................. 24
2.2.1 Tnh hnh chung v nhn dng ch vit tay............................................... 24
2.2.2 Gii thiu bi ton nhn dng ch vit tay................................................ 24
2.2.3 Hng gii quyt cho bi ton nhn dng k t vit tay ........................... 24
2.3 CC PHNG PHP TRCH CHN C TRNG CH VIT TAY....... 25
2.2.1 Phng php trch chn c trng kt hp bin i DCT v thut ton phn
tch thnh phn chnh PCA................................................................................ 25
2.2.1.1 Thut ton PCA ................................................................................. 26
2.2.1.2 Phng php trch chn c trng kt php bin i DCT v thut
ton PCA....................................................................................................... 27
2.2.2 Phng php trch c trng s dng Momen Legendre........................... 28
2.2.2.1 Momen v Momen Legendre ............................................................. 28
2.2.2.2 Phng php trch chn c trng ch vit tay bng Momen Legendre30
2.2.3 Phng php s dng mng Neural nhn chp(Convolution neural
network)............................................................................................................ 32
2.2.3.1 Khi nim c s................................................................................. 32
2.2.3.2 Phng php trch c trng s dng mng Neural nhn chp ......... 33
2.4 THC NGHIM............................................................................................ 34
2.4.1 Kt qu..................................................................................................... 35
2.4.2 Nhn xt................................................................................................... 35
Chng 3 CC PHNG PHP CI THIN HIU SUT CA MNG
NEURAL RBF........................................................................................................ 36
3.1 CC PHNG PHP CI THIU HIU SUT CA MNG NEURAL
RBF...................................................................................................................... 36
3.1 CC PHNG PHP CI THIU HIU SUT CA MNG NEURAL
RBF...................................................................................................................... 36

3.1.1 Tng tp d liu hun luyn ..................................................................... 36
3.1.1.1 Tng tp d liu bng cc php bin i hnh hc ............................. 36
3.1.2 Phng php hc tp hp ......................................................................... 37
3.1.2.1 Phng php hc tp hp ci tin...................................................... 38
3.1.3 Phng php tng tc nhn dng ......................................................... 39
3.1.3.1 Phng php b nhn dng ba lp .................................................... 40
3.2 THC NGHIM............................................................................................ 41
Chng 4 GII THIU CHNG TRNH NHN DNG CH S VIT TAY
V TNG KT...................................................................................................... 42
4.1 GII THIU CHNG TRNH NHN DNG CH S VIT TAY.......... 42
4.1.1 Chng trnh nhn dng ch vit tay........................................................ 42
4.1.1.1 Gii thiu chng trnh...................................................................... 42
4.2 TNG KT V PHNG HNG PHT TRIN CA TI ............... 43
4.2.1 Tng kt ................................................................................................... 43
4.2.1.1 Nhng cng vic lm c............................................................ 43
4.2.2.2 Hng pht trin ca ti ............................................................... 44
TI LIU THAM KHO...................................................................................... 45


BNG DANH MC CC HNH MINH HA

Hnh 1: Minh ha bi ton ni suy hm mt bin..............................................................1
Hnh 2: Minh ha mt Neuron thn kinh sinh hc.............................................................4
Hnh 3: Cu to mt Neural nhn to................................................................................5
Hnh 4: th hm ngng ..............................................................................................6
Hnh 5: th hm tuyn tnh...........................................................................................6
Hnh 7: th hm tanh....................................................................................................6
Hnh 8: th hm Gauss.................................................................................................7
Hnh 9: Kin trc mng Neural truyn ti .........................................................................7
Hnh 10: Minh ha s nh hng ca hm bn knh .........................................................9
Hnh 11: Kin trc ca mng RBF..................................................................................10
Hnh 12: Qu trnh hi t n gi tr cc tiu ca thut ton Gradient .............................12
Hnh 13: Thut ton HDH hun luyn mng RBF...........................................................15
Hnh 14: Cc bc x l trong h thng nhn dng mu.................................................22
Hnh 15 : Cc bc gii quyt bi ton nhn dng ch vit tay .......................................25
Hnh 16: nh hng ca vector ring, gi tr ring ln tp d liu..................................26
Hnh 17 : Cc bc thc hin ca thut ton PCA..........................................................27
Hnh 18: Cc bc trch chn c trng bng bin DCT kt hp PCA............................27
Hnh 19: Bin i DCT v cch ly d liu theo ng zigzag.......................................28
Hnh 21: Cc bc thc hin ca phng php trch chn c trng s dung momen
Legendre.........................................................................................................................32
Hnh 22: Thao tc nhn chp ..........................................................................................33
Hnh 23: Qu trnh trch chn c trng s dng mng Neural nhn chp.......................34
Hnh 24: Minh ha qu trnh ly c trng bng mng Neuron nhn chp ......................34
Hnh 21: Ma trn vector cho php bin i Elastic..........................................................37
Hnh 22: V d v php bin i Elastic..........................................................................37
Hnh 23: Kin trc ca phng php hc tp hp ci tin...............................................39
Hnh 24: Kin trc ca b nhn dng ba lp ...................................................................40
Hnh 25: Biu so snh chnh xc nhn dng v thi gian hun luyn ca cc
phng php hun luyn khc nhau ................................................................................41
Hnh 26: Giao din chnh ca chng trnh.....................................................................43
Hnh27: Bng thng bo kt qu nhn dng....................................................................43



BNG DANH MC T VIT TT

K hiu Ngha ting Anh Ngha ting Vit
ANN Artificial neural network Mng n-ron nhn to
DCT Discrete cosin transform Bin i cosin ri rc
IDE
Integrated Development
Environment
Mi trng thit k hp nht
MLP Multi layer perceptron
Mng n-ron truyn thng nhiu
tng
PCA
Principal component
analysis
Phn tch thnh phn chnh
PDA Personal Digital Assistant
Thit b h tr c nhn(thng
m ch cc my tnh cm tay)
RBF Radial Basis Function Hm c s bn knh
SVM Support Vector Machine My vec-t h tr










M U

Bi ton ni suy v xp x hm s c bit n t lu v n c ng dng trong rt
nhiu lnh vc trong khoa hc k thut cng nh i sng. Ngy nay bi ton ni suy
v xp x hm nhiu bin tr thnh mt vn thi s v gii quyt c cc bi
ton ng dng (v d trong nhn dng mu) nhiu khi buc con ngi phi gii quyt
c bi ton ni suy, xp x hm nhiu bin. Trong ton hc bi ton ni suy, xp x
hm mt bin c gii quyt kh y bng rt nhiu cc phng php khc
nhau. Tuy nhin bi ton ni suy, xp x hm nhiu bin th cc cng c ton hc vn
cn rt hn ch.

Khi nim mng Neural nhn to xut hin u th k 20 trong thi k con ngi
tm cch ch to ra nhng b my c kh nng suy ngh, t duy nh con ngi. Tri
qua mt thi gian di pht trin v nghin cu th c s l thuyt cng nh thc
nghim v mng Neural nhn to t c nhng kt qu rt kh quan. Nh kh
tnh ton mnh ca my tnh, mng Neural nhn to ngy nay l mt cng c rt tt
gii quyt bi ton ni suy v xp x hm nhiu bin. V th mng Neural nhn to
c s dng rt nhiu trong cc lnh vc tnh ton, nhn dng mu cng nh trong
cc lnh vc khoa hc quan trng khc (xem [11]-chapter 4). L mt loi mng Neural
nhn to, mng Neural RBF cng l mt cng c hiu qu gii quyt bi ton ni
suy v xp x hm nhiu bin vi im mnh hn hn cc loi mng Neural khc ch
n c thi gian hun luyn rt nhanh.

Bi ton nhn dng ch vit tay l mt bi ton quen thuc v c ng dng rt ln
trong thc t v th t lu n thu ht rt nhiu ngi nghin cu. Mc d t
c nhng kt qu rt cao trong bi ton nhn dng ch vit tay (mng Neural nhn
chp t chnh xc 99.61% trn b d liu MNIST [8]) song ngy nay ngi ta
vn tip tc nghin cu nhng phng php nhn dng tt hn hng n dng cho
cc thit b di ng, v cc bi ton thi gian thc.

T cc nhn xt trn, vi lng am m mun nghin cu, hc hi v kin trc ca
mng Neural nhn to (c th y l mng Neural RBF) qua ng dng vit
phn mm nhn dng ch vit tay, c s ch bo v gip tn tnh ca thy gio


PGS.TS Hong Xun Hun ti tin hnh thc hin kha lun tt nghip vi ti
Mng Neural RBF v ng dng nhn dng ch vit tay.

Ni dung ca kha lun s i su nghin cu nhng vn sau:
- Kho cu v mng Neural RBF.
- Tm hiu bi ton nhn dng ch vit tay v cc phng php trch chn c
trng ch vit tay.
- Nghin cu cc phng php ci tin hiu sut ca mng Neural RBF p dng
cho bi ton nhn dng ch vit tay.
- Tin hnh ci t cc ng dng thc hin so snh hiu sut cc phng php
hun luyn mng Neural RBF, hiu sut cc phng php trch chn gi tr c
trng, ci t cc phng php ci thin hiu sut ca mng RBF p dng
cho bi ton nhn dng ch vit tay.
- Tin hnh vit chng trnh nhn dng ch s vit tay nhn dng ch vit tay
tng hp tt c cc phn kin thc nghin cu.

Vi mc tiu dn dt t c s l thuyt mng Neural RBF n ng dng nhn dng
ch vit tay, bi kha lun c phn thnh bn chng ln:

+Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF
Chng ny s cung cp nhng khi nim c bn nht v bi ton ni suy, xp x hm
cng nh v nn bc tranh tng quan v mng Neural nhn to. Phn ln ni dung ca
chng ny s tp trung i su nghin cu v mng Neural RBF bao gm kin trc v
cc phng php hun luyn mng. Phn cui chng (1.4) gii thiu kt qu thc
nghim so snh hiu sut cc phng php hun luyn mng Neural RBF thng qua
bi ton phn tch thnh phn trong ng du.

+Chng 2: Nhn dng ch vit tay
Phn u chng s trnh by s lc v bi ton nhn dng mu, phn tip theo ca
chng s lm r hn v cc bc gii quyt bi ton nhn dng ch vit tay. Phn
ln ni dung ca chng s tp trung nghin cu cc phng php ly c trng ch
vit tay. Phn cui chng a ra kt qu thc nghim so snh hiu sut cc phng
php trch chn c trng ch vit tay khc nhau.

+Chng 3: Cc phng php tng hiu sut mng Neural RBF


Ni dung ch yu ca chng ny l gii thiu mt s phng php nhm ci thin
hiu sut mng Neural RBF p dng cho bi ton nhn dng ch vit tay. Phn u
chng gii thiu phng php lm tng s lng d liu hun luyn s dng cc
phng php bin i nh affine, elastic. Tip s gii thiu phng php lm tng
tc v chnh xc nhn dng bng cch s dng b nhn dng ba tng. Phng
php hc tp hp (Ensemble Learning) ci thin chnh xc nhn dng cng c
cp chng ny. chng ny ti xin xut phng php hc tp hp ci tin
t chnh xc nhn dng gn 98% cho b d liu MNIST v c thi gian hun
luyn rt nhanh. cui chng s gii thiu kt qu thc nghim so snh hiu sut
nhn dng ca phng php hp ci tin so vi cc phng php thng thng.

+Chng 4: Gii thiu chng trnh nhn dng ch s vit tay v kt lun
Phn u chng gii thiu phn mm nhn dng k t ch s vit tay m ti xy
dng da trn c s tng hp ton b nn tng l thuyt ca bi kha lun. Cui cng
l phn tng kt ca kha lun.









Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

1
CHNG 1
BI TON NI SUY, XP X HM S
V MNG NEURAL RBF

Ni dung chng ny gm c:
1.1 Pht biu bi ton ni suy v xp x hm s
1.2 Mng Neural nhn to
1.3 Mng Neural RBF
1.4 Cc thut ton hun luyn mng Neural RBF
1.5 Kt qu thc nghim v nh gi



1.1 PHT BIU BI TON NI SUY V XP X HM S

1.1.1 Bi ton ni suy

1.1.1.1 Ni suy hm mt bin s

Bi ton ni suy hm mt bin tng qut c t ra nh sau: Mt hm s ( ) x f y =
cha bit v ch xc nh c ti cc im
0 1 N
x a x x b = < < < = K vi cc gi tr y
i
=
f(x
i
). Ta cn tm mt biu thc gii tch (x) xc nh gn ng gi tr ( ) y x ~ ti
cc im | | b a x , e ca hm f(x) sao cho ti cc im x
i
th hm s trng vi gi tr y
i

bit (vi | | b a x , e ta gi l ngoi suy). V phng din hnh hc, ta cn tm hm (x)
c dng bit sao cho th ca n i qua cc im(x
i
,y
i
) vi mi i=0,1,...,N.

Hnh 1: Minh ha bi ton ni suy hm mt bin
x
0
x
1
x
n
f(x
0
)


f(x)


(x)


Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

2
Hm f thng l hm thc nghim hoc cc hm kh tnh gi tr hm s nn ch o
c cc im nht nh. Cc im { }
0
N
i
i
x
=
s gi l cc mc ni suy.

1.1.1.2 Bi ton ni suy hm nhiu bin

Xt mt hm cha bit : ( )
n m
f D R R c v mt tp hun luyn
{ } ( )
1
, ; ,
N
k k k n k m
k
x y x R y R
=
e e sao cho ( ) ; 1,
k k
f x y k N = = . Chng ta cn tm mt
hm s mt dng bit tha mn iu kin ni suy l :
( ) ; 1,
k k
x y k N = =

1.1.2 Bi ton xp x

Hm ( ) x f y = o c ti N im thuc on | | b a,
1 2 N
x x x < < < L ; ( )
i i
y f x =
Vi 1 k N s , ta tm hm
( ) ( )
1
, , ,
k
x c c x = u K
(1)
Trong u l hm cho trc,
j
c l cc tham s cn tm sao cho sai s trung bnh
bnh phng ( ) ( )
2
1
1
N
i i
i
x y
N

=
=

nh nht khi cc tham s
j
c thay i. Khi ta
ni ( ) x l hm xp x tt nht ca y trong lp hm c dng (1) theo ngha bnh ph-
ng ti thiu. Thng th bi ton tm cc tiu ton cc ca sai s trung bnh bnh ph-
ng l bi ton kh. Trong trng hp u l hm tuyn tnh ca cc
j
c th cc tr
ton cc c th xc nh nh gii h phng trnh tuyn tnh ca iu kin cc o
hm cp mt trit tiu.
( ) ( )
1
1
, , ,
N
k k k
k
c c x c x
=
u =

K
(2)
trong ( ) x
k
l cc hm n gin v c lp tuyn tnh.

1.1.3 Cc phng php gii quyt bi ton ni suy v xp x hm s

Bi ton ni suy hm mt bin l mt lnh vc nghin cu nghin cu kh quan trng
trong ngnh gii tch th k 18. u tin bi ton ni suy c gii quyt bng phng
php s dng a thc ni suy: a thc Lagrange, a thc Chebysev... tuy nhin khi s
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

3
mc ni suy ln th ni suy bng a thc thng xy ra hin tng ph hp tri(over-
fitting) do bc ca a thc thng tng theo s mc ni suy. gii quyt hin tng
ph hp tri thay v tm a thc ni suy ngi ta ch tm a thc xp x (thng gii
quyt bng phng php xp x bnh phng ti thiu ca Gauss...) Mt phng php
khc c xut vo u th k 20 l phng php ni suy Spline. Trong hm
ni suy c xc nh nh ghp trn cc hm ni suy dng n gin (thng dng a
thc bc thp) trn tng on con. Phng hay c p dng niu trong k thut.
hiu r hn v cc phng php trn xem [1,14].

Cng vi pht trin ca cc ng dung CNTT, bi ton ni suy nhiu bin c quan
tm gii quyt v t nhiu tin b trong khong 30 nm gn y, vi cc cch tip
cn nh:
-Hc da trn mu, bao gm cc phng php: k-lng ging gn nht vi trng s
nghch o khong cch v hi quy trng s a phng.
-Mng neural truyn thng MLP
-Mng neural RBF
r hn v cc phng php trn xem [11].

1.2 MNG NEURAL NHN TO

1.2.1 Gii thiu mng Neural nhn to

B no con ngi cha ng nhng b mt m n by gi khoa hc vn cha gii p
c, chnh nh c b no hon chnh m con ngi tr thnh ng vt bc cao
thng tr mun loi. t lu con ngi nghin cu cu trc c bit ca b no t
ng dng gii quyt nhng bi ton khoa hc k thut. Ngi ta pht hin ra
rng b no con ngi l mng li chng cht cc Neural lin kt vi nhau, y l c
s hnh thnh nn cu trc ca mng Neural nhn to.

V bn cht ton hc th mng Neural nhn to nh l mt mt trong khng gian a
chiu xp x mt hm cha bit no y. Nhng mng Neural nhn to li ging
mng Neural sinh hc ch l kh nng c th hun luyn (hc), y l c im
quan trng nht ca mng Neural nhn to. Chnh v c im ny m mng Neural
nhn to c kh nng thc hin tt cc cng vic sau khi c hun luyn, v n
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

4
khi mi trng thay i ta li c th hun luyn li mng Neural nhn to n thch
nghi vi iu kin mi.

1.2.1.1 Mng Neural sinh hc

Mng Neural sinh hc l mt mng li (plexus) cc Neuron c kt ni hoc c lin
quan v mt chc nng trc thuc h thn kinh ngoi bin (peripheral nervous system)
hay h thn kinh trung ng (central nervous system).

Hnh 2: Minh ha mt Neuron thn kinh sinh hc

Trn y l hnh nh ca mt t bo thn kinh (Neuron thn kinh), ta ch thy rng
mt t bo thn kinh c ba phn quan trng:
-Phn u cng c nhiu xc tu (Dendrite) l ni tip xc vi cc vi cc im kt
ni(Axon Terminal) ca cc t bo thn kinh khc
-Nhn ca t bo thn kinh (Nucleus) l ni tip nhn cc tn hiu in truyn t xc
tu. Sau khi tng hp v x l cc tn hiu nhn c n truyn tn hiu kt qu qua
trc cm ng (Axon) n cc im kt ni (Axon Terminal) ui.
-Phn ui c nhiu im kt ni (Axon Terminal) kt ni vi cc t bo thn kinh
khc.

Khi tn hiu vo xc tu kch hot nhn Neuron c tn hiu ra trc cm ng th
Neuron c gi l chy. Mc d W. Mculloch v W.Pitts (1940) xut m hnh
mng neural nhn to kh sm nhng nh Heb (1949) mi l nn tng l lun cho
mng neural nhn to.

Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

5
nh Heb: Khi mt neuron (thn kinh) A gn neuron B, kch hot thng xuyn
hoc lp li vic lm chy n th pht trin mt qu trnh sinh ho cc neuron lm
tng tc ng ny.

1.2.1.2 Mng Neural nhn to

Mng Neural nhn to c thit k m hnh mt s tnh cht ca mng Neural sinh
hc, tuy nhin, khc vi cc m hnh nhn thc, phn ln cc ng dng li c bn cht
k thut. Mng Neural nhn to (ANN) l my m phng cch b no hot ng thc
hin cc nhim v ca n. Mt mng Neural l b x l song song phn tn ln, n
ging b no ngi v 2 mt:
-Tri thc c nm bt bi Neural thng qua qu trnh hc.
- ln ca trng s kt ni Neural ng vai tr khp ni ct gi thng tin.

a) Cu to mt Neuron trong mng Neural nhn to

Hnh 3: Cu to mt Neural nhn to
Mt neuron bao gm cc lin kt nhn tn hiu vo bng s c cc trng s kt ni w
i

tng ng vi tn hiu x
i
, hm F gi l hm kch hot to tn hiu ra da trn gi tr
hm tng c trng s ca cc gi tr u vo, Y l gi tr u ra ca Neuron. Ta c th
biu din mt Neural nhn to theo cng thc ton hc nh sau:
0
1
w
N
i i
i
Y F x w
=
| |
= +
|
\ .

Ty vo thc t bi ton hm F l mt hm c th no y, trong qu trnh hun luyn
(hc) th cc tham s w
i
c xc nh. Trn thc t F thng c chn trong nhng
hm sau:

F

x
1
x
2
x
N
w
1
w
2
w
N
w
0
Y

Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

6
1) Hm ngng
1 0
( ) ( )
1 0
x
F x x
x

<

= =

>

-1.5
-1
-0.5
0
0.5
1
1.5
-6 -4 -2 0 2 4 6

Hnh 4: th hm ngng
2) Hm tuyn tnh

( ) F x ax =
-4
-3
-2
-1
0
1
2
3
4
-6 -4 -2 0 2 4 6
Hnh 5: th hm tuyn tnh
3) Hm sigmoid

1
( )
1
x
F x
e

=
+

0
0.5
1
-6 -4 -2 0 2 4 6
Hnh 6: th hm sigmoid
4) Hm tanh
1
( )
1
x
x
e
F x
e

=
+

-1
-0.5
0
0.5
1
-6 -4 -2 0 2 4 6

Hnh 7: th hm tanh
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

7
5) Hm bn knh
(Gauss)
2
( )
x
F x e

=
0
0.5
1
-6 -4 -2 0 2 4 6

Hnh 8: th hm Gauss


Trn thc t th cc h hm sigmoid thng dng cho mng Neural truyn thng nhiu
tng MLP v cc hm ny d tnh o hm: '( ) ( )(1 ( )) f x f x f x = , trong khi mng
Neural RBF li dng hm kch hot l hm bn knh.

b) Kin trc ca mng Neural nhn to
Kin trc ca mng Neural nhn to ly
t tng chnh ca mng Neural sinh hc
l s kt ni ca cc Neuon. Tuy
nhin, mng Neural nhn to c kin trc
n gin hn nhiu, v c s lng
Neuron v c kin trc mng, trong khi
mng Neural t nhin mt Neuron c th
kt ni vi mt Neuron khc bt k
trong mng th mng Neural nhn to
cc Neuron c kt ni sao cho n c
th d dng c biu din bi mt m
hnh ton hc no y. V d trong mng
Neural truyn ti cc Neuron c phn
thnh nhiu lp, cc Neuron lp trc
ch c kt ni vi cc Neuron lp
sau.
INPUT
OUTPUT
HIDDEN

Hnh 9: Kin trc mng Neural truyn ti


c) Qu trnh hc
Nh ni trn mng Neural nhn to c kh nng hun luyn c (hc), qu trnh
hun luyn l qu trnh m mng Neural nhn to t thay i mnh di s kch thch
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

8
ca mi trng (b d liu hun luyn) ph hp vi iu kin ca mi trng. Qu
trnh hun luyn ch c th c thc hin khi mng Neural nhn to xy dng
c kin trc c th, v hm kch hot F c xc nh. V bn cht qu trnh hc
l qu trnh xc nh cc tham s w
i
ca cc Neuron trong mng Neural. C ba kiu
hc chnh, mi kiu mu tng ng vi mt nhim v hc tru tng. l hc c
gim st, hc khng c gim st v hc tng cng. Di y xin nu ra phng php
hc c gim st cc phng php khc xem thm [10] chapter 4.
Hc c gim st
Trong hc c gim st, ta c cho trc mt tp v d gm cc cp
( , , 1.. ), ,
i i
x y i N x X y Y = e e v mc tiu l tm mt hm : f X Y (trong lp cc
hm c php) khp vi cc v d. Trn thc t ngi ta thng tm hm f sao cho
tng bnh bnh phng sai s t gi tr nh nht trn tp v d:
( )
2
1
( )
N
i i
i
E f x y
=
=

.

1.3 MNG NEURAL RBF

1.3.1 Gii thiu mng Neural RBF

Hm c s bn knh c gii thiu bi M.J.D. Powell gii quyt bi ton ni suy
hm nhiu bin nm 1987. Ngy nay, y l vn ht sc quan trng c nghin
cu trong ngnh gii tch s. Trong lnh vc mng Neural, mng Neural RBF c
xut bi D.S. Bromhead v D. Lowe nm 1988 cho bi ton ni suy v xp x hm
nhiu bin (xem [12]).

1.3.1.1 K thut hm c s bn knh

Bi ton ni suy hm nhiu bin c gii thiu phn 1.1.1.2, nh ni trn
gii quyt bi ton ny D. Powell xut dng ca hm l hm c s bn
knh. Di y s trnh by s lc k thut s dng hm c s bn knh gii quyt
bi ton ni suy hm nhiu bin.

K thut hm c s bn knh
Khng mt tnh tng qut gi s m=1 khi hm c dng nh sau :
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

9
0
1
( ) ( )
N
k k
k
x w x w
=
= +

(1)
y
k
l hm c s bn knh th k. Thng thng
k
c nhng dng sau:
2
2
k
( )
k
k
x v
x e
o

=
(2)
2
2
k
( )
k
k
x x v o = +
(3)
k
2
2
1
( )
k
k
x
x v

o
=
+

(4)
Trn thc t th ngi ta thng cho
k
dng (2) v trong khun kh kha lun ny
ch xt
k
dng (2).
2
2
k
( )
k
k
x v
x e
o

= ch rng y ta dng chun ||.|| l chun Euclide
2
1
N
i
i
u u
=
=

;
k
v
l tm ca mi hm c s bn knh
k
;
k
o l bn knh ca
k
. Vi mi k th gi tr
ca bn knh
k
o iu khin min nh hng ca hm bn knh
k
. Nu
3
k
k
x v o > th gi hm ( )
k
x l rt nh, khng c ngha.


Hnh 10: Minh ha s nh hng ca hm bn knh
V d nh hnh trn mt vng trn to tng trng cho mt hm c s bn knh, cc
hm ny ch nh hng n cc im bn trong bn knh ca n.
Thay cng thc (2) vo (1) ta c biu din ton hc ca k thut hm c s bn knh
nh sau:
2
2
0 0
1 1
( ) ( )
j k
k
x v
N N
j j j
k k k
k k
x w x w w e w y
o


= =
= + = + =


(6)
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

10
Mt c im rt li th khi s dng hm bn knh gii quyt bi ton ni suy hm
nhiu bin, l khi xt gi tr bnh phng sai s
( ) ( )
2
1
N
i i
i
E x y
=
=

th ngi ta
chng minh c rng E ch c mt cc tr duy nht. Do vy vic tm cc tham s ca
cc hm c s bn knh( , ,
k
k k
w v o ) cho E t cc tiu s c gii quyt rt nhanh
v hiu qu.

1.3.1.2 Kin trc mng Neural RBF

Mng RBF l mt loi mng Neural nhn to truyn thng gm c ba lp. N bao gm
n nt ca lp u vo cho vector u vo
n
x R e , N neuron n (gi tr ca neuron n
th k chnh l gi tr tr v ca hm c s bn knh
k
) v m neuron u ra.


Hnh 11: Kin trc ca mng RBF
Nh nu trn mng RBF c th biu din bng cng thc ton hc sau:
( )
2
2
0 0
1 1
( ) ( )
j k
k
x v
N N
j j j
k k k k k
k k
x w x w w e w y
o


= =
| |
|
= + = + =
|
|
\ .



1.3.1.3 ng dng ca mng Neural RBF

Nh u im vt tri l c thi gian hun luyn mng rt ngn ngy nay mng
Neural RBF c s dng trong rt nhiu lnh vc:
-X l nh
-Nhn dng ting ni
I
N
P
U
T

O
U
T
P
U
T

HIDDEN
k

x
i w
i
w
0
w
0
Y
X
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

11
-X l tn hiu s
-Xc nh mc tiu cho Radar
-Chun on y hc
-Qu trnh pht hin li
-Nhn dng mu

1.4 CC PHNG PHP HUN LUYN MNG NEURAL RBF

Hun luyn mng Neural RBF thc ra l qu trnh tm cc tham s ( , ,
k
k k
w v o ) ca cc
hm bn knh ph hp vi bi ton no y. So vi cc mng Neural khc mng
Neural RBF c im mnh hn hn c c thi gian hun luyn ngn. Xt tng bnh
phng sai s
( ) ( )
2
1
N
i i
i
E x y
=
=

, do E ch c mt cc tr duy nht, v th nn vic

i tm im cc tr cho E s rt nhanh chng do khng c cc tr a phng. C rt
nhiu phng php hun luyn mng hu ht cc phng php ny u c c im
chung l u c xu hng cc tiu ha gi tr bnh phng sai s E cho tp d liu
hun luyn. C th chia cc kiu hun luyn mng RBF ra thnh ba loi: hun luyn
mt pha, hun luyn hai pha v hun luyn ba pha (hun luyn y ) (xem [4,5]).

1.4.1 Phng php hun luyn mt pha

Xt tp d liu hun luyn
{ } ( )
1
, ; ,
N
k k k n k m
k
x y x R y R
=
e e phng php ny, ngi ta
thng chn tm
k
v ca cc hm bn knh l mt tp con ca tp d liu hun luyn
{ }
1
N
k
k
x
=
, cn cc bn knh
k
o c gn gi tr l mt hng s no y, trn c s thc
nghim ngi ta thng t
1
1
(2 )
n
k
M
o = (trong M l s hm c s bn knh, n l s
Neural u vo). Gi tr ca cc tham s w
k
thng c tm ra bng cc phng php
hc c gim st nh l phng php gi nghch o hoc phng php tt dc
Gradient. V bn cht th hai phng php ny u tm cc trong s w
k
gi tr bnh
phng sai s E t cc tiu.

1) Phng php gi nghch o
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

12
Vi tp hun luyn
{ } ( )
1
, ; ,
N
k k k n k m
k
x y x R y R
=
e e , gi s mng RBF ca ta c M
Neuron tng n. Ta xt ma trn
N M
H

nh sau
2
( , ) ( )
i
k
k
x v
i
k
H i k x e
o

= = v ma trn
Y l ma trn hng cc y
k
khi gi tr ca cc w
k
c tnh nh sau :
W H Y
+
= trong
1
( )
T T
H H H H
+
= .

2) Phng php tt dc Gradient
Vi phng php ny, u tin cc tham s w
k
c to ra ngu nhin sau cc tham
s ny c cp nht bng cng thc sau :
( 1) ( )
k k k
w i w i w q + = + A
( ) ( ) ( )
1
N
i i i
k k
i
w x y x
=
A =

ch y ta xt mng RBF c mt Neural u ra. H

s q c gi l tc hc, nu q nh th gi tr ca cc trng s w tin chm n
im cc tiu, nhng nu q ln th gi tr ca cc trong s w thng c xu hng dao
ng quanh im cc tiu, ni chung tm c gi tr q hp l th phi qua qu
trnh thc nghim. Thng thng ngi ta vn chn q c gi tr nh m bo qu
trnh lp s hi t v gi tr cc tiu cho d hi mt thi gian.

Hnh 12: Qu trnh hi t n gi tr cc tiu ca thut ton Gradient,
ng nt t ng vi gi tr q ln,
ng nt lin ng vi gi tr q nh

1.4.2 Phng php hun luyn hai pha

Vi phng php hun luyn hai pha thng thng cc gi tr tm v
k
v bn knh
k
o
ca hm c s bn knh
k
c tnh bng cc thut ton phn cm nh thut ton
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

13
phn cm k-mean, k-mean c ngng Sau gi tr ca cc trng s w
k
c tnh
bng cc phng php gi nghch o, hay tt dc Gradient nh nu trn.

1) Thut ton phn cm k-mean
-Pht biu bi ton: Cho tp d liu { }
1 2
, ..., ;
d
n i
X x x x x R = e chng ta cn phn tp
d liu ny thnh k tp { }
1 2
1 1
, ,.. : ; ;
k k
k i i
i i
S S S k n S S X
= =
| |
< = C =
|
\ .
I U
sao cho tha mn:
2
1
argmin
j i
k
j i
S
i x S
x
= e

vi
i
l tm ca tp
i
S .
V mt ton hc th bi ton phn cm trn thuc loi NP-kh tuy nhin trn thc t
th ngi ta thng gii bi ton bng phng php heuristic nh sau:
u tin ta khi to ngu nhin tp { }
1 2
, ,...
k
sau thc hin vng lp qua hai
bc sau:
+Bc 1: to cc cm
{ }
( ) *
*
: ; 1,
t
i j j i j i
S x x x i k = s =
+Bc 2: iu chnh li tm
( )
( 1)
( )
1
i
t
j i
t
j
t
x S
i
x
S

+
e
=


Thut ton s dng cho n khi tp { }
1 2
, ,...
k
khng cn c s thay i gi tr(
hiu chi tit hn v thut ton phn cm k-mean c th tham kho thm [18]).

Sau khi chy thut ton k-mean ta s chn tm v
k
ca cc hm c s bn knh
k
chnh
l tp { }
1 2
, ,...
k
cn bn knh
1
|| ||
j k
k j k
x S
k
x
S
o
e
=

.

1.4.3 Phng php hun luyn 2 pha HDH

Xt tp hun luyn
{ } ( )
1
, ; ,
N
k k k n k m
k
x y x R y R
=
e e , khng mt tnh tng qut, y ta
xt mng RBF c mt Neuron output (m=1), khi biu din ton hc ca mng RBF
l:
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

14
0
1
( ) ( )
N
i i i
k k
k
x w x w y | |
=
= + =

(1)

Xt ma trn
( )
, k i
N N

u = trong
2 2
|| || /
,
( )
i k
k
x x i
k i k
x e
o


= = , ch rng y ta chn
tm ca cc hm c s bn knh chnh l tt c cc im thuc tp d liu input X.
Ta k hiu I l ma trn n v cp N ; W=
(
(
(

N
w
w
...
1
, Z=
(
(
(

N
z
z
...
1
l cc vc t trong khng
gian N-chiu R
N
trong :
z
k
= y
k
w
0
,
N k s
(2)
v t
, k j
N N
I

( + = u =


(3)
th
2 2 ,
|| || /
0
j k
k
k j
x x
k j
e k j
o

=

=

(4)
Khi h phng trnh (1) tng ng vi h :
W= +W +Z (5)
Vi cc tham s
k
o chn v w
0
ty , h (1) v do h (5) lun c duy nht
nghim W. V sau gi tr w
0
c

chn l trung bnh cng ca cc gi tr y
k
:
w
0
=

=
N
k
k
y
N
1
1

(6)
Vi mi ksN, ta c hm q
k
ca
k
o xc nh nh sau:

=
=
N
j
j k k
q
1
,

(7)
Hm q
k
l n iu tng v vi mi s dng q < 1 lun tn ti gi tr
k
o sao cho
q
k
(
k
o )=q.
M t thut ton.
Vi sai s c v cc hng s dng q, o <1 cho trc, thut ton bao gm 2 pha
xc nh cc tham s o
k
v W
*
. Trong pha th nht, ta s xc nh cc
k
o q
k
sq v
gn vi q nht (ngha l nu thay o
k
=o
k
/o th q
k
>q). V vy, vi

mi k, chun ca ma
trn +tng ng vi chun vector
*
. (cho bi cng thc (16) di y) thuc on
ny. Pha sau tm nghim gn ng W
*
ca (5) bng phng php lp n gin. Thut
ton c c t trong hnh 13.
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

15

Hnh 13: Thut ton HDH hun luyn mng RBF
tm nghim W
*
ca h (5) ta thc hin th tc lp nh sau.
Khi to W
0
=Z ;
Tnh
W
k+1
=
k
W + +Z ;
(8)
Nu iu kin kt thc cha tha mn th gn W
0
:= W
1
v tr li bc 2 ;
Vi mi vect N-chiu u, ta k hiu chun

=
=
N
j
j
u u
1
*
, iu kin kt thc c th chn
mt trong biu thc sau.
a)
c s

*
0 1
1
W W
q
q

(9)
b)
q
q Z
q
Z
q
t
ln
) 1 ln( ln ln
ln
) 1 (
ln
* *
+
=

>
c
c
, vi t l s ln lp.
(10)
c tnh hi t.
Vi mi vect N-chiu u, ta k hiu chun
*
u cho bi cng thc :

=
=
N
j
j
u u
1
*

(11)
Th thut ton trn lun kt thc sau hu hn bc v nh gi sau ng.
c s
*
* 1
W W
(12)
K hiu chun
*
+ ca ma trn +tng ng vi chun vect (11) l :
*
+ = { } ] , [ max q q q
k
N k
o e
s
s q
(13)
ta c nh gi :
1 1
1 * 1 0
*
* *
1 1
t t
q q
W W u u Z
q q
+ +
s =


(14)
Proceduce Thut ton 2 pha hun luyn mng RBF
for k=1 to N do
Xc nh cc o
k
q
k
sq, v nu thay o
k
=o
k
/o th q
k
>q; // Pha 1
Tm W
*
bng phng php lp n (hoc phng php lp Seidel); //Pha 2
End

Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

16
Biu thc (10) tng ng vi v phi ca (14) nh hn hoc bng c . Mt khc
bc cui ca pha 2, nu p dng (14) cho t=0 ;
1 1 0 0
; W u W u = = ; p dng (14) cho
t=0 th ta c :
1 * 1 0
* *
1
q
W W w w
q
s

(15)
Thut ton ny c u im l ci t rt n gin v tc hi t rt nhanh v ta c
th iu chnh gi tr sai s ni suy nh ty . Song do kin trc mng phc tp nn
thng xy ra hin tng ph hp tri(over-fitting) cho tp d liu hun luyn.
hiu chi tit hn v thut ton HDH xem thm [2,3].

1.4.4 Phng php hun luyn ba pha y

Phng php ny s dng phng php tt dc Gradient tm kim c ba tham s
dng cc cng thc lp di y:
( )
( ( )) ( ) ( ) 1
1
1 1
2
w
Q J
k q q q
mj mj mj j j m
q j
mj
E
w w z y
w M
q
q
= =
| | (
c | |
= = +
|
( |
|
c
\ .

\ .


( ) ( ) ( )
( )
( ) ( ) ( )
( )
( ) ( ) 2
2
1 1
2
Q J
k q q q q m m m
n n j j mj m n n
q j
v v z w y x v
M
q

o
= =
(

= +
` (

)


( ) ( ) ( )
( )
( ) ( ) ( )
2
2 2 3
4
1 1 1
2
2
q q q
Q J M
mj m
k q q
m m j j
q j m
m
w x v
w z
M

q
o o
o
= = =

(

(
= +
`
(

(
)


Ch y ta dng tp d liu hun luyn
{ } ( )
1
, ; ,
Q
k k k n k m
k
x y x R y R
=
e e trong
( ) k q l d liu hun luyn th q. r hn v phng php trn xem [5]. Nhn chung
phng php ny cho kt qu hun luyn kh tt, song nhc im l thi gian hun
luyn di nn khng ph hp vi cc bi ton c d liu ln.

1.5 KT QU THC NGHIM

1.5.1 Kt qu

Di y ti xin gii thiu kt qu thc nghim khi tin hnh ci t cc phng php
khc nhau hun luyn mng RBF. B d liu hun luyn y l d liu c ly
t bi ton phn tch thnh phn trong ng du. Phng php ly d liu nh sau:
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

17
ngi ta dng tia gamma chiu vo ng du thu c cc c trng khc nhau ng
thi ti thi im ngi ta phn tch t l du v nc c trong ng du. Do t l
thnh phn du v nc c lin quan n cc c trng thu c khi chiu tia gamma,
ngi ta mong mun d on c thnh phn du v nc da vo cc c trng thu
c khi chiu tia gamma vo ng du. B d liu bao gm 900 b d liu hun luyn
v 100 b d liu kim tra. Mi d liu bao gm 12 gi tr u vo (cc c trng thu
c khi chiu tia gamma) v 2 gi tr u ra (t l du v nc), tt c u c biu
din bng s thc. D liu c ti t a ch:
http://sites.google.com/site/rbfneuronnetwork/Home/OilData.zip

Ti tin hnh xy dng mng Neural RBF vi kin trc nh sau: c 12 neuron u
vo, 2 neuron u ra, cn s lng neuron tng n ty vo thut ton hun luyn. Sau
tin hnh hun luyn vi 4 phng php hun luyn khc nhau so snh hiu
sut(thi gian hun luyn, sai s ca b d liu hun luyn, sai s ca b d liu kim
tra) ca chng. Cc phng php dng hun luyn l:
1) Phng php hun luyn 2 pha HDH(xem 1.4.3)
2) Phng php hun 3 pha y (xem 1.4.4)
3) Phng php hun luyn 1 pha, s dng phng php gi nghch o tm
tham s w
i
.(xem 1.4.1)
4) Phng php hun luyn 2 pha, s dng phng php gi nghch o tm
tham s w
i
. (xem 1.4.2)

Chng trnh c chy trn my cu hnh nh sau: HH Windows XP Professinal,
CPU Intel Core 2 Duo E6300 1.86GHz, Ram 1G. Sau y l bng kt qu so snh hiu
sut cc phng php hun luyn:
Chng 1: Bi ton ni suy, xp x hm s v mng Neural RBF

18
Phng php hun luyn
Sai s trn tp d liu
kim tra(tng bnh
phng sai s)
Sai s trn tp d liu
hun luyn(tng bnh
phng sai s)
Thi gian hun
luyn(giy)
Phng php hun luyn 2 pha HDH
q=0.9, alpha=0.7, epsilon= 1e-5 3.095031 3.2938e-11 1.921000
q=0.9, alpha=0.75, epsilon= 1e-5 2.929305 2.8989e-11 1.859000
q=0.9, alpha=0.9, epsilon= 1e-5 2.275569 2.9517e-11 2.609000
q=0.99, alpha=0.7, epsilon= 1e-5 2.93460 3.7486e-11 1.703000
q=0.99, alpha=0.8, epsilon= 1e-5 2.47516 1.0336e-11 2.750000
q=0.99, alpha=0.9, epsilon= 1e-5 2.13895 1.2286e-11 1.984000
Phng php hun luyn 3 pha y
S neuron tng n 20, loop = 1000 0.40966 0.18352 1312.266000
S neuron tng n 50, loop = 500 0.38691 0.17951 4198.204000
S neuron tng n 100, loop = 500 0.38206 0.24483 11961.4530
Phng php hun luyn 1 pha + gi nghch o
S neuron tng n 180 0.2270 1.0226 1.968000
S neuron tng n 280 0.14226 0.54680 2.437000
S neuron tng n 350 0.091694 0.3864 2.828000
Phng php hun luyn 2 pha + gi nghch o
S neuron tng n 180 0.2666 1.2269 2.093000
S neuron tng n 280 0.17365 0.9048 2.593000
S neuron tng n 350 0.20062 0.5255 3.046000
Bng 1: Kt qu thc nghim hiu sut cc phng php hun luyn mng Neural RBF
Chng II: Nhn dng ch vit tay
19
1.5.2 Nhn xt

Da vo bng s liu trn ta thy rng:
+Thut ton 1) c thi gian hun luyn rt nhanh, cho kt qu sai s trn b d liu
hun luyn l rt tt, nhng n c sai s trn b kim tra l kh cao, nh vy phng
php hun luyn HDH l la chn tt gii quyt bi ton ni suy, hoc gii quyt
cc bi ton m d liu u vo c mt tp trung dy c.
+Thut ton 2) cho kt qu sai s trn c b d liu hun luyn v b d liu kim tra
l rt nh, tuy nhin n c im rt hn ch l thi gian hun luyn mng l rt lu.
Ni chung phng php ny khng phi l mt la chn tt gii quyt cc bi ton
c b d liu hun luyn ln.
+Thut ton 3), 4) cho kt qu sai s trn c b d liu hun luyn v b d liu kim
tra l rt nh v thi gian hun l rt nhanh. Tuy nhin c 2 thut ton ny c mt
nhc im l s dng phng php gi nghch o tm cc tham s w
i
m
phng php gi nghch o li s dng nhiu php tnh nhn, nghch o ma trn y
l cc php tnh kh ci t v thng cho sai s tnh ton cao. y ti s dng th
vin MATLAB thc hin cc php tnh ton ny.


Chng 2: Nhn dng ch vit tay
20
CHNG 2
NHN DNG CH VIT TAY

Ni dung chng ny gm c:
2.1 Nhn dng mu
2.2 Bi ton nhn dng ch vit tay
2.3 Cc phng php trch chn c trng ch vit tay
2.4 Kt qu thc nghim

2.1 NHN DNG MU

Nhn dng ch vit tay l mt lnh vc con ca nhn dng dng mu, do vy trc khi
i su vo trnh by chi tit bi ton nhn dng ch vit tay, ti xin trnh by s lc
v lnh vc nhn dng mu v bi ton nhn dng mu.

2.1.1 Nhn dng mu

2.1.1.1 Mu l g ?

Mu(pattern) c th phn thnh 2 loi : mu tru tng v mu c th. Cc tng,
lp lun v khi nim... l nhng v d v mu tru tng, nhn dng cc mu nh vy
thuc v lnh vc nhn dng khi nim. Cc mu c th bao gm cc i tng c
tnh khng gian, thi gian v hnh nh... Cc i tng vt l, ch k, ch vit, k
hiu, nh, on sng m thanh, in no hoc in tm , hm s... l nhng v d
v mu c th.

2.1.1.2 Nhn dng mu l g ?

Khng c mt nh ngha thng nht cho nhn dng mu (Pattern recognition) nhng
iu ny khng gy tranh ci g trong gii nghin cu. Sau y l mt s nh ngha
theo ng cnh nghin cu :
- Duda Et Al: Nhn dng mu l vic quy nhng i tng vt l hay s kin vo mt
loi (nhm) no xc nh t trc.
Chng 2: Nhn dng ch vit tay
21
- Jrgen Schrmann: Nhn dng mu l vic gn nhn w cho mt quan st x.
- Selim Aksoy: Nhn dng mu l vic nghin cu cch lm cho mt my c th thc
hin:
+ Quan st mi trng
+ Hc cch phn bit c cc mu cn quan tm
+ a ra cc quyt nh ng n v loi (nhm) ca cc mu

2.1.1.3 Lch s ca lnh vc nhn dng mu

Nhn dng mu c lch s kh lu i, trong thp k 60 ca th k 20 hu ht vn
nhn dng mu dng li vic nghin cu l thuyt thng k. V sau vi s pht
trin mnh m ca my tnh th phn thc nghim cng tr nn n gin hn. Khi m
x hi chng ta ang pht trin t thi k cng nghip sang hu cng nghip, i vi
vn t ng ha th vic thng tin c nhn v x l mt cch t ng l rt cn
thit. Khuynh hng ny lm cho vn nhn dng mu tr nn rt quan trng trong
ng dng k thut v trong nghin cu ngy nay. Nhn dng mu tch hp hu ht vo
cc h thng my mc thng minh, c kh nng t a ra quyt nh gii quyt vn
.

2.1.1.4 ng dng ca nhn dng mu

Nhn dng mu c rt nhiu ng dng trong i sng cng nh trong khoa hc k
thut :
-Trong nng nghip : Nhn dng mu c s dng phn tch ma mng, d bo
cc i dch nh chu chu, su bnh, cm gia cm, cm ln... Ngoi ra nhn dng
mu cng cn c dng phn loi t t cc nh c chp t v tinh.
-Khm ph tri thc trn Web : Ngy nay vic bng n lng thng tin khng l trn
Internet lm cho vic tm kim v lc thng tin trn mng l ht sc quan trng. Nhn
dng mu c nhng vo cc my tm kim tr li kt qu tm kim thng minh v
chnh xc. Ngoi ra n cng c trong cc h thng lc th rc, nhn dng t ng
cc trang web en.
-Trong lnh vc y hc : Phn tch v biu din gene, phn loi sinh hc da trn thng
tin di truyn.
-Trong lnh kinh t : Phn tch nh gi s thay i kinh t, ch s chng khon...

Chng 2: Nhn dng ch vit tay
22
2.1.1.5 Cc bi ton nhn dng mu

Trn thc t thng gp cc bi ton nhn dng mu sau :
-Phn lp (classify) : Da trn mt tp con bit nhn, a ra mt cch phn cc i
tng thuc tp nn thnh cc lp.
-Phn cm (cluster) : Chia tp i tng thnh nhm sao cho cc i tng trong mi
nhm tng i ging nhau cn cc i tng khc nhm th khc nhau.
-Phn tch hi quy (regression) hay nhn dng hm : Xc nh mt bin (hm) qua tp
cc bin khc.
-Nhn thc (Identify) : Xc nh i tng trong tp cho c l i tng ang quan
tm hay khng. Chng hn nh nhn thc vn tay, nhn thc mt ngi...
-M t : M t cc i tng di hnh thc d phn tch. V d i tng. m t in
tm di dng biu c trng hoc xu m.

2.1.1.6 Cc bc x l trong h thng nhn dng mu

Mc d c rt nhiu loi bi ton nhn dng mu, tuy nhin gii quyt mt bi ton
th mt h thng nhn dng mu phi thc hin qua cc bc c bn di y :

Hnh 14: Cc bc x l trong h thng nhn dng mu

1)Thu nhn tn hiu
Nu l h nhn dng i tng vt l, u vo ca h thng thng l mt loi thit
b chuyn i nh my ghi hnh hay ghi m Thit b ny thu nhn tn hiu v i
tng nhn dng. Cc tn hiu ny thng thng s c s ha, sau s c
tin hnh tin x l nh: lc nhiu, tch ngng

2) Phn on (segmentation)
Phn on l mt trong nhng bi ton rt kh trong nhn dng mu. Chng hn,
trong bi ton nhn dng vn bn in ra d liu text th giai on phn on chnh l
u vo
Thu tn hiu, tin x l
Phn on Trch chn c trng
Nhn dng
Hu x l
Chng 2: Nhn dng ch vit tay
23
vic xc nh u l vng d liu text nhn dng, tip ta phi tch c nhng
vng c th l mt t, ri li tch tip ra tng k t... Nh vy c th ni vic phn
on trong bi ton nhn dng mu l qu trnh xc nh c u l vng d liu cn
quan tm.

3) Trch chn c trng
Ranh gii khi nim gia vic trch chn c trng v phn lp mc no c
phn khng r rng: mt b trch chn c trng l tng phi lm cho cng vic cn
li ca b phn lp tr nn d dng Mc tiu chung ca b trch chn c trng l da
trn tn hiu thu c m t cc i tng bng cc gi tr ca chng m chng c gi
tr gn xp x nhau i vi cc i tng thuc cng loi v khc xa nhau nu khc
loi. Hn na tin x l th cng t c trng cng tt. iu ny dn n vic phi
tm ra cc c trng khc nhau v chng khng ph thuc hon cnh ta thu tn hiu v
i tng. u ra ca cng on ny c gi l vector c trng ca i tng,
thng thng y l mt vector s thc.

4) Nhn dng
Nhim v ca thnh phn ny trong h thng l s dng cc vector c trng c
cung cp t bc trc (trch chn c trng) gn cc i tng vo cc lp hoc
phn tch hi quy hay m t i tng. Cc k thut thng c s dng cho cng
on nhn dng l: thut ton k-lng ging gn nht, mng neural, my vector h
tr SVM... Ni chung, bc ny gn nh c cng thc x l c nh thng
khng b ph thuc vo bi nhn dng mu c th no.

5) Hu x l
Mt b nhn dng him khi ch dng n l. Thay vo n thng dng a ra
thao tc tng ng, mi thao tc mt mt chi ph tng ng. Hu x l s dng u ra
ca b phn lp quyt nh thao tc tng ng. Theo quan nim, cch n gin
nht nh gi hot ng ca mt b nhn dng l xem t l nhn dng sai vi cc
mu mi. Do chng ta cn phi nhn dng vi t l li thp nht. Tuy nhin chng
ta cn cc thao tc tng ng phi lm cho tng chi ph l thp nht. C th phi kt
hp cc tri thc bit v chi ph, v n s c nh hng n vic ra cc quyt nh
hnh ng. Chng ta cng cn c lng trc chi ph xem c tha mn hay
khng.

Chng 2: Nhn dng ch vit tay
24
2.2 BI TON NHN DNG CH VIT TAY

2.2.1 Tnh hnh chung v nhn dng ch vit tay

Bi ton nhn dng ch vit tay c ng dng rt nhiu trong thc th : c tch
hp vo h thng nhn dng form t ng, tch hp trong cc my PDA c mn hnh
cm ng, nhn dng ch k... Do c nhiu ng dng quan trng nh vy nn t lu bi
ton nhn dng ch vit tay thu ht rt nhiu ngi nghin cu, tm cch gii quyt.
Ngy nay bi ton nhn dng ch vit tay c gii quyt gn nh trn vn trn th
gii cng nh Vit Nam. H nhn dng s dng mng Neural nhn chp gii thiu
[8] t chnh xc n 99.60% (trn b d liu MNIST). y l chnh xc gn
nh tuyt i v nhanh chng c p dng vo rt nhiu ng dng. Cc sn phm
ng dng khc c ngha thc t ln c th k n nh sn phm FineReader ca
hng AABYY c th nhn dng 20 th ting khc nhau, sn phm OmniPage ca hng
ScanSoft nhn dng ch in ting Anh,. v Vit Nam, chng ta c sn phm
VNDOCR ca Vin Cng ngh thng tin nhn dng ch in ting Vit vi chnh xc
ti 99%.

2.2.2 Gii thiu bi ton nhn dng ch vit tay

Nhn dng ch vit tay c thc hin qua hai hnh thc l nhn dng online v
nhn dng offline. Nhn dng online c ngha l my tnh s nhn dng cc ch c
vit ln mn hnh ngay khi n c vit. i vi nhng h nhn dng ny, my tnh s
lu li cc thng tin v nt ch nh th t nt vit, hng v tc ca nt vit trong
khi n ang c vit. Cn nhn dng offline tc l vic nhn dng c thc hin sau
khi ch c vit hay in ln giy ri, lc thng tin u vo l hnh nh vn bn
hoc k t cn nhn dng.

Trong khun kh ni dung kha lun ny ti ch xt hnh thc nhn dng offline cho
tng k t mt.

2.2.3 Hng gii quyt cho bi ton nhn dng k t vit tay
Nh ni trn bi ton nhn dng ch vit tay thuc lp bi ton nhn dng mu,
nh vy gii quyt bi ton nhn dng ch vit tay th phi tun theo cc bc ca
bi ton nhn dng mu nu phn 2.1.1.6. Tuy nhin do y ta ch xt vic nhn
Chng 2: Nhn dng ch vit tay
25
dng tng k t vit tay mt nn bc tin x l v bc phn on xem nh khng
cn thit. C th khi qut qu trnh nhn dng ch vit tay thng qua hnh v di
y :

Hnh 15 : Cc bc gii quyt bi ton nhn dng ch vit tay
u vo l bc nh ca mt k t cn nhn dng, sau khi qua b trch chn c trng
ch vit tay ta thu c vector c trng ca k t , vector c trng c chuyn
n b nhn dng thc hin lng gi v a ra kt qu nhn dng. Trong bi ton
nhn dng ch vit tay c rt nhiu phng php trch chn c trng ch vit, di
y ti s trnh by chi tit 3 phng php trch chn c trng ch vit l : phng
php s dng b phn tch thnh phn chnh, phng php s dng momen Legendre,
phng php s dng mng Neural nhn chp. Trong bi ton nhn dng ch vit tay
b nhn dng thng s dng cc cng c sau : phng php k-lng ging gn nht,
mng Neural truyn thng MLP, my vector h tr SVM, mng Neural RBF... Trong
khun kh ca kha lun ny ti ch xt b nhn dng l mng Neural RBF.

2.3 CC PHNG PHP TRCH CHN C TRNG CH VIT TAY

Trch chn c trng ng vai tr ht sc quan trng gii quyt bi ton nhn dng
ch vit tay. Thc cht qu trnh trch chn c trng ca ch vit tay tc l ta i tm
cc i lng biu din cho cc k t vit tay, m cc i lng ny t b thay i
khi k t c s bin i v hnh dng. Mt phng php trch c trng ch vit tt
cho ra c cc c trng t b bin i khi k t b bin dng nhiu. Di y xin
di thiu 3 phng php trch chn c trng hay c s dng cho bi ton nhn
dng ch vit tay l :

2.2.1 Phng php trch chn c trng kt hp bin i DCT v thut ton
phn tch thnh phn chnh PCA
Khi nim gi tr ring v vector ring (xem chi tit [22])c cc nh ton hc tm
ra vo cui th k 19 u th k 20. C hng lot tnh cht quan trng lin quan n
gi tr ring v vector ring, cc tnh cht ny c ng dng rt nhiu trong cc lnh

D liu u
vo (dng
nh)

Trch chn
c trng
ch vit


Nhn dng

a ra kt
qu nhn
dng
Chng 2: Nhn dng ch vit tay
26
vc k thut. Di y l mt trong cc tnh cht : cho tp d liu X, A l ma trn
hip phng sai ca tp d liu X (A l ma trn vung). Khi tp vector ring ca A
l tp cc vector trc giao, v khi chiu tp d liu X ln tp cc vector ring ny, th
d liu s dao ng nhiu quanh cc vector ring tng ng vi gi tr ring ln, d
liu t dao ng xung quanh cc vector ring tng ng vi cc gi tr ring nh.


Hnh 16: nh hng ca vector ring, gi tr ring ln tp d liu

Thut ton phn tch thnh phn chnh (PCA) l h qu trc tip ca tnh cht nu
trn. Trong k thut thut ton PCA l mt thut ton ph bin c s dng gim
s chiu ca d liu nhng vn gi c nhiu thng tin khi phn bit vi cc d liu
khc. Thut ton PCA c xut u tin bi H. Hotelling (ban u c bit vi
tn l bin i Hotelling-Hotellings transform). Di y l ni dung thut ton PCA,
sau xin gii thiu phng php trch chn c trng bng cch s dng bin i
cosin ri rc (DCT) kt hp vi thut ton PCA thu gn s chiu ca d liu.

2.2.1.1 Thut ton PCA

Gi s ta c tp d liu
1
{ }
D N
i i
X x R
=
= e trong cc x
i
c sp xp thnh cc hng
c ma trn X c kch thc N D . Ta mong mun lm gim s chiu d liu tp
X c D chiu thnh tp d liu Y c L chiu, L < D. C th tm tt thut ton PCA nh
sau(xem chi tit thut ton [10,19], cc khi nim c s xem thm [10]) :
+Bc 1: Tnh gi tr trung bnh trn tng chiu d liu
Sau tnh gi tr trung bnh trn tng chiu d liu 1...D ca tp d liu X ta c vector
gi tr trung bnh u c kch thc 1 D trong
1
1
[ ] [ , ]
N
n
u d X d n
N
=
=


Chng 2: Nhn dng ch vit tay
27
+Bc 2: Thc hin gim gi tr ca d liu trn tng chiu vi gi tr trung bnh u[d]
tng ng:
Ta lu gi tr tnh c vo ma trn B
B = X dh trong h l ma trn c kch thc 1 x N; [1, ] 1; 1... h n n N = =
+Bc 3: Tnh ma trn hip phng sai ca ma trn B
| |
1

D D
T T
N

( = = =


C B B BB B B E E
+Bc 4: Tm gi tr ring v vector ring ca ma trn hip phng sai C
u tin ta tnh tp gi tr ring ; 1..
d
d D = ca ma trn C bng cch gii phng trnh
det( ) 0 C I = sau ta tm tp vector ring
1
{ }
D
d d
V v
=
= cch gii phng trnh
1
V CV D

= trong
| |
MxM
;
D i, j
0;
i
i j
i j
=
=

=

+Bc 5 : Sp xp tp gi tr ring
d
v tp vector ring tng ng theo chiu gim
dn ca gi tr ring
+Bc 6 : Chn ly L vector u tin t tp vector ring sau khi sp xp.
[ , ] [ , ]; 1, , ; 1, , W p q V p q p D q L = = . = .
+Bc 7 : Tnh gi tr ca Y
*
T
Y X W =
Hnh 17 : Cc bc thc hin ca thut ton PCA
Nhn chung vic tnh gi tr ring v vector ring l mt bi ton kh trong ton hc.
Trong phn thc nghim ca mnh, ti s dng th vin MATLAB thc hin tm
gi tr ring v vector ring ca ma trn.

2.2.1.2 Phng php trch chn c trng kt php bin i DCT v thut ton
PCA

y ta xem d liu u vo l ma trn im nh, hnh v di y m t cc bc
ca thut ton:

Hnh 18: Cc bc trch chn c trng bng bin DCT kt hp PCA
D liu u
vo (ma
trn im
nh NxM)
Bin
i
DCT
Ly d
liu theo
ng
zigzag

PCA
Output
(vector
c
trng)
Chng 2: Nhn dng ch vit tay
28
T nh d liu ban u kch thc MxN dng php bin i Cosin ri rc (DCT xem
[20]) ta c ma trn MxN cc h s thc. Sau ta ly d liu theo ng zigzag
nh hnh v bn di ta c vector vi MxN c trng. Tip ta dng thut ton
PCA thu gn s chiu ca vector c trng, khi kt qu u ra sau bc ny xem
nh l vector c trng ca bc nh u vo.

Hnh 19: Bin i DCT v cch ly d liu theo ng zigzag

Ta thy rng bc nh gc qua php bin i DCT th ta khng b mt thng tin v ta c
th s dng php bin i DCT nghch o thu c hnh nh gc. ma trn h s
ca php bin i DCT, cc im gn gc (1,1) th hin mc sng nn ca bc nh
gc, cn cc im im cng xa im gc th hin mc chi tit ca bc nh( trong
thut ton nn nh JPEG sau khi bin i DCT ngi ta thng lc b cc h s xa
im gc, m mt ngi vn khng nhn thy s thay i), nh vy sau khi ly d liu
cho vector c trng theo ng zigzag th cc im gn nhau trn ng zigzag(cng
nh trong vector c trng) lun c quan h v mt gi tr cng nh nh hng i vi
nh gc. iu ny gip cho vector c trng mng nhiu thng tin cho vic nhn dng.
Cui cng vector c trng c x l qua thut ton PCA thu gn s chiu d liu
gip cho vic nhn dng c hiu qu hn.

2.2.2 Phng php trch c trng s dng Momen Legendre

Trc khi i sau vo chi tit thut ton xin gii thiu cc khi nim c s:

2.2.2.1 Momen v Momen Legendre
1) Khi nim Momen (ton hc)
Trong ton hc khi nim momen xut pht t khi nim momen trong vt l, gi tr
momen bc n ca hm s thc f(x) ti c c nh ngha nh sau:
Chng 2: Nhn dng ch vit tay
29
( ) ( )
n
n
x c f x dx

' =
}

(1)

Trong trng hp f l hm hai bin s th momen bc (n+m) ca hm s f(x,y) ti
im (c
1
, c
2
) c nh ngha nh sau:
1 2
( ) ( ) ( , )
n m
n m
x c y c f x y dxdy

+

' =
} }

(2)
2) Khi nim Momen trong x l nh:
Momen nh (image moments) l trng s trung bnh c th no ca sng cc
im nh, nu nh l nh lin tc th nh ngha momen nh ging cng thc (2), nu
nh ri rc ha th momen bc (m+n) ti im (c
1
,c
2
) ca nh c nh ngha nh
sau:
1 2
1 1
( ) ( ) ( , )
M N
m n
m n
x y
x c y c I x y
+
= =
' =

(3)
Trong M N l kch thc ca nh, I(x,y) l sng ca nh ti im (x,y). Gi tr
momen nh c c th dng m t cc i tng nh sau khi c phn on.
Mt s thuc tnh c bn ca i tng nh c th c pht hin thng qua momen
nh: din tch, tm, hng ca vt th...

Cc momen bt bin (invarians moments) trong php bin i nh c nghin cu
vo thp nin 60 ca th k 20. Momen bt bin l nhng gi tr momen khng thay
i trong cc php bin i nh nh di hnh, xoay, ng dng... Trong giai on
nghin cu ny rt nhiu loi momen c gii thiu: Momen Legendre, momen
Zernike... Km theo l rt nhiu thut ton nhanh v hiu qu tnh gi tr cc
momen ny.

3) nh ngha momen Legendre:
Momen Legendre bc (m+n) ca nh lin tc f(x,y) c nh ngha nh sau:
1 1
1 1
(2 1)(2 1)
( ) ( ) ( , )
4
mn m n
m n
P x P y f x y dxdy

+ +
' =
} }

Trong , 0,1, 2,..., m n = , P
m
, P
n
l a thc Legendre. H a thc Legendre l tp
cc a thc trc giao trn on [-1,1]. tn ti momen Legendre c xc nh th
hm f(x,y) phi xc nh trn on [-1,1]. a thc Legendre bc n c nh ngha
nh:
Chng 2: Nhn dng ch vit tay
30
0
( )
n
j
n nj
j
P x a x
=
=

trong
( )/2
1 ( )!
( 1) ( ) 0 mod 2
( ) ( )
2
( )!( )! !
2 2
0 ( ) 1 mod 2
n j
n
nj
n j
n j
n j n j
j a
n j

M
M

-1
-0.5
0
0.5
1
1.5
2
-1 -0.5 0 0.5 1
n=0
n = 1
n = 2
n = 3
n = 4
n = 5

Hnh 20: H a thc Legendre
Ta c th xy dng li nh f(x,y) t momen Legendre bng cng thc sau:
0 0
( , ) ( ) ( )
k
kl k l
k l
f x y p x p y

= =
~

Vi nh c ri rc ha P
xy
ta c cng thc sau tnh momen Legendre:
(2 1)(2 1)
( ) ( )
4

mn m n xy
x y
m n
P x P y P
+ +
' =

.
Ngi ta chng minh c rng gi tr momen Legendre l i lng bt bin trong
cc php di hnh, v ng dng (xem [7,15]). V th momen Legendre thng c
s dng nhn dng cc i tng hnh hc khi n b bin i bng php di hnh
hay php ng dng. Tuy nhin nhc im ca momen Legendre l n khng phi
l momen bt bin trong php xoay hnh v vy tm ng dng ca n vn cn hn ch.
Di y xin gii thiu phng php trch chn c trng ch vit bng cch s dng
momen Legendre.
2.2.2.2 Phng php trch chn c trng ch vit tay bng Momen Legendre

Chng 2: Nhn dng ch vit tay
31
T tng chnh ca phng php nh sau: Gi s ta c nh u vo X, ta t
MAX_ORDER l mt gi tr cho trc no y, sau ta tnh tt c cc gi tr Momen
Legendre bc(m+n) sao cho m < n < MAX_ORDER. Cc gi tr Momen ny c lu
v ma trn L:
(2 1)(
[ ,
2 1)
( ) ( )
4
]
mn m n xy
x y
m n
L m n P x P y X
+ +
' = =

. Ta gi ma trn L
l ma trn momen Legendre. Sau khi tnh xong ma trn L ta vit cc gi tr ca ma trn
L thnh mt hng v xem nh l gi tr c trng ca nh u vo X. Do tnh cht
ca Momen Legendre l bt bin vi cc php bin i nh: di hnh v xoay hnh
nn nu d liu kim tra ng dng hay b di hnh so vi d liu hun luyn th s
cho kt qu nhn dng rt tt. Sau y l tng bc c th ca phng php trch chn
c trng ch vit tay bng Momen Legendre:
Bc 1: Chuyn i ta :
Ch rng momen Legendre ch c xc nh trn on [-1,1] nn p dng c vi
mt bc nh kch thc M N bt k th ta phi thc hin bc i ta nh sau: ta
gi i,j l ta trn nh gc 0 , 0 i N j M s s s s , x,y l ta sau khi i
1 1, 1 1 x y s s s s , ,
2 2
c c
N M
i j
( (
= =
( (

v
( )
max ,
c c
D i i j j = khi cng thc
chuyn i ta l ( , ) ,
c c
i i j j
x y
D D

| |
=
|
\ .
.
Bc 2: Tnh ton gi tr ca ma trn Momen Legendre:
Di y l th tc vit bng gi m tnh gi tr ca ma trn Momen Legendre
For k:=0 to MAX_ORDER do
For l:=0 to k do
( , ) : 0 k l l =
For i:= 0 to N do
For j:=0 to M do
:
c
i i
x
D

=
:
c
j j
y
D

=
( , ) : ( , ) ( ) * ( ) * ( , )
k l l
k l l k l l P x P y f x y

= +
End
End

( , ) *(2 2 1) *(2 1)
( , ) :
( 1) *( 1)
k l l k l l
k l l
N M

+
=


End

Chng 2: Nhn dng ch vit tay
32
Ch tnh nhanh c P
n
(x) ta c th s dng cng thc truy hi sau (xem [7,15]):
( )
0 1 1 2
( ) 1; ( ) ; ( ) 2 1 ( ) ( 1) ( ) /
n n n
P x P x x P x n xP x n P x n

= = = (

.
Bc 3: Chuyn ma trn Legendre thnh vector c trng
Thc cht bc ny ta thc hin vit gi tr cc im trong ma trn Legendre thnh mt
hng (hay vector) v ta xem l vector c trng ca nh u vo.
Hnh 21: Cc bc thc hin ca phng php trch chn c trng s dung momen Legendre

2.2.3 Phng php s dng mng Neural nhn chp (Convolution neural
network)

Phng php trch chn c trng s dng mng neural nhn chp (chi tit xem [8,
16]) c xut u tin bi LeCun v Bengio vo nm 1995. y l phng php
trch chn c trng cho chnh xc nhn dng cao nht hin nay (t chnh xc
n 99.6% vi b d liu MNIST). Trc khi i su vo chi tit thut ton xin gii
thiu khi nim c s l: thao tc nhn chp trong x l nh (convolution
operator).

2.2.3.1 Khi nim c s

1) Thao tc nhn chp
Nhn chp l mt khi nim quen thuc trong x l s tn hiu (xem [23]), trong lnh
vc x l nh thao tc nhn chp c dng bin i nh thnh mt dng mong
mun no y: nh lm ni cnh (detect edge), l mt nh (smoothing), lm sc
nt(sharpening)... Trong thao tc nhn chp ngi ta s dng mt n nhn chp
(convolution mask)- l ma trn vung 2 chiu c kch thc l s l(thng thng l
3x3 v 5x5) v tc ng nhn chp vo tt c cc im ca nh gc. Thao tc nhn
chp ln mt im ca nh gc c thc hin nh sau:
Chng 2: Nhn dng ch vit tay
33
+Bc 1: u tin tm ca mt n nhn
chp c t trng vo im cn tnh
nhn chp.
+Bc 2: Thc hin nhn tng gi tr
ca mt n nhn chp vi gi tr sng
ca im nh tng ng
+Bc 3: Cng tng ca tng gi tr
tnh bc 2 kt qu lu vo v tr nhn
chp ca nh gc.

Hnh 22: Thao tc nhn chp
(ngun http://micro.magnet.fsu.edu/)


2.2.3.2 Phng php trch c trng s dng mng Neural nhn chp

T tng chnh ca phng php ny l bin mt bc nh t phn gii cao v nh
c phn gii thp hn nhng li mang nhiu thng tin hn cho qu trnh nhn dng
hn (bng php nhn chp v thu nh nh). Phng php ny t hiu qu cao cho
vic trch c trng ch vit tay l bi v nh thao tc nhn chp n lm ni bt ln
c cc ng nt, hnh dng, cc giao im... ca k t vit tay. Hn na cc c
trng ca n t b thay i bi cc php dch nh v php bin dng bp mo nh
(distortion).
1) Kin trc mng
Ta gi thit nh u vo c kch thc 29x29, di y l kin trc mng ly c
trng:
+Bc 1: u tin ta s dng 5 mt n nhn chp khc nhau kch thc 5x5 thc hin
thao tc nhn chp ln nh gc, sau khi nhn chp ta thc hin php thu nh nh i 2
ln. V qu trnh nhn chp khng tc ng ln nhng im bin nn sau mt php
nhn chp v thu nh th kch thc mi ca nh s l (n-3)/2 (n l kch thc nh ban
u).
+Bc 2: Vi mi nh lp th 2 ta dng 10 mt n nhn chp khc nhau kch thc
5x5 thc hin nhn chp v thu nh nh i 2 ln, ta c 50 nh lp th 3.
Chng 2: Nhn dng ch vit tay
34
+Bc 3: vi mi nh lp th 3 ta li dng 2 mt n nhn chp khc nhau kch thc
5x5 thc hin nhn chp ta thu c 100 nh kch thc 1x1. Ta xem y chnh l 100
c trng ca nh gc ban u.

Hnh 23: Qu trnh trch chn c trng s dng mng Neural nhn chp










Hnh 24: Minh ha qu trnh ly c trng bng mng Neuron nhn chp

2) Hun luyn mng
Qu trnh hun luyn mng Neural nhn chp l qu trnh xc nh cc h s ca cc
mt n nhn chp. lm c iu ny ngi ta xem kin trc hnh 23 nh l mt
mng Neural tuyn tnh truyn thng nhiu tng sau thm b phn lp tuyn tnh
vo sau, ri ngi ta hun luyn mng bng phng php lan truyn ngc (chi tit
xem thm [8,16]).

2.4 THC NGHIM


Input

Lp 2

Lp 3

c trng
Input 29x29 5x13x13 50x5x5
100x1x1
Convolution+
Subsampling
Convolution+
Subsampling
Convolution
Chng 2: Nhn dng ch vit tay
35
2.4.1 Kt qu

Di y xin gii thiu gii thiu kt qu thc nghim so snh hiu sut ca cc
phng php trch chn c trng ch vit nu trn. ti y s dng b d liu
MNIST thc hin hun luyn v kim tra. B d liu gm 60000 d liu hun
luyn, v 10000 d liu kim tra, mt d liu l mt bc nh kch thc 28x28 ca
mt k t ch s Arap. Ti s dng b phn lp l mng Neural RBF vi cc phng
php hun luyn l: phng php hun luyn 2-pha HDH, 1-pha+gi nghch o, 2-
pha + gi nghch o.
Cu hnh my tin hnh thc nghim: HH Windows XP Professional, CPU Intel
Core 2 Duo E6300 1.86GHz, RAM 2G.
Di y l bng kt qu thc nghim:
Bng 2: Kt qu so snh hiu sut ca cc phng php trch chn c trng

2.4.2 Nhn xt

T bng s liu trn ta thy ta thy u im vt tri ca phng php trch c trng
s dng mng Neural nhn chp(Convolution), ta thy c 3 phng php hun luyn
mng u cho kt qu nhn dng chnh xc trn 99.1%. Phng php hun luyn mng
HDH c thi gian hun luyn mng nhanh hn nhng c chnh xc nhn dng km
hn phng php hun luyn mng 2 pha + gi nghch o. Ch thi gian hun
luyn y khng tnh thi gian trch chn gi tr c trng.
Phng php trch
chn c trng
Phng php hun
luyn mng
Thi gian
hun luyn
mng(pht)
S ln nhn
dng sai trn
b d liu
kim tra
chnh
xc(%)
DCT+PCA 2 pha HDH 15.59 834 91.66%
Legendre 2 pha HDH 8.94 818 91.82%
Convolution 2 pha HDH 10.73 82 99.18%
DCT+PCA 1 pha+gi nghch o 15.56 1633 83.67%
Legendre 1 pha+gi nghch o 12.45 1713 82.87%
Convolution 1 pha+gi nghch o 11.34 96 99.14%
DCT+PCA 2 pha+gi nghch o 25.00 354 96.46%
Legendre 2 pha+gi nghch o 15.51 500 95%
Convolution 2 pha+gi nghch o 17.70 72 99.28%
Chng 3: Cc phng php ci thin hiu sut ca mng Neural RBF
36
CHNG 3
CC PHNG PHP CI THIN HIU SUT CA
MNG NEURAL RBF

Ni dung chng ny gm c:
3.1 Cc phng php ci thiu hiu sut ca mng Neural RBF
3.2 Thc nghim



3.1 CC PHNG PHP CI THIU HIU SUT CA MNG NEURAL
RBF

Di y l ni dung mt s phng php nhm ci thin hiu sut ca mng Neural
RBF trong vn nhn dng ch vit tay. Ci thin hiu sut y c ngha l lm
sao tng c cht lng nhn dng cng nh lm gim c thi gian hun luyn
mng v chi ph ci t mng.

3.1.1 Tng tp d liu hun luyn

Ta thy rng trong bi ton phn lp d liu hun luyn ng vai tr rt quan trng,
n nh hng trc tip n cht lng phn lp, tp d liu hun luyn cng ln th
cht lng phn lp cng cao. Tuy nhin to ra mt tp d liu tt th mt rt nhiu
chi ph do a phn vic to tp d liu hun luyn l phi lm th cng. V vy nu c
mt cch no lm tng s lng d liu cho tp d liu hun luyn th s lm cho
kt qu nhn dng s tt hn v chi ph cng gim ng k. Vic tng s lng tp d
liu phi m bo c cht lng v tnh t nhin vn c ca n, cng nh vic trch
chn c trng th vic tng s lng tp d liu hun luyn khng c cch tng qut
tt, phi ty vo bi ton c th. Di y ti gii thiu phng php tng tp d
liu hun luyn cho bi ton nhn dng ch vit tay bng cc bin php bin i hnh
hc.

3.1.1.1 Tng tp d liu bng cc php bin i hnh hc
Chng 3: Cc phng php ci thin hiu sut ca mng Neural RBF
37

a) Bin i Elastic
Php bin i nh ny thc ra ngi ta dng mt ma trn mt n dng vector vi
ngha l im no trn nh gc trng vi im gc ca mt vector no y th s b
dch chuyn n im ngn ca vector tng ng.

Hnh 21: Ma trn vector cho php bin i
Elastic

Hnh 22: V d v php bin i
Elastic

Hnh 22 l v d v php bin i elastic, ta thy rng t nh gc bn tri sau 2 php
bin i elastic ta c thm 2 nh khc cho d liu u vo, bng trc quan ta thy
rng d liu ny rt m bo cht lng. Ngoi php bin i Elastic ngi ta cn
dng kt hp cc php bin i affine nh: dch nh, xoay nh, hay phng to thu
nh cho c d liu mi. Tuy nhin php bin i Elastic vn thng dng hn
trong bi ton nhn dng ch vit tay do d liu n sinh ra m bo cht lng hn v
vic ci t thut ton cng nh thi gian chy ca n l rt nhanh.

3.1.2 Phng php hc tp hp

Hc tp hp l cc phng php hc m hm mc tiu c hc bng cch hun luyn
mt s b hc c lp sau kt hp chng li vi nhau. C th m hnh ton hc
bng cng thc sau:
1 2
( , ,..., )
n
h F h h h = trong h c th coi l phng php hc tp
hp,
1 2
, ,...,
n
h h h l cc b hc c lp, F l mt cch kt hp no y.
Nh vy vi phng php hc tp hp chng ta phi gii quyt 2 vn :
+Th nht: lm sao to ra c cc b hc c lp, c lp y c ngha l cc
b hc phi c cc phng php hun luyn khc nhau, hay phi c cc b d liu
hun luyn khc nhau.
+Th hai: lm sao kt hp hiu qu cc b hc ny.
Chng 3: Cc phng php ci thin hiu sut ca mng Neural RBF
38
Vn th hai thng c gii quyt bng cch kt hp tuyn tnh cc b hc li vi
nhau tc l:
1
n
i i
i
h wh
=
=

trong cc trng s w
i
c nhn gi tr ln nu b h
i
c
nh gi l tin cy hn cc b hc khc, ngc li vi cc b hc h
i
khng ng tin
cy th ta cho trng s w
i
tng ng c gi tr nh.
gii quyt vn th nht c nhiu phng php tip cn, xin gii thiu 2
phng php thng c hay dng l phng php: bagging v boosting.
+Bagging(Bootstrap aggregating)
C th tm tt phng php nh sau: Gi s ta c mt b d liu hun luyn D gm c
n d liu, phng php bagging sinh M b d liu hun luyn
i
D D c c s lng
' n n < , cc b hc h
i
c hun luyn bng b d liu D
i
tng ng. Sau cc b
hc c kt hp bng cch sau:
1
M
i i
i
h wh
=
=

vi
1
; 1..
i
w i M
M
= =
+Boosting
T tng chnh ca thut ton nh sau: Gi s ta c b d liu hun luyn D, ta nh
trng s cho cc d liu trong tp d liu hun luyn, u tin cc trng s c gn
bng nhau. Ti mi bc th t ca thut ton, ta chn b d liu
t
D D c sao cho cc
d liu trong b d liu D
t
l nhng d liu c nh trng s cao nht, sau ta
hun luyn b hc h
t
bng b d liu D
t
. Sau khi hun luyn xong ta dng b hc h
t

thm nh li tp d liu D nhng d liu no b phn lp sai th ta tng trng s ca
n ln 1, nhng d liu phn lp ng ta gim trng s ca n i 1. Lp li T ln nh
th ta c T b hc c lp, ta c th gn trng s cc b hc ny theo s lng m
n phn lp ng.

3.1.2.1 Phng php hc tp hp ci tin
Phng php ny ly t tng chnh ca phng php hc tp hp nh nu trn,
ch khc l kt qu ca u ra cui cng khng phi l t hp tuyn tnh ca cc b
hc m u ra ca cc b hc li c hun luyn mt ln na cho kt qu cao hn
(ngha l ta t hp phi tuyn tnh cc b hc vi nhau, thng qua mng Neural RBF).
Di y l kin trc tng th ca phng php hc tp hp ci tin gii quyt cho
bi nhn dng ch s vit tay:

Chng 3: Cc phng php ci thin hiu sut ca mng Neural RBF
39

Hnh 23: Kin trc ca phng php hc tp hp ci tin
y ti s dng cc phng php trch chn c trng n gin c tc tnh ton
nhanh (PCA, LEGENDRE, SUBSAMPLING-thu nh nh) to ra cc b hc khc
nhau. Sau khi hun luyn chng qua mng neural RBF ti li tng hp cc u ra ca
cc mng Neural ny v hun luyn chng li mt ln na bng mng Neural RBF.
Phng php ny tn dng ti a u im ca mng Neural RBF l thi gian hun
luyn mng rt nhanh, trong khi c mt mng c cht lng tt phng php mng
Neural nhn chp phi mt nhiu gi thm ch l nhiu ngy hun luyn mng trch
chn gi tr c trng th vi phng php ny ch mt hn mt 1h hun luyn
mng v trch c trng cho b d liu MNIST.

3.1.3 Phng php tng tc nhn dng

2 phng php nu trn ti ch mi cp n vn lm tng chnh xc nhn
dng cng nh lm lm gim thi gian hun luyn m cha cp n vn lm
gim thi gian nhn dng ca mng. y l vn then cht trong cc ng dng, v
thc ra qu trnh hun luyn thng thng c chy t hn nhiu ln so vi qu trnh
thc hin nhn dng (qu trnh a d liu u vo cho mng tnh ton). Thi gian
nhn dng = thi gian trch chn c trng + thi gian tnh ton trong mng, nh
vy vic tng hiu sut nhn dng thng qua vic tinh chnh qu trnh trch chn c
trng s lm cho thi gian trch chn c trng tng ln ng ngha vi thi gian nhn
dng cng tng. Ni tm li l vic lm tng hiu sut nhn dng v tng tc hun
luyn mng th s dn n vic lm tng thi gian nhn dng. Di y ti xin gii
thiu mt phng php kh hay va nng cao tc nhn dng nhng li khng
gim chnh xc nhn l phng php B phn nhn dng ba lp (chi tit [13]).

I
N
P
U
T

SUB
SAM
PLIN
G
DCT+
PCA
LEGE
NDRE
RBF
RBF
RBF
RBF
O
U
T
P
U
T

Chng 3: Cc phng php ci thin hiu sut ca mng Neural RBF
40
3.1.3.1 Phng php b nhn dng ba lp

C th tm tt phng php ny nh sau: ngi ta to ra ba b phn lp c c im
nh sau:
+B phn lp th nht s dng phng php trch chn c trng n gin ly t c
trng v s dng mng Neural c kin trc rt n gin thc hin nhn dng tt
nhin l b phn lp ny s c tc nhn dng rt nhanh nhng li c hiu sut nhn
dng km.
+B phn lp th 2 ta s ly nhiu c trng hn b phn lp trn ng thi kin trc
ca mng Neural nhn dng cng phc tp hn b phn lp th nht nh vy b phn
lp ny s c tc nhn dng chm hn nhng b li li c hiu sut nhn dng tt
hn b th nht.
+B phn lp th 3 ta s kt hp gia cch trch chn c trng tt nht v b phn
lp c kin trc phc tp m bo c hiu sut nhn dng cao tt nhin l c thi
gian nhn dng chm.
Sau khi xy dng c ba b phn lp nh trn qu trnh nhn dng c x l nh
sau: d liu u vo c a vo b phn lp th nht sau khi thc hin nhn dng
nu kt qu nhn dng khng b nghi ng l sai th ta chn l kt qu nhn dng
cui cng, nu khng th ta li a d liu u vo cho b phn lp th 2 x l. Sau
khi x l b phn lp th 2 nu kt qu nhn l tt th ta xc nhn kt qu nhn dng
ny cn khng ta li chuyn d liu u vo cho b phn lp cui cng. Tt nhin l
vi phng php ny ta phi c thut ton tt xc nh nhp nhng ca kt qu
nhn dng.

Hnh 24: Kin trc ca b nhn dng ba lp


INPUT
B
nhn
phn
lp
th
nht
OUTPUT
Nhp
nhng?
B
nhn
phn
lp
th 2

Nhp
nhng?

B
nhn
phn
lp
th 3

yes
yes
no no
Chng 3: Cc phng php ci thin hiu sut ca mng Neural RBF
41
Ta thy vi kin trc nh trn th phn ln qu trnh x nhn dng ri vo b phn lp
th nht (v da vo kt qu thc nghim cho d l b phn lp ti nh cng rt d
dng t c chnh xc nhn dng > 70%). Xc xut vic nhn dng x l
trn b nhn dng th 2 s thp v cng thp hn i vi b nhn dng th ba. Hin
nhin ta thy rng kin trc ny s c thi gian nhn dng nhanh hn nhiu so vi cch
ch s dng b phn lp th 3 v cng c chnh xc tt hn nhiu so vi cch ch
s dng b phn lp th nht (xem chi tit [13]).

3.2 THC NGHIM

Biu di y so snh hiu sut ca phng php hc tp hp ci tin so vi cc
phng php thng thng. y s dng b d liu MNIST cho qu trnh hun
luyn v kim tra.
90
91
92
95
94.5
95
97.75
99.3
84
86
88
90
92
94
96
98
100
1 2 3 4 5 6 7 8

8 8 8 8 18 13
63
1020
0
200
400
600
800
1000
1200
1 2 3 4 5 6 7 8

Hnh 25: Biu so snh chnh xc nhn dng v thi gian hun luyn ca cc
phng php hun luyn khc nhau
Ta thy rng phng php mng Neural nhn chp vn cho chnh xc nhn dng l
tt nht, song thi gian hun luyn mng ca n l qu ln. Phng php hc tp hp
ci tin cho kt qu nhn dng l kh cao trong khi thi gian hun luyn ca n thp
hn nhiu ln so vi phng php mang Neural nhn chp.



1,2,3,4.
Subsambling
5. PCA
6.Legedre
7.Hc tp
hp ci tin
8. Mng
Neural nhn
chp
Chng 4: Gii thiu chng trnh nhn dng ch s vit tay v tng kt

42
CHNG 4
GII THIU CHNG TRNH NHN DNG CH S
VIT TAY V TNG KT

Ni dung chng ny gm c:
4.1 Gii thiu chng trnh nhn dng ch vit tay
4.2 Tng kt v phng hng pht trin ca ti

4.1 GII THIU CHNG TRNH NHN DNG CH S VIT TAY

Di y ti xin gii thiu chng trnh ng dng nhn dng ch vit tay m ti
vit. Chng trnh l phn thc nghim ca tt c cc phn l thuyt m ti nu ra
xuyn sut trong ton b ni dng ca kha lun m ti trnh by trn.

4.1.1 Chng trnh nhn dng ch vit tay

4.1.1.1 Gii thiu chng trnh

1) Tnh nng ca chng trnh
Chng trnh c kh nng nhn dng ch s vit tay do ngi dng nhp file nh ca
k t no y thc hin nhn dng. nhn dng chng trnh cho php ngi
dng ty chn phng php trch, v ty chn phng php hun luyn mng RBF.
Sau khi thc hin nhn dng th chng trnh tr v cho ngi dng biu th hin
xc sut bc nh ngi dng nhp vo l s no.

2) Mi trng v cng c pht trin
Chng trnh c vit hon ton bng ngn ng C++, giao din s dng IDE
Microsoft Visio Studio 2008 thit k, ngoi ra chng trnh cn s dng mt s th
vin ca Matlab tnh ton ma trn.

3) Hng dn s dng chng trnh
chng trnh chy c tt, ta phi ci t th vin Matlab vo my v thm th
mc bin ca Matlab vo bin mi trng PATH ca h iu hnh. Ngi dng c th
Chng 4: Gii thiu chng trnh nhn dng ch s vit tay v tng kt

43
nhp nh vi cc nh dng nh *.jpg, *.bmp, *. gif, *.png vo chng trnh sau
thc hin nhn dng ch s, sau khi chng trnh thc hin xong n s a ra biu
nh gi xc sut bc nh l s no da trn kt qu nhn dng ca mng Neural.
Ngoi ra ngi dng c th chn ch hun luyn mng Neural vi cc phng
php trch chn c v phng php hun luyn khc nhau. Sau y l giao din chnh
ca chng trnh:
Hnh 26: Giao din chnh ca chng
trnh
Hnh27: Bng thng bo kt qu nhn dng


4.2 TNG KT V PHNG HNG PHT TRIN CA TI

4.2.1 Tng kt

n y ti hon thnh kha lun tt nghip vi ti Mng Neural RBF v ng
dng nhn dng ch vit tay. Vi mc tiu kt hp gia nghin cu l thuyt mng
Neural RBF i i vi vic tm hiu bi ton nhn dng ch vit tay thc hin to
chng trnh ng dng nhn dng ch vit tay.

4.2.1.1 Nhng cng vic lm c

Chng 4: Gii thiu chng trnh nhn dng ch s vit tay v tng kt

44
+Tm hiu c kin trc mng Neural RBF, hiu c cc phng php hun luyn
mng RBF, nh tin hnh ci t thnh cng c mng Neural RBF vi cc
thut ton hun luyn khc nhau (xem chng 1).
+Tm hiu c bi ton nhn dng ch vit tay, hiu c cc bc x l bi ton
nhn dng ch vit tay. Tm hiu v tin hnh ci t c 3 phng php trch chn
c trng ch vit tay l: phng php kt hp bin i DCT v thut ton PCA,
phng php s dng momen Legendre, phng php s dng mng Neural nhn
chp (xem chng 3).
+Tm hiu cc phng php tng hiu sut cho mng Neural RBF p dng cho bi
ton nhn dng ch vit tay. xut phng php hc tp hp ci tin cho hiu sut
nhn dng 98% vi b d liu MNIST v c thi gian hun luyn mng rt nhanh.
+Xy dng c phn mm nhn dng ch s vit tay.

4.2.2.2 Hng pht trin ca ti

Do thi gian nghin cu c hn nn ti kha lun cha th i st vo cc vn
c a ra. Nu c pht trin thm ti s nghin cu k hn v mng neural RBF,
nghin cu cc cch hun luyn khc cng nh cc kin trc mng RBF ci tin lm
cho vic hun luyn nhanh hn v hiu sut nhn dng tt hn. Bi ton nhn dng
ch vit tay mc d c nghin cu t lu nhng n ngy nay vn c tip tc
pht trin, c bit l cc bi ton nhn dng vn bn ting vit, nhn dng form... Do
cha th u t thi gian nhiu cho vic gii quyt bi ton nhn dng ch vit tay,
nn chc chn ti cn mc thiu st rt nhiu phn ny. Nu pht trin tip th
phn nhn dng ch vit tay ti s nghin cu k hn cc phng php trch chn c
trng ch vit cng nh cc k thut lm tng tc nhn dng ca mng (trong kha
lun ti cha dnh thi gian nhiu gii quyt vn tng tc nhn dng ca
mng). phn ng dng ti cng ch mi dng li mi dng li vic vit phn mm
nhn dng ch s vit tay, ng dng ny khng thc s c ngha thc tin. Nu tip
tc nghin cu pht trin thm th ti s pht trin ng dng hng n vic nhn dng
k t ting vit, cng nh pht trin cc ng dng thi gian thc. Tm li hng pht
trin ca ti cho ti kha lun ny l kt hp gia vic nghin cu st hn phn l
thuyt ca mng Neural RBF v vic a ra cch gii quyt hp l nht cho bi ton
nhn dng ch vit tay ting vit s dng mng Neural RBF.




TI LIU THAM KHO
[1] Hong Xun Hun, Gio trnh cc phng php s, NXB i hc quc gia
H Ni, 2004
[2] Hoang Xuan Huan, Dang Thi Thu Hien and Huu Tue Huynh, A Novel
Efficient Algorithm for Training Interpolation Radial Basis Function
Networks, Signal Processing 87 ,2708 2717, 2007.
[4] F. Schwenker, H.A. Kestler, Gu nther Palm, Three learning phases for
radial-basis-function networks, Neural Networks 14 (2001) 439458
[5] C.G. Looney, Pattern Recognition Using Neural Network, Theory and
algorithm for engineers and scientist, Oxford University press, 1997.
[6] N.B. Karayiannis, Member, IEEE, and Glenn Weiqun Mi. Growing Radial
Basis Neural Networks: MergingSupervised and Unsupervised Learning with
Network Growth Techniques. IEEE transactions on neural networks, vol. 8,
no. 6, November 1997
[7] M. Vatkin, M. Selinger the system of handwritten characters recognition on
the basis of legendre moments and neural network, The International
Workshop on Discrete-Event System Design, DESDes01, June 2729, 2001;
Przytok near Zielona Gora, Poland
[8] P.Y. Simard, Dave Steinkraus, John C. Platt, Best Practices for Convolutional
Neural Networks Applied to Visual Document Analysis, Microsoft Research,
One Microsoft Way, Redmond WA 98052
[9] S.Theodoridis, K.Koutroumbas Pattern recognition Second edition, 2ed.,
Elsevier, 2003
[10] J. Shlens, A Tutorial on Principal Component Analysis, April 22, 2009
[11] T.M. Mitchell, Machine learning, McGraw-Hill, 1997
[12] D.S. Broomhead and D. Lowe. Multivariable functional interpolation and
adaptive networks. Complex Systems, vol. 2, 321-355, 1988.
[13] D. Gorgevik, D. Cakmakov. An Efficient Three-Stage Classifier for
Handwritten Digit Recognition. Proceedings of the 17th International
Conference on Pattern Recognition (ICPR04).
[14] R. H. Bartels, J. C. Beatty and B. A. Barsky, An introduction to Splines for use
in computer graphics & geometrics modeling, Morgan Kaufmann Publishers,
1987
[15] K.M. Hosny, New Set of Rotationally Legendre Moment Invariants,
International Journal of Electrical and Electronics Engineering 2:3 2007
[16] D. Bouchain, Character Recognition Using Convolutional Neural Networks,


2006
[17] R. Esposito, Ensemble Learning
[18] Wikipedia, k-means clustering, http://en.wikipedia.org/wiki/K-
means_clustering
[19] Wikipedia, Principal component analysis,
http://en.wikipedia.org/wiki/Principal_components_analysis
[20] Wikipedia, Discrete cosine transform,
http://en.wikipedia.org/wiki/Discrete_cosine_transform
[21] Wikipedia, Image moments,
http://en.wikipedia.org/wiki/Image_moments
[22] Wikipedia, Eigenvalue, eigenvector and eigenspace,
http://en.wikipedia.org/wiki/Eigenvalue,_eigenvector_and_eigenspace
[23] Wikipedia,Convolution
http://en.wikipedia.org/wiki/Convolution