H C S Tri Thc

HC MY (MACHINE LEARNING)
Gii Thiu
Vn ngn ng v hc
kh nng hc chnh l mt trong nhng thnh t
quan trng ca hnh vi thng minh.
Cc h chuyn gia c pht trin nhng vn cn
gp nhiu kh khn
Mt gii php l cc chng trnh t hc ly cch
gii quyt vn t kinh nghim, t s ging
nhau, t cc v d hay t nhng ch dn, li
khuyn,...

nh Ngha Hc
Theo Herbert Simon: Hc c nh ngha nh l bt
c s thay i no trong mt h thng cho php n tin
hnh tt hn trong ln th hai khi lp li cng mt nhim
v hoc vi mt nhim v khc rt ra t cng mt qun
th cc nhim v
C ba tip cn hc: tip cn k hiu (symbol-based
learning), tip cn mng neuron hay kt ni (neural or
connectionist networks) v tip cn ni tri (emergent)
hay di truyn v tin ha (genetic and evolutionary
learning).
Tip Cn K Hiu: Gii Thut Quy
Np Cy Quyt nh ID3
Gii Thut Quy Np Cy Quyt nh
I. Gii Thiu
II. Gii Thut ID3 xy dng cy quyt nh t trn
xung
III. Thuc Tnh Dng Phn Loi Tt Nht
IV. Tm Kim Khng Gian Gi Thuyt Trong ID3
V. nh Gi Hiu Sut
VI. Chuyn Cy V Cc Lut
VII.Khi No S Dng ID3
Gii Thiu
Gii thut quy np cy (ID3) l 1 gii thut hc n gin c
ng dng trong nhiu lnh vc.
ID3 biu din cc khi nim (concept) dng cy quyt nh
(decision tree)
Gii thut c u vo v u ra l:

Gii Thiu
Gii Thiu
Thuc tnh: quang cnh, nhit , m v gi
Thuc tnh phn loi chi Tennis (c, khng)
Thuc tnh Chi tennis cn c gi l thuc tnh ch
(target attribute).
Thuc tnh quang cnh c ba gi tr (m u, ma, nng)
Nhit c ba gi tr (nng, mt, m p)
m c hai gi tr (cao, TB)
Gi c hai gi tr (mnh, nh).
Cc gi tr ny chnh l k hiu (symbol)
Gii Thiu
Gii Thiu
Sau khi quy np c cy quyt nh th cy
ny s c s dng phn loi cho cc v d
trong tng lai.
ng vi mt tp d liu rn luyn s c nhiu
cy quyt nh c th phn loi ng tt c cc
v d trong tp d liu rn luyn.
Vy lm sao hc c cy quyt nh c th
phn loi ng tt c cc v d trong tp rn
luyn?
Gii thut ID3 xy dng cy quyt
nh t trnxung
Gii thut ID3 xy dng cy quyt
nh t trnxung
Thuc tnh no l thuc tnh dng
phn loi tt nht?
Shannon (1948) cung cp khi nim entropy o
tnh thun nht (hay ngc li l pha trn) ca
mt tp hp. Mt tp hp l thun nht nu nh tt
c cc phn t ca tp hp u thuc cng mt loi,
v khi ta ni tp hp ny c pha trn l thp
nht.

Khi tp v d l thun nht th c th ni: ta bi t
chc chn v gi tr phn loi ca mt v d thuc
tp ny, hay ta c lng thng tin v tp l cao
nht.
Entropy o tnh thun nht ca tp v
d
Khi nim entropy ca mt tp S c nh ngha trong L
thuyt thng tin l s lng mong i cc bt cn thit m
ha thng tin v lp ca mt thnh vin rt ra mt cch ngu
nhin t tp S.
Theo l thuyt thng tin, m c di ti u l m gn -log2p
bits cho thng ip c xc sut l p.
Entropy o tnh thun nht ca tp v
d
Entropy c gi tr nm trong khong [0..1].
Entropy(S) = 0 tp v d S ch ton v d thuc
cng mt loi, hay S l thun nht.
Entropy(S) = 1 tp v d S c cc v d thuc cc
loi khc nhau vi pha trn l cao nht.
0 < Entropy(S) < 1 tp v d S c s lng v d
thuc cc loi khc nhau l khng bng nhau.


Entropy o tnh thun nht ca tp v
d
Lng thng tin thu c o mc
gim entropy mong i
Lng thng tin thu c l php o hiu sut phn loi cc v
d ca mt thuc tnh, n gin l lng gim entropy mong
i gy ra bi vic phn chia cc v d theo thuc tnh ny.
Tm kim khng gian gi thuyt
trong ID3
ID3 thc hin mt php tm kim t n gin n
phc tp, theo gii thut leo-ni (hill climbing)

Tm kim khng gian gi thuyt
trong ID3
Khng gian gi thuyt cc cy quyt nh ca ID3 l mt
khng gian y cc cy quyt nh trn cc thuc tnh
cho trong tp rn luyn.
Trong khi tm kim, ID3 ch duy tr mt gi thuyt hin ti
Gii thut thun ID3 khng c kh nng quay lui trong khi tm
kim.

Tm ki m khng gian gi thuyt
trong ID3
V ID3 s dng tt c cc v d mi bc a ra cc
quyt inh da trn thng k, nn kt qu tm kim ca ID3 rt
t b nh hng bi mt vi d liu sai (hay d liu nhiu).
Trong qu trnh tm kim, gii thut ID3 c xu hng chn cy
quyt nh ngn hn l nhng cy quyt nh di. y l tnh
cht thin lch quy np ca ID3.
nh gi hiu sut ca cy quyt
nh:
c nh gi l tt nu nh cy ny c kh nng phn loi
ng c cc trng hp hay v d s gp trong tng lai,
hay c th hn l c kh nng phn loi ng cc v d khng
nm trong tp d liu rn luyn.
nh gi hiu sut ca mt cy quyt nh ngi ta thng
s dng mt tp v d tch ri v tp d liu tp kim tra
(validation set)
Chuyn cy v cc lut
Khi no nn s dng ID3
Gii thut ny thuc tip cn gii quyt vn
da trn k hiu (symbol based approach).
Tp d liu rn luyn y bao gm cc v d
c m t bng cc cp Thuc tnh gi tr v
mi v d u c mt thuc tnh phn loi.
ID3 s dng cc v d rn luyn dng xc sut
nn n c u im l t b nh hng bi mt vi
d liu nhiu.