PCA - Principal Component Analysis - MlcvGru

28/10/2017 PCA Principal Component Analysis | mlcvGru
mlcvGru
Random thoughts on Machine Learning, Deep Learning

and (sometimes) Computer Vision
PCA Principal Component Analysis
Phn tch thnh phn chnh (Principal Component Analysis PCA) l mt trong nhng phng php
phn tch d liu nhiu bin n gin nht. Phn 1 ca bi ny s ni v ng lc v tng chnh
ca PCA, sau trnh by tng bc trong thut ton PCA. Nh thng l, hiu thut ton th
ci t l cch tt nht, do mt on m MATLAB minh ha tng bc trong PCA cng c
trnh by. Phn cui cng l chi tit cc khai trin Ton hc trong PCA. Do mc tiu ng dng nn
cc phng trnh c th c dnh n phn ny, ngi c nu tht s quan tm nguyn l ca
PCA, ti sao PCA li s dng nhng cng thc nh th v.v th c th tham kho phn ny bit
thm chi tit.
1. Gii thiu
Nh ni trong bi trc
(h ps://phvuresearch.wordpress.com/2011/09/21/multivariate_data_analysis_intro/), trong thng k,
thng thng cn phi nghin cu d liu trc khi xy dng cc m hnh suy din da trn d
liu . Tuy nhin i khi d liu c s chiu ln, khng th visualize (khng bit dch l g) trong
khng gian 2 hay 3 chiu, do cn phi tm cch a d liu v khng gian c s chiu nh hn.
PCA l mt trong nhng phng php nh th, nhng hn th, PCA cn ging nh mt Swiss knife
(h p://en.wikipedia.org/wiki/Swiss_Army_knife) vi nhiu c tnh tt:
Gip gim s chiu ca d liu,

Thay v gi li cc trc ta ca khng gian c, PCA xy dng mt khng gian mi t chiu
hn, nhng li c kh nng biu din d liu tt tng ng khng gian c, ngha l m bo
bin thin (variability (h p://en.wikipedia.org/wiki/Statistical_variability)) ca d liu trn mi
chiu mi.
Cc trc ta trong khng gian mi l t hp tuyn tnh ca khng gian c, do v mt ng
ngha, PCA xy dng feature mi da trn cc feature quan st c. im hay l nhng
feature ny vn biu din tt d liu ban u.
Trong khng gian mi, cc lin kt tim n ca d liu c th c khm ph, m nu t trong
khng gian c th kh pht hin hn, hoc nhng lin kt nh th khng th hin r.
https://phvu.net/2011/10/05/pca-principal-component-analysis/ 1/32
Ni mt cch ngn gn, mc tiu ca PCA l tm mt khng gian mi (vi s chiu nh hn khng
gian c). Cc trc ta trong khng gian mi c xy dng sao cho trn mi trc, bin thin
ca d liu trn l ln nht c th. Ting Vit th di dng, nhng ting Anh th mc tiu ny gi
l maximize the variability. Ba ch ny gi gn tng chnh ca PCA.
(h ps://phvuresearch.les.wordpress.com/2011/10/untitled.png)
Minh ha PCA: php chiu ln cc trc ta khc nhau c th cho cch nhn rt khc nhau v cng mt d
liu.
Mt v d kinh in l hnh nh v con lc . Cng l mt con lc nhng nu nhn t bn hng

th ta c c y thng tin nht, trong khi nhn t pha trc th tht kh ni n l lc .
Mt v d thuyt phc hn c minh ha trong hnh sau
(h ps://phvuresearch.les.wordpress.com/2011/10/untitled1.png)
Minh ha PCA: tm cc trc ta mi sao cho d liu c bin thin cao nht
Gi s tp d liu ban u (tp im mu xanh) c quan st trong khng gian 3 chiu (trc mu
en) nh hnh bn tri. R rng 3 trc ny khng biu din c tt nht mc bin thin ca d
liu. PCA do s tm h trc ta mi (l h trc mu trong hnh bn tri). Sau khi tm c
khng gian mi, d liu s c chuyn sang khng gian ny c biu din nh trong hnh bn
phi. R rng hnh bn phi ch cn 2 trc ta nhng biu din tt hn bin thin ca d liu
so vi h trc 3 chiu ban u.
Mt im rt p na ca PCA l cc trc ta trong khng gian mi lun m bo trc giao i

mt vi nhau, mc d trong khng gian ban u, cc trc c th khng trc giao.
Di dng nh vy l , ta s trnh by tng bc thut ton PCA trong phn ti. Chi tit v tng
v khai trin ton hc c dnh li trnh by cui bi.
2. Thut ton PCA
Cho ma trn . Cc bc ca PCA ln lt nh sau:
1. Tin x l
D liu ban u c th c gi tr thay i bt thng. V d trn feature 1 (ct 1 ca ) gi tr thay

i trong khong (0, 1), trn feature 2 li bin thin trong on (-100, 100). R rng cn phi c mt
bc tin x l chun ha gi tr trn cc ct ca ma trn X. C 2 cch tin x l thng c dng
cho PCA l Centered PCA v Normed PCA.
Centered PCA mang tt c cc feature (cc ct ca X) v cng mt gc ta :
, (1a)
Trong n l s dng ca X, l mean ca ct th j ca X, c tnh nh trn.
Normed PCA mang tt c cc feature v cng mt gc ta , ng thi chun ha v cng mt

qung standard-deviation bng 1:
. (1b)
Trong l lch chun (standard deviation) ca ct th j trong X.
Thng thng Normed PCA hay c dng. Sau bc tin x l, ma trn s l u vo cho bc
tip theo.
2. Xy dng khng gian mi
Tnh ma trn hip phng sai (covariance) ca cc feature trong :
(2)
Do l tch ca ma trn vi chuyn v ca n nn l ma trn positive semidenite kch

thc . Hn na c p tr ring .
Tip theo, PCA tm tr ring v vector ring

(h ps://phvuresearch.wordpress.com/2011/03/19/tr%e1%bb%8b-rieng-va-vector-rieng/) tng ng
ca , sp xp theo th t gim dn ca tr ring. Gi s p tr ring ca V l
, (3)
v p vector ring tng ng l
. (4)
Khi cc trc ca khng gian mi chnh l cc vector ring trn, ng nhin cc vector ring
hon ton c lp tuyn tnh (ngha l trc giao i mt).
C th ni trong PCA, tr ring v vector ring c v tr rt p, tha mn tt c cc yu cu ca PCA.

Bn thn ti khi c n phn ny cng thy bt ng v li gii cho PCA khng g khc li hon ton
trn vn trong tr ring v vector ring. Tuy nhin ti thi im ny, ta s chp nhn nh vy. Phn
c s Ton hc cui bi s gii thch ti sao tr ring v vector ring li xut hin (c phn bt ng)
trong PCA nh vy.
3. Chuyn d liu t khng gian ban u vo khng gian mi
Thng thng khng gian mi khng c xy dng bng tt c p vector ring trong (4), m thng
thng ch t k vector ring u tin, vi k < p. Ti sao l cc vector u tin, v chn k bao nhiu th
tt, ta s bn trong phn cui.
Nh vy gi
Khi ta cc im trong h ta mi l
(5)
Xong. Ta kt thc gii thut PCA, khng th n gin hn.
Trong phn ny ta ch thc hin phn tch cc sample trong khng gian to bi cc feature. Ta cn c
th thc hin kho st cc feature trong khng gian to bi cc sample. ng nhin c th chuyn v
ma trn X ri thc hin tng t, nhng l cch v bin, cy ba. Thc s khng cn phi nh
vy, m nh vo nhng c tnhp ca tr ring v vector ring, ta c th tnh ngay khng gian mi
ca cc sample ( biu din cc feature) t cc vector ring trong (4). Tr ring cn p n ni c
dng trong cc tiu chun chn k. Ta s tr li vi 2 vn ny trong phn cui ca bi.
Ngoi ra nh gi cht lng ca khng gian mi to bi PCA, ta dng 2 o l contribution v

squared cosine. Ta cng dnh 2 phn ny cho phn cui bi.
3. M ngun MATLAB
Sau y l on m MATLAB minh ha cc bc ca thut ton PCA, thc hin phn tch cc
observation trong khng gian cc feature, sau cng tnh cc o contribution v squared cosine.
Download here (h ps://www.box.com/s/l7jgipl8eyqdwzlxnr2o).
1 function [ output_args ] = mypca( X )

2 m = mean(X);
3 st = std(X, 1);
4
5 sz = size(X);
6 % sz(1): number of observations
7 % sz(2): number of features
8
9 % normed PCA
10 Xhat = zeros(sz);
11 for i=1:sz(2)
12 Xhat(:,i) = (X(:, i) - m(i))/(sqrt(sz(1))*st(i));
13 end
14 display('Normed data:');
15 display(Xhat);
16
17 % correlation matrix
18 V = Xhat' * Xhat;
19 display('Correlation between features (columns) of normed data:');
20 display(V);
21
22 %eigenvalues
23 [eivec, eival] = eig(V);
24
25 % display the eigenvalues accumulation and %
26 % sz(2) == number of eigenvalues of V
27 eigenInfo = zeros(sz(2), 3);
28 sum_eival = sum(diag(eival));
29 acc_eival = 0;
30 for i=sz(2):-1:1
31 acc_eival = acc_eival + eival(i, i);
32
33 % eigenvalues, accumulate, %
34 eigenInfo(sz(2)-i+1, : ) = [eival(i, i), acc_eival, acc_eival/sum_eival
35 end
36 display('Eigenvalues Accumulate Acc ');
37 display(eigenInfo);
38
39 %eigenvectors
40 eivec2 = zeros(size(eivec));
41 for i=1:size(eivec, 2)
42 eivec2(:, i) = eivec(:, size(eivec, 2) - i + 1);
43 end
44 eivec = eivec2;
45 display('Full eigenvectors:');
46 display(eivec);
47
48 % Ask for number of new coordinates
49 respone = input('Number of new coordinates: ');
50 dim = respone(1);
51
52 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
53
54 %The new observations
55 F = Xhat*eivec(:, 1:dim);
56 display('Transformed data (New coordinates of observations):');
57 display(F);
58
59 % Contribution of observations
60 C = F.^2;
61 for i=1:size(C, 2)
62 C(:, i) = C(:, i)/eival(sz(2) - i + 1, sz(2) - i + 1);
63 end
64 display('Contribution of observations:');
65 display(C);
66
67 % Squared cosine of observations
68 SC = F.^2;
69 for i=1:size(SC, 1)
70 SC(i, : ) = SC(i, : )/sum(SC(i,: ), 2);

71 end
72 display('Squared cosine of observations:');
73 display(SC);
74 end
4. C s Ton hc ca PCA
Xem ti y (h ps://phvuresearch.wordpress.com/2011/11/15/pca-intuition-maths/).
5. Kt lun
Mt s hn ch ca PCA:
Ch lm vic vi d liu numeric,

Nhy cm vi cc im outlier/extreme,
Khng ph hp vi cc m hnh phi tuyn, do PCA hon ton da trn cc bin i tuyn tnh.
Advertisements
Posted in Linear Algebra, Machine Learning, Mathematics, Statistics and tagged MDA, multivariate,
pca, vietnamese on October 5, 2011 by Vu Pham. 77 Comments
77 comments
1. Thong says:
October 9, 2011 at 4:27 am
Ngoai PCA thi con co ICA cung la 1 pp thong dung de xu ly data.
i
Rate This
REPLY
2. Hc says:
November 5, 2011 at 6:05 pm
Rt mong tc gi c th trnh by tip phn C s Ton hc ca PCA.
i
Rate This
REPLY
3. Pingback: PCA Intuition and Maths | IPhVu::iResearch
4. Pham Dang Khoa says:
December 21, 2011 at 9:58 am
Cho anh V, cho em hi l PCA v Karhunen Loeve khc nhau nh th no
i
Rate This
REPLY
5. phvu says:
December 21, 2011 at 8:34 pm
Hi Khoa,
Trong trng hp ri rc v p dng ln sample ca bin ngu nhin, th PCA v Kerhunen-
Loeve Transforms l mt.
i
Rate This
REPLY
Thanks for your reply
Theo em hiu th Karhunen l ma trn A dng bin Rx (covariance ca X) thnh Ry (l ma trn

cho, c cc phn t trn ng cho l cc gi tr ring ca Rx)
Nhng khi p dng PCA, em thy t Rx l mnh c th suy ra gi tr ring, vector ring lun ri ?
Ti sao li cn ma trn cho ?
i
Rate This
REPLY
7. phvu says:
Hi em,
Em tham kho pp KLT u th? Theo anh c th trong trng hp ri rc, KLT cng ch dng
ma trn to bi cc vector ring thc hin bin i cc vector ban u, ging ht nh PCA.
i
Rate This
REPLY
Em c trong bi ging ca thy L.Q.Ngc
Vy mnh c th hiu PCA l tng c s, cn Karhunen Loeve l ci t c th ca bin i
vector khng anh ?
Khi dng PCA chuyn X thnh Y, th cc ti liu thng chng minh covariance matrix ca Y
l ma trn cho, c cc gi tr trn ng cho l cc gi tr ring ca covariance matrix ca X. Anh
c th ni r hn ch ny c ko ?
Em cm n.
i
Rate This
REPLY
9. phvu says:
1. Anh khng ngh th. Anh thy Karhunen Loeve tng qut hn PCA.
2. Anh khng thy l do ti sao covariance ca Y li phi l ma trn ng cho. Em c th ni r
ni dung chng minh th no ko?
i
Rate This
REPLY
10. Nam says:
Thank v bi vit
ps: visualize c th dch l quan st (mt cch) trc quan (?), tc l quan st bng mt thng.
i
Rate This
REPLY
1. DDC says:
February 8, 2014 at 4:48 pm
ng ngha ting Anh-Vit visualize = hnh dung cc bc
i
Rate This
REPLY
11. Huynh Suong says:
February 24, 2012 at 11:12 pm
Doc xong da hieu duoc rat nhieu thu thac mac, cam on anh nhieu!
i
Rate This
REPLY
12. Ha Nguyen says:
April 24, 2012 at 9:44 am
chao anh
Hien nay e dang lam de tai ve xu ly anh nhan dang mat nguoi su dung thuat toan pca
nhung e co mot van de ko hieu ve khai niem truc giao va truc giao de lam gi
e cam on anh nhieu
Rat hy vong nhan duoc su hoi am cua anh
i
Rate This
REPLY
13. Ha Nguyen says:
April 24, 2012 at 9:48 am
Chao anh
Em hien dang lam de tai lien quan den xu ly anh nhan dang mat nguoi ( face recognition)
Qua tim hieu em co gap mot van de ma khong giai thich duoc
Do la truc giao Vay truc giao la gi- y nghia cua no de lam gi
Em cam on anh nhieu
Chuc anh suc khoe va that nhieu niem vui
PS: neu co tai lieu lien quan den PCA anh gui qua mail giup e nhe
vie.hanguyen89@gmail.com
i
Rate This
REPLY
1. PHAM Hoai Vu says:
April 24, 2012 at 7:01 pm
Hi H,
Trc giao hiu nm na ngha l vung gc. Trong i s tuyn tnh, ta ni 2 vector n chiu
l trc giao khi v ch khi tch v hng ca chng bng 0.
Thn
i
Rate This
REPLY
14. Ha Nguyen says:
April 29, 2012 at 11:38 pm
Anh Vu ah em co mot thac mac mong anh giup do nhe
Theo mot tai lieu em doc duoc ve thuat toan PCA gom co 5 buoc co ban:
1 lay du lieu
2 Tru di tri trung binh ( mean) cua moi chieu ( dimension)
3- Tinh toan ma tran hiep psai
4 Thiet lap vecto dac trung ( feature vector)
5 chuyen du lieu ban dau ve khong gian moi
Buoc thu 2 em ko hieu lam

Em nghi no la buoc tien xu ly du lieu phai khong ah
mong anh giup em giai thich nhe
em cam on anh nhieu!
i
Rate This
REPLY
April 30, 2012 at 12:39 am
Hi Ha,
ng vy, trong bc ngi ta mun chun ha d liu sao cho gi tr trung bnh (mean)
ca n bng 0.
V d X l ma trn sau:
tc l ta c 3 mu d liu trong khng gian 2 chiu. Nu thc hin centered PCA (l cch em
m t trn), th vi mi ct, ta tr i gi tr trung bnh ca ct . C th trong ma trn trn
th gi tr trung bnh ca ct 1 l , v ca ct 2 l . Nh vy sau khi bin i th ma trn mi l:
Nh vy sau khi chuyn i, ta thy gi tr trung bnh ca mi ct trong ma trn mi u

bng 0, y chnh l ngha ca centered PCA.
Nu ta chia thm cho phng sai ca mi ct, th cc ct trong ma trn cui cng s c phng
sai bng 1, v cch lm gi l Normed PCA.
Thc ra y khng phi l 1 bc bt buc ca PCA, tuy nhin v PCA hot ng da trn tr
ring ca ma trn covariance, m ma trn covariance kh nhy vi variance ca cc bin,
nn thc t cho thy normed v centered PCA cho kt qu tt hn PCA trn ma trn gc ban
u.
i
Rate This
REPLY
15. Ha Nguyen says:
April 29, 2012 at 11:48 pm
PS: Em thuc su muon giai quyet bai toan nhan dang mat nguoi theo huong xay dung di tu: Hinh
dung van de cot loi (hieu duoc muc tieu cuoi cung) Xay dung tren co so toan hoc ( tai sao lai su
dung nhung cong thuc va ham toan hoc nhu vay) va cai dat thuat toan tren matlab
Neu A co tai lieu lien quan thi gui giup em nhe anh ( neu duoc thi em mong nhan duoc tai lieu
thong qua gmail) hoac neu anh ban ^^ thi co the giup e ten cac tai lieu day du ve no cung duoc )
Em cam on anh!
PS: Chuc anh ngay le vui ve !
And: Em van phai lam viec de hoan thanh nal project theo dun tien do
( Em rat an tuong voi cach giai quyet bai toan cua anh Dieu ma it giao vien hien nay tiep can va
huong dan students xay dung phuong huong giai quyet van de)
And: Dac biet la hinh con lac da
i
Rate This
REPLY
April 30, 2012 at 12:40 am
Nhn dng mt ngi cng c kh nhiu phng php, khng bit em ang lm theo phng
php no?
i
Rate This
REPLY
16. Ha Nguyen says:
April 30, 2012 at 12:49 am
Em cam on anh nhieu! (^-^) ( Em ko nghi la duoc reply nhanh nhu vay)
Hien em dang lam nal Project su dung phuong phap PCA
Thuc su la em chua qua hieu mo hinh toan hoc cua PP nen Em muon tim hieu no that ky truoc khi
xay dung thuat toan cung nhu can xay dung nhung chu y (uu va nhuoc diem cua no) de khac
phuc
neu con thoi gian em se ket hop voi mot so pp khac de cai thien do chinh xac cua chuong trinh
anh ah
Hien gio e su dung matlab de viet chuong trinh ( GUI cho Thiet lap Camera va GUi cho giao tiep
phan cung em da hoan thanh)
mot so code mau tu tren matlab nguoi ta thuc hien cung cuc ky tuyet voi nhung em van muon tu
minh viet code duoi goc do am hieu thuat toan anh ah
^^ PS: Nhan duoc hoi am cua anh Em rat vui ( quen het met moi lun) ^0^
i
Rate This
REPLY
17. Van Hoang says:
June 20, 2012 at 11:46 am
Cho anh V,
Trong hc my, v d dng SVM luyn, cho tp training set.
Mc tiu ca mnh l dng PCA gim s chiu ca cc quan st (gim s cc feature) t khng
gian trng thi ban u thnh khng gian mi. Ly v d, tp cc samples, mi sample c 10.000
feature (hiu nm na l 10.000 chiu).
1. Trong trng hp ny cn tm tr ring ca X^TX, XX^T.
2. bc phn loi, lm sao bin i tp d liu cn phn loi trong khng gian c sang khng
gian c xy dng bi CPA trn.
Xin cm n anh!
i
Rate This
REPLY
June 20, 2012 at 4:09 pm
Hi em,
1. Anh khng hiu cu ny. Thng thng em c th chy PCA bng cc ci t c sn
(OpenCV hay MATLAB) tm ma trn U. Sau th chiu ton b d liu ban u ln khng
gian ca U. Vic ny hnh nh cng c lm sn lun ri.
2. Trong trng hp PCA l input cho SVM (tng qut l cc thut ton supervised), th em
nn lm nh sau:
Chy PCA trn training set tm ma trn U ca cc vector ring. Lu li ma trn U ny.
Chiu training set vo khng gian mi: X = X*U. Dng X hun luyn SVM.
Trong qu trnh test, v cc vector trong tp test trong khng gian ban u nn cn phi
chiu vo khng gian ca U, tc l tnh Y = Y*U, trong Y l tp test ban u, Y l tp test
trong khng gian PCA.
Dng Y test m hnh hun luyn.
Thn.
i
Rate This
REPLY
1. Van Hoang says:
June 20, 2012 at 4:57 pm
Cm n anh V.
Anh gii thch ng mun hi ri .
Chc anh khe!
i
Rate This
18. dduuyy says:

November 6, 2012 at 9:51 am
Anh Vu oi! A co tai lieu (tieng viet) v thut ton 2D-PCA khng?
E dang nghien cuu ma khong hieu gie het A.
0
i
Rate This
REPLY
Hi em, anh khng c ti liu ting Vit no v 2DPCA c
i
Rate This
REPLY
19. dduuyy says:
Anh vu oi! A co biet gi thuat toan 2D-PCA khong?
A c gi g cho e vi.
Cm n Anh tht nhiu
i
Rate This
REPLY
Hy dduuyy,
Cch lm ca 2DPCA kh n gin. Gi s cho tp nh hun luyn th 2DPCA tm ma trn
sau:
trong l trung bnh cng ca tt c cc nh.
Sau 2DPCA tm tr ring v vector ring ca ma trn G, cc vector ring ng vi cc tr

ring ln nht s l c s cho khng gian mi. Trong PCA th ta dng ma trn covariance, cn
2DPCA th dng ma trn G. Cn li hon ton tng t.
Khng hiu bn khng r ch no.
i
Rate This
REPLY
20. dduuyy says:
Cam on ban tra loi minh!
Cai minh khong hieu la giua pca va 2dpca, thi cai nao tot hon!
va muon xin ban thuat toan cua 2dpca.
Minh lap trinh hoai ma khong duoc, chac tai thuat toan minh bi sai
i
Rate This
REPLY
Theo mnh hiu th 2DPCA c chi ph thp hn PCA truyn thng v ma trn dng tnh tr
ring ca 2DPCA nh hn nhiu so vi ma trn trong PCA. V hiu qu th hnh nh l tng
ng, mnh khng chc lm, bn c th xem thm trong paper ca 2DPCA.
Bn c th download toolbox ny, trong c cha cc hm lin quan n 2DPCA:
h p://www.mathworks.com/matlabcentral/leexchange/12333-statistical-learning-toolbox
i
Rate This
REPLY
21. dduuyy says:
Cm n A. V nhiu lm!
i
Rate This
REPLY
22. Ch Trn Minh says:
January 26, 2013 at 4:20 pm
Mt s hn ch ca PCA:
Ch lm vic vi d liu numeric,

Nhy cm vi cc im outlier/extreme,
Khng ph hp vi cc m hnh phi tuyn, do PCA hon ton da trn cc bin i tuyn tnh.
Nh anh Vu gii thch dm em 3 trn vi , em cn m h qu . Cm n anh
i
Rate This
REPLY
Hi Minh,
1/ Ch lm vic vi d liu numeric: PCA khng th s dng c cho d liu categorical. l
d liu trong ma trn X phi l s thc (lin tc), ch khng phi l cc categorical variable.
2/ Nhy cm vi cc im outlier/extreme: nu c vi im outlier (ngoi l) trong d liu ban

u th cht lng ca PCA c th s khng cao. Cht lng khng cao theo ngha l d liu
sau khi bin i PCA s c th khng gi c variance cao nh ban u.
3/ Khng ph hp vi cc m hnh phi tuyn, do PCA hon ton da trn cc bin i tuyn
tnh: bn cht PCA l thc hin mt bin i tuyn tnh t khng gian ban u sang khng
gian mi. Theo ngha th PCA khng mnh bng cc php bin i phi tuyn nh RBF
v.v
i
Rate This
REPLY
Em cm n anh Vu nhiu lm .
^_^
i
Rate This
REPLY
24. Trn Minh Ch says:

March 11, 2013 at 6:10 am
anh Vu i cho em hi ch ny t :
khi em test bn ca s command matlab Xhat th xut ra c ma trn nghch o nhng khi vo
code trong hm th bo li ngay dng ny V= Xhat * Xhat em dc kiu m cng ko tm c
ma trn nghc o
Chi tit li :
??? Error using ==> ctranspose
Transpose on ND array is not dened.
Error in ==> mypca at 18

V = Xhat * Xhat;
Cm n anh c
i
Rate This
REPLY
March 11, 2013 at 11:15 am
Hi em,
Anh va th li nhng khng thy c li g c. C th ma trn input X ca em c nhiu hn 2
chiu nn mi c li trn. Trc khi gi mypca(), em th gi size(X) xem kt qu l g nh.
Em c th download le mypca.m y: h ps://www.box.com/s/l7jgipl8eyqdwzlxnr2o
Matlab ci t sn PCA trong hm princomp
(h p://www.mathworks.fr/fr/help/stats/princomp.html). Nu em dng PCA trong chng
trnh ca em th nn dng hm ca Matlab. Ci t ca anh ch minh ha cho thut ton
thi.
i
Rate This
REPLY
March 11, 2013 at 5:47 pm
ah hnh ca em 3 chiu lun RGB hn chi .
i
Rate This
REPLY
March 11, 2013 at 5:55 pm
anh Vu c code no trong matlab nhn dng khun mt bng thut ton PCA c s dng Yale
Face Database hok. Thut ton PCA th em c ca anh nm r . Nhng khi p dng b d
liu Yale Face Database th em cha bt . Nu anh c th gi mail cho em vi
(minhchi_a4@yahoo.com) . Cm n Anh Vu nhiu
i
Rate This
REPLY
March 11, 2013 at 7:25 pm
y em: h p://www.mathworks.fr/matlabcentral/leexchange/17032-pca-based-face-
recognition-system
i
Rate This
REPLY
March 11, 2013 at 7:46 pm
em test c ru cm n anh Vu chia s
i
Rate This
REPLY
March 14, 2013 at 5:36 pm
anh Vu i gii thch k dm em vi cu hi m em cn thc mc vi nha :
cu 1 : Ti sao PCA li dng vector ring v tr ring ? Dng n c ch li g ?
cu 2 : cch tm K tr ring tng ng vi vector ring ca anh vit trn v ci link anh chia s
cho em trn n tm k = cch ly nhng tr ring tng ng vi vector ring > 1 th cch chn k
no tt hn anh Vu em test th cch theo ci link ny th c hnh tr v ng , c hnh tr v

kt qu sai h p://www.mathworks.fr/matlabcentral/leexchange/17032-pca-based-face-
recognition-system
cu 3: PCA dng nn d liu th mnh lu li ci g mnh c th gii nn chnh xc nh ban

u ?
Thanks anh c
i
Rate This
REPLY
29. Nguyn Minh Qun says:
April 9, 2013 at 10:53 am
Hi mi ngi, e ang lm ti v nhn dng mt s thao tc c bn ca tay ngi, trc mt e
nh lm trn cc nh tnh, sau nu ngon ngh th s lm trn webcome. Hin e mi bit ngm
cu thut ton PCA, bc no c quyn face recognition using eigenfaces and neural network
khng, share cho e vi.
Mail ca e l: tieudoan208@gmail.com Thanks mi ngi rt nhiu
i
Rate This
REPLY
30. dang hong quan says:
June 19, 2013 at 8:20 am
Hi anh, anh c th gii thch em thc mc ny khng .
Em c mt tp dataset c 58 samples, mi samples c 166200 features. Em dng PCA trong matlab
gim s lng features xung . Sau khi gim th s features cao nht m em c th t c l
57. Em khng hiu ti sao li nh vy.
i
Rate This
REPLY
June 19, 2013 at 10:07 am
hi em,
Em dng hm no trong matlab tnh PCA vy?
i
Rate This
REPLY
1. danghongquan says:
June 24, 2013 at 3:16 am
Hi anh,
Cm n anh tr li cu hi ca em. Em dng hm pca trong th vin cua prtools anh
.
i
Rate This

June 24, 2013 at 9:31 am
hi em,
c th l ma trn d liu ca e khng ng nn prtools hiu l e c 166200 samples v 58
features. Em th transpose ma trn xem.
Ngoi ra e c th dng hm princomp (h p://www.mathworks.fr/fr/help/stats/princomp.html)

ca matlab kim tra kt qu.
i
Rate This
REPLY
31. Minh Khi says:
June 19, 2013 at 2:24 pm
Minh Khi
Cho bn!
kt qu ca output_args l g sao trong code khng thy bn tr v gi tr cho n?
Nu mnh cho pca cho 1 nh th kt qu mnh c g? li t g? c cn nhn thy nh vi hm
imshow c khng?
i
Rate This
REPLY
32. Tran says:
July 5, 2013 at 5:19 am
Cho a V
PCA c th p dng nhn dng ra ngi v khng phi ngi da vo silhoue e c khng
a?
Anh c th hng dn em cch p dng c khng ?..
Anh c code C trong opencv v vn ny khng a, em ang rt mun tham kho hiu v n.
Cm n a.
i
Rate This
REPLY
33. Hoai says:
July 5, 2013 at 5:30 am
a V i, em thy trong opencv nh ngha sn lp PCA ri, vy khng bit n c khc g so vi
nhng ni dung anh cp n y khng anh? nu em mun s dng PCA trong opencv th
c cn nh ngha li khng anh?
i
Rate This
REPLY
July 5, 2013 at 10:46 am
Hi em,
Em c th xem v d y nh: h p://www.bytesh.de/blog/pca_in_opencv/
i
Rate This
REPLY
34. Tam Nguyen says:
July 6, 2013 at 5:45 pm
Cho anh. Bi vit rt b ch m em ang cn..
Em ang lm 1 project nhn dng ra nhng i tng lu sn trong Tranning Set.
Em nh nhn dng i tng lun bng cch o khong cc Eclid v tm MIN n trong trong
tp
Tranning Set a c chun ha theo PCA.
Theo anh th n c kh thi v hiu qu cao khng . ?
Thanks !
i
Rate This
REPLY
July 6, 2013 at 8:05 pm
Hi em,
Ty vo d liu ca em nh th no. Ni chung dng PCA th feature thu c c v hi
low-level, nn chnh xc c th khng cao. Em c th xem xt dng Histogram of
Gradients (HoG) rt c trng (thay v PCA), v dng SVM phn lp (thay v khong
cch Euclide, tng ng kNN vi k=1).
Tuy nhin cn ty vo d liu ca e na.
i
Rate This
REPLY
35. Chi Do Duc says:
Nu s dng Normed PCA m lch chun sigma(i)=0 th gii quyt th no h anh? Liu c
cn s dng phng php ny c na ko?
0
i
Rate This
REPLY
Trong trng hp anh thy ngi ta hay dng smoothing bng cch cng 0.01 vo
variance:
trong l hm tnh variance ca cc ct trong ma trn X (ging hm var() trong matlab).
Cch ny s m bo v em c th chia bnh thng. Nh l nu dng smoothing th em phi

cng 0.01 vo variance ca tt c cc ct, ch khng phi ch cng vo nhng ct c .
ng nhin em c th dng gi tr khc thay v 0.01, nhng khng nn chn gi tr nh qu

hoc ln qu.
i
Rate This
REPLY
1. Chi Do Duc says:
Em cm n anh rt nhiu v va c cch gii quyt tha ng li va nhanh, y l vn
em ang gp thc t .
i
Rate This
36. Duy Vng says:

cho anh. bi viet cua a thuc su rat bo ich.
em mun hi anh 1 vn : em dng thut ton PCA pht hin khun mt th ci ngng
thng dng quyt nh xem c phi l khun mt hay khng thng c chn nh th
no ?? e ko hiu ch .mong a gii p
Rate This
REPLY
1. Duy Vng says:
s< th H l bc nh khun mt ( do H gn vi khng gian mt). cai nguong a o day ly gi
tr nh th no l hp l ??
i
Rate This
REPLY
Hi em,
Cu hi ca em thun tu ph thuc vo m hnh my hc em ang dng. C th anh
khng bit s c tnh th no nn khng th tr li chnh xc c.
Tuy nhin anh on l em s dng thut ton nearest neighborhood (s l khong cch
Euclid hoc i loi vy). Trong trng hp th em cn chn sao cho t l nhn dng trn
tp validation set l cao nht.
Tuy nhin em nn s dng cc thut ton mnh hn, chng hn SVM.
i
Rate This
37. Pingback: Phng php Phn tch thnh phn chnh (Principal Component Analysis PCA) |
MFEPE
38. nguyn quang mnh says:
June 15, 2014 at 2:40 am
Em cho a .
Em nghe c gio em ni th PCA lm gim s chiu theo kiu % chnh xc. vy em mun kiu nh
pca(X, %) th em phi ci nh th no . em cm n a
i
Rate This
REPLY
September 10, 2014 at 5:34 am
Nh anh ni phn 2 bi ny: h ps://phvu.net/2011/11/15/pca-intuition-maths/
Em da vo % chnh xc chn gi tr cho , trong l s tr ring ln nht cn gi li sau
PCA.
i
Rate This
REPLY
39. Quc Cng says:
August 7, 2014 at 12:27 pm
Cho anh.
Em ang tm hiu v PCA. Anh cho em hi t tr ring v vector ring c mi lin h no vi
input (X) khng? v nhn bit c thnh phn no quan trng trong X?
i
Rate This
REPLY
40. truong duc says:
September 9, 2014 at 12:31 pm
Cho Anh!
Em cng ang nghin cu v PCA, em c v d c th: cc nhn t: GPD, dn s, gii, t l h
ngho, s trm y t, c s h tng, trong cc nhn t Em c th dng phng php PCA
xc nh nhn t no b nh hng nhiu khi l v khng Anh?
Em rt vui khi nhn c hi m ca Anh ^^. Chc Anh thnh cng!
i
Rate This
REPLY
41. Nhat Linh says:
April 7, 2015 at 4:00 am
%eigenvectors
eivec2 = zeros(size(eivec));
for i=1:size(eivec, 2)
eivec2(:, i) = eivec(:, size(eivec, 2) i + 1);

end
eivec = eivec2;
display(Full eigenvectors:);
display(eivec);
anh cho em hi dng lnh trn c phi l sp xp eivec2 tng dn khng ? em c m khng hiu
lm, nu vy em c th thay bng dng eivec2 = sort(eivec,2); khng anh?
i
Rate This
REPLY
42. Mai Bui says:
May 12, 2015 at 6:40 am
Gi anh V,
Em l sinh vin nm cui v ang lm n v tm hiu phng php PCA nhn bit ngun
gc du. T hm nhn n em tm kim ti liu v PCA m khng tm c ng. n khi
c c bi vit ny ca anh, em thc s c ng lc lm tip n cng nh em c ci
nhn tng qut PCA l g. Anh V c th cho em xin thm ti liu v PCA c khng ? Em
mun tm hiu thm v cc bc gii quyt bi ton bng PCA, t tp hp s liu cho n khi a
vo khng gian c s chiu t hn (lm th no chuyn i s liu t khng gian nhiu chiu
a vo khng gian c s chiu t hn) v nh gi kt qu.
Em rt mong nhn c hi m ca anh!
Chn thnh cm n anh!
i
Rate This
REPLY
43. Phong says:
Anh c th ch em cch lm PCA trn MiniTab 17 Khng
i
Rate This
REPLY
em xem ci ny cha: h p://support.minitab.com/en-us/minitab/17/topic-library/modeling-
statistics/multivariate/principal-components-and-factor-analysis/perform-pca-with-varimax-
rotation/
i
Rate This
REPLY
44. Lan says:
March 27, 2017 at 10:14 am
#ad cho e hi dng acc_eival = 0; hiu l g vy
i
Rate This
REPLY
1. Vu Pham says:
March 27, 2017 at 10:53 am
n l accumulated eigen values.
i
Rate This
REPLY
1. Lan says:
March 28, 2017 at 2:10 pm
vng , a gii thch gip em vng lp for dng 30 c ko, em bi ri qu
i
Rate This
45. hltomcua says:

May 16, 2017 at 1:36 pm
Anh cho em hi trong bc 2 xy dng khng gian mi, ma trn hip phng sai cov(X,Y) s
c theo cch khc ch khng phi theo cng thc (2), v (2) thc cht l ma trn t tng quan
ca cc c trng.
Trong phn code ca anh, c on
% correlation matrix
V = Xhat * Xhat;
fprintf(1, Correlation between features (columns) of normed data:);
V
nh vy tn bc vi tnh ton c v khng thng nht a..
Rt mong anh gii p thc mc ca em. Thank anh!
i
Rate This
REPLY
1. Vu Pham says:
May 17, 2017 at 11:32 pm
Ah ti v i vi cc bin ngu nhin chun ho (phng sai bng 1) th ma trn correlation
chnh l covariance.
Trong bc 1 ta chun ho cc c trng bng phng sai, nn correlation hay covariance
cng nh nhau.
i
Rate This
REPLY
1. HLtomcua says:
May 20, 2017 at 1:08 am
D vng, em cm n anh . Chc anh khe
i
Rate This
46. Nguyn Thnh Nhn says:

June 20, 2017 at 9:24 pm
Anh i cho em hi 1 t. Theo l thuyt th ci khng gian mi l xp xp cc vecto ring theo tr

ring c ln gim dn. V theo dng code th t dng 27 -> 46 l anh ang o ngc th t
cc vecto ring. Tc l dng 23 khi xut cc vecto rieng n theo th t tng dn ca cc tr
ring sn, nn ch cn o ngc li s c gim dn theo ging l thuyt .
Vy cho em hi l v sao khi xut cc tr ring nh trn, n c sp xp theo tng dn. Em test
th vi 1 ma trn bt k, th em thy tr ring xut ra khng phi tng dn. Nhng nu l tr ring
ca tch (ma trn v chuyn v ca n) ging nh l V= (X * chuyn v X) nh on code trn, th
cc tr ring s xut hin theo th t tng dn. Cho em hi l ti sao li nh v , v cc tr ring khi
tnh = matlab th s xut hin theo th t nh th no
i
Rate This
REPLY
1. Vu Pham says:
June 21, 2017 at 9:30 am
Ci ny ph thuc vo ci t ca matlab thi em. Trong documentation ca matlab khng ni
chc chn l kt qu ca eig() c sort. Thng thng th t ca cc eigenvalues tr v
theo th t tnh c trong thut ton.
Nu em mun chn chn l cc eigenvalues c th t th c th dng thm hm sort:
1 [eivec, eival] = eig(V);

2 [eival, I] = sort(diag(eival));
3 eivec = eivec(:, I);
i
Rate This
REPLY
June 21, 2017 at 5:44 pm
d em cm n anh. Vy mnh c th thay on code t dng 27->36 = 3 dng code trn ca
anh vi [eival,I] = sort (diag(eival),descend); th cng c kt qu tng t phi khng
anh. Ti em thy on for n ri qu
i
Rate This
2. Vu Pham says:
June 21, 2017 at 6:28 pm
Khng, on code t 27->36 l tnh eigenInfo, cha vi thng tin lt vt v cc tr ring
debug (v v mc ch gio dc ) thi, em khng nht thit phi vit y nh vy.
i
Rate This
REPLY
June 21, 2017 at 9:06 pm
d em cm n anh, em tng anh ang xy dng li cc vecto ring theo th t gim dn
tr ring ^^. Em hiu r , cm n v nhng chia s v s nhit tnh ca anh, n tht s gip
em rt nhiu ^^
i
Rate This
WORDPRESS.COM.
Follow
mlcvGru
Customize
Follow
Sign up
Log in
Copy shortlink
Report this content
Manage subscriptions
Collapse this bar

PCA - Principal Component Analysis - MlcvGru

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PCA - Principal Component Analysis - MlcvGru

Uploaded by

Copyright:

Available Formats

28/10/2017 PCA Principal Component Analysis | mlcvGru

Random thoughts on Machine Learning, Deep Learning

PCA Principal Component Analysis

Gip gim s chiu ca d liu,

Mt v d kinh in l hnh nh v con lc . Cng l mt con lc nhng nu nhn t bn hng

Mt v d thuyt phc hn c minh ha trong hnh sau

Mt im rt p na ca PCA l cc trc ta trong khng gian mi lun m bo trc giao i

2. Thut ton PCA

Cho ma trn . Cc bc ca PCA ln lt nh sau:

D liu ban u c th c gi tr thay i bt thng. V d trn feature 1 (ct 1 ca ) gi tr thay

Centered PCA mang tt c cc feature (cc ct ca X) v cng mt gc ta :

Trong n l s dng ca X, l mean ca ct th j ca X, c tnh nh trn.

Normed PCA mang tt c cc feature v cng mt gc ta , ng thi chun ha v cng mt

Trong l lch chun (standard deviation) ca ct th j trong X.

2. Xy dng khng gian mi

Tnh ma trn hip phng sai (covariance) ca cc feature trong :

Do l tch ca ma trn vi chuyn v ca n nn l ma trn positive semidenite kch

Tip theo, PCA tm tr ring v vector ring

v p vector ring tng ng l

C th ni trong PCA, tr ring v vector ring c v tr rt p, tha mn tt c cc yu cu ca PCA.

3. Chuyn d liu t khng gian ban u vo khng gian mi

Xong. Ta kt thc gii thut PCA, khng th n gin hn.

Ngoi ra nh gi cht lng ca khng gian mi to bi PCA, ta dng 2 o l contribution v

1 function [ output_args ] = mypca( X )

70 SC(i, : ) = SC(i, : )/sum(SC(i,: ), 2);

Ch lm vic vi d liu numeric,

Theo em hiu th Karhunen l ma trn A dng bin Rx (covariance ca X) thnh Ry (l ma trn

e cam on anh nhieu

Rat hy vong nhan duoc su hoi am cua anh

Do la truc giao Vay truc giao la gi- y nghia cua no de lam gi

Em cam on anh nhieu

Chuc anh suc khoe va that nhieu niem vui

Buoc thu 2 em ko hieu lam

em cam on anh nhieu!

Nh vy sau khi chuyn i, ta thy gi tr trung bnh ca mi ct trong ma trn mi u

18. dduuyy says:

trong l trung bnh cng ca tt c cc nh.

Sau 2DPCA tm tr ring v vector ring ca ma trn G, cc vector ring ng vi cc tr

Ch lm vic vi d liu numeric,

Nh anh Vu gii thch dm em 3 trn vi , em cn m h qu . Cm n anh

2/ Nhy cm vi cc im outlier/extreme: nu c vi im outlier (ngoi l) trong d liu ban

24. Trn Minh Ch says:

Error in ==> mypca at 18

no tt hn anh Vu em test th cch theo ci link ny th c hnh tr v ng , c hnh tr v

cu 3: PCA dng nn d liu th mnh lu li ci g mnh c th gii nn chnh xc nh ban

Em dng hm no trong matlab tnh PCA vy?

2. PHAM Hoai Vu says:

Ngoi ra e c th dng hm princomp (h p://www.mathworks.fr/fr/help/stats/princomp.html)

Em c th xem v d y nh: h p://www.bytesh.de/blog/pca_in_opencv/

Theo anh th n c kh thi v hiu qu cao khng . ?

Tuy nhin cn ty vo d liu ca e na.

trong l hm tnh variance ca cc ct trong ma trn X (ging hm var() trong matlab).

Cch ny s m bo v em c th chia bnh thng. Nh l nu dng smoothing th em phi

ng nhin em c th dng gi tr khc thay v 0.01, nhng khng nn chn gi tr nh qu

36. Duy Vng says:

Tuy nhin em nn s dng cc thut ton mnh hn, chng hn SVM.

eivec2(:, i) = eivec(:, size(eivec, 2) i + 1);

Em rt mong nhn c hi m ca anh!

Chn thnh cm n anh!

45. hltomcua says:

46. Nguyn Thnh Nhn says:

Anh i cho em hi 1 t. Theo l thuyt th ci khng gian mi l xp xp cc vecto ring theo tr

Nu em mun chn chn l cc eigenvalues c th t th c th dng thm hm sort: