Professional Documents
Culture Documents
)HDWXUHV6HOHFWLRQIRU0XOWLFDPHUD7UDFNLQJ
,QWHUQDWLRQDO,VODPLF8QLYHUVLW\0DOD\VLD
.XDOD/XPSXU0DOD\VLD
HPDLO\DVLU#LLXPHGXP\
Abstract6QDWFK WKHIW LV EHFRPLQJ PRUH SUHYDOHQW LQ
DQG PRWRUF\FOLVW WKDW DUH UHODWHG WR WKH VQDWFK UREEHU\
0DOD\VLD QRZDGD\V DQG SURSHU PHDVXUHV PXVW EH WDNHQ WR
HYHQW 7KH UHVW RI WKH SDSHU LV RUJDQL]HG DV IROORZV
UHGXFHLW&&79VXUYHLOODQFHV\VWHPVKDYHEHHQZLGHO\XVHG
5HODWHG ZRUN RQ PXOWLFDPHUD WUDFNLQJ V\VWHP DQG WKH
DV D VWUHHW FULPH SUHYHQWLRQ WRRO DFURVV SXEOLF DQG SULYDWH
PHWKRGV XVHG DUH GLVFXVVHG LQ 6HFWLRQ ,, 7KH DOJRULWKP
DUHDV 7UDFNLQJ WKH VDPH REMHFW ZLWKLQ GLIIHUHQW FDPHUDV
XVHG IRU REMHFW GHWHFWLRQ LV GLVFXVVHG LQ 6HFWLRQ ,,,
YLHZLVHVVHQWLDOLQPDQ\VXUYHLOODQFHDSSOLFDWLRQV5HFHQWO\
IROORZHG E\ WUDFNLQJ ZLWKLQ VLQJOH FDPHUD LQ 6HFWLRQ ,9
PRVWRIWKHUHVHDUFKHUVKDYHJURZQPRUHLQWHUHVWRQKRZWR
DQGPXOWLFDPHUDWUDFNLQJLQ6HFWLRQ97KHH[SHULPHQWDO
WUDFN REMHFWV DFURVV FDPHUDV GXH WR WKH LQFUHDVLQJ QXPEHU
UHVXOWV DQG GLVFXVVLRQV DUH JLYHQ LQ 6HFWLRQ 9, )LQDOO\
RIFDPHUDV+RZHYHUWKHFXUUHQWDSSURDFKSURSRVHGE\WKH
FRQFOXVLRQLVGHVFULEHGLQ6HFWLRQ9,,
UHVHDUFKHUV VWLOO RIIHU WUDGHRII LQ WHUPV RI LWV DFFXUDF\ DQG
VSHHG $V WKH WUDFNLQJ DFFXUDF\ LQFUHDVHV WKH VSHHG ZLOO
GHFUHDVH WKDW DFWV UHYHUVHO\ SURSRUWLRQDO WR LW 7KLV SDSHU
SUHVHQWV D QRYHO DSSURDFK WR WUDFN PRYLQJ REMHFWV DFURVV
GLVWULEXWHGFDPHUDVWKDWSURYLGHVWKHPRVWRSWLPDOWUDGHRII
EDVHG RQ FRORU WH[WXUH DQG HGJH IHDWXUHV 7KH FRORU HGJH
DQG WH[WXUH IHDWXUHV IRU WDUJHW DQG FDQGLGDWH EOREV DUH
FRPSXWHG E\ D QRYHO FRPSXWDWLRQDO DOJRULWKP 7KLV VWXG\
IRFXVHV RQ DQDO\]LQJ RI YLGHR VXUYHLOODQFH LQ SXEOLF SODFHV
VSHFLILFDOO\LQRXWGRRUHQYLURQPHQW,QWKHUHVXOWVHFWLRQWKH
FRPSDULVRQEHWZHHQWKHHIIHFWLYHQHVVRIWKHIHDWXUHVXVHGLQ
WKHWUDFNLQJDOJRULWKPLVSUHVHQWHG7KHSHUIRUPDQFHRIWKH
PHWKRG LV DQDO\]HG EDVHG RQ LWV DFFXUDF\ DQG VSHHG 7KH
PRUH VXLWDEOH IHDWXUHV DUH LGHQWLILHG ([SHULPHQWDO UHVXOWV
VKRZWKHHIIHFWLYHQHVVRIWKLVPHWKRGIRUUHDOWLPHRSHUDWLRQ
,,
,
,1752'8&7,21
5(/$7(':25.
243
DQG GLUHFWLRQDO FXHV RI WKH KXPDQ SHGHVWULDQ DUH XVHG WR
FRPSHQVDWHZLWKVPDOOYDULDQFHVLQSRVHVDQGWRGHDOZLWK
WKH FKDQJHV LQ YLHZSRLQW +\EULG RI FRORU PRYLQJ
GLUHFWLRQ VSHHG VLGHRI HQWU\ KHLJKW DQG WUDYHO WLPH DUH
XVHGWRWUDFNREMHFWVDFURVVPXOWLSOHFDPHUDV>@
)LJ%DFNJURXQGVXEWUDFWLRQEDVHGRQ0L[WXUHRI*DXVVLDQD7KH
IRUHJURXQGREMHFWLVGLYLGHGLQWRSRUWLRQE2QO\WKHXSSHUSDUWRIWKH
IRUHJURXQGREMHFWLVFRQVLGHUHGGXULQJWUDFNLQJ
,9 75$&.,1*:,7+,16,1*/(&$0(5$9,(:
%RXQGLQJER[HVIRUWKHELQDU\LPDJHLVREWDLQHGXVLQJ
&RQQHFW &RPSRQHQW /DEHOLQJ >@ )RU WKH IHDWXUH
H[WUDFWLRQDVPDOOHUERXQGLQJER[ORFDWHGDWWKHKDOIWRS
UHJLRQRIWKHERXQGLQJER[ORZHUWKDQQHFNDQGEHWZHHQ
WKH XSSHU OLPEV LV VHOHFWHG WR UHGXFH WKH SRVVLELOLW\ RI
IDOVH WUDFN )RU WUDFNLQJ ZLWKLQ VLQJOH FDPHUD YLHZ WKH
REMHFW SRVLWLRQ DQG FRORU IHDWXUHV DUH XVHG $OJRULWKP
GHVFULEHGDQRYHOPHWKRGIRUFRPSXWDWLRQDORI+69FRORU
IHDWXUH 7KH ILUVW VWHS LV WR H[WUDFW WKH +69 FRORU VSDFH
IURP WKH LQWHQVLW\ LPDJH 7KH VHFRQG VWHS LV WR FRPSXWH
WKH PHDQ YDOXH IRU HDFK +69 FKDQQHO 7KH VLPLODULW\ RI
HDFKWDUJHWDQGFDQGLGDWHEOREDUHFRPSXWHGEDVHGRQ
,,, 2%-(&7'(7(&7,21
A. Background Subtraction
%DFNJURXQG VXEWUDFWLRQ >@ LV GRQH E\ ILQGLQJ WKH
GHYLDWLRQVRIHDFKLQFRPLQJIUDPHVIURPWKHEDFNJURXQG
PRGHO WR REWDLQ WKH IRUHJURXQG REMHFW 0L[WXUH RI
*DXVVLDQ LV XVHG WR PRGHO SHUSL[HO EDFNJURXQG FRORU
7KH SL[HOV WKDW GHYLDWH IURP WKH EDFNJURXQG PRGHO DUH
ODEHOOHGDVWKHIRUHJURXQGSL[HOV%DVHGRQ)LJDWKH
IRUHJURXQGREMHFWLVQRWFRUUHFWO\GHWHFWHG7KHRQO\SDUW
WKDWLVGHWHFWHGLVUHSUHVHQWHGE\WKHZKLWHSL[HOVDQGWKH
DFWXDOREMHFWLVVKRZQLQ)LJE
B. Morphological Operation
0RUSKRORJLFDORSHUDWLRQ>@LVDSSOLHGWRDQLPDJHWR
VXSSUHVVWKHQRLVHVLQWKHLPDJH
$OJRULWKP&RPSXWDWLRQIRU+69&RORU
1) Opening
LV
7KH RSHQLQJ RI $ E\ % GHQRWHG E\
GHILQHG DV WKH HURVLRQ RI $ E\ % IROORZHG E\ D
GLODWLRQRIWKHUHVXOWE\%EDVHGRQ
2UFDQEHH[SUHVVHGDV
GHQRWHVWKHXQLRQRIDOOVHWVLQVLGHWKH
:KHUH
EUDFHV ,Q RWKHU ZRUGV
LV WKH XQLRQ RI DOO
WUDQVODWLRQRI%WKDWILWHQWLUHO\ZLWKLQ$
2) Closing
LVGHILQHGDV
7KHFORVLQJRI$E\%GHQRWHG
WKHGLODWLRQRI$E\%IROORZHGE\DHURVLRQRIWKH
UHVXOWE\%EDVHGRQ
LV WKH FRPSOHPHQW RI WKH XQLRQ RI DOO
WUDQVODWLRQVRI%WKDWGRQRWRYHUODS$
H = 0; S = 0; V = 0;,QLWLDOL]HWKHLQWHQVLW\YDOXHVIRU
HDFK FKDQQHOZLWKLQ WKH UHJLRQ Total pixel LV WKH QXPEHU
RISL[HOLQWKHUHJLRQ
IRU(DFKSL[HOx ZLWKLQUHJLRQGR
H H + 1;XSGDWHYDOXHRI+XH
S S + 1;XSGDWHYDOXHRI6DWXUDWLRQ
V V + 1;XSGDWHYDOXHRI9DOXH
HQGIRU
P(H) = H/Total Pixel;FRPSXWHPHDQIRU+XHFKDQQHO
ZLWKLQWKHUHJLRQ
P(S) = S/Total Pixel; FRPSXWH PHDQ IRU 6DWXUDWLRQ
FKDQQHOZLWKLQWKHUHJLRQ
P(V) = V/Total Pixel;FRPSXWHPHDQIRU9DOXHFKDQQHO
ZLWKLQWKHUHJLRQ
)LJ7UDFNLQJUHVXOWIRUVLQJOHFDPHUD
7KH REMHFW SRVLWLRQ LV FRPSXWHG XVLQJ (XFOLGHDQ
GLVWDQFHEDVHGRQ7KHUHVXOWRIWUDFNLQJZLWKLQVLQJOH
FDPHUDYLHZFDQEHVHHQLQ)LJ
:KHUH
244
DQG
9
08/7,&$0(5$75$&.,1*
SL[HOORFDWLRQ
7KLV SDSHU LV LQVSLUHG E\ WKH ZRUN RI >@ ZKLFK
GHSHQGVVROHO\RQFRORUIHDWXUH,QIDFWWKLVVWXG\IRFXVHV
RQ VHOHFWLQJ WKH RSWLPDO IHDWXUHV WR EH XVHG IRU PXOWL
FDPHUD WUDFNLQJ ,Q RUGHU WR PDNH WKH V\VWHP WR EH PRUH
UREXVW HGJH LV XVHG DV DQ DGGLWLRQDO IHDWXUH VLQFH LW LV
LQVHQVLWLYH WR LOOXPLQDWLRQ 2EMHFW DSSHDUDQFH PD\ EH
GLIIHUHQWLDWHGLQWHUPVRILWVFORWKLQJVXFKDVFKHFNHGVKLUW
RU SODLQ VKLUW ,W FDQ EH GLIIHUHQWLDWHG E\ XVLQJ WKH HGJH
IHDWXUH RI WKH FORWKLQJ 7KH HGJH IHDWXUH LV REWDLQHG E\
XVLQJ&DQQ\GHWHFWRU>@$OJRULWKPGHVFULEHGDQRYHO
PHWKRG IRU FRPSXWDWLRQDO RI HGJH IHDWXUH 7KH VLPLODULW\
RIHDFKWDUJHWEOREDQGFDQGLGDWHEOREDUHFRPSXWHGEDVHG
RQ
HQGLI
LIORFDWLRQRISL[HOLVQRWHTXDOWRFHQWHUSL[HO
FRPSXWHWKHGHFLPDO
YDOXHIRUFHQWHUSL[HOEDVHGRQELQDU\YDOXH
RIQHLJKERUKRRGSL[HO
p p + 1XSGDWHWKHSRZHUIRUGHFLPDO
FRPSXWDWLRQ
HQGLI
HQGIRU
D(x) = 05HLQLWLDOL]HWKHQXPEHURIGHFLPDOYDOXH
IRUFHQWHUSL[HOZLWKLQ[ZLQGRZ
$OJRULWKP&RPSXWDWLRQIRU(GJH)HDWXUH
p = 05HLQLWLDOL]HWKHSRZHUIRUGHFLPDOYDOXH
FRPSXWDWLRQ
p = 0,QLWLDOL]HWKHWRWDOQXPEHURIZKLWHSL[HOLQWKH
UHJLRQB(x) LVWKHELQDU\YDOXHDWSL[HOx. Total pixel LVWKH
QXPEHURISL[HOLQWKHUHJLRQ
HQGIRU
IRU(DFKSL[HOx ZLWKLQUHJLRQGR
LIB(x) LVHTXDOWRWKHQ
p p + 1XSGDWHWKHIUHTXHQF\RIHGJH
HQGLI
FDOFXODWHGDQGJLYHQDZHLJKW 7KHVLPLODULW\VFRUHVDUH
UHIHUUHGWRDQG
HQGIRU
P(p) = p/Total PixelFRPSXWHSUREDELOLW\RIRFFXUULQJ
ZKLWHSL[HOZLWKLQWKHUHJLRQ
9, (;3(5,0(17$/5(68/76
7KH FRPSXWDWLRQDO RI /%3 IHDWXUH IRU HDFK SDWFK LQ
$OJRULWKP LV UHSUHVHQWHG E\ PQ PDWUL[ ZKLFK LV
QXPEHU RI URZV DQG FROXPQV UHVSHFWLYHO\ 7KH SDWFKHV
PDWULFHV DUH FRQFDWHQDWHG WR REWDLQ D VHW RI GHFLPDO
YDOXHV 7KHQ WKH GHFLPDO YDOXHV DUH GLYLGHG LQWR
QXPEHU RI ELQV 7KH VLPLODULW\ EHWZHHQ WDUJHW DQG
FDQGLGDWHEOREDUHFRPSXWHGEDVHGRQ
A. Object Recognition
7KHH[SHULPHQWZDVFRQGXFWHGE\XVLQJ9,3H5GDWDVHW
DQG LPDJHV REWDLQHG IURP VWUHHW HQYLURQPHQW EDVHG RQ
HGJH FRORU DQG WH[WXUH IHDWXUHV 7KH UHVXOW IURP )LJ
GHPRQVWUDWHV WKH HIIHFWLYHQHVV RI WKH DOJRULWKP WR
UHFRJQL]HSHGHVWULDQZLWKVLPLODUDSSHDUDQFHZKHUHDV)LJ
VKRZVWKHUHVXOWIRUPRWRUF\FOLVWVUHFRJQLWLRQ
$OJRULWKP&RPSXWDWLRQIRU/%3)HDWXUH
B. Object Tracking
7KH H[SHULPHQW LV EDVHG RQ FDPHUDV LQ RXWGRRU
HQYLURQPHQW ZLWK QRQRYHUODSSLQJ YLHZV 7KH DOJRULWKP
LV WHVWHG EDVHG RQ VLQJOH DQG PXOWLSOH IHDWXUHV 7KH
ZHLJKWV RI FRORU WH[WXUH DQG HGJH XVHG DUH DQG
UHVSHFWLYHO\ ZLWK RYHUDOO WKUHVKROG RI )LJ
VKRZV WKH UHVXOW IRU WUDFNLQJ XVLQJ K\EULG RI HGJH FRORU
DQGWH[WXUHIHDWXUHV
D(x) = 0; p = 0;,QLWLDOL]HWKHQXPEHURIGHFLPDOYDOXH
IRU FHQWHU SL[HO ZLWKLQ [ ZLQGRZ DQG WKH SRZHU IRU
GHFLPDO YDOXH FRPSXWDWLRQ I(x) LV WKH LQWHQVLW\ YDOXH DW
SL[HOx.
IRU(DFKSDWFKGR
IRU(DFKSL[HOx ZLWKLQ[ZLQGRZGR
LII(x)RIQHLJKERUKRRGSL[HO LVJUHDWHUWKDQI(x)
RIFHQWHUSL[HOWKHQ
B(x) 1DVVLJQELQDU\WRWKDWQHLJKERUKRRG
SL[HOORFDWLRQ
HOVH
B(x) 0DVVLJQELQDU\WRWKDWQHLJKERUKRRG
245
)LJ7UDFNLQJUHVXOWIRUSHGHVWULDQVZLWKVLPLODUDSSHDUDQFHD
7DUJHWEORELQFDPHUD$E&DQGLGDWHEORELVFRUUHFWO\DVVLJQHGLQ
FDPHUD%F2WKHUFDQGLGDWHEOREVLQFDPHUD%WKDWDUHQRWEHORQJWR
WKHWDUJHWREMHFWLQFDPHUD$
9,, &21&/86,21
7KLV SDSHU SUHVHQWV D QRYHO VROXWLRQ IRU WUDFNLQJ
REMHFWV DFURVV GLVWULEXWHG FDPHUDV ZLWK QRQRYHUODSSLQJ
YLHZV LQ UHDOWLPH DSSOLFDWLRQ 0XOWLSOH IHDWXUHV DUH
FRPELQHG XVLQJ RYHUDOO VLPLODULW\ VFRUH WR WUDFN REMHFWV
DFURVV GLVWULEXWHG FDPHUDV 7KHVH IHDWXUHV DUH FRORU
WH[WXUHDQGHGJH)URPWKHH[SHULPHQWDOUHVXOWVWKHUHVXOW
IRU WUDFNLQJ SHUIRUPDQFH DFFXUDF\ DQG VSHHG DUH
LPSURYHG 7KH IHDWXUHGEDVHG WUDFNLQJ DOJRULWKP XVHG LQ
WKLV SDSHU ZRXOG EH YHU\ KHOSIXO IRU SUHGLFWLRQ RI WKH
VQDWFKUREEHU\RFFXUUHQFH
)LJ7UDFNLQJUHVXOWIRUPRWRUF\FOLVWVD7DUJHWEORELQFDPHUD$
E&DQGLGDWHEORELVFRUUHFWO\DVVLJQHGLQFDPHUD%F2WKHUFDQGLGDWH
EOREVLQFDPHUD%WKDWDUHQRWEHORQJWRWKHWDUJHWREMHFWLQFDPHUD$
7KH UHVXOW RI WUDFNLQJ SHUIRUPDQFH LV WDEXODWHG LQ
7DEOH %DVHG RQ VLQJOH IHDWXUH FRORU +69 IHDWXUH
SHUIRUPHG WKH EHVW ZLWK WUDFNLQJ DFFXUDF\ RI
SHUFHQWV ZLWK WKH KLJKHVW VSHHG RI IUDPHV SHU
VHFRQG &RPELQDWLRQ RI SRVLWLRQ FRORU HGJH DQG WH[WXUH
IHDWXUHVJLYHWKHKLJKHVWDFFXUDF\RISHUFHQWVDQG
ZLWKWKHKLJKHVWVSHHGRIIUDPHVSHUVHFRQG
$&.12:/('*0(17
7KLV ZRUN LV VXSSRUWHG E\ )XQGDPHQWDO 5HVHDUFK
*UDQW 6FKHPH )5*6 E\ WKH 0LQLVWU\ RI (GXFDWLRQ
0DOD\VLD
5()(5(1&(6
>@ < &DL . +XDQJ DQG 7 7DQ 0DWFKLQJ WUDFNLQJ VHTXHQFHV
DFURVV ZLGHO\ VHSDUDWHG FDPHUDV 3URF ,((( WK ,QW &RQI
,PDJH 3URFHVVLQJ ,&,3 ,((( 3UHVV 2FW SS
GRL,&,3
>@ + + +VX : 0 <DQJ DQG 7 . 6KLK 3HRSOH WUDFNLQJ LQ D
PXOWLFDPHUD HQYLURQPHQW 3URF ,((( &RQI $QWKRORJ\ ,(((
3UHVV
-DQ
SS
GRL
$17+2/2*<
>@ ; &KHQ . +XDQJ DQG 7 7DQ 'LUHFWLRQEDVHG VWRFKDVWLF
PDWFKLQJ IRU SHGHVWULDQ UHFRJQLWLRQ LQ QRQRYHUODSSLQJ FDPHUDV
3URF ,((( WK ,QW &RQI ,PDJH 3URFHVVLQJ ,((( 3UHVV 6HSW
SSGRL,&,3
>@ '7/LQDQG.<+XDQJ&ROODERUDWLYHSHGHVWULDQWUDFNLQJDQG
GDWD IXVLRQ ZLWK PXOWLSOH FDPHUDV ,((( 7UDQV ,QI )RUHQV
6HFXULW\ YRO 'HF SS GRL
7,)6
>@ $ <LOPD] 2 -DYHG DQG 0 6KDK 2EMHFW WUDFNLQJ D VXUYH\
$&0 &RPSXWLQJ 6XUYH\V &685 YRO 'HF SS
GRL
>@ < :DQJ / +H DQG 6 9HOLSDVDODU 5HDOWLPH GLVWULEXWHG
WUDFNLQJ ZLWK QRQRYHUODSSLQJ FDPHUDV 3URF ,((( WK ,QW
&RQI,PDJH3URFHVVLQJ,(((3UHVV6HSWSSGRL
,&,3
>@ 0 2 0HKPRRG DQG $ .KDZDMD 0XOWLFDPHUD EDVHG KXPDQ
WUDFNLQJZLWKQRQRYHUODSSLQJILHOGVRIYLHZ3URF,(((WK,QW
&RQI ,PDJH DQG *UDSKLFV ,((( 3UHVV 6HSW SS
GRL,&,3
>@ 0 6RQND 9 +ODYDF DQG 5 %R\OH ,PDJH SUHSURFHVVLQJ LQ
,PDJH 3URFHVVLQJ $QDO\VLV DQG 0DFKLQH 9LVLRQ QG HG
7KRPVRQSS
>@ 06RQND9+ODYDFDQG5%R\OH0DWKHPDWLFDOPRUSKRORJ\
LQ ,PDJH 3URFHVVLQJ $QDO\VLV DQG 0DFKLQH 9LVLRQ QG HG
7KRPVRQSS
>@ 0 6RQND 9 +ODYDF DQG 5 %R\OH 6KDSH UHSUHVHQWDWLRQ DQG
GHVFULSWLRQ LQ ,PDJH 3URFHVVLQJ $QDO\VLV DQG 0DFKLQH 9LVLRQ
QGHG7KRPVRQSS
)LJ7UDFNLQJUHVXOWIRUPXOWLSOHFDPHUDVD2EMHFWLVDVVLJQHGWR
ODEHOLQFDPHUDE2EMHFWVDUHFRUUHFWO\WUDFNHGLQFDPHUD
7$%/(,75$&.,1*3(5)250$1&(%$6('21326,7,21&2/25
('*($1'7(;785()($785(6)257:29,'(26(48(1&(6
)HDWXUHV
&RORU
(GJH
7H[WXUH
3RVLWLRQ&RORUDQG
(GJH
3RVLWLRQ&RORUDQG
7H[WXUH
3RVLWLRQ(GJHDQG
7H[WXUH
3RVLWLRQ&RORU(GJH
DQG7H[WXUH
$FFXUDF\
6SHHG
)UDPHVVHFRQG
246