Damien Lefloch Master Thesis

Real-Time People Counting system
using Video Camera

Damien LEFLOCH
Master of Computer Siene!
"mage and #rtifiial "ntelligene
$%%&
Super'isors( )on *+ Harde,erg!
Faou-i #laya C.ei/.! Pierre 0outon
Department of Computer Siene
and Media Te.nology
012'i/ 3ni'ersity College
4o5 676
8-$9%$ 012'i/
8or:ay
3FR Sienes et Te.ni;ues
4atiment Mirande
3ni'ersit< de 4ourgogne
4P =&9&%
$6%&9 Cede5
Frane
P+i+D Solutions
Osar 8issensgate >
8-$9=7 012'i/
8or:ay
Real-Time People Counting System using Video Camera
Abstract
# people ounter is a de'ie used to ount t.e num,er of pedestrians :al/ing
t.roug. a door or orridor+ Most of t.e time! t.is system is used at t.e entrane of a
,uilding so t.at t.e total num,er of 'isitors an ,e reorded+ People ounting system
is important for mar/eting resear. ?pedestrian traffi management! tourists flo:
estimation@ or in seurity appliation ?in t.e ase of an e'auation! it is essential to
/no: .o: many people are inside t.e ,uilding at any gi'en time@+
Some sensors are used to ount people in a ,uilding :it. different ad'antages and
dra:,a/s and different auray su. as laser ,eam! infra-red sensor! t.ermal sensor
and reently 'ideo amera+
"n t.e literature o'er'ie:! it :as found t.at t.e most relia,le and aurate sensor
:as t.e 'ideo amera ,ut it :as also t.e most e5pensi'e and .ea'y on omputing+
Se'eral papers .a'e ,een pu,lis.ed t.is last ten years for ounting ?or tra/ing@
people ,y using 'ideo proessing :it. different met.ods ?e5tration of t.e pedestrian
sil.ouette! ma/e retangular pat.es :it. a ertain ,e.a'iour! mounting t.e amera
'ertially to a'oid olusion pro,lem! segmentation of .uman region ,y using stereo
'ideo amera@+
T.e aim of t.is t.esis :as to ma/e a prototype of a real-time ounting people
system 'ideo ,ased ,y using t.e Matla,-Simulin/ programming tool+ #nd t.en try to
measure its auray and ompare to anot.er system ,ased on laser ,eam sensors+
#nd finally! onlude in :.i. situations t.is system is more relia,le and t.erefore
find its ad'antages and dra:,a/s+
A
Preface
T.is master t.esis :as arried out :it. t.e 8or:egian Colour Resear. La,oratory
in olla,oration :it. t.e 8or:egian ompany P+"+D Solutions+ For seurity reasons!
t.is fire ompany as/ed t.e Colour La,oratory to ,uild a system :.i. an ount .o:
many people are in ,uildings at any gi'en time+ #tually! t.is ompany .ad already a
system li/e t.is ,ut it ould only measure t.e flo: of people B :it.out any distintion
a,out t.e diretion - and its maintenane :as e5tremely e5pensi'e+
T.is ,ig pro1et :as di'ided in A parts ,et:een four studentsC T:o students!
Mat.ias Sa'elle and 8iolas Mauduit! :or/ed :it. t.e eletronis tea.ers to set-up a
people ounting system ,ased :it. laser ,eam+ One student! *orri/ )ansen! :as in
.arge of t.e soft:are part - implementing a :e, appliation and a entralised store
data,ase+ #nd finally! " :or/ed on a people ounting system ,ased :it. Video Camera+
During t.is master t.esis! :e also .ad to set-up an operational prototype of t.e
system ,eause of t.e s.ort delay timeC o,'iously! four mont.s :ere a too s.ort period
to ,uild a full people ounting system+
Damien Leflo.! $%%&D%7D%&
Acknowledgements
During t.is master t.esis! " .a'e gotten many good ad'ies+ First! " :ould li/e to
t.an/ my super'isors in partiularly Professor Faou-i #laya C.ei/ :.o :as a 'ery
great .elp for me! .e .elps me to ,egin :it. t.e Matla,-Simulin/ programming tool
t.at " didnEt /no: ,efore and ,orro: me t.e different :e,-ams+ Seond! " :ould li/e
to t.an/s my friend *orri/ for .is good ad'ise and good time :e .a'e toget.er in
8or:ay+ " :ould also li/e to t.an/ an anot.er friend of me! 4ar,ora Fomin/o'G! :.o
.elped me and moti'ated to :rite t.is final report+ #nd o,'iously! t.an/s to all my
family and friends in partiularly my fat.er :.o al:ays ,elie'ed in me ?:it.out you! i
ouldnEt ,eome :.at " am no:@+
H
Presentation of our work
#s said ,efore! t.is pro1et :as di'ided in t.ree ,ig parts+ "n t.is setion! " :ill 1ust
present you a ,rief analysis of t.e full pro1et! its important features and t.e different
e5isting system+
T.e first :or/s to do :as to tal/ :it. t.e fire engineer a,out t.e system .e :ould
li/e to .a'e ?:.atEs t.e main purpose and features@! to /no: :.y .is urrent system
doesnEt suit .imself+ Hopefully! .is ans:ers :as really lear+ He :ould li/e a system
apa,le of /no:ing e5atly .o: many people are in ,uildings ?for e5ample
supermar/ets@ at any gi'en time+ T.e main goal of t.is is to manage! in t.e se;uel! t.e
different fire e5its ?add or remo'e fire e5it! get ,igger or smaller fire e5it depending of
t.e ati'ity of t.e ,uildings@ so it .as to ,e relia,le to ,e omplying :it. la:+
List of features
T.e people ounting system must fit :it. t.e e5isting Et.ernet net:or/ of t.e
ompany+
# .ig. auray+
Management of se'eral ,uildings ?need a entralised data,ase updated ,y ea.
,uildingEs main omputer@+
E'olution ?in order to add some e5tra features li/e e'aluate t.e impat of an
ad'ertisement@+
Rea.a,le from any plae ?,y using a :e, appliation onneted :it. t.e
entralised data,ase@+
Easy management of onfiguration+
4rea/do:n management ?detet t.e malfuntion of sensors ,y using t.e
TCPD"P protool@+
Easy aess to statistis for e'ery ,uildings or e'ery doors of ea. ,uilding :it.
different inter'al of time ?daily! mont.ly or yearly@+
3se! :.en itEs possi,le! free te.nologies to redue t.e ost of t.e system+
Researches of existing people counting system
Ie didnEt /no: anyt.ing a,out t.at /ind of te.nologies! t.en :e .ad to go furt.er
in t.e su,1et and get some doumentation and e5amples a,out people ounting
systems+ Here are t.e main features :e found out during our resear.es+
4esides! t.eses resear.es :ere arried out at t.e same time :e defined t.e
features :it. t.e engineer from P+"+D+ Solutions+ "ndeed! :e :ere loo/ing for
information :.ile :e proposed se'eral solutions and features to t.e lient+ Some
meetings allo:ed us to meet .is demand :it. all information :e gat.ered from our
sur'ey+
Sine our resear.es :ere ,ased on t.e internet! :e diso'ered plenty ompaniesE
:e, sites! among t.ose+ Here are t.e most interesting ones+
&
Name of the
company
Web site Field
#orel
.ttp(DD:::+aorel+omDenDsoft:a
re+asp
People ounting
systems in pu,li
transport! s.opping
entre
SPSL
.ttp(DD:::+ustomerounting+o
mD
People ounting! people
,e.a'iour analysis
"nfode' .ttp(DD:::+infode'+aD
People ounting in
,uildings and 'e.iles
#,te/ontrols
.ttp(DD:::+a,te/ontrols+om
D
People ounting
systems :it. different
sensors su. as laser
,eams
Video turnstile
.ttp(DD:::+'ideoturnstile+om
D
People ounting
systems ,ased on 'ideo
proessing
Table 1: Interesting companies and their web sites
#s you sa: a,o'e! se'eral ompanies set-up people ounting systems! ,esides one
of t.ose .as integrated a po:erful tool to analyse people ,e.a'iour at t.e same time
t.an ounting .o: many people are inside t.e ,uilding+ 8e'ert.eless! :e are going to
fous on people ounting people systems :.i. is ,asially are di'ided into t:o parts(
sensors ?detetion and ounting@ and soft:are ?data management@+
The different type of sensors
# lot of systems use 'ideo ameras to detet people lea'ing or entering t.roug. a
door+ T.e 'ideo ameras are generally set-up on a :all! ,et:een 6!H% and = meters
a,o'e t.e floor+ Sine t.e system get only images from t.e ameras! an ot.er
omponent re;uired to analyse t.e images and allo: people ounting+ T.is de'ie is
plaed lose to ea. amera in order to a'oid sending too mu. data t.roug. t.e
ompanyEs net:or/+ "n fat! ta/e onsider of a 'ideo amera apa,le of a;uiring a
'ideo in R04$=
6
format at A$%5$=% Pi5els
$
+ T.us! ea. image of t.e 'ideo :ill ,e
oded in A5A$%5$=% J $A%=%% 4ytes J $$H F4+ 8o:! if t.is 'ideo amera operates in
6H frames per seond ?FPS@! one seond of t.e 'ideo :ill ,e oded in 6H5$$H J AA&H F4
J A!A M4+ T.is is totally inonei'a,le! in order to send t.e 'ideo data! t.e proess :ill
use A!AM4Ds of t.e net:or/Es ,and:idt. for one 'ideo amera+ T.e net:or/ :ill ,e
saturated+ To resol'e t.is pro,lem! t.e image proessing algorit.m lose to t.e
amera ?for e5ample a miro-proessor ,uilt-in t.e 'ideo amera@ and t.en 1ust send
t.e ounting data+ 4ut t.e fat is t.at t.is proess is ompliated and so 'ery ostly in
terms of omputing and money+ Ho:e'er! t.is system is also t.e most aurate and
an ,e used and impro'ed to ot.er appliations li/e seurity ?detet KdangerousL
situations@ or mar/eting ?analyse t.e ,e.a'iour of lients@+
"nfra-red lig.t produed ,y red LED is anot.er :ay to ount people+ # de'ie
6 R04$= format is omposed of t.ree .annels ?Red! 4lue! 0reen@+ Ea. olour of t.is format is oded in
A 4ytes ?one ,yte per .annel@ so it an reate $
$=
different olours+
$ # pi5el ?piture element@ is a single point in a images+ T.e sreensE resolution is e5pressed in pi5el unit+
9
omposed ,y se'eral red LED generates infra-red lig.t and anot.er de'ie :.i.
inludes a sensor detets :.en a person ross t.e line made ,y t.ose t:o de'ies+
O,'iously a miro ontroller is inluded :it. t.e sensor! and t:o groups of su.
de'ies are needed to dedue t.e diretion of a person+ "n fat! t:o lines are needed to
dedue :.et.er a person is entering or lea'ing t.roug. a door+
T.e same priniple as t.e pre'ious one is used for t.e laser-,eam ,ased sensors+
T.e ad'antage of su. systems is a minimum ost ,ut t.is systems annot ,e
KintelligentL means t.at it annot differentiate .umans+ #nd its ot.er dra:,a/s is t.e
fat if t:o persons ross t.e lines in t.e same time! it annot identified t.is t:o
persons+ So in order to limit t.is pro,lem! it is ,etter to plae t.is type of system in
Ksmall doorL ?door t.at only 1ust one person an ross o'er it@+
T.e last :ell-/no:n :ay to ount people is to use t.ermal sensors+ T.is sensor is
a,le to detet t.e 'ariation of temperature in its field so it an identified t.e
temperature of .umans+ T.e pro,lem is t.at it is 'ery sensi,le to small and .ig.
en'ironment temperature+ "n fat! if t.e temperature of t.e am,ient air is too .ig.
t.en t.e sensors :ill ,eome unusa,le ,eause of too lose temperatures ?.uman and
air@+
Ot.er systems e5ist li/e foot step sensor ?sensiti'e arpet or Differential :eig.t
systems@ or sound sensor ,ut t.ey are not so fre;uent ,eause t.ey re;uire .ea'y
modifiation of t.e en'ironment and signifiant amounts of maintenane+ T.ey also
do not pro'ide an easy :ay of determining diretionality of passers-,y+
3nmista/a,ly! t.e most aurate met.od is t.e system ,ased :it. turnstiles
?me.anial ounters@+ T.e aurate is loser of t.e perfetion+ 4ut it annot ,e used in
supermar/ets ,eause it reates a ,arrier to traffi flo: and redue onsidera,ly t.e
Kfeeling of freedomL in a ommerial entre ?t.is ounting met.od o,struts people as
t.ey entered and e5ited@+
7
Illustration 1: Simple scheme of the entire project
Table of contents
Abstract.................................................................................................... 3
Preface...................................................................................................... 5
#/no:ledgements++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ H
Presentation of our work...........................................................................
List of features+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ &
Resear.es of e5isting people ounting system++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ &
T.e different type of sensors+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 9
Table of contents...................................................................................... !!
List of figures........................................................................................... !3
List of tables............................................................................................ !3
! "ntroduction............................................................................................ !
6+6 Topi++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6
6+$ "ntrodution++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6
6+A Pro,lem desription+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6
6+A T.esis struture++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $
# Related $ork......................................................................................... 3
3 Analysis..................................................................................................
A+6 "mage #;uisition++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 9
A+$ Frame Differening++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 7
A+A 4a/ground Estimation++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 66
A+= 4a/ground Su,tration and Segmentation+++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6=
A+=+6 Erosion+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6H
A+=+$ Dilatation+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6>
A+=+A Opening++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6&
A+H Tra/ing++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 67
A+> Counting++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $$
% "mplementation.................................................................................... #5
=+6 Learning of ne: de'elopment tool++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $H
=+$ Video ameras++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $H
=+A Doumentation++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $>
=+A+6 People Counting Model+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $>
=+A+$ 4a/ground Proess Su,system+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $9
=+A+A Segmentation Su,system++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A%
=+A+= Tra/ing and Counting Su,system+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A$
=+A+H 8et:or/ Communiation :it. t.e soft:are+++++++++++++++++++++++++++++++++++++++++++++++++ A=
5 Results.................................................................................................. 3
5 Results.................................................................................................. 3
5 Results.................................................................................................. 3
H+6 Set-up and .ard:are used+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A&
H+$ Tests and e'aluations++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A9
& "mpro'ements...................................................................................... %!
>+6 People ounting system++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ =6
>+$ Ot.er appliations++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ =6
(onclusion............................................................................................ %3
)ibliography........................................................................................... %5
66
List of figures
Simple s.eme of t.e entire pro1et++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6%
Counting people Module+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ &
"mage #;uisition and 0ray-sale++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 9
Frame Differening algorit.m :it. no motion+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 7
Frame Differening algorit.m :it. motion++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 7
4a/ground Estimation - Median Filter ,et:een H frames+++++++++++++++++++++++++++++++++++++++++66
4a/ground Estimation module++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6A
4a/ground Su,tration and Segmentation+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6=
Erosion of a ,inary image :it. a dis/ struturing element++++++++++++++++++++++++++++++++++++++++ 6H
A5A s;uare struture element++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6>
Erosion of a ,inary image :it. a A5A s;uare struturing element+++++++++++++++++++++++++++++ 6>
Dilatation of a ,inary image++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6>
Dilatation of a ,inary image :it. a A5A s;uare struturing element++++++++++++++++++++++++++ 6&
Opening of a ,inary image+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 6&
Opening of a ,inary image :it. a A5A s;uare struturing element++++++++++++++++++++++++++++ 69
Segmentation Module++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 69
Tra/ing :it. a ,lo, analysis++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 67
T:o rules for tou.ing pi5els++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $%
"mportane of t.e lattie used++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $%
Count t.e different o,1ets :it. one 'irtual line++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $$
E5ample of ounting :it. one 'irtual line++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $A
To t.e left t.e Creati'e :e,am and to t.e rig.t t.e infra-red :e,am++++++++++++++++++++ $>
People Counting model+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $&
4a/ground Proess Su,system++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $7
Segmentation Su,system+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A6
Tra/ing and Counting Su,system+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ AA
S.eme of t.e Ser'er program++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ AH
Demonstration 4lueprint++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A9
List of tables
"nteresting ompanies and t.eir :e, sites+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 9
6A
! "ntroduction
T.is setion o'ers t.e introdution and t.e ,a/ground for t.is t.esis+
!.! Topic
T.e main goal of t.is t.esis :as to ma/e different resear. a,out e5isting people
ounting system ,ased on 'ideo amera and ,uild a prototype of t.is using! in a first
time! Matla,-Simulin/ programming tool+ T.en! lin/ t.is system to t.e entralised
data,ase ,uild ,y *orri/+ #nd finally disuss and ompare it :it. ot.er e5isting
people ounting system ,ased on laser ,eam ,uild ,y t.e t:o eletronis students+
!.# "ntroduction
Real-time people flo: estimation an ,e a 'ery useful information for se'eral
appliations li/e seurity or people management su. as pedestrian traffi
management or tourists flo: estimation+ T.e use of 'ideo ameras to tra/ and ount
people inrease onsidera,ly in t.e pas fe: years due to t.e ad'anement of image
proessing algorit.ms and omputersE te.nology+ Se'eral attempts .a'e ,een made to
tra/ people ,ut all t.ose different :ays an ,e lassify in t.ree ategories of different
omple5ity(
Met.ods using region tra/ing features+ To impro'e t.is met.ods some
adding a lassifiation s.eme of pi5els ,ased on olour or te5tures+
Met.ods using $D appearane of .umans ?using different models of .umans@
Met.ods using multiple ameras to ma/e a full AD modelling+
T.e t.ird ategory is more aurate t.an t.e t:o ot.ers ,eause it re,uild preisely
t.e sene ?so it deals in a ,etter :ay t.e olusion pro,lems@ ,ut it is also t.e most
diffiult :it. omple5 algorit.ms+ Some of t.e time! t.is system re;uired a omple5
amera set-up ?ali,ration@ and annot operate in real-time ,eause t.e AD models are
too slo:+ T.is is :.y most of t.e systems used t.e t:o ot.er ategories+
!.3 Problem description
Today! a lot of resear. .a'e ,een pu,lis.ed in order to resol'e su. pro,lem
:.i. is ount people using 'ideo amera+ T.is is not a simple tas/! t.ere are some
situations diffiult to sol'e e'en :it. todayEs omputer speeds ?t.e algorit.m .as to
operate in real-time so it ma/es limits for t.e omple5ity of met.ods for detetion and
tra/ing@+ May,e one of t.e most diffiult! is people olusions+ I.en people entering
or e5iting of t.e field of 'ie: in group! it is 'ery .ard to distinguis. all t.e .umans in
t.is group+
T.an/s to all t.ose resear.! a lot of ompanies propose people ounting system
6
,ased on 'ideo amera+ T.eir system are 'ery aurate and relia,le ,ut are also 'ery
e5pensi'e+ T.e aim of t.is entire pro1et is to ma/e a .eap prototype a,le to ount
people ?o,'iously! it annot ompete in term of auray and performane :it. t.e
ompaniesE system@ and may,e! in a lose future! t.is prototype :ill ,eome a KtrulyL
people ounting system and ould ,e ommeriali-ed+
!.3 Thesis structure
T.is t.esis is organi-ed as follo:s( Setion $ desri,ed all t.e related :or/ done in
people ounting system ,ased on 'ideo amera+ "n t.e setion A! t.e people ounting
and tra/ing algorit.ms are gi'en in details+ Setion = presents t.e .oie of
implementation and t.e ad'anement of t.e prototype+
$
# Related $ork
T.is part :ill 1ust gi'e an idea of all t.e resear. during t.is last ten years to resol'e
t.is ompliated pro,lem :.i. is ount people+ "n fat! t.an/s to t.e fast e'olution of
omputing! it is possi,le to ount people using omputer-'ision e'en if t.e proess is
e5tremely ostly in terms of omputing operations and resoures+
Computer-'ision ,ased people ounting offers an alternati'e to t.ese ot.er
met.ods+ T.e first and ommon pro,lem of all omputer-'ision systems is to separate
people from a ,a/ground sene ?determine t.e foreground and t.e ,a/ground@+
Many met.ods are proposed to resol'e t.is pro,lem+ Se'eral suggest ounting people
systems use multiple ameras ?most of t.e time $@ to .elp :it. t.is proess+
Terada et al+ reates a system t.at an determine people diretion mo'ement and
so ount people as t.ey ross a 'irtual line MTER77N+ T.e ad'antages of t.is met.od is
it a'oids t.e pro,lem of olusion :.en groups of people pass t.roug. t.e ameraEs
field of 'ie:+ To determine t.e diretion of people! a spae-time image is used+
Li/e Terada et al! 4eymer and Fonolige also use stereo-'ision in people tra/ing
M4E*77N+ T.eir system uses ontinuous tra/ing and detetion to .andle people
olusion+ Template ,ased tra/ing is a,le to drop detetion of people as t.ey ,eome
oluded! eliminating false positi'es in tra/ing+ 3sing multiple ameras impro'e t.e
resolution of olusion pro,lem+ 4ut t.e pro,lem is t.e need to .a'e a good
ali,ration of t:o ameras ?:.en AD reonstrution is used@+
Has.imoto et al+ resol'e t.e pro,lem of people ounting using a speiali-ed
imaging system designed ,y t.emsel'es ?using "R sensiti'e eramis! me.anial
.opping parts and "R-transparent lenses@ MH#S7&N+ T.e system uses ,a/ground
su,tration to reate Kt.ermalL images ?plae more or less importane depending of
t.e regions of t.e imageC Region Of "nterest@ t.at are analysed in a seond time+ T.ey
de'eloped an array ,ased system t.at ould ount persons at a rate of 7HO+ So t.eir
system is e5tremely aurate ,ut :it. ertain onditions+ "n order to :or/ in good
onditions! t.e system re;uires a distane of at least 6% m ,et:een passing people to
distinguis. t.em and t.us to ount t.em as t:o separate persons+ T.e system also
s.o:s some pro,lem in ounting :it. large mo'ements from arms and legs+ So t.is
system :ill ,e not so appropriate in ommerial entre ,eause of t.e .ig. density
traffi :.en people entering or e5iting+ "n fat! most of t.e time! person ome in
supermar/ets :it. t.eir family so ma/e a lose group of people :.i. is t.e most
diffiult pro,lem to resol'e for ounting people system+
Tesei et al+ use image segmentation and memory to tra/ people and .andle
olusions MTES7>N+ "n order to .ig.lig.t regions of interests ?,lo,s
A
@! t.e system uses
,a/ground su,tration+ "t onsists to su,trat a referene frame ?,a/ground image
pre'iously ompute@ from t.e urrent frame and t.en t.res.old it ?t.is algorit.m :ill
,e more detailed in t.e analysis section@+ 3sing features su. as ,lo, area! .eig.t and
:idt.! ,ounding ,o5 area! perimeter! mean gray le'el! t.e ,lo,s are tra/ed from
frame to frame+ 4y memori-ing all t.is features o'er time! t.e algorit.m ans resol'e
t.e pro,lem of merging and separating of ,lo,s t.at ours from olusion+ "n fat!
:.en ,lo,s merge during olusion a ne: ,lo, is reated :it. ot.er features ,ut t.e
idea of t.is algorit.m is t.at in t.is ne: ,lo,! it stores t.e ,lo,sE features :.i. form
A # ,lo, is defined as a region of onneted pi5els+ T.e pi5els ontained in a ,lo, must ,e loally
distinguis.a,le from pi5els :.i. are not part of t.e ,lo,+
A
it+ So :.en t.e ,lo,s separate t.emsel'es! t.e algorit.m ans assigned t.eir original
la,els+ T.is system doesnEt resol'e all t.e pro,lems ,ut itEs a good idea and does not
re;uest a lot of omputing operations+
S.io and S/lan/sy try to impro'e t.e ,a/ground segmentation algorit.m ?detet
people olusion@ ,y simulating t.e .uman 'ision more partiularly t.e effet of
pereptual grouping
=
MSH"76N+ First! t.e algorit.m alulates an estimation of t.e
motion from onseuti'e frames ?frames differening is more detailed in t.e analysis
section@ and use t.is data to .elp t.e ,a/ground su,tration algorit.m ?segment
people from t.e ,a/ground@ and try to determine t.e ,oundary ,et:een loser
persons ?:.en olusions ours! ma/e ,oundaries to separate people ,y using frame
differening information@+ T.is segmentation uses a pro,a,ilisti o,1et model :.i.
.as some information li/e :idt.! .eig.t! diretion of motion and a mergingDsplitting
step li/e t.is seen ,efore+ "t :as found t.at using an o,1et model is a good
impro'ement for t.e segmentation and a possi,le :ay to resol'e t.e olusions
pro,lem+ 4ut using pereptual grouping is totally ineffeti'e in some situations li/e!
for e5ample! a group of people mo'ing in t.e same diretion at speed almost e;uals+
#not.er met.od of separation of people from a ,a/ground image is used ,y
S.ofield et al+ MSCH7HN+ #ll t.e ,a/ground segmentation algorit.m is done ,y
simulating a neural net:or/s
H
and uses a dynamially ad1usted spaing algorit.m in
order to sol'e olusions+ 4ut ,eause of t.e redue speed of neural net:or/! t.e
algorit.m only deal :it. ounting people in a speifi image+ T.is paper is 1ust an
approa. of .o: resol'e people ounting ,y using neural net:or/s+ Tra/ing people is
not onsidered+
#s simple and effeti'e approa.! Se5ton et al+ use simplified segmentation
algorit.m MSEP7HN+ T.ey test t.eir system in a Parisian rail:ay station and get error
rate ranging 6O to $%O+ T.eir system uses a ,a/ground su,tration to isolate people
from t.e ,a/ground+ T.e ,a/ground image ?referene frame@ is onstantly updated
to impro'e t.e segmentation and redue t.e effet of lig.ting or en'ironment
modifiation+ T.e tra/ing algorit.m is simply done ,y mat.ing t.e resulting ,lo,s!
gi'en ,y t.e ,a/ground su,tration proess! :it. t.e losest entroids
>
+ Means t.at
t.e tra/ing operation is operated frame to frame and t.e la,el of t.e ,lo, resulting
:it. t.e urrent frame is t.e same t.at t.e ,lo, resulting :it. t.e pre'ious frame
:.i. .as t.e losest entroid+ "n order to a'oid t.e ma1ority of olusions! an
o'er.ead 'ideo amera is used+
Segen and Pingali onentrate on image proessing after segmentation MSE07>N+ #
standard ,a/ground algorit.m is used to determine t.e different regions of interest+
T.en! in ea. of t.ose areas! t.e algorit.m identifies and tra/s features ,et:een
frames+ #ll t.e pat.s of ea. feature is stored and represent t.e motion of person
during all t.e proess+ T.en! ,y using t.ose pat.s! t.e algorit.m an easily determine
.o: many people rossed a 'irtual line and t.e diretion of t.is rossing+ T.is system
does not deal :it. olusion pro,lems and an ,e redue in performane if t.ere is a
lot of persons in t.e field of t.e 'ideo amera+ "n fat! t.e pat.sE data :ill ,e ,ig :.i.
:ill ompliate t.e alulation of intersetion ,et:een t.e line and all t.e pat.s+
Haritaoglu and Fli/ner adopt an anot.er met.od to resol'e t.e pro,lem of real-
time tra/ing of people MH#R%6N+ "n order to segment sil.ouettes from t.e
= Pereptual grouping is a psy.ologial p.enomena and refers to .uman 'isual a,ility to e5trat some
primiti'es image features of important regions of t.e image and group t.em to ma/e a meaningful
struture+ T.e .uman 'isual system an detet many lasses of patterns+
H "n artifiial intelligene! neural net:or/s is modelled li/e a simplified ,rainEs neural net:or/ and
omposed ,y a lot of simple elements :.i. are onneted :it. different :eig.ts+ T.e most diffiult part
is to find t.e ,est :eig.ts for ea. onnetions in order to resol'e t.e urrent pro,lem+ T.ere is a lot of
appliations li/e spee. reognition! fae reognition! image analysis+++ T.e most impressi'e t.ing in
neural net:or/s is t.at some onnetions ,et:een simple elements an e5.i,it a omple5 ,e.a'iour+
> Centroid of a $D s.ape represents its ,aryentre+
=
,a/ground! t.ey .oose to use a ,a/ground su,tration ,ased :it. olour and
intensity of pi5el 'alues+ T.ose informations :ill .elp to lassify all t.e pi5els in t.e
image+ T.ree lassifiations are used ( foreground! ,a/ground and s.ado:+ T.en all
t.e pi5els lassified as foreground ma/e different regions+ #ll t.ese foreground groups
are t.en segmented into indi'idual people ,y using $ different motion onstraints as
temporal and glo,al+ "n order to tra/ t.ese indi'iduals! t.e algorit.m uses an
appearane model ,ased on olour and edge densities+
0ary Conrad and Ri.ard )o.nson,aug. simplify t.e entire people ounting
proess ,y using an o'er.ead amera ?it permits to greatly redue t.e pro,lem of
olusions@ MCO87=N+ To a'oid t.e pro,lem of lig.t modifiation! t.ey use onseuti'e
frames differening instead of using ,a/ground su,tration+ To limit omputation!
t.eir algorit.m redues t.e :or/ing spae in a small :indo: of t.e full sene
perpendiular to t.e flo: traffi+ #t any gi'en time! t.eir algorit.m is a,le to
determine t.e num,er of people in t.e :indo: and t.e diretion of tra'el ,y using t.e
entre of mass in ea. little images of t.e :indo:+ Iit. a ;ui/ and simple algorit.m!
t.ey o,tained 'ery good results and a.ie'ed a 7H!>O auray rate o'er &=76 people+
H
3 Analysis
T.is .apter gi'es an o'er'ie: for t.e resear. of t.is t.esis+ "n t.is setion all t.e
different algorit.ms used to resol'e people ounting :ill ,e e5plained+
Here a ,rief flo:.art of t.e entire people ounting algorit.m ?ea. module of t.e
flo:.art :ill ,e detailed in t.eir respeti'e part@+
&
Illustration 2: Counting people Module
3.! "mage Ac*uisition
"mage are first re;uired in R04$= format at A$%5$=% Pi5els+ T.en t.e image is
pre-proessed ,y transforming it to a gray-sale
&
image+ T.is transformations is easy
and 1ust onsists to ta/e t.e a'erage of t.e t.ree .annels of ea. pi5el+
& # gray-sale image is omposed of one .annel ?intensity@+ Ea. intensity ?gray olour@ is oded in 6
4yte so it an reate $
9
J$H> different olours+
9
Illustration 3: Image Acquisition and Gray-scale
3.# +rame ,ifferencing
7
Illustration 4: rame !ifferencing algorithm with no motion
Illustration 5: rame !ifferencing algorithm with motion
T.is first step of t.e people ounting algorit.m is 'ery important and onsists to
ma/e a Pi5el ,y Pi5el a,solute differene
9
,et:een t:o onseuti'e frames+ T.e result
of t.is operation is a ne: image t.at s.o:s all t.e differenes ,et:een t.is $ frames+
T.e differene image represents a motion detetor+ "f itEs not an empty image ?a full
,la/ image@ t.en t.ere is modifiation ,et:een t.e t:o onseuti'e frames so t.ere is
motion in t.e field of t.e 'ideo amera+ 4ut! due to t.e ;uality of t.e amera used
?noises
7
@ and some automati proessing! e'en if no motion is present during t.e t:o
onseuti'e frames! t.e result of t.e frame differening proess :ill not ,e empty+ 4ut
all t.e Pi5el 'alues resulted ,y t.e differene operation :ill not ,e ,ig+ T.is means t.at
t:o onseuti'e frames are not e5atly t.e same ,ut t.ey are 'ery lose+ E'en if t.ere
is noise in t.e 'ideo amera! t.e KnoisyL Pi5els .a'e lose gray-sale 'alue+ #
t.res.old
6%
must ,e introdue to deide of t.e e5istene of motion+
T:o different :ays of applying t.res.old an ,e done(
T.e first one is a t.res.old ,et:een t.e differene of ea. Pi5el! all t.e
differene smaller t.an a ertain t.res.old is onsidering li/e a null Pi5el
?,la/ Pi5el@+ #n anot.er to onsider t.at is to ta/e t.e ma5imum gray-sale
'alue of t.e differene image and ompare to t.e t.res.old+ "f t.is ma5imum
'alue is greater t.an t.e t.res.old t.en t.e onlusion is t.at t.ere is motion
in t.e field of t.e 'ideo amera+
T.e seond :ay! is to ma/e a t.res.old in t.e full differene image! if t.e sum
of all Pi5elEs 'alues is smaller t.an a ertain t.res.old t.en t.e differene
image is onsider li/e an empty image+ T.e pro,lem of t.is met.od is t.e
diffiulty to find t.e t.res.old ,eause it depends of t.e resolution of t.e
image ?t.e num,er of Pi5els in t.e image@+
T.is algorit.m an ,e use also to detet t.e regions of interest
66
in t.e image ?t.e
regions :.ere t.ere is motion@+
9 #n a,solute differene of t:o pi5els onsists to ma/e a differene ,et:een t.e t:o intensity 'alues of t.e
pi5el and ta/e t.e a,solute 'alue+ "t orresponds at t.e distane ,et:een t.is t:o 'alues+
7 8oise is a random! usually un:anted! flutuation of pi5els 'alue in a image due to t.e ;uality of t.e
input de'ie ?sanner! digital amera+++@+
6% # t.res.old is a :ay to filter informations in t:o ases ?t.e 'alues :.i. are smaller to t.is t.res.old
and t.e 'alues :.i. are greater+
66 Region of interests ?RO"@ is a :ay to redue omputing operations ,y gi'ing more and less importane
to t.e different regions of t.e image+
6%
3.3 )ackground -stimation
4efore t.e system attempts to loate person on t.e sene! it must learn t.is sene in
order to detet motions ?,ig 'ariation of t.e sene@+ T.e ,a/ground estimation
algorit.m ma/es a referene image :.i. represent t.e ,a/ground part of t.e sene
'ie:+ T.e ,a/ground image is fundamental to detet mo'ing o,1ets and tra/ing
t.em! and :ill ,e used to separate t.e ,a/ground and t.e foreground+ To a'oid t.e
ma5imum effet of t.e noises Pi5el! a median filter is applying ,et:een fi'e
onseuti'e frames in order to o,tain a good estimate of t.e model+ For ea. pi5el! t.e
median filter ta/es t.eir 'alue in t.e H frames! t.en sort t.is 'alues and ta/e t.e
median+ T.is proess is repeated for ea. pi5el and t.en :ill ma/e a ne: median
image ?t.e ,a/ground image@+
T.e ,a/ground estimation .as to ,e dynami! means t.at t.e ,a/ground image
must ,e update+ T.is idea is 'ery important in order to ma/e a good people ounting
algorit.m+ For e5ample! if t.e people ounting system is plaed in t.e entrane of one
,uilding! some little and slo:ly modifiations our during all t.e day and ans
parasite t.e people ounting algorit.m ?more partiularly t.e ,a/ground differene
algorit.m@+ "n fat! during t.e day! t.e lig.t intensity of t.e sun .anges or !for some
reasons! some o,1ets an ,e remo'ed or added in t.e :or/ sene+ So! if t.e
,a/ground estimation is ne'er reompute! t.e algorit.m :ill al:ays detet 'ariations+
4ut ,efore laun.ing t.e ,a/ground estimation! t.e algorit.m must ,e ertain
t.at t.ere isnEt motion in t.e field of t.e 'ideo amera during t.is moment+ "f t.ere is
no motion! t.e referene image ?,a/ground image@ is updated :.ereas if t.ere is
66
Illustration 6: "ac#ground $stimation - Median ilter between % frames
motion t.e ,a/ground image is not updated and :ill ,e try to ,e updated in t.e ne5t
,a/ground estimation+
For e5ample! ta/e onsider of a gray-sale image format at A5A Pi5els+ Here H
matries :.i. represent t.e H onseuti'e frames(
4y sorting all t.e 'alues of t.e H frames for ea. Pi5el! 7 'etors are reated
T.e 4a/ground image is omposed of median 'alues for ea. 'etors Pii
6$
6A
Illustration 7: "ac#ground $stimation module
3.% )ackground .ubtraction and .egmentation
One t.e ,a/ground estimation is ompleted! t.e algorit.m used to separate t.e
foreground and t.e ,a/ground ?t.e foreground 1ust represent t.e ,ig 'ariation
,et:een t.e urrent ,a/ground and t.e image of t.e 'ideo amera@+ T.is proess is
'ery lose of t.e frame differening algorit.m ,ut not e5atly t.e same+ "t is 'ery
diffiult to separate t.e foreground and t.e ,a/ground of t.e image ,y using t.e
frame differening proess ?differene ,et:een t:o onseuti'e frames@ ause t.at it
.as tendeny to 1ust .ig.lig.t t.e edges of o,1ets in t.e foreground so t.e image
analysis :ill ,e more diffiult after:ards+ #nd an anot.er reason to not use frame
differening algorit.m is t.at it is totally ineffeti'e to detet stationary o,1ets of t.e
foreground+ Conse;uently! people ounting system uses t.e ,a/ground su,tration
step to detet o,1ets of t.e foreground and frame differening to detet motion+
T.e first step of t.is algorit.m! is to ma/e a ne: image ?,a/ground su,tration@
resulting from t.e a,solute differene ,et:een t.e urrent ,a/ground and t.e urrent
frame+ T.e ,a/ground su,tration image is a gray-sale image so it .as to ,e
transformed in a ,inary image to ma/e t.e segmented image ?i+e+ separation of t.e
foreground and t.e ,a/ground@+ To transform a gray-sale image ?$HH 'alues@ in a
,inary image ?$ 'alues@ a t.res.old must interfere+ #ll t.e Pi5elEs 'alues smaller t.an
t.is t.res.old is 'ie:ed as t.e ,a/ground of t.e sene ? 'alue %@+ T.is :ill eliminate a
lot of KnoisyL pi5els :.i. .a'e! t.e most of t.e times! a lose 'alue and :ill eliminate
too some of t.e Pi5els :.i. represents t.e s.ado:s ma/e ,y t.e mo'ing o,1ets+ "n
6=
Illustration : "ac#ground Subtraction and Segmentation
fat! in a gray-sale image! t.e s.ado: of an o,1et! most of t.e time! doesnEt .ange a
lot t.e feature ?olour@ of t.e Pi5el+ So t.is s.ado:! in t.e ,a/ground su,tration! .as
a small 'alue+
4ut t.is t.res.old :ill not eliminate all t.e K:rongL Pi5els ?Pi5els :.i. are in t.e
,a/ground of t.e sene@+ T.e seond step of t.is algorit.m! is to ma/e a morp.ology
opening in t.e ,inary image ?t.e segmented one@+ T.e interest of t.is morp.ologial
operation is t.at it ans remo'e small o,1ets reated ,y noise+ T.e morp.ology
opening is omposed of t:o ,asis operator in t.e area of mat.ematial morp.ology
:it. t.e same struture element ?Erosion and Dilatation@+ #t first sig.t! ma/e an
erosion and t.en a dilatation :it. t.e same struture element ans ,e ta/en ,y an
identity+ 4ut itEs not t.e ase! an erosion is apa,le of deleting some little o,1ets! so
ma/e a dilatation after :ill not ma/e t.is o,1ets reappeared so t.e result of a
morp.ology opening on a ,inary image is not e5atly t.e same t.an t.is original
image+
4a/ground su,tration reates images :it. ,lo,s t.at orrespond to foreground
o,1ets+
8o:! " :ill ma/e a small presentation of t.e different mat.ematial morp.ology
operations used and illustrate it ,y some e5amples+
3.%.! -rosion
T.e first morp.ologial operation used is t.e erosion+ "tEs a ,asi operation and its
primary feature is to erode a:ay t.e ,oundaries of t.e different foreground regions+
T.us t.is foreground o,1ets :ill ,eome smaller ?little of t.em :ill totally ,e
'anis.ed@ and .oles in o,1ets :ill ,e ,igger+
Let P ,e a su,set of E and let 4 denote t.e struture element+ T.e morp.ologial
erosion is defined ,y(
"n outline! all t.e pi5els of t.e foreground o,1et :.i. an totally ontain t.e
struture element 4 :ill ,e ontained in t.e eroded o,1et+ For e5ample! ta/e onsider
of a A5A s;uare struture element .a'ing its morp.ologial entre t.e same as t.e
geometrial entre+ "t is as follo:s(
6H
Illustration !: $rosion of a binary image with a dis# structuring element
To ompute a ,inary erosion! all t.e Pi5els of t.e foreground must ,e proess+ For
ea. pi5el of t.e foreground! t.e algorit.m puts t.e struture element ?t.e entre of
t.e struture element mat.es :it. t.e pi5el@ and tests if t.e struture element is
ompletely ontained in t.e foreground+ "f it is not! t.e urrent pi5el :ill ,e onsidered
li/e t.e ,a/ground and on t.e ontrary! if it is! t.e urrent pi5el :ill ,e ontained in
t.e eroded foreground+
3.%.# ,ilatation
Li/e t.e erosion! t.e dilatation is t.e seond ,asi operation and its primary
feature is to dilate t.e ,oundaries of t.e different foreground regions+ T.us t.is
foreground o,1ets :ill ,eome ,igger and .oles in o,1ets :ill ,e smaller ?little of
t.em :ill totally disappear@+
6>
Illustration 1": &'& square structure element
Illustration 11: $rosion of a binary image with a &'& square structuring element
Illustration 12: !ilatation of a binary image
"n outline! all t.e pi5els of t.e ,a/ground :.i. an tou. t.e foreground regions! ,y
putting on it t.e struture element 4! :ill ,e ontained in t.e dilated o,1et+
For e5ample! ta/e onsider of a A5A s;uare struture element .a'ing its morp.ologial
entre t.e same as t.e geometrial entre ?see illustration &@+ To ompute a ,inary
dilatation! all t.e Pi5els of t.e ,a/ground must ,e proess+ For ea. pi5el of t.e
,a/ground! t.e algorit.m puts t.e struture element ?t.e entre of t.e struture
element mat.es :it. t.e pi5el@ and tests if t.e struture element is in tou. :it. at
least one pi5el of t.e foreground+ "f it is! t.e urrent pi5el :ill ,e onsidered li/e t.e
foreground and on t.e ontrary! if it is not! t.e pi5el :ill stay a ,a/ground pi5el+
3.%.3 /pening
T.e opening operation is a om,ination of t.e t:o ,asis operation ?Erosion and
Dilatation@+ "tEs t.e dilation of t.e erosion and its primary feature is too eliminate
noise ?small o,1ets@+ T.is operation :ill separate ,lo,s :.i. are lin/ed :it. a small
layer ?see fig+66@+
"n outline! #ll pi5els :.i. an ,e o'ered ,y t.e struturing element :it. t.e
struturing element ,eing entirely :it.in t.e foreground region :ill ,e preser'ed+
Ho:e'er! all foreground pi5els :.i. annot ,e rea.ed ,y t.e struturing element
:it.out parts of it mo'ing out of t.e foreground region :ill ,e eroded a:ay+ #s
6&
Illustration 13: !ilatation of a binary image with a &'& square structuring element
Illustration 14: (pening of a binary image
follo:s! an e5ample of an opening operation :it. a A5A s;uare struturing element+
69
Illustration 15: (pening of a binary image with a &'& square structuring element
Illustration 16: Segmentation Module
3.5 Tracking
One t.e segmentation is done! an anot.er image proessing must ,e laun.ed in
t.e ,inary image+ "n fat! in order to tra/ o,1ets! t.e first step to do is to identify all
t.e o,1ets on t.e sene and alulate all t.eir features+ T.is proess is alled a 4lo,
#nalysis
6$
+
"t onsists to analyse t.e ,inary image! find all t.e ,lo,s present and ompute
statistis for ea. one+ Typially! t.e ,lo,s features usually alulated are area
?num,er of pi5els :.i. ompose t.e ,lo,@! perimeter! loation and ,lo, s.ape+ "n t.is
proess! it is possi,le to filter t.e different ,lo,s ,y t.eir features+ For e5ample! if t.e
sear.ing ,lo,s .a'e to .a'e a minimum area! some ,lo,s an ,e eliminate :it. t.is
algorit.m if t.ey donEt respet t.is onstraint ?it permits to limit t.e num,er of ,lo,s!
t.us redue t.e omputing operations@+ T:o different :ays of onnetion an ,e
defined in t.e ,lo, analysis algorit.m depending of t.e appliation+ One onsists to
ta/e t.e ad1aent pi5els along t.e 'ertial and t.e .ori-ontal as tou.ing pi5els and t.e
ot.er ,y inluding diagonally ad1aent pi5els ?illustration )*@+
6$ 4lo, analysis is t.e identifiation and study of t.e different regions of onneted pi5els in t.e image+ T.e
algorit.m diserns pi5els ,y t.eir 'alue and lassifies t.em in t:o ategories( foreground ?typially t.e
non--ero 'alue@ and ,a/ground ?pi5els :it. a -ero 'alue@+
67
Illustration 17: +rac#ing with a blob analysis
Setting t.e rules for tou.ing pi5els is important ,eause t.e outome of t.e ,lo,
analysis an ,e different+ For e5ample ?illustration ),@! t.e group of pi5els :ould ,e
onsidered as one ,lo, if t.e algorit.m used t.e lattie :it. eig.t onnetions and as
t:o different ,lo,s if it used t.e ot.er lattie+
T.e performane of t.e ,lo, analysis algorit.m depends totally of t.e ;uality of t.e
segmentation+ Iit. a ,ad segmentation! t.e ,lo, analysis an detet some not
interesting ,lo,s or :orse an merge some different ,lo,s due to lig.ting ondition or
noise in t.e image+
I.ile all t.e ,ounding ,o5
6A
are reated! t.e tra/ing proess put a la,el
6=
for ea.
one in order to memori-e t.e different o,1ets present to t.e sene+ 4ut! too /eep t.e
6A # ,ounding ,o5 is a t.e smallest retangular s.ape :.i. surround ,lo,s+
6= # la,el is an identifiation ?for e5ample a num,er@ gi'en to ea. o,1ets+
$%
Illustration 1: +wo rules for touching pi'els
Illustration 1!: Importance of the lattice used
good la,els for ea. ,lo,s in a se;uene of frames! t.e algorit.m must memori-e some
features of ea. ,lo,+ T.e most simple :ay to do t.is is to memori-e t.e position of
t.e entroid! ,ounding ,o5! :idt. and .eig.t for ea. ,lo, and t.en in t.e urrent
frame try to find t.e ,lo, :.i. is t.e ,est mat.ing+ 4ut sometimes! it is impossi,le
to find a good mat.ing ,lo,! for e5ample in t.e ase of merging ,lo,s+ So :.en t.is
ase is deteted! t.e algorit.m .as to determine :it. :.i. ,lo,s t.is ne: merging
,lo, :as reated+ Li/e t.is! t.e algorit.m does not lose informations a,out ,lo,s in
t.e sene and t.is is fundamental for t.e ounting proess+
$6
3.& (ounting
"n t.is paragrap.! " :ill 1ust e5plain t.e ounting proess :.en an o'er.ead 'ideo
amera is used ,eause my model :as made for t.is situation ?t.e different .oies are
e5plained in t.e implementation section@+ 4ut all t.e pre'ious algorit.ms are a,le to
:or/ in any ot.er situations ?it is 1ust t.e ounting :.i. depends of t.ese different
situations@+
T.e ounting proess onsists to determine t.e diretion of ,lo,s :.i. ross a
'irtual line in order to inrement t.e good ounter+ T.ere is t:o different :ay to ma/e
t.is proess in t.e ase of using an o'er.ead amera+
T.e first one is to use t:o 'irtual lines :.i. represent t.e Entering ?"8@ and t.e E5it
?O3T@+ "f a ,lo, ross o'er ompletely a 'irtual line! in t.e good diretion! t.en t.e
algorit.m inrements t.e orrespondent ounter ?if t.e urrent ,lo, is a merging ,lo,!
t.e algorit.m loo/s .o: many ,lo,s omposed it and inrements :it. t.e same
num,er@+ 4eause t.e algorit.m memori-es t.e features of ea. ,lo,s it an easily
dedue t.e diretion of t.e motion ?for e5ample! ,y loo/ing t.e position of t.e
entroid@+
T.e seond :ay is to use 1ust one 'irtual line :.i. demarate learly t.e "8 area and
t.e O3T area+ T.is met.od :as preferred for t.e final prototype ,eause of its
simpliity+ To ount people! t.e algorit.m 1ust loo/ t.e position of t.e ,ottom segment
of ea. ,ounding ,o5 in t:o onseuti'e frames+ "f! in frame T! t.e segment is under
t.e 'irtual line and! in t.e ne5t frame! t.e same segment is upper t.en t.e algorit.m
$$
Illustration 2": Count the different objects with one -irtual line
inrement its "8 ount 'alue+ 4ut it is really important to use also t.e ,ottom segment
of ,ounding ,o5 for t.e O3T ount ?or else! t.e algorit.m an ma/e error of ounting
li/e ount se'eral times a person "8 :it.out ount .im O3TC for e5ample a person
:.i. ma/e small round trip ne5t to t.e "8 area@+ T.e ot.er :ay to ount using one
'irtual line is to loo/ t.e position of t.e entroid of ea. ,lo,s+
$A
Illustration 21: $'ample of counting with one -irtual line
% "mplementation
T.is setion o'ers t.e different .oies ta/en for t.e people ounting systemEs
implementation and t.e ad'anement of t.e prototype+
%.! Learning of new de'elopment tool
My super'isor ad'ise me to use t.e Matla,-Simulin/ programming tool ,eause .e
/ne: t.is te.nology and some ,asis :or/s .a'e ,een done onerning t.e tra/ing
of o,1ets ,y t.e Mat.:or/sE team+ So ,efore starting t.e programming! " tried to
learn t.is te.nology and found 'ery good doumentation and tutorials in t.e offiial
:e,site of Mat.:or/s K .ttp(DD:::+mat.:or/s+omD K+ Learning of Matla, language
:as not 'ery .ard for me ,eause it loo/s li/e C language programming and during my
H-year ollege! " used to program :it. su. language+ 4ut " enounter some pro,lems
:it. t.e Matla,-Simulin/ te.nology ,eause it is a totally different approa. to ma/e
programs ?" .a'enEt any /no:ing a,out su. system@+ Simulin/ is a tool for modelling!
simulating and analysing multi-domain dynami system and is :idely used in digital
signal proessing and ontrol t.eory+ "t furnis.ed a grap.ial ,lo/ en'ironment and a
ustomi-a,le set of ,lo/ li,raries and ,ased on dataflo: programming+ T.e
ad'antage of su. te.nology is t.at it proposed 'ery good tools to see in real-time
different ,e.a'iours of t.e simulation :.en some modifiations of situations or
parameters our+ T.e ot.er ad'antage is Simulin/ an generate po:erful C ode+
#fter t.is learning proess! " study different models of Mat.:or/s more
partiularly t:o of t.em ?'iptrackpeople0win.mdl and 'iptraffic0win.mdl@+
T.e first one is a,le to tra/ multiple mo'ing o,1ets in t.e field of t.e amera ?detet
and memori-e mo'ing o,1ets frame to frame@ and t.e seond one ount t.e num,er
of ars ?in fat mo'ing o,1ets@ in .ig.:ay+ " did not enounter pro,lems to
understand t.ose t:o models t.an/s to a good doumentation and a lear ode+ 4ut "
anEt use t.em for my appliation! " .a'e 1ust ta/en some ideas+ For e5ample! in t.e
tra/ing people model! t.ere is not ,a/ground estimation! t.e first frame of t.e 'ideo
is ta/en as a ,a/ground and all t.e segmentation proess is done :it. auto-t.res.old
+ #nd in t.e ars ounting model! t.e algorit.m 1ust ount t.e num,er of mo'ing
o,1ets :.i. are under a 'irtual line ?in fat! t.e diretion of t.e ars in one side of
t.e .ig.:ay is t.e same so no need to detet t.e arEs rossing@+ T.ose t:o models
:ere tested :it. t.e t:o ameras :.i. " used and t.e result :as not 'ery onlusi'e+
T.e segmentation proess :as 'ery ,ad due to t.e automati t.res.old and t.e
different automati proesses of t.e ameras! as a onse;uene t.e tra/ing proess
:asnEt good too+
%.# 1ideo cameras
During my :or/ plaement! " used t:o types of 'ideo amera o:ned ,y my
super'isor+ T.e first one :as a Creati'e Li'eQ Cam Voies :e,am a,le to a;uire
'ideo at 6$9%57>% Pi5els ?6!A Mpi5els@ and ost appro5imately 6%%R+ T.e main
$H
pro,lem :it. t.is 'ideo amera :as all t.eir automati proesses :.i. annot ,e
turned off su. as .ig.lig.t ompensation! ,a/ground noise remo'al and
en'ironmental side .atter remo'al+ #ll t.ose automati proesses ma/e t.e :or/
.arder partiularly for t.e segmentation algorit.m+ T.e seond :e,am :as a
V"M"CRO 3S4 PC Camera A%65 ?infra-red amera so 'ery sensiti'e to lig.t@ a,le to
a;uire 'ideo at A$%5$=% pi5els and ost appro5imately $%R+ T.e result :it. t.is
amera :as learly ,etter ,eause it .as less automati proesses t.an t.e ot.er one
?1ust .ig.lig.t ompensation proess@+
4ut to a'oid t.e effet of t.e automati proesses and ,eause of no'ie s/ill in
Simulin/! " .oose to mount t.e amera 'ertially li/e Rossi and 4o--oli MROS7=N+ "t is
true t.at using an o'er.ead amera limits t.e diretions of people entering in t.e
sene ?top and ,ottom side of t.e image@ ,ut remo'e totally t.e olusion pro,lem so
ma/e t.e pro,lem easier to resol'e+
%.3 ,ocumentation
T.is part desri,es t.e implementation in Simulin/ of t.e different algorit.ms used
to ma/e a people ounting system+
%.3.! People (ounting 2odel
T.e people Counting Model is omposed of four su,system ?4a/ground Proess!
Segmentation! Tra/ing and Counting and Display Results@+ #ll t.ose su,systems :ill
,e desri,e later+ T.e model uses t.ree glo,al 'aria,les :.i. are t.e CountS"n
?represents t.e num,er of people enter@! t.e CountSOut ?represents t.e num,er of
people e5it@ and t.e Connetion ?represent t.e so/et onnetion using TCPD"P
protool ,et:een t.e omputer :.i. e5eute t.e model and t.e main omputer of t.e
,uildingC t.is onnetion is done at t.e laun.ing of t.e program@+ T.e Edit
Parameters ,lo/ allo: to fi5 different parameter su. as t.e position of t.e 'irtual
line or t.e minimum and ma5imum area for ea. ,lo,s deteted+ T.e input of t.is
model is t.e signal of t.e 'ideo amera! t.e yle is repeated for ea. urrent frame
$>
Illustration 22: +o the left the Creati-e webcam and to the right the infra-red
webcam
a;uired at A$%5$=% in R04$= format+ M40N and MF0N are 1ust flags ?li/e Label and
3oto in ot.er programming language@ and represent respeti'ely t.e e5istene of t.e
,a/ground ?,oolean@ and t.e presene of mo'ing o,1ets in t.e foreground ?,oolean@+
$&
Illustration 23: .eople Counting model
%.3.# )ackground Process .ubsystem
T.e 4a/ground Proess is omposed of t:o main ,lo/s :.i. are t.e motion
Detetor 4lo/ and t.e 4a/ground Estimation 4lo/+ T.e input of t.is su,system is a
olour image ?urrent frame of t.e 'ideo amera@+ T.e t.ree outputs are t.e e5istene
of ,a/ground! t.e ,a/ground image and t.e urrent frame transformed in intensity+
T.e Pulse 0enerator is a ,lo/ :.i. allo:ed to reate a pulsation ?in t.is ase t.e
amplitude of t.e pulsation is 6 and its lengt. is 'ery small@ e'ery ea. period ?t.e time
of t.is period is ustomi-a,le@+
T.e Motion Detetor ,lo/ is ena,led e'ery ea. period :.ile t.e Pulse 0enerator
4lo/ reate a pulsation and determine t.e e5istene of motion during t.e urrent
sene represented ,y its output ?,oolean@+ "ts input is t.e urrent frame of t.e 'ideo
transformed in intensity+ To detet t.e motion t.is ,lo/ ma/e an a,solute differene
,et:een t:o onseuti'e frames ?urrent frame and pre'ious frame@ and find t.e
ma5imum 'alue of t.is frame differening operation+ "f t.ere is ,ig modifiation
,et:een t.e t:o frames ?i+e+( t.e ma5imum 'alue is greater t.an a t.res.old@ t.en
t.ere is motion+ 4ut in order to .a'e t.e pre'ious frame! a ,lo/ Memory is needed+
T.is ,lo/ allo: to memori-e t.e last 'alue of its input ?i+e+ T.e last 'alue of t.e
urrent frame is 1ust its pre'ious frame@+
T.e ,a/ground Estimation ,lo/ is ena,led :.en t.ere is no motion on t.e field
of t.e amera and :.en t.e Motion Detetor is ena,led ?i+e+( :.en t.e Pulse
0enerator 4lo/ reates a pulsation@ and ompute t.e ,a/ground image+ Li/e t.is t.e
,a/ground :ill ,e updated e'ery ea. period if t.ere is no motion in t.e field of t.e
amera during t.is moment+ "ts input is t.e urrent frame of t.e 'ideo transformed in
intensity and its t:o outputs are respeti'ely a ,oolean :.i. represent t.e e5istene
of t.e ,a/ground and t.e ,a/ground image+ To ompute t.e ,a/ground image t.e
algorit.m merge t.ree onseuti'e frames t.an/s to t.e 4uffer 4lo/ and ta/e t.e
median filter+ #nd to /no: if t.e ,a/ground e5ists ?t.e ,a/ground is not a ,la/
image@! t.e algorit.m 1ust loo/ t.e ma5imum 'alue of t.is image and ompare to t.e
-ero 'alue ?if t.e ma5imum 'alue is -ero t.en t.e ,a/ground does not e5ist@+
$9
$7
Illustration 24: "ac#ground .rocess Subsystem
%.3.3 .egmentation .ubsystem
T.e Segmentation su,system is ena,led :.en t.e ,a/ground image e5ists and
ma/e a ,inary image :.i. represent t.e foreground and t.e ,a/ground+ T:o inputs
are needed to ma/e t.e segmentation! one is t.e urrent frame transformed in
intensity and t.e ot.er is t.e last updated ,a/ground image+ "ts outputs are
respeti'ely t.e segmented image ?,inary image@ and a ,oolean :.i. represent t.e
e5istene of mo'ing o,1ets ?presene of foreground in t.is ,inary image@+ To reate
t.e ,inary image! t.e algorit.m ma/e an a,solute differene ,et:een t.e t:o inputs
?,a/ground image and urrent frame@+ T.en t:o ases are possi,le! t.ere is a lot of
non--ero pi5els ,ut loser of -ero means t.at t.ere is no mo'ing o,1ets ?t.is small
'ariations ,et:een t.e urrent frame and t.e ,a/ground is due to t.e ;uality of t.e
'ideo amera@+ #nd t.e seond ase is t.e presene of ,ig 'ariations means t.at t.ere
is mo'ing o,1ets in t.e sene+ T.is e5plains t.e use of t.e S:it. 4lo/ :.i. almost
e;ui'alent to a "F THE8 ELSE in programming+ T.e S:it. ,lo/ needs t.ree inputs!
t.e input of t.e middle is t.e 'alue ompare to a t.res.old ?.ere t.e sum of ea. 'alue
pi5els of t.e frame differening image@+ "f t.is test is positi'e t.en t.e algorit.m :ill
ta/e t.e first input :ay else it :ill ta/e t.e t.ird input :ay+ More learly! if t.e sum of
all 'alue pi5els is smaller t.an a t.res.old ?no mo'ing o,1ets in t.e sene@ t.en t.e
frame differening image :ill ,e transformed in a empty image ?,la/ image@ else t.e
algorit.m apply a opening operation to eliminate little ,lo,s due to noise+ Finally to
ma/e a ,inary image of t.is frame differening image! t.e Data Type Con'ersion 4lo/
is used+
A%
A6
Illustration 25: Segmentation Subsystem
%.3.% Tracking and (ounting .ubsystem
T.e Tra/ing and Counting Su,system is ena,led :.en t.ere is mo'ing o,1ets in
t.e urrent frame ?i+e+( t.e segmented image is not an empty image@+ "ts input is t.e
,inary image ?segmented image@ and its outputs are 1ust t.e different positions of
,ounding ,o5 for t.e display+ T.ree ases are distinguis.! t.e ,ounding ,o5 :ill ,e
displayed in 4lue t.e moment :.en it go t.roug. t.e 'irtual line and ount as a "8 or
O3T! displayed in Magenta if t.e ,ounding ,o5 uts t.e 'irtual line and else displayed
in 0reen+ T.e algorit.m laun. a 4lo, analysis to detet all t.e ,lo,s .e/ing t.e
onstraints of minimum and ma5imum areas+ To tra/ t.e different ,lo,s t.e
algorit.m need to memori-e t.e position of ea. ,ounding ,o5 and entroid+ "n order
to /no: t.e diretion of t.e motion of ea. ,lo,! t.e algorit.m ompare t.e position
of t.e ,ounding ,o5 in t.e urrent frame :it. t.e position of t.e ,ounding ,o5 in t.e
pre'ious frame+ # "8 is deteted if and only if! t.e position of t.e ,ottom segment of
t.e ,ounding ,o5 in t.e pre'ious frame is under t.e 'irtual line and is upper t.e
'irtual line in t.e urrent frame+ T.e same KmirrorL reasoning is done for t.e O3T
detetion+ I.en a "8 or O3T is deteted t.en t.e algorit.m inrement t.e good
ounter and send data to t.e main omputer of t.e ,uilding t.an/s to t.e Connetion
glo,al 'aria,le ?t.is proess is done in t.e #dd and Send Count 4lo/@+
A$
AA
Illustration 26: +rac#ing and Counting Subsystem
%.3.5 4etwork (ommunication with the software
(lient
To ma/e TCPD"P onnetion :it. my Simulin/ model! " use a totally free tool,o5
e'en for ommerial use named TCPD3DPD"P Tool,o5 $+%+H and oded in C ,y Peter
RydesTter+ T.is tool,o5 an ,e found in t.e Matla, Central File E5.ange :e,site
.ttp(DD:::+mat.:or/s+omDmatla,entralDfilee5.angeDloadFile+doUo,1et"dJA=HV
o,1etTypeJFile+ T.is tool,o5 is 'ery easy to use and allo: a lot of funtionalities
neessary for t.e appliation+ Here are some e5ample of ode used in my model(
#s seen ,efore! t.e Connetion glo,al 'aria,le is initiali-ed at t.e laun.ing of
t.e program+ "ts initial 'alue is Kpnet?EtponnetE!E67$+6>9+%+6E!69%%%@CL+ T.is
line of ode opens a so/et :it. a TCPD"P onnetion in t.e port 69%%% and to
t.e omputer :.i. .as t.e loal "P address E67$+6>9+%+6E+
"n order to send data :.en a rossing is deteted! an em,edded Matla,
funtion 4lo/ is used+ T.is ,lo/ allo: to add Matla, ode in Simulin/
model+ T.e ode for t.e "8 ount is Kpnet?u!EprintfE!EVideoCam6(6E@CL+ EuE is t.e
name of t.e so/et and EVideoCam6(6E is t.e sent message+ VideoCam6
represents t.e identifying of t.e sensor ? t.is is useful for t.e update of t.e
entralised data,ase@ and t.e E6E after t.e separator E(E represent an "8 ount
?E-6E is t.e O3T ount@+
.er'er
T.e ser'er program :as oded in )#V# using t.e Elipse en'ironment ?free:are@
,eause it used some ot.er program oded ,y *orri/ also in )#V#+ T.e ser'er
program .as to ,e laun.ed in t.e main omputer of ea. ,uilding! all t.e sensors ?or
data logger depending of t.e type of t.e sensors@ are onneted ,y TCPD"P protool to
t.is omputer+ "n fat! t.e eletroni students de'elop a program :.i. an deal :it.
se'eral sensors ?1ust one onnetion@ :.ereas my model deal :it. only one amera ?it
is not possi,le to laun. t:o people ounting models in one omputer due to
performane@+ So! ,eause t.e program must deal :it. a lot of sensors! its ar.iteture
loo/s li/e a Ser'er C.at ?multi-t.read lient onnetions@+ For ea. sensor ?omputer@!
t.e program reate one Thread(lient in order to ommuniate ,eause different
omputer an send data to t.e ser'er in t.e same time+ T.e algorit.m .as an anot.er
t.read alled Thread5ueue+ T.is t.read is laun.ed all t.e seonds and is used for
updating t.e entralised data,ase :it. all t.e informations sent ,y t.e sensors during
t.is seond+
"n order to :or/ orretly! t.is program uses a onfiguration file spring6
beans.xml :.ere se'eral informations are stored li/e t.e "P address of t.e
entralised data,ase ?in order to ommuniate :it. it@ and t.e identifiation of t.e
,uilding :.ere t.e program is laun.ed ?ea. ,uilding .as a uni;ue identifiation in
t.e entrali-e data,aseC t.is part is more detailed in t.e *orri/ report@+ #t t.e
,eginning! t.e program reads t.e onfiguration file and reates an o,1et 5ueue :it.
t.e urrent "uildingI!+ T.is o,1et is onneted :it. t.e entralised data,ase and gets
,a/ all t.e informations of t.e urrent ,uilding ?Doors and Sensors@ t.an/s to t.e
funtion of t.e Remote .er'ice O,1et named getAll,oors)y)uilding7
"uildingI!8+ I.en a lient ?a sensor@ send a message to t.e Ser'er! T.e
orresponding Thread(lient :ill analyse t.e message and update t.e different
ounts ?"8 and O3T for ea. sensor@ :.i. are stored in t.e o,1et Wueue ?update
:it. t.e orresponding SensorI!@+ T.e Thread5ueue ?:.i. is laun.ed all t.e
A=
seonds@ read t.ose different ounts and ma/e a onnetion :it. entralised data,ase
in order to update t.e orresponding ,uilding+ "f t.e updating operation is going :ell!
t.e Thread5ueue resets all t.e ount of t.e o,1et 5ueue+
"f you :ant to understand more! " ad'ise you to read t.e report of *orri/ :.i.
detailed t.e ar.iteture of t.e entralised data,ase+
AH
Illustration 27: Scheme of the Ser-er program
5 Results
"t :as impossi,le for me to test my model :it. a truly sientifi approa. ,eause
of t.e s.ort time and la/ of materials+ To measure t.e auray of my model! " :ill
.a'e to plae t.e 'ideo amera in t.e entrane of t.e uni'ersity during one full day
lin/ed to a omputer :.i. laun. t.e people ounting algorit.m+ Furt.ermore! t.e
'ideo amera is a 3S4 :e,am so need speial 3S4 a,le due to t.e limit of & meters
of 3S4 protool+ T.en! to ompare t.e result! someone :ill .a'e to ount people
manually during one full day+ 4ut one demonstration of our entire system :as do:n in
t.e end of our :or/ plaement to s.o: t.e ad'anement to t.e lient+ "t :as at t.is
moment :.en i ould ma/e a real test of my program+
5.! .et6up and hardware used
During t.is demonstration! " use t.e seond amera ?V"M"CRO 3S4 amera@
lin/ed to my laptop :.i. laun. t.e Simulin/ model+ Li/e seen in t.e
implementation section! t.e 'ideo amera :as plaed in o'er.ead to limit olusions
and too ,ig 'ariation of lig.t+ T.e room :as spaious and in normal ondition of lig.t+
My program operated in 6H frames per seond :it. my laptop ?"ntel Centrino T$A%%
Dual ore at 6!>> 0H- and $ 04ytes of R#M@+ "n order to ma/e a demonstration of our
entire system as real as possi,le! :e .oose to s.o: t.e system for one ,uilding+ First!
:e ma/e a loal net:or/ ,et:een t.e ser'er ?main omputer of t.e ,uilding@! my
laptop ?for t.e 'ideo amera@ and t.e laptop of t.e eletroni students ?for t.e laser
,eam@+ #nd for pratial reasons! t.e entralised data,ase :as plaed in t.e same
room ,ut not integrated in t.e loal net:or/+ "n order to aess to t.is data,ase! t.e
ser'er ommuniate t.an/s to an internet onnetion ?s.o: t.e fat t.at t.e loation
of t.e entralised data,ase an ,e e'ery:.ere in t.e :orld! not neessarily loser of
t.e main omputers@+ # s.eme of our system in t.e room is present on t.e ne5t page+
A&
5.# Tests and e'aluations
During t.is demonstration! :e test some situations :.i. ould ause pro,lems+
T.e first one :as to test my multi-t.reading ount-insertor program+ "f se'eral sensors
detet se'eral persons in t.e same time and after send t.e data ount to t.e ser'er
?main omputer@! may,e t.ere :ill ,e some loss data+ T.is test :as positi'e! t.an/s to
t.e TCPD"P protool and )#V# language :.i. an easily and surely deal :it. t.e
pro,lem of simultaneous multi-t.reading+ T.e ot.ers tests onern my Simulin/ ode+
Ie /ne: t.at laser-,eam sensor .as pro,lem :it. multi rossing in t.e same time+ "n
fat! for e5ample! if t:o persons ross t.e .ori-ontal laser ,eam in t.e same time
?same or opposite diretion@ t.en one person :ill .ide t.e ot.er and t.e sensor :ill
1ust ount one+ T.ose t:o different situations :ere tested and :or/ed in some ases+
"n fat! t.e algorit.m an deal :it. multiple rossings if and only if t.e persons are not
too loser ?no tou. ,et:een people@ ,eause it is not enoug. e'olute onerning t.e
segmentation and t.e tra/ing+ #ll t.ose impro'ements :ill ,e tal/ later in t.e
impro-ement section+ Ot.er tests .a'e ,een done and :as positi'e too for e5ample
A9
Illustration 2: !emonstration "lueprint
'ery fast and slo: rossings+ "n some ase! fast people rossing an ause pro,lem if
t.e num,er of frames per seond of t.e 'ideo a;uisition and t.e speed of t.e
algorit.m are not enoug. .ig.+
So :e an onlude t.at t.e 'ideo amera an ,e a 'ery good alternati'e to deal
:it. ,ig entrane ?se'eral persons an enter or e5it t.e door in t.e same time@ t.us
more aurate t.an ot.er sensor+ 4ut t.e fat is using 'ideo amera is more e5pensi'e
t.an using ot.er sensors so to redue t.e ost of t.e system! it is not neessary to use
'ideo amera for small entrane ?normal door@C for e5ample t.e .ori-ontal laser ,eam
is enoug.+
A7
& "mpro'ements
4eause of t.e s.ort time limit to de'elop t.e people ounting system! t.e
prototype is not 'ery ad'aned and an easily ,e impro'ed+ T.is setion o'ers all t.e
features :.i. an ,e added in order to impro'e t.e entire people ounting algorit.m+
T.is setion is di'ided in t:o parts! t.e first one desri,es all t.e different
impro'ement for a people ounting appliation and t.e seond one gi'es some ideas to
e5tend t.e prototype to ot.er appliations+
&.! People counting system
#s said in t.e /esult section some situations are resol'ed li/e t.e ount of se'eral
persons :.i. ross a 'irtual line in t.e same or opposite diretion+ 4ut if t.e persons
are too loser ?ontat@! t.e algorit.m is not a,le to separate orretly t.ose ,lo,s+ T.e
first important impro'ement is a,out t.e segmentation+ To ma/e t.e segmentation
,etter! t.e prototype .as to use a ,etter 'ideo amera ?:.i. .a'enEt automati
proess li/e lig.t ompensation@! use t.e information of t.e motion detetor in order
to detet olusions and ma/e a ,etter sil.ouette of t.e o,1ets and may,e use olour
analysis to lassify ea. pi5el in order to remo'e s.ado:s+ T.en t.e tra/ing module
ans ,e impro'ed ,y memori-ing more informations of ea. ,lo,s frame to frame li/e
t.e ,ounding ,o5! position of t.e entroid! num,er of pi5el in t.e area! mean of t.e
pi5el gray 'alues+ Li/e t.is t.e tra/ing module :ill deal! in a ,etter :ay! t.e situations
of merging and splitting ,lo,s+ Finally! t.e ot.er impro'ement onerned t.e ounting
module+ "f t.e program detets a mo'ing o,1ets t.en laun. a .uman reognition
?s/in detetor! .ead detetor or ot.er modules to detet .uman@+
"n t.e future! t.is model .as to ,e e5ported in ot.er language to :or/ :it. a DSP
miroproessor ,uild-in t.e 'ideo amera+ Li/e t.is! it :ill redue t.e a,les and spae
ta/en ,y t.e urrent system and ma/e it more ommeriali-ed+
&.# /ther applications
# people tra/ing system ans ,e used for ot.ers appliations not neessarily 1ust
for ounting people+ For e5ample! t.is systems an ,e e5tended for seurity
appliation+ "n fat! a real-time people tra/ing system pro'ides enoug. information
in order to ma/e a good 'ideo sur'eillane+ Detet strange ,e.a'iours of people ?li/e
'iolent gesture! fig.t or running people@ and store t.ose informations on a data,ase+
T.is type of system ans ,e 'ery interesting for store/eeper or supermar/et+ #n ot.er
appliation is for mar/eting+ "t an analyse t.e ,e.a'iours of lients and ma/e
onlusion+ For e5ample! measure t.e impat of an ad'ertisement or modifiation in
t.e arrangement+ Determine t.e period and plae of good and ,ad influene+++
=6
(onclusion
T.is master t.esis presents an approa. to ount people passing t.roug. a 'irtual
gate using a fi5ed .eap 'ideo amera mounting 'ertially and Matla,-Simulin/
programming tool lin/ed to a :e, appliation+ "n section %! t.e results s.o: t.at using
a amera to ount people is good alternati'e to ot.er sensors for ,ig entrane ,eause
more aurate+ 4ut it s.o:s also t.at t.e system needs a lot of impro'ements to ,e
really relia,le+
T.is topi :as 'ery interesting for me! ,eause it merges a lot of appliations su.
as image and 'ideo proessing! :e, appliation and data,ases and miro-ontroller
programming+ "t :as good to :or/ in a su. 'arious pro1et and disuss a,out
pro,lems :it. all t.e person :.o partiipated to t.is pro1et+ "t permits also to
impro'e my s/ills in Matla, de'eloping and learning a ne: de'elopment tool
Simulin/+
4ut! ,eause of t.e s.ort time and t.e time " lost to ode :it. Simulin/! " am not
'ery satisfied of my resulting :or/+ " .a'e a lot of ideas of algorit.ms ,ut " ouldnEt
implement t.em in Simulin/ due to my le'el+ " donEt regret using Matla,-Simulin/
tool! ,ut if! one day! " .ad to ontinue t.is pro1et " :ill restart it from t.e ,eginning
and use a language :it. :.i. " am more omforta,le su. as Matla, for e5ample+ "
:as .appy to do my :or/ plaement in t.is great and ,eautiful ountry :.i. is
8or:ay! and if " .a'e t.e opportunity to ontinue t.is pro1et t.en " :ill return t.ere
:it.out any .esitations XX+
=A
)ibliography
MTER77N ( F+ Terada! D+ *os.ida! S+ Oe! and )+ *amagu.i! # met.od of ounting t.e
passing people ,y using t.e stereo images! "nternational onferene on image
proessing! %-&9%A-H=>&-$!6777
M4E*77N ( D+ 4eymer and F+ Fonolige! Real-time tra/ing of multiple people using
stereo! ! !6777
MH#S7&N ( Has.imoto! F+ Morina/a! F+ *os.ii/e! 8+ Fa:agu.i! C+ Matsueda! S!
People ount system using multi-sensing appliation! 677& "nternational onferene
on solid state sensors and atuators! %-&9%A-A9$7-=!677&
MTES7>N ( #+ Tesei! #+ Tes.ioni! C+S+ Rega--oni! 0+ Verna--a! YLong-MemoryZ
mat.ing of interating omple5 o,1ets from real image se;uenes! 677> Conferene
on Time Varying "mage Proessing and Mo'ing O,1ets Reognition! !677>
MSH"76N ( #+ S.io and )+ S/lans/y! Segmentation of people in motion! 6776
Proeedings of t.e "EEE :or/s.op on Visual Motion! %-969>-$6HA-$!6776
MSCH7HN ( #+)+ S.ofield! T+)+ Ston.am! and P+#! # R#M ,ased neural net:or/
approa. to people ounting! Fift. "nternational Conferene on "mage Proessing and
its #ppliations ! %-9H$7>->=$-A!677H
MSEP7HN ( 0+ Se5ton! P+ [.ang! D+ Redpat.! and D+ 0rea'es! #d'anes in automated
pedestrian ounting! European Con'ention on Seurity and Detetion ! %-9H$7>->=%-
&!677H
MSE07>N ( )+ Segen and S+0+ Pingali! # amera-,ased system for tra/ing people in real
time! Proeedings of t.e 6At. "nternational Conferene on Pattern Reognition! %-
969>-&$9$-P!677>
MH#R%6N ( "+ Haritaoglu and M+ Fli/ner! Detetion and tra/ing of s.opping groups in
stores! Proeedings of t.e $%%6 "EEE Computer Soiety Conferene on Computer
Vision and Pattern Reognition ! %-&>7H-6$&$-%!$%%6
MCO87=N ( 0ary Conrad and Ri.ard )o.nson,aug.! # real-time people ounter!
Proeedings of t.e 677= #CM symposium on #pplied omputing! %-97&76->=&-> !677=
MROS7=N ( M+ Rossi and #+ 4o--oli! Tra/ing and Counting Mo'ing People! "EEE Pro+
of "nt+ Conf+ "mage Proessing! !677=
=H

Damien Lefloch Master Thesis

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Damien Lefloch Master Thesis

Uploaded by

Copyright:

Available Formats

Real-Time People Counting system

using Video Camera

You might also like