Through the Interface

September 1 2, 201 2

Creating a face-recognising security cam with a Raspberry Pi Part 1

This series of posts builds upon the mini-series on building a motion-detecting security cam based around the Raspberry Pi. Once y ou hav e y our motion detecting security cam up and running, y ou should be able to mov e on to the nex t stage: enabling that sy stem to recognize faces that it has been trained against. My specific project (which Im calling the Facecam, although I hav ent applied for a trademark ;-) pulls data down from Facebook and uses that to train the face recognition sy stem, but thats far from being a requirement: its also v ery possible to train the database in other way s. Well go into detail on the v arious parts ov er the coming weeks (probably once a week, as I want to continue blogging about other topics, too), but lets start by introducing the project and the sy stem architecture. Heres the elev ator pitch: Facecam is a security camera that recognises a residents Facebook friends when they come to their front door and allows for tailored communication both to the resident and the v isitor. This would basically mean that instead of the resident receiv ing a notification email say ing Y ou hav e a v isitor (as in the case of the motion-detection sy stem we saw prev iously ), the email would read John Smith v isited y ou. And John could potentially receiv e a customized were out message while standing at the door and ev en a post on his Facebook wall thanking him for the v isit. A lot of this is clearly contingent on the accuracy of the facial recognition sy stem something well look at much more closely later on. In case the accuracy isnt adequate, it would be nice at least to prov ide some feedback to the v isitor, say ing Welcome, John! (ev en if its actually Megan at the door ;-). Right, now lets div e into the sy stem architecture. Heres a quick diagram of the Facecam sy stem:

Heres a look at the new components (in addition to the Motion detection box we saw prev iously ): 1. A com ponent to download inform ation on the residents Facebook friends (.NET desktop application) This component will build a friend database (stored in a file named facedata.xml) to be used at runtime by the security cam. The database will be trained with the friends photos as downloaded from Facebook. So the component will need to access for each friend both their names and the photos in which they hav e been

tagged (as well as the tag information, of course). Using the tag information, the component will use OpenCV to attempt to ex tract the friends face (by checking the list of detected faces against the tag location, to get the closest match), and if it finds it it will resize, crop and equalize the grey scale intensity of the image before sav ing it to disk for later use in the database training process. This is ultimately an offline, CPU-intensiv e operation we will simply need the database transferred across to the Raspberry Pi from time to time so the approach well take is to create a .NET desktop application that performs this task on a higher-powered dev ice. 2. A com ponent to analy se fram es sav ed by the m otion-detection sy stem , checking for friends (face recognition) This is the guts of the sy stem: the runtime component that will again use OpenCV to process images captured from the webcam, detect faces and then check the results against the facial recognition database on the Raspberry Pi. This needs to be pretty efficient as the feedback should be in close to real-time which may be a challenge: giv en the relativ ely low power of the Raspberry Pi CPU, well hav e to jump through some interesting hoops to get this working well enough. 3. A com ponent to prov ide feedback to the v isitor (LED m essaging) If were going to tell the v isitor that they hav e been recognised (or in some way warn unwanted v isitors that their photo has been sent to the property s residents), then were going to need some kind of screen. The Raspberry Pi has both HDMI and component v ideo outputs, but Iv e decided to go the way of a USB-powered LED message-board. More on this in due course. Right, thats it for this introductory post. Well look in more detail at each of these components ov er the coming weeks, starting with the .NET desktop application to download friend information. Y ou m ight also like:

Creating a facerecognising security cam with a Raspberry ...

Creating a motiondetecting security cam with a Raspberry ...

Creating a motiondetecting security cam with a Raspberry ...

Creating a facerecognising security cam with a Raspberry ...

LinkWithin

Posted at 07 :45 AM in Raspberry Pi | Permalink Com m ents

Jeremy Tammik dijo... Hi Kean, Sounds neat and clear and interesting.

I really wonder what kind of data is going to go into the face data XML file. I would not put pix el data there, but store that in ex ternal indiv idual image files, and reference those. Some of my friends boy cott internet-based social networking and thus hopefully cannot be identified by facebook, so I would enable priv ate non-facebook photos from the local sy stem to be used as well. Cheers, Jeremy . Responder September 1 2, 201 2 at 09:1 1 AM

Kean Walmsley dijo en respuesta a Jeremy Tammik... Hi Jeremy , We'll of course talk about the XML format (although it's something this project has inherited), which uses an algorithm called Eigenfaces to greatly reduce the amount of data stored. And y es - I'm probably not ev en going to share the Facebook-specific code (for once), but will certainly ex plain more about the file/folder structure y ou'd need to build to train the database in some other way . Cheers, Kean Responder September 1 2, 201 2 at 09:1 5 AM

Alex Fielder dijo... Hi Kean, This looks really interesting! An API I'm not familiar with (but which I believ e might prov e useful here) is the Google Picasa Web API; As I say , I only know there ex ists an API from Google which (in some way ) ties in with the Picasa Web offering from the big G. In case y ou aren't aware, Picasa (the desktop application) offers y ou the ability to tag faces of the people it finds within y our picture library , and will (with y our permission) sy nc them to y our Picasa Web Photos albums. I'v e tagged all my family with it, so if it's possible to use this API instead of Facebook, that might be another av enue that would likely get around the friends-not-tagged-on-Facebook issue that Jeremy mentions. Here is the best link I could find that will be of help: Retriev ing face tag data from the Picasa Web API (which in turn links to this page: Google Groups Picasa issues page Cheers, Alex . Responder September 1 2, 201 2 at 04:52 PM

Kean Walmsley dijo en respuesta a Alex Fielder... Hi Alex , That's interesting... it should certainly be possible to swap out the Facebook-oriented piece to pull down data

from Picasa. I'll be describing the process I'v e used to isolate and ex tract faces to train the database nex t week. Cheers, Kean Responder September 1 2, 201 2 at 04:56 PM

Philipp Wagner dijo... This will probably help y ou: http://docs.opencv .org/trunk/modules/contrib/doc/facerec/index .html. Responder September 1 2, 201 2 at 1 0:39 PM

Philipp Wagner dijo en respuesta a Philipp Wagner... Especially the part on Face Recognition in V ideos might be interesting to y ou: * http://docs.opencv .org/trunk/modules/contrib/doc/facerec/tutorial/facerec_v ideo_recognition.html Responder September 1 2, 201 2 at 1 0:45 PM

Kean Walmsley dijo en respuesta a Philipp Wagner... Thanks, Philipp - that's v ery helpful. I'm pretty much done with that side of things, but I'll certainly rev isit the code once I'v e looked at the tutorial more closely . Kean Responder September 1 2, 201 2 at 1 0:58 PM

Hly tin dijo... That was an interesting thing,security camera purpose is really a lot specially on giv ing a security a shop and home.In Finland country many of people are still hav ing doubt on putting security cameras howev er most of the already did.Any way this is really ex cellent one. Responder Nov ember 07 , 201 2 at 1 0:27 AM

Py ry l dijo... Hi Kean, hope y ou are still following this thread. I'm wondering whether to start a face recognition project. My concept would be based on linkedin profile pics, thus only 1 reference pic per face. Based on y our ex perience is recognizing a face with only one reference pic av ailable from a photo by any chance possible? Did y ou encounter in y our ex periment cases where the user had only one pic in fb? Responder March 25, 201 3 at 03:22 PM

Kean Walmsley dijo en respuesta a Py ry l... Hi Py ry l, Based on my ex perience the problems would be: * 1 photo is way too few, at least for the sy stem I tried to use. * Profile photos aren't representativ e enough of how people look when they come to y our door (or sit at their PC, if somehow recognising people's faces v ia their webcam). Cheers, Kean Responder March 25, 201 3 at 04:00 PM

Py ry l dijo... Thanks Kean! I think y ou just sav ed me a week of hacking towards a dead end. Responder April 01 , 201 3 at 1 2:44 PM Comment below or sign in with Ty pePad Facebook Twitter and more...