Professional Documents
Culture Documents
1, February 2014
ABSTRACT
Face detection is one of the most relevant applications of image processing and biometric systems. Artificial neural networks (ANN) have been used in the field of image processing and pattern recognition. There is lack of literature surveys which give overview about the studies and researches related to the using of ANN in face detection. Therefore, this research includes a general review of face detection studies and systems which based on different ANN approaches and algorithms. The strengths and limitations of these literature studies and systems were included also.
KEYWORDS
Face Detection, Face Recognition, Artificial Neural Networks
1. INTRODUCTION
In the past few years, face recognition has received a significant attention and regarded as one of the most successful applications in the field of image analysis [1]. The human faces represent complex, multidimensional, meaningful visual stimulant. Developing a computational model for face recognition is difficult [2]. Face detection can be regarded as fundamental part of face recognition systems according to its ability to focus computational resources on the part of an image containing a face. The process of face detection in images is complex because of variability present across human faces such as: pose; expression; position and orientation; skin color; presence of glasses or facial hair; differences in camera gain; lighting conditions; and image resolution [3]. The analysis of facial expression was primarily a research field for psychologists in the past years [4]. At the same time, advances in many domains such as: face detection [5][6]; tracking[7]; and recognition [1]; pattern recognition; and image processing contributed significantly to research in automatic facial expression recognition. Face detection should be performed before recognition system. This is done to extract relevant information for face and facial expression analysis. Two classes of techniques for face representation and relevant information extraction. And geometrical feature extraction relies on parameters of distinctive features such as eyes, mouth and nose. At the same time, a face is represented as an array of pixel intensity values suitably pre-processed in appearance based approaches (texture). This array is then compared with a face template using a suitable metric [4]. Research [8] compared the performances of these representation techniques in face recognition. Therefore, according to the complexity of face detection process, many applications based on human face detection have been developed recently such as surveillance systems, digital monitoring, intelligent robots, notebook, PC cameras, digital cameras and 3G cell phones. These
DOI : 10.5121/ijma.2014.6101 1
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
applications play an important role in our life. Nevertheless, the algorithms of the applications are complicated and hard to meet real-time requirements of specific frame-rate [9]. Over the past decade, many approaches for improving the performance of face detection have been proposed [9][10][11][12][13][14][15][16][17][18][19]. At the same time, many literature studies focused on survey on face detection techniques [20][21][6][22][23]. Artificial neural networks (ANN) were used largely in the recent years in the fields of image processing (compression, recognition and encryption) and pattern recognition. Many literature researches used different ANN architecture and models for face detection and recognition to achieve better compression performance according to: compression ratio (CR); reconstructed image quality such as Peak Signal to Noise Ratio (PSNR); and mean square error (MSE). Few literature surveys that give overview about researches related to face detection based on ANN. Therefore, this research includes survey of literature studies related to face detection systems and approaches which were based on ANN. The rest of this paper is organized as follows: Section 2 includes the main steps of face detection and recognition. Section 3 includes literature studies related to face detection systems based on ANN. Section 4 includes comparisons between these literature studies. Section 5 includes recommendations. Finally section 6 concludes this work.
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
enhanced user experience. To achieve high-performance face recognition system, each processing stage in the system has to be designed to satisfy application requirements [14]. Face recognition involves comparing an image with a database of stored faces to identify individual in input image. The related task of face detection has direct relevance to face recognition because images must be analysed and faces identified, before they can be recognized. Detecting faces in an image can help to focus the computational resources of the face recognition system, optimizing the systems speed and performance [3].
The main steps of face detection system are shown in Figure2. Face detection separate image windows into two parts: one containing faces, and one containing the background. The process is difficult because the: commonalities exist between faces (vary in terms of age, skin color and facial expression); and also differing in: lighting conditions; image qualities; and geometries. The face detector would be able to detect the presence of any face under any set of lighting conditions, upon any background.
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
(a)
(b)
Figure 7. (a): Hybrid algorithm. (b): Visualization of input and field connectivity [34] .
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
Avinash and Raina (2010) [40] presented face detection approach with Gabor wavelets transform & feed forward neural network for finding feature points and extracting feature vectors. Gabor filter used for feature extraction for face detection. The classifier (FFNN) take the feature vectors as input. The location of feature points contains information about the face in this approach. The graph is constructed from the general face idea. Instead of fitting this graph, the feature points are obtained from the characteristics of each face automatically. Facial features allow to make a decision from face parts because the facial features are compared locally instead of using a general structure. Two measures were used in this study : false negative and false positive. These two measures can be calculated using Equation 1 and Equation 2 as follows [39]: = = (1)
(2)
Mohammad Abadi, et al, (2011) [41] proposed approach based on ANN and Gabor wavelets to detect desirable number of faces in fixed photo with gray background. They used correlation of a window with a face with photo. Then they estimated areas of candidate of face presence. After that, they used step algorithm and referred these areas and around them to section of extraction of Gabor wavelets characteristics and neural network classifier. The resultant areas lead to detection of face locations in photo. They examined the result of estimation of efficiency of this method by different tests. The method is simulated in MATLAB. They used 70 face photos and 60 non face photos in training phase. Every face photo, its mirror photo and with the angle of 5,10,15 degrees in positive and negative directions and photos with one pixel shift in every 4 directions are placed in training set for reducing network sensitivity. For no face photos also, their mirror and their 180 degrees transformation is placed in training data. They obtained 5% right answer, error limit of 0.0001, false negative error= 5%. And for tested image with size=254600, the positive false=12, and detection= 56 from 57. Also for test image of size = 50100, the positive false=0, and detection= 2 from 3.
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
Anissa Bouzalmat, et al (2011) [42] presented BPNN for face recognition. The BPNN input is feature vector based on Fourier Gabor filters. They used an algorithm for detecting face regions in images using the color of skin which presents overlooked in different background, accessory and clothing. After that, they introduced Gabor filters with 8 orientations and 5 resolutions to get maximum information (to extract the maximum of information by varying the resolution and orientation). This is done to generate and extract the features vector of the whole face in image. BPNN is then applied to perform the recognition task. This solution was implemented using Java environment. Results indicate that the proposed method achieves good results. Figure 10 shows the BPNN with Gabor Wavelet for face detection.
10
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
Figure 11. Skin Color and BPNN for Face Detection [43]
3.12 Cascaded Neural Network Finally, Zuo & de With (2008) [45] proposed a fast face detector based on a hierarchical cascade of neural network ensembles to enhance detection accuracy and efficiency. They used a number of neural network classifiers to form a neural network ensemble. Each classifier is specialized in a sub region in the face-pattern space. These classifiers complement each other to perform the detection task. Then, they organized the neural network ensembles in a pruning cascade to reduce the total computation cost of face detection. In this stage, simpler and more efficient ensembles used at earlier stages in the cascade are able to reject a majority of non face patterns in image backgrounds by improving the overall detection efficiency while maintaining detection accuracy. Their results showed that the proposed neuralnetwork ensembles improve the detection accuracy as compared to traditional ANN. Their approach reduced training and detection cost by achieving detection rate equal 94%.
11
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
Table 1 shows that the literature studies used different data bases for image training and testing set. Other studies were not included in table 1 because they werent use known database and just used camera image samples. The number of samples is different from one study to another.as example the research [38] take only 50 image samples. Whereas other studies such as [27][32] used more than one image sets as samples. We can note from table 1 also that the studies adopted different image dimensions: (384384), (92112) and so on.
Table 1. DB and Performance measures used in literature studies Research Topology Retinal connected neural network Data Base: Training & Testing Three training sets of images. Test SetA: 42 scanned photographs Test SetB: 23 images contain 155 faces Test SetC: 65 images, 183 faces (images with more complex backgrounds and without faces to measure false detection ) Select 700 pictures in Kah-Kay Sungs data set of 1488 faces to train ANN with 700 random noise pictures as negative examples remaining 788 faces in Kah-Kays data set, followed by 788 random noise pictures First set: 3257 images downloaded from several websites (384384), with one face in each image. Second set: 130 images downloaded from website of CMU training of CNN, the number of facial fragment images used is 2900 for the FD2 layer, 5290 for the FD3, and 14,700 (face) for the FD4 layer, respectively. Number of non-face images, also used for the FD4 layer, is 137. Training set contains 12000 face images collected from various face DBs. These samples also include the scaled versions at the same face with factor (0.8 - 1.12) ORL dataset: 400 frontal faces: 10 tightly cropped (92112) with 256 grey images of 40 individuals with variations in pose, illumination, ..etc Training set: face images from MIT DB. Images (scaled to 2020) Test set: 2000 face/non-face images from MIT DB. Non-face patterns generated at different locations and scales. 50 real images taken under different lighting conditions (digital camera images and web images from several websites). Performance Detect 78.9% - 90.5% of faces in a set of 130 test images Acceptable number of false detections. 1.2% error after training for 50 epochs 1566 examples, 35 mis classifications made (2.23% error). Detection rate = 84.6% False rate=3:51 106 Recognition rate = 97.6% for 5600 still images of more than 10 subjects
[27]
[30]
[32]
PNN
[33]
CNN
[37]
BPNN
[39]
Detection rates measured for separate test set of 500 faces and 4000 non-faces. Performance=94%. Detect (77.9% - 90.3%) of faces in a set of 130 test images Detection rate = 91.6% Error rate = 7.54%
[36]
[38]
BPNN
Detect 97.3% of faces in a set of 50 real images. Processing Time (s) of image (63180)= 3.8 Processing Time (s) of image (200219)= 6.2 12
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
Finally we can note from table 1 that the factors used as performance evaluations are different from one study to other. Many studies used detection rate, others used false rate, and so on. Figure 12 shows the detection rate of different ANN approaches from many studies. We can note from Figure 12 that the highest face detection rate can be obtained from using the CNN [33] approach. At the same time the BPNN approach adopted in [38] and [37] result in obtaining good and high detection rate.
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014
ANN can be adopted in combination with other algorithms to obtain better results for face detection. At the same time, we must focus on how to simplify the combined algorithms steps to reduce the memory required and processing time. Try to use other ANN architectures: self-organizing map, PatternNet, FitNet and Fast BPNN. Try to use different optimization ANN training algorithms such as: Levenberg-Marquardt (TRAINLM); TRAINBFG; Bayesian regularization (TRAINBR); TRAINCGF algorithm; Gradient descent (TRAINGD); and Gradient descent with momentum (TRAINGDM) [46] to obtain best results for face detection system. Try to use genetic algorithm (GA) as an optimization algorithm to obtain the best values of ANN algorithm parameters that result to optimal results.
6. CONCLUSION
This paper includes a summary review of literature studies related to face detection systems based on ANNs. Different architecture, approach, programming language, processor and memory requirements, database for training/testing images and performance measure of face detection system were used in each study. Each study has its own strengths and limitations. In future work, a face detection system will be suggested based on using Pattern Net and Back propagation neural network (BPNN) with many hidden layers. Different network architectures and parameters values of BPNN and PatternNet will be adopted to determine PatternNet architecture that will result in best performance values of face detection system.
ACKNOWLEDGEMENTS
The author would like to thanks Al-Zaytoonah university of Jordan- Amman- Jordan for supporting this research.
REFERENCES
W. Zhao, et al (2000) Face recognition: a literature survey, Technical Report CAR-TR-948, University of Maryland, October 2000. [2] Turk M & Pentland A (1991) Eigenfaces for recognition, Journal of Cognitive Neuroscience, Vol.3, pp7186. [3] Phil Brimblecombe (2002) Face Detection using Neural Networks, H615 Meng Electronic Engineering, School of Electronics and Physical Sciences, URN: 1046063. [4] Bouchra Abboud, et al (2004) Facial expression recognition and synthesis based on an appearance model, Signal Processing: Image Communication, Vol. 19, Issue. 8, pp723-740. [5] P. Viola & M.J. Jones (2001) Robust real-time object detection, Technical Report CRL/2001/01, Cambridge Research Laboratory, USA, February 2001 [6] Ming-Hsuan Yan, et al (2002), Detecting Faces in Images: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 1, pp34-58, January2002 [7] Minyoung Kim, et.al (2008) Face tracking and recognition with visual constraints in real-world videos, IEEE Conference on Computer Vision and Pattern Recognition, pp23-28, June. [8] Brunelli R & Poggio T (1993) Face recognition: features versus templates, IEEE Transaction Pattern Analysis and Machine Intelligence, Vol. 15, No.10, pp10421052. [9] Yao-Jiunn Chen & Yen-Chun Lin (2007) Simple Face-detection Algorithm Based on Minimum Facial Features, The 33rd Annual Conference of the IEEE Industrial Electronics Society (IECON) 58 Nov 2007, Taipei, Taiwan, pp455-460. [10] Sanjay Singh, et. Al (2003) A Robust Skin Color Based Face Detection Algorithm, Tamkang Journal of Science and Engineering, Vol. 6, No. 4, pp.227-234 . [1]
14
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014 [11] Abdenour Hadid, Matti Pietikainen & Timo Ahone (2004) A Discriminative Feature Space for Detecting and Recognizing Face, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2. [12] Elise Arnaud et al (2005) A Robust And Automatic Face Tracker Dedicated To Broadcast Videos, IEEE International Conference On Image Processing. [13] Zhonglong Zheng, Jie Yang & Yitan Zhu (2006) Face Detection and Recognition using Colour Sequential Images, Journal of Research and Practice in Information Technology, Vol. 38, No. 2, pp.135-149, May 2006. [14] Fei Zuo (2006) Embedded Face Recognition Using Cascaded Structures, Thesis, Technische Universiteit Eindhoven, China. [15] Zuo F. and P.H.N. de With (2002) Automatic Human Face Detection for a Distributed Video Security System, Proceedings of the Progress workshop on Embedded Systems, pp269274, Oct. [16] Stan Z. Li & Anil K. Jai (2005) Handbook of Face Recognition, Springer Science &Business Media. [17] Jun-Su Jang & Jong-Hwan Kim (2008) Fast and Robust Face Detection Using Evolutionary Pruning, IEEE Transactions on Evolutionary Computation, pp1-10. [18] Bernd Menser & Michael Brunig (2000) Face Detection and Tracking for Video Coding Application, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers, 29Oct-1Nov2000, Pacific Grove, CA, USA [19] Pedro Alexandre Dias Martin (2008) Active Appearance Models for Facial Expression, Recognition and Monocular Head Pose Estimation, master thesis, Dept. of Electrical and Computer Eng., Faculty of Sciences and Technology, University of Coimbra. [20] Hjelmas and Low (2001) Face Detection: A Survey, Computer Vision and Image Understanding, vol. 83, pp236-274, doi:10.1006/cviu.2001.0921, http://www.idealibrary.com [21] Yongzhong Lu, Jingli Zhou & Shengsheng Yu (2003) A Survey of Face Detection, Extraction and Recognition, Computing and Informatics, Vol. 22, pp.163-195. [22] W. Zhao et al (2003) Face recognition: A literature survey, ACM Computing Surveys, Vol.35, No.4, December 2003, pp. 399458. [23] Cha Zhang & Zhengyou Zhang (2010) A Survey of Recent Advances in Face Detection, Technical Report, MSR-TR-2010-66, Microsoft Research, Microsoft Corporation One Microsoft Way, Redmond, WA 98052, http://www.research.microsoft [24] Abboud B, Davoine F & Dang M (2004) Facial expression recognition and synthesis based on an appearance model, Signal Processing: Image Communication, Vol.19, No.8, pp723740. [25] Mohammad Alia, Abdelfatah Tamimi and Omaima Al-Allaf, "Integrated System For Monitoring And Recognizing Students During Class Session", AIRCCs: International Journal Of Multimedia & Its Applications (IJMA), Vol.5, No.6, December 2013, pp:45-52. Airccse.org/journal/ijma.html [26] Jain A, Ross A & Prabhakar S (2004) An introduction to biometric recognition, IEEE Transactions on Circuits and Systems for Video Technology, Vol.14, No.1, pp420, Jan. [27] Henry Rowley, Baluja S. & Kanade T. (1999) Neural Network-Based Face Detection, Computer Vision and Pattern Recognition, Neural Network-Based Face Detection, Pitts-burgh, Carnegie Mellon University, PhD thesis. [28] KahKay Sung & Tomaso Poggio (1994) Example Based Learning For View Based Human Face Detection, Massachusetts Institute of Technology Artificial Intelligence Laboratory and Center For Biological And Computational Learning, Memo 1521, CBCL Paper 112, MIT, December. [29] Henry A. Rowley, Shumeet Baluja &Takeo Kanade. (1997) Rotation Invariant Neural Network-Based Face Detection, December, CMU-CS-97-201 [30] Jeffrey S. Norris (1999) Face Detection and Recognition in Office Environments, thesis, Dept. of Electrical Eng. and CS, Master of Eng in Electrical Eng., Massachusetts Institute of Technology. [31] Hazem M. El-Bakry (2002), Face Detection Using Neural Networks and Image Decomposition Lecture Notes in Computer Science Vol. 22, pp:205-215. [32] Lin-Lin Huang, et al (2003) Face detection from cluttered images using a polynomial neural network, Neurocomputing, Vol.51, pp197 211. [33] Masakazu Matsugu (2003) Subject independent facial expression recognition with robust face detection using a convolutional neural network, Neural Networks, Vol.16, pp555559. [34] Stefan W., Christian I. & Uwe H (2004) Evolutionary Optimization of Neural Networks for Face Detection, Proceedings of the 12th European Symposium on Artificial Neural Networks, Evere, Belgium: d-side publications. [35] Marian Beszedes & Milos Oravec (2005) A System For Localization Of Human Faces In Images Using Neural Networks, Journal Of Electrical Engineering, Vol. 56, No 7-8, pp195199. 15
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.1, February 2014 [36] N. Shilbayeh and G. Al-Qudah (2008) Face Detection System Based On MLP Neural Network, Recent Advances in Neural Networks, Fuzzy Systems & Evolutionary Computing, ISSN: 1790-5109, ISBN: 978-960-474-195-3, pp 238-243. [37] Zoran Bojkovic & Andreja Samcovic (2006) Face Detection Approach In Neural Network Based Method For Video Surveillance, 8th Seminar on Neural Network Applications in Electrical Engineering, Neurel, Faculty Of Electrical Eng., University Of Belgrade, Serbia, September 25-27. [38] Aamer Mohamed, et al (2008) Face Detection based Neural Networks using Robust Skin Color Segmentation, 5th International Multi-Conference on Systems, Signals and Devices, IEEE. [39] Sahoolizadeh, Sarikhanimoghadam and Dehghani (2008) Face Detection using Gabor Wavelets and Neural Networks, World Academy of Science, Engineering and Technology, Vol. 45, pp552- 554. [40] Avinash Kaushal, J P S Raina (2010) Face Detection using Neural Network & Gabor Wavelet Transform, International Journal of Computer Science and Technology (IJCST), Vol. 1, Issue.1, pp58-63, September 2010, ISSN : 0976 - 8491. [41] Mohammad Abadi, et al, (2011) Face Detection with the Help of Gabor Wavelets Characteristics and Neural Network Classifier, American Journal of Scientific Research, Issue.36, pp67-76, ISSN 1450-223X, http://www.eurojournals.com/ajsr.htm [42] Anissa Bouzalmat, et al (2011) Face Detection And Recognition Using Back Propagation Neural Network And Fourier Gabor Filters, Signal & Image Processing: An International Journal (SIPIJ) Vol. 2, No. 3, September 2011, DOI : 10.5121/sipij.2011.2302 15 [43] Kalavdekar Prakash N. (2010) Face Detection using Neural Network, International Journal of Computer Applications (0975 8887),Vol.1, No.14, pp36-39. [44] Mansaf M Elmansori & Khairuddin Omar (2011) An Enhanced Face Detection Method Using Skin Color and Back-Propagation Neural Network, European Journal of Scientific Research, ISSN 1450216X, Vol.55 No.1, pp80-86, http://www.eurojournals.com/ejsr.html [45] Zuo F & P.H.N. de With (2008) Fast face detection using a cascade of neural network ensembles, EURASIP Journal on Advances in Signal Processing, Volume 2008, Article ID 736508, Hindawi Publishing Corporation, pp1-13, doi:10.1155/2008/736508 [46] Hudson, Hagan and Demuth, Neural Network Toolbox Users Guide R2012a, The MathWorks, Inc., 3 Apple Hill Drive Natick, MA 01760-2098, 2012, www.mathworks.com
AUTHOR
Dr. Omaima N. A. Al-Allaf received the M.Sc. degree in CS from the Dept. of CS\ Faculty of Computers and Mathematical Sciences\ University of Mosul\ Mosul\ Iraq in 1999. She received the Ph.D. degree in CIS from CIS department\ Faculty of IS and Technology\ AABFS \ Jordan in 2008. Currently, she is an Assistant Professor at CIS department\ Faculty of Sciences and IT\ AlZaytoonah University of Jordan\ Amman\ Jordan (from 2009). Her research interests include image compression and recognition, Artificial Neural Networks, Genetic Algorithms. She is member of International Association of Engineers (IAENG).
16