A Method of 3D Face Recognition Based on Principal Component Analysis Algorithm

Xue Yuan, Jianming Lu, and Takashi Yahagi,

and head position. In the last ve years, tasks such as the creation of 3D facial models, 3D pose etimation, 3D models recognition have been active research topics. In [8], the 3D techniques based on the extraction of 3D facial features from depth images by means of differential geometry techniques are presented. In [9], point signatures have also been proposed for face identication. In [7], a face recognition technique is developed based on 3D shape and texture imformation. In the recognition procedure, we propose a method for face recognition based on fuzzy clustering and parallel neural networks. In recent years neural computing has emerged as a practical technology, with successful application in the pattern recognition eld [4]. However, the conventional backpropagation neural network has difculty in learning as the number of categories increase. Even when learning is possible, too much time is required. As a solution to this type of problem, CombNET-II, by which to produce several smallscale neural networks and recognize them in an integrated form, is proposed [5]. CombNET-II is a very efcient algorithm that is applicable even to a large-scale system. However, even close patterns may not be learned by the neural network because the CombNet-II restricts each point of data set to exactly one cluster and patterns belonging to each cluster or small-scale neural network are not overlapped. In order to solve this problem, this paper proposes a method of dividing face patterns by fuzzy clustering when determining patterns belonging to each small-scale neural network and to the neural networks. Fuzzy clustering produces the idea of uncertainty of belonging with respect to a membership function, and permits one pattern to belong to several small-scale neural networks (subnetworks). Since patterns belonging to each cluster or small-scale neural networks are overlapped, patterns that are comparatively similar to each other can be recognized in the same neural network efciently. In our system, we extract 2D texture data and the 3D shape data from 3D facial images. The main objective of our system is to explore a more efcient feature-based approach by considering both shape and texture information and capitalize on the advantages afforded by solely using either 2D or 3D information. II. P RE - PROCESSING A. Facial image acquisition In our research, original facial images were obtained by RangeFinder Scansion 3D [Danae-R]. Input device is illustrated in Fig. 1. The original facial images are illustrated in Fig. 2.
Abstract In this paper, we present a method of face recognition using 3D images. We rst compensate the poses of 3D original facial images using geometrical measurement and extract 2D texture data and the 3D shape data from 3D facial images for recognition. Based on principal component analysis (PCA) Algorithm, all the 2D texture images and the 3D shape images are normalized to 32 32 pixels. In the second step, we propose a method for face recognition based on fuzzy clustering and parallel neural networks. Experimental results for 70 persons with different pose demonstrate the efciency of our algorithm.

I. I NTRODUCTION Face recognition plays an important role in many applications such as building/store access control, suspect identication and surveillance. Over the past 30 years many different face recognition techniques were proposed, motivated by the increased number of real world applications requiring recognition of humans. There are several problems that make automatic face recognition to be a very difcult task. The input face image of a person presented to a face recognition system is not usually taken under similar conditions as the face image of the same person in the database. Therefore, it is important that an automatic face recognition system is able to cope with numerous possible variations among different images of the same face. The image variations are mostly due to changes in the following parameters: viewpoint, illumination, expression, age, disguise, facial hair, glasses, and background. Several of the methods that have been reported in the literature have demonstrated some ability to reduce their sensitivity to variations in illumination condition and pose[1], [2]. In previous researchs, most face recognition systems are based on the evaluation of 2D characteristic points or colour images [2], geometric feature-based methods [10], using properties and relations among features such as eyes, mouth, nose, and chin to perform recognition. An eigenface system was proposed in [11], which projects face images onto a feature space that span the signicant variation among known face images. Isodensity lines [12], which are the boundaries of constant gray level areas after quantizing an image, were also investigated for face recognition. Since the extraction of reliable features, compensation for illumination effects, and compensation for pose are difcult and are subject to a variety of possible interpretation errors. The recognition accuracy of such systems is limited to a small set of individuals. The use of additional 3D information is expected to improve the reliability of the recognition scheme, since biometric features are reliably extracted from 3D measurements, due to their relative independence from illumination inuences, head rotation,

0-7803-8834-8/05/$20.00 2005 IEEE.

3211

E. Face region extraction We extract the face region with the center at the tip of the nose, and cut out the surplus in the z-direction up to 65. F. 3D shape data
Fig. 1. Input device

The 3D shape data are represented by the z-coordinates of each pixel. In order to coordinate with 2D gray scale data, the 3D shape data are normalized to 8bit by quantization.

Fig. 2. image

Example of original facial

Fig. 3.

Feature points Fig. 4. 3D shape data Fig. 5. Texture data

B. Converting to gray-scale images RGB format images are obtained from the device. However, the color images are liable to vary under the change of color or reectance properties [3]. In order to solve this problem, in this research, we convert the color images to gray scale images. The brightnesses(Y) of the gray scale images are calculated as follows: Y 0.299 I = 0.596 Q 0.211 0.587 0.274 0.523 0.114 R 0.322 G 0.312 B

G. Principal Component Analysis In order to reduce the amount of calculation of training and recognition by neural network, Principal Component Analysis (PCA) are used for face compression. PCA is a powerful technique for reducing a large set of correlated variables to a smaller number of uncorrelated components. It has been applied extensively for both face representation and recognition [11]. We use the PCA on the set of shape and texture vectors Si and Ti of facial images i = 1...m. Ignoring the correlation between shape and texture data, we analyze shape and texture separately. For shape, we subtract the average s =
1 m m

(1)

C. Locating the feature points We locate 7 points on a human face and then use them for face pose compensation and extraction. These points are chosen based on their signicance in representing a face, and the reliability of extracting them automatically. These points are described as follows: P1: the tip of nose, P2, P3: the midpoint of two eyes and two brows, P4, P5: the points on each side of the nose, P6, P7: the right-and-left eyes, L1:the nose center-line D. Face pose compensation Variant posed facial images are obtained with the input device. Consequently, 3D pose compensation is a critical requirement for accurate face recognition using the information of height. In this section, we present a method for face pose compensation. The process of the face rotation is projected as follows: The rst step is to rotate on the x-axis of the template, until the gradient of nose is equal to the xed value. The next step is to rotate on y-axis of the template, until the z-coordinates of the mouth corners are equal. Finally , there is a rotationon z-axis of the template, until the y-coordinates of the eyes are equal.

shape vector, ai = Si s , and dene a data matrix A = (a1 , a2 , ..., am ). The essential step of PCA is to compute the eigenvectors s1 , s2 , ... of the covariance matrix C =
1 T m AA

i=1

si from each

1 m

2 2 The eigenvalues of C, S, 1 S,2 ..., are the variances of the data along each eigenvector. Let U = (U1 , U2 , ..., Ur ) be the r eigenvectors corresponding to the rth largest eigenvalues. 2 Thus, for a set of original facial images S Rm n , their 2 corresponding eigenface-based feature X Rr n can be obtained by projecting S into the eigenface space X = U T S . By the same procedure, we obtain texture eigenvectors ti and 2 . variances T,i Results are illustrated in Fig. 4 and Fig. 5. In order to determine the size of test images, we performed a number of experiments. Increasing the size improves the accuracy of the system, but it takes much time. We have determined the size to be 32 32 empirically.

i=1

aT i .

III. F UZZY C LUSTERING AND N EURAL N ETWORKS A. Fuzzy c-means FCM is a data clustering algorithm in which each data point is associated with a cluster through a membership degree. This technique partitions a collection of N data points into r fuzzy

3212

groups and nds a cluster center in each group, such that a cost function of a dissimilarity measure is minimized. The algorithm employs fuzzy partitioning such that a given data point can belong to several groups with a degree specied by membership grades between 0 and 1. A fuzzy r-partition of input feature vector X = {x1 , x2 , , xN } n is represented by a matrix U=[ik ], where the entries satisfy the following constraints: ik [0, 1] , 1 i r, 1 k N
r

(2) (3)

ik = 1, 1 k N
i=1 N

1) Learning algorithm: Regarding facial images for learning, a standard pattern is obtained from several patterns of each registrant. The distances of standard patterns of registrants and input patterns are calculated and the registrants are classied into several subnetworks by FCM so that similar patterns (similar persons) will be entered into the same neural network. Then one image for learning is entered into the input layer of the subnetwork to which the registrant of the image belongs. As a teaching signal, 1 is entered to a neuron corresponding to the registrant, and 0 is entered to any other neuron for learning by back propagation. This learning procedure is repeated until the sum of differences between input pattern and teaching signal becomes equal to or smaller than a certain value for all image patterns of all registrants.

0<
k=1

ik < N , 1 i r

(4)

U can be used to describe the cluster structure of X by interpreting ik as the degree of membership of ik to cluster i. A proper partition U of X may be dened by the minimization of the following cost function:
N r

} }

}
Fig. 6. Structure of proposed parallel NN

Jm (U, C ) =
k=1 i=1

(ik )m d2 ik

(5)

where m[1,+] is a weighting exponent called a fuzzier, C=[c1 ,c2 ,...,cr ] is the vector of the cluster centers, and dik is the distance between xk and the ith cluster. Bezdek [6] proved that if m 1,d2 ik > 0,1 i r, then U and C minimize Jm (U, C ).
TABLE I S UBNETWORKS AFTER PARTIAL UNIFICATION Subnet 1 Subnet 2 Subnet 3 Subnet 4 Subnet 5 Subnet 6 Subnet 7 Subnet 8 Subnet 9 Subnet 10 Subnet 11 Subnet 12 Subnet 13 Subnet 14 Subnet 15 Subnet 16 1,5,7,14,21,29 2,3,14,4,29,51 5,6,14,21,38,9 8,55,10,53,11,12 13,5,7,15,21,16 17,51,18,19,20,22 23,25,51,26,28,30 5,7,21,24,29,53 5,27,29,38,53,31 32,33,34,35,55,39 7,14,36,53,37,55 40,41,42,55,43,44 45,51,46,47,55,48 7,14,21,29,38,49 50,52,53,54,56,58 38,57,59,60

2) Recognition algorithm: Step 1: Exclusion by the negation ability of the neural network First, patterns learned by a certain neural network (subnetwork) but not reaching the maximum output are excluded. If all the element values are smaller than the threshold in a subnetwork, all patterns in this subnetwork are deleted. In this experiment, the threshold was set to 0.5. Step2: Exclusion by the negation ability of parallel neural networks A face pattern once negated in a subnetwork is deleted from the answers in other subnetworks. If all face patterns are excluded, the input face patterns are judged as those of a nonregistrant. Step 3: Judgment by the similarity method If face patterns remain after Step 2, then the similarity method is used for judgment. The similarity between the standard pattern of these face patterns and the input pattern (face pattern for recognition) is calculated. The pattern having the greatest similarity is regarded as the nal answer. If this value is smaller than the threshold of similarity, the input face pattern is regarded as that of a nonregistrant. The similarity is calculated as follows: 1 k (l )
k(l) k=1

B. Parallel neural networks In this research, the parallel neural networks used are composed of 3 layers back-propagation type neural networks. A connected NN with 2048 (32 32 2) input neurons, 128 hidden neurons, and 6 output neurons have been simulated. The structure of the proposed parallel neural networks is illustrated in Fig. 6. similarityc = (x, xk ) x xk
(l ) (l )

(6)

where k (l) is the number of trained patterns, l is the number of individuals.

3213

IV. E XPERIMENTS In this research, a device named RangeFinder Scansion is used to acquire 3D images. Experiments have been carried out using facial images of 70 individuals in Chiba university (60 individuals are selected as registered persons and the others are used for nonregistered persons). Each person provides 5 facial images with varied poses. In our system, for each individual, 3 images were selected for each subject as the learning set, and the rest of the images are taken as the test facial images for recognition. A. Computing by FCM Eqs (3) and (4) were calculated repeatedly with the membership function dened in Chapter III. From the denition, the sum of each line was 1. The last number of clusters was 51. 1) Defuzzication of lines: The maximum number of clusters to which a pattern may belong is set to 5. In order to reduce the number of elements, the threshold of defuzzication is set to 1/c = 0.019. Here, c is the number of the clusters, in this work, c is 51. If each element value is less than this threshold, the value of this element is set to 0. If the element value is greater than the threshold but there are more elements than the maximum number of elements in one cluster, only the top 5 elements may belong to the cluster. 2) Defuzzication of rows: The rows determines how many elements one cluster may contain. The maximum number of elements in a cluster is set to 6. The top 6 elements that are not 0 are saved and the other elements are excluded. In order to guarantee an element to belong to at least one cluster, an element that is not among the top 6 must be saved only if the value of this element in other clusters are 0 in the same line. 3) Arrangement of subnetworks: In order to reduce the amount of calculation, some subnetworks are integrated . However, the maximum number of face patterns to one subnetwork is 6. This integration produces the nal classication results given in Table I. Based on these results, face patterns can be learned and recognized using a parallel neural network system that consists of small-scale subnetworks that comprise a largescale neural network. Even when subnetworks are merged, this method can be applied. Integration of the subnetworks can reduce the amount of calculation, as shown in Table I, which enhances the efciency. B. Learning and recognition by parallel neural networks For 60 registrants, 120 face patterns are used for recognition. In addition, 50 face patterns are prepared for 10 nonregistrants to determine whether the parallel neural network system can judge that the persons are not registered. The recognition procedure is based on the algorithm presented in Section III. When this method was applied to frontal face patterns, the recognition rate was 99.17%(1 rejected among 120 patterns) for registrants and 100% for nonregistrants. Table II gives the results of recognizing whole face patterns. The results prove that the proposed method is useful for face recognition.

TABLE II R ECOGNITION RESULTS WHEN WHOLE PATTERNS ARE USED Recognition rate Rejection rate Falseness rate 99.17% 0.00% 0.83% 0.00% 100.00% 0.00%

Registered 120 Nonregistered 50

V. C ONCLUSIONS In this paper, a 3D face recognition system based on texture and shape data is presented. It can be used to identify facial images with facial pose variations. Firstly, the poses of 3D original facial images are compensated for using feature points and geometrical measurement. The texture and shape data are extracted for recognition. In the recogniton process, the face patterns are divided into several small-scale subnetworks based on fuzzy c-means. Since the fuzzy c-means permits one facial pattern to belong to several small-scale networks, patterns that are similar to each other can be recognized in the same neural network efciently. Then, the parallel neural networks are used for face recognition. R EFERENCES
[1] A.Z.Kouzani, F.He and K.Sammut, Towards Invariant Face Recognition, Information Sciences, vol.123, pp.75-101, 2000. [2] K. Lam and H.Yan, An Analytic-to-Holistic Approach for Face Recognition Based on a Single Frontal View, IEEE Trans. Pattern Analysis and Machine Intelligence, vol.20, no.7 pp.673-686, July 1998. [3] P. J. Phillips, Matching Pursuit Filters Applied to Face Identication, IEEE Trans. Image Processing, vol.7, no.8, pp.1150-1164, Aug 1998. [4] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, Face Recognition: A Convolutional Neural-Network Approach, IEEE Trans. Neural Networks, vol.8, no.1, pp.98-113, Jan 1997. [5] J. Bezdek, A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms, IEEE Trans, Pattern Analysis and Machine Intelligence, vol.PAMI-2, no.1, pp.1-8, 1981. [6] W. Pedrycz, Conditional Fuzzy Clustering in the Design of Radial Basis Function Neural Networks, IEEE Trans. Neural Networks, vol.9, no.4, pp.601-612, July 1998. [7] F. Tsalakanidou and D. Tzovaras, Use of Depth and Colour Eigenfaces for Face Recognition Pattern Recognition Letters , vol.24, no.9-10, pp.1427-1435, June 2003. [8] C.Zhang and F.S.Cohen, 3-D Face Structure Extraction and Recognition From Images Using 3-D Morphing and Distance Mapping, IEEE Trans. Image Processing, vol.11, no.11, pp.1249-1259, Nov 2003. [9] C.S.Chua and R.Jarvis: Point Signatures, A New Representation for 3D Object Recognition, International Journal of Computer Vision, vol.25, no.1, pp.63-85, Oct 1997. [10] S.Li and J.Lu, Face Recognition Using Nearest Feature Line, IEEE Trans. Neural Networks, vol.10, No.2, pp.439-443, March 1997. [11] M.A.Turk and A.P.Pentland, Face recognition using eigenfaces, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.586-591, 1991. [12] O. Nakamura, S. Mathur, Identication of Human Faces Based on Isodensity Maps, Pattern Recognition , vol.24, no.3, pp.263-272, 1991.

3214