Integrating Local and Global Features in Automatic Fingerprint Verification

Anna Vallarta Ceguerra and Irena Koprinska School of Information Technologies, University of Sydney, Sydney, Australia {anna,irena}@it.usyd.edu.au

Abstract
This paper presents a new approach for combining local and global recognition schemes for Automatic Fingerprint Verification (AFV), by using matched local features as the reference axis for generating global features. In our specific implementation, minutia-based and shape-based techniques were combined. The first one matches local features (minutiae) by a point-pattern matching algorithm. The second one generates global features (shape signatures) by using the matched minutiae as its frame of reference. Shape signatures are then digitised to form a feature vector describing the fingerprint. Finally, a LVQ neural network was trained to match the fingerprints by using the difference of a pair of feature vectors. The experimental results show that the integrated system significantly outperforms the minutiae-based system in terms of classification accuracy and stability. This makes the new approach a promising solution for biometric applications.

1. Introduction
A fingerprint is a unique identifier for any person, and can be used as a biometric. There are two applications for such a biometric. Verification involves matching two fingerprint images, in order to verify a persons claimed identity. Identification involves searching a fingerprint database for a matching fingerprint, in order to determine a persons identity based on their physical characteristics. Currently, the predominant approach for verification is to match local features, commonly minutiae, using pointpattern matching algorithms [1]. This is effective assuming that the set of minutiae is unique to that fingerprint alone. However, errors introduced before the matching itself, such as image acquisition, falsifies this assumption. Global recognition schemes, traditionally used for fingerprint classification as part of identification, were seen to replace local techniques as these features contain more information than minutiae points. It has become more feasible for common use over the past few years, with the increasing availability of highly compact and more powerful computers. These schemes are effective when the representation is invariant to translation, rotation and

scale. These are addressed by a consistently detected reference axis, consisting of a point and a line corresponding between the two images [2]. For classification purposes, singular points have been used as a reference point. However, the detection of these points is not precise enough for matching, and it is difficult to determine the reference line with a high degree of accuracy. Due to the known weaknesses of either technique, general methods for combining any form of biometric techniques have been developed. One such method relies on the probability that the set of techniques is correct [3], however it does not consider the actual input for this decision. To the best of our knowledge, there are no proposed approaches for uniting the two techniques within a single biometric. This paper presents an approach for combining local and global features of one biometric in a clear and direct manner, creating improvements in performance. The approach, as well as a possible implementation, is described in the following section. Section 3 displays the experimental results of the performance of this particular implementation, and the final section concludes the paper.

2. Automatic Fingerprint Verification

This section is divided into two subsections. The first describes the approach that allows any local and global recognition schemes to be combined in this way. The second subsection outlines the particular algorithms used.

2.1. AFV system approach

In order to combine local and global features, the approach uses the output of the local matching algorithm as the input to the global recognition scheme. The general order is as follows: Use local feature matching to find the reference axis, consisting of a reference point and line, Use this reference axis to generate the global features, ultimately resulting in an n-dimensional feature vector, and Take the difference of these feature vectors to form the n-dimensional concept of the similarity of the pair of fingerprints, and decide if they match.

1051-4651/02 $17.00 (c) 2002 IEEE

In the first step, a reference axis is found within each image. The corresponding pair of reference points for the two images is a pair of matching local features. The reference line is drawn from the reference point to another local feature, where the local feature forms one of a matching pair. By using local features as a reference axis for global features, the two schemes are combined. There are two advantages in combining local and global features in this fashion. Firstly, by creating a reference axis derived from matching pairs of local features, it is guaranteed that the global features are consistently generated. Moreover, accuracy of the global scheme clearly builds on that of the local matching algorithm, as the former refines the decision of the latter. The second step involves the creation of the feature vector from the raw global features. The set of raw global features themselves may not be suitable as a feature vector, as each feature may represent a single pixel in the image. Since each entry in the feature vector must correspond to the same feature, regardless of the image, this suggests a specialised form of compression of the global features. This transformation may be highly dependent on the type of global features in use. The final step provides a form of abstraction, from describing two fingerprints, to describing the similarity between them. Learning techniques, such as neural networks, can be used to learn to make the distinction between matching and non-matching fingerprints.

of the counterclockwise sequence of its adjacent minutiae {pik : k = 1, , K}, where two minutiae are defined to be adjacent if their Voronoi cells share a border (Figure 1a).
Pi Pi
3 2

Pi

d3 a3 d2 a4 Pi a5 d5 Pi
5

Pi

Pi Pi
4

Pi

a2 d1 a1 Pi
1

d4 Pi
4

Pi

Figure 1: Representing the minutia pi: a) adjacent minutiae of pi defined by the Voronoi tesselation, b) parameters used to generate the representation of pi The position of the adjacent minutia pik is expressed in terms of the minutia pi, by finding the distance (dk) between pik and pi, and the polar angle (k) between pik and pik 1 with respect to pi (Figure 1b). The respective distances and polar angles are combined and normalised, to form a sequence of complex numbers, and the Discrete Fourier Transform (DFT) is applied to this sequence: d exp{j k } f(pi) = DFT{uk : k = 1, , K}, uk = k K (1) dl
l =1

2.2. AFV system implementation

Both the local and global features were extracted from a thinned fingerprint ridge pattern. These thinned ridges were found using the following steps. Firstly, preprocessing of the images were performed using Gabor filters, in order to remove noise and enhance the fingerprint ridge pattern. Segmentation limited the image to those areas that contain ridge information. This image was then binarised, creating a black and white image from a grey-scale image. Finally, skeletonisation of the binarised image was performed, where ridges with a width of multiple pixels were simplified to ridges of single pixel width. 2.2.1. Generate the reference axis For the first step, the detected minutiae were matched using a point-pattern matching algorithm, and each matched pair of points created a reference axis. Minutiae, represented as single pixels in the image, were detected by examining the neighbourhood around each pixel of the thinned ridges [1]. The position of a pixel was recorded as a ridge termination, if its neighbourhood contained only one other pixel, or ridge bifurcation if it contained three other pixels. Following [4], each minutia (pi) is represented in terms

The matching algorithm in [4] was also implemented. It makes use of a function, which quantifies the similarity between two minutiae pm and qo. Our implementation extended this function, by incorporating a check of the type (termination or bifurcation) of the minutiae. This similarity function is defined below: 0 if dim{f ( p ) } m } dim{f (q o ) s(pm, qo) = otherwise The set of matching pairs of minutiae C is found by determining, for each minutia pm in one fingerprint, the best matching minutia qo is found in the other fingerprint. Another constraint for this pair to be in C is that they must be sufficiently similar, that is the similarity value of the two minutiae must occur above a threshold T: (j = 1, , M )(k = 1, , O ) s ( p m , qo ) s ( p j , qo ) C = ( pm , qo ) : (3) ( ) ( ) s p , q s p , q m o m k ( ) s p , q T m o Each matched pair of minutiae (pm, qo) forms a reference point that corresponds between the two fingerprints. Another pair of matched minutiae (pmj, qok) is found by finding correspondences between the adjacent minutiae of pm and qo, and these two points within each image forms the reference line. In this way, a matching pair of reference axes (refpm, refqo) is found.
2 *

0 if type( p m ) type(q o ) f ( p ) f (q ) m o exp

(2)

1051-4651/02 $17.00 (c) 2002 IEEE

2.2.2. Generate the feature vector Within the second step, the shape signatures for both fingerprints were generated using the matching pair of reference axes, then digitising to form the feature vector. The shape signature, or turning function, represents a two-dimensional boundary as a one-dimensional function [5]. It is generated in terms of the reference axis (refpm) corresponding to that fingerprint. For each pixel (lu) on the thinned ridges, the distance (ru) from the reference point, the clockwise angle (u) from the reference line, and the average tangent angle (tu) are calculated. To compute tu, the ridge pixels {luk : k = 1;;K} in the neighbourhood of lu are first found. Then, tu is the average of the tangent angles tuk of lu defined below, where tuk is made by the 8 neighbouring pixels of lu in the shape signature space: tuk = ( arg[(ru - ruk) + j(u - uk)] ) mod (4) Finally, the n-dimensional feature vector is calculated, by digitising the shape signature. Digitisation compresses the amount of information in the shape signature, while remembering the general shape as the clockwise angle changes. This is done using a hybrid of the compression approaches presented by [2, 6, 7]. The entire set of ridge pixels {lu : u = 1, , M} is divided into n subsets, where lu belongs to the ith subset if u is within the range of [(i 1) / 2n , i / 2n ] The average of all tu for the ith subset is then calculated, forming the ith entry in the n-dimensional feature vector. Figure 2 below is a visual summary of this step.

tures their similarity. This difference is an input to a Learning Vector Quantisation (LVQ) neural network (NN) [8] that was trained to distinguish between a match and non-match. LVQ is a supervised competitive algorithm. Given a set of pre-classified feature vectors (training examples), it creates a few prototypes for each class, adjusts their positions by learning and then classifies the unseen examples by the nearest-neighbour principle. As there are several possible reference axes, two fingerprints match if at least one is a match concept.

3. Experimental Results
The goal of the experiments was to compare the performance of the combined AFV system (minutiae matching and shape signature analysis) with the minutiae matching alone. AFV local was implemented according to the algorithm in Sec. 2.2.1. AFV combined is the complete system described in Sec. 2. Both use T = 0.9 and = 0.1. Three databases were used for these comparisons: hioptical, synthetic and db-ipl. The first two are Set B of DB3 and DB4 respectively, described in [9]. Both contained 10 fingers, each with 8 impressions, and a proportion of 0.088 matching pairs. The Image Processing Laboratory, University of Trieste, Italy, captured the last database, containing 16 fingers with 8 impressions each, and 0.055 matching pairs. The NN was trained on the first database only, and tests were performed on all. The LVQ network architecture was determined using stratified ten-fold cross validation, by comparing the performance of different combinations of feature vector size and number of neurons within the competitive layer. The best combination was a feature vector size of 24, and 12 competitive neurons. Of the two output neurons, the first represents a match, and the other a non-match.
Accuracy Class +ve Class -ve Recall (TAR) Prediction +ve TA FA matched Specificity (TRR) didn't match Prediction -ve FR Predictive value +ve TR should shouldn't match match all pairs Predictive value -ve (TA + TR) / (all pairs) (TA) / (should match) (TR) / (shouldn't match) (TA) / (matched) (TR) / (didn't match)

Table 1: Performance measures Figure 2: Generating the feature vector: a fingerprint with the reference axis (top), its shape signature (middle), its digitised shape signature (bottom), with the direction of each entry of the feature vector. 2.2.3. Determining the similarity Once the feature vectors for both fingerprints are generated, taking the difference of two feature vectors capFive performance measures were used for this comparison, and are described in Table 1 above. Most performance measures within fingerprint recognition only make use of TAR and TRR. However, the final two values measure the trustworthiness of the output of the AFV system, and are most useful when they are compared with the proportion of matches and non-matches respectively. To determine the performance of the AFV combined system over all T levels, an ROC curve was generated. This was done by calculating the TAR and FAR of the system, at a specific threshold level, and plotting the TAR

1051-4651/02 $17.00 (c) 2002 IEEE

against the FAR. For this experiment, threshold levels appear between 0 and 1, at intervals of 0.01.
Accuracy TAR hi-optical AFV local AFV combined synthetic AFV local AFV combined db-ipl AFV local AFV combined 0.8554 0.9032 0.8883 0.9079 0.9305 0.9445 0.1286 0.0357 0.0393 0.0107 0.0491 0.0424 TRR 0.9260 0.9875 0.9708 0.9951 0.9819 0.9971 "+ve" 0.1446 0.2174 0.1158 0.1765 0.1366 0.4634 "-ve" 0.9162 0.9133 0.9122 0.9119 0.9465 0.9470

4. Conclusions
A new approach for combining local and global features for AFV systems was developed. It uses matched local features as the reference axis for generating global features. Any local and global recognition schemes can be combined in this way. In our implementation, minutiabased and shape-based algorithms were integrated. The first one matches local features by using a point-pattern matching algorithm. The second one generates global features by using the matched minutiae as its frame of reference. The difference of two feature vectors, describing the corresponding global features of two fingerprints, was used to train a LVQ NN to distinguish between matching and non-matching fingerprints. Experimental evaluation shows that the integrated system outperforms the local one on various accuracy measures, with significant improvement in the trustworthiness. It also confirms the ability of the trained NN to have consistent performance on unseen databases. Hence, this approach has clear potential in AFV systems that require excellent accuracy. However, the best combination of local and global recognition schemes is still to be found.

Table 2: Comparison of performance. Table 2 shows the comparisons between two AFV systems. The complete AFV system has equal or better performance on most performance measures, with the most significant improvement on +ve, or the trustworthiness of the match decision. The only measure with no improvement is TAR, since this approach cannot increase the number of matches it only refines the match decision. It can also be seen that this improvement is consistent over all databases, regardless of whether the NN has been trained on the dataset or not, which confirms the high generalization ability of LVQ.
ROC curve
1 0.9 0.8 0.7 0.6 TAR 0.5 0.4 0.3 0.2 0.1 0 0 0.2 0.4 FAR 0.6 0.8 1

5. Acknowledgements
We are very grateful to Fabio Vitali from the University of Trieste, Italy and Piero Calucci from Tender S.p.A., Trieste, who provided the implementations for the pre-processing and minutiae extraction algorithms.

References
[1] [2] A. Jain, L. Hong, R. Bolle, "On-line Fingerprint Verification," IEEE trans. on PAMI, v. 19, pp. 302-314, 1997. A. K. Jain, S. Prabakhar, L. Hong, S. Pankanti, "FingerCode: A Filterbank for Fingerprint Representation and Matching," In Proc. CV&PR Conf., Fort Collins, 1999. S. Prabakhar and A. K. Jain, "Decision-level Fusion in Biometric Verification," TR MSU-CSE-00-24, 2000. M. Tico, C. Rusu, and P. Kuosmanen, "A Geometric Invariant Representation for the Identification of Corresponding Points," In Proc. ICIP, pp. 462-466, 1999. S. Loncaric, "A survey of shape analysis techniques," Pattern Recognition, vol. 31, pp. 983-1001, 1998. B. Logan and A. Salomon, "A Content-Based Music Similarity Function," TR CRL 2001/02, June 2001. J. L. Blue, G. T. Candela, P. J. Grother, R. Chellappa, and C. L. Wilson, "Evaluation of Pattern Classifiers for Fingerprint and OCR Applications," Pattern Recognition, v. 27, pp. 485-501, 1994. T. Kohonen, Self-Organizing Maps: Springer, 2001. D. Maio, D. Maltoni, R. Cappelli, A. K. Jain, and J. L. Wayman, "FVC2000: Fingerprint Verification Competition," BIOLAB, University of Bologna, Italy 2000.

Figure 3: ROC curve of the combined AFV system Another interesting observation is that AFV local has trustworthiness values that are close to the actual proportions within the database, whereas for AFV combined, these values are higher than the actual proportions. This suggests that the trustworthiness values of the combined system are not dependent on the proportions within the database, while AFV local shows this dependency. Figure 3 is the ROC curve of the complete AFV system described in Section 2, showing an Equal Error Rate (EER) of 4. This is an average performance, when compared to those in the FVC2000 benchmark [9]. However, the implementation could be highly improved by a better combination of local and global recognition schemes, where the minutiae matching algorithm has a high TAR and the global scheme has a high accuracy overall.

[3] [4]

[5] [6] [7]

[8] [9]

1051-4651/02 $17.00 (c) 2002 IEEE