Figures
Abstract
Establishing correspondences between two minutia sets is a fundamental issue in fingerprint recognition. This paper proposes a new tensor matching strategy. First, the concept of minutia tensor matrix (simplified as MTM) is proposed. It describes the first-order features and second-order features of a matching pair. In the MTM, the diagonal elements indicate similarities of minutia pairs and non-diagonal elements indicate pairwise compatibilities between minutia pairs. Correct minutia pairs are likely to establish both large similarities and large compatibilities, so they form a dense sub-block. Minutia matching is then formulated as recovering the dense sub-block in the MTM. This is a new tensor matching strategy for fingerprint recognition. Second, as fingerprint images show both local rigidity and global nonlinearity, we design two different kinds of MTMs: local MTM and global MTM. Meanwhile, a two-level matching algorithm is proposed. For local matching level, the local MTM is constructed and a novel local similarity calculation strategy is proposed. It makes full use of local rigidity in fingerprints. For global matching level, the global MTM is constructed to calculate similarities of entire minutia sets. It makes full use of global compatibility in fingerprints. Proposed method has stronger description ability and better robustness to noise and nonlinearity. Experiments conducted on Fingerprint Verification Competition databases (FVC2002 and FVC2004) demonstrate the effectiveness and the efficiency.
Citation: Fu X, Feng J (2015) Minutia Tensor Matrix: A New Strategy for Fingerprint Matching. PLoS ONE 10(3): e0118910. https://doi.org/10.1371/journal.pone.0118910
Academic Editor: Wen-Bo Du, Beihang University, CHINA
Received: May 8, 2014; Accepted: January 12, 2015; Published: March 30, 2015
Copyright: © 2015 Fu, Feng. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Comparative experiments are conducted on the International Fingerprint Verification Competition (simplified as FVC) databases: FVC2002 and FVC2004 databases. They are international competition databases for fingerprint verification algorithms, which are open to all researchers (FVC2002 can be downloaded in http://bias.csr.unibo.it/fvc2002/ and FVC2004 can be downloaded in http://bias.csr.unibo.it/fvc2004/).
Funding: The study was granted by National 973 project (No. 61333015, No. 2011CB302400) in China. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Fingerprint matching is a classical and hot topic in computer vision and pattern recognition [1]. Researchers have set up a series of special fingerprint verification competition databases [2] [3]. Establishing correspondences between fingerprint minutia sets is a fundamental issue in fingerprint recognition. It is challenging to find perfect correspondences for fingerprints due to various reasons: non-linear distortion, partial overlap, noise and so on.
Many minutia-based fingerprint matching papers have been published these years. They can be mainly classified into two categories. The first category of these papers is to exact more matching features besides minutia locations and orientations. Jain et al. used pores and ridge contours besides minutia points and proposed a three-level (level 1: pattern, level 2 minutia points and level 3: pores and ridge contours) matching strategy [4]. Liu et al. proposed the minutiae phase difference feature to describe the minutiae [5]. Choi et al. incorporated ridge features with minutiae and achieved good results [6]. Izadi et al. introduced cylinder quality measure into minutia cylinder-code based fingerprint matching [7]. Wen et al. applied a set of global level minutia dependent features, including minutia qualities and the area of overlapping regions [8]. Thuy et al. increased the ridge-valley structure features and improved the classical Thin-Plate-Spline deformation model [9], and they proposed the local Thin-Plate-Spline to deal with non-linear distorted fingerprints [10]. In summary, these papers are trying to seek for more matching features to improve the effectiveness.
The other category of these papers just uses the minutia features and tries to construct more complex structures or to find more efficient algorithms. Since only the minutia features are used, this category of approaches faces greater challenges. Previous pioneers have achieved big progress. Ratha et al. proposed a local similarity calculation algorithm for local matching [11]. Chen et al. constructed the minutia topological structure to increase the tolerance for noise [12]. Chikkerur et al. proposed K-PLET structures and a graph-based representation for minutia matching [13]. Xu et al. grew minutia triangle structures into a growing area and then performed the global match by fusing them[14]. Cappelli et al. proposed a novel representation based on 3D cylinder structures, called Minutia Cylinder-Code [15]. Medina-Pérez et al. improved fingerprint verification by using minutia triplets [16]. In summary, minutia topologic structures constructed in local areas are considered to be less impacted by nonlinear distortion and noise. However, as the radius of minutia structures is not big, it limits the use of global minutia information. Based on this consideration, some researchers try to introduce global information into the matching process. Tea et al. presented an approach based on convex hull to deal with incomplete or partial fingerprints [17]. Feng et al. proposed the compatibility of local minutia pairs and adopted a relaxation iterative process [18]. Cao et al. applied compatibility to local minutia star-structures and achieved better results [19]. All these papers are trying to construct more complex minutia topologic structures or seek for more efficient matching strategies.
Establishing correspondences between two feature sets is a fundamental issue in computer vision and pattern recognition [20]. Horaud et al. proposed a graph matching framework and formulated point matching as recovering the maximal cliques in the correspondence graph [21]. Duchenne et al. proposed the affinity tensors to describe high-order affinities between each feature pairs [22]. First-order tensor describes similarities of each feature pair. Second-order tensor describes compatibilities between each two feature pairs. Leordeanu et al. proposed a spectral technique for correspondence problems using pairwise constraints [23]. They recovered the matching relationship based on spectral matching methods, by using the principal eigenvector of adjacency matrix and imposing the one-to-one mapping constraints. This notion can be introduced into minutia matching.
Based on these insights, we propose a tensor matching strategy. We construct the minutia tensor matrix (simplified as MTM) for fingerprint minutiae. It unifies both the first-order features and the second-order features. The diagonal elements in MTM indicate similarities of each minutia pair and other elements indicate pairwise compatibilities between minutia pairs. Correct minutia pairs are more likely to establish both large similarities within them and large compatibilities among them, thus they form a dense sub-block. Incorrect pairs establish links with the other pairs accidentally, so they are unlikely to belong to dense sub-blocks (The definition of dense sub-block can be seen in section 2.1). Minutia matching is formulated as recovering the main dense sub-block in the MTM.
Our previous papers tried to apply the tensor idea to fingerprint matching [24] [25]. We applied the spectral matching strategy to fingerprint global matching [24]. The disadvantage was that it was only applied to global matching, while other strategies were still required for local matching. We proposed extended clique models to deal with local matching and global matching [25]. The disadvantage is that the local matching process is time-consuming. Meanwhile, local matching is not associated with global matching. This paper is innovative on the basis of previous papers. First, the tensor strategy is used in local matching for the first time. Second, by use of the proposed “MTM”, local matching and global matching are unified into an integrated framework.
Main contributions of this paper are: First, the proposed MTM unifies both similarities and compatibilities appropriately. Second, minutia matching is formulated as recovering the dense sub-block in the MTM. Optimal matching relationship corresponds to the dense sub-matrix in the MTM. It gives a clear mathematical meaning for “optimal matched pairs”. Third, spectral matching methods are then used for recovering the dense block. It is efficient and effective, which can be seen in the following experiments. Forth, two MTMs with different constraints are constructed, which makes use of local rigidity and global compatibility, respectively. In the local matching level, we build the local minutia topologic structures and construct local MTM for each minutia structure pair. Calculating similarities of minutia pairs corresponds to recovering the dense sub-block in the local MTM. In the global matching level, we construct the global MTM for entire minutia sets. Calculating similarities of minutia sets corresponds to recovering the dense sub-block in the global MTM. Proposed approach has stronger description ability and better robustness to non-linear deformation and noise.
Materials and Methods
2.1 Problem formulation
Suppose there are two fingerprint minutia sets P and P', with Np and Np' minutiae, respectively. We define two attributed graphs GP = (V,E,A) for minutia set P and GP' = (V',E',A') for minutia set P'. Each edge e = ij ∊ E in GP is assigned an attribute Aij, which is the distance vector between minutia i and minutia j in P. We represent node attributes as special edge attributes, i.e. Aii for node i. So it is with P'. Let SA(ii', jj') indicate the similarity score between attributes Aij and A'i'j'. We want to find a mapping that best preserves the attributes of nodes and edges between attributed graphs GP and GP'. Equivalently, we seek for a set of correct minutia pairs M = (ii',jj',kk'…) so as to maximize the matching score, defined as:
High-order tensors are constructed to represent high-order affinities [22]. Inspired by this notion, we propose the minutia tensor matrix (MTM). It is proposed based on these considerations: First-order tensor matrix describes similarities of each minutia pair. Second-order tensor matrix describes compatibilities between each two minutia pairs. For fingerprint minutia sets P and P', each minutia pair (i, i') is assigned an similarity attribute T1 (i, i') and each two minutia pairs (ii', jj') is assigned a compatibility attribute T2 (ii', jj'). (Calculation of T1 (i, i') and T2 (ii', jj') can be seen in section 2.3 and 2.4). Thus we can build the first-order tensor matrix T1 and the second-order tensor matrix T2. Here we don’t use tensor matrixes with three or more orders as the computation is unacceptable. After that we fuse T1 and T2 into a fused high-order tensor matrix TF, which is also the proposed MTM. The fusion rule is shown in the formula:
(2)
Where T1 is the first-order tensor matrix, whose element T1 (i, i') describes the similarity of minutia pair (i, i'). T2 is the second-order tensor matrix, whose element T2 (ii', jj') describes compatibility between (i, i') and (j, j'). The fused tensor matrix TF (MTM) is a two-dimensional matrix with (Np × Np') × (Np × Np') elements. It unifies both similarities and compatibilities among minutia pairs. When the condition (i = i' and j = j') holds, minutia pairs (i, i') and (j, j') are the same pair, thus T1 (i, i') = T1 (j, j'). As minutia matching is one-to-one matching, minutia i in P mapped to both minutia i' and j' in P' is impossible. When (i = j and i' ≠ j') or (i ≠ j and i' = j') holds, T2 (ii', jj') is 0.We use this priori knowledge to prune and can get a very sparse MTM. The parameter λ is a weighting factor. As T1 and T2 have different dimensions, we use λ to adjust the values. It is set manually. TF is a symmetric matrix as TF (ii', jj').
As shown in Fig. 1, for two minutia sets P and P', we can build the correspondence graph GPP'. In graph GPP', the dense subgraph GA is the biggest strong-connected subgraph in GPP', whose nodes have strong links with all the other nodes. Correct minutia pairs are likely to establish links among each other and thus form a strongly connected subgraph, which is also the dense subgraph. Incorrect pairs establish links with the other pairs only accidentally. Minutia matching can be formulated as seeking for the dense subgraph in the correspondence graph.
The fingerprint images are from FVC2004 DB1 27–3 and 27–5. For clarity, only a small subset of minutia pairs are shown. Candidate minutia pairs shown in (a) form the correspondence graph in (b) and the minutia tense matrix (MTM) in (c). Genuine minutia pairs corresponds to the dense subgraph of correspondence graph, and also the dense block of MTM. Minutia matching is formulated as recovering the dense sub-block in the MTM. It can be solved by the spectral correspondence methods.
The correspondence graph GPP' corresponds to the MTM TF. In TF, the dense sub-block TD is the biggest sub-matrix in TF, whose elements in all the rows and all the columns have big values. The element values in the dense sub-block TD should be greater than a certain threshold. As shown in Fig. 1, for minutia sets P and P', we can build the MTM TF of the correspondence graph GPP'. Correct pairs are likely to establish elements with big values among each other and thus form a strongly connected cluster, which is also the dense sub-block TD. Minutia matching corresponds to recovering the dense sub-block.
Equivalently, we seek for a set of correct matching pairs M = (ii', jj', kk'…), so as to maximize the MTM score, defined as:
The main idea is illustrated in Fig. 1. The optimization problem (3) can be solved by spectral matching method, which will be discussed in section 2.2.
2.2 Spectral matching methods
Formula (3) yields to the following binary optimization problem:
(4)
The optimal solution is the binary vector that maximizes the mTTFm score. m is a row-wise vectored replica of the MTM TF. It can be derived by using the principal eigenvector of TF and imposing the one-to-one mapping constraints, which is proposed in [23].
Here we make some improvements. Firstly we prune the elements with small similarities or low compatibilities. Secondly we relax both the mapping constraints and the integral constraints on m, so that its elements can take real values in [1]. Thirdly we don’t initialize m0 (the initialization value of m) randomly like [23], we use some priori knowledge. As minutia pairs with big local structural similarities are more likely to have large compatibilities, we initialize m0 by normalized first-order tensor matrix T1. It converges much faster using this strategy.
We can derive m* by computing the leading eigenvalue of TF, using Algorithm 1.
Algorithm 1 Spectral relaxation iterations.
Input: minutia tensor matrix TF
Output: m*, the main eigenvector of TF
1 initialize
2 repeat
3 ↓m* ← TFm*;
4;
5 until convergence;
Indeed, each step of the iteration algorithm requires only O (z) operations, where z is the number of non-zero elements of the matrix. Also, typically, in our situation, the algorithm converges in a few dozen steps.
Given the continuous solution m*, we can rewrite it in a Np × Np' matrix form and discretize m* by a greedy approach and derive the optimal solution. Because of the existence of partial overlapping, only a subset of size NS ≤ min(Np, Np') can be matched. Here we set the m* elements whose value is smaller than the pre-set threshold δ to zero. The parameter δ is set manually. Meanwhile, minutia matching is one-to-one matching, minutia i in P mapped to both minutia i' and j' in Q is impossible. We add this compatibility constraint during the iteration process. It is summarized as Algorithm 2.
Algorithm 2 Greedy approach for discretization.
Input: continuous matrix m*
Output: binary matrix
1 initialize
2 set m* (i,j) = 0, ∀i, ∀j, m*(i,j)< δ
3 repeat
4 ↓ find the maximal element m*(i,j)
5 ↓ set, ∀k, set m*(i,k) = 0,m*(k,j) = 0
6 until m* = 0;
The matching constraints (m1 ≤ 1 and mT 1 ≤ 1) are ignored in the spectral relaxation step and then are induced during the discretization step. Corrected matched pairs can be gained and similarities of minutia sets can be evaluated. We can get the number Nm of matched pairs.
After that we adjust the similarity score with the number of matched pairs. Take minutia sets P and P' for example, the similarity SPP' is calculated using the following formula:
Sii' measures the similarity of matched pairs ii'. indicates the maximum value of the optimization problem shown in formula (3). Nm indicates the number of matched pairs, and Np and Np' indicate the minutia number of P and P'.
In practice, we use the tensor matching strategy both in local matching level and global matching level. There are some differences in these two levels. We will give detailed descriptions in section 2.3 and 2.4.
2.3 Local minutia tensor matrix (Local MTM)
Local minutia topologic structure (simplified as LMTS) is firstly introduced in [12]. A typical LMTS is constructed from a center minutia and a list of neighboring minutiae in a specified area. Fingerprint images show rigidity within the range of LMTS. Each LMTS can be seen as a small minutia set. Here we propose the local MTM for minutia structures and design a novel strategy to calculate similarities of LMTS pairs.
As shown in Fig. 2, minutia a and i, j, k form a minutia topologic structure La, minutia a' and i', j', k' form a minutia topologic structure La'.
The fingerprint images are captured from FVC2004 DB1 6–2 and 6–5. Minutia pairs within two local minutia topologic structures shown in (a) form the local correspondence graph in (b) and the local minutia tense matrix (local MTM) in (c). Correct pairs corresponds to the dense subgraph of local correspondence graph, and also the dense block of local MTM. Minutia matching is formulated as recovering the dense sub-block in the local MTM.
According to the coordinates and orientations of minutiae in La, we calculate the distance vector between each two minutiae within La, so it is with La'. Each candidate minutia pair (i, i') is assigned a local similarity attribute LT1 (i,i') and each two minutia pairs (ii', jj') is assigned a local compatibility attribute LT2 (ii', jj'). They can be calculated in the following formula:
(7)
(8)
Where
is the distance vector between minutia a and minutia i.
have similar meanings. Here
. D (,) measures the 1-norm distance of vectors
and
. Sii' indidates the similarity of minutia pair (i, i'). Cii',jj' indidates the compatibility between pairs (i, i') and (j, j'). α, α', β, β' are positive parameters. They are set manually.
We calculate all the similarities of minutia pairs in LT1 and all the compatibilities of each two minutia pairs in LT2. Thus, we can construct the local first-order tensor matrix LT1 and the local second-order tensor matrix LT2. The local fused tensor matrix LTF (local MTM) can be constructed using the method proposed in formula (2). Calculating similarities of La and L'a is formulated as recovering the dense sub-block in the local minutia tensor matrix LTF.
In experiments, we make some pruning after building the local MTM. Suppose there are two minutia topologic structures La,La', with na, na' minutiae, respectively. Because of the one-to-one mapping constraint, only subsets of size ns ≤min (na, na') can be matched. We retain nr = 3 * min (na, na') candidate minutia pairs with the maximal similarity values. The pruned local MTM has only nr * nr elements, which is much smaller than the original scale (na * na') * (na, * na'). Then the optimal matching pairs within LMTS can be gained through spectral methods proposed in section 2.2.The spectral iteration process is very fast. It usually will converge in several iterations.
After relaxation and discretization, we can gain the optimal matched pairs and the number nm of matched pairs. After that we adjust the similarity score with the number of matched pairs. The similarity of LMTS pairs La and La' can be calculated using the following formula.
(10)
indicates the maximal score of the local MTM LTF. nm indicates the number of matched pairs within (La, La'), and na indicates the number within LMTS La, na' has a similar meaning.
2.4 Global minutia tensor matrix (Global MTM)
The whole minutia set can be seen as a large global minutia topologic structure and the local minutia topologic structures (LMTSs) are basic matching units. Fingerprint images show nonlinearity in the global range. Minutia pairs far apart may not look so compatible. Here we propose the global MTM for minutia sets and design a novel strategy to calculate similarities of fingerprints.
As shown in Fig. 3, La,Lb,Lc,Ld are four minutia structures in minutia set P, La',Lb',Lc',Ld' are four minutia structures in minutia set P'. For clarity, only a small subset of the candidate LMTS pairs is shown. LMTS pair (La,La') is assigned a global similarity attribute GT1 (La, La') and each two minutia pairs (LaLa',LbLb') is assigned a global compatibility attribute GT2(LaLa',LbLb').
The fingerprint images are from FVC2004 DB1 27–3 and 27–5. For clarity, only four minutia structure pairs are shown. Candidate structure pairs shown in (a) form the global correspondence graph in (b) and the global minutia tense matrix (global MTM) in (c). Correct structure pairs corresponds to the dense subgraph of global correspondence graph, and also the dense block of global MTM. Minutia matching is formulated as recovering the dense sub-block in the global MTM.
Fingerprint images show nonlinearity in the global range. Minutia pairs far apart may not look so compatible. We lower the compatibility threshold and take nonlinearity into account. The first-order tensor GT1(La,La') can be evaluated using the structural similarity gained in section 2.2. According to the coordinates and orientations of LMTS center minutiae, we can calculate the distance vector
between each two center minutiae, such as
. The compatibility GT2(LaLa',LbLb') can be evaluated using the two pairs of center minutiae (aa', bb'). They can be calculated in the following formula:
(11)
(12)
Where
indidates the similarity of LMTS pair (La, La') gained in section 2.3.
is the distance vector between center minutiae a and b.
has a similar meaning. Caa',bb' indidates the compatibility between pairs (a, a') and (b, b'). γ, γ' are positive parameters. They are set manually.
We calculate all the similarities of LMTS pairs in LT1 and all the compatibilities of each two LMTS pairs in LT2. Thus, we can construct the global first-order tensor matrix GT1 and the local second-order tensor matrix GT2. The global fused tensor matrix GTF (global MTM) can be constructed using the method proposed in formula (2). Calculating the similarity of minutia sets P and P' is formulated as recovering the dense sub-block in the global minutia tensor matrix GTF.
In experiments, we make some pruning after building the global MTM. Suppose there are two minutia sets P and P', with Np and Np' minutiae, respectively. Because of the one-to-one mapping constraint, only subsets of size NS ≤ min (Np, Np') can be matched. We retain Nr = 3 * min (Np,Np') candidate minutia pairs with the maximal similarity values. The pruned global MTM can be gained. It has only Nr * Nr elements, which is much smaller than the original scale (Np * Np') * (Np * Np'). Then the optimal LMTS pairs can be gained through spectral methods proposed in section 2.2.The spectral iteration process is very fast. It usually will converge in several iterations.
After relaxation and discretization, we can gain the optimal matched pairs and the number Nm of matched pairs. After that we adjust the similarity score with matched pairs. As isolated minutiae or those minutiae in the border of fingerprint images have few surrounding minutiae, they may be removed at the step of constructing LMTSs. It is necessary to regain the lost matches. We use the moving least squares (MLS) model proposed in [26] to regain lost genuine minutia pairs. First, the parameters of MLS deformation are estimated from the matched minutia pairs. Then the estimated parameters are used to transform the model minutia set to the target minutia set. Third, a tolerance box is adopted to scan each minutia. When two minutiae from different fingerprints drop into the same tolerance box, they are judged as matched. For fingerprints from the same finger, many genuine minutia pairs can be retrieved. For fingerprints from different fingers, few minutiae can be regained as the randomness of minutiae’s positions. In this way we can get the final matched pairs.
Here we design a new scoring strategy to get the final matching scores. We adopt the convex hull strategy proposed in [17] to adjust the final similarity score. The convex hull can be gained and the number of unmatched points within the convex hull can be calculated. The evaluation strategy is based on convex hull and can be represented as follows:
(14)
indicates the maximal score of the global MTM GTF. Nm indicates the number of matched points, and mp indicates the number of unmatched points within the convex hull in minutia set P and mp' indicates the number of unmatched points within the convex hull in minutia set P'. It is more reasonable to adopt this new score evaluation strategy.
Algorithm
In this section, we describe our proposed algorithm in detail. It mainly contains the following parts: initialization, local matching and global matching.
Initialization
Suppose there are two minutia sets P and P', with Np and Np' minutiae, respectively. For each minutia i in P, we construct the LMTS with i as the center minutia and a list of neighboring minutiae in the circular area. In this way, Np LMTSs in P can be constructed. We take the same operations in P' and Np' LMTSs can be constructed, too. According to the coordinates and orientations of minutiae, we calculate all the distance vectors within minutia P and all the distance vectors
within minutia P'. We calculate all the similarities of minutia pairs (i, i'), ∀i ∊ P, ∀i' ∊ P' using the formula (7). Meanwhile, we calculate all the compatibilities of each two pairs (ii', jj'), ∀i,j ∊ P, ∀i', j' ∊ P' using the formula (8). In this way, we can gain Np × Np' similarities and (Np × Np') × (Np × Np') compatibilities in all.
Local matching
For minutia sets P and P', with Np and Np' minutiae, we construct Np and Np' LMTSs, respectively, thus we gain Np × Np' LMTS pairs. For each LMTS pair from 1 to Np × Np', we conduct a local matching operation.
The local matching operation contains three steps:
(a) Building the local MTM: For the LMTS pair (La, La'), we select the corresponding minutia pairs within La and La'. (The similarities and compatibilities have been calculated in the initialization step and we just need to select them.) We build the local first-order tensor matrix LT1 and the local second-order tensor matrix LT2. Next we construct the local MTM LTF using formula (2).
(b) Spectral matching: Calculating the similarity of LMTS pair (La, La') yields to the optimization problem shown in formula (9). We use the spectral matching methods proposed in section 2.2 to deal with it. The maximal value and optimal matching pairs can be gained.
(c) Scoring: After gaining the maximal and the optimal matching pairs, we calculate the similarity
of LMTS pair (La, La') using formula (10).
After Np × Np' local matching operations, we can gain all the similarities of candidate LMTS pairs.
Global matching
(a) Building the global MTM: After gaining the similarities of all candidate LMTS pairs, we can build the global first-order tensor matrix GT1. For each two LMTS pairs (LaLa', LbLb'), we evaluate their compatibility through their center minutiae (aa', bb'). The calculation function can be seen in formula (12). In this way, (Np × Np') × (Np × Np') LMTS compatibilities can be calculated and the global second-order tensor matrix GT2 can be built. Next we construct the global MTM GTF with (Np × Np') × (Np × Np') elements using formula (2).
(b) Spectral matching: Calculating the similarity of minutia sets P and P' yields to the optimization problem shown in formula (13). We use the spectral matching methods proposed in section 2.2 to deal with it. The maximal value and the optimal LMTS pairs can be gained.
(c) Regaining: After calculating the maximal value and the optimal LMTS pairs, we regain lost genuine pairs using the MLS strategy. In this way the final matched pairs can be gained.
(d) Scoring: After gaining the maximal value and the final matched pairs. We adopt the convex hull strategy to adjust the final similarity score. We calculate the entire similarity SPP' of minutia sets P and P' using formula (14).
The novel fingerprint matching algorithm is summarized as algorithm 3.
Algorithm 3 High-order tensors matching algorithm.
Input: Two fingerprint minutia sets P and P', with Np and Np' minutiae, respectively.
Output: The entire similarity SPP' of minutia sets P and P'
1 Initialization:
constructing local minutia topologic structures:
Np and Np'structures, respectively.
calculating similarities and compatibilities of all minutia pairs:
Np × Np' similarities and (Np × Np')×(Np × Np') compatibilities.
2 Local matching: calculating LMTS similarities
repeat: from 1 to Np × Np'
calculating similarity of LMTS pairs for LMTS pairs (La,La')
building the local first-order tensor LT1 and the local second-order tensor LT2
constructing the local minutia tensor matrix LTF
spectral matching for and matched pairs
calculating the similarity of LMTS pair (La,La')
until all candidate LMTS pairs are calculated.
3 Global matching: calculating entire similarity
building the global first-order tensor GT1and the global second-order tensor GT2
constructing the global minutia tensor matrix GTF
spectral matching for and matched pairs
regaining lost genuine minutia pairs using MLS model
constructing the convex hulls of minutia sets P and P'.
calculating the entire similarity SPP' of minutia sets P and P'
Results
4.1 Fingerprint Database
Comparative experiments are conducted on the International Fingerprint Verification Competition (simplified as FVC) databases: FVC2002 and FVC2004 databases. FVC2002 contains four different sub-databases and each database has 800 fingerprints (100 fingers and 8 impressions per finger) [2]. For each sub-database, each two impressions per finger are tested and there are 2800 true matches. The first impressions of each two fingers are tested and there are 4950 fake matches. We conduct 7750 matches in all. It is the same with FVC2004 [3]. They are international competition databases for fingerprint verification algorithms, which are open to all researchers (FVC2002 can be downloaded in http://bias.csr.unibo.it/fvc2002/ and FVC2004 can be downloaded in http://bias.csr.unibo.it/fvc2004/). The image quality of FVC2002 is good, while the image quality of FVC2004 is relatively bad. It contains much noise and nonlinearity.
For each algorithm and for each dataset, the following performance indicators are considered: Equal-Error-Rate (simplified as EER); False Match Rate (simplified as FMR); False Non Match Rate (simplified as FNMR); the lowest FNMR for FMR ≤ 0.1% (simplified as FMR-1000), average time for per match (simplified as Time) and the corresponding receiver operating characteristic curves (simplified as ROC cures). These experiments run on Intel i5–760 (2.80 GHz) processor (single-thread) and 32-bit Windows 7 systems.
4.2 Experiments
The matrix scale of local MTM plays an important role for local matching. It is directly affected by the radius of LMTS. Here we make a special experiment on FVC2004-DB1 to test the influence. We use the proposed matching algorithm and vary the radius size from 30 pixels to 160 pixels. We observe the relationship between EER, average time and the LMTS radius. Fig. 4 shows the results. The radius ranges from 30 pixels to 160 pixels. The average time increases with the radius length as the matrix scale of MTM is directly affected by the radius of LMTS. EER decreases at first and then increases with the radius. It is because that fingerprints show rigidity in local areas and nonlinearity in global areas.
In the following experiments, we set each parameter the same value for all algorithms. The LMTS radius is set to 80 pixels. The matrix fusion parameter λ in formula (2) is set to 0.1. The minimal similarity threshold δ in section 2.2 is set to e-4. The local similarity parameters α, α' in formula (7) is set to 1, 0.5, respectively. The local compatibility parameters β, β' in formula (8) is set to 2, 0.33, respectively. The global compatibility parameters γ, γ' in formula (12) is set to 2, 0.2, respectively.
Comparative experiments are conducted on FVC2002 and FVC2004 databases. In order to demonstrate the effectiveness of the new algorithm, some other matching algorithms are used, including the efficient minutia-based fingerprint matching algorithm (simplified as EMF) [8], the local structural similarity algorithm (simplified as LSS) [11], the local topologic structure algorithm (simplified as LTS) [12], the grow and fuse algorithm (simplified as GF) [14], the minutia cylinder-code algorithm (simplified as MCC) [15], the minutiae triplets algorithm (simplified as MT) [16] and the local structure compatibility algorithm (simplified as LSC) [18]. Here we use only minutia features (coordinates and orientations) for every algorithm. We do not use other matching features such as qualities or ridges to ensure the comparison effectiveness.
The proposed algorithm has two levels: local MTM matching and global MTM matching. We make validation for each level, respectively.
4.2.1 Validation of local MTM.
Local MTM is proposed to calculate the similarity of LMTSs. In order to test the effectiveness of the local matching part, we do a special group of experiments on FVC databases. Four different methods for measuring LMTS are used: the local matching part of LSS (simplified as local LSS) [11], the local matching part of LTS (simplified as local LTS) [12], the local matching part of MCC (simplified as local MCC) [15] and the proposed local MTM. We choose the four algorithms for local matching as all of them have the step of local matching. Local LSS used a matching strategy based on graph representation. Local LTS used a longest common subsequence strategy to calculate the similarity of two LMTSs. Local MCC proposed a novel representation based on 3D data structures. For local MTM, we only use the local matching part proposed in section 2.3.
In order to ensure the comparison effectiveness of these four methods, we use the four different methods to measure similarities of LMTS pairs, and then using the same post-processing operations. After gaining the similarity matrix of LMTS pairs, we discard the different post-processing operations of these original algorithms and use the same discretization operation proposed in algorithm 2 to gain the optimal matching pairs. The final scores are simply evaluated using the same convex hull strategy, which is shown in the following formula.
(15)
In this formula SLL' indicates the similarity of LMTS pair (L, L'), Nm indicates the number of matched pairs. mp indicates the number of unmatched points within the convex hull in minutia set P and mp' indicates the number of unmatched points within the convex hull in minutia set P'.
For each algorithm and for each dataset, the performance indicator of EER and Time is considered. EER Results on FVC2002 and FVC2004 are shown in the Table 1. The experimental results show that our local MTM algorithm is close to well-known algorithm MCC in the EER indicator. It has promising performances in terms of both the EER and FMR-1000. Meanwhile, Time results on FVC2004-DB1 is shown in Table 2. We can see that the proposed local MTM is better than other algorithms in the Time indicator. Specially, the proposed local MTM is much better than local MCC, as we don’t need to construct minutia structures as complex as local MCC.
4.2.2 Validation of global MTM.
From the experiments for local matching we can see, MCC and the proposed local MTM are two better methods for measuring similarities of LMTSs pairs. They can be used to calculate the structural similarity for global matching. Take global MTM into consideration, we design two combined algorithms: local MCC + global MTM, and local MTM + global MTM. We compare these two algorithms with some other algorithms, including EMF [8], LSS [11], LTS [12], GF [14], MCC [15], MT [16] and LSC [18]. EMF applied a set of global level minutia dependent features including qualities and the area of overlapping regions. Here we use only the area of overlapping regions for fairness. LSS used a matching technique based on graph representation and then an expanding strategy based on distances. LTS proposed a novel matching algorithm based on local topologic structures and then used a novel method to compute the similarity. GF proposed a robust matching approach based on the growing and fusing of local structures. MCC proposed a novel representation based on 3D data structures. MT proposed a novel fingerprint matching algorithm named M3gl. LSC proposed the concept of compatibility to the minutiae triangle structures and adopted a relaxation process for global adjustment. In order to ensure the comparison effectiveness of these four methods, the radius of minutia structures is set to 80 pixels equally.
For each algorithm and for each dataset, the performance indicators of EER, FMR-1000, Time are considered. Results on FVC2002 and FVC2004 are shown in the Table 3 and Table 4. The corresponding ROC curves for FVC2004-DB1 are shown in Fig. 5.
We can get two conclusions: First, the combined two algorithms based on global MTM outperforms the other algorithms. Second, the algorithm based on local MTM and global MTM has the best results. It has promising performances in terms of both the EER, FMR-1000 and Time. Fig. 6 shows a true match with much noise and large nonlinearity. Thanks to the strong robustness of new method, many true matches are still found and they are judged as matched.
For FVC2004-DB1, there are 800 fingerprints. Each two impressions per finger are tested and there are 2800 true matches. The first impressions of each two fingers are tested and there are 4950 fake matches. The average matching time for FVC2004-DB1 is shown in Table 5. The result shows that our algorithm is significantly efficient. It is reasonable as both local MTM and global MTM run very fast. We don’t need to construct minutia structures as complex as MCC.
4.2.3 Comparative Experiments with our previous papers.
As we have described in the introduction part, this paper is innovative on the basis of two previous papers [24] [25]. We firstly applied the spectral matching strategy (simplified as SM) to global matching [24]. The spectral method was only applied to global matching, while other strategies were still required in local matching. Afterwards, we proposed the extended clique models (simplified as ECM) to deal with local matching and global matching [25].The process in local matching is time-consuming. Meanwhile, local matching is not associated with global matching. This paper firstly unifies local matching and global matching into an integrated framework. We design local MTM for local matching and global MTM for global matching, respectively. It is efficient in both levels. We conduct a group of experiments for the previous SM, ECM algorithms and the proposed MTM algorithm.
Results on FVC2002 and FVC2004 are shown in Table 6 and Table 7. The experimental results show that our local MTM algorithm has the best performances in terms of both the EER and Time. The ECM algorithm has promising effectiveness, but low efficiency.
Discussion
5.1 Effectiveness of this method
The proposed algorithm, including local MTM and global MTM, has promising effectiveness. It fully takes fingerprint characteristic into consideration. The local MTM is proposed to describe local rigidity. It contains similarities and compatibilities of minutia pairs within LMTSs. The global MTM is proposed to deal with global nonlinearity. It contains similarities and compatibilities of LMTS pairs within two minutia sets. For true minutia pairs, most of them have both high similarities and high compatibilities, thus they will be judged as matched. Some of them may have high similarities and low compatibilities due to the nonlinearity, they will still be selected in the global MTM and judged as matched. Some of them may have low similarities and high compatibilities due to the sparseness or noise, they will be excluded at the local matching step firstly and then regained at the global matching step. True minutia pairs with both low similarities and low compatibilities are unsolvable. They may be regained through other matching features, such as ridges.
5.2 Efficiency of this algorithm
The proposed algorithm has promising efficiency. Minutia matching is formulated as seeking for the dense sub-block in the corresponding tensor matrix. Fingerprint matching is divided into two levels: local matching and global matching. In each level it contains two steps: building the tensor matrix and spectral iteration. We make some pruning when building the tensor matrixes. The pruned tensor matrix is much smaller than original matrix. The time for building tensor matrixes is acceptable. Meanwhile, the spectral relaxation is very fast, it usually converges in 20 iterations. Suppose there are two minutia sets P and P', with Np and Np' minutiae, respectively, this algorithms contains these operations: Np × Np' times of local spectral optimization at the local matching step, one time of global spectral optimization at the global matching step, a MLS expanding operation and a convex hull operation. Each operation runs very fast.
5.3 Future work
As we have described in section 4.1, true minutia pairs with both low similarities and low compatibilities are unsolvable. They may be regained through other matching features, such as qualities and ridges. Our future work shall focus on: (1) adding new matching features, such as phase difference feature, ridges; (2) taking minutia quality into consideration. We will try to add other features into our algorithm framework.
Conclusions
In this paper, we propose the novel tensor strategy for fingerprint matching. First, the concept of minutia tensor matrix is proposed. It unifies both the first-order features and the second-order features appropriately. Second, correct minutia pairs correspond to the dense sub-block in the MTM. Minutia matching is formulated as recovering the main dense sub-block. Spectral matching methods are then used for recovering the dense sub-block. It is efficient and effective. Third, two different kinds of MTMs are constructed. The local MTM is constructed to calculate similarities of LMTS pairs. It makes use of local rigidity. The global MTM is constructed to calculate entire similarity. It takes global nonlinearity into consideration.
In summary, this paper proposes a novel structural similarity calculation and a novel approach for fingerprint recognition. It makes better use of similarities and compatibilities. It has stronger description ability and better robustness to nonlinearity and noise. Experimental results demonstrate the effectively and accuracy of our algorithm.
Author Contributions
Conceived and designed the experiments: XF. Performed the experiments: XF. Analyzed the data: JF. Contributed reagents/materials/analysis tools: JF. Wrote the paper: XF.
References
- 1.
Maltoni D, Maio D, Jain AK, Prabhakar S. Handbook of fingerprint recognition. springer; 2009.
- 2. Maio D, Maltoni D, Cappelli R, Wayman JL, Jain AK. FVC2002: Second fingerprint verification competition. Pattern Recognition, Proceedings, 16th IEEE International Conference on. 2002; 3: 811–814.
- 3. Cappelli R, Maio D, Maltoni D, Wayman JL, Jain AK. Performance evaluation of fingerprint verification systems. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2006; 28: 3–18.
- 4. Jain AK, Chen Y, Demirkus M. Pores and ridges: high-resolution fingerprint matching using level 3 features. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2007; 29: 15–27.
- 5. Liu C, Cao J, Gao X, Fu X, Feng J. A novel fingerprint matching algorithm using minutiae phase difference feature. Image Processing (ICIP), 18th IEEE International Conference on. 2011; 3201–3204.
- 6. Choi H, Choi K, Kim J. Fingerprint matching incorporating ridge features with minutiae. Information Forensics and Security, IEEE Transactions on. 2011; 6: 338–345.
- 7. Izadi MH, Mirmohamadsadeghi L, Drygajlo A. Introduction of cylinder quality measure into minutia cylinder-code based fingerprint matching. Biometrics: Theory, Applications and Systems (BTAS), 15th IEEE International Conference on. 2012; 353–358.
- 8. Wen W, Qi Z, Li Z, Zhang J, Gong Y, Cao P. A Robust and Efficient Minutia-Based Fingerprint Matching Algorithm. Pattern Recognition (ACPR), 2nd IAPR Asian Conference on. 2013; 201–205.
- 9. Li J, Tulyakov S, Zhang Z, Govindaraju V. Fingerprint matching using correlation and thin-plate spline deformation model. Biometrics: Theory, Applications and Systems (BTAS), 2nd IEEE International Conference on. 2008; 1–4.
- 10. Thuy H, Thi N, Huan HX, Ky NN. An efficient method for fingerprint matching based on local point model. Computing, Management and Telecommunications (ComManTel), 2013 IEEE International Conference on. 2013; 334–339.
- 11. Ratha NK, Bolle RM, Pandit VD, Vaish V. Robust fingerprint authentication using local structural similarity. Applications of Computer Vision, 15th IEEE Workshop on. 2000; 29–34.
- 12.
Chen X, Tian J, Yang X. A matching algorithm based on local topologic structure. In: Image Analysis and Recognition. Springer; 2004. pp. 360–367.
- 13.
Chikkerur S, Cartwright AN, Govindaraju V. K-plet and coupled BFS: a graph based fingerprint representation and matching algorithm. In: Advances in Biometrics. Springer; 2005. pp. 309–315.
- 14.
Xu W, Chen X, Feng J. A robust fingerprint matching approach: Growing and fusing of local structures. In: Advances in Biometrics. Springer; 2007. pp. 134–143.
- 15. Cappelli R, Ferrara M, Maltoni D. Minutia cylinder-code: A new representation and matching technique for fingerprint recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2010; 32: 2128–2141. pmid:20975113
- 16. Medina-Pérez MA, García-Borroto M, Gutierrez-Rodríguez AE, Altamirano-Robles L. Improving fingerprint verification using minutiae triplets. Sensors. 2012; 12: 3418–3437. pmid:22737016
- 17. Jea TY, Govindaraju V. A minutia-based partial fingerprint recognition system. Pattern Recognition. 2005; 38: 1672–1684.
- 18. Feng Y, Feng J, Chen X, Song Z. A novel fingerprint matching scheme based on local structure compatibility. Pattern Recognition (ICPR), 18th IEEE International Conference on. 2006; 4: 374–377.
- 19. Cao J, Feng J. A robust fingerprint matching algorithm based on compatibility of star structures. Sixth International Symposium on Multispectral Image Processing and Pattern Recognition, International Society for Optics and Photonics. 2009; 74983X–74983X–7.
- 20. Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2002; 24: 509–522.
- 21. Horaud R, Skordas T. Stereo correspondence through feature grouping and maximal cliques. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 1989; 11: 1168–1180.
- 22. Duchenne O, Bach F, Kweon I-S, Ponce J. A tensor-based algorithm for high-order graph matching. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2011; 33: 2383–2395.
- 23. Leordeanu M, Hebert M. A spectral technique for correspondence problems using pairwise constraints. Computer Vision (ICCV), Tenth IEEE International Conference on. 2005; 2:1482–1489.
- 24.
Fu X, Liu C, Bian J, Feng J. Spectral correspondence method for fingerprint minutia matching. Pattern Recognition (ICPR), 21st IEEE International Conference on. 2012; 1743–1746.
- 25.
Fu X, Liu C, Bian J, Feng J, Wang H, Mao Z. Extended clique models: A new matching strategy for fingerprint recognition. Biometrics (ICB), 6th IEEE International Conference on. 2013; 1–6.
- 26. Schaefer S, McPhail T, Warren J; Image deformation using moving least squares. ACM Transactions on Graphics (TOG). 2006; 25: 533–540.