## Figures

## Abstract

Establishing correspondences between two minutia sets is a fundamental issue in fingerprint recognition. This paper proposes a new tensor matching strategy. First, the concept of minutia tensor matrix (simplified as *MTM*) is proposed. It describes the first-order features and second-order features of a matching pair. In the *MTM*, the diagonal elements indicate similarities of minutia pairs and non-diagonal elements indicate pairwise compatibilities between minutia pairs. Correct minutia pairs are likely to establish both large similarities and large compatibilities, so they form a dense sub-block. Minutia matching is then formulated as recovering the dense sub-block in the *MTM*. This is a new tensor matching strategy for fingerprint recognition. Second, as fingerprint images show both local rigidity and global nonlinearity, we design two different kinds of *MTM*s: local *MTM* and global *MTM*. Meanwhile, a two-level matching algorithm is proposed. For local matching level, the local *MTM* is constructed and a novel local similarity calculation strategy is proposed. It makes full use of local rigidity in fingerprints. For global matching level, the global *MTM* is constructed to calculate similarities of entire minutia sets. It makes full use of global compatibility in fingerprints. Proposed method has stronger description ability and better robustness to noise and nonlinearity. Experiments conducted on Fingerprint Verification Competition databases (*FVC2002* and *FVC2004*) demonstrate the effectiveness and the efficiency.

**Citation: **Fu X, Feng J (2015) Minutia Tensor Matrix: A New Strategy for Fingerprint Matching. PLoS ONE 10(3):
e0118910.
https://doi.org/10.1371/journal.pone.0118910

**Academic Editor: **Wen-Bo Du, Beihang University, CHINA

**Received: **May 8, 2014; **Accepted: **January 12, 2015; **Published: ** March 30, 2015

**Copyright: ** © 2015 Fu, Feng. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

**Data Availability: **Comparative experiments are conducted on the International Fingerprint Verification Competition (simplified as FVC) databases: FVC2002 and FVC2004 databases. They are international competition databases for fingerprint verification algorithms, which are open to all researchers (FVC2002 can be downloaded in http://bias.csr.unibo.it/fvc2002/ and FVC2004 can be downloaded in http://bias.csr.unibo.it/fvc2004/).

**Funding: **The study was granted by National 973 project (No. 61333015, No. 2011CB302400) in China. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests: ** The authors have declared that no competing interests exist.

## Introduction

Fingerprint matching is a classical and hot topic in computer vision and pattern recognition [1]. Researchers have set up a series of special fingerprint verification competition databases [2] [3]. Establishing correspondences between fingerprint minutia sets is a fundamental issue in fingerprint recognition. It is challenging to find perfect correspondences for fingerprints due to various reasons: non-linear distortion, partial overlap, noise and so on.

Many minutia-based fingerprint matching papers have been published these years. They can be mainly classified into two categories. The first category of these papers is to exact more matching features besides minutia locations and orientations. Jain et al. used pores and ridge contours besides minutia points and proposed a three-level (level 1: pattern, level 2 minutia points and level 3: pores and ridge contours) matching strategy [4]. Liu et al. proposed the minutiae phase difference feature to describe the minutiae [5]. Choi et al. incorporated ridge features with minutiae and achieved good results [6]. Izadi et al. introduced cylinder quality measure into minutia cylinder-code based fingerprint matching [7]. Wen et al. applied a set of global level minutia dependent features, including minutia qualities and the area of overlapping regions [8]. Thuy et al. increased the ridge-valley structure features and improved the classical Thin-Plate-Spline deformation model [9], and they proposed the local Thin-Plate-Spline to deal with non-linear distorted fingerprints [10]. In summary, these papers are trying to seek for more matching features to improve the effectiveness.

The other category of these papers just uses the minutia features and tries to construct more complex structures or to find more efficient algorithms. Since only the minutia features are used, this category of approaches faces greater challenges. Previous pioneers have achieved big progress. Ratha et al. proposed a local similarity calculation algorithm for local matching [11]. Chen et al. constructed the minutia topological structure to increase the tolerance for noise [12]. Chikkerur et al. proposed K-PLET structures and a graph-based representation for minutia matching [13]. Xu et al. grew minutia triangle structures into a growing area and then performed the global match by fusing them[14]. Cappelli et al. proposed a novel representation based on 3D cylinder structures, called Minutia Cylinder-Code [15]. Medina-Pérez et al. improved fingerprint verification by using minutia triplets [16]. In summary, minutia topologic structures constructed in local areas are considered to be less impacted by nonlinear distortion and noise. However, as the radius of minutia structures is not big, it limits the use of global minutia information. Based on this consideration, some researchers try to introduce global information into the matching process. Tea et al. presented an approach based on convex hull to deal with incomplete or partial fingerprints [17]. Feng et al. proposed the compatibility of local minutia pairs and adopted a relaxation iterative process [18]. Cao et al. applied compatibility to local minutia star-structures and achieved better results [19]. All these papers are trying to construct more complex minutia topologic structures or seek for more efficient matching strategies.

Establishing correspondences between two feature sets is a fundamental issue in computer vision and pattern recognition [20]. Horaud et al. proposed a graph matching framework and formulated point matching as recovering the maximal cliques in the correspondence graph [21]. Duchenne et al. proposed the affinity tensors to describe high-order affinities between each feature pairs [22]. First-order tensor describes similarities of each feature pair. Second-order tensor describes compatibilities between each two feature pairs. Leordeanu et al. proposed a spectral technique for correspondence problems using pairwise constraints [23]. They recovered the matching relationship based on spectral matching methods, by using the principal eigenvector of adjacency matrix and imposing the one-to-one mapping constraints. This notion can be introduced into minutia matching.

Based on these insights, we propose a tensor matching strategy. We construct the minutia tensor matrix (simplified as *MTM*) for fingerprint minutiae. It unifies both the first-order features and the second-order features. The diagonal elements in *MTM* indicate similarities of each minutia pair and other elements indicate pairwise compatibilities between minutia pairs. Correct minutia pairs are more likely to establish both large similarities within them and large compatibilities among them, thus they form a dense sub-block. Incorrect pairs establish links with the other pairs accidentally, so they are unlikely to belong to dense sub-blocks (The definition of dense sub-block can be seen in section 2.1). Minutia matching is formulated as recovering the main dense sub-block in the *MTM*.

Our previous papers tried to apply the tensor idea to fingerprint matching [24] [25]. We applied the spectral matching strategy to fingerprint global matching [24]. The disadvantage was that it was only applied to global matching, while other strategies were still required for local matching. We proposed extended clique models to deal with local matching and global matching [25]. The disadvantage is that the local matching process is time-consuming. Meanwhile, local matching is not associated with global matching. This paper is innovative on the basis of previous papers. First, the tensor strategy is used in local matching for the first time. Second, by use of the proposed “*MTM*”, local matching and global matching are unified into an integrated framework.

Main contributions of this paper are: First, **the proposed MTM unifies both similarities and compatibilities appropriately**. Second, minutia matching is formulated as recovering the dense sub-block in the

*MTM*. Optimal matching relationship corresponds to the dense sub-matrix in the

*MTM*.

**It gives a clear mathematical meaning for “optimal matched pairs”**. Third,

**spectral matching methods are then used for recovering the dense block. It is efficient and effective**, which can be seen in the following experiments. Forth,

**two**. In the local matching level, we build the local minutia topologic structures and construct local

*MTM*s with different constraints are constructed, which makes use of local rigidity and global compatibility, respectively*MTM*for each minutia structure pair. Calculating similarities of minutia pairs corresponds to recovering the dense sub-block in the local

*MTM*. In the global matching level, we construct the global

*MTM*for entire minutia sets. Calculating similarities of minutia sets corresponds to recovering the dense sub-block in the global

*MTM*. Proposed approach has stronger description ability and better robustness to non-linear deformation and noise.

## Materials and Methods

### 2.1 Problem formulation

Suppose there are two fingerprint minutia sets *P* and *P'*, with *N*_{p} and *N*_{p'} minutiae, respectively. We define two attributed graphs *G*_{P} = (*V*,*E*,*A*) for minutia set *P* and *G*_{P'} = (*V'*,*E'*,*A'*) for minutia set *P'*. Each edge e = ij ∊ E in *G*_{P} is assigned an attribute *A*_{ij}, which is the distance vector between minutia *i* and minutia *j* in *P*. We represent node attributes as special edge attributes, i.e. *A*_{ii} for node *i*. So it is with *P'*. Let *S*_{A}(*ii'*, *jj'*) indicate the similarity score between attributes *A*_{ij} and *A'*_{i'j'}. We want to find a mapping that best preserves the attributes of nodes and edges between attributed graphs *G*_{P} and *G*_{P'}. Equivalently, we seek for a set of correct minutia pairs *M* = (*ii'*,*jj'*,*kk'*…) so as to maximize the matching score, defined as:

High-order tensors are constructed to represent high-order affinities [22]. Inspired by this notion, we propose the minutia tensor matrix (*MTM*). It is proposed based on these considerations: First-order tensor matrix describes similarities of each minutia pair. Second-order tensor matrix describes compatibilities between each two minutia pairs. For fingerprint minutia sets *P* and *P'*, each minutia pair (*i*, *i'*) is assigned an similarity attribute *T*_{1} (*i*, *i'*) and each two minutia pairs (*ii'*, *jj'*) is assigned a compatibility attribute *T*_{2} (*ii'*, *jj'*). (Calculation of *T*_{1} (*i*, *i'*) and *T*_{2} (*ii'*, *jj'*) can be seen in section 2.3 and 2.4). Thus we can build the first-order tensor matrix *T*_{1} and the second-order tensor matrix *T*_{2}. Here we don’t use tensor matrixes with three or more orders as the computation is unacceptable. After that we fuse *T*_{1} and *T*_{2} into a fused high-order tensor matrix *T*_{F}, which is also the proposed *MTM*. The fusion rule is shown in the formula:
(2)
Where *T*_{1} is the first-order tensor matrix, whose element *T*_{1} (*i*, *i'*) describes the similarity of minutia pair (*i*, *i'*). *T*_{2} is the second-order tensor matrix, whose element *T*_{2} (*ii'*, *jj'*) describes compatibility between (*i*, *i'*) and (j, *j'*). The fused tensor matrix *T*_{F} (*MTM*) is a two-dimensional matrix with (*N*_{p} *× N*_{p'}) × (*N*_{p} *× N*_{p'}) elements. It unifies both similarities and compatibilities among minutia pairs. When the condition (*i = i' and j = j'*) holds, minutia pairs (*i*, *i'*) and (j, *j'*) are the same pair, thus *T*_{1} (*i*, *i'*) = *T*_{1} (*j*, *j'*). As minutia matching is one-to-one matching, minutia *i* in *P* mapped to both minutia *i'* and *j'* in *P'* is impossible. When (*i = j and i' ≠ j'*) or (*i ≠ j and i' = j'*) holds, *T*_{2} (*ii'*, *jj'*) is 0.We use this priori knowledge to prune and can get a very sparse *MTM*. The parameter λ is a weighting factor. As *T*_{1} and *T*_{2} have different dimensions, we use *λ* to adjust the values. It is set manually. *T*_{F} is a symmetric matrix as *T*_{F} (*ii'*, *jj'*).

As shown in Fig. 1, for two minutia sets *P* and *P'*, we can build the correspondence graph *G*_{PP'}. In graph *G*_{PP'}, the dense subgraph *G*_{A} is the biggest strong-connected subgraph in *G*_{PP'}, whose nodes have strong links with all the other nodes. Correct minutia pairs are likely to establish links among each other and thus form a strongly connected subgraph, which is also the dense subgraph. Incorrect pairs establish links with the other pairs only accidentally. Minutia matching can be formulated as seeking for the dense subgraph in the correspondence graph.

The fingerprint images are from FVC2004 DB1 27–3 and 27–5. For clarity, only a small subset of minutia pairs are shown. Candidate minutia pairs shown in (a) form the correspondence graph in (b) and the minutia tense matrix (*MTM*) in (c). Genuine minutia pairs corresponds to the dense subgraph of correspondence graph, and also the dense block of *MTM*. Minutia matching is formulated as recovering the dense sub-block in the *MTM*. It can be solved by the spectral correspondence methods.

The correspondence graph *G*_{PP'} corresponds to the *MTM T*_{F}. In *T*_{F}, the dense sub-block *T*_{D} is the biggest sub-matrix in *T*_{F}, whose elements in all the rows and all the columns have big values. The element values in the dense sub-block *T*_{D} should be greater than a certain threshold. As shown in Fig. 1, for minutia sets *P* and *P'*, we can build the *MTM T*_{F} of the correspondence graph *G*_{PP'}. Correct pairs are likely to establish elements with big values among each other and thus form a strongly connected cluster, which is also the dense sub-block *T*_{D}. Minutia matching corresponds to recovering the dense sub-block.

Equivalently, we seek for a set of correct matching pairs *M* = (*ii'*, *jj'*, *kk'*…), so as to maximize the *MTM* score, defined as:

The main idea is illustrated in Fig. 1. The optimization problem (3) can be solved by spectral matching method, which will be discussed in section 2.2.

### 2.2 Spectral matching methods

Formula (3) yields to the following binary optimization problem: (4)

The optimal solution is the binary vector that maximizes the *m*^{T}*T*_{F}*m* score. *m* is a row-wise vectored replica of the *MTM T*_{F}. It can be derived by using the principal eigenvector of *T*_{F} and imposing the one-to-one mapping constraints, which is proposed in [23].

Here we make some improvements. Firstly we prune the elements with small similarities or low compatibilities. Secondly we relax both the mapping constraints and the integral constraints on *m*, so that its elements can take real values in [1]. Thirdly we don’t initialize *m*_{0} (the initialization value of *m*) randomly like [23], we use some priori knowledge. As minutia pairs with big local structural similarities are more likely to have large compatibilities, we initialize *m*_{0} by normalized first-order tensor matrix *T*_{1}. It converges much faster using this strategy.

We can derive *m** by computing the leading eigenvalue of *T*_{F}, using Algorithm 1.

**Algorithm 1** Spectral relaxation iterations.

**Input:** minutia tensor matrix *T*_{F}

**Output:** *m**, the main eigenvector of *T*_{F}

**1** initialize

**2** repeat

**3** ↓*m** ← *T*_{F}*m**;

**4**;

**5** until convergence;

Indeed, each step of the iteration algorithm requires only *O* (*z*) operations, where *z* is the number of non-zero elements of the matrix. Also, typically, in our situation, the algorithm converges in a few dozen steps.

Given the continuous solution *m**, we can rewrite it in a N_{p} × N_{p'} matrix form and discretize *m** by a greedy approach and derive the optimal solution. Because of the existence of partial overlapping, only a subset of size *N*_{S} ≤ *min*(*N*_{p}, *N*_{p'}) can be matched. Here we set the *m** elements whose value is smaller than the pre-set threshold δ to zero. The parameter δ is set manually. Meanwhile, minutia matching is one-to-one matching, minutia *i* in *P* mapped to both minutia *i'* and *j'* in *Q* is impossible. We add this compatibility constraint during the iteration process. It is summarized as Algorithm 2.

**Algorithm 2** Greedy approach for discretization.

**Input:** continuous matrix *m**

**Output:** binary matrix

**1** initialize

**2** set *m** (*i*,*j*) = 0, ∀i, ∀j, m*(*i*,*j*)< δ

**3** repeat

**4** ↓ find the maximal element m*(*i*,*j*)

**5** ↓ set, ∀k, set *m**(*i*,*k*) = 0,*m**(*k*,*j*) = 0

**6** until *m** = **0**;

The matching constraints (m**1** ≤ **1** and m^{T} **1** ≤ **1**) are ignored in the spectral relaxation step and then are induced during the discretization step. Corrected matched pairs can be gained and similarities of minutia sets can be evaluated. We can get the number *N*_{m} of matched pairs.

After that we adjust the similarity score with the number of matched pairs. Take minutia sets *P* and *P'* for example, the similarity *S*_{PP'} is calculated using the following formula:

*S*_{ii'} measures the similarity of matched pairs *ii'*. indicates the maximum value of the optimization problem shown in formula (3). *N*_{m} indicates the number of matched pairs, and *N*_{p} and *N*_{p'} indicate the minutia number of *P* and *P'*.

In practice, we use the tensor matching strategy both in local matching level and global matching level. There are some differences in these two levels. We will give detailed descriptions in section 2.3 and 2.4.

### 2.3 Local minutia tensor matrix (*Local MTM*)

Local minutia topologic structure (simplified as *LMTS*) is firstly introduced in [12]. A typical *LMTS* is constructed from a center minutia and a list of neighboring minutiae in a specified area. Fingerprint images show rigidity within the range of *LMTS*. Each *LMTS* can be seen as a small minutia set. Here we propose the local *MTM* for minutia structures and design a novel strategy to calculate similarities of *LMTS* pairs.

As shown in Fig. 2, minutia *a* and *i*, *j*, *k* form a minutia topologic structure *L*_{a}, minutia *a'* and *i'*, *j'*, *k'* form a minutia topologic structure *L*_{a'}.

The fingerprint images are captured from FVC2004 DB1 6–2 and 6–5. Minutia pairs within two local minutia topologic structures shown in (a) form the local correspondence graph in (b) and the local minutia tense matrix (*local MTM*) in (c). Correct pairs corresponds to the dense subgraph of local correspondence graph, and also the dense block of *local MTM*. Minutia matching is formulated as recovering the dense sub-block in the *local MTM*.

According to the coordinates and orientations of minutiae in *L*_{a}, we calculate the distance vector between each two minutiae within *L*_{a}, so it is with *L*_{a'}. Each candidate minutia pair (*i*, *i'*) is assigned a local similarity attribute LT_{1} (*i*,*i'*) and each two minutia pairs (*ii'*, *jj'*) is assigned a local compatibility attribute LT_{2} (*ii'*, *jj'*). They can be calculated in the following formula:
(7)
(8)
Where is the distance vector between minutia *a* and minutia *i*. have similar meanings. Here. *D* (,) measures the 1-norm distance of vectors and. *S*_{ii'} indidates the similarity of minutia pair (*i*, *i'*). *C*_{ii'},_{jj'} indidates the compatibility between pairs (*i*, *i'*) and (j, *j'*). α, α', β, β' are positive parameters. They are set manually.

We calculate all the similarities of minutia pairs in *LT*_{1} and all the compatibilities of each two minutia pairs in *LT*_{2}. Thus, we can construct the local first-order tensor matrix *LT*_{1} and the local second-order tensor matrix *LT*_{2}. The local fused tensor matrix *LT*_{F} (local *MTM*) can be constructed using the method proposed in formula (2). Calculating similarities of *L*_{a} and *L'*_{a} is formulated as recovering the dense sub-block in the local minutia tensor matrix *LT*_{F}.

In experiments, we make some pruning after building the local *MTM*. Suppose there are two minutia topologic structures *L*_{a},*L*_{a'}, with *n*_{a}, *n*_{a'} minutiae, respectively. Because of the one-to-one mapping constraint, only subsets of size n_{s} ≤min (*n*_{a}, *n*_{a'}) can be matched. We retain n_{r} = 3 * min (*n*_{a}, *n*_{a'}) candidate minutia pairs with the maximal similarity values. The pruned local *MTM* has only *n*_{r} ** n*_{r} elements, which is much smaller than the original scale (*n*_{a} ** n*_{a'}) * (*n*_{a}, ** n*_{a'}). Then the optimal matching pairs within *LMTS* can be gained through spectral methods proposed in section 2.2.The spectral iteration process is very fast. It usually will converge in several iterations.

After relaxation and discretization, we can gain the optimal matched pairs and the number *n*_{m} of matched pairs. After that we adjust the similarity score with the number of matched pairs. The similarity of *LMTS* pairs *L*_{a} and *L*_{a'} can be calculated using the following formula.
(10)
indicates the maximal score of the local *MTM LT*_{F}. *n*_{m} indicates the number of matched pairs within (*L*_{a}, *L*_{a'}), and *n*_{a} indicates the number within *LMTS L*_{a}, *n*_{a'} has a similar meaning.

### 2.4 Global minutia tensor matrix (*Global MTM*)

The whole minutia set can be seen as a large global minutia topologic structure and the local minutia topologic structures (*LMTS*s) are basic matching units. Fingerprint images show nonlinearity in the global range. Minutia pairs far apart may not look so compatible. Here we propose the global *MTM* for minutia sets and design a novel strategy to calculate similarities of fingerprints.

As shown in Fig. 3, L_{a},L_{b},L_{c},L_{d} are four minutia structures in minutia set *P*, L_{a'},L_{b'},L_{c'},L_{d'} are four minutia structures in minutia set *P'*. For clarity, only a small subset of the candidate *LMTS* pairs is shown. *LMTS* pair (*L*_{a},*L*_{a'}) is assigned a global similarity attribute *GT*_{1} (*L*_{a}, *L*_{a'}) and each two minutia pairs (*L*_{a}*L*_{a'},*L*_{b}*L*_{b'}) is assigned a global compatibility attribute *GT*_{2}(*L*_{a}*L*_{a'},*L*_{b}*L*_{b'}).

The fingerprint images are from FVC2004 DB1 27–3 and 27–5. For clarity, only four minutia structure pairs are shown. Candidate structure pairs shown in (a) form the global correspondence graph in (b) and the global minutia tense matrix (*global MTM*) in (c). Correct structure pairs corresponds to the dense subgraph of global correspondence graph, and also the dense block of *global MTM*. Minutia matching is formulated as recovering the dense sub-block in the *global MTM*.

Fingerprint images show nonlinearity in the global range. Minutia pairs far apart may not look so compatible. We lower the compatibility threshold and take nonlinearity into account. The first-order tensor *GT*_{1}(*L*_{a},*L*_{a'}) can be evaluated using the structural similarity gained in section 2.2. According to the coordinates and orientations of *LMTS* center minutiae, we can calculate the distance vector between each two center minutiae, such as. The compatibility *GT*_{2}(*L*_{a}*L*_{a'},*L*_{b}*L*_{b'}) can be evaluated using the two pairs of center minutiae (*aa'*, *bb'*). They can be calculated in the following formula:
(11)
(12)
Where indidates the similarity of *LMTS* pair (*L*_{a}, *L*_{a'}) gained in section 2.3. is the distance vector between center minutiae *a* and *b*. has a similar meaning. *C*_{aa',bb'} indidates the compatibility between pairs (*a*, *a'*) and (*b*, *b'*). *γ*, *γ'* are positive parameters. They are set manually.

We calculate all the similarities of *LMTS* pairs in *LT*_{1} and all the compatibilities of each two *LMTS* pairs in *LT*_{2}. Thus, we can construct the global first-order tensor matrix *GT*_{1} and the local second-order tensor matrix *GT*_{2}. The global fused tensor matrix *GT*_{F} (global *MTM*) can be constructed using the method proposed in formula (2). Calculating the similarity of minutia sets *P* and *P'* is formulated as recovering the dense sub-block in the global minutia tensor matrix *GT*_{F}.

In experiments, we make some pruning after building the global *MTM*. Suppose there are two minutia sets *P* and *P'*, with *N*_{p} and *N*_{p'} minutiae, respectively. Because of the one-to-one mapping constraint, only subsets of size *N*_{S} ≤ min (*N*_{p}, *N*_{p'}) can be matched. We retain *N*_{r} = 3 * min (*N*_{p},*N*_{p'}) candidate minutia pairs with the maximal similarity values. The pruned global *MTM* can be gained. It has only *N*_{r} ** N*_{r} elements, which is much smaller than the original scale (*N*_{p} ** N*_{p'}) * (*N*_{p} ** N*_{p'}). Then the optimal *LMTS* pairs can be gained through spectral methods proposed in section 2.2.The spectral iteration process is very fast. It usually will converge in several iterations.

After relaxation and discretization, we can gain the optimal matched pairs and the number *N*_{m} of matched pairs. After that we adjust the similarity score with matched pairs. As isolated minutiae or those minutiae in the border of fingerprint images have few surrounding minutiae, they may be removed at the step of constructing *LMTS*s. It is necessary to regain the lost matches. We use the moving least squares (*MLS*) model proposed in [26] to regain lost genuine minutia pairs. First, the parameters of *MLS* deformation are estimated from the matched minutia pairs. Then the estimated parameters are used to transform the model minutia set to the target minutia set. Third, a tolerance box is adopted to scan each minutia. When two minutiae from different fingerprints drop into the same tolerance box, they are judged as matched. For fingerprints from the same finger, many genuine minutia pairs can be retrieved. For fingerprints from different fingers, few minutiae can be regained as the randomness of minutiae’s positions. In this way we can get the final matched pairs.

Here we design a new scoring strategy to get the final matching scores. We adopt the convex hull strategy proposed in [17] to adjust the final similarity score. The convex hull can be gained and the number of unmatched points within the convex hull can be calculated. The evaluation strategy is based on convex hull and can be represented as follows:
(14)
indicates the maximal score of the global *MTM GT*_{F}. *N*_{m} indicates the number of matched points, and *m*_{p} indicates the number of unmatched points within the convex hull in minutia set *P* and *m*_{p'} indicates the number of unmatched points within the convex hull in minutia set *P'*. It is more reasonable to adopt this new score evaluation strategy.

## Algorithm

In this section, we describe our proposed algorithm in detail. It mainly contains the following parts: initialization, local matching and global matching.

### Initialization

Suppose there are two minutia sets *P* and *P'*, with *N*_{p} and *N*_{p'} minutiae, respectively. For each minutia *i* in *P*, we construct the *LMTS* with *i* as the center minutia and a list of neighboring minutiae in the circular area. In this way, *N*_{p} *LMTS*s in *P* can be constructed. We take the same operations in *P'* and *N*_{p'} *LMTS*s can be constructed, too. According to the coordinates and orientations of minutiae, we calculate all the distance vectors within minutia *P* and all the distance vectors within minutia *P'*. We calculate all the similarities of minutia pairs (*i*, *i'*), ∀*i* ∊ *P*, ∀*i*' ∊ *P'* using the formula (7). Meanwhile, we calculate all the compatibilities of each two pairs (*ii'*, *jj'*), ∀*i*,*j* ∊ *P*, ∀*i*', *j'* ∊ *P'* using the formula (8). In this way, we can gain *N*_{p} *× N*_{p'} similarities and (*N*_{p} *× N*_{p'}) × (*N*_{p} *× N*_{p'}) compatibilities in all.

### Local matching

For minutia sets *P* and *P'*, with *N*_{p} and *N*_{p'} minutiae, we construct *N*_{p} and *N*_{p'} *LMTS*s, respectively, thus we gain *N*_{p} × *N*_{p'} *LMTS* pairs. For each *LMTS* pair from 1 to *N*_{p} × *N*_{p'}, we conduct a local matching operation.

The local matching operation contains three steps:

(a) Building the local *MTM*: For the *LMTS* pair (*L*_{a}, *L*_{a'}), we select the corresponding minutia pairs within *L*_{a} and *L*_{a'}. (The similarities and compatibilities have been calculated in the initialization step and we just need to select them.) We build the local first-order tensor matrix *LT*_{1} and the local second-order tensor matrix *LT*_{2}. Next we construct the local *MTM LT*_{F} using formula (2).

(b) Spectral matching: Calculating the similarity of *LMTS* pair (*L*_{a}, *L*_{a'}) yields to the optimization problem shown in formula (9). We use the spectral matching methods proposed in section 2.2 to deal with it. The maximal value and optimal matching pairs can be gained.

(c) Scoring: After gaining the maximal and the optimal matching pairs, we calculate the similarity of *LMTS* pair (*L*_{a}, *L*_{a'}) using formula (10).

After *N*_{p} *× N*_{p'} local matching operations, we can gain all the similarities of candidate *LMTS* pairs.

### Global matching

(a) Building the global *MTM*: After gaining the similarities of all candidate *LMTS* pairs, we can build the global first-order tensor matrix *GT*_{1}. For each two *LMTS* pairs (*L*_{a}*L*_{a'}, *L*_{b}*L*_{b'}), we evaluate their compatibility through their center minutiae (*aa'*, *bb'*). The calculation function can be seen in formula (12). In this way, (N_{p} × N_{p'}) × (N_{p} × N_{p'}) *LMTS* compatibilities can be calculated and the global second-order tensor matrix *GT*_{2} can be built. Next we construct the global *MTM GT*_{F} with (N_{p} × N_{p'}) × (N_{p} × N_{p'}) elements using formula (2).

(b) Spectral matching: Calculating the similarity of minutia sets *P* and *P'* yields to the optimization problem shown in formula (13). We use the spectral matching methods proposed in section 2.2 to deal with it. The maximal value and the optimal *LMTS* pairs can be gained.

(c) Regaining: After calculating the maximal value and the optimal *LMTS* pairs, we regain lost genuine pairs using the *MLS* strategy. In this way the final matched pairs can be gained.

(d) Scoring: After gaining the maximal value and the final matched pairs. We adopt the convex hull strategy to adjust the final similarity score. We calculate the entire similarity *S*_{PP'} of minutia sets *P* and *P'* using formula (14).

The novel fingerprint matching algorithm is summarized as algorithm 3.

**Algorithm 3** High-order tensors matching algorithm.

**Input:** Two fingerprint minutia sets *P* and *P'*, with *N*_{p} and *N*_{p'} minutiae, respectively.

**Output:** The entire similarity *S*_{PP'} of minutia sets ** P** and

*P'***1** Initialization:

constructing local minutia topologic structures:

*N*_{p} and *N*_{p}'structures, respectively.

calculating similarities and compatibilities of all minutia pairs:

*N*_{p} *× N*_{p'} similarities and **( N**

_{p}

*× N*_{p'}

**)×(**

*N*_{p}

*× N*_{p'}) compatibilities.

**2** Local matching: calculating *LMTS* similarities

repeat: from 1 to *N*_{p} *× N*_{p'}

calculating similarity of *LMTS* pairs for *LMTS* pairs (*L*_{a},*L*_{a'})

building the local first-order tensor *LT*_{1} and the local second-order tensor *LT*_{2}

constructing the local minutia tensor matrix *LT*_{F}

spectral matching for and matched pairs

calculating the similarity of *LMTS* pair (*L*_{a},*L*_{a'})

until all candidate *LMTS* pairs are calculated.

**3** Global matching: calculating entire similarity

building the global first-order tensor *GT*_{1}and the global second-order tensor *GT*_{2}

constructing the global minutia tensor matrix *GT*_{F}

spectral matching for and matched pairs

regaining lost genuine minutia pairs using *MLS* model

constructing the convex hulls of minutia sets ** P** and

**.**

*P'* calculating the entire similarity *S*_{PP'} of minutia sets ** P** and

*P'*## Results

### 4.1 Fingerprint Database

Comparative experiments are conducted on the International Fingerprint Verification Competition (simplified as *FVC*) databases: *FVC2002* and *FVC2004* databases. *FVC2002* contains four different sub-databases and each database has 800 fingerprints (100 fingers and 8 impressions per finger) [2]. For each sub-database, each two impressions per finger are tested and there are 2800 true matches. The first impressions of each two fingers are tested and there are 4950 fake matches. We conduct 7750 matches in all. It is the same with *FVC2004* [3]. They are international competition databases for fingerprint verification algorithms, which are open to all researchers (*FVC2002* can be downloaded in http://bias.csr.unibo.it/fvc2002/ and *FVC2004* can be downloaded in http://bias.csr.unibo.it/fvc2004/). The image quality of *FVC2002* is good, while the image quality of FVC2004 is relatively bad. It contains much noise and nonlinearity.

For each algorithm and for each dataset, the following performance indicators are considered: Equal-Error-Rate (simplified as *EER*); False Match Rate (simplified as *FMR*); False Non Match Rate (simplified as *FNMR*); the lowest *FNMR* for *FMR* ≤ 0.1% (simplified as *FMR-1000*), average time for per match (simplified as *Time*) and the corresponding receiver operating characteristic curves *(*simplified as *ROC cures*). These experiments run on Intel i5–760 (2.80 GHz) processor (single-thread) and 32-bit Windows 7 systems.

### 4.2 Experiments

The matrix scale of *local MTM* plays an important role for local matching. It is directly affected by the radius of *LMTS*. Here we make a special experiment on *FVC2004*-*DB1* to test the influence. We use the proposed matching algorithm and vary the radius size from 30 pixels to 160 pixels. We observe the relationship between *EER*, average time and the *LMTS* radius. Fig. 4 shows the results. The radius ranges from 30 pixels to 160 pixels. The average time increases with the radius length as the matrix scale of *MTM* is directly affected by the radius of *LMTS*. *EER* decreases at first and then increases with the radius. It is because that fingerprints show rigidity in local areas and nonlinearity in global areas.

In the following experiments, we set each parameter the same value for all algorithms. The *LMTS* radius is set to 80 pixels. The matrix fusion parameter *λ* in formula (2) is set to 0.1. The minimal similarity threshold *δ* in section 2.2 is set to *e*^{-4}. The local similarity parameters α, α' in formula (7) is set to 1, 0.5, respectively. The local compatibility parameters *β*, *β'* in formula (8) is set to 2, 0.33, respectively. The global compatibility parameters *γ*, *γ'* in formula (12) is set to 2, 0.2, respectively.

Comparative experiments are conducted on *FVC2002* and *FVC2004* databases. In order to demonstrate the effectiveness of the new algorithm, some other matching algorithms are used, including the efficient minutia-based fingerprint matching algorithm (simplified as *EMF*) [8], the local structural similarity algorithm (simplified as *LSS*) [11], the local topologic structure algorithm (simplified as *LTS*) [12], the grow and fuse algorithm (simplified as *GF*) [14], the minutia cylinder-code algorithm (simplified as *MCC*) [15], the minutiae triplets algorithm (simplified as *MT*) [16] and the local structure compatibility algorithm (simplified as *LSC*) [18]. Here we use only minutia features (coordinates and orientations) for every algorithm. We do not use other matching features such as qualities or ridges to ensure the comparison effectiveness.

The proposed algorithm has two levels: local *MTM* matching and global *MTM* matching. We make validation for each level, respectively.

#### 4.2.1 Validation of *local MTM*.

*Local MTM* is proposed to calculate the similarity of *LMTS*s. In order to test the effectiveness of the local matching part, we do a special group of experiments on *FVC* databases. Four different methods for measuring *LMTS* are used: the local matching part of *LSS* (simplified as *local LSS*) [11], the local matching part of *LTS* (simplified as *local LTS*) [12], the local matching part of *MCC* (simplified as *local MCC*) [15] and the proposed local *MTM*. We choose the four algorithms for local matching as all of them have the step of local matching. *Local LSS* used a matching strategy based on graph representation. *Local LTS* used a longest common subsequence strategy to calculate the similarity of two *LMTS*s. *Local MCC* proposed a novel representation based on 3D data structures. For *local MTM*, we only use the local matching part proposed in section 2.3.

In order to ensure the comparison effectiveness of these four methods, we use the four different methods to measure similarities of *LMTS* pairs, and then using the same post-processing operations. After gaining the similarity matrix of *LMTS* pairs, we discard the different post-processing operations of these original algorithms and use the same discretization operation proposed in algorithm 2 to gain the optimal matching pairs. The final scores are simply evaluated using the same convex hull strategy, which is shown in the following formula.
(15)
In this formula *S*_{LL'} indicates the similarity of *LMTS* pair (*L*, *L'*), *N*_{m} indicates the number of matched pairs. *m*_{p} indicates the number of unmatched points within the convex hull in minutia set *P* and *m*_{p'} indicates the number of unmatched points within the convex hull in minutia set *P'*.

For each algorithm and for each dataset, the performance indicator of *EER* and *Time* is considered. *EER* Results on *FVC2002* and *FVC2004* are shown in the Table 1. The experimental results show that our *local MTM* algorithm is close to well-known algorithm *MCC* in the *EER* indicator. It has promising performances in terms of both the *EER* and *FMR-1000*. Meanwhile, *Time* results on *FVC2004-DB1* is shown in Table 2. We can see that the proposed *local MTM* is better than other algorithms in the *Time* indicator. Specially, the proposed *local MTM* is much better than *local MCC*, as we don’t need to construct minutia structures as complex as *local MCC*.

#### 4.2.2 Validation of *global MTM*.

From the experiments for local matching we can see, *MCC* and the proposed local *MTM* are two better methods for measuring similarities of *LMTSs* pairs. They can be used to calculate the structural similarity for global matching. Take *global MTM* into consideration, we design two combined algorithms: *local MCC* + *global MTM*, and *local MTM* + *global MTM*. We compare these two algorithms with some other algorithms, including *EMF* [8], *LSS* [11], *LTS* [12], *GF* [14], *MCC* [15], *MT* [16] and *LSC* [18]. *EMF* applied a set of global level minutia dependent features including qualities and the area of overlapping regions. Here we use only the area of overlapping regions for fairness. *LSS* used a matching technique based on graph representation and then an expanding strategy based on distances. *LTS* proposed a novel matching algorithm based on local topologic structures and then used a novel method to compute the similarity. *GF* proposed a robust matching approach based on the growing and fusing of local structures. *MCC* proposed a novel representation based on 3D data structures. *MT* proposed a novel fingerprint matching algorithm named *M3gl*. *LSC* proposed the concept of compatibility to the minutiae triangle structures and adopted a relaxation process for global adjustment. In order to ensure the comparison effectiveness of these four methods, the radius of minutia structures is set to 80 pixels equally.

For each algorithm and for each dataset, the performance indicators of *EER*, *FMR-1000*, *Time* are considered. Results on *FVC2002* and *FVC2004* are shown in the Table 3 and Table 4. The corresponding *ROC curves* for *FVC2004-DB1* are shown in Fig. 5.

We can get two conclusions: First, the combined two algorithms based on *global MTM* outperforms the other algorithms. Second, the algorithm based on *local MTM* and *global MTM* has the best results. It has promising performances in terms of both the *EER*, *FMR-1000* and *Time*. Fig. 6 shows a true match with much noise and large nonlinearity. Thanks to the strong robustness of new method, many true matches are still found and they are judged as matched.

For *FVC2004-DB1*, there are 800 fingerprints. Each two impressions per finger are tested and there are 2800 true matches. The first impressions of each two fingers are tested and there are 4950 fake matches. The average matching time for *FVC2004-DB1* is shown in Table 5. The result shows that our algorithm is significantly efficient. It is reasonable as both *local MTM* and *global MTM* run very fast. We don’t need to construct minutia structures as complex as *MCC*.

#### 4.2.3 Comparative Experiments with our previous papers.

As we have described in the introduction part, this paper is innovative on the basis of two previous papers [24] [25]. We firstly applied the spectral matching strategy (simplified as *SM*) to global matching [24]. The spectral method was only applied to global matching, while other strategies were still required in local matching. Afterwards, we proposed the extended clique models (simplified as *ECM*) to deal with local matching and global matching [25].The process in local matching is time-consuming. Meanwhile, local matching is not associated with global matching. This paper firstly unifies local matching and global matching into an integrated framework. We design local *MTM* for local matching and global *MTM* for global matching, respectively. It is efficient in both levels. We conduct a group of experiments for the previous *SM*, *ECM* algorithms and the proposed *MTM* algorithm.

Results on *FVC2002* and *FVC2004* are shown in Table 6 and Table 7. The experimental results show that our *local MTM* algorithm has the best performances in terms of both the *EER* and *Time*. The *ECM* algorithm has promising effectiveness, but low efficiency.

## Discussion

### 5.1 Effectiveness of this method

The proposed algorithm, including *local MTM* and *global MTM*, has promising effectiveness. It fully takes fingerprint characteristic into consideration. The *local MTM* is proposed to describe local rigidity. It contains similarities and compatibilities of minutia pairs within *LMTS*s. The *global MTM* is proposed to deal with global nonlinearity. It contains similarities and compatibilities of *LMTS* pairs within two minutia sets. For true minutia pairs, most of them have both high similarities and high compatibilities, thus they will be judged as matched. Some of them may have high similarities and low compatibilities due to the nonlinearity, they will still be selected in the *global MTM* and judged as matched. Some of them may have low similarities and high compatibilities due to the sparseness or noise, they will be excluded at the local matching step firstly and then regained at the global matching step. True minutia pairs with both low similarities and low compatibilities are unsolvable. They may be regained through other matching features, such as ridges.

### 5.2 Efficiency of this algorithm

The proposed algorithm has promising efficiency. Minutia matching is formulated as seeking for the dense sub-block in the corresponding tensor matrix. Fingerprint matching is divided into two levels: local matching and global matching. In each level it contains two steps: building the tensor matrix and spectral iteration. We make some pruning when building the tensor matrixes. The pruned tensor matrix is much smaller than original matrix. The time for building tensor matrixes is acceptable. Meanwhile, the spectral relaxation is very fast, it usually converges in 20 iterations. Suppose there are two minutia sets *P* and *P'*, with *N*_{p} and *N*_{p'} minutiae, respectively, this algorithms contains these operations: *N*_{p} × *N*_{p'} times of local spectral optimization at the local matching step, one time of global spectral optimization at the global matching step, a *MLS* expanding operation and a convex hull operation. Each operation runs very fast.

### 5.3 Future work

As we have described in section 4.1, true minutia pairs with both low similarities and low compatibilities are unsolvable. They may be regained through other matching features, such as qualities and ridges. Our future work shall focus on: (1) adding new matching features, such as phase difference feature, ridges; (2) taking minutia quality into consideration. We will try to add other features into our algorithm framework.

## Conclusions

In this paper, we propose the novel tensor strategy for fingerprint matching. First, the concept of minutia tensor matrix is proposed. It unifies both the first-order features and the second-order features appropriately. Second, correct minutia pairs correspond to the dense sub-block in the *MTM*. Minutia matching is formulated as recovering the main dense sub-block. Spectral matching methods are then used for recovering the dense sub-block. It is efficient and effective. Third, two different kinds of *MTM*s are constructed. The local *MTM* is constructed to calculate similarities of *LMTS* pairs. It makes use of local rigidity. The global *MTM* is constructed to calculate entire similarity. It takes global nonlinearity into consideration.

In summary, this paper proposes a novel structural similarity calculation and a novel approach for fingerprint recognition. It makes better use of similarities and compatibilities. It has stronger description ability and better robustness to nonlinearity and noise. Experimental results demonstrate the effectively and accuracy of our algorithm.

## Author Contributions

Conceived and designed the experiments: XF. Performed the experiments: XF. Analyzed the data: JF. Contributed reagents/materials/analysis tools: JF. Wrote the paper: XF.

## References

- 1.
Maltoni D, Maio D, Jain AK, Prabhakar S. Handbook of fingerprint recognition. springer; 2009.
- 2. Maio D, Maltoni D, Cappelli R, Wayman JL, Jain AK. FVC2002: Second fingerprint verification competition. Pattern Recognition, Proceedings, 16th IEEE International Conference on. 2002; 3: 811–814.
- 3. Cappelli R, Maio D, Maltoni D, Wayman JL, Jain AK. Performance evaluation of fingerprint verification systems. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2006; 28: 3–18.
- 4. Jain AK, Chen Y, Demirkus M. Pores and ridges: high-resolution fingerprint matching using level 3 features. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2007; 29: 15–27.
- 5. Liu C, Cao J, Gao X, Fu X, Feng J. A novel fingerprint matching algorithm using minutiae phase difference feature. Image Processing (ICIP), 18th IEEE International Conference on. 2011; 3201–3204.
- 6. Choi H, Choi K, Kim J. Fingerprint matching incorporating ridge features with minutiae. Information Forensics and Security, IEEE Transactions on. 2011; 6: 338–345.
- 7. Izadi MH, Mirmohamadsadeghi L, Drygajlo A. Introduction of cylinder quality measure into minutia cylinder-code based fingerprint matching. Biometrics: Theory, Applications and Systems (BTAS), 15th IEEE International Conference on. 2012; 353–358.
- 8. Wen W, Qi Z, Li Z, Zhang J, Gong Y, Cao P. A Robust and Efficient Minutia-Based Fingerprint Matching Algorithm. Pattern Recognition (ACPR), 2nd IAPR Asian Conference on. 2013; 201–205.
- 9. Li J, Tulyakov S, Zhang Z, Govindaraju V. Fingerprint matching using correlation and thin-plate spline deformation model. Biometrics: Theory, Applications and Systems (BTAS), 2nd IEEE International Conference on. 2008; 1–4.
- 10. Thuy H, Thi N, Huan HX, Ky NN. An efficient method for fingerprint matching based on local point model. Computing, Management and Telecommunications (ComManTel), 2013 IEEE International Conference on. 2013; 334–339.
- 11. Ratha NK, Bolle RM, Pandit VD, Vaish V. Robust fingerprint authentication using local structural similarity. Applications of Computer Vision, 15th IEEE Workshop on. 2000; 29–34.
- 12.
Chen X, Tian J, Yang X. A matching algorithm based on local topologic structure. In: Image Analysis and Recognition. Springer; 2004. pp. 360–367.
- 13.
Chikkerur S, Cartwright AN, Govindaraju V. K-plet and coupled BFS: a graph based fingerprint representation and matching algorithm. In: Advances in Biometrics. Springer; 2005. pp. 309–315.
- 14.
Xu W, Chen X, Feng J. A robust fingerprint matching approach: Growing and fusing of local structures. In: Advances in Biometrics. Springer; 2007. pp. 134–143.
- 15. Cappelli R, Ferrara M, Maltoni D. Minutia cylinder-code: A new representation and matching technique for fingerprint recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2010; 32: 2128–2141. pmid:20975113
- 16. Medina-Pérez MA, García-Borroto M, Gutierrez-Rodríguez AE, Altamirano-Robles L. Improving fingerprint verification using minutiae triplets. Sensors. 2012; 12: 3418–3437. pmid:22737016
- 17. Jea TY, Govindaraju V. A minutia-based partial fingerprint recognition system. Pattern Recognition. 2005; 38: 1672–1684.
- 18. Feng Y, Feng J, Chen X, Song Z. A novel fingerprint matching scheme based on local structure compatibility. Pattern Recognition (ICPR), 18th IEEE International Conference on. 2006; 4: 374–377.
- 19. Cao J, Feng J. A robust fingerprint matching algorithm based on compatibility of star structures. Sixth International Symposium on Multispectral Image Processing and Pattern Recognition, International Society for Optics and Photonics. 2009; 74983X–74983X–7.
- 20. Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2002; 24: 509–522.
- 21. Horaud R, Skordas T. Stereo correspondence through feature grouping and maximal cliques. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 1989; 11: 1168–1180.
- 22. Duchenne O, Bach F, Kweon I-S, Ponce J. A tensor-based algorithm for high-order graph matching. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2011; 33: 2383–2395.
- 23. Leordeanu M, Hebert M. A spectral technique for correspondence problems using pairwise constraints. Computer Vision (ICCV), Tenth IEEE International Conference on. 2005; 2:1482–1489.
- 24.
Fu X, Liu C, Bian J, Feng J. Spectral correspondence method for fingerprint minutia matching. Pattern Recognition (ICPR), 21st IEEE International Conference on. 2012; 1743–1746.
- 25.
Fu X, Liu C, Bian J, Feng J, Wang H, Mao Z. Extended clique models: A new matching strategy for fingerprint recognition. Biometrics (ICB), 6th IEEE International Conference on. 2013; 1–6.
- 26. Schaefer S, McPhail T, Warren J; Image deformation using moving least squares. ACM Transactions on Graphics (TOG). 2006; 25: 533–540.