Figures
Abstract
Due to being derived from linear assumption, most elastic body based non-rigid image registration algorithms are facing challenges for soft tissues with complex nonlinear behavior and with large deformations. To take into account the geometric nonlinearity of soft tissues, we propose a registration algorithm on the basis of Newtonian differential equation. The material behavior of soft tissues is modeled as St. Venant-Kirchhoff elasticity, and the nonlinearity of the continuum represents the quadratic term of the deformation gradient under the Green- St.Venant strain. In our algorithm, the elastic force is formulated as the derivative of the deformation energy with respect to the nodal displacement vectors of the finite element; the external force is determined by the registration similarity gradient flow which drives the floating image deforming to the equilibrium condition. We compared our approach to three other models: 1) the conventional linear elastic finite element model (FEM); 2) the dynamic elastic FEM; 3) the robust block matching (RBM) method. The registration accuracy was measured using three similarities: MSD (Mean Square Difference), NC (Normalized Correlation) and NMI (Normalized Mutual Information), and was also measured using the mean and max distance between the ground seeds and corresponding ones after registration. We validated our method on 60 image pairs including 30 medical image pairs with artificial deformation and 30 clinical image pairs for both the chest chemotherapy treatment in different periods and brain MRI normalization. Our method achieved a distance error of 0.320±0.138 mm in x direction and 0.326±0.111 mm in y direction, MSD of 41.96±13.74, NC of 0.9958±0.0019, NMI of 1.2962±0.0114 for images with large artificial deformations; and average NC of 0.9622±0.008 and NMI of 1.2764±0.0089 for the real clinical cases. Student’s t-test demonstrated that our model statistically outperformed the other methods in comparison (p-values <0.05).
Citation: Zhang J, Wang J, Wang X, Gao X, Feng D (2015) Physical Constraint Finite Element Model for Medical Image Registration. PLoS ONE 10(10): e0140567. https://doi.org/10.1371/journal.pone.0140567
Editor: Alena Talkachova, University of Minnesota, UNITED STATES
Received: December 22, 2014; Accepted: September 28, 2015; Published: October 23, 2015
Copyright: © 2015 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: This study supported by the Natural Science Foundation of Jiangsu Province China through Grant Number BK2008159 (http://jstd.gov.cn) and the Natural Science Foundation of Suzhou Number SYG201113 (http://www.szkj.gov.cn) provided to the co-author Jiajun Wang. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Widely available biomedical images are essential for more accurate diagnosis and management of patients with a variety of diseases. Registration of these medical images into a spatial correspondence and alignment is important for fusion and maximization of the underlying information in the imaging datasets. Image registration is achieved by estimating an optimal transformation or displacement field [1]. It is one of the most fundamental research areas for various clinical applications, including planning and delivery of radiotherapy treatment [2–3], image-guided surgery [4–5], cross modality image fusion [6], morphometric study [7] and treatment monitoring [8].
After more than three decades of intensive research, various registration methods have been proposed [9–11]. Among these methods, the intensity based registration has the advantage of direct usage of the image intensity information without necessary preprocessing of segmentation and extraction of features such as salient points, curves and surfaces as in the feature based registration. However, compared with feature based counterpart, these intensity based methods normally face the challenges of reducing high computational cost and avoiding local optimization.
Basically, two types of geometric transformations are used in the intensity based methods which are derived from interpolation theory or derived from physical models. Free-form deformation (FFD) method [12] and local affine transformation (LAT) method [13] are two most common types of interpolation strategies. In FFD method, a rectangular grid of control points is defined in order to determine the deformation. Displacements between control points are propagated by interpolation. This method requires few degrees of freedom to describe local deformations and is able to efficiently provide smooth results. However, the dependence on a regular grid of control points restricts their adaptability and it is difficult to change control points topology. Many extensions of FFD methods have been proposed [14–15]. LAT methods parameterize the transformation by locally linear deformation. The images are hierarchically partitioned into contiguous blocks and an affine transformation is recovered for each one of them. However, the underlying image content has not been considered in the splitting process, and the regularization providing for the global smoothness in LAT method may lead to the ambiguous matching [16].
The diffusion model and the elastic body model in the framework of finite element method are the currently recognized physical models for medical image registration. The diffusion model upon the optical flow (OF) constraint describes deformations by assuming a constant brightness constraint of floating voxels [17–18]. However, it lacked a sound theoretical justification [9]. In finite element elastic model, the deformation of the image was modeled as an elastic body that is described by the Navier-Cauchy partial differential equation [19]. The gradient of the similarity measure is used as an external force field which tries to deform the floating image to fit the reference image configuration, while the internal elastic forces of the solid oppose the deformation. Thus, the floating image is deformed until the internal and external forces reach an equilibrium state. On the basis of a linear assumption [20], the governing equation can be simplified and the nodal displacement matrix can be obtained under the nodal external force matrix in the framework of finite element analysis method. However, the linear assumption may be not valid for soft tissues with complex nonlinear behavior and cannot accurately simulate large organ deformations [21].
To address the challenging issue in the elastic registration, in this paper, we propose a new registration method based on the St. Venant–Kirchhoff (VK) model that is the simplest hyper-elastic material model for representing the material behavior of soft tissues. Furthermore, we extend this model in a dynamic fashion by using the Newtonian differential equation [22–23]. The geometrical nonlinearity of the continuum is taken into account by considering the quadratic term of the deformation gradient in the strain tensor when deriving the elastic force. For a more efficient implementation of our proposed realistic model, we adopt a hierarchical (also referred to as a “global-to-local”) strategy in our registration algorithm.
Methods and Materials
In the proposed deformable registration algorithm, the displacement field is obtained subject to external constraints imposed by the similarity metric of the image pair for registration. The equilibrium of our model is achieved by the minimization of the global energy functional that consists of elastic deformation energy and the energy enforced by the external image force.
2.1 Governing Equation for the Nonlinear Elastic Model
In the framework of the finite element analysis method, the displacement of any image pixel can be interpolated with the nodal displacement and the shape function. On the basis of an isotropic linear assumption, a matrix equation describing the motion of the elastic model can be expressed as:
(1)
where the stiffness matrix K is obtained from the mesh and elastic parameters.
is the nodal external force matrix obtained by optimizing the spatially encoded mutual information (SEMI), and u is the displacement vector of element nodes.
The linear elastic finite element model (LFEM) has an important limitation when coping with large deformations. To account for large deformations, we propose a nonlinear elastic model governed by the second order differential equation as below:
(2)
where M and C are the matrices of the mass and damping, respectively. These two matrices are interrelated to each other with C = αM where α is a scale factor. The mass matrix M is chosen to be a diagonal matrix with diagonal elements as
, where Vel is the volume of element el, and ρ is the mass density.
The elastic force matrix constrains the solution space and together with the external force, this elastic force guides optimization process to converge. Under the linear elastic condition,
according to Eq (1), thus Eq (2) is deduced to the linear elastic dynamic finite element model (DFEM) as below:
(3)
In our proposed nonlinear finite element model (NFEM), the elastic force matrix is derived from the elastic deformation energy and innovatively models the geometrical nonlinearity based on hyper-elastic material properties. Fig 1 illustrates the components governing equations for LFEM, DFEM and NFEM, and elaborates the differences of these three FEM registration methods.
The nonlinear governing Eq (2) is solved with an explicit integration scheme [24], where the displacement vector at time t+1 is computed from the elastic force and external force estimated at time t as follows:
(4)
where △t is the integration time step.
2.2 Modeling of Elasticity via Green-St.Venant Strain Measure
The hyperelastic material model has been proven appropriate to represent the material behavior of soft tissues and has been successfully applied in surgery simulation [25–26]. We used the simplest isotropic hyperelastic material model as proposed by Wittek et al. [27]. In this paper, we derive the elastic force to model tissue deformation on the basis of St.Venant-Kirchhoff (VK) material model [28]. The VK material is the simplest hyper-elastic material and holds nonlinear property between the strain and the displacement gradient, and in the meantime obeys Hooke’s law in constitutive relationships.
The deformation of the image pair for registration is assumed to be resulted from a large displacement such as the flexible plates or shell deformed with external forces. The strain relative to the initial (base) configure which is usually called the Green-St.Venant Strain and is defined as
(5)
where
(6)
where
is the right Cauchy-Green strain tensor, I is an identity tensor and JX is the deformation gradient. Physically, the right Cauchy-Green strain tensor leads to the square of local change in distances due to deformation. By substituting
in Eq (6) to Eq (5), we have
(7)
and with the assumption that |∇u|<<1, the quadratic term ∇uT∇u can be neglected, which is encountered in the linear elastic finite element model analysis.
The components of e can be rearranged as a 3-component strain vector ε for the finite element analysis of our elastic model as follows:
(8)
where
(9)
where hm is a 4×1 vector, Hm is a 4×4 sparse symmetric matrix, and g is the displacement gradient vector, i.e.,
(10)
(11)
(12)
where ux and uy are the displacement in x and y directions.
Assume an elastic continuum without initial stresses of strains, the elastic deformation energy We of an elastic body can be expressed as:
(13)
where Ω is the image domain, σ is the stress tensor which relates to the Green-St.Venant Strain in the following manner:
(14)
Eq (14) can be rewritten in a more compact form as σ = Dε. For the material with isotropic characteristics, matrixD D can be determined with the Young’s modulus E and Poisson’s ratio v as follows:
(15)
By substituting Eq (8) and Eq (14) to Eq (13), the elastic energy can be rewritten as:
(16)
where Wmn is usually called the strain energy density and can be expressed as follows:
(17)
where L is a differential operator given in Eq (12) and Dmn is the element of matrix D.
2.3 Elastic Force
According to the Delaunay criteria [29], uniform meshes that are composed of triangular elements Ωel are generated for both the reference image IR and the floating image IT. If the displacement vectors uiel(i = 1,2,3) of the triangular nodes are obtained, the displacement vectors u(x) of other points in the triangular element can be obtained approximately by the following interpolations:
(18)
where Niel(x)(i = 1,2,3) is the shape function of an element. Using linear interpolation, the shape functions are defined as the so-called natural coordinates Li of the element. i.e.,
(19)
With natural coordinates, the area Ael and the coefficients aiel, biel, ciel are all defined as following:
(20)
where (xi, yi) i = 1,2,3 is the global coordinates of element nodes, and the other coefficients are found by cyclic interchange of the indexes.
Given the expression of the elastic energy, the elastic force acting on node i of element el can be derived as:
(21)
and by substituting Eq (18) to Eq (17),
can be written as:
(22)
where
, and L is a differential operator given in Eq (12).
as determined in Eq (21) can be assembled to the global
in the image domain Ω.
2.4 External Force
In the framework of FEM, the nodal external force matrix can be assembled by the external force
of all element el, which is defined as follow:
(23)
where R(x) is the external force field subjected to the gradient flows of the matching metric, Nel is the shape function vector assembled by Niel(x)(i = 1,2,3).
Our aim of registration is to find a deformation field so that the registration similarity, whch is represented as spatially encoded mutual information (SEMI) [30], can be maximized. Mathematically, the matching metric increases faster along its gradient direction. Therefore, a plausible choice of the external force would be the gradient flows of the matching metric which would then drive the tissue to deform in a trend increases the matching similarity.
The gradient flows of the similarity metric between reference image r(x) and floating image m(x) is defined as follow:
(24)
where
is the intensity value domain and Gs(x, x′) is the weighting function for spatial information encoding, i.e.,
(25)
where γ is the standard deviation, Ns = ∫Ω Gs(x, x′)dx is the normalization factor and ϕ(⋅,⋅) is the Parzen intensity kernel, i.e.,
(26)
(27)
where
(28)
where β is the smoothing parameter for the Parzen estimates, ps(r’) and ps(m’) are the region marginal intensity probability density of the reference image and floating image respectively.
2.5 Hierarchical Registration Strategy
For a more efficient implementation of our proposed registration method for images with large deformation, a hierarchical strategy (often referred to as the “global-to-local” strategy) was adopted in our registration procedure. A schematic illustration of the block diagram of our hierarchical registration method is shown in Fig 2.
The global registration is to find the transformation matrix maximizing the similarity metric that is mutual information in our paper [31]. After the global registration, a subsequent local non-rigid registration was performed based on our proposed NFEM. We constructed the coarse-to-fine registration pyramid by firstly performing the filtering of the images with a Gaussian low pass filter for noise reduction and then setting uniform mesh nodes using the predefined element size to generate the mesh pyramid.
2.6 Parameters Setting
We set three types of parameters in our method. The first one is the elastic material related parameters. For simplicity, we considered our model as a homogeneous isotropic hyperelastic model and we fixed the elastic modulus E = 100kPa and the Poisson’s ratio v = 0.45 as in [32]. The second type is the parameters related to the optimization procedure including the number of hierarchical levels, the element sizes and the time step for the explicit integration scheme. In our experiments, the spatial resolution of the images is 256×256. Upon pyramid generation, excessively larger number of nodes would increase computational requirement while excessively smaller number of nodes would lead to unacceptable loss of information. In our experiments, meshes with 64×64, and 32×32 nodes were used to construct a two–level multi-resolution pyramid for better balance of computational efficiency and registration accuracy.
Since the explicit integration scheme for the nonlinear elastic model is only conditionally stable, a thorough analysis about it has been presented by determining a suitable time step. Similar to the procedure as in [33], we chose △t = 0.004, the value of the damping coefficient and the mass density were deduced according to the predefined spectral radius ρ* (ρ* = 0.99) [33]. These parameters are summarized in Table 1.
The third type of parameters is for SEMI metric including the size of the local region denoted as the diameter d, the standard deviation γ and the smoothing parameter β for the Parzen estimates. As analyzed in [30], there was a tradeoff between the global robustness and the local accuracy. The large local region contributes more information for estimating the joint intensity probability density, while the small local region contributes the better accuracy in the local region. For obtaining optimized values for the region diameter d, analysis of registration accuracy over 20 abdominal CT image pairs for different d values was conducted.
2. 7 Materials
Evaluations of our nonlinear finite element model (NFEM) were performed on 30 clinical CT image pairs with artificial deformations. These abdominal and chest CT images were acquired from the First Affiliated Hospital of Soochow University and used as the reference images. 10 B-Spline based small synthetic deformation fields and 10 large synthetic deformation fields were used to warp the reference image to generate floating images for the nonrigid registration. In order to compare our method as well as the LFEM and the DFEM methods with the RBM method, 10 different non-uniform artificial fields were used to generate ten floating images, instead of the B-Spline based predefined fields used above. The deformation exceeding 15 pixels in local regions was considered as “large” deformation and was included to validate our proposed NFEM algorithm.
We also evaluated the performance of our proposed algorithm for clinical images including 10 chest CT image pairs and 20 brain MRI image pairs. The chest CT image pairs were acquired from two different treatment periods of 10 patients in the First Affiliated Hospital of Soochow University. The brain MRI images from different healthy volunteers were collected and made available by the CASILab at The University of North Carolina at Chapel Hill and distributed by the MIDAS Data Server at Kitware, Inc [34]. We randomly selected 21 T1-Flash Images and chose one T1-Flash image as the reference and the rest 20 T1-Flash images as floating images.
2.8 Ethics Statement
The study about CT images was carried out according to the Helsinki Declaration and approved by the ethical committee of the First Affiliated Hospital of Soochow University. The need for informed consent was waived, because the data used in this study had already been collected for clinical purposes. Furthermore, the present study did not interfere with the treatment of patients and the database was organized in a way that makes the identification of an individual patient impossible. The study about MR images was carried out according to the Helsinki Declaration and approved by the ethical committee of the University of North Carolina at Chapel Hill. All subjects provided signed consent allowing images to be made publicly available on the website. The database was organized in a way that makes the identification of an individual patient impossible.
2.9 Registration Accuracy Assessment
To evaluate the proposed NFEM algorithm for non-rigid registration, we compared it with 1) the conventional LFEM registration algorithm; 2) DFEM registration algorithm; and 3) the robust block matching (RBM) based registration software which was called “Nifty_Reg” and developed at University College London containing programs to perform rigid, affine and the RBM algorithms for nonlinear registration [35].
Evaluation was carried out by visual assessment in subtraction images between the reference image and the floating image after registration and also by three popular measures: the mean square difference (MSD), the normalized correlation (NC) and the normalized mutual information (NMI). Furthermore, for the registration of the images with artificial deformations, 40 random foreground points were automatically selected from the reference images. Then the mean and max distance of the corresponding points in the image pairs before and after registration was calculated to measure the registration accuracy.
Results
Curves in Fig 3 show the impact of parameter d on the average values of NC and NMI. We found d = 15 contributed the highest NC and NMI values. The standard deviation γ was set to a small value, typically γ = 0.25*d.
(a) Values at level 0 with the mesh resolution 8×8; (b) Values at level 1 with the mesh resolution 4×4.
3.1 Registration Results of Medical Images with Uniform Deformations
Fig 4 illustrates two examples of the image pairs for registration, where Fig 4B is the floating image derived from reference image Fig 4A upon large uniform artificial deformation, and Fig 4C is the floating image derived from Fig 4A upon relatively small uniform deformation. 40 random positions in the object region chosen as label points were marked in red in Fig 4A. The corresponding positions in the floating image were acquired from pre-defined artificial large deformation fields and marked in green in Fig 4B; As shown in Fig 4C, the amount of this kind of deformation was set to exceed 15 pixels in local regions. Fig 4D and 4E show the color images of the distance of a point in the reference to its counterpart in the floating image to display the deformation degree in local regions.
(a) The reference image attached 40 points (in red color and with white indexes) as registration ground truth; (b) and (c) are the floating image attached 40 points (in green color and with white indexes); (d) and (e) are the color images of deformations.
The visual assessment and comparison of the registration results by observing the sub-regions in the squares were performed as shown in Fig 5. The more corresponding and similar the registration results and the reference images are, the more capable the algorithm of tackling large deformations. As demonstrated in Fig 5D, our registration result was more corresponding to the reference image.
(a) the registration result corresponding to the floating image with the LFEM method; (b) the registration result with DFEM method; (c) the registration result with RBM method; (d) the registration result with NFEM method. Squares in images highlight the local differences.
Table 2 shows the quantitative evaluations of different methods on the ten pairs of images with large uniform deformations in terms of the mean distance (mean_x and mean_y) and maximal distance (max_x and max_y) between true and recovered deformation in the x and y direction. The mean values of the mean and max distances of our method were the lowest when compared to the mean values from other three methods, which indicated that our method achieved the highest registration accuracy. Student’s t-test (p-values <0.05) validated this improvement was statistically significant.
The comparisons of our proposed NFEM method with other three methods in terms of MSD, NC and NMI (shown in Table 3) demonstrated that the mean values of NC and NMI using our NFEM was greater than the corresponding values using other three methods, and consistently, the mean value of MSD using NFEM was the lowest. Student’s t-test (p-values <0.05) consistently validated our method statistically outperformed its counterpart comparison methods.
The quantitative validation on ten pairs of images with small uniform deformations is displayed in Table 4. Table 4 summarized the mean distances and different metrics for 10 image pairs with small deformation and demonstrated that the RBM achieved the best registration results followed by our NFEM. In comparison, although there were no obvious matching errors in the registration results shown in Fig 6, LFEM and DFEM algorithms slightly fell behind regarding the registration accuracy because of the infinitesimal deformation assumption.
(a) the registration result of the LFEM method; (b) the registration result of DFEM method; (c) the registration result of RBM method; (d) the registration result of NFEM method. Squares in imags highlight the local differences.
3.2 Registration Results of Medical Images with Non-uniform Deformations
Fig 7 illustrates the examples of the registration results over the image pairs with non-uniform artificial fields. Circles in the subtraction images highlight the difference in the local regions in the results from different methods. The performance comparisons among different methods in term of MSD, NC and NMI are illustrated in Table 5. In addition, the Student’s t-test (p-values <0.05) shown in Table 5 indicated that our NFEM method significantly outperformed other three methods. The visual assessment displayed as the subtraction images in Fig 7 further consolidated the same conclusion.
The first two rows present the reference image, the floating image and the registration results with the RBM method, LFEM method, DFEM method and NFEM method. The bottom row shows the subtraction images before registration and the ones using RBM and NFEM methods. Circles in (e) and (f) highlight the local differences in the subtraction image.
3.3 Registration Results of Real Medical Images
Fig 8 illustrates two example image pairs used in our registration method evaluation. Fig 8A is a pre-chemotherapy image and Fig 8B is the post-chemotherapy image after about three month’s treatment. Red circles in Fig 8A and 8B highlight the local variations because of the chemotherapy, while the variation in blue squares is resulted from the deformation of chest tissues in the different body position. Fig 8C and 8D are a pair of T1-Flash images from different subjects.
(a) reference image of chest CT image pairs,(b) floating image of chest CT image pairs, (c) reference image of brain MR image pairs, (d) floating image of brain MR image pairs.
The statistical performance in terms of NC and NMI for different methods in both chest CT and brain MRI cases is shown in Fig 9 where these two matching metrics are illustrated by means of statistic box-plots. The t-tests were implemented and results are illustrated in Table 6 and Table 7, which indicated that, at the 0.05 level, the mean value for NC and NMI of all registration validation using method NFEM is significantly greater than the ones using Global, LFEM, DFEM, RBM method.
(a) NC parameters for box-plots. (b) NMI parameters for box-plots. In CT image registration cases, these methods are represented by appending a suffix 1 (e.g. Global1) to the name abbreviations, while in MRI image cases, the suffix 2 is added to their name abbreviations (e.g. Global2).
All evaluations were implemented with C on Windows 7 operating system and performed on a DELL desktop with Intel(R) Core(TM) i7-4770 @ 3.4GHz CPU. Using the aforementioned two-level pyramid, the total freedom degree for computing the displacement field was 10240. The average computation time using LFEM was 3 minutes and 35 seconds and the one using DFEM was 4 minutes and 3 seconds. The average computation time using our NFEM method was 5 minutes and 26 seconds.
Discussion
Complicated nonlinear characteristics of the soft tissues or the tremendous deformations introduced by the internal organ movements or treatment interventions pose significant challenges for the current non-rigid image registration algorithms. Underlying linear hypotheses of these algorithms are one of the foremost reasons to underpin the requirement for further improvement of these nonlinear methods. We took into account the geometric nonlinearity of elastic body model. In our algorithm, the elastic force was modeled as the derivative of the deformation energy with respect to the nodal displacement vectors of the finite element; the external force was derived from the registration similarity gradient flow.
We validated our algorithm firstly on image pairs with artificial deformations and then on real CT and MRI images. We compared our NFEM method with three other methods: 1) the conventional LFEM registration algorithm; 2) DFEM registration algorithm; and 3) the robust block matching (RBM) algorithm.
To validate the performance of our algorithm when registering images with large deformations, we conducted experiments on ten pairs of images with large artificial deformations. The visual assessment and comparison of the registration results were performed by observing the sub-regions in the squares as shown in Fig 5. As demonstrated in Fig 5D, our registration result was more corresponding to the reference image, which indicated that our proposed NFEM algorithm outperformed other three algorithms. The better registration results from our algorithm were mainly due to the contribution of the internal elastic force that took into account the geometrical nonlinearity to compensate for the local large displacements.
The quantitative evaluations of different methods (as shown in Table 2 and Table 3) on the ten pairs of images with large uniform deformations consistently demonstrated that our method statistically outperformed its counterpart comparison methods. These better statistical results from our algorithm also indicated that the nonlinear internal elastic force played the positive role in the registration.
The quantitative validation on ten pairs of images with small uniform deformations as displayed in Table 4 demonstrated that the RBM achieved the best registration results followed by our NFEM. The better accuracy from RBM method benefited from the fact that RBM was a B-Spline based method and it was therefore inherently more suitable for handling the small uniform deformations generated by B-Spline interpolation. In comparison, although there were no obvious matching errors in the registration results shown in Fig 6, LFEM and DFEM algorithms slightly fell behind regarding the registration accuracy because of the infinitesimal deformation assumption. The registration results in Table 2 and Table 4 demonstrated that our method was more robust for the cases with both large deformations and small deformations.
Meanwhile, 10 corresponding floating images were generated by pre-defined non-uniform artificial fields to investigate the feasibility of our method for estimating large non-uniform deformations. The performance of different methods was compared in terms of MSD, NC and NMI. The Student’s t-test (p-values <0.05) shown in Table 5 indicated that our NFEM method significantly outperformed other three methods. The visual assessment displayed as the subtraction images in Fig 7 further consolidated the same conclusion. The nonlinear elasticity modeling via Green-St.Venant strain measure contributed to the improved deformation estimation and hence led to the improved registration accuracy for the images with non-uniform deformations.
In addition to the evaluations of our proposed algorithm on artificially deformed images, we validated our algorithm on clinical CT and MRI images as shown in Fig 8. The statistic box-plots illustrated in Fig 9 demonstrated that our method was able to effectively compensate the deformations. Statistical analysis as illustrated in Table 6 and Table 7 indicated that, at the 0.05 level, the mean value for NC and NMI of all ten registration validation using method NFEM is significantly greater than the ones using Global, LFEM, DFEM, RBM method, which demonstrated that our proposed NFEM algorithm outperformed other methods for registering real clinical image pairs and would be potentially usable for assessing disease progression or patient response to treatment.
While our method achieved improved registration accuracy compared to other linear elastic registration models, it has a tradeoff in regard to the computation time. Theoretically, explicit integration in our NFEM method should contribute to improve the computational efficiency. However, updating the relatively complicated elastic force in the iterations was computationally costly. The convergence towards the steady-state solution would be increased if the integration time step could be adaptively updated during iterations.
Conclusion
In this paper, a novel physics-based nonlinear registration algorithm is proposed based on the FEM elastic body model in the framework of hierarchical strategy. The novelty of this algorithm is the inclusion of nonlinearity between the strain and the displacement gradient based on St.Venant-Kirchhoff model. This nonlinear model is represented as a second order differential equation to obtain equilibrium between the internal and the external forces. Experimental validation on images with artificial deformations and real medical images demonstrated that our nonlinear FEM elastic model outperformed the traditional LFEM model, DFEM method, and RBM method in terms of the quantitative and qualitative analysis.
Author Contributions
Conceived and designed the experiments: JZ JW XW. Performed the experiments: JZ. Analyzed the data: JZ. Contributed reagents/materials/analysis tools: JZ JW XW XG DF. Wrote the paper: JZ JW XW XG DF. Designed the software used in analysis: JZ.
References
- 1. Maintz JB, Viergever MA (1998) A survey of medical image registration. Medical Image Analysis 2: 1–36. pmid:10638851
- 2. Markelj P, Tomaževič D, Likar B, Pernus F (2012) A review of 3D/2D registration methods for image-guided interventions, Medical image analysis 16: 642–661. pmid:20452269
- 3. Roujol S, Benois-Pineau J, de Senneville BD, Ries M, Quesson B, Moonen CTW (2012) Robust Real-Time-Constrained Estimation of Respiratory Motion for Interventional MRI on Mobile Organs. IEEE Transactions on Information Technology in Biomedicine 16:365–374. pmid:22411045
- 4. Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, et al. (2012) Intraoperative Image-based Multiview 2D/3D Registration for Image-Guided Orthopaedic Surgery: Incorporation of Fiducial-Based C-Arm Tracking and GPU-Acceleration. IEEE Transactions on Medical Imaging 31: 948–962. pmid:22113773
- 5. Ferrant M, Nabavi A, Macq B, Black P, Jolesz F, Kikinis R, et al. (2012) Serial registration of intraoperative MR images of the brain. Medical Image Analysis 6:337–360.
- 6.
Wang X, Eberl S, Fulham M, Som S, Feng DD. Data registration and fusion. In: Feng DD (Eds). Biomedical information technology. California: Academic Press. 2008. 187–210.
- 7. Han Z, Tricia ATW, Elisabeth MD, John CG, Benoit MD (2012), Effect of nonrigid registration algorithms on deformation-based morphometry: a comparative study with control and Williams syndrome subjects. Magnetic Resonance Imaging 30: 774–788. pmid:22459439
- 8. Xue H, Derrek PH, Christopher RKC, Boyle CP, Rajagopalan P, Gutman BA, et al. (2013) Unbiased tensor-based morphometry: Improved robustness and sample size estimates for Alzheimer's disease clinical trials. NeuroImage 66: 648–661. pmid:23153970
- 9. Sotiras A, Davatzikos C, Paragios N (2013) Deformable Medical Image Registration: A Survey. IEEE Transactions on Medical Imaging 32:1153–1190. pmid:23739795
- 10. Oliveira FPM, Tavares JMRS (2014) Medical image registration: a review. Computer methods in biomechanics and biomedical engineering 17: 73–93. pmid:22435355
- 11. Wei L, Eraldo R (2011) A survey on image-based continuum-body motion estimation. Image and Vision Computing 29: 509–523.
- 12. Rueckert D, Sonoda L, Hayes C, Hill D, Leach M, Hawkes D (1999) Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging 18: 712–721. pmid:10534053
- 13. Periaswamy S, Farid H (2003) Elastic registration in the presence of intensity variations. IEEE Transactions on Medical Imaging, 22(7):865–874. pmid:12906240
- 14. Zhuang X, Arridge S, Hawkes D J, Ourselin S (2011) A Nonrigid Registration Framework Using Spatially Encoded Mutual Information and Free-Form Deformations. IEEE Transactions on Medical Imaging 30:1819–1828. pmid:21550878
- 15. De Craene M, Piella G, Camara O, Duchateau N, Silva E, Doltra A, et al. (2012) Temporal diffeomorphic free-form deformation: Application to motion and strain estimation from 3D echocardiography. Medical Image Analysis 16: 427–450. pmid:22137545
- 16. Christian B, Tobias S, Andrew PK (2011) Hierarchical adaptive local affine registration for fast and robust respiratory motion estimation. Medical Image Analysis 15:551–564. pmid:21454119
- 17. Lu X, Zhao Y, Zhang B, Wu JS, Li N, Jia WT (2013) A non-rigid cardiac image registration method based on an optical flow model. Optik—International Journal for Light and Electron Optics 124: 4266–4273.
- 18. Lombaert H, Grady L, Pennec X, Ayache N, Cheriet F (2014) Spectral Log-Demons: Diffeomorphic Image Registration with Very Large Deformations. International Journal of Computer Vision 107:254–271.
- 19. Christos D (1997) Spatial Transformation and Registration of Brain Images Using Elastically Deformable Models. Computer Vision and Image Understanding 66:207–222. pmid:11543561
- 20. Zhang J, Wang J, Wang X, Feng DG (2014), The adaptive FEM elastic model for medical image registration. Physics in Medicine and Biology 59: 97–118. pmid:24334618
- 21. Guillaume P, Hervé D, Nicholas A (2003) Non-linear anisotropic elasticity for real-time surgery simulation. Graphical Models 65: 305–321.
- 22. Xu SP, Liu XP, Zhang H, Hu LY (2011) A Nonlinear Viscoelastic Tensor-Mass Visual Model for Surgery Simulation. IEEE Transactions on Instrumentation and Measurement 60:14–20.
- 23. Horton A, Wittek A, Joldes GR, Miller K (2010) A meshless Total Lagrangian explicit dynamics algorithm for surgical simulation. Int J Numer Meth Biomed Eng 26: 977–998.
- 24. ZHang GY, Wittek A, Joldes GR, Jin X, Miller K (2014) A three-dimensional nonlinear meshfree algorithm for simulating mechanical responses of soft tissue. Engineering Analysis with Boundary Elements 42: 60–66.
- 25. Taylor ZA, Cheng M, Ourselin S (2008) High-Speed Nonlinear Finite Element Analysis for Surgical Simulation Using Graphics Processing Units. IEEE Transactions on Medical Imaging 27: 650–663. pmid:18450538
- 26. Miller K, Horton A, Joldes GR, Wittek A (2012) Beyond finite elements: A comprehensive, patient-specific neurosurgical simulation utilizing a meshless method. Journal of Biomechanics 45: 2698–2701. pmid:22935689
- 27. Wittek A, Joldes G, Couton M, Warfield SK, Miller K (2010) Patient-specific non-linear finite element modelling for predicting soft organ deformation in real-time; Application to non-rigid neuroimage registration. Progress in Biophysics and Molecular Biology 103: 292–303. pmid:20868706
- 28. Van Dyke TJ, Hoger A (2000) A comparison of second-order constitutive theories for hyperelastic materials. International Journal of Solids and Structures 37: 5873–5917.
- 29.
Georges PL. Automatic mesh generation and finite element computation. In: Ciarlet P., Lions J. (Eds.), Handbook of Numerical Analysis (Part 2) Finite Element methods, Numerical Methods for Solids. North Holland: Elsevier Science Ltd. 1996. 4: 69–192.
- 30. Zhuang X, Arridge S, Hawkes DJ, Ourselin S (2011) A Nonrigid Registration Framework Using Spatially Encoded Mutual Information and Free-Form Deformations. IEEE Transactions on Medical Imaging 30: 1819–1828. pmid:21550878
- 31. Pluim JPW, Maintz JBA, Viergever MA (2003) Mutual-information-based registration of medical images: a survey. IEEE Transactions on Medical Imaging 22(8):986–1004. pmid:12906253
- 32.
Marami B, Sirouspour S, Capson DW (2011) Model-Based Deformable Registration of Preoperative 3D to Intraoperative Low-Resolution 3D and 2D Sequences of MR Images. In: Fichtinger G, Martel A, Peters T (Eds). Medical Image Computing and Computer-Assisted Intervention–MICCAI. Berlin Heidelberg: Springer 6891: 460–467.
- 33. Joldes GR, Wittek A, Miller K (2009) Computation of intra-operative brain shift using dynamic relaxation. Computer Methods in Applied Mechanics and Engineering 198(41–44): 3313–3320
- 34. Bullitt E, Zeng D, Gerig G, Aylward S, Joshi S, Smith JK, et al. (2005) Vessel tortuosity and brain tumor malignancy: A blinded study. Academic Radiology 12:1232–1240. pmid:16179200
- 35. Modat M, Ridgway GR, Taylor ZA, Lehmann M, Barnes J, Fox NC, et al. (2010) Fast free-form deformation using graphics processing units. Computer Methods Programs in Biomedicine 98:278–284. pmid:19818524