Introduction

The increasing availability of magnetic resonance images (MR images, MRI) at 1 mm³ resolution has made it possible to build realistic high-resolution models of individual human heads. Accurate segmentations of the whole-head anatomy are important for the “forward modeling” of current flow in electroencephalography (EEG) and trancranial electric stimulation (TES), as well as their magnetic equivalents—MEG and TMS [1–8]. It is becoming increasingly clear that individual anatomy including complete cerebrospinal fluid (CSF), skull and scalp are key to obtain meaningful source localization in EEG and targeting of TES for individual subjects [1, 5, 9–12].

Unfortunately, currently available segmentation tools are of limited utility for this purpose. Statistical Parametric Mapping version 8 (SPM8, [13]) provides segmentation of the brain, CSF, skull and scalp using the Unified Segmentation algorithm [14], but its field of view (FOV) does not cover the whole head. FieldTrip [15] and the Brain Extraction Tool (BET, [16]) in the FMRIB Software Library (FSL, [17]) can extract the skull and scalp surfaces, but they also only operate on the standard FOV (the brain area only). The FMRIB Automated Segmentation Tool (FAST, [18]) and Integrated Registration and Segmentation Tool (FIRST, [19]) are designed, respectively, only for segmentation of brain tissues and subcortical structures. FreeSurfer [20–23], the Expectation–Maximization Segmentation tool (EMS, [24]), the Atlas Based Classification (ABC) and the EMSegmenter [25–27] in 3D Slicer [28], BrainSuite [29, 30], BrainVISA Morphologist [31, 32], the CIVET segmentation [33], the Sub-Volume Probabilistic Atlases Segmentation tool (SVPASEG, [34]), and the segmentation module in BrainVoyager [35, 36] also only focus on brain tissues. ITK-SNAP [37] and Neuroelectromagnetic Forward Head Modeling Toolbox (NFT, [38]) are semi-automated tools since they need user-specified seed point(s) to start. Commercial software tools, such as ASA (ANT Software B.V., Enschede, Netherlands), Curry (Compumedics NeuroScan, Charlotte, NC), BESA (BESA GmbH, Gräfelfing, Germany) and ScanIP (Simpleware Ltd, Exeter, UK), either are semi-automated or cannot generate (accurate) segmentation for CSF.

Therefore, researchers have attempted several workarounds for the segmentation of the whole head for electromagnetic forward modeling, e.g., combination of different segmentation tools [2, 39], use of computed tomography (CT) images for skull segmentation [40], simple heuristics using thresholding and morphological operations [10, 38], addition of dummy model for the lower head and/or the neck [3], or manual segmentation [1]. In short, these approaches are either too specific, heuristic, or not fully automated for segmenting out CSF, skull, scalp and air cavities. In our view, the simplest way to adapt current automated segmentation tools for this purpose is to extend the atlas representing anatomical prior information to include those non-brain tissues and cover the whole head. This is implemented in SPM8 or in [41]. However, the resulting segmentations still have morphological errors (gray matter, CSF and skull layers are fragmented and some tissue voxels erroneously fall inside other tissues) and the tissue boundaries are unnecessarily rough making subsequent meshing and forward modeling intractable. An automated post-processing routine based on morphological and Boolean operations is proposed in [41], but the technique is ad hoc with arbitrary parameters. Since the morphological constraints can be easily encoded by Markov random field (MRF, [42, 43]), here in this work we further improve the method in [41] by combining the atlas with an MRF prior.

The goal of this work is to develop an open-source, fully automated segmentation tool for the purpose of forward modeling. It can separate the whole head into gray matter (GM), white matter (WM), CSF, skull, scalp and air cavities. The method combines the atlas and the MRF prior with a finite mixture (FM) model [44, 45] of image intensities into a single rigorous Bayesian framework. A variational expectation–maximization (EM) algorithm is used with an asynchronous updating scheme that guarantees convergence [46]. We extend the Potts model [47] such that its interaction energy encodes all possible combinations of neighboring tissue types to express strict anatomical constraints. The core algorithm is implemented in the C programing language and integrated into SPM8, where the image registration and bias-field correction are solved using Unified Segmentation [14]. Evaluation is performed on both simulated and real MR images. When comparing to the segmentation tools that are already part of SPM8, we find smoother and less fragmented tissue segmentations without compromising accuracy. As another contribution, this paper summarizes the research on MRI segmentation using atlas and the MRF over the last two decades and presents the general framework with clear mathematical derivations.

Methods

This section only gives an overview of the segmentation algorithm. For detailed mathematical derivations, the readers are referred to the Appendix, which carefully describes the probabilistic formalism and derives equations that combine spatial and morphological priors.

The probabilistic approach to automated segmentation aims to fit an FM model [45] to the MR image(s) such that the likelihood of observing the image(s) is maximized. This maximum likelihood (ML) estimation is usually implemented by the EM algorithm [48, 49]. When morphological constraints are incorporated as MRF prior to probability model, an extension of ML known as variational EM (VEM) [46, 50, 51] is used. It iterates between estimating the tissue probability for each voxel (E-step) and updating the model parameters (M-step). The iterative update equation for the E-step is given by (1) Here x_i is a label indicating the tissue class at voxel i. For mathematical convenience it is expressed as a K-dimensional unit vector with zeros for all K tissues, except for the tissue present in that voxel. Eq (1) iterates q(x_i), which quantifies the estimated probability of finding label x_i. After convergence of this iteration, the estimated probability gives the desired segmentation. Variable y_i represents the image intensity in voxel i (or several intensities in case of multi-image segmentation) and P(y_i|x_i, θ) is the probability of finding this intensity for a given tissue class. As in many other segmentation algorithms, this probability is assumed to be a Gaussian and its parameters θ are adjusted to the image during the segmentation process in the M-step (Eq. 3) of the EM algorithm. 𝓝_i is the set of neighbors of voxel i. In this work, we consider only the 6 immediate face-connected neighbors in 3D space. The two terms in the exponential come from the prior model, with matrix J_ij representing the MRF interaction between neigboring voxels and vector h_i representing the effect of the atlas on single voxels. β is a weighting constant that adjusts the relative contribution of these two terms and will be tuned using training data (see Evaluation). The quadratic term here can be seen as an extension to the traditional Potts model [47], whose neighboring interaction coefficients J_ij are given by the identity matrix.

Our implementation distinguishes between tissue class x_i and tissue type ${\overline{\mathbf{\text{x}}}}_i$ to allow for several classes to belong to the same tissue (e.g. skull bone encompasses cortical and cancellous bone, each with different image intensities). This results in an FM model for each tissue type, which is used in many segmentation algorithms [14, 44, 49]. The complete E-step then follows (2a) (2b) Here we are interested in the probability of tissue type, i.e., $q({\overline{\mathbf{\text{x}}}}_i)$, since we aim to segment out six tissue types: GM, WM, CSF, skull, scalp and air. The parameters ${\gamma }_{x\bar{x}}$ are equivalent to the conventional “mixing proportion”. More strictly, they represent the conditional probability of a class belonging to a specific tissue type $P({\mathbf{\text{x}}}_i|{\overline{\mathbf{\text{x}}}}_i)$ (see Appendix B.8). These parameters will also be adjusted to the image in the M-step. The complete M-step update equations including the update for the mean, covariance and mixing proportion of the Gaussian FM model are given by (3a) (3b) (3c) Here x_i = x is equivalent to x_i = x, representing the tissue class at voxel i (see Appendix B.1).

The parameters quantifying morphological and spatial priors, J_ij and h_i, respectively, can be obtained from prior data. Suppose we have a dataset of validated segmentations for a group of subjects, and these truth data are spatially normalized, then we can easily get (4a) (4b) where ⟨…⟩ refers to the sample average over the subjects, and j ∈ 𝓝_i. This m_i gives the prior probability of observing a specific tissue type at voxel i and is usually referred as “atlas”, or “tissue probability map (TPM)”. C_ij describes the joint probability of co-occurrence of tissues in neighboring voxels i and j, and will be referred as “tissue correlation map (TCM)” in the sequel. It is not trivial to compute J_ij and h_i from C_ij and m_i. In this work we will use the solution for the most simple case of two sites, N = 2 (see Appendix C). Up to a scalar that can be captured by parameter β it is given by (5a) (5b) Note that the mean value m_i is simply the sample estimate of P(x_i) and the correlation C_ij is the sample estimate of P(x_i, x_j). Consequently, C_ij ⋅ diag⁻¹(m_j) is the sample estimate of the conditional P(x_i∣x_j) and is column-normalized.

Since the TCM C_ij is dependent on locations, it may not be practical. First, this local TCM simply requires too much memory (for a typical 200×200×200-site lattice, each site having a 6×6 matrix with one of its 6 neighbors, that is 200³ × 6³ × 4bytes ≈ 6.4GB under 32-bit precision). Second, unless the database to compute the local TCM is very large, the values will be poorly estimated. Therefore, we propose a simplified TCM that is independent of locations with C_ij = C. Furthermore, since we have prior knowledge from neuroanatomy that some combinations of tissue neigboring are not possible (e.g., WM cannot be neighbor of scalp), we can directly set the corresponding element in C to 0. Motivated by the solution for 2 sites, we will also require C to be column-normalized and symmetric. Therefore, (6) Each row in C represents the possible tissue type that voxel i may belong to, given that its neighbor j is one of the six types indicated by the columns of C. Each of the diagonal elements d₁, d₂, …, d₆ is simply 1 minus the sum of other elements in the corresponding column since C is column-normalized. Then there are only 8 free parameters C₁, C₂, …, C₈ (each one between 0 and 1) to be learned from the prior data (see Evaluation for details). This C will be referred as global TCM.

A global C is a practical solution to the difficulties of estimating and storing a local TCM, yet it does lose some of the available prior information. For instance, in boundary regions where different tissue types are likely to co-occur, the values of C should be quite different from those well inside a tissue (e.g., within scalp, or the air outside of head) where co-occurrence of different tissues is all but impossible. For these interior region the identity matrix, C = I, used in the conventional Potts model [47], is more appropriate as it favors only neighbors with the same class label. On the boundary, however, co-occurrence of different tissues is likely and thus a more general C is needed. To balance between the encoding of this locality information and the potentially huge memory consumption, we propose a simple variant of global TCM, called regional TCM, which is simply an identity matrix I inside a specific tissue, and the same as global TCM elsewhere. To define the inside-tissue regions, we use the TPM as a reference. For each tissue type, those voxels that have a probability higher than 0.95 in the TPM are treated as the area inside that tissue. In short, (7) Note that for global and regional TCM, Eq 5a simply changes to J = logC.

Implementation

The proposed segmentation algorithm is implemented based on the Unified Segmentation algorithm of SPM8 [13]. SPM8 is a software package for Matlab (R2010b, MathWorks, Natick, MA) that is freely available and widely used to analyze brain signals. It implements an extended version of the Unified Segmentation (eUS) algorithm [14], which performs bias-field correction and registration with the atlas as part of the segmentation process. The actual Matlab function for this—“spm_preproc8.m”—is referred to as “New Segment” in the SPM8 manual. It is an extended implementation of Unified Segmentation allowing the use of multimodal data, and a different treatment of the mixing proportions γ that is in agreement with the present paper. We leverage this capability by initializing the segmentation algorithm with eUS. Therefore, the value of y_i is the intensity after the optimal bias-field correction, and the voxel locations i of the prior parameters h_i and J_ij have been optimally registered to the MRI that is to be segmented, and the initial values of posteriors $q({\mathbf{\text{x}}}_i,{\overline{\mathbf{\text{x}}}}_i)$ and the Gaussian FM model parameters μ_x, Σ_x, ${\gamma }_{x\bar{x}}$ are all initialized by the result of eUS. Note the local TCM is registered to the MRI using the same algorithm applied by the eUS for TPM registration. This step is used in general but was omitted in the evaluation of the simulated data from BrainWeb (see Evaluation), which is already aligned to the TPM/TCM. The implementation is published as a freely available add-on to SPM8, named Morphologically and Anatomically accuRate Segmentation (MARS, available to download at http://neuralengr.com/mars/, the TPM and TCM used for evaluation in this work are also available there). The user can configure all the options (image data, TPM, TCM, running mode, etc.) in a similar batch editor as that in the eUS. The evaluation of this algorithm is based on six-class segmentation. However, the implementation allows any number of classes provided by the corresponding TPM and TCM. In fact, the user can specify any custom TPM/TCM in MARS. With different running modes, the user can select running the original eUS, the eUS with a subsequent MRF-based clean-up (disabled by default in the latest update of SPM8, but can be enabled to a specific level by the user), or eUS followed by the present algorithm.

The E-step (Eq 2a) is implemented in the C programing language for speed and is integrated with Matlab. Each voxel is accessed and updated sequentially in a checkerboard manner. As shown by [46], this sequential (or asynchronous) updating scheme guarantees both theoretical and practical convergence, i.e., the free energy of the system will monotonically decrease to a finite value, which is not guaranteed using parallel (or synchronous) updating. A similar convergence criterion is employed as the one in [46], i.e., the maximal relative change of tissue volumes between consecutive iterations (8) where t is the iteration number, and L_x is the volume of the xth tissue class, defined by $L_x=\sum _{i=1}^{N}q(x_i=x)$. Convergence is declared once ɛ falls below 0.01%. The M-step (Eqs 3a, 3b and 3c) has a same form as that in [14]. Therefore, the same Matlab code is used as the implementation for M-step (mixture coefficient Eq 3c is different from that in [14], but “spm_preproc8.m” actually implements Eq 3c in this paper).

The total time needed to run the algorithm on a typical T1-weighted MRI of size 256×256×181 and 1 mm³ resolution is about 60–75 min, in which 10–15 min is consumed by the initialization with eUS. The peak memory load is approximately 50 GB when using local TCM and 2 GB for the global TCM. Note all processing time and memory usages in this work were measured on a machine with a 6-core Intel Xeon E5645 CPU at 2.4 GHz, and 96 GB physical memory.

Evaluation

Learning of prior model

Dataset I.

BrainWeb also provides for each of the 20 subjects with the hard segmentation, i.e., each voxel is assigned exclusively to one tissue class. The detailed 12-class hard segmentation was reduced to 6 classes by combining similar tissues together (e.g., fat and muscle integrated with skin). These segmentation ground truth are registered into the same dimension as the MRI scans. Small obvious errors in these hard segmentations were corrected manually in the commercial software ScanIP (version 4.2). The algorithm was evaluated on this dataset using the leave-one-out scheme. At the evaluation for each subject, the hard segmentations from other 19 subjects were used as the prior data to generate the local TCM and TPM using Eqs 4a and 4b, respectively, and then the energy terms were calculated by Eqs 5a and 5b. The MRF weighting constant β was tuned using the first subject by running the algorithm for different β and selecting the value with the smallest difference (least-square sense) from the known hard segmentation. The optimal value, β = 0.1, was then used for all the evaluations using the local TCM. As to the global TCM, it was learned also using the first subject. A general purpose optimization algorithm (Pattern Search [55], available in Matlab) was used to find the C in Eq 6 that gives the segmentation results closest to the true hard segmentations in a least-square sense. Reasonable C was obtained after 1000 computations of the objective function (the mean square error between generated segmentations and true segmentations), which took approximately 6 days when utilizing the parallel computing ability of Matlab. The resulting C was then used for the evaluation of the algorithm on the other 19 subjects.

Dataset II and III.

The images in Dataset II were first automatically segmented by the eUS, binarized and then manually improved in ScanIP and used as ground truth. As to Dataset III, it provides the truth data with 10-class labeling, which was reduced to 6 classes by combining similar tissues. Since the number of subjects is limited, the local TCM and TPM cannot be generated from the true segmentation, we only evaluated the algorithm on the global TCM and used the TPM from our previous work (see [41], the TPM developed by Dr. C. Rorden). The global TCM C was learned using Head 1 in Dataset II by the same procedure as adopted in Dataset I, and then used for the evaluation on the other 7 real MRI volumes. Table 1 shows the learned parameters in the global TCM matrix as in Eq 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The optimal values for the parameters in Eq 6, learned from prior data.

https://doi.org/10.1371/journal.pone.0125477.t001

For the three datasets, the regional TCM was obtained following Eq 7.

Results

The proposed algorithm was evaluated on both simulated and real MRI data. For each subject in the simulated data (Dataset I), we compared the performance of the proposed algorithm (with three types of TCM: global, local, regional) to the performance of the eUS in SPM8, and the eUS with an MRF-based clean-up provided by SPM8. For the real data (Datasets II and III), same evaluations were conducted except for the local TCM due to limited prior data. As two examples, Figs 1 and 2 exhibit the 3D renderings of the segmentation results from different methods and for different tissue types. Fig 1 is the results of Subject 3 in Dataset I, and Fig 2 is of Head 3 in Dataset II. The tissue segmentations appear similar across different methods in Fig 1. However, the proposed algorithm improves on smoothness (as indicated by those red circles), especially for CSF and skull. The local TCM is able to remove isolated voxels (indicated by the red squares) due to its detailed location-specific neighborhood constraints. The difference among methods are more obvious in the real MRI (Fig 2), where the proposed algorithm significantly improves smoothness. This head is a good example as the original MRI contains some artifacts in the air outside of the head, leading to large clusters of disconnected voxels in the CSF and skull when using eUS. This segmentation error is removed by the global TCM in CSF, because CSF and air cannot be co-located, as dictated by the 0 probability of CSF–air in the global TCM (Eq 6). However, the probabilities of skull–air and scalp–air are not zero in Eq 6, leading to enhancement of this error in the skull and scalp. In fact, the skull can be adjacent with the inner air (air cavities) but not the air outside of the head, and the scalp only touches outside air at the scalp surface. Fortunately, the regional TCM encodes this local property of neighborhood information and removes these errors. Another important improvement is the continuity of CSF, which is crucial in applications involving current flow [41]. Fig 3 shows that the proposed algorithm fixes the discontinuity problem in the CSF generated by eUS, which cannot be corrected by the MRF clean-up. As a matter of fact, the global TCM (Eq 6) guarantees that the CSF is continuous after segmentation, as indicated by the 0 probability of GM–skull.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The 3D renderings of the results using different methods on Subject 3 in Dataset I.

From the first row to the last row: GM, WM, CSF, skull and scalp. The first column is the results by eUS, the second column is after the MRF-based clean-up, and the third to the last columns are from the proposed algorithm using different TCMs (global, local, regional). Red circles/squares highlight the areas where improvements can be observed.

https://doi.org/10.1371/journal.pone.0125477.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The 3D renderings of the results using different methods on Head 3 in Dataset II.

From the first row to the last row: GM, WM, CSF, skull and scalp. The first column is the results by eUS, the second column is after the MRF-based clean-up, and the third and fourth column corresponds to the proposed algorithm using different TCMs (global, regional).

https://doi.org/10.1371/journal.pone.0125477.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Axial slices of the CSF generated by different methods.

The first and second row corresponds to Subject 3 in Dataset I and Head 3 in Dataset II, respectively. Red circles indicate the discontinuities from eUS, which cannot be removed by applying the MRF-based clean-up but can be fixed by the TCM (global, local, or regional).

https://doi.org/10.1371/journal.pone.0125477.g003

The quantitative assessment of the results is shown in Fig 4. The fuzzy Dice coefficient fD, the total squared Gaussian curvature sK_G, and the porosity ϕ averaged across subjects for different tissues and methods are plotted in the first, second and third row, respectively. The three columns correspond to Dataset I, II and III, respectively. The red error bars indicate the standard deviations of the results across different subjects. Note that since the first subject in Datasets I and II was used for parameter learning (see Evaluation), it is thus excluded in the first two columns in Fig 4. The accuracy of each method seems almost the same in the figure (panels (a), (b), (c)), while the smoothness of the segmentation appears improved by the proposed algorithm (panels (d), (e), (f)). A two-way (tissue, method) repeated measures analysis of variance (ANOVA) was conducted on the accuracy and smoothness of Dataset I. Results show that different methods do not significantly change the accuracy (F(4, 72) = 0.55, p = 0.70) but do have significant effect on the smoothness (F(4, 72) = 1484.83, p = 0). Furthermore, a pairwise t-test was conducted between different methods on the tissue-averaged values of sK_G. Results show that global, local and regional TCM all significantly improve the smoothness of the segmentation results (p < 0.001, see inset of panel (d)). It is obvious that the proposed algorithm gives lower porosity for CSF and skull as compared to the methods in SPM8 (panels (g), (h), (i)). For the porosity results in Dataset I, a one-way (method) repeated measures ANOVA shows that there is significant difference (CSF: F(4, 72) = 494.57, p = 0, skull: F(4, 72) = 380.93, p = 0). The pairwise t-test indicates that our algorithm (global, local and regional) significantly reduces the porosity of the CSF and the skull (p < 0.001), compared to those obtained from SPM8 (eUS and eUS with MRF clean-up). See panel (g). For Datasets II and III, no statistical testing was conducted due to limited number of subjects.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The accuracy, smoothness and porosity of the segmentation results averaged across subjects for Dataset I (Subject 2–20, left column), Dataset II (Head 2 and 3, middle column), and Dataset III (Head 1–5, right column).

The red error bars indicate standard deviations across subjects. The inset in (d) shows tissue-averaged quantities. All the colors refer to the legend in (i). (***: significant with p < 0.001)

https://doi.org/10.1371/journal.pone.0125477.g004

We also note that the accuracy for Dataset III is generally lower than that of Dataset I or II. It is mainly because the quality of the truth data provided by Dataset III is lower compared to the ground truth we manually improved in Datasets I and II. For example, Fig 5 shows the truth images of Head 3 in Dataset II and Head 1 in Dataset III in the first two rows, respectively. It is obvious that the latter has lower quality, as shown in the noisy GM slice, discontinuous CSF surface, and incomplete skull volume. Qualitatively speaking, the results from the proposed algorithm using global TCM on this head are more accurate and realistic than its truth data, as shown in the third row of Fig 5. However, we admit that although the ground truth in Dataset II is obtained by manual improvement, it is by no means a gold standard. In fact, it has the same origin as the automated results, i.e., initially generated by the eUS in SPM8. Therefore, it is in some sense biased. Nevertheless, the main argument here is that based on the same truth, the accuracy is not significantly altered by different methods in Dataset I, and meanwhile the proposed algorithm can achieve better smoothness and lower porosity of the segmentation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. The truth data of Head 3 in Dataset II (first row) and Head 1 in Dataset III (second row).

As a comparison, the third row shows the results from the proposed algorithm using global TCM on Head 1 in Dataset III. The columns correspond to axial slices of GM, 3D rederings of the CSF and skull, respectively.

https://doi.org/10.1371/journal.pone.0125477.g005

Discussion

Automated MRI segmentation combining a probabilistic model for image intensity with an atlas and MRF regularization was first proposed in the late 1990s [24, 26], with implementations in many neuroimaging software tools, such as EMS [24], ABC and EMSegmenter [25–27] in 3D Slicer, FAST [18], Freesurfer volumetric approach [22, 23], the SVPASEG [34], and Atropos [60]. Unfortunately these tools only focus on brain tissues. Moreover, Freesurfer volumetric approach only present a heuristic derivation for its update equations, and does not include any EM iteration (the Gaussian parameters are obtained from prior data and never updated). This limits its use to imaging modalities (and scanning parameters) for which calibration data is available. In contrast, the EM algorithm used here estimates the intensity parameters and can thus be applied to arbitrary image modalities (T1, T2, FLAIR, CT, etc.) without concern for historical data or even calibration and image intensity artifacts. The algorithm implemented in the Atropos software is conceptually similar to the present work. It is an open-source segmentation tool that combines user-defined atlas, MRF prior and image model into the EM iterations. In the present work we have focused in addition on smoothness and porosity of the resulting segmentations, which is crucially important for current-flow modeling or lead-field calculations.

This paper summarizes the framework of probabilistic automated MRI segmentation developed over the last two decades and presents it with a systematic mathematical derivation (see Appendix). When applied to the FM model with an atlas as prior one obtains the update equations used in SPM8 (“New Segment” function). When including in addition an MRF as part of the EM update we obtain the algorithm of [24]. To our knowledge the present paper provides the first systematic and complete derivations of these update equations.

The present paper proposes an extension to the conventional energy function of the MRF, namely, an extended Potts model that is consistent with the maximum entropy prior that specifies mean and covariance on historical data. The method is implemented using an updating scheme that guarantees convergence, and is integrated into SPM8 where the bias field correction and image registration are handled by the “New Segment” function. Both simulated and real MRI data are used to evaluate the algorithm. The proposed method is shown to successfully improve the smoothness and porosity without compromising accuracy. More importantly, since the implementation can segment out 6 tissues including CSF, skull, scalp and air in an FOV covering the whole head, it can be used for forward modeling for TES/TMS/EEG/MEG.

Smooth segmentations and a CSF without any discontinuity are crucial for current-flow modeling, in particular modeling of TES [41]. Previously, we achieved this by using morphological and Boolean operations as a post-processing step to the segmentation results [41]. Such heuristic algorithms have a number of parameters that have to be selected judiciously. In this work, we prevent ad hoc procedures with arbitrary parameters by incorporating spatial priors in the TCM (see Eq 6), which regularizes the segmentation at each iteration. Importantly, by combining morphological priors into the segmentation algorithm, spatial considerations can be taken into account while at the same time respecting the intensity information. Post-processing operations in contrast typically smooth the segmentation results while entirely ignoring the original image(s). The proposed algorithm can generate tissue segmentations with significantly lower porosity compared to those from SPM8. Therefore, the results from this algorithm can be used for TES modeling directly without any need of automated clean-up or manual correction.

We would like to point out some differences in the use of the mixing proportion variable γ in this work and that of [14]. By enforcing normalization of the mixing proportions ${\gamma }_{x\bar{x}}$ for each tissue type $\bar{x}$, we are biasing the segmentation to match the relative volume fractions of the TPM. This bias may introduce errors. In stroke or geriatric populations, for instance, subjects present typically with substantially larger CSF fraction. Enforcing tissue volume fraction of the normal TPM may not be desired in these populations. In such a case, the approach as in Eq. 27 in [14] may be preferable to our Eq 3c. [14] introduces an additional free parameter (also denoted as γ in [14], but with different meaning) to globally rescale and re-normalize the atlas, thus accounting for variation of tissue volumes fractions in different individuals. Interestingly, SPM8 implements Eq 3c and only SPM12 implements as originally proposed in [14]. In order to stay within a strict probabilistic model yet account for different volume fractions, we propose instead to construct an atlas that matches the target population.

Except for SPM8, we did not compare our algorithm with the other software tools mentioned above, simply because they cannot generate segmentations for non-brain tissues in an FOV covering the whole head. For this comparison, we would need to extend the FOV and tissue types for all the candidate tools, which is beyond the scope of this work. Similarly, except BrainWeb, we did not use other public datasets that are commonly adopted for evaluations in the neuroimaging community (e.g., IBSR, LPBA40 [61], segmentation validation engine [62]), because these datasets do not provide ground truth for non-brain tissues. We did however provide results for 8 subjects with non-brain ground truth (3 from our lab and 5 from the Neurodevelopmental MRI Database of John Richards).

Since the TPM provides the anatomical information, and the TCM gives the neighborhood constraints, the accuracy of this algorithm is strongly dependent on the quality of the two maps. For reliable estimates, the size of the dataset used to generate these maps may be important, in particular for the TCM. In this work, only 19 subjects are available, which is small compared to the 36 × 6 parameters in C_ij that are to be estimated for each location (6 stands for 6 face-connected neighbors). Additionally, the exact calculation of the MRF parameters J_ij, h_i from the observed correlations C_ij and means m_i is an open problem. The analytic solution for the trivial 2-site lattice which we used here likely overestimates the strength of interaction (as it ignores the indirect influence of distant neighbors and loops) and thus we had to adjust it by tuning parameter β. The correct solutions should be obtained from solving Eq 6, which is known as the “inverse Ising problem” in statistical physics. Exact solutions are known for dependencies with an acyclic graph structure (e.g., tree structure, [63]), and approximate analytical solution for network with loops (e.g., regular lattice, [64]). However, these solutions are only available for 2-state variables of the Ising model, where J_ij and h_i are only scalars. In this work, we are dealing with a K-state variable x_i, for which theoretical solutions are not yet available. Thus we have taken a pragmatic approach and have made the parameter J independent of locations ij and tuned it so that we obtain the best possible segmentations on known data. Finally note that the VEM algorithm we used here to compute approximate posterior probabilities may find improvements with newer techniques for computing probabilities in networks with loops such as “loopy belief propagation” and “tree-reweighted message passing” [65].

Appendix

The probabilistic formalism for image segmentation has a long history starting in the 1980s, if not earlier, and encompasses efforts to include spatial priors through an atlas as well as morphological priors using MRF. Parameters of the models are adjusted to the images using the EM algorithms and variations thereof. The earliest work we found on combining morphological and spatial priors in this formalism are [24, 26]. The goal of this appendix is to summarize this body of work with a consistent notation and provide a complete derivation of the EM update equations, which we have not been able to find elsewhere in the literature. This appendix starts with a review of work in this field, introduces notation and formalism, derives the EM update equations, and discusses a few novel choices of our implementation.

A Review of probabilistic model for image segmentation

Probabilistic inference techniques have been developed for MRI brain segmentation over the years (e.g., [14, 18, 24, 25, 34, 49, 60]). The key idea is to establish a probabilistic model of the image intensity distribution using a finite mixture (FM) model [45]. Automated segmentation is then a process of fitting the FM model to maximize the likelihood of observing the MR image(s). Since the true tissue types of voxels are unknown (“missing data”), this maximum likelihood (ML) estimation is usually implemented by the expectation–maximization (EM) algorithm [48]. Early work dates back to [49] and was reviewed in [44]. In [25] the EM algorithm is also used to correct intensity inhomogeneities and segment the MRI iteratively, which was later implemented as the EMSegmenter. However, such FM models only use intensity and ignore available spatial contextual information, as pointed out by [18]. Two approaches have been independently pursued to incorporate prior knowledges into the ML estimation process.

The first type of prior model is a brain atlas, which quantifies for each voxel location the prior likelihood of observing a given tissue type. It is built by registering many MRI volumes into a common standard space, assigning a specific tissue type to each voxel and then averaging across these volumes [66, 67]. Such an atlas is also known as tissue probability map (TPM) as it quantifies the anatomical prior probability irrespective of the observed intensity in a given image. Since the atlas is in a standard space, image registration is required in order to map the atlas onto the image that is to be segmented. Yet, mapping the image requires an estimate of what tissue is contained in each voxel, i.e. an estimated segmentation. An integrative approach to resolve this interdependence is the Unified Segmentation algorithm [14], which is now publicly available in the Statistical Parametric Mapping 8 (SPM8) software [13]. It combines registration, segmentation as well as bias-field correction into a single optimization problem (bias-field correction compensates for intensity inhomogeneities often observed in MRI). A similar approach is taken in [68] and [27] where the EM algorithm is used to optimize segmentation and registration parameters simultaneously, and registration is achieved in a hierarchical manner such that detailed anatomical structures can be recursively differentiated. An implementation of this method is made into the EMSegmenter in 3D Slicer [28].

The second type of prior knowledge that has been leveraged for segmentation is neighborhood relationships (e.g., not all tissue types can be adjacent to each other). These relationships have been quantified using a Markov random field (MRF), which assigns an “energy” to combinations of neighboring tissue types [42, 43]. Such an energy effectively provides (prior) probabilities making some combinations highly likely (low energy) and others very unlikely (high energy), irrespective of image intensity. MRF has been widely used to regularize the segmentation to ensure spatial consistency [18, 24, 26, 34, 60, 69–72]. Unfortunately, computing posterior probabilities exactly becomes intractable once the spatial dependencies are introduced into the prior probabilities with an MRF. Researchers have proposed a variety of solutions to this problem. Monte Carlo simulation by drawing samples from the MRF [73, 74] is a classical numerical technique to approximate the posterior probability of the voxel label, but it is not computationally efficient. Another classical approach, based on the mean-field theory, is to approximate the influence of the neighbors onto a voxel using their mean influence. The posterior distribution of voxel labels can then be approximated by the product of voxelwise marginals [75]. This has been successfully applied to MRI segmentation [24, 70, 74]. A variant of mean-field approximation is proposed by [18], where a voxel is affected by the most probable label of each of its neighbors (the pseudo-likelihood maximization as in [47]). This has been used in many segmentation tools incorporating the MRF, such as the FMRIB Automated Segmentation Tool (FAST) in FSL [17], the CIVET segmentation [33], and Atropos [60]. Recently, [46] pointed out that both [75] and [18] adopt a synchronous updating scheme which is not guaranteed to converge. If instead voxels are updated sequentially (asynchronous update), convergence of the approximated EM in [75] and [18] is guaranteed. Finally, we note that the interaction energy used in most MRF are based on the Potts model [47] which cannot express constraints for all possible combinations of neighboring tissue types.

These two types of priors have also been combined into a single inference framework and implemented in a number of software tools (see Discussion in the main text). However, there is no publication that summarizes the complete formalism with a consistent notation. This is the goal of the next few sections.

B Derivation of the variational EM formalism

We start this section by introducing notation, and then we provide an outline of the estimation process. We will review the ML optimization scheme which is applicable when only an atlas is used as prior. To incorporate neighborhood priors we will need an extension of ML known as variational EM (VEM). Then we present the probabilistic model for image intensity and prior probabilities. With these we can then derive the entire estimation algorithm.

C 2-site problem

Consider a trivial 2-site lattice. x₁ and x₂ are the class vectors for the two sites. According to the prior model Eq 27, we can write the distribution of the lattice configuration as (42) To get the expression of J_ij and h_i in terms of C_ij and m_i, we need to solve (43a) (43b) For this 2-site lattice, the sum $\sum _{\mathbf{\text{X}}\in 𝓧}$ can be expanded and thus the partition function Z can be analytically expressed. When i = 1, j = 2, plugging the P(X) in Eq 42 into Eq. 43, we have (44a) (44b) Note the denominator in Eq. 44 is the expanded partition function Z. These two equations can be simplified by using matrix subscripts k and l (1 ≤ k ≤ K, 1 ≤ l ≤ K), (45a) (45b) By observation, we can immediately get (46) and similarly, (47) In order to satisfy Eq 45a, we can guess (48a) (48b) (48c) (48d) Eqs 48c and 48d use the relations in Eqs 46 and 47, and also the fact that (C₁₂)_kl = (C₂₁)_lk. Plugging Eq. 48 into the left hand side of Eq 45a, combining the sum of logarithms into logarithm of the products, and noticing that the terms (m₁)_k, (m₂)_l are cancelled out, we finally get (49) which is equal to the right hand side of Eq 45a, because $\sum _k\sum _l{(C_{12})}_{kl}=1$ by definition. Therefore, Eq. 48 satisfies Eq 45a, and thus the solution to the 2-site problem is (50a) (50b) These give Eqs 30a and 30b.

D Derivation of the free energy function

Noticing the fact that the probability distribution Q(X) sums to 1, i.e., $\sum _{\mathbf{\text{X}}\in 𝓧}Q(\mathbf{\text{X}})=1$, we can write the free energy function F(Q, θ) in Eq 18 as (51) Expanding the complete data likelihood P(X, Y|θ) into the product of the intensity distribution P(Y|X, θ) and the prior P(X), and plugging Eq 15, Eq 20 and Eq 31 into Eq 51, we get (52) The above equation can be simplified, for example, (53) (54) The fact that $\sum _{{\mathbf{\text{x}}}_m}q({\mathbf{\text{x}}}_m)=1$ is used from Eq 53 to Eq 54. Similarly, we have (55a) (55b) (55c) Plugging Eqs 54 and 55 into Eq 52, we get Eq 32.

E Derivation of the updating equation in the E-step

With the objective function F(Q, θ) as in Eq 32 and the constraint $\sum _{{\mathbf{\text{x}}}_i}q({\mathbf{\text{x}}}_i)=1$, we introduce a generalized function using the Lagrange multipliers λ_i (56) Then we need to minimize 𝓕 with respect to q(x_i). To avoid notation confusion, we take derivative of 𝓕 with respect to q(z_n) (57) Note that q(x_j) is treated as a constant when the above derivative is taken. Simplifying Eq 57 and setting it to 0, we can solve q(z_n) as (58) Plugging the above solution into $\sum _{{\mathbf{\text{z}}}_n}q({\mathbf{\text{z}}}_n)=1$ to enforce normalization, and substituting z_n with x_i, we can get the Eq 34.

F Derivation of the updating equation in the M-step

The M-step is to optimize the free energy function F(Q, θ) with respect to the parameters θ subject to the constraint $\sum _x{\gamma }_{x\bar{x}}=1$. For generality, we need to use the one involving tissue mixtures (Eq 37). To enforce the constraint we use again the Lagrange multipliers ${\lambda }_{\bar{x}}$ (59) For generality, we consider the case of multimodal data, and thus we need to minimize 𝓕 with respect to μ_x, Σ_x and ${\gamma }_{x\bar{x}}$. To avoid confusion, we take derivative with respect to μ_z (60)

Setting this to 0, solving for μ_z, replacing z with x and using the fact that $q(x_i=x)=\sum _{\bar{x}}q(x_i=x,{\bar{x}}_i=\bar{x})$, we get Eq 41a. Similarly, we can obtain the Σ_x as in Eq 41b.

To get ${\gamma }_{x\bar{x}}$, we take derivative of Eq 59 with respect to ${\gamma }_{z\bar{z}}$ (61) Set this to 0, solve for ${\gamma }_{z\bar{z}}$ and replace z with x. Using the normalization condition $\sum _x{\gamma }_{x\bar{x}}=1$, we find ${\lambda }_{\bar{x}}=\sum _i\sum _{x}q(x_i=x,{\bar{x}}_i=\bar{x})$. Inserting this into ${\gamma }_{x\bar{x}}$ and using Eq 39, one obtains Eq 41c.

Acknowledgments

The authors would like to thank Andrea Cavagna, Irene Giardina, and Flaviano Morone for valuable discussions on statistical physics. We also thank Clay Spence for discussion on model construction and Christopher Alvino for his help with the evaluation of smoothness. We thank John Richards for providing segmentation from his Neurodevelopmental MRI Database, and also Marom Bikson for providing the de-identified MRI scans. Finally we thank John Ashburner for pointing us to the work of [46] and highlighting the importance of accommodating differing volume fractions. This work was supported by grants from the National Institute of Health (NS076123 and MH-092926).