Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A fast two-stage active contour model for intensity inhomogeneous image segmentation

  • Yangyang Song ,

    Roles Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    yangys@mail.nwpu.edu.cn

    Affiliation Department of Applied Mathematics, Northwestern Polytechnical University, Xi’an, Shaanxi 710072, PR China

  • Guohua Peng

    Roles Software

    Affiliation Department of Applied Mathematics, Northwestern Polytechnical University, Xi’an, Shaanxi 710072, PR China

A fast two-stage active contour model for intensity inhomogeneous image segmentation

  • Yangyang Song, 
  • Guohua Peng
PLOS
x

Abstract

This paper presents a fast two-stage image segmentation method for intensity inhomogeneous image using an energy function based on a local region-based active contour model with exponential family. In the first stage, we preliminary segment the down-sampled images by the local correntropy-based K-means clustering model with exponential family, which can fast obtain a coarse result with low computational complexity. Subsequently, by taking the up-sampled contour of the first stage as initialization, we precisely segment the original images by the improved local correntropy-based K-means clustering model with exponential family in the second stage. This stage can achieve accurate result rapidly as the result of the proper initialization. Meanwhile, we converge the energy function of two-stage by the Riemannian steepest descent method. Comparing with other statistical numerically methods, which are used to solve the partial differential equations(PDEs), this method can obtain the global minima with less iterations. Moreover, to promote regularity of energy function, we use a popular regular method which is an inner product and applies spatial smoothing to the gradient flow. Extensive experiments on synthetic and real images demonstrate that the proposed method is more efficient than the other state-of-art methods on intensity inhomogeneous images.

Introduction

Image segmentation is still a popular problem in the field of image processing and computer vision [1]. The intensity inhomogeneity exits in images, which is caused by the imperfections of image acquisition, influence of the illumination and other factors of environment, is one of the main issues in subject of image segmentation. The existences of intensity inhomogeneity in images may lead to misjudgments of doctors and researchers. Therefore, the subject for segmenting images with intensity inhomogeneity attracts more and more researchers to study. Recently, active contour models [2] have been a successful branch for image segmentation.

The existing active contour models can be divided into two major types: edge-based models [37] and region-based models [1, 2, 815]. Edge-based models utilize the image gradient information to drive the evolution of active contours. Therefore, these models can hardly obtain right segmentation results for images with noisy or weak edge. Alternatively, region-based models use the statistical and curvature information inside and outside of the contour to guide the contour evolution. Hence, the region-based models have better performance than the edge-based models and are able to deal with noisy and blurred images.

One of the most popular region-based models is the Chan-Vese model [2], which computes the intensity averages by using constant intensity information across the region. Thus, this model is a global region-based model and assumes images are homogeneous. For images with intensity inhomogeneity, many local region-based methods [1015] have been proposed. These models are able to handle image intensity in local region and can be used to segment the images with intensity inhomogeneity. However, most of these region-based models are solved by the standard gradient method [5, 7], which needs a large number of iterations to converge and easily falls into local minima. The convergence problem for active contour models is inherently to solve of the Euler-Lagrange differential equations. Therefore, various methods have been proposed recently to solve the Euler-Lagrange differential equation.

These methods can be mainly categorized into three groups: the first group focuses on investigating more sophisticated differential equations, the second group immerses in searching other alternating direction method, while the third group concentrates on describing Euler’s method as a gradient descent method and applying more advanced optimization techniques. The development of the first group comes from the work of Scheuermann et al. [16] and Badshah et al. [17]. Scheuermann et al. [16] studied the RK-2 and RK-3 order Runge-Kutta methods and the explicit Euler method. Badshah et al. [17] proposed two related multigrid algorithm. The second group derives from Xie et al. [18] and Song et al. [19]. Xie et al. [18] and Song et al. [19] considered Bregman method and Split Bregman method for image segmentation. These methods offered another line of fast algorithms and can get global minima for some active contour models. But the Bregman method and Split Bregman method are used to solve L0 minimization directly and do not work for different types active contour models.

As far as the optimization-based approach, Chartrand et al. [20] and Mendi et al. [21] proposed the quasi-Newton method for minimizing the active contour energy function within Chan-Vese model. Following these two works, Bar et al. [22] modified the quasi-Newton method by adding an analytic functional Hessian and presented a generalized Newton method. Alternatively, Sundaramoorthi et al. [23] creatively proposed the Sobelev gradient descent method by using Sobolev metrics. The Sobelev metrics outperform the traditional metrics for the same segmentation energy function.

Motivated by the work of Sundaramoorthi et al. [23], Pereyra et al. [24] first considered the Riemannian steepest descent method on model’s intrinsic manifold for the Chan-Vese model, which converged extremely fast. However, this method can only be applied to images with Gaussian distribution. Considering that the images have other specific distributions, such as Gamma [25], Rayleigh [26], Laplace [10] and Weibull [27]. Pereyra et al. [28] derived a general Riemannian optimization method for active contour model with the exponential family [29]. More precisely, Pereyra et al. presented a smooth descent algorithm [30] for nonparametric active contour models. Above all, Pereyra et al. have given appropriate mathematical development for natural metric of the statistical manifold, which makes the computation of natural matric possible. But in the paper [28], Pereyra et al. only demonstrated the feasibility of the natural metric for Chan-Vese model, which still cannot achieve satisfactory segmentation results for images with intensity inhomogeneity.

To achieve the object of fast segmentation for images with intensity inhomogeneity, we combine the Riemannian steepest descent method with local correntropy-based K-means clustering (LCK) model [13], which could obtain ideal segmentation results for intensity inhomogeneous images. In addition, in order to further improve the speed of operation and weaken the dependence of LCK model on the initial position, we divide our segmentation algorithm into two stages. The first stage can roughly but fast obtain the coarse contour near the object boundaries with low computational complexity in coarse space. The second stage can easily get the accurate segmentation results with suitable initialization in original space. Meanwhile, the two minimization problems of the energy function with exponential family [29] are iterated by natural gradient method, which is faster than other methods for solving active contour models to find the global minima.

The remainder of this paper is structured as follows. In the next section we introduce the classic region-based active contour modes and the frameworks of region-based active contour models with exponential family. In Section 3 we derive a fast two-stage segmentation model. More precisely, we develop the mathematical implement for local region-based active contour models. In Section 4 we list the experimental results illustrating the performance of the proposed method. Finally, in Section 5 we describe the conclusion and future work.

Background and theoretical foundations

Traditional region-based active contour models

Chan-Vese model.

Chan and Vese [2] proposed a global region-based active contour model for image segmentation. This model approximates the image intensities outside and inside of contour by the average image intensities outside and inside of contour, respectively. For the input image I: Ω → ℜ2, we divide the image domain Ω into two regions: an object region Ωinside and a background region Ωoutside = Ω/Ωinside by a closed contour C(s): [0, 1] → ℜ2, which corresponding to zero level set function: C = {x: ϕ(x) = 0}. The energy function of Chan-Vese model can be written as: (1) Where μ ≥ 0, ν ≥ 0, λ1 ≥ 0 and λ2 ≥ 0 are fixed constants. c1 and c2 are the average image intensities outside and inside contour C, respectively. Hϵ(ϕ) is the smooth approximate of the Heaviside function (2) and the derivative of Hϵ is defined as (3) Chan and Vese minimized the energy function (1) by the standard gradient descent method [31]. The gradient descent flow corresponding to ϕ can be written as (4)

As a global active contour model, the curve convolution of the CV model is only related to global characteristic of the image region. Therefore, the CV model can achieve the satisfactory results for images with intensity homogeneity, whereas it can not deal with the images with intensity inhomogeneity.

Local binary fitted model.

Li et al. [11] proposed the local binary fitting (LBF) model by considering a kernel function into a local region-based model. The LBF model can deal with intensity inhomogeneity images. The energy function of LBF model is defined as (5) where λ1, λ2 ≥ 0, ν ≥ 0 and μ ≥ 0 are fixed constants. is Gaussian kernel function with kernel width σ, which is introduced to control the spatial distance between the x-th and the y-th; m1(x) and m2(x) are the local clusters of the x-th pixel. Meanwhile, in order to regularize the level set function ϕ, the third and the forth terms are added as level set regularization term.

The level set function and the local clusters can be updated from Eq (5) by the standard gradient descent method, which can be computed as (6) Where e1 and e2 are

The LBF model considers the Gaussian kernel function to capture the local region information of images and can segment the images with intensity inhomogeneity. However, the Gaussian kernel function is not sufficient to introduce the image information and the LBF model cannot segment images with severe intensity inhomogeneity. Moreover, this model is very sensitive to the initialization and the iterative approach of it easily falls into local minima.

Local correntropy-based K-means clustering model.

Wang et al. [13] presented a more accurate segmentation method (LCK) for images with unknown complex noise. The main difference between the LCK model and LBF model is that the LCK model utilizes the pixel-to-cluster distance, which makes this model is more robust to unknown complex noise. The objective function of the LCK model is (7) The weight of the y-th is calculated by (9) Minimizing Eq (7) by using the standard gradient method, the corresponding local clusters and level set formulation is obtained by (8)

The LCK model is robust to images with complex noise and intensity inhomogeneity, whereas it is also a little bit sensitive to the initialization. Meanwhile, the iterative method of the LCK model also easily falls into minima and needs much more times to obtain the final segmentation results.

Region-based model with exponential family observation

Due to the fact that the energy functions of region-based active contour model does not have a structure of a vector space, Lecellier et al. [29] proposed the exponential family observation framework for region-based model. This framework is specifically calculated when coping with global region-based information such as statistical image features (histogram, variance and mean).

Exponential family.

In this section, we provide the necessary mathematical concepts on the exponential family, which covers most noise models of the image acquisition system. For the given point x ∈ ℜd, the image values can be distributed by (9) Where the m1(x) and m2(x) are the clusters of the foreground and background respectively and the function f: ℜp → ℜ+ is the probability density function with exponential family distribution.

Definition 0.1 The family of distributions of a random variable, is called a k-parameter canonical exponential family. If there exists canonical parameter vector η = (η1, …, ηk)T and log-normalizer A(η), and real-valued functions h, T1, …, Tk: ℜk → ℜ, the probability density function f(⋅|m) may be written as (10) here T = (T1, …, Tk)T is the sufficient statistic.

Note the Definition 0.1 gives the general distribution of exponential family that most of them have distribution for signal and image processing. Table 1 shows some common canonical distributions of the exponential family, such as the Gamma, Beta, Poisson, Gaussian, Exponential, and Rayleigh.

Region-based active contour model with exponential family observation.

Following the definition of exponential family, the region-based active contour model (this paper takes LCK model as an example)(Eq (7)) with exponential family observation can be rewritten as (11)

The functional optimization problem Eq (12) can be solved by alternative minimization method. In this paper, we take the Gaussian distribution as an example and give the iteration with respect to ϕ, m1(x) and m2(x) as following: (12)

Although we give more general distribution for LCK model, it still can not overcome its inherent limitations. The segmentation method of the above LCK models is not convex and the weights of the pixels wx are always decreased faster than other region pixels during the iterated process. Hence, the main difficulty for the region-based active contour model is still the solution of energy function.

Proposed model

Inspired by the work of Wang et al. [13] and Pereyra et al. [28], we proposed a novel two-stage segmentation method, which can fast segment the intensity inhomogeneous images with accuracy results. In the real-world, the accuracy segmentation results for intensity inhomogeneity images are difficult to obtain and also need to spend a large amount of time, because these intensity inhomogeneity images are always big and complex. Due to the above factor, we split the process of the segmentation into two stages which makes the computation with lower computational complexity. In the first stage, we implement the segmentation model in coarse spaces. Following it, in the accurate segmentation stage, we utilize the up-sampled coarse results as initializations and segment the images in fine space. Finally, to get more accurate segmentation results faster, we present the Riemannian steepest descent method for the two-stage segmentation model.

Coarse segmentation

The LCK model is robust to images with severe intensity inhomogeneity and complex noise. However, as a local region-based active contour, this model needs a perfect initialization to obtain the satisfactory segmentation results. In the first stage, we propose a novel LCK model with exponential family observation in coarse space (RDLCK), which can obtain the segmentation results fast. The energy functional of this model can be defined as (13) Where Ωτ denotes the uniform sub-space of Ω with the down-sampled factor τ, and function f is exponential family function. wτy is computed by and g(x) is Gaussian kernel function g(x) = exp(−x2/2σ2). mτ1(x) and mτ2(x) are the local clusters in the coarse space, which can be calculated by gradient descent method. For f is Gaussian distribution, then mτ1(x) and mτ2(x) can be computed by (14)

Most traditional region-based active contour models always converged by using the standard gradient descent method, whereas it easily falls into minima and needs a large number of iterations. Therefore, fixing mτ1(x) and mτ2(x), we utilize the Riemannian steepest descent method for RDLCK model. The update for level set function ϕτ is (15) Here γτt is a positive parameter, is the standard gradient flow, which is similar to Eq (12) (16) and is the Riemannian steepest descent of ERDLCK on Euclidean tangent space [30] and is (17) Therefore, the key to Eq (16) is that the is a positive definite matrix. (refer to the appendix for a detailed computation for ).

This paper makes the computation of the possible. Moreover, the fast image segmentation method based on natural gradient Eq (15) is applied to local region-based active contour model successfully.

Lastly, to promote the solutions of Eq (13) smooth, we need to regularise ϕ. The traditional region-based active contour model is regularised by adding the penalty term , which regularises the energy function by selecting contours with minimal length. In this paper, we promote regularity by using an increasingly popular alternative, which applies spatial smoothing to the natural gradient flow. This method can be written as (18) Where and the low values of σ are applied for images with low noise, which preserves details of images.

We can obtain the coarse contour ϕτ in the down-sampled space Ωτ, but the coarse contour ϕτ is not suitable for the original image due to down-sampling process. Therefore, we need to up-sample the coarse contour ϕτ and get the up-sampling version ϕ* of the coarse contour ϕτ. As shown in Fig 1, we give the segmentation process of a heart CT image (the size of the image is 152 × 128).

thumbnail
Fig 1. The process of obtaining the initialization for accurate segmentation.

https://doi.org/10.1371/journal.pone.0214851.g001

Accurate segmentation

The coarse segmentation can obtain the coarse contour with low cost of computation, whereas the coarse contour is not accurate due to the loss of region information from down-sampling. Therefore, we further segment the objects by improved LCK model with satisfactory initialization ϕ*. The improved LCK model with exponential family in original space (ROLCK) is defined as (19) where ξ is a small positive parameter and ϕ* is the up-sampling vision of the coarse contour. The last term promise that the distance between ϕ* and accurate result is small.

Using the Riemannian steepest descent method for converging ROLCK model, the iterative scheme for minimizing the Eq (19) is similar with the one for solving Eq (13). (20) Where γt is positive constant and wx is the final weight of the x-th pixel, which is calculated by wx = ∫Ω H(ϕ(y))g(‖I(x) − m1(y)‖2) + (1 − H(ϕ(x)))g(‖I(x) − m2(y)‖2), the computation of is also similar to the computation of (21) The only difference is the iteration of , it is computed by (22) At the original space, there is no region information loss. Since the initialization for Eq (19) is very close to the object, the convergence of Eq (19) is fast and the accurate segmentation stage can obtain the final result after a few iterations.

Segmentation procedures

The procedures of the proposed method can be summarized in Algorithm. 1. The initial level function of the coarse segmentation is defined as (23) We provide a positive parameter for ρ in this paper.

The proposed method can be summarized in Algorithm 1.

Experimental results and comparison

In order to prove and compare the performance with the CV model [2], the GCV model [10], the LBF model [11], the LIC model [12], the LCK model [13], the RCV model [28] and the LGFI model [14], we apply our approach to several images with intensity inhomogeneity in this section. All the experiments are implemented by using MATLAB R2015a and running on a person computer with Intel core i5, 2.5GHz, and 4.00 GB RAM.

Algorithm 1 Two-stage Segmentation

Stage 1: Coarse Segmentation on Ωτ

1: Input: Ωτ

2: Initialization: by Eq (23)

3: for 1 to MaxIter do

4:  Compute

5:  Clusters mτ1(x) and mτ2(x) by Eq (14)

6:   by Eq (16)

7:   by Eq (17)

8:  Updating the level set function ϕτ by Eq (15)

9: end for

10: Output: Coarse segmentation result ,

Stage 2: Accurate Segmentation on Ω

11: Let ϕ* be the up-sampling version of ,

12: for 1 to MaxIter do

13:  Compute

14:  Clusters m1(x) and m2(x) by Eq (20)

15:  ∇ϕ EROLCK(I(x); ϕt) by Eq (22)

16:   by Eq (21)

17:  Updating the level set function ϕ by Eq (20)

18: end for

Comparisons with state-of-art methods for intensity inhomogeneity images

Comparisons with global active contour models (the CV [2] model, GCV [10] model and RCV [28] model.

In this section, we compare our method with some global active contour models. The global active contour models assume that the intensities of images are piecewise constant and thereby these models can not segment the images with intensity inhomogeneity.

Fig 2 shows the segmentation results on two synthetic images (The first row is a T-shape image and the fourth row is a synthetic image.) and two medical images (The second row is a blood image and the third row is a CT heart image.) with intensity inhomogeneity by using the CV model [2], the GCV model [10], the RCV model [28] and the proposed method, respectively. The first column shows the segmentation results with CV model. The second column shows the segmentation results with GCV model. The third column shows the segmentation results with RCV model. Finally the last column shows the segmentation results with our method. It can be seen that the results of the CV model [2], the GCV model [10] and the RCV model [28] are unsatisfactory because these models are global active contour models and use the global intensity means of images to fit them. On the other hand, our method can achieve satisfying results due to taking the local image region information into account, which can deal with the intensity inhomogeneous images well.

thumbnail
Fig 2. Comparisons with global active contour models using two synthetic images and two medical images.

From left to right: segmentation results by the CV model, the GCV model and the RCV model.

https://doi.org/10.1371/journal.pone.0214851.g002

Comparisons with local active contour models (the LIC [12] model, the LCK [13] model and the LGFI model [14]).

We demonstrate the superior performance of the proposed method to previous local active contour methods on a synthetic image (see the third row of Fig 3) and three medical images (The first and last row give two brain images, whereas the second row gives a X-ray image.) with severe intensity inhomogeneity (see the last column of Fig 3). Fig 3 shows the results of the LIC model [12], the LCK model [13] and the LGFI model [14] and our method, all of which consider the local region information and fit them by local intensities means. Therefore, these methods can achieve much better results for images with intensity inhomogeneity than global active contour models. However, The LIC model [12] only uses the local mean information and thereby it fails to segment the objects when the intensity inhomogeneity of images are severe, which can be seen from the first column of Fig 3. Although the LCK model (see the second column of Fig 3) and LGFI model (see the third column of Fig 3) consider the gaussian kernel function as constraint and improve the segmentation results to some extent, these models still fail to discriminate the intensities of the objects. Due to the proper initialization from the coarse segmentation, our method obtain more satisfactory results (see the forth column of Fig 3) than other methods.

thumbnail
Fig 3. Comparisons with local active contour model using a synthetic image and three medical images with severe intensity inhomogeneity.

From left to right: segmentation result by LIC model, LCK model, LGFI model and our method, and the histograms of the image intensity.

https://doi.org/10.1371/journal.pone.0214851.g003

Table 2 introduces the computation complexity for Fig 3 in terms of total iterations and CPU time. The proposed method achieves the accurate segmentation results with less CPU time and less number of iterations. The LIC model [12], the LCK model [13] and the LGFI model [14] are all solved by standard gradient method, which take a large number of iterations to converge and make the level set function easily fall into local minima. In order to overcome the drawback of the standard gradient method, our model constructs a steepest descent on the model’s intrinsic manifold which converges extremely fast. Moreover, we split the process of segment into two stage and the second stage can get a suitable initialization from the first stage. This experiment proves our method can obtain the best segmentation results with the least time.

thumbnail
Table 2. Iterations and running time (in seconds) for the experiments shown in Fig 3.

https://doi.org/10.1371/journal.pone.0214851.t002

Results on natural images with intensity inhomogeneity.

In Fig 4, we have shown the performance of our method for natural images with different types of region properties. In the third column of Fig 4, the images (see the first column of Fig 4) with severe intensity inhomogeneity see the last column of Fig 4 and some of them with complex textures are used for this experiment. We can see that these images are well segmented into objects and backgrounds from the middle column of Fig 4. This section demonstrates that our method can be applied to handle different types of images.

thumbnail
Fig 4. Segmentation results for natural images with intensity inhomogeneity.

From left to right: the original images, segmentation results using our methods and the histogram of image intensity.

https://doi.org/10.1371/journal.pone.0214851.g004

Comparisons with the-state-of-art models on medical images with Poisson noise.

Fig 5 presents the segmentation results obtained by our method for five medical images (Two brain images are represented in column 1 and column 3, respectively. A blood image is given in column 2. Two ultrasound images are shown in column 4 and column 5.) with Poisson noise, from which we evaluate the performance of proposed method for noise images. The results of CV model, GCV model, RCV model are shown in first, second, third row of the Fig 5, respectively. As can be seen from these segmentation results, the global active contour models can not segment the noise images well. The results of LCK model, LGFI model and the proposed method are shown in forth, fifth and last row of the Fig 5. These local active contour methods all can segment objects from images, whereas the results of the proposed method are clearly better than other local active contour models.

thumbnail
Fig 5. Comparisons with state-of-art methods for niose images.

From top to bottom: segmentation results of CV model, GCV model, RCV model, LCK model, LGFI model and our method, respectively.

https://doi.org/10.1371/journal.pone.0214851.g005

Comparisons of the various methods by the total iterations and CPU time (in seconds) are listed in Table 3, from which we can see that our method converges obviously faster than other methods. On the one hand, our method converges with the Riemannian steepest method is much faster. On the other hand, the first stage provides the proper initialization for second stage. Therefore, our method is also applicable to noise images.

thumbnail
Table 3. Iterations and running time (in seconds) for the experiments shown in Fig 5.

https://doi.org/10.1371/journal.pone.0214851.t003

Quantitative comparisons with the-state-of-art models

To quantitatively analysis the performance of the proposed method, we adopt the F-score value [13] and Jaccard similarity (JS) [32] metrics as the evaluation framework of segmentation accuracy. The metric JS is defined as (24) where A is the computed object region and B is the ground truth region. Obviously, the closer the value of JS is to 1, the better segmentation result is obtained. Furthermore, to evaluate the results more precise, we also give the value of F-score, which can be calculated by (25) Where TP = AB (true positive) corresponds the correct segmented regions, FN (false negative) corresponds the false unsegmented regions, FP (false positive) corresponds the undetected segmented regions. Similar to the value of JS, the higher value of F-score means the better segmentation results.

Robustness to the severe intensity inhomogeneity.

Fig 6 shows the results of competing method segmentation on five synthetic images with changed intensity inhomogeneity. The first row of Fig 6 shows the five original images with initial contour. As can be seen, the object of each image is gradual difficult to be segmented from top to bottom. The segmentation results obtained by CV model are presented in the second row, GCV model are presented in the third row, RCV model are presented in the forth row, LBF model are presented in fifth row, LCK model are presented in the sixth row and the proposed model are presented in the last row, respectively. We can see that our method provides the best segmentation results, especially when the strength of the intensity inhomogeneity for objects is strong, which proves that the proposed method is more robust to image intensity inhomogeneity. The corresponding F-score values and JS values are shown in Fig 7, all the models can achieve high F-score values and JS values, when the strength of intensity inhomogeneity is slow (see the first column and second column of Fig 6). Meanwhile, the total iterations and CPU time of all the models, which are shown in Table 4, are close. Whereas, when the strength of intensity inhomogeneity becomes strong, the F-score values and JS values of the proposed method are still close to 1 and higher than other models. Furthermore, our method converges with 2 steps for all images, even the object of image is difficult to be segmented (see the last column of Fig 6). The main difference between our method and other local active contour is the proper initialization and iterative method.

thumbnail
Fig 6. Quantitative comparisons among CV model, GCV model, RCV model, LBF model, LCK model and our method for the images, which the intensity inhomogeneity is gradually increased from left to right.

https://doi.org/10.1371/journal.pone.0214851.g006

thumbnail
Table 4. Iterations and CPU time (in seconds) for the experiments shown in Fig 6.

https://doi.org/10.1371/journal.pone.0214851.t004

Robustness to different types of noises.

To quantitative illustrate the robustness of our method for different types of noises, we also evaluate the segmentation results by the F-score values and JS values. On the top row of the Fig 8, we show the segmentation results of the proposed method for images with different level of Gaussian noises. And on the bottom row of the Fig 8, we present the segmentation results of the proposed method for images with different level of speckle noise. As can be seen, the proposed method can achieve the satisfactory results even the noise is very strong. Furthermore, the F-score values and JS values for proposed method, which are shown in Fig 9, are also higher than 0.9, so the accuracy of the proposed method is ideal.

thumbnail
Fig 8. Segmentation results by the proposed method for images with different types of noise.

The first row: images with Gaussian white noise (zero means and different variances (σ = 0.01, 0.02, 0.03, 0.04)). The second row: images with speckle noise (zeros means and different variances (σ = 0.01, 0.02, 0.03, 0.04)).

https://doi.org/10.1371/journal.pone.0214851.g008

Conclusion

In this paper, we propose a fast two-stage segmentation method for the segmentation problem of synthetic and real-world (including medical) images with complex noise and severe intensity information. The first stage obtains coarse segmentation results in coarse space with low calculation complexity. Following it, the second stage takes the up-sampled contour of the first stage as the initialization and achieves accurate segmentation results in original space quickly. To minimize the energy function with low time complex, we propose a method measuring intensity under the framework of exponential family and converge the model by Riemannian steepest method. Finally, a smooth operator, which is Toeplitz and separable, is used to regularize the level set function to preserve more image details.

The main contribution of this paper is the formulation of two-stage energy function based on LCK model with exponential family for acquiring the intensity information and solving the two-stage method by smooth natural gradient method. Qualitative and quantitative analysis showed that the performance of the proposed method is superior to the LCK model and other state-of-art methods for intensity inhomogeneity images.

There are still some limitations for the proposed method, such as model still depends on the initializations to some extent and model is difficult to segment images with wispy and clutter targets. In the future, we plan to add shape constraints to optimization model and focus on developing the application of the proposed methodology for more types real-life images.

Appendix

The computation for .

Theorem 0.1 Suppose that Eq (10) holds. Then the matrix is diagonal with elements where the Kσ(xy) is a Gaussian kernel function with width σ, wτy is the final weight of the y-th pixel. we rearrange the above equation by using convolution method (26) as i = j and otherwise, the Bf (m*||m) is the fBregman divergence (27) In above equations, A(⋅) denotes the logarithm of the normalising constant of f andη A(m) is the gradient of A(⋅) with respect to the canonical constant vector η = η(θ). Furthermore, δ′(u) is evaluated by a regularised approximation , which is bounded away from zero, then matrix is positive definite.

Proof. To prove these results we first give the definition of the Fisher information matrix (FIM) [33] (28) where EI|ϕ represents the expectation operator of the function f(I(x)|ϕ) and f(I(x)|ϕ(x)) = ∫{x:ϕ(x)>0} f(I(x)|m1(x)) dx + ∫{x:ϕ(x)<0} f(I(x)|m2(x)) dx. Following above definition, we develop the derivatives of Fisher information matrix (FIM) (29) and then the is (30) if x = y and (G(ϕ))(x,y) = 0, then . Otherwise, we need to prove the diagonal elements of matrix are strictly positive.

In Eq (30), the E(…|ϕτ(x)) is the expectation with respect to the marginal likelihood (31) According to the fact that δ′(−x) = −δ′(x) and the Eq (30) can be rewritten as (32) In terms of Kullback-Leibler divergences [34], we obtain (33) In above equation, the Kullback-Leibler divergence is Finally, in this paper the function f belongs to exponential family and we can rewritten (33) in terms of Bregman divergences [34] Due to the fact that the terms Bf(mτ1(x)||mτ2(x))} and Bf(mτ2(x)||mτ1(x))} are strictly positive, and then the diagonal elements of matrix are strictly positive.

Supporting information

S1 Dataset. Contains synthetic images, real-world images and medical images used in Figs 1, 2, 3, 4, 5, 6 and 8.

https://doi.org/10.1371/journal.pone.0214851.s001

(ZIP)

Acknowledgments

The authors would like to thank the three anonymous reviewers for all the constructive and helpful review comments that have helped improved the manuscript significantly.

References

  1. 1. Withey DJ, Koles ZJ. Medical Image Segmentation: Methods and Software. In: 2007 Joint Meeting of the 6th International Symposium on Noninvasive Functional Source Imaging of the Brain and Heart and the International Conference on Functional Biomedical Imaging; 2007. p. 140–143.
  2. 2. Chan TF, Vese LA. Active contours without edges. IEEE Transactions on Image Processing. 2001;10(2):266–277. pmid:18249617
  3. 3. Kass M, Witkin A, Terzopoulos D. Snakes: Active contour models. International Journal of Computer Vision. 1988;1(4):321–331.
  4. 4. Cohen LD. On active contour models and balloons. CVGIP: Image Understanding. 1991;53(2):211–218.
  5. 5. Kichenassamy S, Kumar A, Olver P, Tannenbaum A, Yezzi A. Gradient flows and geometric active contour models. Proceedings of IEEE International Conference on Computer Vision. 1995; p. 4–9.
  6. 6. Malladi R, Sethian JA, Vemuri BC. Shape modeling with front propagation: a level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1995;17(2):158–175.
  7. 7. Xu Chenyang, Prince JL. Snakes, shapes, and gradient vector flow. IEEE Transactions on Image Processing. 1998;7(3):359–369. pmid:18276256
  8. 8. Li Bing Nan, Chui Chee Kong, Chang Stephen, Ong S.H. Integrating spatial fuzzy clustering with level set methods for automated medical image segmentation Computers in Biology and Medicine. 2011;41(1):1–10. pmid:21074756
  9. 9. Pratondo Agus, Chui Chee-Kong, Ong Sim-Heng Integrating machine learning with region-based active contour models in medical image segmentation Journal of Visual Communication and Image Representation. 2017; 43:1–9.
  10. 10. Bresson X, Esedoglu S, Vandergheynst P, Thiran JP, Osher S. Fast global minimization of the active contour/snake model. Journal of Mathematical Imaging and Vision. 2007;28(2):151–167.
  11. 11. Li Chunming, Kao Chiu-Yen, Gore JC, Ding Zhaohua. Minimization of Region-Scalable Fitting Energy for Image Segmentation. IEEE Transactions on Image Processing. 2008;17(10):1940–1949. pmid:18784040
  12. 12. Zhang K, Song H, Zhang L. Active contours driven by local image fitting energy. Pattern Recognition. 2010;43(4):1199–1206.
  13. 13. Wang L, Pan C. Robust level set image segmentation via a local correntropy-based K-means clustering. Pattern Recognition. 2014;47(5):1917–1925.
  14. 14. Akram F, Garcia MA, Puig D. Active contours driven by local and global fitted image models for image segmentation robust to intensity inhomogeneity. PLoS ONE. 2017;12(4):e0174813. pmid:28376124
  15. 15. Soomro S, Munir A, Choi KN. Hybrid two-stage active contour method with region and edge information for intensity inhomogeneous image segmentation. PLoS ONE. 2018;13(1):e0191827. pmid:29377911
  16. 16. Scheuermann B, Rosenhahn B. Analysis of Numerical Methods for Level Set Based Image Segmentation. Springer, Berlin, Heidelberg; 2009. p. 196–207.
  17. 17. Badshah N, Chen K. On two multigrid algorithms for modeling variational multiphase image segmentation. IEEE Transactions on Image Processing. 2009;18(5):1097–1106. pmid:19342341
  18. 18. Xie X, Wu J, Jing M. Fast two-stage segmentation via non-local active contours in multiscale texture feature space. Pattern Recognition Letters. 2013;34(11):1230–1239.
  19. 19. Song Y, Xie X. Fast two-stage segmentation based on local correntropy-based K-means clustering. In: 2017 9th IEEE International Conference on Communication Software and Networks, ICCSN 2017. vol. 2017-Janua. IEEE; 2017. p. 1317–1323.
  20. 20. Chartrand R, Staneva V. A faster-converging algorithm for image segmentation with a modified Chan-Vese model. In: Proceedings of the 2008 International Conference on Image Processing, Computer Vision, and Pattern Recognition; 2008. p. 212–216.
  21. 21. Mendi E, Milanova M. Quasi-Newton Minimization for Active Contours with Chan-Vese Model. Image (Rochester, NY). 2008;(3):212–216.
  22. 22. Bar L, Sapiro G. Generalized Newton-Type Methods for Energy Formulations in Image Processing. SIAM Journal on Imaging Sciences. 2009;2(2):508–531.
  23. 23. Sundaramoorthi G, Yezzi A, Mennucci AC, Sapiro G. New possibilities with Sobolev active contours. International Journal of Computer Vision. 2009;84(2):113–129.
  24. 24. Pereyra M, Batatia H, McLaughlin S. Exploiting information geometry to improve the convergence properties of variational active contours. IEEE Journal on Selected Topics in Signal Processing. 2013;7(4):700–707.
  25. 25. Ben Ayed I, Mitiche A, Belhadj Z. Multiregion level-set partitioning of Synthetic Aperture Radar images. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2005;27(5):793–800. pmid:15875799
  26. 26. Sarti A, Corsi C, Mazzini E, Lamberti C. Maximum likelihood segmentation of ultrasound images with rayleigh distribution. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control. 2005;52(6):947–960. pmid:16118976
  27. 27. Ben Ayed I, Hennane N, Mitiche A. Unsupervised variational image segmentation/classification using a weibull observation model. IEEE Transactions on Image Processing. 2006;15(11):3431–3439. pmid:17076402
  28. 28. Pereyra M, Batatia H, McLaughlin S. Exploiting information geometry to improve the convergence of nonparametric active contours. IEEE transactions on image processing: a publication of the IEEE Signal Processing Society. 2015;24(3):836–45.
  29. 29. Lecellier F, Fadili J, Jehan-Besson S, Aubert G, Revenu M, Saloux E. Region-based active contours with exponential family observations. Journal of Mathematical Imaging and Vision. 2010;36(1):28–45.
  30. 30. Amari S, Douglas SC. Why natural gradient? In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing. vol. 2. IEEE; 1998. p. 1213–1216.
  31. 31. Chan TF, Esedoglu S, Nikolova M. Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models. SIAM Journal on Applied Mathematics. 2006;66(5):1632–1648.
  32. 32. Zhang K, Zhang L, Yang MH. Fast Compressive Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2014;36(10):2002–2015. pmid:26352631
  33. 33. Amari S. Differential-Geometrical Methods in Statistics. vol. 28; 1990.
  34. 34. Nielsen F, Nock R. Entropies and cross-entropies of exponential families. In: Proceedings—International Conference on Image Processing, ICIP. IEEE; 2010. p. 3621–3624.