Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Selective image segmentation driven by region, edge and saliency functions

  • Shafiullah Soomro ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    ‡ SS and AN are contributed equally to this work and share the first authorship.

    Affiliations Department of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea, Department of Computer Science and Media Technology, Linnaeus University, Vaxjo, Sweden

  • Asim Niaz ,

    Roles Project administration, Resources, Software, Writing – review & editing

    ‡ SS and AN are contributed equally to this work and share the first authorship.

    Affiliation Department of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea

  • Toufique Ahmed Soomro,

    Roles Validation, Writing – review & editing

    Affiliation School of Computing and Mathematics, Charles Sturt University, Bathurst, Australia

  • Jin Kim,

    Roles Funding acquisition

    Affiliation SecuLayer Inc., Seoul, South Korea

  • Adnan Manzoor,

    Roles Writing – review & editing

    Affiliation Department of Artificial Intelligence, Quaid-e-Awam University of Engineering Science and Technology, Nawabshah, Sindh, Pakistan

  • Kwang Nam Choi

    Roles Formal analysis, Funding acquisition, Supervision, Writing – review & editing

    knchoi@cau.ac.kr

    Affiliation Department of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea

Abstract

Present active contour methods often struggle with the segmentation of regions displaying variations in texture, color, or intensity a phenomenon referred to as inhomogeneities. These limitation impairs their ability to precisely distinguish and outline diverse components within an image. Further some of these methods employ intricate mathematical formulations for energy minimization. Such complexity introduces computational sluggishness, making these methods unsuitable for tasks requiring real-time processing or rapid segmentation. Moreover, these methods are susceptible to being trapped in energy configurations corresponding to local minimum points. Consequently, the segmentation process fails to converge to the desired outcome. Additionally, the efficacy of these methods diminishes when confronted with regions exhibiting weak or subtle boundaries. To address these limitations comprehensively, our proposed approach introduces a fresh paradigm for image segmentation through the synchronization of region-based, edge-based, and saliency-based segmentation techniques. Initially, we adapt an intensity edge term based on the zero crossing feature detector (ZCD), which is used to highlight significant edges of an image. Secondly, a saliency function is formulated to detect salient regions from an image. We have also included a globally tuned region based SPF (signed pressure force) term to move contour away and capture homogeneous regions. ZCD, saliency and global SPF are jointly incorporated with some scaled value for the level set evolution to develop an effective image segmentation model. In addition, proposed method is capable to perform selective object segmentation, which enables us to choose any single or multiple objects inside an image. Saliency function and ZCD detector are considered feature enhancement tools, which are used to get important features of an image, so this method has a solid capacity to segment nature images (homogeneous or inhomogeneous) precisely. Finally, the adaption of the Gaussian kernel removes the need of any penalization term for level set reinitialization. Experimental results will exhibit the efficiency of the proposed method.

Introduction

Image segmentation plays a pivotal role in image processing and computer vision, serving as a crucial step for various applications, such as image recognition and object detection. Over the last decade, numerous methods have emerged for image segmentation, encompassing techniques like region merging, thresholding-based methods, edge detection approaches, as well as clustering-based methods. Active contours, pioneered by Kass [1], have found widespread application in image segmentation, with subsequent adaptations introduced by various researchers [24].

The core concept of active contour is to guide the curve’s motion toward the object boundary using energy minimization algorithms. The initial structure of active contours, however, exhibited two distinct drawbacks. Firstly, it was highly sensitive to its starting position. Secondly, managing topological changes that arose during the curve’s evolution posed a challenge. Addressing this concern, Osher and Sethian [5] introduced a curve model for the moving front. In this model, the curve was implicitly represented by a zero level set function. Subsequently, a multitude of variations on the level set models emerged, which could be categorized into two primary classes: edge-based and region-based models. Edge-based models [1, 6] operate by responding to the image’s gradient to achieve image segmentation. These approaches enable level set segmentation without the need for re-initialization. Nonetheless, edge-based models often encounter challenges when dealing with delicate boundaries, making accurate identification problematic.

Region-based models aim to identify distinct regions by utilizing specific descriptors, such as color or intensity, to guide contour movement. Recently, these models have gained significant attention for their ability to capture regional data [721]. Initially designed to segment objects with uniform intensity, traditional region-based models, like the Chan-Vese (CV) model [8], excel at capturing images with similar intensity levels. However, the CV model struggles when faced with intensity variations or texture information in images. To address this, region-based models have been enhanced and adapted into local region-based methods for segmenting images with inhomogeneous intensity variation [10]. Notably, Li et al. introduced the first local fitting model, known as LBF [10]. This model employs the concept of region scalable fitting energy (RSF) to capture inhomogeneous intensity patterns, facilitated by local adaptation using Gaussian kernels.

Similarly, in the context of addressing local energy considerations, the VLBCS (Variational Level Set Approach for Bias Correction and Segmentation) method, as described in [22], and the LSACM (Local Statistical Active Contour Model) approach, as presented in [23], were introduced. These techniques have further improved the outcomes of image segmentation for cases involving inhomogeneous images by integrating bias field correction into their formulations. This integration allows them to approximate the distribution of inhomogeneities across the entire image [24].

Additionally, the LGDF (Local Gaussian Distribution Fitting) method, as detailed in [25], and the LSD (Local Signed Difference Energy) approach, as outlined in [26], were proposed. These methods incorporate considerations of local intensity into the evolution of the level set and have shown promising results when dealing with images exhibiting intensity inhomogeneities. However, it’s important to note that these methods come with notable limitations, particularly concerning the need for level set re-initialization and computationally complex formulations.

Recently, there has been a trend in reconfiguring edge-based models by substituting their edge term with a region term in order to construct the Signed Pressure Force (SPF) function. Among these approaches, one of the most noteworthy is the work of Zhang et al. as detailed in [9]. They were the first to introduce an SPF function that effectively captures regions in an adaptive manner. Subsequently, several techniques, as discussed in [15], have emerged in the field of image segmentation, focusing on both local and global SPF formulations for homogeneous and inhomogeneous images.

Recent research in cognitive psychology, neurobiology, computer vision, and image processing has revealed a strong connection between saliency detection and the human visual system. Consequently, numerous investigators have made efforts to incorporate saliency features into image segmentation processes, as demonstrated in previous studies such as those by Bai and Wang in [27], Qin et al. in [28], and Zhi et al. in [29].

For instance, Bai and Wang introduced the Saliency-SVM model in [27], which takes into account saliency data and formulates the image segmentation problem accordingly. In a similar vein, Qin et al., as described in [28], combined region and affinity propagation clustering algorithms with region saliency to develop a random walk-based image segmentation model. More recently, Zhi et al., detailed in [29], devised an active contour method based on regions and edges, incorporating saliency knowledge into the energy functional to enhance the segmentation process.

Real-world images often exhibit variations in intensity, color, and texture, presenting a significant challenge for traditional segmentation methods in effectively delineating distinct regions. This challenge becomes particularly pronounced when dealing with images that contain objects demonstrating both homogeneous and inhomogeneous characteristics. Additionally, certain existing approaches rely on complex mathematical formulations, which can result in sluggish computational performance during the energy minimization process. Our objective is to develop a method that strikes a balance between precision and computational efficiency, rendering it suitable for practical applications.

Furthermore, achieving precise boundaries for image regions, especially when dealing with objects characterized by faint or subtle edges, poses a substantial challenge for many segmentation techniques. Our aim is to devise an approach capable of accurately capturing these boundaries, even in such intricate cases. Moreover, different regions within an image may necessitate distinct segmentation strategies. Therefore, we endeavor to create a method that seamlessly integrates both local and global information to adaptively capture diverse image structures.

In our investigations, we have also observed that saliency detection, inspired by human visual perception, has yielded promising results in enhancing image comprehension and segmentation. This observation motivates us to leverage saliency information as a means to improve the accuracy and robustness of our segmentation method.

Guided by these motivations, our newly proposed active contour method has been meticulously crafted to tackle these formidable challenges head-on. Our method leverages a combination of key components, including a zero-crossing detector (ZCD) to extract edge information, saliency detection to discern the significance of regions, and a globally tuned region-based signed pressure force (SPF) term. Through this work, we strive to contribute a method that not only advances the field of image segmentation but also has practical implications for a wide range of applications.

This research proposes a novel active contour method with the following major contributions.

  1. First this method captures the global structures of an image by using zero crossing feature detector and using it as a driving force for contour evolution.
  2. This method has designed a new energy term by amalgamating saliency map function in level set formulation.
  3. Proposed method can segment an inhomogeneous and homogeneous set of images by considering global SPF, ZCD and saliency information at some scaled value.
  4. Proposed method is using a selective level set formulation from Zhang et al [9], this property allows the proposed method to capture the region of interest inside an image.

Proposed model

As discussed earlier, active contours represent curve C by zero level set function as: C = (x, y)|ϕ(x, y) = 0 and describe level set energy functional as: (1) The force F in above energy function relies on image data and level set function ϕ.

Motivated by the work of [9, 29] methods, we propose a novel homogeneous and inhomogeneous image segmentation method by combining global SPF and ZCD terms with saliency function at some scaled value. The Proposed formulation of the saliency, region and ZCD based ESRZCD method is defined as: (2)

Equation above is scaled with a parameter s which ranges between 0 ≤ s ≤ 1. This parameter evaluate the effect of the intensity and Saliency terms based on the image we have. We set s large when we want the intensity term as a leading force to capture homogeneous and inhomogeneous images with less saliency information. Similarly, we choose s small if we want saliency as a leading force to capture textual features from an image.

In Eq 2, w is also an important positive parameter ranges between 0 ≤ w ≤ 1, and it plays a key part in dealing with an image region and edge information. When w is close to 0 then the proposed energy work as a leading global segmentation method with some saliency information. Likewise, when w it is close to 1 then it has a predominant ZCD function which makes the proposed method act as an inhomogeneous image segmentation method with some saliency information.

Many researchers have previously employed various zero-crossing detectors to extract edge information. However, in our proposed energy function, we utilize the Difference of Gaussian (DoG) detector as a force term, denoted as FDoG, for detecting image boundaries. This method highlights regions where intensity changes rapidly, effectively serving as a zero-crossing edge detector. Additionally, we incorporate a DoG force term, FDoG, as an image enhancement feature. This feature is derived by subtracting a blurred version of the original image from a less blurred version of the same original image. It enhances the visibility of edges and is employed as an energy source for image segmentation. For an image denoted as I(x, y): Ω ← R2, the DoG term in Eq (2) is defined as: (3) where σ1 and σ2 are the standard deviations of the Gaussian filters, where σ1 < σ2. These parameters plays a crucial role in obtaining the result and removing unwanted noise from an image. For noisy images, the difference of σ1 < σ2 must be chosen high and for images with less noise σ1 < σ2 must be chosen small to retain image details properly.

Edge-based active contour methods rely solely on gradient-based information, which can lead to imprecise segmentation outcomes. Consequently, these models often become trapped in local minimums or struggle to accurately delineate fuzzy and noisy edges. Therefore, in Eq (2), our proposed approach incorporates a global region-based SPF (Signed Pressure Force) function inspired by [11]. This function utilizes the global region information within an image, guiding the level set flow to shift the contour away from boundaries and effectively capture homogeneous regions. The formulation of the SPF function is described as follows: (4) where j1 and j2, are the intensity mean values taken from CV model [8] defined as: (5) SPF function is very adaptive with respect to its position so that it can shrink or expand it movement if placed inside or outside of object. Hε(ϕ) Eq (8) and in Eq (5) is the Heaviside term written as: (6) where ϵ stabilize the smoothness of Heaviside term.

Saliency function in Eq (2) used to capture color and texture feature of an image. For I(x, y): Ω ← R2, we take saliency function adapted as [27]. (7) Where Im is mean pixel value of an image and IG = I(x, y) * G is an image smoothed by Gaussian kernel. Subsequently, by taking the derivative of 7 with respect to s1, and s2 we get the corresponding values of s1 and s2 inside Ωin and outside Ωout as: (8)

By looking at the adaptivity of the SPF function from [9], we have also redefined saliency function and formulated new adaptive saliency function as: (9) The traditional geodesic active contour (GAC) method [6] is considered as a pioneer active contour method, which uses an object gradient information from an edge. Let I: Ω ⊂ R2 is an image domain, I: Ω → R is an input image and C(q) is a closed curve. GAC has formulated the following energy formulation: (10) h is the edge stopping function designed as: (11)Gσ * I is known as the convolution of an image I with a kernel having standard deviation σ. We get the following formulation after minimizing above energy functional. (12) where α is included to increase the propagation, k is curvature of the curve and is inward normal of the curve. The final level set equation is defined as follows: (13) Finally, by substituting the FSMLOG from 2 in to 13, the Gateaux derivative of the proposed method can be written as. (14) It can be seen that the 1st term of the equation above is using some penalizing energy term to maintain the level set deviation from a signed distance function. However, the proposed method adopts instead a Gaussian kernel to regularize the level set function, which maintains the stability of the level set as a signed distance function. Therefore, we can eliminate the from 14. Moreover, we are also incorporating regional information of an image in our formulation, which avoids edge leakage problems occurring in only edge-based methods. In this context, we can also remove unnecessary term ∇FSMLOG(I(x).∇ϕ from 14. Therefore, the final level set equation is rewritten as: (15) The level set function in active contours is often required to be initiated as SDF (signed distance function) and also to be reinitialized to maintain its stability. However, the proposed method has deployed the Gaussian kernel after each iteration, which not only removes the problem of costly re-initialization the but also maintains the stability of the level set function. We define the initial level set function ϕ0 as: (16) ρ ≥ 0 is a constant value, Ω0 represents a region inside an initial contour, Ω is the image domain and ∂Ω is an initial contour. Finally in order to obtain the results of our proposed method, we follow the following iterative procedure:

Algorithm

1. Initialize level set as constant function, ϕ(x, t = 0) from Eq (16).

2. n = 1.

3. while the solution is not converged do

4. Compute the values of DoG, SPF and SAL function from (9), (4) and from (4).

5.  Solve the final level set from (15).

6.  n = n + 1.

7. end while

8. Output: Final result, ϕ.

Results and quantitative comparisons

Each experiment of this paper executed over MATLAB 2018 on a Windows 10 in a PC with Intel i7, 2.9 GHz with 16 GB RAM. We have chosen the parameters for our experiments as ρ = 1, ϵ = 1.5, σ = 1, σ1 = 1, σ2 = 2, Δt = 1 and α = varies for image to image.

To demonstrate the effectiveness of our proposed method, we initially applied it to both grayscale intensity and real RGB images. The results generated by our approach can be seen in Fig 1. In 1st row of Fig 1, we present the outcome obtained from the grayscale intensity image. Given that this image primarily consists of intensity regions, we set the parameters as follows: σ1 = 1, σ2 = 10, w = 0.9, and s = 0.999. These parameter settings prioritize the utilization of the Difference of Gaussians (DoG) and Signed Pressure Force (SPF) components while minimizing the emphasis on saliency information, resulting in a precise delineation of regions. Similarly, In the 2nd of Fig 1, we display the results we obtained from a real RGB texture image. For this image, we adjusted the settings as follows: σ1 = 2, σ2 = 20, w = 0.01, and s = 0.01. These settings were chosen to improve the segmentation by giving more importance to the saliency force while reducing the influence of edge and region details.

thumbnail
Fig 1. Proposed method outcomes for Inhomogeneous (1st row) and real texture (2nd row) image segmentation.

Column I: Original images with initial level set, column II: DOG profile of an image, column III: Saliency profile of an image and column IV: Final segmentation result.

https://doi.org/10.1371/journal.pone.0294789.g001

Further, we have shown intensity inhomogeneity analysis of five different images with other state of art methods in Fig 2. The results demonstrate that CV [8] is unable to handle inhomogeneity across the images and eventually failed to segment objects. Bai et al. [27] is using only saliency information, therefore it has also failed to get accurate results. As LSACM [23] and VLSBCS [2] are using local pixel information with bias intensity correction, therefore this method has got some positive outcomes to some extent. However, the proposed method has successfully segmented all the images accurately compared to all previous methods. Computational complexity of Fig 2 is compared in Table 1 in terms of time and iterations taken by each method. CV [8] method is very fast compared to other methods yet failed to get accurate outcome. Proposed method consumed a smaller number of iterations than other methods while getting accurate outcomes.

thumbnail
Fig 2. Proposed method comparision on images having intensity inhomogeneity.

Column 1st: (Initial Position), Column 2nd: (CV [8]), Column 3rd: (LSACM [23]), Column 4th: (Bai et al. [27]), Column 5th: VLSBCS. [2], Column 6th: GLFIF. [30], Column 7th: HLFRA. [31] and Column 8th: (proposed method) respectively.

https://doi.org/10.1371/journal.pone.0294789.g002

thumbnail
Table 1. CPU time comparison (in seconds) and iterations consumed by each method in Fig 2.

https://doi.org/10.1371/journal.pone.0294789.t001

Subsequently, the result of the proposed method is obtained over the real set of images and compared with other methods in Fig 3. The results demonstrate that CV [8] is unable to handle real set of images and eventually failed to segment objects. Bai et al. [27] is using only saliency information, which is not sufficient to segment out an inhomogeneous real set of images. Moreover, LSACM [23] and VLSBCS [2] are using only local pixel information, which cannot capture texture details of images. on the other hand, the proposed method has successfully segmented all the images accurately compared to all previous methods. In Table 2, we have measured the time complexity and number of iterations of each method consumed over every single image in Fig 3,. The results have proved that the proposed method has consumed a smaller number of iterations and lesser time than other methods.

thumbnail
Fig 3. Proposed method comparision over real set of images.

Column 1st: (Initial Position), Column 2nd: (CV [8]), Column 3rd: (LSACM [23]), Column 4th: (Bai et al. [27]), Column 5th: VLSBCS. [2], Column 6th: GLFIF. [30], Column 7th: HLFRA. [31] and Column 8th: (proposed method) respectively.

https://doi.org/10.1371/journal.pone.0294789.g003

thumbnail
Table 2. CPU time comparison (in seconds) and iterations of previous methods and proposed method in Fig 3.

https://doi.org/10.1371/journal.pone.0294789.t002

Fig 4 illustrates the selective segmentation property of the proposed method. The first column shows the selective initialization, whereas the second column demonstrate the selective segmentation result of the proposed method.

thumbnail
Fig 4. Selective segmentation result of the proposed method.

Column 1st: (Initial Position), Column 2nd: (Selective outcome).

https://doi.org/10.1371/journal.pone.0294789.g004

By taking an advantage of the selective property of the proposed method. In Fig 5, we have performed segmentation over five different images to capture the region of interest. The first row illustrates the initial position of the contour, which is placed inside of the region and the second row illustrates the corresponding outcomes. In the first column, we have used a liver tumor image taken from [32], the second column shows a brain tumor image taken from the BRATS 2015 dataset [33], the third column shows a lung CT image taken from the covid dataset [34], the fourth column shows a synthetic image and the last column shows a cardiac image taken from public repository.

thumbnail
Fig 5. Segmentation results of the proposed method with region of interest.

Row I: (Initial Position), Row II: (Result of the proposed method).

https://doi.org/10.1371/journal.pone.0294789.g005

In Fig 6, we have performed the proposed method segmentation and its comparison with other methods over an image with varying noise details. The results have demonstrated that the proposed method has outperformed previous methods in terms of noisy image segmentation. Moreover, we have measured the time complexity and number of iterations of each method in Table 3. It is clearly illustrated that the proposed method has achieved the required results in less time and in a less number of iterations.

thumbnail
Fig 6. Proposed method comparision over noisy images.

Column 1st: (Initial Position), Column 2nd: (CV [8]), Column 3rd: (LSACM [23]), Column 4th: (Bai et al. [27]), Column 5th: VLSBCS. [2], Column 6th: GLFIF. [30], Column 7th: HLFRA. [31] and Column 8th: (proposed method) respectively.

https://doi.org/10.1371/journal.pone.0294789.g006

thumbnail
Table 3. CPU time comparison (in seconds) and iterations of previous methods and proposed method in Fig 6.

https://doi.org/10.1371/journal.pone.0294789.t003

In Fig 7, we have taken mammogram images from mini-Mias [35] dataset to validate our method performance, the results have illustrated that all previous methods were not capable to capture the region of interest due to the deficit of characteristic function. Since, the proposed method is using the selective characteristic function, which can capture the desired region based on contour initialization. Therefore, the proposed methods have segmented all regions perfectly. We have also measured the time complexity and number of iterations of each method in Table 4. It shows that the proposed method has acquired segmentation results in minimum time and iterations.

thumbnail
Fig 7. Proposed method comparision over mammogram tumor images.

Column 1st: (Initial Position), Column 2nd: (CV [8]), Column 3rd: (LSACM [23]), Column 4th: (Bai et al. [27]), Column 5th: VLSBCS. [2], Column 6th: GLFIF. [30], Column 7th: HLFRA. [31] and Column 8th: (proposed method) respectively.

https://doi.org/10.1371/journal.pone.0294789.g007

thumbnail
Table 4. CPU time comparison (in seconds) and iterations of previous methods and proposed method in Fig 7.

https://doi.org/10.1371/journal.pone.0294789.t004

To ensure a fair qualitative analysis, we present in Table 5 an asymptotic analysis alongside a comparison with alternative approaches. It is evident from the table that the proposed method shares a cubic time complexity (O(n3)) with some of the existing methods, like LSACM and HLFRA but it has been designed to deliver specific and intended results that might not be achievable with methods that have a lower time complexity. The choice of the appropriate algorithm depends on the specific requirements of the task and the trade-offs between time complexity and the quality of results. In this context, the proposed method has been shown to achieve its intended results effectively, even with its cubic time complexity, making it a valuable addition to the available techniques for the given problem.

thumbnail
Table 5. Asymptotic analysis (Big O) previous methods and proposed method.

https://doi.org/10.1371/journal.pone.0294789.t005

Quantitative analysis

For the quantitative analysis of the proposed method. We have taken 100 images with sizes 1024 X1024 from the publicly available dataset [36] and compared the proposed method results with previous methods. Some of the results are illustrated in Fig 8 with the initial position of the contour shown in column I, ground truth is shown in Column II and their segmentation result is displayed in column III.

thumbnail
Fig 8. Proposed method validation over [36] dataset.

Column 1st: (Initial Position), Column 2nd: (Ground truth, 3rd: (Proposed method result).

https://doi.org/10.1371/journal.pone.0294789.g008

For quantitative analysis, we use the Dice index, accuracy, specificity, Jaccard index and F1 score metrics. As per definitions of these metrics, the obtained result will be viewed as satisfactory when their values are near 1. The dice index estimates how much perceived cancer matches the ground truth. The accuracy metric alludes to the closeness of the segmented region to its ground truth region. Specificity is how precisely the framework keeps away from true negative regions. Jaccard index coefficient estimates the cross-over of the result over the ground truth and F1 score is the harmonic mean of the precision and recall. The formulation of these metrics is elaborated as below: (17) (18) (19) (20) (21) where, TP (true positive) exhibits segmented cancer tissues, TN (true negative) describes precisely unsegmented tissues, FP (false positive) describes wrongly segmented tissues considered as tumor or region of interest and FN (false negative) describes unsegmented tumor or region.

The quantitative analysis of [36] is illustrated in Table 6 and it shows that proposed formulation has achieved better Dice index, accuracy, specificity and Jaccard index values in comparision with previous formulations.

thumbnail
Table 6. Quantitative analysis of the proposed method over [36] dataset.

https://doi.org/10.1371/journal.pone.0294789.t006

Similarly, we have validated the proposed method quantitatively over BRATS 2015 [33] for brain tumor detection. This dataset contains MRI data of four successions for every case known as T1-weighted(T1), T1 with gadolinium-upgrading contrast (T1c), T2-weighted (T2), and FLAIR. More recently, active contours have been adapted frequently by researchers for brain tumor detection [37]. In this regard, we executed our method using BRATS 2015 [33] dataset over 200 HGG(high-grade glioma) and 44 LGG (low-grade) patient volumes. Some of the results of brain tumor detection are demonstrated in Fig 9, where first column illustrates the initial position of the contour, second column shows tumor segmentation and third column shows segmented tumor from an image. Similarly, quantitative results are shown in Table 7. The results show that proposed method have achieved better outcomes for DSC, accuracy, specificity, Jaccard index and F1 score compared to previous methods.

thumbnail
Fig 9. Proposed method validation over BRATS 2015 [33] dataset.

Column I: (Initial Position), Column II: (Proposed method segmentation, Column III: (extracted tumor).

https://doi.org/10.1371/journal.pone.0294789.g009

thumbnail
Table 7. Quantitative analysis of proposed method over BRATS 2015 [33] dataset.

https://doi.org/10.1371/journal.pone.0294789.t007

Discussion

Parameter selection

In this section, we outline the specific parameters utilized in our proposed method to achieve accurate and effective results. The following parameters are discussed in detail:

  1. ϵ
    The parameter ϵ is employed in both the Dirac delta and Heaviside step functions. It serves the purpose of introducing a finite width or regularization to these functions, rendering them suitable for practical calculations and numerical simulations. Modifying the value of ϵ can lead to discontinuities or computational challenges. For our experiments, a fixed value of ϵ = 1.5 was used consistently.
  2. σ1 and σ2
    σ1 and σ2 are integral parameters in the Difference of Gaussians (DoG) filter, a widely adopted technique in image processing. The DoG filter serves various purposes, including image enhancement, edge detection, and feature extraction.
    σ1 represents the standard deviation of the first Gaussian function applied in the DoG filter. A larger σ1 corresponds to a wider and smoother Gaussian, preserving low-frequency image information while minimizing noise.
    σ2 signifies the standard deviation of the second Gaussian function in the DoG filter. Typically, σ2 is chosen to be larger than σ1. The choice of these parameters affects the scale at which edges are detected. Smaller values of σ1 and σ2 highlight fine details, while larger values emphasize broader features.
  3. ρ
    The parameter ρ is crucial for contour initialization and depends on the segmentation task’s requirements and object characteristics within the image. Setting ρ to 1 establishes an initial contour that is relatively impartial and straightforward. This allows the active contour model to adapt flexibly and move towards object boundaries, accommodating a broader range of potential shapes during the segmentation process.
  4. w
    The parameter “w” holds significance within the range of 0 to 1 and plays a pivotal role in balancing image regions and edge information. The behavior of the proposed energy function is contingent on the value of “w.” When “w” approaches 0, the energy function functions primarily as a global segmentation method with an added emphasis on saliency information. Conversely, as “w” approaches 1, the energy function functions as a Zero Crossing Detector (ZCD), transforming the proposed method into an inhomogeneous image segmentation technique. This parameter is manually adjusted depending on the image’s intensity levels.
  5. s
    The parameter “s” is defined within the range of 0 to 1 and plays a crucial role in assessing the impact of both intensity and saliency terms based on the image’s characteristics. A larger value of “s” prioritizes the intensity term, making it the dominant factor, which is useful for capturing both homogeneous and inhomogeneous images with minimal emphasis on saliency information. A smaller value of “s” shifts the balance in favor of saliency, ideal for highlighting and extracting textual features from the image. The selection of the “s” parameter is made manually, tailored to the specific image type under consideration.

By clearly defining and explaining these parameters, we aim to provide a better understanding of how they are utilized in our proposed method to achieve the desired results.

Comparision with machine learning based segmentation techniques

Contemporary segmentation techniques have incorporated machine learning, particularly deep learning architectures, to enhance object segmentation [38] due to their remarkable accuracy and resilience. Nonetheless, the adoption of deep learning methods comes with significant resource demands and prerequisites, such as access to ample computational resources and prior data knowledge. In contrast, the proposed approach relies on a level set-based partial differential structure [39], offering advantages that include not necessitating GPUs, substantial physical resources, or any prior familiarity with the data.

Limitations of the proposed method

Proposed hybrid active contour model, which combines Difference of Gaussians, Chan-Vese global terms, and saliency information, has a few limitations. It relies on manual parameter tuning, making it less automated, and can be computationally intensive. It’s also sensitive to variations in image quality and may require significant computational resources. Additionally, it may not generalize well across different types of images and might demand more manual intervention compared to recent, more user-friendly methods.

Conclusion

This research has proposed a novel selective homogeneous and inhomogeneous object segmentation driven by an image global SPF (signed pressure force), a difference of Gaussian (DoG) and saliency functions. With the hybrid combination of DoG, Saliency and SPF we have formulated an extraordinary force function, which has disposed of the restrictions of previous level set based methods. Further, the proposed method has expanded its proficiency by including a selective segmentation behavior, which allows us to segment desired region or the region of interest from any image. In the end, we have adjusted the Gaussian filter to regularize the level set and to maintain a strategic distance from a costly reinitialization. We performed several experiments with real and synthetic images for the validation of the proposed method. Moreover, we adapted the Dice index, accuracy, specificity, F1 score and Jaccard index metrics for the quantitative validation of our method. Results show that the proposed procedure has accomplished improved results that stood out from past methods.

References

  1. 1. Kass M, Witkin A, Terzopoulos D. Snakes: Active contour models. international journal of computer vision. 1988 1(4)(321–331).
  2. 2. Li C, Huang R, Ding Z, Gatenby JC, Metaxas DN, Gore JC. A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI. IEEE Transactions on Image Processing. 2011 20(07):2007–2016. pmid:21518662
  3. 3. Li C, Gore J, Davatzikos C. Multiplicative intrinsic component optimization (MICO) for MRI bias field estimation and tissue segmentation. Magnetic resonance imaging. 2014 32:913–923. pmid:24928302
  4. 4. Ivanovska T, Laqua R, Wang L, Schenk A, Yoon JH, Hegenscheid K, et al. An efficient level set method for simultaneous intensity inhomogeneity correction and segmentation of MR images. Computerized Medical Imaging and Graphics. 2016 48:9–20. pmid:26741125
  5. 5. Stanley O and James A.S Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of computational physics. 1988 79:12–49.
  6. 6. Caselles V, Kimmel R, Sapiro G. Geodesic active contours. International journal of computer vision. 1997 22(1):61–79.
  7. 7. Mumford D, Shah J. Optimal approximations by piecewise smooth functions and associated variational problems. Communications on pure and applied mathematics. 1989 42(5):577–685.
  8. 8. Chan TF, Vese LA. Active contours without edges. IEEE Transactions on image processing. 2001 10(2):266–277. pmid:18249617
  9. 9. Zhang K, Zhang L, Song H, Zhou W. Active contours with selective local or global segmentation: a new formulation and level set method. Image and Vision computing. 2010 28(04):668–679.
  10. 10. Li C, Kao CY, Gore JC, Ding Z. Minimization of region-scalable fitting energy for image segmentation. IEEE transactions on image processing. 2008 17(10):1940–1949. pmid:18784040
  11. 11. Zhang K, Song H, Zhang L Active contours driven by local image fitting energy. Pattern recognition., vol 43, 2010, Elsevier, pp. 1199–1206.
  12. 12. Zhu S, Bu X, Zhou Q. A Novel Edge Preserving Active Contour Model Using Guided Filter and Harmonic Surface Function for Infrared Image Segmentation. in IEEE Access., vol 6, 2018, IEEE, pp. 5439–5510.
  13. 13. Soomro S., Munir A., Choi K. N. Fuzzy c-means clustering based active contour model driven by edge scaled region information. in Expert Systems with Applications., vol 120, 201900000, Elesevier, pp. 387–3960.
  14. 14. Memon A A, Soomro S., Shahid M T, Munir A., Niaz A. and Choi K. N. Segmentation of Intensity-Corrupted Medical Images Using Adaptive Weight-Based Hybrid Active Contours. Computational and Mathematical Methods in Medicine., 2020, Hindawi. pmid:33204300
  15. 15. Memon A. A, Niaz A., Soomr S., Iqbal E., Munir A. and Choi K.N. Hybrid SPF and KD Operator-Based Active Contour Model for Image Segmentation. in IEEE Access., vol 8, 2020, IEEE, pp. 198368–198383.
  16. 16. Niaz A., Memon A.A., Rana K., Joshi A., Soomro S., Kang S.J. et al. Inhomogeneous Image Segmentation Using Hybrid Active Contours Model With Application to Breast Tumor Detection. in IEEE Access., vol 8, 2020, IEEE, pp. 186851–186861.
  17. 17. Munir A., Soomro S., Shahid M.T, Soomro T.A. and Choi K.N. Hybrid Active Contours Driven by Edge and Region Fitting Energies Based on p-Laplace Equation. in IEEE Access., vol 7, 2019, IEEE, pp. 135399–135412.
  18. 18. Soomro S., Soomro T.A., Choi K.N. An Active Contour model based on region based fitting terms driven by p-Laplace length regularization. in IEEE Access., vol 6, 2018, IEEE, pp. 58272–58283.
  19. 19. Hachia B.E., Benkhechibaa A.E., Kireda M.R., Hachib D. and Haboussi M. Some investigations on 3D homogenization of nano-composite/nano-porous materials with surface effect by FEM/XFEM methods combined with Level-Set technique. in Computer Methods in Applied Mechanics and Engineering., vol 371, 2020, Elsevie, pp. 113319.
  20. 20. Niaz A., Iqbal E., Akram F., Kim J., and Choi K. N., Self-initialized active contours for microscopic cell image segmentation, Scientific Reports, vol. 12, no. 1, p. 14947, 2022, Nature Publishing Group, UK, London. pmid:36056042
  21. 21. Zhang K, Liu Q, Song H, Li X. A variational approach to simultaneous image segmentation and bias correction. IEEE Transactions on Cybernetics., vol 45, 2015, IEEE, pp. 1426–1437. pmid:25347891
  22. 22. Li C, Huang R, Ding Z, Gatenby C, Metaxas D, Gore J. A variational level set approach to segmentation and bias correction of images with intensity inhomogeneity.
  23. 23. k. Zhang, L. Zhang, KM. Lam, and D. Zhang. A local active contour model for image segmentation with intensity inhomogeneity. arXiv preprint arXiv:1305.7053. 2013.
  24. 24. K. Zhang, L. Zhang and S. Zhang A variational multiphase level set approach to simultaneous segmentation and bias correction. 2010 IEEE International Conference on Image Processing 2010, IEEE, pp. 4105–4108.
  25. 25. Li W, Jim M, Quansen S, Deshen X and Chunming L Level set segmentation based on local Gaussian distribution fitting. Asian Conference on Computer Vision., 2009, Springer, pp. 293-302.
  26. 26. Lingfeng W, Huaiyu W and Chunhong P Region-based image segmentation with local signed difference energy. Pattern Recognition Letters., vol 34, 2013, Elsevier, pp. 637–645.
  27. 27. Bai Xuefei and Wang Wenjian Saliency-SVM: An automatic approach for image segmentation. Neurocomputing., 2014, Elsevier, pp. 243–255.
  28. 28. Qin Chanchan, Zhang Guoping, Zhou Yicong, Tao Wenbing and Cao Zhiguo Integration of the saliency-based seed extraction and random walks for image segmentation. Neurocomputing. vol 129, 2014, Elsevier, pp. 378–391.
  29. 29. Zhi Xu-Hao and Shen Hong-Bin Saliency driven region-edge-based top down level set evolution reveals the asynchronous focus in image segmentation. Pattern Recognition., vol 80, 2018, Elsevier, pp. 241–255.
  30. 30. Fang Jiangxiong and Liu Huaxiang and Liu Jun and Zhou Haiying and Zhang Liting and Liu Hesheng, Fuzzy region-based active contour driven by global and local fitting energy for image segmentation, Applied Soft Computing, vol 100, 2021, Elsevier, pp. 106982.
  31. 31. Fang Jiangxiong and Liu Huaxiang and Zhang Liting and Liu Jun and Liu Hesheng. Region-edge-based active contours driven by hybrid and local fuzzy region-based energy for image segmentation, Information Sciences, vol 546, 2021, Elsevier, pp. 397–419.
  32. 32. Pratondo A., Chui C., Chee-Kong and S.H Ong. Robust edge-stop functions for edge-based active contour models in medical image segmentation. IEEE Signal Processing Letters., vol 23, 2015, IEEE, pp. 222–226.
  33. 33. Menze Bjoern H. and Jakab Andras and Bauer Stefan and Kalpathy-Cramer Jayashree and Farahani Keyvan and Kirby Justin et al. “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS),” in IEEE Transactions on Medical Imaging., vol 34, number = 10, IEEE, pp. 1993–2024
  34. 34. J. Zhao, Y. Zhang, Yichen, X. He, Xuehai and P. Xie. Covid-ct-dataset: a ct scan dataset about covid-19. arXiv preprint arXiv:2003.13865., vol 490, 2020.
  35. 35. The mini-MIAS database of mammograms, available: http://peipa.essex.ac.uk/info/mias.html, 2003.
  36. 36. Borji A, Ming-Ming C, Huaizu J and Jia L. Salient Object Detection: A Benchmark IEEE TIP., vol 24, 2015, IEEE, pp. 5706–5722. pmid:26452281
  37. 37. Ilunga-Mbuyamba Elisee and Avina–Cervantes Juan Gabriel and Garcia–Perez Arturo and de Jesus Romero–Troncoso Rene and Aguirre–Ramos Hugo and Cruz–Aceves Ivan et al. Localized active contour model with background intensity compensation applied on automatic MR brain tumor segmentation. Neurocomputing, vol 220, 2017, Elsevier, pp. 84–97.
  38. 38. Harrison K and Pullen H and Welsh C and Oktay O and Alvarez-Valle J and Jena R. Machine learning for auto-segmentation in radiotherapy planning, Clinical Oncology, vol 34, 2022, Elsevier, pp. 74–88. pmid:34996682
  39. 39. Aubert G, Kornprobst P. Mathematical problems in image processing: partial differential equations and the calculus of variations. Springer Science & Business Media. 2006(147).