Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

An efficient method to remove mixed Gaussian and random-valued impulse noise

Abstract

Mixed Gaussian and Random-valued impulse noise (RVIN) removal is still a big challenge in the field of image denoising. Existing denoising algorithms have defects in denoising performance and computational complexity. Based on the improved “detecting then filtering” strategy and the idea of inpainting, this paper proposes an efficient method to remove mixed Gaussian and RVIN. The proposed algorithm contains two phases: noise classification and noise removal. The noise classifier is based on Adaptive center-weighted median filter (ACWMF), three-sigma rule and extreme value processing. Different from the traditional “detecting then filtering” strategy, a preliminary RVIN removal step is added to the noise removal phase, which leads to three steps in this phase: preliminary RVIN removal, Gaussian noise removal and final RVIN removal. Firstly, RVIN is processed to obtain a noisy image approximately corrupted by Gaussian noise only. Subsequently, Gaussian noise is re-estimated and then denoised by Block Matching and 3D filtering method (BM3D). At last, the idea of inpainting is introduced to further remove RVIN. Extensive experimental results demonstrate that the proposed method outperforms quantitatively and visually to the state-of-the-art mixed Gaussian and RVIN removal methods. In addition, it greatly shortens the computation time.

Citation: Xing M, Gao G (2022) An efficient method to remove mixed Gaussian and random-valued impulse noise. PLoS ONE 17(3): e0264793. https://doi.org/10.1371/journal.pone.0264793

Editor: Qichun Zhang, University of Bradford, UNITED KINGDOM

Received: November 18, 2021; Accepted: February 16, 2022; Published: March 3, 2022

Copyright: © 2022 Xing, Gao. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting information files.

Funding: This work was supported by Chinese Universities Scientific Fund, [Grant No.2452016097].

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

Noise is everywhere in life, not only in the signal will have noise interference [1, 2], noise removal in the image is also a big challenge. Noise is an inevitable random phenomenon in images, which may degrades the visual quality of images, distorts its original content and burdens any preprocessing step that may be undertaken [3]. Meanwhile, the existence of the noise will also bring adverse effects to image analysis, recognition and interpretation in the later stage. In practice, the most common types of noise introduced into images can be considered as Gaussian noise and impulse noise [4]. Impulse noise can be further divided into salt and pepper noise (SPN) and random-valued impulse noise (RVIN), where RVIN is more general than SPN because SPN can be converted to RVIN in some cases. There are many studies on the single noise image denoising of Gaussian noise or RVIN, such as the methods in [5–9] for Gaussian noise removal and the algorithms in [10–14] for RVIN reduction. However, in most cases, the noise contained in natural images may not be single, it can be regarded as a mixture of Gaussian noise and impulse noise. Based on the generality of RVIN, this paper aims to study the noise removal of mixed Gaussian and RVIN.

Due to the totally different distributions of Gaussian noise and RVIN, applying the single noise removal methods directly to remove mixed noise always leads to undesirable results. Some important denoising methods for mixed Gaussian and RVIN noise removal based on “detecting then filtering” strategy [4] has been found in recent years [15–22]. “Detecting then filtering” is a strategy that detecting the impulse noise firstly, then abandoning them and using the remaining pixels to estimate the original noise-free image. In 2005, Roman Garnett et al. [15] proposed a trilateral filter (TF) based on this strategy to remove Gaussian and impulse noise simultaneously. They defined a statistic named rank-ordered absolute differences (ROAD) to detect impulse noise and further incorporated it into the bilateral filter (BF) for mixed removal. TF can remove mixed noise effectively. But when the noise level is a bit high, the denoised image is undesirable due to the lack of useful information in a local filtering neighbor. Inspired by the “detecting then filtering” strategy, a two-phase approach [16] for mixed Gaussian and impulse noise removal was proposed in 2008. First, it identifies the pixels that are likely to be corrupted by the impulse noise, and removes them from the dataset. Then, the image is deblurred and denoised simultaneously using essentially the noise-free data. However, the denoised image performs not very well because there are still some Gaussian noises in the image. In 2011, Li Bing et al. [17] put forward a new method which called non-local mixed noise filter (NLMNF) for mixed noise removal. Though it can remove Gaussian noise and RVIN simultaneously, it suffers from unstable results when the images are corrupted with a high level of mixed noise. In 2012, a method named ROR-NLM [18] was also presented based on “detecting then filtering” strategy. By using the statistic robust outlyingness ratio (ROR) to detect noise, the non-local means (NLM) filter [19] is extended to remove mixed noise. However, due to the use of an iterative framework, this algorithm will take a lot of time. Zhou et al. [20] proposed their method of restoring the image corrupted by mixed Gaussian and RVIN in 2013. It used a detector based on Bayesian theory to detect noises and then trained an adaptive overcomplete dictionary to get the final recovered image. However, this algorithm is also time consuming due to the training of the structured dictionary. In 2016, a new method named customized block-wise non-local means filter (CBNLMF) [21] was proposed. This filter is based on block-wise NLM. Unfortunately, this algorithm does not perform well for images with a lot of texture details. Later, Zhou et al. [21] proposed an image denoising algorithm combining CBNLMF and sparse representation (SR) technique to remove mixed Gaussian and RVIN. It used a CBNLMF-based detector to classify mixed noises and then removed them by SR. Nevertheless, this algorithm still needs some time to operate. Yamaguchi, T et al. [22] proposed a new mixed noise removal method in 2017, which utilizing Direction Weighted Median filter (DWMF) [23] and Block Matching and 3D filtering method (BM3D) [5]. This method leads to high performance of mixed noise removal. However, the denoising effect still needs improve, especially at high noise levels.

Due to the limitations of the methods mentioned above [15–22], it is essential to design an efficient method to deal with the mixed Gaussian and RVIN noise removal problem. Hence, this paper proposes a new algorithm based on improved “detecting then filtering” strategy and the idea of inpainting. It effectively addresses the mixed Gaussian and RVIN noise removal problem and greatly reduces the computation time. In general, the proposed denoising algorithm is divided into two phases: (i) noise classification phase and (ii) noise removal phase. In the noise classification phase, we put forward a novel mixed noise classifier which based on adaptive center weighted median filter (ACWMF) [24] and three-sigma rule [25]. In addition, the extreme values similar to impulse noise are also processed. Compared with classifiers in directional weighted median filter (DWMF) [23], 2013 Zhou’s [20] and 2016 Zhou’s [21], ours is fast and efficient. In the noise removal phase, different from the traditional “detecting then filtering” strategy, we add a preliminary RVIN removal step to this phase, which leads to three steps: (i) preliminary RVIN removal step, (ii) Gaussian noise removal step and (iii) final RVIN removal step. At the first step, the adaptive median filter (AMF) is used on those noisy pixels corrupted by RVIN for initial processing to obtain a noisy image approximately corrupted by Gaussian noise only. At the second step, the Gaussian noise level is re-estimated first and then the Gaussian noises, which are in the approximate Gaussian noise image obtained at the previous step, are removed by employing BM3D [5] algorithm. At the last step, the RVIN is removed for the second time based on the idea of inpainting. Extensive experimental results show that our proposed algorithm not only work well in mixed Gaussian and RVIN noise removal, but also has minimal computation time compared with NLMNF [17], CBNLMF [21], 2013 Zhou’s [20], 2016 Zhou’s [21] and 2017 Yam’s [22] methods.

The rest of the paper is organized as follows. Section 2 introduces the noise model of mixed Gaussian and RVIN. In Section 3, the proposed novel noise classification scheme for classifying mixed noise pixels is described. Then, the proposed noise removal scheme based on the improved “detecting then filtering” strategy and the idea of inpainting is presented in Section 4. The summary of the whole proposed denoising algorithm is given in Section 5. In Section 6, experimental results and discussions are shown in the noise classification phase and noise removal phase, respectively. Finally, the conclusion is drawn in Section 7.

2. Mixed Gaussian and RVIN noise model

The noise removal of a natural image corrupted by mixed Gaussian and RVIN remains a challenging problem in the field of image denoising. In general, images are destroyed first by Gaussian noise in the acquisition process, and then by impulse noise in the transmission process [3]. Therefore, the noise model of mixed Gaussian and RVIN, which denoted as Y, is expressed as follows: (1) where (i, j) (i = 1, …, M. j = 1, …, N.) denotes the location of a pixel in the whole image, Y(i, j) refers to the gray value of noisy image pixel, X(i, j) is the gray value of original noise-free image, R(i, j) shows gray value of uniformly distributed noise in range [Xmin, Xmax] while p0 is the probability of the RVIN, and n(i, j) stands for the noisy value which is drawn from a Gaussian distribution with zero mean and σ standard deviation.

In this paper, we use σ&p0 to represent the mixed noise level, where σ denotes the standard deviation of Gaussian noise and p0 refers to the probability of the RVIN. With the mixed noise model, the goal of our image denoising method is to restore the unknown true pixels from the pixels in Y which are corrupted by mixed Gaussian and RVIN. Since the distribution of mixed noise cannot be described by a fixed function, different noise pixels should be considered differently. Therefore, we first propose a noise classification algorithm to separate mixed noise pixels.

3. Proposed noise classification scheme

The purpose of the noise classification phase is to separate the additive Gaussian noise and the RVIN in the image, so as to further process the two kinds of noise separately. In our noise classification scheme, a good initial denoised image is obtained on the basis of ACWMF [24] firstly. Then, according to the feature of mixed noise, three-sigma rule is employed to get the classification result. Finally, the extreme pixel values of similar impulses are processed to achieve better classification. The proposed noise classifier is efficient and accurate. In Section 3.1, the acquisition of initial denoised image based on ACWMF is described. The proposed noise classification algorithm is generalized in Section 3.2.

3.1. Acquisition of initial denoised image based on ACWMF

The first step of our proposed noise classification scheme is to acquire the initial denoised image , where the superscript “ACWMF” stands for the adaptive center weighted median filter. Here, the ACWMF algorithm proposed in [24] is chosen to generate from Y. Motivated by the two advantages of ACWMF’s strong noise removal capability and less time consumption, it can ensure the high efficiency of our mixed noise classifier.

ACWMF devises a novel adaptive operator, which forms estimates based on the differences between the current pixels and the outputs of center-weighted median filter (CWMF) [24] with varied center weights. The details are as follows.

Consider a window W defined symmetrically around the image coordinates of the current pixel, which is described as: (2) where a, b are positive integer and the window size is defined as 2L + 1 (L > 0). Throughout the following discussion, unless otherwise stated, the window size is assumed to be 3 × 3 (i.e. a = b = 1, 2L + 1 = 9).

Let X(i, j) be the gray value of original noise-free image pixel at position (i, j). The gray value of mixed noise image pixel Y(i, j) is obtained from the mixed noise model. The output of CWMF [26], in which a weight adjustment is applied to the target noise image pixel Y(i, j) at position (i, j) within the sliding window in order to obtain the denoised image, can be described as: (3) where (4)

In the above equations, Y(i, j) is the gray value of current pixel (i, j) in mixed noise image, refers to the output of CWMF which represents the estimated gray value of original noiseless image X(i, j), ω = 2k + 1 denotes the center weight and k is the nonnegative integer. When ω = 1, the standard median filter which expressed as is obtained. Besides, operator ◊ represents the repetition operation and W is the window defined above.

Define dk as the differences between the output of CWMF and noise pixels, the differences dk is given by: (5) where k = 0, 1, …, L−1. It is easy to know that dk ≤ dk−1 (k ≥ 1) based upon the derivation shown in [27]. These differences provide information about the likelihood of corruption for the current pixel.

Consider four thresholds Tk (k = 0,1,2,3), which are described as: (6) where Tk < Tk−1 (k ≥ 1), and MAD is a robust estimate of dispersion [28], which is given by: (7) where is the output of standard median filter.

In this paper, parameters are set to [δ0, δ1, δ2, δ3] = [40, 25, 10, 5] and s = 0.6, which are selected following the suggestions in original literature [24]. From the simulation conducted on a broad variety of images, it has been observed that the selection above yields satisfactory results.

In a word, the ACWMF can be realized as follows: (8) where denotes the output of ACWMF, Tk refers to a set of thresholds (k = 0, 1, …, L − 1) where Tk < Tk−1 (k ≥ 1), and dk is the difference defined above.

Specifically, if dk < Tk, the pixels are regarded as noise and then treated with SMF. Otherwise, the current pixel remains unchanged. The algorithm of ACWMF is summarized in Algorithm 1.

Algorithm 1. The algorithm of ACWMF.

Input: a mixed noise image Y corrupted by mixed Gaussian and RVIN

Output: a denoised image

 Step1: Calculate dk value according to Formula (5);

 Step2: Calculate corresponding thresholds Tk according to Formulas (6) and (7);

 Step3: For every pixel, we estimate its true gray value based on Formula (8). After processing all pixels, the denoised image is obtained.

3.2. Proposed mixed noise classifier

Based on the output of ACWMF, an absolute difference image between the noisy image Y and denoised image is obtained: (9) Here abs(·) represents the operation of getting the absolute value for every element in a matrix.

Since is an initial estimation of the original noise-free image X, there are two types of values in Id: One is the absolute value of Gaussian noise and the other is the difference between impulse noise and the noise-free values. Three-sigma rule [25] is a classical statistical method through which a simple threshold can use to differentiate the noisy pixels. Let l be a label matrix corresponding to Y. Then, (10) where “1” represents the pixel corrupted by Gaussian noise, and “0” indicates that the pixel is distorted by RVIN.

For better classification, two kinds of extreme values are processed: the maximum and the minimum values of the mixed noise image, which can be recognized as impulse noise. It can be realized as follows: (11) where Ymax and Ymin represent the maximum and minimum value in the mixed noise image Y, respectively. The proposed novel mixed noise classifier is concluded in Algorithm 2.

Algorithm 2. The proposed mixed noise classifier.

Input: a mixed noise image Y corrupted by mixed Gaussian and RVIN

Output: a label matrix l labeling the type of every pixel in Y

 Step1: Operate ACWMF filter for Y as Algorithm 1. and an initial denoised image is generated;

 Step2: Construct an absolute difference image Id by ;

 Step3: Generate a label matrix l according to Formulas (10) and (11).

4. Proposed noise removal scheme

After the noise classification phase, we separate the Gaussian noise and RVIN effectively. Then, a noise removal scheme is needed to remove the mixed noise to restore the image. To improve the “denoising then filtering” strategy, we added an initial step which named preliminary RVIN removal in the denoising stage. Therefore, the noise removal phase comprises three steps: (i) preliminary RVIN removal, (ii) Gaussian noise removal, and (iii) final RVIN removal.

4.1. Preliminary RVIN removal

Since the RVIN only destroys part of the pixels in the image while Gaussian noise destroys all the pixels, it is inevitably to use the Gaussian noise in the surrounding neighborhood to estimate the original pixel when removing the RVIN. So that there is still an error in the estimation. Based on the above analysis, the first step that we perform in the noise removal phase is the preliminary RVIN removal. Since the impulse noise destroys the pixel information, the purpose of the preliminary RVIN removal step is to provide a pixel value containing image information to the position of the impulse, which facilitates the Gaussian noise removal in the next step. In the case of mixed Gaussian and RVIN, it is important to first suppress the RVIN [29], so an impulse removal filter which is robust to Gaussian noise is need to use. Therefore, the method applied in the preliminary RVIN removal step needs to meet the following two conditions:

  1. The algorithm is robust to the existence of Gaussian noise;
  2. The algorithm is simple with fast speed.

Based on the above two conditions, the adaptive median filter (AMF) algorithm mentioned in Fig 1 is chosen for preliminary RVIN removal. At this time, the pixel value at the RVIN position is estimated by using pixels in its neighborhoods, which can be approximately regarded as Gaussian noise, and then it can be removed as Gaussian noise in the following steps.

thumbnail
Download:
Fig 1. The flowchart of AMF algorithm.

https://doi.org/10.1371/journal.pone.0264793.g001

AMF algorithm works with flow A and flow B, the flowchart is in Fig 1:

where W(i, j) is a window centered at (i, j) with (2a + 1)×(2b + 1) as window size. Zmed is the median of the grayscale value in the window W(i, j). In the same way, we define Zmax as the maximum and Zmin as the minimum, Z(i, j) represents the grayscale value at coordinate (i, j), Wmax denotes the maximum allowed window in W(i, j). In this paper, we apply initial window as W3×3 (i.e. a = b = 1) and Wmax as W5×5 (i.e. a = b = 2) which are selected based on a broad variety of images simulation. Then, the output of AMF is named as .

In a word, the proposed preliminary RVIN removal steps can be realized as follows: (12)

The pixels identified as RVIN in the noise classification phase are processed by AMF algorithm in Fig 1, while the pixels classified into Gaussian noise remain unchanged. Through this step, we obtain a noisy image approximately corrupted only by Gaussian noise through preliminary processing of RVIN, which named as approximate Gaussian noisy image.

4.2. Gaussian noise removal

As the second step of noise removal phase, Gaussian noise removal is also very important. Due to the fact that Gaussian noise destroys all pixels in the image, it is a great challenge to completely remove Gaussian noise. The proposed Gaussian noise removal step is further divided into two small steps: (i) Gaussian noise level re-estimation and (ii) Gaussian noise removal by BM3D.

4.2.1. Gaussian noise level re-estimation.

Since the approximate Gaussian noisy image is obtained through the preliminary RVIN removal step, in which the RVIN after preliminary processing is approximately regarded as Gaussian noise, it is necessary to re-estimate the Gaussian noise level.

Here, the Gaussian noise level estimation method proposed in [30] is applied to re-estimate the Gaussian noise level of the approximate Gaussian noisy image obtained in the preliminary RVIN removal step. The method is described as follows.

When re-estimating Gaussian noise level, it is essential to re-estimate standard deviation σ of the Gaussian noise. So first divide the whole approximate Gaussian noisy image into four sub-images(sub-groups), and the sample variances of these four sub-groups are calculated respectively. Then, the estimation of Gaussian noise variance is re-estimated by averaging all the sample variances: (13) where M×N refers to the size of approximate Gaussian noisy image and s(i, j)2 represents the sample variances of sub-groups. At last, the estimation of the standard deviation we need is obtained by taking the square root of the . For more detailed information about this Gaussian noise level estimation method, please refer to the literature [30].

4.2.2. Gaussian noise removal by BM3D.

After re-estimating the Gaussian noise level, the BM3D algorithm proposed in [5] is used to remove Gaussian noise in this step. BM3D algorithm is a filtering algorithm based on 3-D transform and block-wise estimation. It consists of two steps: (i) basic estimate and (ii) final estimate. In the first step, the noisy image to be processed is divided into fixed size sub-groups and each block of image is estimated one by one. Blocks are grouped according to how similar they are to the currently processed one and then, they are stacked together in a 3-D array(group). Then, the collaborative hard-thresholding is performed to the formed 3D group and finally the basic estimate of the true original image from the overlapping blocks is computed by aggregation. The second step is using the basic estimate, performing improved grouping and collaborative wiener filter. Above all, use the basic estimate obtained in the first step to estimate each block for the second time. Through blocking matching (BM), the locations of the blocks similarly to the currently processed one can be found. After BM, two 3-D arrays (groups) are obtained: one is from the noisy image and the other is from the basic estimate. Then, perform collaborative Wiener filter on the two 3-D arrays mentioned above. At last, the final estimate is computed by the weighted average of the overlapping block estimations which obtained by aggregation. More algorithm details are presented in S1 File. The output of the BM3D algorithm is named as and the flowchart of this algorithm is shown in Fig 2:

thumbnail
Download:
Fig 2. The flowchart of BM3D algorithm.

https://doi.org/10.1371/journal.pone.0264793.g002

4.3. Final RVIN removal

The third step of the noise removal phase is the final RVIN noise removal step. It is assumed that all the Gaussian noises in the image have been removed after the Gaussian noise removal step. Obviously, the location of the RVIN is already known in the noise classification phase. Hence, the final RVIN noise removal step only deals with the pixels at the RVIN position, while the pixels in the neighbors used for estimating the original pixel are all regarded as clean pixels. Pursuing the denoising performance, the idea of image inpainting is introduced into this step to remove RVIN. The main idea of image inpainting is to use the observed information to reconstruct the damaged or obscured area [31]. In this step, by cleverly combining the idea of inpainting, the processing of damaged areas is transformed into the processing of the points of RVIN. The inpainting method based on mixed median [32] is chosen to modify and perform here. The details are in Algorithm 3.

Algorithm 3. The image inpainting algorithm based on the mixed median.

Input: & label matrix l

Output:

 Step1: Initialize the set of RVIN pixels . For every RVIN pixel in Y(RVIN), if there is at least one noise-free pixel in W3×3 (the window of 3×3 size), eliminate RVIN pixels from W3×3 and compute the median of the remaining pixels, then update Y(RVIN). Else if do this process in W7×7 window as the same.

 Step2: Initialize a dynamic window whose size starts from 3×3 and increases by d = 1. For every RVIN pixel in Y(RVIN) obtained from the step1, when there is at least one noise-free pixel in the considered dynamic window, compute the maximum repetitive pixels values, and evaluate the median of it, then update Y(RVIN). End this step until d > dmax, where dmax = (min{M, N} − 1)/2. Finally, the is obtained.

In a word, the proposed final RVIN removal step can be realized as follows: (14)

The pixels identified as RVIN in the noise classification phase are processed by inpainting method in Algorithm 3, while the remaining pixels remain unchanged at the last step. Through this step, the final denoised image is obtained.

5. Summary of the proposed denoising scheme

In general, the proposed denoising algorithm is divided into two phases: (i) noise classification phase and (ii) noise removal phase. The proposed novel mixed noise classifier is based on ACWMF, three-sigma rules and extreme value processing. Based on the improved “detecting then filtering” strategy, the noise removal phase contains three steps: (i) preliminary RVIN removal step, (ii) Gaussian noise removal step, and (iii) final RVIN removal step. The idea of inpainting is introduced to the final RVIN removal step. The flowchart of the whole proposed denoising algorithm is in Fig 3.

thumbnail
Download:
Fig 3. The proposed algorithm flowchart.

https://doi.org/10.1371/journal.pone.0264793.g003

6. Experimental results and discussion

To assess the capability of the proposed algorithm for mixed Gaussian and RVIN noise removal, comprehensive numerical results and visual quality are compared with other state-of-the-art methods. Now, the proposed algorithm is tested from two perspectives. One is the performance of noise classification in Section 6.1, the other is the image denoising performance in Section 6.2.

6.1. Comparison of noise classification

For good performance, the capability of noise classification is exceedingly important. The performance of the proposed noise classifier is compared with three state-of-the-art ones coping with mixed Gaussian and RVIN. The compared methods are DWMF [23], 2013 Zhou’s [20] and 2016 Zhou’s [21].

In this section, four noise classification evaluation indexes are introduced. They are Miss_error [21], False_error [21], Total_error [21] and Computation time. Miss_error is the number of pixels distorted by RVIN but wrongly classified into Gaussian noise. False_error counts the number of pixels corrupted by Gaussian noise but wrongly identified as RVIN. Total_error equals to Miss_error plus False_error. Computation time refers to the running time of the algorithm. By considering these four evaluation indexes in a comprehensive way, if Total_error and computation time of the proposed classification algorithm are the minimum among these comparison methods mentioned above, we have the confidence to employ it into our whole denoising algorithm.

To test our classifier in the noise classification phase, three standard test images: “House”, “Boat” and “Barbara” are chosen. The selected images are shown in Fig 4, they are 512 × 512 size which have a dynamic range of pixels varying from 0 to 255. It is reasonable to select these three images as the test images for they contain different level of edges and textures.

thumbnail
Download:
Fig 4. Test images used in this paper.

https://doi.org/10.1371/journal.pone.0264793.g004

In Tables 1–3, the results of the noise classification evaluation indexes for noisy “House”, “Boat” and “Barbara” images are presented. These noisy images are corrupted with different mixed noise levels σ = 15, 20, 25, 30 & p0 = 0.2, 0.3. The simulated additive Gaussian noise is with zero mean and four standard deviations σ = 15, 20, 25, 30. Meanwhile, the noise densities p0 of RVIN are 0.2 and 0.3. The bold numbers in the tables are the smallest ones in the related columns.

thumbnail
Download:
Table 1. Comparison of classifiers for noisy “House” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t001

thumbnail
Download:
Table 2. Comparison of classifiers for noisy “Boat” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t002

thumbnail
Download:
Table 3. Comparison of classifiers for noisy “Barbara” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t003

From the data in Tables 1–3, it is obvious that our proposed novel mixed noise classifier can guarantee the lowest classification errors which defined as Total_error and the shortest computation time at almost all the noise level, even when the noise level is as high as σ = 30 & p0 = 0.3. So, there is no doubt that the proposed noise classifier is efficient and accurate. The good noise classification performance benefits from the good initial denoised image obtained by ACWMF, classical statistical principles of three-sigma rules and the treatment of extreme values. Hence, we have the confidence to employ the proposed noise classification scheme into our whole denoising algorithm.

6.2. Comparison of noise removal

6.2.1. Image quality metrics introduction.

For quantitative performance evaluation, two image quality metrics are employed. They are: peak signal to noise ratio (PSNR) and structural similarity index (SSIM).

PSNR is one of the most classical metrics for image quality assessment (IQA). It is computed as the ratio of the peak intensity value of the reference image to the RMS reconstruction error relative to the reference. Also, its values are usually given in logarithmic decibel units(dB). PSNR provides a global statistical similarity over intensity distribution [9]. PSNR is used as quantitative measurement to compare the proposed method with different well-known techniques. The PSNR is defined as: (15) where MSE is mean squared error and is defined as: (16)

In the above equations, X is the original noise-free image while is the output denoised image, and M × N is the size of the image. The larger the PSNR value is, the better the denoising effect is.

SSIM [33] is based on the computation of three terms, which named as the luminance term, the contrast term and the structural term. The overall index is a multiplicative combination of the three terms. The SSIM is defined as: (17) (18) (19) (20) where μx, μy, σx, σy and σxy are the local means, standard deviations, and cross-covariance for images x, y. If α = β = γ = 1, and C3 = C2 / 2, the SSIM simplifies to: (21)

Substitute the images x, y with the original clean image X and the output denoised image when calculating SSIM. Similarly, the higher the value of SSIM is, the better the denoising effect of the algorithm is.

6.2.2. Numerical results and visual quality.

The denoising performance of the proposed method is tested on three test images mentioned above in Fig 4. Consistent with the noise classification phase, we add different mixed noise level σ = 15, 20, 25, 30 & p0 = 0.2, 0.3 to the above three test images in order to obtain mixed noise images. The performance of our whole denoising algorithm is shown on numerical results and visual quality respectively. To validate the superiority of the proposed method, its performance is compared in terms of PSNR, SSIM, computation time and visual quality of the denoised images using the various methods available in literature such as NLMNF [17], CBNLMF [21], 2013 Zhou’s [20], 2016 Zhou’s [21] and 2017 Yam’s [22]. The parameters in these algorithms are set to values suggested by the authors.

All algorithms have been implemented using MATLAB 2019a (The MathWorks, Inc., Natick, MA, USA). Also, data processing was carried out on a Dell workstation system with a 2.80 GHz Intel Core i7 processor and 16 GB memory.

PSNR and SSIM comparisons of the proposed method and other baseline algorithms on test images corrupted by mixed Gaussian and RVIN with different noise levels are tabulated in Tables 4–6, respectively.

thumbnail
Download:
Table 4. Comparison of PSNR and SSIM for noisy “House” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t004

thumbnail
Download:
Table 5. Comparison of PSNR and SSIM for noisy “Boat” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t005

thumbnail
Download:
Table 6. Comparison of PSNR and SSIM for noisy “Barbara” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t006

The bold numbers in Tables 4–6 are the best values in the related columns. It is clear from the tables that the proposed denoising algorithm always generate the highest PSNR and SSIM. So, it can demonstrate that the proposed method can generate the best denoised results whatever noisy image is and whatever noise level is. Especially in “Barbara” with the most detailed textures, NLMNF, CBNLMF, 2013 Zhou’s, 2016 Zhou’s and 2017 Yam’s do not perform well, but ours still get best results in the values of PSNR and SSIM.

Now, we discuss the time complexity of our whole mixed noise algorithm. As shown in Tables 7–9, five other related algorithms are chosen for comparison. Here, we operate the MATLAB codes of the above six algorithms on the same platform- MATLAB 2019a. The computer is equipped with Dell workstation system with a 2.80 GHz Intel Core i7 processor and 16 GB memory. The data in Tables 7–9 is the CPU consuming time of each algorithm. The bold numbers are the smallest time in the related columns. Obviously, with the help of improved “detecting then filtering” strategy and the idea of inpainting, the proposed method consumes the least time among the six algorithms. In other words, our algorithm obtains the best denoising performance in shortest time.

thumbnail
Download:
Table 7. Comparison of Time (seconds) for noisy “House” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t007

thumbnail
Download:
Table 8. Comparison of Time (seconds) for noisy “Boat” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t008

thumbnail
Download:
Table 9. Comparison of Time (seconds) for noisy “Barbara” with σ = 15,20,25,30&p0 = 0.2,0.3.

https://doi.org/10.1371/journal.pone.0264793.t009

In order to have a close-up observation to the noise removal images, some typical results are chosen to show. They are denoised “House”, “Boat” and “Barbara” with the mixed noise level of σ = 20 & p0 = 0.2. The denoised images compared with NLMNF [17], CBNLMF [21], 2013 Zhou’s [20], 2016 Zhou’s [21] and 2017 Yam’s [22] are shown in Figs 5–7.

thumbnail
Download:
Fig 5. Results of different algorithm in denoised “House” with σ = 20&p0 = 0.2: (a) Mixed noise image; (b) NLMNF; (c) CBNLMF; (d) 2013 Zhou’s; (e) 2016 Zhou’s; (f) 2017 Yam’s; (g) Proposed; (h) Original image.

https://doi.org/10.1371/journal.pone.0264793.g005

thumbnail
Download:
Fig 6. Results of different algorithm in denoised “Boat” with σ = 20&p0 = 0.2: (a) Mixed noise image; (b) NLMNF; (c) CBNLMF; (d) 2013 Zhou’s; (e) 2016 Zhou’s; (f) 2017 Yam’s; (g) Proposed; (h) Original image.

https://doi.org/10.1371/journal.pone.0264793.g006

thumbnail
Download:
Fig 7. Results of different algorithm in denoised “Barbara” with σ = 20&p0 = 0.2: (a) Mixed noise image; (b) NLMNF; (c) CBNLMF; (d) 2013 Zhou’s; (e) 2016 Zhou’s; (f) 2017 Yam’s; (g) Proposed; (h) Original image.

https://doi.org/10.1371/journal.pone.0264793.g007

Fig 5 shows the comparisons on visual results from the test cases on “House”. It can be clearly observed that there are more fine structures remained in the “House” of the proposed denoising algorithm shown in Fig 5e, especially some of the textures on the exterior walls. However, the “House” images denoised by NLMNF, CBNLMF, 2013 Zhou’s, 2016 Zhou’s and 2017 Yam’s which shown in Fig 5b–5f respectively do not perform well. The denoised images obtained by CBNLMF and 2016 Zhou’s are distorted a lot. 2017 Yam’s owns fast speed but the image after denoising is a bit blurry. Furthermore, there are still some noises in these five “House” images.

Similar conclusions can be drawn from Figs 6 and 7, the texture details and edge structure of the denoised image obtained by the algorithm in this paper are clearer and the visibility of our denoised result is better. For instance, the boat and wave in Fig 6g is more clearer than that in Fig 6b–6f. Especially in Fig 7, although there are most texture details in “Barbara” image, the proposed algorithm in Fig 7g still preserves these textures well.

7. Conclusion

In this paper, we propose an efficient method for mixed Gaussian and RVIN noise removal. Experimental results verify that the proposed algorithm performs well not only in noise classification phase but also in noise removal phase compared with several state-of-the-art methods. The proposed mixed noise removal method has achieved good denoising performance both in numerical results and visual quality, and greatly reduced the computation time.

Supporting information

S1 File. The BM3D algorithm in details.

https://doi.org/10.1371/journal.pone.0264793.s001

(PDF)

S1 Table. Abbreviation comparison.

https://doi.org/10.1371/journal.pone.0264793.s002

(PDF)

References

  1. 1. Zhang Q, Wang H. A Novel Data-based Stochastic Distribution Control for Non-Gaussian Stochastic Systems. IEEE Transactions on Automatic Control. 2021.
  2. 2. Yin X, Zhang Q., Wang H. and Ding Z. Rbfnn-based minimum entropy filtering for a class of stochastic nonlinear systems. IEEE Transactions on Automatic Control. 2019;65(1):pp.376–381.
  3. 3. Mafi M, Martin H, Cabrerizo M, Andrian J, Barreto A, Adjouadi M. A comprehensive survey on impulse and Gaussian denoising filters for digital images. Signal Processing. 2019;157:236–260.
  4. 4. Chen LL, Licheng L, Philip Chen, C. L. A robust bi-sparsity model with non-local regularization for mixed noise reduction. Information Sciences. 2016;354:101–111.
  5. 5. Dabov K, Foi A, Katkovnik V, Egiazarian K. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process. 2007;16(8):2080–2095. pmid:17688213
  6. 6. Shao L, Yan R, Li X, Liu Y. From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms. IEEE Trans Cybern. 2014;44(7):1001–1013. pmid:24002014
  7. 7. Guo Q, Zhang C, Zhang Y, Liu H. An Efficient SVD-Based Method for Image Denoising. IEEE Transactions on Circuits and Systems for Video Technology. 2016;26(5):868–880.
  8. 8. Rakhshanfar M, Amer MA. Efficient cascading of multi-domain image Gaussian noise filters. Journal of Real-Time Image Processing. 2019;17(5):1183–1195.
  9. 9. Wu G, Luo S, Yang Z. Optimal weighted bilateral filter with dual-range kernel for Gaussian noise removal. IET Image Processing. 2020;14(9):1840–1850.
  10. 10. Wu TCaHR. Space variant median filters for the restoration of impulse noise corrupted images. IEEE Transactions on Circuits and Systems for Video Technology. 2001:784–789.
  11. 11. Schulte S, De Witte V, Nachtegael M, Van der Weken D, Kerre EE. Fuzzy random impulse noise reduction method. Fuzzy Sets and Systems. 2007;158(3):270–283.
  12. 12. Turkmen I. A new method to remove random-valued impulse noise in images. AEU—International Journal of Electronics and Communications. 2013;67(9):771–779.
  13. 13. Gupta V, Chaurasia V, Shandilya M. Random-valued impulse noise removal using adaptive dual threshold median filter. Journal of Visual Communication and Image Representation. 2015;26:296–304.
  14. 14. Nadeem M, Hussain A, Munir A, Habib M, Naseem MT. Removal of random valued impulse noise from grayscale images using quadrant based spatially adaptive fuzzy filter. Signal Processing. 2020;169.
  15. 15. Garnett R., Huegerich T., Chui C., He W. A universal noise removal algorithm with an impulse detector. IEEE TRANSACTIONS ON IMAGE PROCESSING. 2005;14(11):1747–1754. pmid:16279175
  16. 16. Cai JF, Chan RH, Nikolova M. Two-phase approach for deblurring images corrupted by impulse plus Gaussian noise. Inverse Probl Imaging. 2008;2(2):187–204.
  17. 17. Li B, QuanSheng L, Xu JW, Luo XJ. A new method for removing mixed noises. Science China Information Sciences. 2011;54(1):51–59.
  18. 18. Xiong B, Yin Z. A universal denoising framework with a new impulse detector and nonlocal means. IEEE Trans Image Process. 2012;21(4):1663–1675. pmid:22020688
  19. 19. Buades A, Bartomeu C, Morel J.M. A Review of Image Denoising Algorithms, with a New One. Multiscale Modeling & Simulation. 2005;4(2):490–530.
  20. 20. Zhou Y, Ye Z, Xiao Y. A restoration algorithm for images contaminated by mixed Gaussian plus random-valued impulse noise. Journal of Visual Communication and Image Representation. 2013;24(3):283–294.
  21. 21. Yingyue Zhou M L, Su Xu, Hongbin Zang, Hongsen He, Qiang Li, Jin Guo. An image denoising algorithm for mixed noise combining nonlocal means filter and sparse representation technique. Journal of Visual Communication and Image Representation. 2016;41:74–86.
  22. 22. Yamaguchi T, Suzuki A, Ikehara M. Detail Preserving Mixed Noise Removal by DWM Filter and BM3D. IEICE Trans Fundam Electron Commun Comput Sci. 2017;E100A(11):2451–2457.
  23. 23. Dong Y, Xu S. A New Directional Weighted Median Filter for Removal of Random-Valued Impulse Noise. IEEE Signal Processing Letters. 2007;14(3):193–196.
  24. 24. Wu TCaHR. Adaptive impulse detection using center-weighted median filters. IEEE SIGNAL PROCESSING LETTERS. 2001;8.
  25. 25. Maronna R M R., and Yohar V. Robust Statistics: Theory and Methods. Chichester, UK:Wiley. 2006.
    • 26. Lee Y.-H., Ko S.-J. Center weighted median filters and their applications to image enhancement. IEEE TransCircuits Syst. 1991;38:984–993.
    • 27. Chen T., Ma K.-K., and Chen L.-H. Tri-state median filter for image denoising. IEEE TransImage Processing. 1999;8:1834–1838. pmid:18267461
    • 28. Hampel F.R., Ronchetti E.M., Rousseeuw P.J., and Stahel W.A. Robust Statistics: The Approach Based on Influence Functions. New York:Wiley. 1986.
      • 29. Astola SMAKOEDZGJT. Combining the discrete wavelet transforms and rank-order based filters for image restoration. Optical Engineering. 1998.
        • 30. Li M, Ghosal S. Bayesian Multiscale Smoothing of Gaussian Noised Images. Bayesian Analysis. 2014;9(3):733–758.
        • 31. Wan W, Liu J. Nonlocal patches based Gaussian mixture model for image inpainting. Applied Mathematical Modelling. 2020;87:317–331.
        • 32. D. N. H. Thanh, V. B. S. Prasath, L. M. Hieu and H. Kawanaka, “Image Inpainting Method Based on Mixed Median,” 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), 2019, pp. 24–29.
          • 33. Zhou W, Bovik A. C., Sheikh H. R., and Simoncelli E. P. Image Qualifty Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing. 2004;13 (4):600–612. pmid:15376593