Figures
Abstract
The basic principle of nonlocal means is to denoise a pixel using the weighted average of the neighbourhood pixels, while the weight is decided by the similarity of these pixels. The key issue of the nonlocal means method is how to select similar patches and design the weight of them. There are two main contributions of this paper: The first contribution is that we use two images to denoise the pixel. These two noised images are with the same noise deviation. Instead of using only one image, we calculate the weight from two noised images. After the first denoising process, we get a pre-denoised image and a residual image. The second contribution is combining the nonlocal property between residual image and pre-denoised image. The improved nonlocal means method pays more attention on the similarity than the original one, which turns out to be very effective in eliminating gaussian noise. Experimental results with simulated data are provided.
Citation: Wang X, Wang H, Yang J, Zhang Y (2016) A New Method for Nonlocal Means Image Denoising Using Multiple Images. PLoS ONE 11(7): e0158664. https://doi.org/10.1371/journal.pone.0158664
Editor: Juri G. Gelovani, Wayne State University, UNITED STATES
Received: October 23, 2015; Accepted: June 20, 2016; Published: July 26, 2016
Copyright: © 2016 Wang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data (images) have been uploaded to the following link:https://figshare.com/articles/A_New_Method_for_Nonlocal_Means_Image_Denoising_Using_Multiple_Image/3384115.
Funding: This work is partially supported by the NSFC fund (61471213, 61571259, 61531014), Shenzhen Fundamental Research fund (JCYJ20140509172959961, JCYJ20160331185006518), Natural Science Foundation of Guangdong Province(2015A030310173) and National High-tech R&D Program of China (863 Program, 2015AA015901).
Competing interests: The authors have declared that no competing interests exist.
Introduction
Image denoising [1–10] is a low-level image processing tool, but it’s an important preprocessing tool for high-level vision tasks such as object recognition [11,12], image segmentation and remote sensing imaging. In the practical imaging system, there exists different kinds of noise. This paper only focus on the zero mean additive gaussian noise, which can be formulated as:
(1.1)
yi is the i-th patch of the observed image y, xi is the original patch with the i-th pixel at it’s center, and ηi is the independent additive gaussian white noise. The gaussian term can be used to simulate the noise generated by the intrinsic thermal or electronic fluctuations of the acquisition devices [13].
In image denoising, an image is often divided into many small patches which are repeatedly appear. We can remove the noise by taking advantages of the redundant information of patches while preserving the slight structure of images at the same time. By utilizing the redundant patches, the nonlocal means (NLM) image denoising method [14] could achieve impressive performance which be regarded as the most popular denoising method. The basic principle of nonlocal means is to denoise a pixel by averaging its local neighborhood pixels with the clues of similarities of the redundant patches. It has shown to be an effective image denoising technique. But the definition of similarities between the patch of the noisy pixel and its spatially local neighborhood patches in NLM is not strict, it’s just calculated by a block matching process.
Many researches devote to reduce the computational complexity [15,16] of NLM method. To achieve this, one important issue is to define and select similar patches, and many researchers have been focused on modifying the block matching process hence to find the true similar patches. In [17] and [18], a moment-based and rotation invariant feature representations have been applied for robust block matching by Ji et al and Zimmer et al respectively. Both of Kervrann et al [19] and Thaipanich et al [20] utilized the width of local window to represent different kinds of texture patterns, such as smooth and edge regions. The work in [21] analyze a rotationally invariant similarity for nonlocal image denoising. Among these researches, it is easy to see that finding ‘true similar’ patches are the critical part in patch-based denoising algorithms. But what is the ‘true’ similarity is not easy to define.
Since the accuracy of similarity computation is clearly affected by how rich the priori information of noised images is, we hereby apply multiple noised images hence to improve the performance of NLM method. First, we propose to use two noisy images to denoise pixels. The input to our system is two different noisy images but with the same noise deviation. Instead of only one image, we calculate the weight from two noisy images. Another important issue is how to design the weight of the similar patches. In the original nonlocal methods, every block within the scope of the search will be assigned a weight. Although the weight is very small, large numbers will reduce the denoising performance. So we redesign the similarity weight which rules out blocks with large difference. Third, the residual image implies a lot of edge information. We combine the nonlocal information of residual image and pre-denoised image to improve the denoising performance.
Nonlocal Means Denoising
The basic principle of the nonlocal means denoising is to replace the noisy gray-value I(i) of pixel i with a weighted average of the gray-values of all the pixels on the image. Because it needs too much computation, it is more practical to average the pixels in a smaller scope. We denote the pixel to be denoised by i and the pixels in the neighborhood of i by j and use them to denoise i. The estimated value for a pixel i is computed based on the weighted average of the neighborhood pixels j around pixel i:
(1.2)
Where
is the search window of size (2n+1)*(2n+1) centered at i and wij is the weight of two pixels i and j which is calculated depending on the similarity of their patches and is defined as:
(1.3)
where Zi is a normalizing term,
, and h acts as a filtering parameter. It controls the decay of the weights as a function of the Euclidean distances. This parameter is typically adjusted manually in the algorithm.
Fig 1A shows the original nonlocal means denoising result using the wij under the noise standard deviation δ = 20, and Fig 1B is the corresponding method noise, which appears obvious image structures that are similar with those in Fig 1A. The method noise here is defined as
(1.4)
there exists some edge information in the residual image.
A denoised image. B residual image.
Improved Nonlocal Means
In this section, we will introduce the improved methods. In the first step we utilize the nonlocal information in two noisy images to improve the accuracy of similarity calculation. In the way, we can acquire the smoothed pre-denoised image and the residual image. In the second denoising step, we get the final denoised image by combining the nonlocal information in pre-denoised image and residual image. The flow chart is showed in Fig 2.
Multi-Image
The nonlocal means algorithm utilizes similar blocks to denoise the point. In the original algorithm, polluted pixels are recovered from weighted average of the similar pixels. Many researches devoted to find the ‘true’ similar pixels. In this work, we try to solve this problem in another way, we calculate the weight from two noisy images but with same noise deviation. We add noise to I, the original clean image, and get Fig 3A and 3B, two different noisy images with same noise deviation, which is denoted by I1 and I2. In the real world, we could get such two images using the very fast camera with the continuous capture function.
A Noisy Image 1. B Noisy Image 2.
We denote the pixel to be denoised in image I1 by i1 and the neighborhood pixels of i1 by j1. And we denote by i2 the pixel with the same position as i1 in I2 and the neighborhood pixels of i2 by j2. So the new weight between i1 and j1 is calculated by
(1.5)
where
is a normalizing term,
, h1 and h2 act as filtering parameters.
Redesigning the weight
In the original nonlocal methods, every block within the search scope would be assigned a weight. Although the weights of the unsimilar blocks are very small, the large number of them will reduce the denoising performance. We redesign the similarity weight which rules out blocks with large difference in the same time. We denote the low limit by ϵ1 and the up limit by ϵ2. If is less than ϵ1, the similarity of two blocks is small, so we set the weight
as 0 to rule out bad pixels. The redesigned weight is as follow:
(1.6)
where
is the normalizing term,
, With the new weight, we can calculate the estimated value of pixel i1 by
(1.7)
Method noise
In this subsection, we will show how to approximately estimate the similarities between different patches using residual image and pre-denoised image, and thus obtain the new weight. In the first denoising process, we got one residual image, shown in Fig 4B. In order to improve the accuracy of similarity calculation, we combine the similarity of the pre-denoised image and the residual image. According to the observation, we find that the residual image only provides abundant information in edge area. However, the flat area is largely covered by noise, without any useful information. So we should calculate the gradient image Fig 4A to determine the usage of residual nonlocal properties.
We denote the pixel in the pre-denoised image by ip, and denote the neighborhood pixels of ip by jp. We then denote the pixel with same position as ip in the residual image by im, and denote the neighborhood pixels of i2 by jm. So the new weight between pixel i1 and j1 is calculated by
(1.8)
where Zi is a normalizing term,
, hp is used to control the attenuation of weight of pre-denoising image, and hm is used to control the attenuation of weight of residual image. hp and hm will be adjusted in the experiment.
Experiments
To validate the effectiveness of our method, we have used standard images such as Barbara, Boat, Fingerprint, House, Lena, Baboon with an additive white Gaussian noise (δn = 20 and δn = 10). We compare our approach to the well-known filtering techniques such as the bilateral filter [22], the traditional nonlocal means [14], the total variation [23] and the anisotropic diffusion [24]. The parameters of the methods should be tuned to get a good balance between texture preservation and noise suppression, measured by the peak signal noise ratio(PSNR). The parameters in our method for all experiments are as follows: δ = δn and a window size of 5*5.
In image denoising, the peak signal to noise ratio is often considered as the objective criteria. Given a noise-free m×n monochrome image I and its noisy approximation K, MSE and PSNR is defined as:
(1.9)
(1.10)
Another image quality assessment index is the structural similarity (SSIM) index, which is based on the computation of three terms, namely the luminance term, the contrast term and the structural term, the overall index is a multiplicative combination of the three terms.
(1.11)
Where
,
,
, and μx, μy, σx, σy and σxy are the local means, standard deviations, and cross-covariance for images.
We show the PSNR and SSIM by different methods in Table 1, and it confirms that our method has better performances than the other methods. We can find that the proposed algorithm can improve 0.8dB than the original nonlocal means.
In Fig 5, we compared the denoising performance of 6 images, the above row is the result using the origin nonlocal means, the following row is result using the improved method.
(a) Barbara. (b)Boat. (c) Fingerprint. (d) Lena. (e) man.
In Fig 6, we show the residual of four methods, gaussian filter(a), bilateral filter(b), the origin nonlocal(c) and the improved nonlocal means(d). Although some image structure is still visible in the residual image of our method, the amount of remaining structure is much less than those using gaussian, bilateral and origin NLM.
(a) gaussian. (b) bilateral. (c) original NLM. (d) improved NLM.
Conclusion
In this paper we redesign the weight in nonlocal image denoising. There are two main contributions of this paper. In the first denoising process, we add a noisy image of which the noise deviation is equal to that of the original noisy image. It can improve the accuracy of finding similar blocks by using the nonlocal property of two images. At the same time, we rule out the smaller weight blocks, thus reduce the interference of unsimilar blocks. In the second denoising step, it can further improve the accuracy by using the nonlocal similarity of the residual image. The experimental results show that the proposed algorithm can contribute an improvement of about 0.8 dB than the result of the original nonlocal means. As the speed of camera becoming faster, we will consider using multiple images in the camera denoising technology in the later study. It is also a good research direction to find how to better utilize the edge information in the residual image. However, since our methods need multiple images with the same noise deviation, the implementation of the image acquisition and the processing speed are limited and need to be further improved.
Acknowledgments
This work is partially supported by the NSFC fund (61471213, 61571259, 61531014), Shenzhen Fundamental Research fund (JCYJ20140509172959961, JCYJ20160331185006518),Natural Science Foundation of Guangdong Province(2015A030310173) and National High-tech R&D Program of China (863 Program,2015AA015901).
Author Contributions
Conceived and designed the experiments: XW. Performed the experiments: XW JY. Analyzed the data: XW JY. Contributed reagents/materials/analysis tools: HW YZ. Wrote the paper: XW.
References
- 1. Deledalle CA, Denis L, Tupin F, Reigber A. NL-SAR: A Unified Nonlocal Framework for Resolution-Preserving (Pol)(In)SAR Denoising. IEEE Trans Geosci Remote Sens. 2015;53(4):2021–38.
- 2. Lebrun M, Colom M, Morel JM. Multiscale Image Blind Denoising. IEEE Trans Image Process. 2015;24(10):3149–61. pmid:26080052
- 3. Maggioni M, Katkovnik V, Egiazarian K, Foi A. Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction. IEEE Trans Image Process. 2013;22(1):119–33. pmid:22868570
- 4. Rajwade A, Rangarajan A, Banerjee A. Image Denoising Using the Higher Order Singular Value Decomposition. IEEE Trans Pattern Anal Mach Intell. 2013;35(4):849–62. pmid:22732660
- 5. Salimi-Khorshidi G, Douaud G, Beckmann CF, Glasser MF, Griffanti L, Smith SM. Automatic denoising of functional MRI data: Combining independent component analysis and hierarchical fusion of classifiers. NeuroImage. 2014;90:449–68. pmid:24389422
- 6. Shao L, Yan R, Li X, Liu Y. From Heuristic Optimization to Dictionary Learning: A Review and Comprehensive Comparison of Image Denoising Algorithms. IEEE Transactions on Cybernetics. 2014;44(7):1001–13. pmid:24002014
- 7. Tan J, Ma Y, Baron D. Compressive Imaging via Approximate Message Passing With Image Denoising. IEEE Trans Signal Process. 2015;63(8):2085–92.
- 8.
Luo E, Chan SH, Pan S, Nguyen TQ, editors. Adaptive non-local means for multiview image denoising: Searching for the right patches via a statistical approach. 2013 IEEE International Conference on Image Processing; 2013 15–18 Sept. 2013.
- 9. Huang Y, Chen X, Zhang J, Zeng D, Zhang D, Ding X. Single-trial ERPs denoising via collaborative filtering on ERPs images. Neurocomputing. 2015;149, Part B:914–23. http://dx.doi.org/10.1016/j.neucom.2014.07.043.
- 10. He S, Chen H, Zhu Z, Ward DG, Cooper HJ, Viant MR, et al. Robust twin boosting for feature selection from high-dimensional omics data with label noise. Information Sciences. 2015;291:1–1811.
- 11. Wong WK, Lai Z, Xu Y, Wen J, Ho CP. Joint Tensor Feature Analysis For Visual Object Recognition. IEEE Transactions on Cybernetics. 2015;45(11):2425–36. pmid:26470058
- 12. Liu F, Zhang D, Shen L. Study on novel Curvature Features for 3D fingerprint recognition. Neurocomputing. 2015;168:599–608. http://dx.doi.org/10.1016/j.neucom.2015.05.065.
- 13. Luisier F, Blu T, Unser M. Image Denoising in Mixed Poisson Gaussian Noise. IEEE Trans Image Process. 2011;20(3):696–708. pmid:20840902
- 14.
Buades A, Coll B, Morel JM, editors. A non-local algorithm for image denoising. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05); 2005 20–25 June 2005.
- 15. Mahmoudi M, Sapiro G. Fast image and video denoising via nonlocal means of similar neighborhoods. IEEE Signal Process Lett. 2005;12(12):839–42.
- 16. Tasdizen T. Principal Neighborhood Dictionaries for Nonlocal Means Image Denoising. IEEE Trans Image Process. 2009;18(12):2649–60. pmid:19635697
- 17. Ji Z, Chen Q, Sun Q-S, Xia D-S. A moment-based nonlocal-means algorithm for image denoising. Information Processing Letters. 2009;109(23):1238–44. http://dx.doi.org/10.1016/j.ipl.2009.09.007.
- 18. Chen Q, Sun Q-s, Xia D-s. Homogeneity similarity based image denoising. Pattern Recognit. 2010;43(12):4089–100. http://dx.doi.org/10.1016/j.patcog.2010.07.002.
- 19. Kervrann C, Boulanger J. Optimal Spatial Adaptation for Patch-Based Image Denoising. IEEE Trans Image Process. 2006;15(10):2866–78. pmid:17022255
- 20. Thaipanich T, Oh BT, Wu PH, Xu D, Kuo CCJ. Improved image denoising with adaptive nonlocal means (ANL-means) algorithm. IEEE Trans Consum Electron. 2010;56(4):2623–30.
- 21. Grewenig S, Zimmer S, Weickert J. Rotationally invariant similarity measures for nonlocal image denoising. J Visual Commun Image Represent. 2011;22(2):117–30. http://dx.doi.org/10.1016/j.jvcir.2010.11.001.
- 22.
Tomasi C, Manduchi R, editors. Bilateral filtering for gray and color images. Computer Vision, 1998 Sixth International Conference on; 1998 4–7 Jan 1998.
- 23. Rudin LI, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Physica D. 1992;60(1):259–68. http://dx.doi.org/10.1016/0167-2789(92)90242-F.
- 24. Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell. 1990;12(7):629–39.