http://orcid.org/0000-0002-2654-8643
Figures
Abstract
We proposed a new efficient image denoising scheme, which mainly leads to four important contributions whose approaches are different from existing ones. The first is to show the equivalence between the group-based sparse representation and the Schatten-p norm minimization problem, so that the sparsity of the coefficients for each group can be measured by estimating the underlying singular values. The second is that we construct the proximal operator for sparse optimization in ℓp space with p ∈ (0, 1] by using fixed-point iteration and obtained a new solution of Schatten-p norm minimization problem, which is more rigorous and accurate than current available results. The third is that we analyze the suitable setting of power p for each noise level σ = 20, 30, 50, 60, 75, 100, respectively. We find that the optimal value of p is inversely proportional to the noise level except for high level of noise, where the best values of p are 1 and 0.95, when the noise levels are respectively 75 and 100. Last we measure the structural similarity between two image patches and extends previous deterministic annealing-based solution to sparsity optimization problem through incorporating the idea of dictionary learning. Experimental results demonstrate that for every given noise level, the proposed Spatially Adaptive Fixed Point Iteration (SAFPI) algorithm attains the best denoising performance on the value of Peak Signal-to-Noise Ratio (PSNR) and structure similarity (SSIM), being able to retain the image structure information, which outperforms many state-of-the-art denoising methods such as Block-matching and 3D filtering (BM3D), Weighted Nuclear Norm Minimization (WNNM) and Weighted Schatten p-Norm Minimization (WSNM).
Citation: Cai S, Liu K, Yang M, Tang J, Xiong X, Xiao M (2018) A new development of non-local image denoising using fixed-point iteration for non-convex ℓp sparse optimization. PLoS ONE 13(12): e0208503. https://doi.org/10.1371/journal.pone.0208503
Editor: Mathieu Hatt, INSERM, FRANCE
Received: May 25, 2018; Accepted: November 19, 2018; Published: December 12, 2018
Copyright: © 2018 Cai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant date are within the paper.
Funding: This work was funded by NSF140928 of the United States, National Natural Science Foundation of China (61201392), and in part by the Natural Science Foundation of Guangdong Province, China (No. 2015A030313497).
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Images are generally contaminated by noise during acquisition, transmission and compression and real-life images are often degraded with mixed noise and it is hard to identify the type and model the noise [1–8]. Images with high resolutions are desirable in many applications, e.g., object recognition, image classification, and image segmentation in medical and biological science. As an essential low-level image processing procedure, image denoising has been studied extensively and belong to a special type of classical inverse problems. The general observation with additive noise can be modeled as Y = X + N, where Y is the noisy observation, and X and N present the original image and white Gaussian noise, respectively. Though a plethora of noise removal techniques have appeared in recent years, for example, Convolutional Neural Network (CNN) [9, 10] have proved very promising on denoising tasks for which large training sets are available, but when the training data are scarce, their performance suffers from overfitting. Therefore image denoising for real-life noise still remains an important challenge in order to recover the images with high quality [11].
Image denoising problem is in general ill-posed and it requires appropriate regularization. Over the past few decades, numerous image denoising methods have been developed [12]. This is usually achieved by minimizing a suitable energy functional that characterizes a trade-off between data-fidelity and regularity. Frobenius norm is often employed to measure the data fitting loss for additive Gaussian noise.
Sparse signal representation describes a signal that can be approximated as a linear combination of as few as possible atoms from a given dictionary. Recently, Elad [13] showed that sparse overcomplete representation approach is quite effective in denoising images, supported by recent study that better denoising performance can be achieved by using a variant of sparse coding methods [14, 15]. In order to promote sparsity more extensively than convex regularization, it is also standard practice to employ non-convex optimization [16].
In image denoising, following [17], each noise patch yi is extracted from the noisy image Y. In order to better exploit group sparsity, we group a set of similar patches . Thus, denoising problem becomes the recovery problem of xi from yi. Now let us consider the group sparsity defined by a group norm ||A||p,2:
(1)
where
is related to image patches by X = DA. We note that the group norm (quadratic symmetric gauge function, see 2.4.2 of [18])||⋅||p,2 is defined by
where αi = [αi,1, …, αi,m]T denotes the ith column of matrix A in
. In recent years, many research is devoted to address the group sparse optimization problem (1), aiming at the improvement of efficiency and accuracy (e.g., see survey paper [16] and references therein).
Once all group sparse codes A are achieved, the latent clean image X can be reconstructed as X = DA by standard approach(see Theorem 1 and 2 in [19]).
The main contributions of this paper are illustrated as follows:
- We unify the group-based sparse coding in [20] and the Schatten-p norm minimization problem in [21] by proving their mathematical equivalence.
- A fixed-point iteration scheme is developed for sparse optimization in ℓp space with p ∈ (0, 1] by using proximal operator and we a new solution to Schatten-p norm minimization problem is obtained, which appears to be more accurate and rigorous than [21].
- Regarding to image denoising, we find that the optimal value of p is inversely proportional to the noise level except for high level noise, where the best values of p are 1 and 0.95, when the noise levels are 75 and 100, respectively.
- The proposed Spatially Adaptive Fixed Point Iteration (SAFPI) algorithm attains the best denoising performance on the value of PSNR and SSIM, being able to retain the image structure information, which outperforms many state-of-the-art denoising methods such as BM3D, WNNM and WSNM.
The rest of the paper is organized as follows. In Section 2.1, we prove the equivalence of group-based sparse coding and the Schatten-p norm minimization problem and propose a new solution to Schatten-p norm minimization problem. A fixed point iteration for solving sparse optimization in ℓp space with p ∈ (0, 1] is formulated and discussed. In Section 2.2, we establish an image denoising scheme using nonlocal self-similarity and Schatten-p norm minimization. In Section 3, based on the new developed Spatially Adaptive Fixed Point Iteration (SAFPI) algorithm, we present the experimental results using a set of standard benchmark images. And the comparison with several existing methods are also provided to demonstrate our improvement. Finally, the paper ends with concluding remarks.
2 Materials and methods
2.1 Proximal operator for Schatten-p norm minimization
2.1.1 Background.
Consider a matrix , then YT Y is a positive semidefinite matrix. The eigenvalues of YT Y are called the singular values of Y, denoted by σ1(Y), …, σmin{m,n}(Y) in decreasing order (see page 246 of [22]). Let r = rank(Y), it is clear that
The matrix Y also has the following Singular Value Decomposition (SVD) Y = UΣVT, where
(
is the set of orthogonal matrices) and σn is an m×n diagonal matrix with diagonal entries σ1(Y), …, σmin{m,n}(Y). We introduce the Schatten-p norm (0 < p < ∞) of Y, which is defined as
Special cases of the Schatten-p norm include the nuclear norm (p = 1) and the Frobenius norm (p = 2).
Next we analyze the relationship between group-based sparse coding and the Schatten-p norm minimization problem, which improves Theorem 2 in [23]. But our approach is based on the “symmetry” technique (similar to [17] for other purpose), which is essentially different from [23].
Theorem 1 The group-based sparse coding problem (1) is equivalent to a Schatten-p norm minimization problem.
Eqs (12), (13) and (14) imply that any operation designated for sparse coefficient vector α’s can be conveniently implemented with singular values of X (only differs by a constant scalar).
The Schatten-p norm (0 < p ≤ 1) has been widely used to replace the nuclear norm for better approximating the rank function. There are extensive study for the Schatten-p norm optimization problem (14) in literature [24, 25]. Note that the main difference between group sparse coding and the Schatten-p norm minimization problem is that group sparse coding has a dictionary learning operator while the Schatten-p norm minimization problem does not involve such operation.
2.1.2 Computation of proximal mapping using fixed point iterative method.
Now let us recall the definition of proximal mapping.
Definition 2 The proximal mapping of a mapping
is
The proximal mapping of
is defined as:
(2)
And we have the following celebrated theorem:
Theorem 3 [Theorem 1 of [26]] If matrix has the following Singular Value Decomposition (SVD) Y = UΣVT, where
and σn is an m × n diagonal matrix with diagonal entries σ1(Y), …, σmin{m,n}(Y). Then we have in Eq (2)
(3)
where
is defined as the scalar proximal mapping in (2):
(4)
In order to be transparent for our proposed approach to solve Eq (4), we recall two important concepts in convex optimization next.
Definition 4 (see Chapter 2 p 82 of [27]) Let be paired by a bilinear functional (inner product) 〈,〉 and let
be any extended real-valued function on
. Then the function f* on
defined by
is called the Fenchel conjugate of f (with respect to the given pairing). Note that f* is always a closed convex function, regardless of the structure of f.
Definition 5 Given the proper convex function , the subdifferential of such a function is the (generally multivalued) mapping
defined by
The elements x* ∈ ∂f(x) are called subgradients of f at x. Actually, same definition works for nonconvex f (however, subgradient need not exist).
A point is a minimizer of a function f (not necessarily convex) over
if and only if f is subdifferentiable at x and 0 ∈ ∂f(x).
Lemma 6
is closed and convex, but it has no close form solution for general p.
If p = 1, it is well-known that the function ϕ(w) = |w| is not differentiable but still convex, and can be described by a subgradient (see Section 2.3 of [28]) as ∂ϕ(w) = sign(w) and from Lemma 6, we have
Furthermore, we can obtain the following theorem, which improves Theorem 1 in [29] and Theorem 1 in [19] using fixed-point iteration (see Chapter 1 of [30] for details).
Definition 7 Given a function , find ξ ∈ [a, b] such that ξ = g(ξ). If such ξ exists, it will be called a fixed point of g and it could be computed by the following algorithm: ξ(n) = g(ξ(n−1)), n ≥ 1. And g is said to be a contraction on [a, b] if there exists a constant L such that 0 < L < 1 and |g(x) − g(y)| < L|x − y| for any x, y ∈ [a, b].
2.2 Spatially Adaptive Fixed Point Iteration (SAFPI) denoising algorithm
In [20, 21], the authors proposed a group sparse representation framework and a Schatten-p norm minimization framework for image denoising. In Theorem 1, we have shown these two approaches are equivallent. From combining Theorem 3 and Theorem 8, we obtained a fixed point iteration solution of Eq (14) in Theorem 1, which is more rigorous than [20, 21].
After grouping a set of similar patches , the denoising problem becomes the recovery problem of xi from yi. And as was shown in Theorem 1, the Schatten-p norm minimization problem (14) converts the denoising problem to recover the low-rank matrix X from the non-low-rank matrix Y, and thus filtering out the noise of the structure set. And the second identity in Eq (14) can be solved using Theorem 3 and Theorem 8.
Wavelet-based image denoising assumes that the wavelet coefficients obey the Laplace distribution, and the threshold method is used to filter the noise in the image. The prior distribution of the block matrix singular values can also approximate the Laplace distribution in space. The parameter λ for each group that balances the fidelity term and the regularization term should be adaptively determined for better denoising performance. Using the Spatial Adaptive Laplacian Transcendental as appeared in [17, 31], the threshold parameter can be set to where σi denotes the locally estimated variance at the position i. Now the second identity in Eq (14) becomes
(5)
From Eq (4), if Y has singular value decomposition Y = UΣVT, we have
, where
. And
can be computed using Theorem 3 and Theorem 8.
Recently some developed iterative regularization techniques in [17] offers an alternative approach toward spatial adaptation. The basic idea of iterative regularization is to add filtered noise back to the denoised image i.e.,
(6)
where k denotes the iteration number and δ is a relaxation parameter. Besides, we can execute the above denoising procedures for better results after several iterations. In the k + 1-th iteration, the iterative regularization strategy in [17] is used to update the estimation of noise variance. Then the standard deviation of noise in k + 1-th iteration is adjusted as
(7)
where γ is a scaling factor controlling the re-estimation of noise variance and the local estimated variance at the i-th position is
(8)
where χi is the i-th singular value of image y.
The higher the structural similarity of the blocks in the structure group is, the more correlative the column vectors in the block matrix will be, which means that it has a low rank property corresponding to the noise-free matrix. The information is mainly concentrated in those largest singular values. During the proximal operation, selecting the appropriate threshold parameter for those with larger singular value makes the processed singular value closer to the noise-free singular value, which can well preserve the useful information in the image while filtering out the noises.
Therefore, choosing blocks with more similar structure will help to improve the image denoising effect. There are many commonly used similarity measures such as Euclidean distance, cosine angle, and correlation coefficient. The traditional block similarity measure function have some shortcomings in measuring the similarity between blocks. Euclidean distance simply calculate the difference between the pixel gray value of the blocks, and then add up as a standard measure of the degree of similarity. Although this method is simple and easy to implement, it only treats the blocks as isolated pixels and neglects the statistical relevance between local pixels, which leads to the inaccuracy of similarity measure. This is because the blocks are not in an Euclidean space. There is a very strong correlation between the pixels in the block. The local pixel correlation carries important structural information of the blocks.
In order to solve this problem, Structural SIMilarity (SSIM) index [32, 33] is often used to evaluate the image quality. SSIM is defined as , where
,
, and
denote the average of X, the average of
, the variance of X and the variance of
, respectively. c1 and c2 are two variables to stabilize the division with weak denominator.
A detailed step-by-step description of Spatially Adaptive Fixed Point Iteration (SAFPI) denoising algorithm is given by Algorithm 1
Algorithm 1 Image Denoising via SAFPI Algorithm
Require: Initialization: ;
Iterate on i = 1,2, …, iter
- Iterative regularization:
and compute its variance
;
- Divide y(k+1) into several blocks, the SSIM is used to classify the blocks with structural similarities into one structural group Yi;
- Noise variance update: re-estimate σω from y(k+1) via
;
- SVD for each noisy data matrix Yi: (Ui, Σi, Vi) = SVD(Yi), where Σi = diag{ε1, …, εmin{m,n}};
- Thresholds update: compute λi using
and
;
- Compute
- Application of proximal operator: Updating the εi value by using the follow formula
with computed λi and εi from step 5 and 4;
- Image update: obtain an improved denoised image
by weighted averaging all denoised patches
, where
;
Output: .
3 Results and discussion
In recent years, many denoising algorithms have been developed and the adaptive image removal algorithms [34–36] is a hot trend in signal and image denoising. To demonstrate the effectiveness of the proposed denoising algorithm, in this section, we compared the denoising performance with recently proposed state-of-the-art denoising methods, such as BM3D [37], WNNM [38], WSNM [21], Expected Patch Log-likelihood (EPLL) [39], Spatially Adaptive Iterative Singular-value Thresholding (SAIST) [17], Patch-Based Near-Optimal image denoising (PBNO) [40], Global Image Denoising (GID) [41], iterative denoising system based on Wiener filtering (WIENER) [34], and Linear Complex Diffusion Process (LCDP) [35]. We have used some well known images that are commonly used in the literature such as [17, 21, 38, 42]. We added noise to them, and test the proposed denoising algorithm with different power p under different noise levels. The experimental images are shown in Fig 1.
There are several image quality evaluation indicators measuring success of denoising such as kurtosis, low signal-to-noise-ratio(SNR). Low kurtosis indicate superior performance and it is defined as [43], where Ck(.) is the k-th cumulant function. In our work, we evaluated the performance with three criterion: Structure Similarity Index (SSIM), kurtosis and Peak Signal-to-Noise Ratio (PSNR) which defined as
, where M denotes the maximum intensity of the underlying image and
is the mean squared error between the denoised image
and the noiseless image X. All the experiments were carried out on Matlab (R2016a) of a PC with Intel(R) Xeon(R) CPU E5 − 1630 V4@3.7GHz and 32GB RAM.
3.1 Analysis of over-shrinkage problem and optimal power p
Firstly, we noticed that not all values of power p applied well to the proposed Spatially Adaptive Fixed Point Iteration (SAFPI) algorithm. It would conduct an approximation deviation with the solved singular values and produce excessive contraction, when the value of p is not suitable. As shown in Fig 2, we tested SAFPI to process low rank approximation on the red patch in Fig 2B with the noise level be 50, which is randomly marked from “Monarch” Fig 2A. In Fig 2C, represents the singular values of the denoised similar patches with different power p. The ground-truth line (denoted by blue line) is the singular value connection line for the similar blocks of the noiseless red patch in Fig 2A. Now we can see that
(shown on green line) is more close to the ground-truth line. This means that the other
’s (denoted by black, red, blue lines) conducted a serious over-shrinkage problem. In this case, setting p = 1 as in WNNM in denoising will lead to bad processing results. So the advantage of SAFPI algorithm is to overcome the over-shrinkage problem, in case we can find the optimal value of power p.
(A) Original image. (B) Noisy image with σn = 50. (C) Singular values of .
Secondly, in order to find the optimal values of p under different noise levels for SAFPI algorithm, we randomly chose 10 test images in Fig 1 for our experiments and set the values of power p to be from 0.05 to 1 with an interval of 0.05. The zero mean additive white Gaussian noise levels were set to be σn = {20, 30, 50, 60, 75, 100}, and the other parameters were the same as WSNM [21]. The results are shown in Fig 3, the horizontal coordinate denotes the different values of p and the vertical coordinate represents the average value of PSNR under given noise level. And the red dots are the optimal points for each given noise level.
(A) σn = 20 (B) σn = 30 (C) σn = 50 (D) σn = 60 (E) σn = 75 (F) σn = 100.
We can see that the best values of power p are 1.0, 0.90, 0.85 and 0.6, when the noise levels are low or medium 20, 30, 50 and 60, respectively. While handling very high noise levels 75, 100, the average PSNR values decrease firstly and then increase, the best values of p are 0.95 and 0.9 respectively. To sum up, we find that the optimal value of p is inversely proportional to the noise level except for high level of noise, where the best values of p are 1 and 0.95. And then we applied the best empirical values for the next experiments.
3.2 Performance comparison with different methods
We set p = {1.0, 0.9, 0.85, 0.6, 0.95, 0.9} for σ = {20, 30, 50, 60, 75, 100} in our proposed SAFPI algorithm. And then we compared the performance with seven standard algorithms (BM3D, WNNM, WSNM, EPLL, SAIST, PBNO, GID, WIENER, LCDP) from 13 widely used images from Fig 1. The results (thanks to the source codes provided by the authors) are in Tables 1, 2, 3, 4, 5 and 6. It can be seen from Table 7 that our algorithm always obtains the best average values of PSNR under different noise levels. The proposed approach achieves 0.3dB to 0.51dB improvement on average over the BM3D, when the noise levels are between 20 and 100. It also achieves 0.02dB, 0.06dB and 0.14dB improvement on average over the WSNM, when the noise levels are 30, 50 and 100, respectively. And our average values of SSIM are the best when the noise levels are 20, 30, 50 and 60. To sum up, for every given low and medium noise level, our algorithm attains the best denoising performance on the values of SSIM and PSNR for all noise levels. This leads to a better image denoising performance and high robustness to noise strength in comparison to several existing denosing algorithms.
For visual quality, some comparative images are shown in Figs 4, 5, 6 and 7. As shown in Fig 4, our algorithm resumed the structure of the ear (which is magnified in the highlighted red window) better than other algorithms. When the noise level is very high, as shown in the zoom-in window in Fig 7, our algorithm could reconstruct clear texture structures, while the competing methods get more blurred textures. Other visual improvements can be seen in Figs 5 and 6. Sometimes the variation of noise is too big and too small in the same image (in different parts of the image). To demonstrate our method, we randomly selected two small pieces from the given image. Although their local noise level would be different, our algorithm always gets the best visual texture. Now we could conclude that the proposed SAFPI algorithm can display excellent denoising performance, producing good visual effect and rebuilding better textures.
(A) Ground Truth (B) Noisy Image (C) BM3D, PSNR = 30.48 (D) EPLL, PSNR = 30.34 (E) SAIST, PSNR = 30.45 (F) WNNM, PSNR = 30.64 (G) WSNM, PSNR = 30.64 (H) SAFPI, PSNR = 30.68.
(A) Ground Truth (B) Noisy Image (C) BM3D, PSNR = 28.36 (D) EPLL, PSNR = 28.35 (E) SAIST, PSNR = 28.03 (F) WNNM, PSNR = 28.94 (G) WSNM, PSNR = 29.02 (H) SAFPI, PSNR = 29.09.
(A) Ground Truth (B) Noisy Image (C) BM3D, PSNR = 29.69 (D) EPLL, PSNR = 28.76 (E) SAIST, PSNR = 29.99 (F) WNNM, PSNR = 30.28 (G) WSNM, PSNR = 30.21 (H) SAFPI, PSNR = 30.51.
(A) Ground Truth (B) Noisy Image (C) BM3D, PSNR = 23.62 (D) EPLL,PSNR = 22.14 (E) SAIST, PSNR = 23.98 (F) WNNM, PSNR = 24.39 (G) WSNM, PSNR = 24.43 (H) SAFPI, PSNR = 24.5.
If noise is non-Gaussian, one popular method is to transform the non-Gaussian noise into a more tractable Gaussian model such as the generalized Anscombe transformation (GAT) [43, 44]. In this paper, we deal with non-Gaussian noise using the proposed algorithms. In the first experiment, we assumed the noise was a mix of Gaussian noise (σn = 20, 50, 100) and speckle noise (the density is d = 1*10−3). Then we used the SAFPI algorithm directly to remove the noise. We randomly selecteded six images (Lena, Monarch, Barbara, Cameraman, House, Peppers) on Fig 1 for experimental verification and compared with some excellent denoising algorithms which has been mentioned in the previous experiments. The results are shown on Table 8. In the second experiment, we assumed the noise was mixed Poisson-Gaussian noise. Then we transformed the Poisson-Gaussian hybrid noise into an approximate Gaussian noise using the GAT [43] algorithm and obtained the repaired images by using the proposed denoising algorithm and the exact unbiased inverse GAT [44]. We used Lena and C.man as the test images and set eight different peak values to be (1, 2, 5, 10, 20, 30, 60, 120). The Poisson-Gaussian noise were set to be σ = peak/10. We compared with BM3D, SAIST, WNNM and recorded the average PSNR and kurtosis parameters of these two images. The results are shown on Table 9. All bold numbers represent the best evaluation index values. From Table 8, we can see when the standard deviation is not big (σn = 20, 50), our proposed algorithm almost achieved the best values of all three quality evaluation indicators, and obtained the best PSNR values on all of hybrid noises experiments. From Table 9, it can be seen that the SAFPI algorithm almost get the highest averaged PSNR value under all peaks experiments. In most cases, our proposed algorithm obtained relatively good kurtosis metrics and optimal PSNR value, which is 0.2 to 0.3dB higher than the BM3D algorithm and about 0.1dB to 1.11dB higher than WNNM.
Finally, we bravely attempted to discuss the complexity of SAFPI algorithm. We assume each patch size is A * A, where A represents the length or width of each block, and k is the number of similar patches in each structural group yi. Now calculating SVD (step 4 in Algorithm 1) needs flops in each iteration. And it also costs
to compute the singular values in step 6. Next since the image y(k+1) can be divided into N blocks in step 2, then it needs
flops, where i is the number of iterations in Algorithm 1. Then we recorded the execution times of several excellent denoising algorithms spent on the above experiments with the standard deviation σn of the white Gaussian noise to be 20: SAFPI 4843.339s, WSNM 5453.311s, WNNM 4410.991s, SAIST 923.9837s, BM3D 17.6242s and EPLL 1550.607s. Our algorithm did not take much longer time while maintaining the best denoising results.
4 Conclusions
In this paper, a fixed-point iteration scheme was developed for sparse optimization in ℓp space with p ∈ (0, 1] by using proximal operator. We showed that group sparse coding was equivalent to Schatten-p norm minimization problem, and thus the sparse coefficient of each group were measured by estimating the singular values of each group. When analyzing the optimal value of power p, we can find that the optimal value of Schatten p-norm is related to the noise level. As the noise level increases, the optimal value of p decreases gradually. And if the noise reaches a high level, the optimal value of p will be close to 1. The developed SAFPI algorithm can obtain higher PSNR indices and is able to retain promising texture structure information and visual quality. The methods developed in this paper leads to a better image denoising compared to other competing denoising algorithms. There are several future research directions. We are further exploring other non-convex optimization strategies for more effective convergence and further improvement. The convolutional neural networks(CNN) based denoising methods become more and more popular now and we will investigate CNN architectures for the denoising of images in the future.
5 Appendix
Proof 9 (Proof of Theorem 1) Let D = U and A = ΣVT in Eq (1), where is a diagonal matrix and each column of V in
is decomposed of vi = (αi)T/εi. Then we have
(9)
Let σi denotes the standard deviation of the sparse coefficients αi in the i-th column, then the sum of standard deviations associated with sparse coefficient vector in each column is
(10)
And then it is not hard to see
(11)
Using Eq (10) and the unitary property of V, we have
(12)
Then it is ready to see
(13)
By substituting Eq (13) into Eq (9) we could obtain
(14)
which appears to be better approximation to the rank function by using the Schatten-p quasi-norm.
Proof 10 (Proof of Lemma 6) Let ϕ = |w|p, by using subdifferential and Corollary 2.59 of [27], one has
If ϕ is convex, this is an equality.
If we further define , since
(15)
it is ready to see
(16)
For Π*(x) is the pointwise minimum of the collection of affine functions, we know it is closed and convex. Thus Proxλ ∥ ⋅∥p(x) is also closed and convex.
But it is easy to see Π*(x) is a discontinues mapping, so generally there is no closed form expression for it.
Proof 11 (Proof of Theorem 8) For 0 < p < 1, let ϕ(w) = |w|p, we have ∂ϕ(w) = ∅ when w = 0.
In order to overcome the singularity of (|w|p)′ = pw/|w|2−p near w = 0, following Section 4 of [28], we consider for 0 < ϵ << 1 the approximation
It is important to observe that Proxλ ∥⋅∥p(x) = 0 if
(17)
which is equivalent to
equality obtained when
. Otherwise, the necessary optimality condition is given by
(18)
To solve Eq (18) for nonnegative w, let
we consider the iteration
One can easily see that
if and only if
. Notice that the first and second order derivatives of f(w) = (w − x)2 + 2λ|w|p are
and one can easily verify that f(w) is concave in the range of
, and is convex in the range of
. By using the Contraction Mapping Theorem (Theorem 1.5 on page 11 of [30]), we have
The iteration will eventually converge to a fixed-point, which is the root w = g(w) in the interval
.
Moreover, by noting that the derivative of x(w) = w + λpwp−1 is
by solving
, we have
then
and
. Combined with (17), we denote
Thus the proof is completed.
Acknowledgments
This research was supported in part by the National Natural Science Foundation of China (61201392),in part by the Natural Science Foundation of Guangdong Province, China (No. 2015A030313497), and in part by the NSF140928 of the United States.
References
- 1.
Dong B, Shen Z. Image restoration: a data-driven perspective. Proceedings of the International Congress of Industrial and Applied Mathematics (ICIAM), Beijing, China. 2015;p. 65–108.
- 2. Kang Z, Peng C, Cheng Q. Kernel-driven similarity learning. Neurocomputing. 2017;267:210–219.
- 3.
Kang Z, Wen L, Chen W, Xu Z. Low-rank kernel learning for graph-based clustering. Knowledge-Based Systems. 2018;.
- 4. Peng C, Kang Z, Cai S, Cheng Q. Integrate and Conquer: Double-Sided Two-Dimensional k-Means Via Integrating of Projection and Manifold Construction. ACM Transactions on Intelligent Systems and Technology (TIST). 2018;9(5):57.
- 5. Sun P, Mahdi A, Xu J, Qin J. Speech Enhancement in Spectral Envelop and Details Subspaces. Speech Communication. 2018;.
- 6. Han Y, Xu C, Baciu G, Li M, Islam MR. Cartoon and texture decomposition-based color transfer for fabric images. IEEE transactions on multimedia. 2017;19(1):80–92.
- 7. Li M, Xu C. Variational Image Restoration and Decomposition in Shearlet Smoothness Spaces. Chinese Journal of Electronics. 2017;26(5):1017–1021.
- 8. Dong H, Ye D, Feng J, Wang J. Almost sure cluster synchronization of Markovian switching complex networks with stochastic noise via decentralized adaptive pinning control. Nonlinear Dynamics. 2017;87(2):727–739.
- 9.
Slavova A, Rashkova V. A novel CNN based image denoising model. In: Circuit Theory and Design (ECCTD), 2011 20th European Conference on. IEEE; 2011. p. 226–229.
- 10.
Tian C, Xu Y, Fei L, Yan K. Deep Learning for Image Denoising: A Survey. arXiv preprint arXiv:181005052. 2018;.
- 11. Chatterjee P, Milanfar P. Is denoising dead? IEEE Transactions on Image Processing. 2010;19(4):895–911. pmid:19932997
- 12. Lebrun M, Colom M, Buades A, Morel JM. Secrets of image denoising cuisine. Acta Numerica. 2012;21:475–576.
- 13.
Elad M. Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer; 2010.
- 14.
Mairal J, Bach F, Ponce J. Sparse modeling for image and vision processing. arXiv preprint arXiv:14113230. 2014;.
- 15. Cai S, Kang Z, Yang M, Xiong X, Peng C, Xiao M. Image Denoising via Improved Dictionary Learning with Global Structure and Local Similarity Preservations. Symmetry. 2018;10(5):167.
- 16. Zhang Z, Xu Y, Yang J, Li X, Zhang D. A survey of sparse representation: algorithms and applications. IEEE access. 2015;3:490–530.
- 17. Dong W, Shi G, Li X. Nonlocal image restoration with bilateral variance estimation: A low-rank approach. IEEE transactions on image processing. 2013;22(2):700–711. pmid:23047875
- 18. Kakade SM, Shalev-Shwartz S, Tewari A. Regularization techniques for learning with matrices. Journal of Machine Learning Research. 2012;13(Jun):1865–1890.
- 19.
Zuo W, Meng D, Zhang L, Feng X, Zhang D. A generalized iterated shrinkage algorithm for non-convex sparse coding. In: Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE; 2013. p. 217–224.
- 20. Wang Q, Zhang X, Wu Y, Tang L, Zha Z. Nonconvex Weighted lp Minimization Based Group Sparse Representation Framework for Image Denoising. IEEE Signal Processing Letters. 2017;24(11):1686–1690.
- 21. Xie Y, Gu S, Liu Y, Zuo W, Zhang W, Zhang L. Weighted Schatten p -Norm Minimization for Image Denoising and Background Subtraction. IEEE Transactions on Image Processing. 2016;25(10):4842–4857.
- 22.
Voyevodin VV. Linear Algebra, translated from the Russian by Vladimir Shokurov. Mir Publishers; 1983.
- 23.
Zha Z, Liu X, Huang X, Shi H, Xu Y, Wang Q, et al. Analyzing the group sparsity based on the rank minimization methods. In: Multimedia and Expo (ICME), 2017 IEEE International Conference on. IEEE; 2017. p. 883–888.
- 24. Lin Z. A review on low-rank models in data analysis. Big Data and Information Analytics. 2016;1:136–161.
- 25.
Xu C, Lin Z, Zha H. A Unified Convex Surrogate for the Schatten-p Norm. In: AAAI; 2017. p. 926–932.
- 26.
Lu C, Zhu C, Xu C, Yan S, Lin Z. Generalized Singular Value Thresholding. In: AAAI; 2015. p. 1805–1811.
- 27.
Barbu V, Precupanu T. Convexity and Optimization in Banach Spaces. Springer; 2014.
- 28. Ito K, Kunisch K. A variational approach to sparsity optimization based on Lagrange multiplier theory. Inverse problems. 2013;30(1):15001–15023.
- 29.
Lorenz D. Variational denoising in Besov spaces and interpolation of hard and soft wavelet shrinkage. Citeseer;.
- 30.
Süli E, Mayers DF. An Introduction to Numerical Analysis. Cambridge University Press; 2003.
- 31.
Zha Z, Liu X, Zhou Z, Huang X, Shi J, Shang Z, et al. Image denoising via group sparsity residual constraint. In: Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Conference on. IEEE; 2017. p. 1787–1791.
- 32. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing. 2004;13(4):600–612. pmid:15376593
- 33. Brunet D, Vrscay ER, Wang Z. On the mathematical properties of the structural similarity index. IEEE Transactions on Image Processing. 2012;21(4):1488–1499. pmid:22042163
- 34. Lahmiri S. An iterative denoising system based on Wiener filtering with application to biomedical images. Optics Laser Technology. 2017;90:128–132.
- 35. Lahmiri S. Denoising techniques in adaptive multi-resolution domains with applications to biomedical images. Healthcare Technology Letters. 2017;4(1):25–29. pmid:28529760
- 36. Lahmiri S. Image denoising in bidimensional empirical mode decomposition domain: the role of Student’s probability distribution function. Healthcare Technology Letters. 2017;3(1):67–71.
- 37. Dabov K, Foi A, Katkovnik V, Egiazarian K. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing. 2007;16(8):2080. pmid:17688213
- 38.
Gu S, Zhang L, Zuo W, Feng X. Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2014. p. 2862–2869.
- 39.
Zoran D, Weiss Y. From learning models of natural image patches to whole image restoration. In: Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE; 2011. p. 479–486.
- 40. Chatterjee P, Milanfar P. Patch-based near-optimal image denoising. IEEE Transactions on Image Processing. 2012;21(4):1635–1649. pmid:22020683
- 41. Talebi H, Milanfar P. Global image denoising. IEEE Transactions on Image Processing. 2014;23(2):755–768. pmid:26270916
- 42. Dong W, Shi G, Ma Y, Li X. Image restoration via simultaneous sparse coding: Where structured sparsity meets gaussian scale mixture. International Journal of Computer Vision. 2015;114(2-3):217–232.
- 43. Xu Z, Gao M, Papadakis GZ, Luna B, Jain S, Mollura DJ, et al. Joint solution for PET image segmentation, denoising, and partial volume correction. Medical Image Analysis. 2018;46. pmid:29627687
- 44. Makitalo M, Foi A. Optimal Inversion of the Generalized Anscombe Transformation for Poisson-Gaussian Noise. IEEE Transactions on Image Processing. 2013;22(1):91–103. pmid:22692910