Total Variation with Overlapping Group Sparsity for Image Deblurring under Impulse Noise

Gang Liu; Ting-Zhu Huang; Jun Liu; Xiao-Guang Lv

doi:10.1371/journal.pone.0122562

Abstract

The total variation (TV) regularization method is an effective method for image deblurring in preserving edges. However, the TV based solutions usually have some staircase effects. In order to alleviate the staircase effects, we propose a new model for restoring blurred images under impulse noise. The model consists of an ℓ₁-fidelity term and a TV with overlapping group sparsity (OGS) regularization term. Moreover, we impose a box constraint to the proposed model for getting more accurate solutions. The solving algorithm for our model is under the framework of the alternating direction method of multipliers (ADMM). We use an inner loop which is nested inside the majorization minimization (MM) iteration for the subproblem of the proposed method. Compared with other TV-based methods, numerical results illustrate that the proposed method can significantly improve the restoration quality, both in terms of peak signal-to-noise ratio (PSNR) and relative error (ReE).

Citation: Liu G, Huang T-Z, Liu J, Lv X-G (2015) Total Variation with Overlapping Group Sparsity for Image Deblurring under Impulse Noise. PLoS ONE 10(4): e0122562. https://doi.org/10.1371/journal.pone.0122562

Academic Editor: Daoqiang Zhang, Nanjing University of Aeronautic and Astronautics, CHINA

Received: October 20, 2014; Accepted: February 22, 2015; Published: April 15, 2015

Copyright: © 2015 Liu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: The work of GL, TZH and JL is supported by China 973 Program (2013CB329404), NSFC (61370147, 61170311), Sichuan Province Sci. and Tech. Research Project (2012GZX0080). The work of XGL is supported by Nature Science Foundation of Jiangsu Province (BK20131209), NSFC (61401172).

Competing interests: The authors have declared that no competing interests exist.

Introduction

Image deblurring and denoising problems have been widely studied in the past decades. In the literature, it is widely assumed that observed images are the convolution of standard linear and space invariant blurring functions with true images plus some noise. Let g denote the blurred and noisy image, h the blur kernel, f the original image and η the noise. The image f is assumed to be a real function defined on a bounded and piecewise smooth open subset Γ of ℝ². In general, the image formation process can be modeled as: g = h ⋆ f + η, where “⋆” denotes the two-dimensional convolution operation. Image deblurring is to estimate the true image f from the blurred and noisy image g. As is well known, image deblurring and denoising is a typically ill-posed problem [1, 2]. To handle this problem, regularization technique is usually considered to obtain a stable and accurate solution. In this way, we need to solve the following problem (1) where the first term is called the regularization term, the second term is called the fidelity term (ℓ₂-fidelity), μ > 0 is the regularization parameter, and ψ is the regularization functional.

Without loss of generality, a discretized image may have n₁ × n₂ pixels. Simply, in our work, we assume that n₁ = n₂ = n, then f, g and η are vectors of length n². It is easy to extend to any n₁ × n₂ image. Let H be the corresponding blurring matrix of n² × n² from h [3]. Then the discretized form of the minimization problem (1) is equivalent to the following matrix-vector form (2) where ‖ ⋅ ‖₂ denotes the Euclidean ℓ₂ norm. Notice that H is a matrix of block circulant with circulant blocks (BCCB) structure when periodic boundary conditions are applied [3].

How to choose a good regularization functional is an active area of research in the imaging science. In the early 1960s, D. L. Phillips [4] and A. N. Tikhonov [5] proposed the definition of ψ as an ℓ₂-type norm (academically called Tikhonov regularization), that is, $ψ = ‖ L f ‖_{2}^{2}$ with L an identity operator or difference operator. Although the functional ψ of this type has the advantage of facilitating the calculations, it is rarely used in current practice because it has the drawback of penalizing discontinuities in resulting solutions, for instance, over-smoothing edges. Therefore, this is not a good choice since natural images have many edges.

To overcome this drawback, many different types of regularization functionals have been proposed. One popular model was introduced by Rudin, Osher and Fatemi (ROF) in [6]. They proposed a total variation (TV) regularization with an ℓ₂-fidelity term (ℓ₂-TV) for image restoration. Its corresponding minimization task is: (3) where BV(Γ) denotes the space of functions of bounded variation. ‖f‖_TV is defined by ${‖ f ‖}_{TV} ≔ \sum_{1 ⩽ i, j ⩽ n} {‖ {(\nabla f)}_{i, j} ‖}_{2} = \sum_{1 ⩽ i, j ⩽ n} \sqrt{{∣ {(\nabla_{x} f)}_{i, j} ∣}^{2} + {∣ {(\nabla_{y} f)}_{i, j} ∣}^{2}}$ which is called as isotropic TV, or ${‖ f ‖}_{TV} ≔ \sum_{1 ⩽ i, j ⩽ n} {‖ {(\nabla f)}_{i, j} ‖}_{1} = \sum_{1 ⩽ i, j ⩽ n} ∣ {(\nabla_{x} f)}_{i, j} ∣ + ∣ {(\nabla_{y} f)}_{i, j} ∣$ which is named as anisotropic TV, where ‖ ⋅ ‖₁ denotes the Euclidean ℓ₁ norm. Operator ∇: ℝ^n² → ℝ^2×n² denotes the discrete gradient operator (under periodic boundary conditions) which is defined by (∇f)_i,j = ((∇_x f)_i,j, (∇_y f)_i,j), with for i, j = 1, 2, ⋯, n, where f_i,j refers to the ((j − 1)n + i)th entry of the vector f (it is the (i, j)th pixel location of the n × n image, and this notation remains valid throughout the paper unless otherwise specified). More details for the definition of ‖f‖_TV can be referred to [7, 8].

Many methods have been proposed to solve the restoration model (3) such as the fast TV deconvolution (FTVd) [9, 10], the augmented Lagrangian method (ALM) [11–14], the dual methods [13, 15], and the split Bregman method [11, 16]. These studies provided many efficient iterative algorithms to make the computation on TV related models be more convenient for Gaussian noise removal. Furthermore, in many cases, the noise does not satisfy the Gaussian assumption, for instance, the noise may follow a Laplace distribution [17]. There has been a growing interest in using an ℓ₁-fidelity term instead of the ℓ₂-fidelity term for image restoration in many literature such as [8, 18–27] for considering another non-Gaussian noise–impulse noise. We consider the corresponding regularization model with an ℓ₁-fidelity term as: (4)

A classic approach is to use the TV regularizer by ψ(f) = ‖f‖_TV, which we call ℓ₁-TV. Recently, Yang et al. [20] used the FTVd method to solve the ℓ₁-TV model fast. Guo et al. [8] proposed a fast algorithm for image restoration in the ℓ₁-TV model. Their method was to add a penalty term by using the variable substitution method, which belongs to penalty methods in optimization. They employed an alternating minimization method to solve it. They also proved the convergence of their method. Numerical tests showed that their method got better restorations and faster than FTVd. Wu et al. [25] used ALM to solve the ℓ₁-TV model. More recently, Chan et al. [26] proposed a constrained total variation (TV) regularization method for image restoration for ℓ₁-TV. Their method used a box constrained projection to ensure the restored images stay in a given dynamic range. They used the alternating direction method of multipliers (ADMM) [11–14, 28–33] to solve the model based on augmented Lagrangian method. They got better results than other methods such as FTVd and ALM. Their numerical results showed that for some images where there are many pixels with values lying on the boundary of the dynamic range, the gain could get very high numerical superiority in the peak signal-to-noise ratio. As far as our knowledge goes, the box constrained projection in image restoration is significant both in practice and theory.

Although the TV regularization using in the restoration problems can recover sharp edges of a degraded image, it also gives rise to some undesired effects and transforms smooth signal into piecewise constants, the so-called staircase effects [34, 35]. To overcome this deficiency, one usual method is to replace the original TV norm by a high-order TV norm. The high-order TV regularization schemes have been studied so far mainly for overcoming the staircase effects while preserving the edges in the restored image. More details can be referred to [35–38]. The high-order TV based methods may have some other behaviors. For example, it may transform the smooth signal to over-smoothing and take more time to compute.

More recently, Bredies et al. [39] proposed total generalized variation for image restoration with overcoming the staircase effects while preserving the edges. Pryre and Fadili [40] considered group sparsity and overlapping group sparsity (OGS) for image processing, such as denoisng, compressed sensing. Their numerical experiments showed that the PSNR (defined in Section 5) gain by OGS is a consistent improvement on comparing with group sparsity across a wide range of natural images. Selesnick and Chen [41] proposed an OGS TV regularizer to one-dimensional signal denoising. They applied the majorization minimization (MM) method to solve their model. Their numerical experiments showed that their method can overcome staircase effects effectively. Their method has the disadvantages of the low speed of computation and the difficulty to directly extend to the two-dimensional case. However, all these methods only considered the ℓ₂-fidelity term under Gaussian noise. To our knowledge, the methods for image deblurring and denoising under impulse noise with these regularization terms are still missing in the literature. Particularly, Liu-Huang-Liu in [27] proposed a hybrid model for image deblurring and denoising under impulse noise, but their model is not convex and it may take more time since its regularization term is the combination of TV and high-order TV with adapted parameter selection.

In this paper, inspired by the works from [40] and [41], we propose a new model for images deblurring under impulse noise by setting ψ in (4) to be the OGS-TV functional. In this model we first extend the OGS-TV functional in [41] to general two-dimensional case as a new regularization term, then consider an ℓ₁-fidelity term for application in images deblurring under impulse noise. Moreover, we impose a box constraint to the proposed model to obtain more accurate solutions similarly as [26]. Using the technique of variable substitution, we provide an efficient algorithm to solve the model under the framework of ADMM. We use an inner loop which is nested inside the majorization minimization (MM) iteration for the subproblem of the proposed method. Our main contributions are firstly combining the former three parts together in one model in application and making the 2D TV with OGS case be more easily solved than the 1D case in [41]. According to a brief explanation of our model and our numerical results, we can observe that our new model using the OGS-TV regularizer coincides to maintain the edge-preserving property of TV methods and overcome the staircase effects, which is similar as total generalized variation [39]. In addition, the numerical results also show that our method is very effective and competitive with other TV-based methods, such as Chan et al.’s method [26] and Guo et al.’s method [8], especially on getting higher PSNR and better visual quality.

The outline of the rest of this paper is as follows. In the next section, we will briefly introduce the definition of the OGS regularization functional. We will also review the MM method and ADMM, which are used in our proposed method. In Section 3, we propose the OGS-TV based model for recovering images under blur and impulse noise. We also provide the efficient solving algorithm in this section. The numerical results are given in Section 4. Finally, we conclude this paper in Section 5.

Some Preliminaries

OGS-TV

In [41], the authors denoted a K-point group (K denotes the group size) of the vector t ∈ ℝⁿ by (5) Note that t_i,K can be seen as a block of K contiguous samplings of t staring at index i. With the notation (5), a group sparsity regularizer for one-dimensional case is defined in [41] as (6)

Similarly, we can define a K-square-point group of a two-dimensional signal such as images considered in this work v ∈ ℝ^n², where vector v is stacked in column-wisely, in other words, the (i, j)th entry of a matrix is assigned to be the ((j − 1)n + i)th entry of the vector v. Clearly, (7) where $K_{l} = [\frac{K - 1}{2}]$ , $K_{r} = [\frac{K}{2}]$ and [x] denotes the largest integer less than or equal to x. The group size is denoted by K². Note that ${\tilde{v}}_{i, j, K, K}$ can be seen as a square block of K × K contiguous samplings of v with the center at index (i, j). Here we choose a group entries around the objective point rather than a group following the objective point like one-dimensional in [41] because of the faster and easier computation in the experiments. Moreover, to the best of our knowledge, the former is much better than the latter in image restoration because the pixels in image are related to or depended on all the ambient pixels rather than partial surrounding pixels. Let v_i,j,K,K be a K²-vector obtained by arranging the K × K elements of ${\tilde{v}}_{i, j, K, K}$ in lexicographic order. This notation also remains valid throughout the paper unless otherwise specified. Then the overlapping group sparsity functional of the two-dimensional array can be defined by (8)

From the definition above, we can easily get that this function is convex. Consequently, we define the regularization functional ψ in (4) to be the form (9)

We call the regularizer ψ in (9) as the OGS anisotropic TV functional because we handle the ∇_x f and ∇_y f separately (while the isotropic TV is defined differently by ${‖ f ‖}_{I T V} = \sum_{i}^{n^{2}} \sqrt{{(\nabla_{x} f)}_{i}^{2} + {(\nabla_{y} f)}_{i}^{2}}$ ), and call the corresponding convex minimization model (4) L1-OGS-ATV.

The MM method

The MM method is an asymptotical method in solving optimization problems, for instance, a minimization problem of the form as follows, (10) where α is a positive parameter and the functional φ is defined in (8). The point of the MM method is that, instead of directly solving the difficult minimization problem P(v), the MM approach solves a sequence of easier optimization problems Q(v, v_k) (k = 0, 1, 2, …) firstly and then manages to get the minimizer of P(v). Generally, an MM iterative algorithm for minimizing P(v) has the form (11) where Q(v, v′) ≥ P(v) for all v, v′, and Q(v, v) = P(v), i.e., each functional Q(v, v′) is a majorizor of P(v). When P(v) is convex, then under the former conditions, the sequence v^k produced by (11) converges to the minimizer of P(v) [42, 43].

We aim to solve the special problem (10), and it is obvious that P(v) in (10) is convex. Therefore, the MM approach is available for solving the problem (10). While we were concluding this manuscript, we became aware of very recent related work in [44]. The authors in [44] have studied the problem (10) elaborately, which is a subproblem of our method. Moreover, for the sake of completeness, we briefly introduce the solving method here by our way independently.

First of all, to derive an efficient algorithm with the MM scheme for solving the problem (10), we want to find a majorizor of P(v). Here, we only need to find a majorizor of φ(v) because of the simple enough quadratic term of the first term in (10). Note that (12) for all v and u ≠ 0 (u, v ∈ ℝ^n²) with equality when u = v. Substituting each group of φ(v) into (12) and summing them, we get a majorizor of φ(v) (13) with (14) provided ‖v_i,j,K,K‖ ≠ 0 for all i, j. After simple calculation, S(v, u) can be rewritten as (15) where C(u) is independent of v, and Λ(u) is a diagonal matrix with each diagonal component (16) with m = 1, 2, ⋯, n². The entries of Λ can be easily computed by using Matlab built-in function conv2. Then a majorizor of P(v) can be easily given by (17) with Q(v, u) ≥ P(v) for all u, v, and Q(u, u) = P(u). To minimize P(v), the MM aims to iteratively solve (18) with the solution (19) where I is an identity matrix with the same size of Λ(v^k). We can easily get that Λ²(v^k) is also a diagonal matrix with each diagonal component [Λ²(v^k)]_m,m equaling to the form of removing the out root of the right term of (16). Moreover, the inversion of the matrix $I + \frac{1}{α} Λ^{2} (v^{k})$ can be computed very efficiently since it only requires simple componentwise calculation. Therefore, we obtain the Algorithm 1 for solving the problem (10).

Algorithm 1 The MM method for solving (10)

1. initialization: Starting point v = v₀, α, group size K², $K_{l} = [\frac{K - 1}{2}]$ ,

$K_{r} = [\frac{K}{2}]$ , ϵ, Maximum inner iterations NIt, k = 0.

2. iteration:

Do

$\begin{matrix} {[Λ^{2} (v^{k})]}_{m, m} & = \sum_{i = - K_{l}}^{K_{r}} \sum_{j = - K_{l}}^{K_{r}} {[\sum_{k_{1} = - K_{l}}^{K_{r}} \sum_{k_{2} = - K_{l}}^{K_{r}} {∣ v_{m - i + k_{1}, m - j + k_{2}}^{k} ∣}^{2}]}^{- \frac{1}{2}}, \\ v^{k + 1} & = {(I + \frac{1}{α} Λ^{2} (v^{k}))}^{- 1} v_{0}, \\ k & = k + 1, \end{matrix}$

until ‖v^k+1 − v^k‖₂/‖v^k‖₂ < ϵ or k > NIt.

3. get v^k.

Variable splitting and ADMM

Consider an unconstrained optimization problem in which the objective function is the sum of two functions as (20) where ϕ_i: ℝ^n_i → ℝ are closed proper convex functions, χ_i ⊆ ℝ^n_i are closed convex sets, A_i ∈ ℝ^l×n_i, and b ∈ ℝ^l is a given vector. The augmented Lagrangian function ([28]) of (20) (21) where λ ∈ ℝ^l is the Lagrange multiplier, β is a penalty parameter which controls the linear constraint, and C does not depend on x₁, x₂. The idea of ADMM is to find a saddle point of 𝓛. Usually, ADMM consists in alternated minimizing 𝓛 on x₁, x₂, λ, for instance, minimizing 𝓛 with respect to x₁ by fixing x₂ and λ. That delivers to the following simple but powerful algorithm classic ADMM.

Algorithm 2 Classic ADMM for the minimization problem (20)

initialization: Starting point $x_{1}^{0}$ , $x_{2}^{0}$ , λ⁰, β.

iteration:

$\begin{matrix} x_{1}^{k + 1} & = & \arg \min ϕ_{1} (x_{1}) + \frac{β}{2} ‖ A_{1} x_{1} + A_{2} x_{2}^{k} - b - \frac{λ^{k}}{β} ‖_{2}^{2}, \\ x_{2}^{k + 1} & = & \arg \min ϕ_{2} (x_{2}) + \frac{β}{2} ‖ A_{1} x_{1}^{k + 1} + A_{2} x_{2} - b - \frac{λ^{k}}{β} ‖_{2}^{2}, \\ λ^{k + 1} & = & λ^{k} - β (A_{1} x_{1}^{k + 1} + A_{2} x_{2}^{k + 1} - b), \\ k & = & k + 1, \end{matrix}$

until a stopping criterion is satisfied.

According to [12], we can see classic ADMM is convergent because of the nonexpansive and absolute summable properties of the x₁ and x₂ subproblems. More details can be found in [12, 28–31]. However, the convergence speed is not too fast. In order to speed the convergence, we can introduce a step length parameter γ for updating the multiplier [29, 32, 33]. The algorithm framework is outlined as follows called general ADMM.

Algorithm 3 General ADMM for the minimization problem (20)

initialization: Starting point $x_{1}^{0}$ , $x_{2}^{0}$ , λ⁰, β.

iteration:

$\begin{matrix} x_{1}^{k + 1} & = & \arg \min ϕ_{1} (x_{1}) + \frac{β}{2} ‖ A_{1} x_{1} + A_{2} x_{2}^{k} - b - \frac{λ^{k}}{β} ‖_{2}^{2}, \\ x_{2}^{k + 1} & = & \arg \min ϕ_{2} (x_{2}) + \frac{β}{2} ‖ A_{1} x_{1}^{k + 1} + A_{2} x_{2} - b - \frac{λ^{k}}{β} ‖_{2}^{2}, \\ λ^{k + 1} & = & λ^{k} - γ β (A_{1} x_{1}^{k + 1} + A_{2} x_{2}^{k + 1} - b), \\ k & = & k + 1, \end{matrix}$

until a stopping criterion is satisfied.

Here, γ > 0 is also called a relax parameter. In fact, if γ = 1, general ADMM is classic ADMM. From [29, 32, 33], general ADMM is convergent if $γ \in (0, (\sqrt{5} + 1) / 2)$ . Moreover, γ = 1.618 makes it converge noticeably faster than γ = 1. Therefore, we set γ = 1.618 in our work.

Proposed Method

With the definition of (9), we will consider a minimization problem of the form (L1-OGS-ATV) (22) Note that for any true digital image, its pixel values can attain only a finite number of values. Hence, it is natural to require all pixel values of the restored image to lie in a certain interval [a, b], see [26] for more details. For example, for 8-bit images, we would like to restore them in a dynamic range [0, 255]. More in general, with the easy computation and the certified results in [26], we only consider all the images located on the range [0, 1]. Therefore, the images we mentioned all lie in the interval [0, 1]. We define a projection operator 𝓟_Ω on the set Ω = {f ∈ ℝ^n×n∣0 ⩽ f ⩽ 1}, (23) Similarly as [26], we will solve the problem (24) We refer to this model as CL1-OGS-ATV. Obviously, this model is also convex.

Particularly, each term of the model (24) has the properties of additivity and separability. Therefore, we can rewrite it as follows (under the periodic boundary conditions), (25)

From the Eq (25), we can observe that our model (24) can be seen as a combination of n × n coupled subproblems, which are all (i, j) terms of the last line in (25). Each subproblem is approximate to the original TV regularization model (4) with ψ(f) = ‖f‖_TV in a local region, which has the edge-preserving property. More details please refer to [45].

Therefore, our model (24) can coincide to 1) maintain the edge-preserving property of TV methods, and 2) have the property of smoothing the local regions, which can be seen as overcoming the staircase effects. However, we did not solve our model decoupled as (25) in this work, so we choose the following algorithm to solve (24) based on ADMM.

For the model (24), by introducing new auxiliary variables v_x, v_y, z, w, we transform the minimization problem (24) to the equivalent constrained minimization problem (26) Note that the constraint is now imposed on w instead of f. The augmented Lagrangian function of (26) is (27) where β₁, β₂, β₃ > 0 are penalty parameters and λ₁, λ₂, λ₃, λ₄ ∈ ℝ^n² are the Lagrange multipliers. According to the scheme of general ADMM mentioned above (Algorithm 3), for a given ( $v_{x}^{k}, v_{y}^{k}, z^{k}, w^{k}, f^{k}$ ; $λ_{1}^{k}, λ_{2}^{k}, λ_{3}^{k}, λ_{4}^{k}$ ), the next iteration $(v_{x}^{k + 1}, v_{y}^{k + 1}, z^{k + 1}, w^{k + 1}, f^{k + 1}$ ; $λ_{1}^{k + 1}, λ_{2}^{k + 1}, λ_{3}^{k + 1}, λ_{4}^{k + 1})$ is generated as follows:

Fix f = f^k, $λ_{1} = λ_{1}^{k}, λ_{2} = λ_{2}^{k}, λ_{3} = λ_{3}^{k}, λ_{4} = λ_{4}^{k}$ , z = z^k, w = w^k, and minimize (27) with respect to v_x and v_y. The minimizers are obtained by (28) (29) It is obvious that problems (28) and (29) match the framework of the problem (10), thus the solutions of (28) and (29) can be obtained by using Algorithm 1, respectively.
Compute z^k+1. The minimization with respect to z can be given by the well-known Shrinkage [20] explicitly: (30) where ∣ ⋅ ∣, sgn and “∘” represent the componentwise absolute value, signum function, and componentwise product, respectively.
Compute w^k+1. The minimizer is given explicitly by (31)
Compute f^k+1 by solving the normal equation (32) where “*” denotes the conjugate transpose, see [13] for more details. Since all the parameters are positive, the coefficient matrix in (32) are always invertible and symmetric positive definite. In addition, note that H, ∇_x, ∇_y and their conjugate transpose have BCCB structure under periodic boundary conditions. We know that the computations with BCCB matrix can be very efficient by using fast Fourier transforms (FFTs).
Update the multipliers via (33)

Based on the discussions above, we present the algorithm on ADMM using inner MM iteration for solving the convex CL1-OGS-ATV model (24) shown as Algorithm 4.

Algorithm 4 CL1-OGS-ATV-ADM4 for the minimization problem (24)

initialization:

Starting point $v_{x}^{0} = v_{y}^{0} = g$ , k = 0, β₁, β₂, β₃, γ, μ, group size K × K,

$λ_{i}^{0} = 0$ , i = 1, 2, 3, 4, Maximum inner iterations NIt.

iteration:

1. Compute $v_{x}^{k + 1}$ and $v_{y}^{k + 1}$ according to (28) and (29).

2. Compute z^k+1 according to (30).

3. Compute w^k+1 according to (31).

4. Compute f^k+1 by solving (32).

5. update $λ_{i}^{0} = 0, i = 1, 2, 3, 4$ according to (33).

6. k = k + 1.

until a stopping criterion is satisfied.

CL1-OGS-ATV-ADM4 is a special form of general ADMM for the case with two blocks of variables (v_y, v_x, w, z) and f. If the Step (1) of Algorithm 4 can be solved exactly, the convergence for CL1-OGS-ATV-ADM4 can be guaranteed. In this case, if the relax parameter $γ \in (0, \frac{\sqrt{5} + 1}{2})$ , Algorithm 4 is convergent, more details can be referred to [29, 30, 32, 33]. Besides, although step (1) of Algorithm 4 can not be solved exactly, we can find a convergent series to ensure the convergence as [12]. Particularly, our numerical experiments verify the convergence of Algorithm 4.

Numerical Results

In this section, we present several numerical results to illustrate the performance of the proposed method. We compare our method CL1-OGS-ATV-ADM4 (“Ours” for short) with other state-of-the-art methods, Chan et al.’s ADM2CTVL1 proposed in [26] (“CTY” for short, Algorithm 2 in [26] for the constrained TV-L1 model), Guo et al.’s fast ℓ₁-TV proposed in [8] (“GLN” for short) and a high-order method derived by ourselves.

All experiments are carried out on Windows 7 32-bit and Matlab 2010a running on a desktop equipped with an Intel Core i3-2130 CPU with 3.4 GHz and 3.4 GB of RAM.

The quality of the restoration results is measured quantitatively by using the peak signal-to-noise ratio (PSNR) in decibel (dB) and the relative error (ReE): where $\overline{f}$ and f denote the original and restored images respectively, and Max_I represents the maximum possible pixel value of the image. In our experiments, Max_I = 1. The stopping criterion used in our work is set to be (34) where 𝓕^k is the objective function value of the respective model in the kth iteration, which is (35)

The stopping criterions of CTY (same as ours) and GLN are set to default as their literature mentioned.

All the test images are shown in Fig 1, seven 256-by-256 images as: (a) Cameraman.tif, (b) Satellite.pgm, (c) House.png, (d) Boat.pgm, (e) Barbara.tiff, (f) Einstein.pgm, (g) Peppers.png and one 460-by-460 image (h) Weatherstation.tif For the sake of simplicity, the pixel values in all of our tests are lied in [0, 1] which have been explained above.

Download:

Fig 1. Original images.

Top row: from left to right, (a) Cameraman, (b) Satellite, (c) House, (d) Boat. Bottom row: from left to right, (e) Barbara, (f) Einstein, (g) Peppers, (h) Weatherstation.

https://doi.org/10.1371/journal.pone.0122562.g001

Remark 1. The image “House” is downloaded from http://sipi.usc.edu/database/database.php?volume=misc&image=5top. The image “Satellite” is from “restore tools” downloaded from http://www.mathcs.emory.edu/~nagy/RestoreTools/. The images “Cameraman”, “Boat”, “Barbara”, “Einstein” and “Peppers” are from http://decsai.ugr.es/cvg/dbimagenes/. The image “Weatherstation” is photographed by one of the authors–Jun Liu in 2011, which can be found from https://www.flickr.com/photos/98566316@N06/14092821600/in/photostream/. The versions of the images in our paper are other special formats which are converted by Photoshop from the sources above.

We set the penalty parameters β₁ = 1, β₂ = 500, β₃ = 1, and relax parameter γ = 1.618 throughout all the experiments. Three blur kernels are generated by Matlab built-in function (i) fspecial(‘gaussian’, 7, 5) for 7 × 7 Gaussian blur with standard deviation 5, (ii) fspecial(‘gaussian’, 15, 5) for 15 × 15 Gaussian blur with standard deviation 5 and (iii) fspecial(‘average’, 7) for 7 × 7 average blur. We generate all blurring effects using the Matlab built-in function imfilter(I, psf, ‘circular’, ‘conv’) under periodic boundary conditions with “I” the original image and “psf” the blur kernel. We generate all noise effects by Matlab built-in function imnoise(B, ‘salt & pepper’, level) with “B” the blurred image and fix the same random matrix for different methods. We only consider the salt-and-pepper noise in our experiments, since the variation method is easy to extend to the random value noise case.

Study on the rest parameters

Firstly, we set the group size parameter K = 3 to find a good maximum inner iterations NIt. Our experiments are on the image “Cameraman” blurred by Gaussian blur kernel with 7 × 7 and standard deviation 5 and corrupted by 40% salt-and-pepper noise. The results are shown in Table 1. From Table 1, we can choose maximum inner iterations NIt = 5 for the best. Here we find that the larger the inner iteration NIt (> 5) is, the lower PSNR values are, which is a drawback for the selection of NIt. This is because we do not tune the other parameters to be best for different NIt. After we test more similar tests, and due to the CPU time and little error between different selections of NIt, we choose NIt = 5 for balance with the following experiments. Then we fix NIt = 5 and repeat more experiments for choosing a good group size parameter K. We operate the three 256-by-256 images (a) “Cameraman”, (b) “Satellite”, and (c) “House” for this best option of parameter K. The results are shown in Fig 2. From the figure, we can see that K = 3 is better for all the tests both on CPU time and PSNR. Therefore, we fix that NIt = 5 and K = 3 for balance in our work.

Download:

Table 1. PSNR (dB) and time (s) depending on maximum inner iterations NIt on the image “Cameraman” with Gaussian blur kernel 7 × 7 and standard deviation 5 and 40% salt-and-pepper noise.

https://doi.org/10.1371/journal.pone.0122562.t001

Download:

Fig 2. Results of our proposed method depending on group size parameter K.

The test images (“Cameraman”, “Satellite”, “House”) are blurred by Gaussian blur kernel with 7 × 7 and standard deviation 5 and corrupted by 40% salt-and-pepper noise. From left to right, results on CPU time, PSNR, ReE respectively.

https://doi.org/10.1371/journal.pone.0122562.g002

Then, we test how to select a good regularization parameter μ for different images. We will point out several important advantages of our method in the following experiments. For the sake of simplicity, we focus on the above three test 256-by-256 images (a) (b) and (c). Under the Gaussian blur with 7 × 7 window size and standard deviation 5, the images corrupted by added salt-and-pepper noise from 30% to 60% are tested. In Fig 3, we plot PSNR, ReE, and Time for our algorithm against different values of the regularization parameter μ. Each row in Fig 3 corresponds to the four salt-and-pepper noise levels. In fact, for all μ, our method always gives high PSNR values. Moreover, the PSNR curves of our method are very flat, which shows that our method is stable for a wide range of μ, which is wider than that in [26]. That is to say, our method is more robust than the method in [26]. This is the first advantage of our method.

Download:

Fig 3. Results of our proposed method depending on regularization parameter μ.

The tests are on the images “Cameraman”, “Satellite”, “House” that are blurred by 7 × 7 Gaussian blur kernel with standard deviation 5 and corrupted by salt-and-pepper noise from 30% to 60%. From left to right, results on PSNR, ReE, CPU time respectively. From top to bottom, results on noise level 30%, 40%, 50%, 60% respectively.

https://doi.org/10.1371/journal.pone.0122562.g003

In addition, as far as we know, a good image restoration algorithm should satisfy the following two properties. a) It is fast and can reach good results in term of both numerical values and high visual quality. b) It is not sensitive to parameters. Our method meets the requirement of these two properties, as it obtains good restoration results with the same parameters for different images under the same blur and noise. This is the second advantage of our method. Here and in the following experiments, under the same blur and noise, we choose the same parameters for all the test images. Particularly, for the images under the Gaussian blur with 7 × 7 window size and standard deviation 5 and corrupted by salt-and-pepper noise from 30% to 60%, we set μ = 100, 80, 60, 40 respectively. After similar tests as Fig 3, we list all the selection rule of μ: for the images under the Gaussian blur with 15-by-15 window size and standard deviation 5 and corrupted by salt-and-pepper noise from 30% to 60%, μ = 120, 110, 100, 90 respectively; for the images under the average blur with 7 × 7 window size and corrupted by salt-and-pepper noise from 30% to 60%, μ = 100, 80, 60, 40 respectively.

Comparison with CTY and GLN for the test image “Cameraman”

In this subsection, we mainly compare our proposed method to CTY and GLN for deblurring problems under salt-and-pepper noise. We use image “Cameraman” for the experiments in this subsection. Our purposes are (1) to show the improvement of PSNR and to demonstrate the efficiency of our proposed method mainly via a comparison with CTY and GLN, and (2) to illustrate that our proposed method can overcome the staircase effects effectively and get better visual quality than CTY and GLN, which is the third advantage of our method.

Firstly, we generate the blurred images by two Gaussian blurs (i) and (ii) with periodic boundary conditions as mentioned above, and then corrupt the blurred images by salt-and-pepper noise from 30% to 60%. For CTY and GLN, we have tuned the parameters manually to give the best PSNR improvement. The numerical results by the three methods are shown in Table 2. From the table, we see that both our proposed method and CTY are much faster and can get higher PSNR than GLN. Our proposed method needs the fewest iterations than the other two methods, and the time is always close to CTY. Particularly, the iterations of the GLN method always reach the maximum number of iterations which we set to be 200.

Download:

Table 2. Numerical comparison of the fast ℓ₁-TV method (GLN) [8], the ADM2CTVL1 method (CTY) [26], and our proposed method (Ours) under two Gaussian blurs (Bls) (i) fspecial(‘gaussian’, 7, 5) and (ii) fspecial(‘gaussian’, 15, 5) and corrupted by salt-and-pepper noise from 30% to 60%.

https://doi.org/10.1371/journal.pone.0122562.t002

We also show the images restored by the three methods. We display the degraded images and the restored images by three methods under two Gaussian blurs (i) and (ii) and 50% level of noise. The results are show in Fig 4. We can easily see the third advantage of our proposed method that our method can overcome the staircase effects effectively and get better visual quality than others. Moreover, we also plot the evolution of the PSNR over time and iterations for the three different methods in Fig 5 for the image blurred by 7 × 7 Gaussian blur and corrupted by 40% level salt-and-pepper noise.

Download:

Fig 4. Degraded and restored images of CTY, GLN and our method.

Left column: blurred and noisy images under 7 × 7 and 15 × 15 Gaussian blur with standard deviation 5 and corrupted by 50% salt-and-pepper noise. Right columns: restored images by GLN, CTY, and our proposed method respectively. The PSNR results can be found clearly from Table 2.

https://doi.org/10.1371/journal.pone.0122562.g004

Download:

Fig 5. PSNR over time and iterations results of GLN, CTY, and our proposed method.

Restoration of the “Cameraman” image under 7 × 7 Gaussion blur with standard deviation 5 and corrupted by 40% salt-and-pepper noise: evolution of the PSNR over time and iterations of GLN, CTY, and our proposed method.

https://doi.org/10.1371/journal.pone.0122562.g005

From the expriments and the description in [8], the GLN method has three sensitive parameters that depend on blur, noise level and test images rather than only one sensitive regular parameter μ in CTY and our proposed method. Besides, the results of GLN is nearly same as FTVd, while CTY is much better than FTVd in [26]. From the above tests, we also observe that both our proposed method and CTY can get better results than GLN. Moerever, the staircase effects by GLN method is also existent. Therefore, we omit the following comparison with GLN and only list the comparison with CTY.

Remark 2. Here and in the following tests for CTY, we tune the regularization parameter μ to be optimal by checking the highest PSNR and the lowest ReE under a “for loop” of Matlab code from 1 to 70 by step length 1 for different images under different blurs and noise levels. Besides, in the experimentsunder the 15 × 15 Gaussion blur with standard deviation 5, we find that if we increase the inner penalty parameter β₂ we will get higher PSNR. We do not change the parameter in our proposed method because of the good properties of a good restoration algorithm introduced above. However, in Table 2 for the CTY method under 15 × 15 Gaussion blur, we set the inner penalty parameter β₂ = 50 instead of β₂ = 20 in [26] for higher PSNR. For other tests following we also set this inner penalty parameter of CTY to be as default (β₂ = 20) in [26].

Comparison with CTY for other test images

In this subsection, we focus on comparisons between our proposed method and CTY for deblurring problems under salt-and-pepper noise. Fifty-six degraded test images are generated in the way similar to that in the last subsection. That is, we first generated the blurred images operating on images (b)-(h) with the periodic boundary condition by two blurs Gaussian blur (i) (also as G) and average blur (iii) (also as A), then corrupted the blurred images by salt-and-pepper noise from 30% to 60%. The parameters of our proposed method are set as above description and the parameters of CTY as Remark 2.

Conclusions similar to those in the last subsection can be made based on the results in Table 3 and Table 4. For example, our proposed method is always more accurate, with a possible improvement of more than 2.90 dB in PSNR (see image (c) with Gaussian blur and a 30% level of noise). For all images, the lower the noise level is, the better the improvement of PSNR will be. Even in high noise level, our method is more accurate than CTY by sacrificing partial time. For a further step, the iterations of our method are fewer than CTY for almost all test.

Download:

Table 3. Numerical comparison of CTY and our proposed method under 7 × 7 Gaussian blur with standard deviation 5 and corrupted by salt-and-pepper noise from 30% to 60%.

https://doi.org/10.1371/journal.pone.0122562.t003

Download:

Table 4. Numerical comparison of CTY and our proposed method under 7 × 7 average blur and corrupted by salt-and-pepper noise from 30% to 60%.

https://doi.org/10.1371/journal.pone.0122562.t004

Finally, in Fig 6 and Fig 7, we display the zoom parts of the degraded images examples and the zoom parts of the restored test images by two methods respectively for Gaussian blur and average blur with noise level from 30% to 60%. We can easily see the visual improvement in the images by using our method. More specifically, from the third column in Fig 7, although the numerical result does not improve too much, the image visual quality of our proposed method is much better than CTY. This superiority is obvious for almost all the test images in our work.

Download:

Fig 6. Several random examples of degraded and restored images of CTY and our method under Gaussian blur.

Top row, zoom parts of blurred and noisy images under 7 × 7 Gaussian blur with standard deviation 5 and corrupted by salt-and-pepper noise. Second row, zoom parts of restored images by CTY respectively. Third row, zoom parts of restored images by our proposed method respectively. The PSNR results can be found clearly from Table 3.

https://doi.org/10.1371/journal.pone.0122562.g006

Download:

Fig 7. Several random examples of degraded and restored images of CTY and our method under average blur.

Top row, zoom parts of blurred and noisy images under 7 × 7 average blur and corrupted by salt-and-pepper noise. Second row, zoom parts of restored images by CTY respectively. Third row, zoom parts of restored images by our proposed method respectively. The PSNR results can be found clearly from Table 4.

https://doi.org/10.1371/journal.pone.0122562.g007

Comparison with other TV-based methods

For image restoration under Gaussian noise, there has been many literature for high order TV or other TV-based methods, such as Lysaker-Lundervold-Tai (LLT) model [36], Wu-Tai’s work [13], total generalized variation [39]. However, to our knowledge, the methods for image deblurring and denoising under impulse noise with these TV-based methods are still missing in the literature. More recently, Liu-Huang-Liu in [27] proposed a hybrid model for image deblurring and denoising under impulse noise, but their method might spend more time due to the combination.

In order to get a fair comparison of our method and other TV-based methods, we take the high-order TV model by changing the ℓ₂-fidelity term in LLT to ℓ₁ (“HOTV” for short) and the total generalized variation method by changing the ℓ₂-fidelity term in [39] to ℓ₁ (“TGV” for short) as examples. Other TV-based methods can be similarly treated. Moreover, we also impose the box constraint in these examples and solve them by ADMM to compare them with our method under Gaussian blur (i) and 30% to 40% impulse noise for diffident images. We only list the results for three images, “Cameraman”, “House” and “Weatherstation” for examples. The codes for HOTV and TGV are projected by ourselves based on ADMM. Furthermore, we think that the superiority of our method can be seen clearly from these tests.

The numerical results are summarised in Table 5. The numerical improvement by our method is obvious from the table. Moreover, TGV usually performs better than HOTV but spends more time. Particularly, the time of each iteration by HOTV is less than other two methods ours and TGV. In Fig 8, we display the zoom parts of the degraded images examples and the zoom parts of the restored test images respectively for Gaussian blur with noise level 40% by three methods. We can see the visual improvement in the images by using our method. In addition, after test more experiments, we find that TGV preforms better than our method on the smooth region of the image, and our method performs better on the edges.

Download:

Table 5. Numerical comparison of our method, TGV and HOTV under Gaussian blur (i) and 30% to 40% impulse noise.

https://doi.org/10.1371/journal.pone.0122562.t005

Download:

Fig 8. Degraded and restored images of TGV, HOTV and our method.

Left column: zoom parts of blurred and noisy image. Second column: zoom parts of restored images by TGV. Third column: zoom parts of restored images by HOTV. Right column: zoom parts of restored images by proposed.

https://doi.org/10.1371/journal.pone.0122562.g008

Remark 3. In this work, we do not compare our method with other two stage methods such as Cai et. al [23] and Yan [24], because our method is a variational method by imposing a new regularization term. Our contribution is that we proposed a new convex model for handling the impulse noise without distinction of salt-and-pepper noise or random-value noise. We could easily apply our new regularization term in the former two stage methods and would consider it in future.

Discussion and Conclusion

In this work, we study a new regularization model by applying TV with OGS in the classic ℓ₁-TV model for the image deblurring under impulse noise. We provided the efficient algorithm CL1-OGS-ATV-ADM4 under the framework of general ADMM. In particular, an MM inner iteration is proposed to solve the subproblem instead of Shrinkage [20] in the classic ℓ₁-TV model. The numerical results illustrate that our method outperforms CTY [26], GLN [8] and some other TV-based methods both in terms of the PSNR values, ReE, iterations or image visual quality. Our main contributions are firstly combining three existent parts together in one convex model for image deblurring under impulse noise without distinction of salt-and-pepper noise or random-value noise. Furthermore, we could easily apply our new regularization term in the popular two stage methods and would consider it in future.

Acknowledgments

The authors would like to thank Prof. M. Tao for providing us the code ADM2CTVL1 (CTY) in [26]. Also, they would like to thank the referees and the academic editor for their careful reviews and helpful comments.

Author Contributions

Conceived and designed the experiments: GL. Performed the experiments: GL TZH JL XGL. Analyzed the data: GL TZH JL. Contributed reagents/materials/analysis tools: GL TZH JL XGL. Wrote the paper: GL. Designed the software used in analysis: GL TZH JL XGL. Revised the manuscript: GL TZH XGL.

References

1. Karayiannis N, Venetsanopoulos A (1990) Regularization theory in image restoration-the stabilizing functional approach. IEEE Transactions on Acoustics, Speech and Signal Processing 38: 1155–1179.
- View Article
- Google Scholar
2. Tikhonov A, Goncharsky A, Stepanov V, Yagola AG (1995) Numerical methods for the solution of ill-posed problems. 328.Netherlands: Springer, 254 pp.
3. Hansen PC, Nagy JG, O’Leary DP (2006) Deblurring images: matrices, spectra, and filtering. Society for Industrial and Applied Mathematics, 131 pp.
- View Article
- Google Scholar
4. Phillips DL (1962) A technique for the numerical solution of certain integral equations of the first kind. J ACM 9: 84–97.
- View Article
- Google Scholar
5. Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Mathematics Doklady 4: 1035–1038.
- View Article
- Google Scholar
6. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena 60: 259–268.
- View Article
- Google Scholar
7. Chambolle A (2004) An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20: 89–97.
- View Article
- Google Scholar
8. Guo X, Li F, Ng M (2009) A fast ℓ1–TV algorithm for image restoration. SIAM Journal on Scientific Computing 31: 2322–2341.
- View Article
- Google Scholar
9. Wang Y, Yang J, Yin W, Zhang Y (2008) A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences 1: 248–272.
- View Article
- Google Scholar
10. Yang J, Yin W, Zhang Y, Wang Y (2009) A fast algorithm for edge–preserving variational multichannel image restoration. SIAM Journal on Imaging Sciences 2: 569–592.
- View Article
- Google Scholar
11. Esser E (2009) Applications of Lagrangian–based alternating direction methods and connections to split Bregman. UCLA CAM report: 9–31.
12. Eckstein J, Bertsekas D (1992) On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming 55: 293–318.
- View Article
- Google Scholar
13. Wu C, Tai X (2010) Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences 3: 300–339.
- View Article
- Google Scholar
14. Goldstein T, O’Donoghue B, Setzer S (2012) Fast alternating direction optimization methods. UCLA CAM report: 12–35.
15. Chan T, Golub G, Mulet P (1999) A nonlinear primal–dual method for total variation–based image restoration. SIAM Journal on Scientific Computing 20: 1964–1977.
- View Article
- Google Scholar
16. Cai J, Osher S, Shen Z (2010) Split Bregman methods and frame based image restoration. Multiscale Modeling & Simulation 8: 337–369.
- View Article
- Google Scholar
17. Alliney S, Ruzinsky S (1994) An algorithm for the minimization of mixed l1 and l2 norms with application to bayesian estimation. IEEE Transactions on Signal Processing 42: 618–627.
- View Article
- Google Scholar
18. Nikolova M (2004) A variational approach to remove outliers and impulse noise. Journal of Mathematical Imaging and Vision 20: 99–120.
- View Article
- Google Scholar
19. Yin W, Goldfarb D, Osher S (2007) The total variation regularized l¹ model for multiscale decomposition. Multiscale Modeling & Simulation 6: 190–211.
- View Article
- Google Scholar
20. Yang J, Zhang Y, Yin W (2009) An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM Journal on Scientific Computing 31: 2842–2865.
- View Article
- Google Scholar
21. Clason C, Jin B, Kunisch K (2010) A duality–based splitting method for l¹–tvimage restoration with automatic regularization parameter choice. SIAM Journal on Scientific Computing 32: 1484–1505.
- View Article
- Google Scholar
22. Dong Y, Hintermüller M, Neri M (2009) An efficient primal–dual method for L¹TV image restoration. SIAM Journal on Imaging Sciences 2: 1168–1189.
- View Article
- Google Scholar
23. Cai J, Chan R, Nikolova M (2008) Two–phase approach for deblurring images corrupted by impulse plus Gaussian noise. Inverse Problems and Imaging 2: 187–204.
- View Article
- Google Scholar
24. Yan M (2013) Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting. SIAM Journal on Imaging Sciences 6: 1227–1245.
- View Article
- Google Scholar
25. Wu C, Zhang J, Tai X (2011) Augmented Lagrangian method for total variation restoration with non–quadratic fidelity. Inverse Problems and Imaging 5: 237–261.
- View Article
- Google Scholar
26. Chan R, Tao M, Yuan X (2013) Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers. SIAM Journal on Imaging Sciences 6: 680–697.
- View Article
- Google Scholar
27. Liu G, Huang TZ, Liu J (2014) High–order TVL1–based images restoration and spatially adapted regularization parameter selection. Computers & Mathematics with Applications 67: 2015–2026.
- View Article
- Google Scholar
28. Nocedal J, Wright SJ (2006) Numerical optimization. 1431–8598. New York: Springer, 664 pp.
29. Gabay D, Mercier B (1976) A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Computers & Mathematics with Applications 2: 17–40.
- View Article
- Google Scholar
30. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 3: 1–122.
- View Article
- Google Scholar
31. Afonso M, Bioucas–Dias J, Figueiredo M (2011) An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Transactions on Image Processing 20: 681–695. pmid:20840899
- View Article
- PubMed/NCBI
- Google Scholar
32. Glowinski R (1984) Numerical methods for nonlinear variational problems. 1434–8322. Berlin Heidelberg: Springer, 493 pp.
33. He B, Yang H (1998) Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Operations Research Letters 23: 151–161.
- View Article
- Google Scholar
34. Chambolle A, Lions PL (1997) Image recovery via total variation minimization and related problems. Numerische Mathematik 76: 167–188.
- View Article
- Google Scholar
35. Chan T, Marquina A, Mulet P (2000) High–order total variation–based image restoration. SIAM Journal on Scientific Computing 22: 503–516.
- View Article
- Google Scholar
36. Lysaker M, Lundervold A, Tai XC (2003) Noise removal using fourth–order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Transactions on Image Processing 12: 1579–1590. pmid:18244712
- View Article
- PubMed/NCBI
- Google Scholar
37. Lysaker M, Tai XC (2006) Iterative image restoration combining total variation minimization and a second–order functional. International Journal of Computer Vision 66: 5–18.
- View Article
- Google Scholar
38. Steidl G (2006) A note on the dual treatment of higher–order regularization functionals. Computing 76: 135–148.
- View Article
- Google Scholar
39. Bredies K, Kunisch K, Pock T (2010) Total generalized variation. SIAM Journal on Imaging Sciences 3: 492–526.
- View Article
- Google Scholar
40. Peyré G, Fadili J (2011) Group sparsity with overlapping partition functions. In: EUSIPCO. Barcelona, Spain, pp. 303–307.
- View Article
- Google Scholar
41. Selesnick I, Chen PY (2013) Total variation denoising with overlapping group sparsity. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5696–5700.
42. Figueiredo M, Bioucas–Dias J, Nowak R (2007) Majorization—minimization algorithms for wavelet–based image restoration. IEEE Transactions on Image Processing 16: 2980–2991. pmid:18092597
- View Article
- PubMed/NCBI
- Google Scholar
43. Oliveira JP, Bioucas–Dias JM, Figueiredo MA (2009) Adaptive total variation image deblurring: a majorization—minimization approach. Signal Processing 89: 1683–1693.
- View Article
- Google Scholar
44. Chen PY, Selesnick IW (2014) Translation–invariant shrinkage/thresholding of group sparse signals. Signal Processing 94: 476–489.
- View Article
- Google Scholar
45. Liu G, Huang TZ, Lv XG, Liu J (2013) New explicit thresholding/shrinkage formulas for one class of regularization problems with overlapping group sparsity and their applications. ArXiv e–prints: 1312.6813.

[ref1] 1. Karayiannis N, Venetsanopoulos A (1990) Regularization theory in image restoration-the stabilizing functional approach. IEEE Transactions on Acoustics, Speech and Signal Processing 38: 1155–1179.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Tikhonov A, Goncharsky A, Stepanov V, Yagola AG (1995) Numerical methods for the solution of ill-posed problems. 328.Netherlands: Springer, 254 pp.

[ref3] 3. Hansen PC, Nagy JG, O’Leary DP (2006) Deblurring images: matrices, spectra, and filtering. Society for Industrial and Applied Mathematics, 131 pp.
View Article
Google Scholar

[6] View Article

[7] Google Scholar

[ref4] 4. Phillips DL (1962) A technique for the numerical solution of certain integral equations of the first kind. J ACM 9: 84–97.
View Article
Google Scholar

[9] View Article

[10] Google Scholar

[ref5] 5. Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Mathematics Doklady 4: 1035–1038.
View Article
Google Scholar

[12] View Article

[13] Google Scholar

[ref6] 6. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena 60: 259–268.
View Article
Google Scholar

[15] View Article

[16] Google Scholar

[ref7] 7. Chambolle A (2004) An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20: 89–97.
View Article
Google Scholar

[18] View Article

[19] Google Scholar

[ref8] 8. Guo X, Li F, Ng M (2009) A fast ℓ1–TV algorithm for image restoration. SIAM Journal on Scientific Computing 31: 2322–2341.
View Article
Google Scholar

[21] View Article

[22] Google Scholar

[ref9] 9. Wang Y, Yang J, Yin W, Zhang Y (2008) A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences 1: 248–272.
View Article
Google Scholar

[24] View Article

[25] Google Scholar

[ref10] 10. Yang J, Yin W, Zhang Y, Wang Y (2009) A fast algorithm for edge–preserving variational multichannel image restoration. SIAM Journal on Imaging Sciences 2: 569–592.
View Article
Google Scholar

[27] View Article

[28] Google Scholar

[ref11] 11. Esser E (2009) Applications of Lagrangian–based alternating direction methods and connections to split Bregman. UCLA CAM report: 9–31.

[ref12] 12. Eckstein J, Bertsekas D (1992) On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming 55: 293–318.
View Article
Google Scholar

[31] View Article

[32] Google Scholar

[ref13] 13. Wu C, Tai X (2010) Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences 3: 300–339.
View Article
Google Scholar

[34] View Article

[35] Google Scholar

[ref14] 14. Goldstein T, O’Donoghue B, Setzer S (2012) Fast alternating direction optimization methods. UCLA CAM report: 12–35.

[ref15] 15. Chan T, Golub G, Mulet P (1999) A nonlinear primal–dual method for total variation–based image restoration. SIAM Journal on Scientific Computing 20: 1964–1977.
View Article
Google Scholar

[38] View Article

[39] Google Scholar

[ref16] 16. Cai J, Osher S, Shen Z (2010) Split Bregman methods and frame based image restoration. Multiscale Modeling & Simulation 8: 337–369.
View Article
Google Scholar

[41] View Article

[42] Google Scholar

[ref17] 17. Alliney S, Ruzinsky S (1994) An algorithm for the minimization of mixed l1 and l2 norms with application to bayesian estimation. IEEE Transactions on Signal Processing 42: 618–627.
View Article
Google Scholar

[44] View Article

[45] Google Scholar

[ref18] 18. Nikolova M (2004) A variational approach to remove outliers and impulse noise. Journal of Mathematical Imaging and Vision 20: 99–120.
View Article
Google Scholar

[47] View Article

[48] Google Scholar

[ref19] 19. Yin W, Goldfarb D, Osher S (2007) The total variation regularized l¹ model for multiscale decomposition. Multiscale Modeling & Simulation 6: 190–211.
View Article
Google Scholar

[50] View Article

[51] Google Scholar

[ref20] 20. Yang J, Zhang Y, Yin W (2009) An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM Journal on Scientific Computing 31: 2842–2865.
View Article
Google Scholar

[53] View Article

[54] Google Scholar

[ref21] 21. Clason C, Jin B, Kunisch K (2010) A duality–based splitting method for l¹–tvimage restoration with automatic regularization parameter choice. SIAM Journal on Scientific Computing 32: 1484–1505.
View Article
Google Scholar

[56] View Article

[57] Google Scholar

[ref22] 22. Dong Y, Hintermüller M, Neri M (2009) An efficient primal–dual method for L¹TV image restoration. SIAM Journal on Imaging Sciences 2: 1168–1189.
View Article
Google Scholar

[59] View Article

[60] Google Scholar

[ref23] 23. Cai J, Chan R, Nikolova M (2008) Two–phase approach for deblurring images corrupted by impulse plus Gaussian noise. Inverse Problems and Imaging 2: 187–204.
View Article
Google Scholar

[62] View Article

[63] Google Scholar

[ref24] 24. Yan M (2013) Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting. SIAM Journal on Imaging Sciences 6: 1227–1245.
View Article
Google Scholar

[65] View Article

[66] Google Scholar

[ref25] 25. Wu C, Zhang J, Tai X (2011) Augmented Lagrangian method for total variation restoration with non–quadratic fidelity. Inverse Problems and Imaging 5: 237–261.
View Article
Google Scholar

[68] View Article

[69] Google Scholar

[ref26] 26. Chan R, Tao M, Yuan X (2013) Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers. SIAM Journal on Imaging Sciences 6: 680–697.
View Article
Google Scholar

[71] View Article

[72] Google Scholar

[ref27] 27. Liu G, Huang TZ, Liu J (2014) High–order TVL1–based images restoration and spatially adapted regularization parameter selection. Computers & Mathematics with Applications 67: 2015–2026.
View Article
Google Scholar

[74] View Article

[75] Google Scholar

[ref28] 28. Nocedal J, Wright SJ (2006) Numerical optimization. 1431–8598. New York: Springer, 664 pp.

[ref29] 29. Gabay D, Mercier B (1976) A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Computers & Mathematics with Applications 2: 17–40.
View Article
Google Scholar

[78] View Article

[79] Google Scholar

[ref30] 30. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning 3: 1–122.
View Article
Google Scholar

[81] View Article

[82] Google Scholar

[ref31] 31. Afonso M, Bioucas–Dias J, Figueiredo M (2011) An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Transactions on Image Processing 20: 681–695. pmid:20840899
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref32] 32. Glowinski R (1984) Numerical methods for nonlinear variational problems. 1434–8322. Berlin Heidelberg: Springer, 493 pp.

[ref33] 33. He B, Yang H (1998) Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Operations Research Letters 23: 151–161.
View Article
Google Scholar

[89] View Article

[90] Google Scholar

[ref34] 34. Chambolle A, Lions PL (1997) Image recovery via total variation minimization and related problems. Numerische Mathematik 76: 167–188.
View Article
Google Scholar

[92] View Article

[93] Google Scholar

[ref35] 35. Chan T, Marquina A, Mulet P (2000) High–order total variation–based image restoration. SIAM Journal on Scientific Computing 22: 503–516.
View Article
Google Scholar

[95] View Article

[96] Google Scholar

[ref36] 36. Lysaker M, Lundervold A, Tai XC (2003) Noise removal using fourth–order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Transactions on Image Processing 12: 1579–1590. pmid:18244712
View Article
PubMed/NCBI
Google Scholar

[98] View Article

[99] PubMed/NCBI

[100] Google Scholar

[ref37] 37. Lysaker M, Tai XC (2006) Iterative image restoration combining total variation minimization and a second–order functional. International Journal of Computer Vision 66: 5–18.
View Article
Google Scholar

[102] View Article

[103] Google Scholar

[ref38] 38. Steidl G (2006) A note on the dual treatment of higher–order regularization functionals. Computing 76: 135–148.
View Article
Google Scholar

[105] View Article

[106] Google Scholar

[ref39] 39. Bredies K, Kunisch K, Pock T (2010) Total generalized variation. SIAM Journal on Imaging Sciences 3: 492–526.
View Article
Google Scholar

[108] View Article

[109] Google Scholar

[ref40] 40. Peyré G, Fadili J (2011) Group sparsity with overlapping partition functions. In: EUSIPCO. Barcelona, Spain, pp. 303–307.
View Article
Google Scholar

[111] View Article

[112] Google Scholar

[ref41] 41. Selesnick I, Chen PY (2013) Total variation denoising with overlapping group sparsity. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5696–5700.

[ref42] 42. Figueiredo M, Bioucas–Dias J, Nowak R (2007) Majorization—minimization algorithms for wavelet–based image restoration. IEEE Transactions on Image Processing 16: 2980–2991. pmid:18092597
View Article
PubMed/NCBI
Google Scholar

[115] View Article

[116] PubMed/NCBI

[117] Google Scholar

[ref43] 43. Oliveira JP, Bioucas–Dias JM, Figueiredo MA (2009) Adaptive total variation image deblurring: a majorization—minimization approach. Signal Processing 89: 1683–1693.
View Article
Google Scholar

[119] View Article

[120] Google Scholar

[ref44] 44. Chen PY, Selesnick IW (2014) Translation–invariant shrinkage/thresholding of group sparse signals. Signal Processing 94: 476–489.
View Article
Google Scholar

[122] View Article

[123] Google Scholar

[ref45] 45. Liu G, Huang TZ, Lv XG, Liu J (2013) New explicit thresholding/shrinkage formulas for one class of regularization problems with overlapping group sparsity and their applications. ArXiv e–prints: 1312.6813.

Figures

Abstract

Introduction

Some Preliminaries

OGS-TV

The MM method

Variable splitting and ADMM

Proposed Method

Numerical Results

Study on the rest parameters

Comparison with CTY and GLN for the test image “Cameraman”

Comparison with CTY for other test images

Comparison with other TV-based methods

Discussion and Conclusion

Acknowledgments

Author Contributions

References