Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Blurred image restoration using knife-edge function and optimal window Wiener filtering

  • Min Wang ,

    Roles Data curation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Visualization, Writing – original draft, Writing – review & editing

    yu0801@163.com

    Affiliation College of Meteorology and Oceanography, National University of Defense Technology, Jiangsu Province, PR of China

  • Shudao Zhou,

    Roles Conceptualization, Formal analysis, Funding acquisition, Validation, Visualization, Writing – review & editing

    Affiliations College of Meteorology and Oceanography, National University of Defense Technology, Jiangsu Province, PR of China, Collaborative Innovation Center on Forecast and Evaluation of Meteorological Disasters, Nanjing University of Information Science and Technology, Jiangsu Province, PR of China

  • Wei Yan

    Roles Investigation, Resources, Supervision

    Affiliation College of Meteorology and Oceanography, National University of Defense Technology, Jiangsu Province, PR of China

Blurred image restoration using knife-edge function and optimal window Wiener filtering

  • Min Wang, 
  • Shudao Zhou, 
  • Wei Yan
PLOS
x

Abstract

Motion blur in images is usually modeled as the convolution of a point spread function (PSF) and the original image represented as pixel intensities. The knife-edge function can be used to model various types of motion-blurs, and hence it allows for the construction of a PSF and accurate estimation of the degradation function without knowledge of the specific degradation model. This paper addresses the problem of image restoration using a knife-edge function and optimal window Wiener filtering. In the proposed method, we first calculate the motion-blur parameters and construct the optimal window. Then, we use the detected knife-edge function to obtain the system degradation function. Finally, we perform Wiener filtering to obtain the restored image. Experiments show that the restored image has improved resolution and contrast parameters with clear details and no discernible ringing effects.

Introduction

Camera movements might result in motion blur in captured images. The blurred image is usually modeled as a convolution between the original image and a known point spread function (PSF). Image restoration techniques are used to remove or minimize known degradations in an image. There are several classical image restoration methods, such as the iterative Lucy–Richardson algorithm and the non-iterative Wiener algorithms [12]. Several complex methods such as the Bussgang algorithm [3] have also been proposed. In [4], an adaptive restoration method to adaptively correct retinal images is proposed. This is performed by using deconvolution to remove the residual wave-front aberrations and provide an improvement over the Wiener filter with respect to the quality of restoration. An efficient technique based on physical optics is presented in [5]. In this, space-variant blurs are restored by sectioning using modified Wiener filtering. In [6], an image reconstruction and restoration method using the simplified topological ε-algorithm is proposed. In [7], the generalized Hermitian and skew-Hermitian splitting (GHSS) iterative method is applied to the problem of image restoration.

A novel method is presented in [8] to estimate the depth map in an underwater scenario. The authors also describe a method to handle the presence of artificial illumination in underwater images. In [9], a novel Bayesian image restoration method that integrates Total Variation (TV) and Poisson singular integral (PSI) models is proposed. An image restoration method that combines the conditional gradient with the Tikhonov regularization technique is described in [10]. In [11], a new filtering process based on adaptive diffusion function in the image restoration is discussed. A fast and automatic method for finding the direction and size of the blur is presented in [12]. In [13], a new variational model for joint multiplicative denoising and deblurring is proposed, which combines a total generalized variation filter and shearlet transform. A two-level domain decomposition method for image restoration is proposed in [14], which consists of an overlapping domain decomposition technique and a coarse mesh correction. A method for image restoration is presented in [15], which can adaptively determine the optimal norms for both fidelity term and regularization term in the (SR) restoration model. In [16], a fast half-quadratic method for image restoration and reconstruction is proposed. A class of upper and lower triangular splitting iteration algorithm for image restoration is proposed in [17]. In [18], an image restoration method for fiber-coupled image using space-variant impulse response characterization is proposed. A nonconvex and nonsmooth total generalized variation (TGV) model for image restoration is presented in [19], which can provide an even sparser representation of the variation of the image function than the traditional TGV model that uses convex L1 norm to measure the variation. A new two-step iterative shrinkage/ thresholding method for image restoration is proposed in [20], which exhibiting much faster convergence rate than IST for ill-conditioned problems. In [21], an image restoration method based on multi-scale patch, which imposes the very same prior on different scale patches extracted from the target image. A variational method for joint optical flow estimation and edge-aware image restoration is presented in [22]. A new approach of multi-frame super-resolution using diffusion registration and a nonlocal variational image restoration is proposed in [23]. In [24], a new image restoration method based on new adaptive anisotropic diffusion function is proposed, which is able to overcome the drawbacks of the traditional process. A novel optimization method for image restoration based on quadratic data fitting and smooth non-quadratic regularization is presented in [25]. A novel Laplacian-based visibility restoration method for real-world hazy scenes to effectively solve inadequate haze thickness estimation and alleviate color cast problems is proposed in [26]. Two image restoration methods using simultaneous sparse coding (SSC) and nonlocally centralized sparse representation are proposed in [27] and [28], respectively. In [29], an edge-preserving image restoration is proposed, which combines a masking operator that prevents wraparound artifacts.

In this paper, we present an image restoration technique based on the knife-edge function and optimal window Wiener filtering. First, we use the Prewitt edge detection operator and autocorrelation function to calculate the direction and scale of the motion-blur. Following this, we add the optimal window to the edge extension image. Then, we use the detected knife-edge function to obtain the system degradation function. Finally, we use Wiener filtering to obtain the restored image and truncate the border.

Experimental evaluations show that this method is effective in restoring images. The resolution and contrast parameters are improved, details are clear, and no ringing effects are observed. The proposed method exhibits superior performance in comparison with other techniques on the basis of metrics like the gray mean grads (GMG), peak signal-noise ratio (PSNR), and improvement signal-to-noise rate (ISNR).

The rest of the paper is organized as follows. Section 2 describes the principles of motion-blurred image degradation, restoration, and optimal window construction. Section 3 explains the restoration method proposed in this paper. Section 4 describes the experimental evaluation results. Section 5 concludes this paper.

Theoretical foundations

The principles of image restoration

The digital representation of a photographic image consists of a matrix that has information about the intensity levels of the individual pixels. In the linear space translation, a blurred image can be described using a two-dimensional convolution model, as shown in Fig 1.

(1)

Here, g(x,y) represents the blurred image, f(x,y) refers to the original image, h(x,y) is the PSF or degradation function, and n(x,y) is random noise caused by the camera sensor. The notation “*” represents the convolution operation. On applying Fourier transformation to the above equation, we obtain the blurred model expression in the frequency domain based on the convolution theorem. (2) where G(u,v), F(u,v), H(u,v), and N(u,v) are the two dimensional Fourier transformations of g(x,y), f(x,y), h(x,y), and n(x,y), respectively, and the operator “·” represents element-wise multiplication.

The PSF is determined by the length (L) and angle (θ) of the camera motion. Hence, we can construct a PSF or filtering function H(u,v) to restore the original image f(x,y). (3) The Wiener filter uses the follow filtering function. (4) where |H(u,v)|2 = H*(u,v)H(u,v). H*(u,v) is the conjugate complex of H(u,v), and k is the constant signal-to-noise ratio chosen from 0.0001 to 0.01.

Wiener filtering takes into consideration the degradation function and noise statistical features, but it can produce the edge error.

The principles of optimal window Wiener filtering

Optimal window Wiener filtering is used to suppress the edge error [30], as shown in Fig 2 and Fig 3. We calculate the window function ω(x,y) of each image region based on the amount of motion experienced by that segment. For optimal window regional range and coordinates parameter values, see Table 1. The window function ω(x,y) is used as the weighting factor while performing a Fourier transform of the motion-blurred image g(x,y).

(5)(6)
thumbnail
Table 1. Optimal window regional range and coordinates parameter values.

https://doi.org/10.1371/journal.pone.0191833.t001

The steps followed in the motion-blurred image restoration procedure using optimal window Wiener filtering are given below.

  1. Compute the Fourier transformation of the windowed motion-blurred image g(x,y)*ω(x,y).
  2. Assign PSF h(m,n) and compute its Fourier transformation H(u,v).
  3. Compute the inverse Fourier transformation of to obtain the restored image.

Wiener filtering using an optimal window can prevent edge errors effectively, but this technique also has its drawbacks. There is a L-shaped strip, at the right bottom corner, of width PSFH and height PSFV, that exists due to the incompleteness of the image data. In this paper, we consider a two-dimensional extension to the edge of the image, such that the extension of width and height is consistent with the motion component PSFH and PSFV. After performing the optimal window Wiener filtering, we truncate the edge extensions.

Determination of PSF

According to the degradation model, the key to image restoration is understanding and estimating the PSF or H(u,v) of the imaging system accurately. From Eq (4), if H(u,v) is known, we can calculate , and consequently f(x,y). In general, the PSF or H(u,v) is determined using prior knowledge or experimental information.

In real-life applications, the PSF is determined according to the image and motion features. It is difficult to obtain the clear point and line objects for a motion-blurred image. However, it is easy to obtain the straight edge image, and consequently the knife-edge function, of the object through edge detection. The knife-edge function has many advantages. First, there is no need for prior knowledge of the specific degradation model of the motion-blurred image, as we can use the knife-edge function to construct PSF and estimate the degradation function. Second, the knife-edge function is not sensitive to any specific form of motion-blurring, and hence is suitable for modeling various types of motion-blur.

A brief summary of the steps followed in the PSF computation process is given below.

  1. Compute the direction of blurring, theta, by using the Prewitt operator to detect the motion trajectory of the edge feature.
  2. Rotate the image such that the motion-blurred direction is parallel to the horizontal direction (x-axis).
  3. Locate the maximum point of in the horizontal direction and assign this as the center pixel of the knife-edge function.
  4. Use the center pixel point to construct the knife-edge function.
  5. Compute the differential of the knife-edge function to get the line spread function (LSF) of the degradation system such that the PSF is the LSF along the x-axis direction.
  6. Fourier transform the PSF to obtain the system degradation function H(u,v).

In order to improve the detection accuracy of the knife-edge function, we can detect the knife-edge function along several edges and compute the average gray value of the corresponding position.

Proposed image restoration technique

The algorithm of the proposal presented in this paper is as follows (S1 Fig):

  1. Use the method in [31] to eliminate noise from the image.
  2. Compute the blurred direction θ and length L, and obtain the horizontal PSFH and vertical PSFV motion components by decomposing θ and L.
  3. Extend the edge to obtain the extension image of .
  4. Add the optimal window to the edge extension image.
  5. Detect the knife-edge function to get the system degradation function.
  6. Use Wiener filtering to get an image free of blurs, and truncate the border to obtain the restored image.

We use simulations to evaluate the proposed algorithm on sample datasets from two application scenarios. The images in the first dataset consist of images with horizontal motion-blur (L = 30, θ = 0°), while the images in the second dataset are original and have motion-blur in an arbitrary direction (L = 30, θ = 45°). This is shown in Figs 4A, 5A and 5B. Fig 4G–4I) show the process and restored versions of the images in Fig 4A. Fig 5E–5G are the fused images for Fig 5A and 5B, respectively. The images shown in Fig 4G and 4H and Fig 5E–5G are listed below (S2 Fig).

  1. ○. The restoration result based on the proposed method (edge correction),
  2. ○. The restoration result based on rectangular PSF
  3. ○. The restoration result based on traditional Wiener filtering,
  4. ○. The restoration result based on traditional Lucy-Richardson, and
  5. ○. The restoration result based on traditional Blind-deconvolution.

Different methods are used to restore Figs 4 and 5 respectively, and the parameters obtained are shown in Figs 6 and 7.

thumbnail
Fig 4. The deblurred performances of different methods on the first test image.

(a) The motion-blurred simulation image; (b) the final restoration result based on the proposed technique after edge correction; (c) the restoration result based on rectangular PSF; (d) the restoration result based on traditional Wiener filtering; (e) the restoration result based on traditional Lucy-Richardson; (f) the restoration result based on traditional Blind-deconvolution.

https://doi.org/10.1371/journal.pone.0191833.g004

thumbnail
Fig 5. The deblurred performances of different methods on the second test image.

(a) The source image; (b) the motion-blurred simulation image; (c) the final restoration result based on the proposed technique; (d) the restoration result based on rectangular PSF; (e) the restoration result based on traditional Wiener filtering; (f) the restoration result based on traditional Lucy-Richardson; (g) the restoration result based on traditional Blind-deconvolution.

https://doi.org/10.1371/journal.pone.0191833.g005

We use three metrics (GMG, PSNR, and ISNR) to evaluate the restoration effect. In the case of the motion-blurred image whose original sharp image is not known (Fig 4), we use GMG to evaluate the restoration effect. The larger the GMG value, the clearer the image (S1 Table). In the case of the motion-blurred image for which we have clear image (Fig 5), we use PSNR and ISNR to evaluate the restoration effect. The larger the PSNR and ISNR values, the greater the improvement (S2 Table).

From Figs 6 and 7, we can see that the performance of the method proposed in this paper is better than that of other techniques. The resolution and contrast parameters are improved, the details are clear, and there is an overall improvement in the image quality.

Conclusion

This paper proposed an image restoration method using knife-edge function and optimal window Wiener filtering. We constructed a knife-edge function as the PSF to estimate the degradation function without prior knowledge of the specific degradation model. Experiments demonstrated the effectiveness of the proposed technique when compared with other restoration methods.

It is significant to explore the blurred image restoration method to improve the effects of the deblurring without knowledge of the specific degradation model. In the future, we plan to extend the proposed method by integrating more sampling and develop more advanced filtering rules.

Supporting information

S1 Fig. Schematic depiction of the algorithm design used in our method.

See Proposed image restoration technique for details of sample description.

https://doi.org/10.1371/journal.pone.0191833.s001

(TIF)

S2 Fig. Schematic depiction of the experimental design used in the current study.

See Proposed image restoration technique for details of sample description.

https://doi.org/10.1371/journal.pone.0191833.s002

(TIF)

S1 Table. GMG results with five methods of Fig 4.

GMG is one of evaluation metrics of restoration effect to the motion-blurred image whose original image is not known, calculated GMG of the restored images from different deblurring.

https://doi.org/10.1371/journal.pone.0191833.s003

(DOCX)

S2 Table. PSNR and ISNR results with five methods of Fig 5.

PSNR and ISNR is two of evaluation metrics of restoration effect to the motion-blurred image whose original sharp image is known, calculated PSNR and ISNR between the original and restored images after different deblurring.

https://doi.org/10.1371/journal.pone.0191833.s004

(DOCX)

Acknowledgments

We thank Prof. W. Li from Nanjing University for excellent technical assistance. We would like to thank Editage [www.editage.cn] for English language editing.

References

  1. 1. Yang HL, Huang PH, Lai SH. A novel gradient attenuation Richardson–Lucy algorithm for image motion deblurring. Signal Processing. 2014;103(10): 399–414.
  2. 2. Hillery AD, Chin RT. Iterative wiener filters for image restoration. Signal Processing IEEE Transactions on. 1991; 39(8):1892–1899.
  3. 3. Panci G, Campisi P, Colonnese S, Scarano G. Multichannel blind image deconvolution using the Bussgang algorithm: Spatial and multiresolution approaches. IEEE Transactions on Image Processing. 2003; 12(11):1324–1337. pmid:18244691
  4. 4. Christou JC, Roorda A, Williams DR. Deconvolution of adaptive optics retinal images. Journal of the Optical Society of America A Optics Image Science and Vision. 2004; 21(8): 1393–1401.
  5. 5. Costello TP, Mikhael WB. Efficient restoration of space-variant blurs from physical optics by sectioning with modified Wiener filtering. Digital Signal Processing. 2003; 13(1):1–22.
  6. 6. Gazzola S, Karapiperi A. Image reconstruction and restoration using the simplified topological ε-algorithm. Applied Mathematics and Computation. 2016; 274(2): 539–555.
  7. 7. Aghazadeh N, Bastani M, Salkuyeh DK. Generalized Hermitian and skew-Hermitian splitting iterative method for image restoration. Applied Mathematical Modelling. 2015; 39(20):6126–6138.
  8. 8. Galdran A, Pardo D, Picón A, Gila AA. Automatic Red-Channel underwater image restoration. Journal of Visual Communication and Image Representation.2015; 26(1): 132–145.
  9. 9. Ruiza P, Orozcob MH, Mateosa J, Villegas OOV, Molina R, Katsaggelos AK. Combining Poisson singular integral and total variation prior models in image restoration. Signal Processing. 2014; 103(10:296–308.
  10. 10. Bouhamidia A, Enkhbatb R, Jbilou K. Conditional gradient Tikhonov method for a convex optimization problem in image restoration. Journal of Computational and Applied Mathematics. 2014; 255(1):580–592.
  11. 11. Tebini S, Mbarki Z, Seddik H, Ben BE. Rapid and efficient image restoration technique based on new adaptive anisotropic diffusion function. Digital Signal Processing. 2016; 48(1): 201–215.
  12. 12. Dobeš M, Machala L, Fürst T. Blurred image restoration: a fast method of finding the motion length and angle. Digital Signal Processing. 2010; 20(12):1677–1686.
  13. 13. Ullah A, Chen W, Khan MA, Sun HG. A New Variational Approach for Multiplicative Noise and Blur Removal. Plos One. 2017; 12 (1):e0161787. pmid:28141802
  14. 14. Xu J, Tai XC, Wang LL. A two-level domain decomposition method for image restoration. Inverse Problems and Imaging. 2017;4 (3): 523–545.
  15. 15. Shen HF, Peng L, Yue LW, Yuan Q, Zhang LP. Adaptive Norm Selection for Regularized Image Restoration and Super-Resolution. IEEE Transactions on Cybernetics. 2017; 46 (6):1388–1399. pmid:26208375
  16. 16. Yang XJ, Wang L. Fast half-quadratic algorithm for image restoration and reconstruction. Applied Mathematical Modelling. 2017; 50(10):92–104.
  17. 17. Fan HT, Bastani M, Zheng B, Zhu XY. A class of upper and lower triangular splitting iteration methods for image restoration. Applied Mathematical Modelling. 2017;9
  18. 18. Motamedi N, Lomakin V, Ford JE. Image restoration in fiber-coupled imagers using space-variant impulse response characterization. Applied Optics. 2017; 56(14): 4003–4011. pmid:29047531
  19. 19. Zhang HL, Tang LM, Fang Z, Xiang CC, Li CY. Nonconvex and nonsmooth total generalized variation model for image restoration. Signal Processing. 2017;8:
  20. 20. Bioucasdias JM, Figueiredo MAT. A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing. 2007;16(12):2992–3005. pmid:18092598
  21. 21. Papyan V, Elad M. Multi-scale patch-based image restoration. IEEE Transactions on Image Processing. 2016; 25(1):249–261. pmid:26571527
  22. 22. Tu ZG, Xie W, Cao J, Veltkamp RC, Poppe R. Variational method for joint optical flow estimation and edge-aware image restoration. Pattern Recognition. 2017; 65(5):11–25.
  23. 23. Laghrib A, Ghazdali A, Hakim A, Raghay S. A multi-frame super-resolution using diffusion registration and a nonlocal variational image restoration. Computers and Mathematics with Applications. 2016; 72(9):2535–2548.
  24. 24. Tebini S, Mbarki Z, Seddik H, Braiek EB. Rapid and efficient image restoration technique based on new adaptive anisotropic diffusion function. Digital Signal Processing. 2016; 48(1):201–215.
  25. 25. Skariah DG, Arigovindan M. Nested conjugate gradient algorithm with nested preconditioning for non-linear image restoration. IEEE Transactions on Image Processing. 2017; 26(9): 4471–4482. pmid:28641260
  26. 26. Huang SC, Ye JH, Chen BH. An advanced single-image visibility restoration algorithm for real-world hazy scenes. IEEE Transactions on Industrial Electronics. 2015; 62(5):2962–2972.
  27. 27. Dong WS, Shi GM, Ma Y, Li X. Image restoration via simultaneous sparse coding: where structured sparsity meets gaussian scale mixture. International Journal of Computer Vision. 2015;114(2):217–232.
  28. 28. Dong WS, Zhang L, Shi GM, li X. Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing. 2013; 22(4):1620–1630. pmid:23269751
  29. 29. Matakos A, Ramani S, Fessler JA. Accelerated edge-preserving image restoration without boundary artifacts. IEEE Transactions on Image Processing. 2013; 22(5): 2019–2029. pmid:23372080
  30. 30. Lim H, Tan KC, Tan BTG. New methods for restoring motion-blurred images derived from edge error considerations. Cvgip Graphical Models & Image Processing, 1991; 53 (5):479–490.
  31. 31. Wang M, Li Z, Duan XJ, Li W. An image denoising method with enhancement of the directional features based on wavelet and SVD transforms. Mathematical Problems in Engineering, 2015;11:1–9.