http://orcid.org/0000-0001-8586-9511
Figures
Abstract
Speckle reduction remains a critical issue for ultrasound image processing and analysis. The nonlocal means (NLM) filter has recently attached much attention due to its competitive despeckling performance. However, the existing NLM methods usually determine the similarity between two patches by directly utilizing the gray-level information of the noisy image, which renders it difficult to represent the structural similarity of ultrasound images effectively. To address this problem, the NLM method based on the simple deep learning baseline named PCANet is proposed by introducing the intrinsic features of image patches extracted by this network rather than the pixel intensities into the pixel similarity computation. In this approach, the improved two-stage PCANet is proposed by using Parametric Rectified Linear Unit (PReLU) activation function instead of the binary hashing and block histograms in the original PCANet. This model is firstly trained on the ultrasound database to learn the convolution kernels. Then, the trained PCANet is utilized to extract the intrinsic features from the image patches in the pre-denoised version of the noisy image to be despeckled. These obtained features are concatenated together to determine the structural similarity between image patches in the NLM method, based on which the weighted mean of all pixels in a search window is computed to produce the final despeckled image. Extensive experiments have been conducted on a variety of images to demonstrate the superiority of the proposed method over several well-known despeckling algorithm and the PCANet based NLM method using ReLU function and sigmoid function. Visual inspection indicates that the proposed method outperforms the compared methods in reducing speckle noise and preserving image details. The quantitative comparisons show that among all the evaluated methods, our method produces the best structural similarity index metrics (SSIM) values for the synthetic image, as well as the highest equivalent number of looks (ENL) value for the simulated image and the clinical ultrasound images.
Citation: Yu H, Ding M, Zhang X, Wu J (2018) PCANet based nonlocal means method for speckle noise removal in ultrasound images. PLoS ONE 13(10): e0205390. https://doi.org/10.1371/journal.pone.0205390
Editor: Zhangyang Wang, Texas A&M University, XX
Received: November 2, 2017; Accepted: September 21, 2018; Published: October 12, 2018
Copyright: © 2018 Yu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work is supported by the National Key Research and Development Program of China (Grant No: 2017YFBI1303100), the National Natural Science Foundation of China (Grant No: 51679101) and Wuhan Science and Technology Project (Grant No: 2016060101010056). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Medical imaging plays a critical role in disease monitoring and diagnosis. Compared with other imaging techniques such as X-ray, CT and MRI, ultrasound imaging is a noninvasive, real-time and radiation-free imaging modality. However, the ultrasound images are inevitably corrupted by speckle noise due to the coherent imaging mechanism from the scatters [1]. Such a noise reduces the sharpness of image details and complicates the diagnosis of the tiny structure of lesions. Therefore, despeckling is of great significance for improving ultrasound image quality. Moreover, as a pre-processing step, denoising will also benefit image post-processing tasks such as image segmentation, classification and registration.
In ultrasound imaging, speckle noise has a random granular pattern. The distribution of speckle noise is signal dependent and is governed by Fisher-Tippett distribution [2] or Gamma distribution [3], which can be represented [4, 5] as:
(Eq 1)
where (x,y) is the pixel location, v(x,y) is the clean image, u(x,y) is the noisy image, η is a Gaussian noise distribution with zero-mean and variance σ2, and the factor γ is related to the ultrasound devices and additional processing. Extensive studies indicate that γ = 0.5 can be used to model speckle noise in the ultrasound images [4, 6].
Many methods have been developed for ultrasound images despeckling [7]. In general, the existing despeckling methods can be classified as the spatial domain based methods and the frequency domain based ones. The traditional spatial domain based methods, such as Frost filter [8], Kuan filter [9], squeeze box filter (SBF) [10] and speckle reducing anisotropic diffusion (SRAD) filter [11], are based on the local comparison of pixels. One disadvantage of these approaches is that they cannot deliver sufficient noise reduction while preserving image details effectively. For the frequency domain based methods, the most popular techniques are the wavelet based methods [12–14]. These methods work by transforming speckle noise into additive noise and then removing it within the wavelet domain. They tend to introduce the artifacts related to the choice of mother wavelet.
Recently, the nonlocal means (NLM) filter proposed by Buades et al. [15] is considered as a state-of-the-art algorithm for eliminating the additive noise. This method explores the self-similarities between image patches instead of individual pixels, and each image pixel is restored by the weighted average of all pixels in a search window. Nowadays, the NLM method has been widely applied to denoise the medical images such as MR images and CT images [16–18]. Despite the success in removing Gaussian noise, the traditional nonlocal means (TNLM) method by its very nature becomes unsuitable for speckle noise reduction since speckle noise is different from Gaussian noise significantly. To overcome this drawback, several modified NLM-based approaches have been presented [5, 19–22] for despeckling. Specifically, Zhan et al. [5] have proposed a weight refining method for speckle noise reduction in which the weights are calculated in the lower-dimensional principal component analysis (PCA) subspace. Coupe et al. [19] have introduced the optimized Bayesian nonlocal means (OBNLM) filter. In contrast to the TNLM, the OBNLM filter determines the similarity between two image patches based on the Pearson distance derived by the Bayesian framework instead of the Euclidean distance of TNLM approach. Yang et al. [21] have presented a hybrid despeckling approach which combines the NLM with the local statistics of noise (NLMLS). In this method, the local statistics of speckle noise are used to pre-filter the ultrasound image and the similarity is computed based on the pre-filtered image to produce the final denoised result. Two total variation (TV) model based NLM methods have been proposed by Dong et al. [22] to reduce the multiplicative noise. Compared with the classical TV-based methods, the nonlocal TV methods perform better in preserving image textures and repetitive structures. The above-mentioned methods as the extension of TNLM method have contributed to the development of ultrasound image restoration techniques.
However, most of the NLM-based despeckling methods depend on the utilization of gray-level information and hand-crafted features. These features are not sufficient to accurately represent the structural similarity between image patches in the ultrasound images. If the intrinsic features can be learned from the images of interest for structural similarity computation, the better despeckled results can be obtained.
Deep learning, as a popular algorithm within the research community of machine learning, can automatically learn the intrinsic features from the training data. Up to now, various deep learning models, such as deep belief network (DBN), stacked auto-encode (SAE), convolutional neural networks (CNN) and nonlocal deep network [23], have been proposed. Among these models, the CNN is very popular due to the use of convolutional architectures. In recent years, many CNN-based variations have been introduced and successfully applied to various visual tasks such as feature extraction, classification [24, 25], super-resolution [26], object detection [27, 28] and image denoising [29–31]. In particular, Zhang et al. [31] have proposed a denoising convolutional neural network (DnCNN) model for removing the additive white Gaussian noise (AWGN), in which a very deep convolutional architecture is constructed while residual learning strategy and batch normalization are integrated to speed up the training process. Although the DnCNN and other CNN-based denoising models exhibit the promising denoised results, they generally involve the following problems. The training of CNN network is very complicated because it involves numerous parameters to be learned and requires too much empirical knowledge and special skills in parameter setting. In addition, most of the CNN-based denoisers are specially designed for denoising AWGN, and they cannot handle speckle noise very well. Thus, how to construct a simple and effective learning network for ultrasonic speckle reduction will pose a great challenge.
More recently, the principal components analysis network (PCANet), a very simple unsupervised deep learning baseline introduced by Chan et al. [32], has been proposed for image classification and hand-written digit recognition. This network consists of three simple data processing components: cascaded PCA, binary hashing and blockwise histograms. In this network, the basic PCA is first employed for learning the multistage convolution kernels, and the subsequent binary hashing and the blockwise histogram are utilized to produce the output features. Compared with the CNN-based models, the training of the PCANet model is extremely simple and efficient because no any regularized parameters or numerical optimization solvers are required for the involved filter learning [32].
Due to the advantage of the PCANet, it will be of significance to introduce this model into the NLM method to characterize structural similarity of image patches by extracting the robust intrinsic features from the ultrasound images. However, the binary hashing used in the PCANet may lead to the loss of useful feature information. To overcome the problem, we will present an improved PCANet in which an activation function Parametric Rectified Linear Unit (PReLU) [33, 34] is used instead of the binary hashing and block histograms at the output stage to extract the intrinsic features. To test the restoration performance of the proposed method, the extensive experiments on the synthetic image, the simulated image and the real ultrasound images are performed to make the comparisons among the proposed method and other despeckling methods. The experimental results demonstrate the superiority of the proposed method in speckle reduction and image detail preservation.
2 Methodology
2.1 The modified PCANet model
The PCANet model is a simple and valuable baseline for image classification and recognition tasks. The intrinsic features can be effectively extracted through three processing layers including the PCA-based convolutional layer, the binary hashing-based nonlinear layer and the histograms-based output layer. These processing methods used in the nonlinear layer and the output layer, which are similar to sparse representation strategies [35, 36], will result in the loss of image features. To overcome the disadvantage, we propose a modified PCANet, which consists of three processing components: the two convolutional layers and the output layer. The detailed structure of this network is given as follows.
A. The convolutional layer.
Let the number of training images be N, the patch size be k1×k2 at all stages. For the p-th training image, all the patches around each pixel are collected step by step. For each patch in this training image, its mean is subtracted from the intensities of all pixels in this patch. Accordingly, the mean-removed matrix Ap = [ap,1,ap,2,…,ap,S] is produced for the whole image, where ap,s denotes the s-th mean-removed vectorized patch, and S is the number of patches produced from the p-th image. By processing all the training images in the same way, the matrix A is obtained as:
(Eq 2)
The PCA operation is then implemented on A to derive the convolution kernels. The reconstruction error within a family of orthonormal filters is minimized by the PCA algorithm, i.e.,
(Eq 3)
where L1 is the number of filters in the first layer,
is the identity matrix of size L1×L1, and ||·||F denotes the Frobenius norm. The solutions of Eq (3) are the L1 principal eigenvectors of AAT. Therefore, the PCA filters are expressed as:
(Eq 4)
where ql(AAT) represents the l-th principal eigenvector of AAT, and
is the function that maps ql(AAT) to the matrix
.
Similar to the deep neural networks (DNNs), the multiple stages of PCA filters can be stacked for extracting higher level features. All outputs of the first convolutional layer are utilized as the inputs of the second convolutional layer. By repeating almost the same process as in the first convolutional layer, the leading L2 eigenvectors are obtained as the filters in the second one.
B. The output layer.
For the p-th training image, there are L1 outputs from the first convolutional layer. Similarly, each input
will produce L2 corresponding outputs
from the second convolutional layer. In the original PCANet, all these L1×L2 outputs are binarized using the Heaviside step function while each group of L2 binary outputs is weighted summed to obtain L1 decimal-valued images. However, the previous analysis has shown that this operation will degrade the effectiveness of the extracted features. To ensure that all those features obtained from the input image can be kept as accurate as possible, the binary hashing and block histograms will be replaced by the PReLU function [34]. The reason of using PReLU function instead of ReLU and sigmoid functions is that it can help to preserve image details better by additionally utilizing the structural information carried by the negative entries. In this modified PCANet model, the PReLU will be utilized as the final output layer to map nonlinearity into the data to ensure the accuracy and robustness of the extracted intrinsic features. Here, the PReLU function is defined as:
(Eq 5)
When a = 0, the function degenerates into the Rectified Linear Unit (ReLU). If a is a very small fixed value, the PReLU will be reduced to the Leaky ReLU (LReLU). In this paper, a is fine-tuned as 0.25 to ensure the optimal despeckled results.
2.2 Feature extraction using the modified PCANet model
To realize robust intrinsic features extraction, the modified PCANet must be trained to obtain the convolution filter kernels. Considering that the PCANet uses the image patch based training strategy, we have chosen 200 general ultrasound images from the open ultrasound database [37] to train the PCANet. They consist of abdomen images, urinary tract images, pediatrics images, gynaecology images and musculo skeletal joints images, among which the number of each type of ultrasound images is 40. All the 200 images are cropped to be the size of 480×320 and pre-processed by the OBNLM filter.
By using the trained PCA filters, the modified PCANet can extract the intrinsic features. A flowchart is given to illustrate how the modified PCANet extracts the features from the considered patches of an input image. As shown in Fig 1, the original noisy ultrasound image is pre-processed by the OBNLM filter before input into the PCANet. The patches in the pre-processed ultrasound image are convoluted with the trained PCA filters to produce L1 feature maps in the first convolutional layer. Then, each feature map is convoluted with the trained PCA filters to generate L2 feature maps in the second convolutional layer. All the L1×L2 feature maps are processed by the PReLU function to produce the final outputs.
2.3 The modified PCANet based nonlocal means method
The main aim in this study is to improve the despeckling performance of NLM method. To refine the calculation of structural similarity between image patches in the NLM method, the proposed method exploits the robust intrinsic features extracted by the modified PCANet instead of the pixel intensities of the noisy image due to the strong feature learning abilities of PCANet. Fig 2 illustrates how to construct the feature vectors of size L for the computation of similarity weight, where L = L1×L2 is the number of feature images. The pixel intensities of the same location in a search window in all feature images are concatenated into a feature vector as shown in Fig 2. The structural similarity ω(i,j,m,n) between two image patches centered at (i,j) and (m,n) is computed based on the difference between the corresponding feature vectors, i.e.,
(Eq 6)
where h acts as a decay parameter controlling the degree of filtering, X(i,j) and X(m,n) are the concatenated feature vectors of image patches centered at (i,j) and (m,n), respectively. ||·||2,α is a Gaussian weighted Euclidean distance with α denoting the standard deviation of Gaussian function.
Based on the structural similarity ω(i,j,m,n), the restored intensity NLM[I(i,j)] at (i,j) in the noisy image I is determined as:
(Eq 7)
where Ω(i,j) is the search window centered at (i,j).
To further improve the denoised results, the structural similarity between image patches will be refined by using the image restored by the proposed method instead of the pre-filtered results produced by the OBNLM method. The refined structural similarity will help the NLM filter to provide better despeckling performance.
2.4 Implementation of the proposed method
The implementation of the proposed method is summarized in Fig 3, the detailed description is as follows.
Step 1: Pre-processing.
The original noisy image is pre-filtered by the OBNLM filter to produce the pre-processed image. The noise standard deviation is estimated, and the decay parameter is set as suggested in [38].
Step 2: Generation of the feature images.
The pre-filtered image is input into the modified PCANet which has been trained using the open ultrasound database [37] to generate L1×L2 feature images as the outputs of this network.
Step 3: Computation of similarity weight.
For each considered image patch in the original noisy image, the corresponding feature vector is constructed by using the feature images obtained in Step 2. The produced feature vectors are employed to compute the structural similarity between two image patches via Eq (6).
Step 4: Image restoration.
Based on the similarity weight and the pre-fixed decay parameter, each pixel in the noisy image is restored by the NLM method based on Eq (7).
Step 5: Refinement of similarity weight.
The restored image obtained in Step 4 is input into the modified PCANet to produce L1×L2 feature images, and then they are used to compute the structural similarity via Eq (6).
Step 6: Output of the final despeckled image.
Based on the derived similarity weight in Step 5, the final despeckled image is produced by Eq (7).
3 Experimental results and discussion
In this section, the proposed method is tested on the synthetic image, the simulated image and the real ultrasound images. To demonstrate the superiority of the proposed PCANet based NLM method (PPCA-NLM), it will be compared with such traditional well-known despeckling algorithms as Frost, Kuan, SBF, SRAD, TNLM, OBNLM and NLMLS. Meanwhile, the proposed method will be also compared with the DnCNN method and the NLM methods using the original PCANet model (OPCA-NLM), the ReLU-based PCANet model (RPCA-NLM) and the sigmoid-based PCANet model (SPCA-NLM). In all experiments, the window size of Frost and Kuan filters is fixed to be 3×3. For SBF and SRAD filters, the parameters are fine-tuned as referred in [10, 11]. The sizes of similarity window and search window are set as 7×7 and 17×17 in the TNLM, OBNLM, NLMLS and PCANet based NLM filters, respectively. For the DnCNN denoiser, the same database is used as the PCANet to train the DnCNN model based on Eq (1), and the parameters and network structures are set as suggested in [31]. For these NLM-based filters, the decay parameter is determined using the rule-of-thumb, i.e., h = β·σ, where β and σ denote a predefined constant and the noise standard deviation [38], respectively. In these PCANet based NLM methods, the number of PCA filters in the two convolutional layers is fixed to be 12 and the patch size is 7×7, respectively.
3.1 The synthetic image
The experiment is conducted on the synthetic image corrupted by various levels of speckle noise with σ = 3, 4, 5, and 6, which is simulated based on Eq (1).
Fig 4 shows the learned feature images by different activation functions based PCANet on the synthetic image with σ = 3. It can be seen from Fig 4 that the PReLU based PCANet model is more effective in learning the different features from the noisy image, which can facilitate the accurate computation of structural similarity between image patches.
The top row, the second row, the third row and the bottom row show the learned feature images by the original PCANet, ReLU-based PCANet, sigmoid-based PCANet and PReLU-based PCANet models, respectively.
Fig 5 presents a visual comparison of restored results for the different filters operating on the synthetic image. Obviously, speckle noise cannot be suppressed effectively by Frost and Kuan filter. In contrast, the SBF and SRAD filters perform better in noise removal, but they cause the blurring of image details. The evaluated NLM-based filters deliver sufficient speckle reduction. However, the TNLM and OBNLM filters produce the artifacts as shown in Fig 5(G) and 5(H), while the NLMLS filter damages some small image structure although it can reduce the artifacts to some extent. The DnCNN filter can facilitate enhancing the image contrast, but significant speckle noise and artifacts are observed in Fig 5(J). For the despeckling results based on PCANet methods, one can notice that there are obvious artifacts in Fig 5(K) and 5(L), whereas useful structure information cannot be preserved in Fig 5(M). By comparison, the PPCA-NLM method has better performance in removing speckle noise and avoiding artifacts.
(a) original image, (b) synthetic image with speckle noise (σ = 3), (c) Frost filter, (d) Kuan filter, (e) SBF filter, (f) SRAD filter, (g) TNLM filter, (h) OBNLM filter, (i) NLMLS filter, (j) DnCNN filter, (k) OPCA-NLM filter, (l) RPCA-NLM filter, (m) SPCA-NLM filter, and (n) PPCA-NLM filter.
Furthermore, the zoomed views of a region of interest (ROI) from Fig 5 are shown in Fig 6 to demonstrate the advantage of the proposed method in preserving the small structures and edges. Clearly, Fig 6(C)–6(G) obtained with the Frost, Kuan, SBF, SRAD, and TNLM filters are unsatisfactory since these methods cannot provide good edge preservation. In Fig 6(H)–6(J), it can be seen that the OBNLM and NLMLS methods blur the edges and details of the rectangle, lines and circular point, while the DnCNN denoiser tends to distort the small structures and produce the obvious artifacts in the smooth region. Similarly, one can see from Fig 6(K)–6(M) that the OPCA-NLM and RPCA-NLM filters generate the blurred edges, while the SPCA-NLM filter causes serious damage to the image details. Compared with the other evaluated filters, the PPCA-NLM method is the most effective for preserving the fine structures and maintaining the edge sharpness in the image.
(a) Original ROI, (b) Noisy ROI, (c) Frost filter, (d) Kuan filter, (e) SBF filter, (f) SRAD filter, (g) TNLM filter, (h) OBNLM filter, (i) NLMLS filter, (j) DnCNN filter, (k) OPCA-NLM filter, (l) RPCA-NLM filter, (m) SPCA-NLM filter, and (n) PPCA-NLM filter.
The quantitative evaluations are also made among all the tested methods. Two well-known evaluation indexes as peak signal-to-noise ratio (PSNR) and structural similarity index metrics (SSIM) [39] are used for performance appreciation, which are defined as:
(Eq 8)
(Eq 9)
where W and H represent the width and height of the image, respectively. u is the noise-free image and
is the denoised image.
and μu are the mean intensity of images
and u, respectively.
is the covariance between images
and u,
and σu are the standard deviations of images
and u, respectively. C1 and C2 are the small constants to stabilize SSIM.
Tables 1 and 2 list the PSNR and SSIM values for all evaluated methods operating on the synthetic image corrupted with different levels of speckle noise, where the best values are marked in bold. It can be seen that the DnCNN can produce the maximum PSNR at all noise levels, and the PPCA-NLM method ranks the second. However, the proposed method provides significantly higher SSIM values than the DnCNN. The overall evaluation based on the above-mentioned visual inspection and the objective indexes demonstrates that the PPCA-NLM method exhibits the excellent despeckling performance for the synthetic image.
3.2 The simulated image
A more challenging and relevant ultrasound image has been generated for the “cyst” phantom based on FieldⅡ simulation, which is a program to simulate ultrasound transducer fields and ultrasound imaging using the linear acoustics. The “cyst” phantom consists of a collection of point targets, five cyst regions, and five highly scattering regions [40]. Fig 7 presents the “cyst” phantom, the simulated image, and the despeckled results of all the tested methods. The Frost, Kuan and SBF filters keep much noise in the images as shown in Fig 7(C)–7(E). The SRAD filter performs well in noise removal, but it leads to the blurry edges and the staircase effect in the despeckled image. All NLM-based methods smooth out speckle noise more effectively than the above-mentioned local filters. However, the TNLM and NLMLS filters cannot maintain the sharpness of point targets and the OBNLM method produces the obvious artifacts. For the DnCNN method, many unwanted artifacts can be observed in the smooth region as shown in Fig 7(J). From Fig 7(K)–7(M), one can see that the OPCA-NLM and RPCA-NLM filters tend to retain speckle noise, while the SPCA-NLM filter produces an over-smoothed image. The comparison with the other eleven methods shows that the PPCA-NLM method can smooth out speckle noise well while providing the clearer boundaries of points as depicted in Fig 7(N).
(a) the “cyst” phantom, (b) simulated image generated by FieldⅡ and four ROIs marked with various colors, (c) Frost filter, (d) Kuan filter, (e) SBF filter, (f) SRAD filter, (g) TNLM filter, (h) OBNLM filter, (i) NLMLS filter, (j) DnCNN filter, (k) OPCA-NLM filter, (l) RPCA-NLM filter, (m) SPCA-NLM filter, and (n) PPCA-NLM filter.
Moreover, two other widely used evaluation indexes, i.e., equivalent number of looks (ENL) and contrast-to-noise ratio (CNR) [41] are utilized to quantitatively appreciate the despeckling performance, which are defined as:
(Eq 10)
(Eq 11)
where μb and μo are the mean intensity of background area and object area, σb and σo are the standard deviation of background area and object area, respectively.
Four pairs of ROIs are selected to evaluate the despeckling performance for these tested methods, and the ROIs are marked with different colors as shown in Fig 7(B). The ENL and CNR are listed in Tables 3 and 4, respectively. As regards ENL, the PPCA-NLM method provides the maximum values at all ROIs. Likewise, the proposed method performs the best for three ROIs except that its CNR is slightly less than that of the RPCA-NLM method for the blue ROI. The performance appreciation based on ENL and CNR confirms that the PPCA-NLM method is superior to other compared methods.
3.3 The real ultrasound images
Three real ultrasound images are also used to further assess the effectiveness of the proposed method. Fig 8 shows the despeckled results for the compared methods on a real ultrasound image of benign lymph nodes. Visually, the Frost and Kuan filters maintain much speckle in the image as shown in Fig 8(B) and 8(C), while the SBF and SRAD filters result in the blurry object boundaries as shown in Fig 8(D) and 8(E). Compared with the TNLM and NLMLS filters, the PCANet based NLM filters are more effective in smoothing out speckle noise. However, the SPCA-NLM filter damages many image details. For the OBNLM and NLMLS filters, we can see from Fig 8(G) and 8(H) that they tend to generate the artifacts in some regions. By comparison, the PPCA-NLM method can preserve the sharpness of node boundaries better than the OBNLM and DnCNN methods. In contrast to the despeckled results of the OPCA-NLM and RPCA-NLM methods, the image details can be preserved with higher sharpness by the PPCA-NLM filter. The experiments on the real ultrasound image demonstrate the superiority of the proposed method to the other despeckling methods.
(a) original noisy image and four ROIs marked with the different colors, (b) Frost filter, (c) Kuan filter, (d) SBF filter, (e) SRAD filter, (f) TNLM filter, (g) OBNLM filter, (h) NLMLS filter, (i) DnCNN filter, (j) OPCA-NLM filter, (k) RPCA-NLM filter, (l) SPCA-NLM filter, and (m) PPCA-NLM filter.
The quantitative results of four ROIs shown in Fig 8(A) are listed in Tables 5 and 6. With regard to ENL results, the PPCA-NLM method produces the maximum values at three ROIs among all the filters. In terms of CNR, the OPCA-NLM method provides the competitive results, followed by the PPCA-NLM and DnCNN methods. The above quantitative comparisons and visual inspection illustrate that the PPCA-NLM method is very suitable and practicable for despeckling the real ultrasound image.
Besides, two other clinical ultrasound images including the fetal image and the parotid gland image are used to further display the visual impression for the TNLM, OBNLM, NLMLS, DnCNN and PPCA-NLM filters. As indicated in Figs 9 and 10, the PPCA-NLM method outperforms the compared methods in that it not only effectively removes speckle noise but also enhances the sharpness of boundaries and retains image structures very well.
(a) original image, (b) TNLM filter, (c) OBNLM filter, (d) NLMLS filter, (e) DnCNN filter, and (f) PPCA-NLM filter.
(a) original image, (b) TNLM filter, (c) OBNLM filter, (d) NLMLS filter, (e) DnCNN filter, and (f) PPCA-NLM filter.
4 Conclusion
In this paper, the modified PCANet based deep learning baseline is introduced into NLM method for reducing speckle noise in the ultrasound images. This proposed method utilizes the intrinsic features extracted from an input noisy image by the PCANet to refine the similarity computation in the NLM method. The introduction of the intrinsic features instead of the gray-level information into the NLM method can facilitate the effective despeckling of ultrasound images due to its effectiveness in representing their structural information. The experimental results demonstrate that the proposed method outperforms the other compared methods in speckle reduction and image detail preservation. Therefore, the proposed approach has the great potential application to ultrasound-based clinical diagnosis. In future, the new deep learning models will be developed to denoise the images corrupted by Rician noise or Poisson noise.
References
- 1. Zhang F, Yoo Y M, Koh L M, Kim Y. Nonlinear diffusion in Laplacian pyramid domain for ultrasonic speckle reduction. IEEE Transactions on Medical Imaging. 2007; 26(2): 200–211. https://doi.org/10.1109/TMI.2006.889735 pmid:17304734
- 2. Slabaugh G, Unal G, Fang T, Wels M. Ultrasound-specific segmentation via decorrelation and statistical region-based active contours. In: 2006 IEEE Conference on Computer Vision and Pattern Recognition. 2006; p. 45–53. https://doi.org/10.1109/CVPR.2006.318
- 3. Tao Z, Tagare H D, Beaty J D. Evaluation of four probability distribution models for speckle in clinical cardiac ultrasound images. IEEE Transactions on Medical Imaging. 2006; 25(11): 1483–1491. https://doi.org/10.1109/TMI.2006.881376 pmid:17117777
- 4. Coupe P, Hellier P, Kervrann C, Barillot C. Nonlocal means-based speckle filtering for ultrasound images. IEEE Transactions on Image Processing. 2009; 18(10): 2221–2229. https://doi.org/10.1109/TIP.2009.2024064 pmid:19482578
- 5. Zhan Y, Ding M, Wu L, Zhang X. Nonlocal means method using weight refining for despeckling of ultrasound images. Signal Processing. 2014; 103: 201–213. https://doi.org/10.1016/j.sigpro.2013.12.019
- 6. Loupas T, McDicken W N, Allan P L. An adaptive weighted median filter for speckle suppression in medical ultrasonic images. IEEE Transactions on Circuits and Systems. 1989; 36(1): 129–135. https://doi.org/10.1109/31.16577
- 7. Kaur A, Ranade S K. Review of speckle noise reduction techniques for ultrasound imaging. Imperial Journal of Interdisciplinary Research. 2016; 2(9): 29–35.
- 8. Frost V S, Stiles J A, Shanmugan K S, Holtzman J C. A model for radar images and its application to adaptive digital filtering of multiplicative noise. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1982; 4(2): 157–166. http://doi.org/10.1109/TPAMI.1982.4767223 pmid:21869022
- 9. Kuan D, Sawchuk A, Strand T, Chavel P. Adaptive restoration of images with speckle. IEEE Transactions on Acoustics, Speech, and Signal Processing. 1987; 35(3): 373–383. https://doi.org/10.1109/TASSP.1987.1165131
- 10. Tay P C, Garson C D, Acton S T, Hossack J A. Ultrasound despeckling for contrast enhancement. IEEE Transactions on Image Processing. 2010; 19(7): 1847–1860. https://doi.org/10.1109/TIP.2010.2044962 pmid:20227984
- 11. Yu Y, Acton S T. Speckle reducing anisotropic diffusion. IEEE Transactions on Image Processing. 2002; 11(11): 1260–1270. https://doi.org/10.1109/TIP.2002.804276 pmid:18249696
- 12. Zaki F, Wang Y, Su H, Yuan X, Liu X. Noise adaptive wavelet thresholding for speckle noise removal in optical coherence tomography. Biomedical Optics Express. 2017; 8(5): 2720–2731. https://doi.org/10.1364/BOE.8.002720 pmid:28663901
- 13. Pizurica A, Philips W, Lemahieu I, Acheroy M. A versatile wavelet domain noise filtration technique for medical imaging. IEEE Transactions on Medical Imaging. 2003; 22(3): 323–331. https://doi.org/10.1109/TMI.2003.809588 pmid:12760550
- 14. Rahman M M, Azim M, Mina, Uddin S. Speckle noise reduction in ultrasound images by wavelet thresholding based on subband mean difference. International Journal of Tomography and Statistics. 2012; 20(2): 91–97.
- 15. Buades A, Coll B, Morel J M. A non-local algorithm for image denoising. In: 2005 IEEE Conference on Computer Vision and Pattern Recognition. 2005; p. 60–65. http://doi.org/10.1109/CVPR.2005.38
- 16. Singh C, Ranade S. K, Singh K. Invariant moments and transform-based unbiased nonlocal means for denoising of MR images. Biomedical Signal Processing and Control. 2016; 30: 13–24. https://doi.org/10.1016/j.bspc.2016.05.007
- 17. Zeng D, Huang J, Zhang H, Bian Z, Niu S, Zhang Z, et al. Spectral CT image restoration via an average image-induced nonlocal means filter. IEEE Transactions on Biomedical Engineering. 2016; 63(5): 1044–1057. https://doi.org/10.1109/TBME.2015.2476371 pmid:26353358
- 18. Zhang X, Hou G, Ma J, Yang W, Lin B, Xu Y, et al. Denoising MR images using non-local means filter with combined patch and pixel similarity. PloS ONE. 2014; 9(6): e100240. https://doi.org/10.1371/journal.pone.0100240 pmid:24933024
- 19. Coupe P, Hellier P, Kervrann C, Barillot C. Nonlocal means-based speckle filtering for ultrasound images. IEEE Transactions on Image Processing. 2009; 18(10): 2221–2229. https://doi.org/10.1109/TIP.2009.2024064 pmid:19482578
- 20. Wen T, Gu J, Li L, Qin W, Wang L, Xie Y. Nonlocal total-variation-based speckle filtering for ultrasound images. Ultrasonic Imaging. 2015; 38(4): 254–275. https://doi.org/10.1177/0161734615600676 pmid:26316172
- 21. Yang J, Fan J, Ai D, Wang X, Zheng Y, Tang S, et al. Local statistics and non-local mean filter for speckle noise reduction in medical ultrasound image. Neurocomputing. 2016; 195: 88–95. https://doi.org/10.1016/j.neucom.2015.05.140
- 22. Dong F, Zhang H, Kong D. Nonlocal total variation models for multiplicative noise removal using split Bregman iteration. Mathematical and Computer Modelling. 2012; 55(3): 939–954. https://doi.org/10.1016/j.mcm.2011.09.021
- 23. Wang X, Girshick R, Gupta A, He K. Non-local neural networks. [Online]. 2017; Available: https://arxiv.org/abs/1711.07971
- 24. Zieliński B, Plichta A, Misztal K, Spurek P, Brzychczy-Włoch M, Ochońska D. Deep learning approach to bacterial colony classification. PloS ONE. 2017; 12(9): e0184554. https://doi.org/10.1371/journal.pone.0184554 pmid:28910352
- 25. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2015. p. 1–9. https://doi.org/10.1109/CVPR.2015.7298594
- 26. Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2016. p. 1646–1654. https://doi.org/10.1109/CVPR.2016.182
- 27. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014; p. 580–587. http://doi.org/10.1109/CVPR.2014.81
- 28. Girshick R. Fast R-CNN. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015; p. 1440–1448. http://doi.org/10.1109/ICCV.2015.169
- 29. Du W, Chen H, Wu Z, Sun H, Liao P, Zhang Y. Stacked competitive networks for noise reduction in low-dose CT. PloS ONE. 2017; 12(12): e0190069. https://doi.org/10.1371/journal.pone.0190069 pmid:29267360
- 30. Divakar N, Babu R V. Image denoising via CNNs: an adversarial approach. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017; p. 1076–1083. http://doi.org/10.1109/CVPRW.2017.145
- 31. Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing. 2017; 26(7): 3142–3155. https://doi.org/10.1109/TIP.2017.2662206 pmid:28166495
- 32. Chan T H, Jia K, Gao S, Lu J, Zeng Z, Ma Y. PCANet: A simple deep learning baseline for image classification? IEEE Transactions on Image Processing. 2015; 24(12): 5017–5032. https://doi.org/10.1109/TIP.2015.2475625 pmid:26340772
- 33. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. In: the 25th International Conference on Neural Information Processing Systems. 2012; p. 1097–1105. https://doi.org/10.1145/3065386
- 34. He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: the IEEE International Conference on Computer Vision. 2015; p. 1026–1034. https://doi.org/10.1109/ICCV.2015.123
- 35. Zhu P, Zhu W, Hu Q, Zhang C, Zuo W. Subspace clustering guided unsupervised feature selection. Pattern Recognition. 2017; 66 (C): 364–374. https://doi.org/10.1016/j.patcog.2017.01.016
- 36. Zhu P, Hu Q, Han Y, Zhang C, Du Y. Combining neighborhood separable subspaces for classification via sparsity regularized optimization. Information Sciences. 2016; s370–371: 270–287. https://doi.org/10.1016/j.ins.2016.08.004
- 37.
http://www.ultrasoundcases.info
- 38. Kervrann C, Boulanger J. Local adaptivity to variable smoothness for exemplar-based image regularization and representation. International Journal of Computer Vision. 2008; 79(1): 45–69. https://doi.org/10.1007/s11263-007-0096-2
- 39. Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 2004; 13(4): 600–612. https://doi.org/10.1109/TIP.2003.819861 pmid:15376593
- 40.
http://field-ii.dk/?examples/cyst_phantom/cyst_phantom.html
- 41. Szkulmowski M, Gorczynska I, Szlag D, Sylwestrzak M, Kowalczyk A, Wojtkowski M. Efficient reduction of speckle noise in optical coherence tomography. Optics Express. 2012; 20(2): 1337–1359. https://doi.org/10.1364/OE.20.001337 pmid:22274479