https://orcid.org/0009-0000-7943-8432
Figures
Abstract
The Perona-Malik (P-M) model exhibits deficiencies such as noise amplification, new noise introduction, and significant gradient effects when processing noisy images. To address these issues, this paper proposes an image-denoising algorithm, ACE-GPM, which integrates an Automatic Color Equalization (ACE) algorithm with a gradient-adjusted P-M model. Initially, the ACE algorithm is employed to enhance the contrast of low-light images obscured by fog and noise. Subsequently, the Otsu method, a technique to find the optimal threshold based on between-class variance, is applied for precise segmentation, enabling more accurate identification of different regions within the image. After that, distinct gradients enhance the image’s foreground and background via an enhancement function that accentuates edge and detailed information. The denoising process is finalized by applying the gradient P-M model, employing a gradient descent approach to further emphasize image edges and details. Experimental evidence indicates that the proposed ACE-GPM algorithm not only elevates image contrast and eliminates noise more effectively than other denoising methods but also preserves image details and texture information, evidenced by an average increase of 0.42 in the information entropy value. Moreover, the proposed solution achieves these outcomes with reduced computational resource expenditures while maintaining high image quality.
Citation: Li W, Zhou G, Wang X (2024) Low illumination fog noise image denoising method based on ACE-GPM. PLoS ONE 19(5): e0302492. https://doi.org/10.1371/journal.pone.0302492
Editor: Sen Xiang, Wuhan University of Science and Technology, CHINA
Received: September 19, 2023; Accepted: April 5, 2024; Published: May 7, 2024
Copyright: © 2024 Li et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data files are available from the https://doi.org/10.6084/m9.figshare.25375222 All data files are available from the Github https://github.com/WuyiLi1999/Low_illumination_fog_noise_images.
Funding: This work was supported in part by Natural Science Foundation of Heilongjiang Province under Grant LH2020C048, and in part by the Harbin Science and Technology Innovation Talent Research Foundation under Grant 2017RAQXJ108." Funders played a vital role in supporting this research. Both funders were the main responsible persons for the project, contributing 10,0000 RMB each.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
With the rapid development of information technology and computer vision, the application scope of digital image processing is expanding daily. Uncrewed Aerial Vehicles (UAVs) and cameras are widely used to obtain desired images in image acquisition work. However, the collected images are inevitably disturbed by noise, resulting in blurred images and low brightness [1]. These noises can be classified according to their characteristics, such as additive noise, shot noise, and multiplicative noise [1].
Additive noise [1] refers to adding noise to the image unrelated to the original image. This noise will be directly superimposed on the pixel values of the original image, resulting in changes in the brightness or color of the image. Additive noise may originate from electromagnetic interference of electronic devices, sensor noise, and many other factors. The Additive noise is usually randomly distributed, and standard additive noise models include Gaussian noise salt and pepper noise [1–6].
Shot noise [6] is the noise caused by the limitation or defect of the optical system, which will affect the image’s clarity, contrast, and color accuracy. This noise is usually related to the image acquisition equipment, such as the camera lens, which may produce spots, color aberration, distortion, and other noise in the imaging process. The common shot noise is divided into aberration noise, chromatic aberration noise, defocus noise, halo noise, malformed noise, etc [2].
Multiplicative noise is a noise that affects the signal strength or energy of the image. Unlike additive noise, multiplicative noise can affect the overall brightness level of the image, reducing the contrast or loss of details in the image. Multiplicative noise is commonly seen in images under low light conditions, such as noise and graininess in images taken in the low light environment. Many types of multiplicative noise exist, including Rayleigh, Gamma, exponential, and fog [2].
Standard image noise denoising algorithms include filters based on spatial domain, wavelet threshold denoising based on wavelet domain [5, 6], image denoising based on Partial Differential Equation(PDE), image denoising based on Total variation (TV), and DnCNN denoising network model based on convolutional neural network. Filters based on the spatial domain are divided into mean filter, median filter, Gaussian filter, bilateral filter, and trilateral filter [2, 3].
In reference [2], a coupled Neural Network (PCNN) was combined with a regularized P-M equation to remove mixed noise in images. This method fully uses the biological heuristic characteristics of PCNN and introduces a threshold function to quickly locate the nonlinear noise information in the image. Then, the second-generation wavelet filter is used to suppress the positioning noise. Then, by regularizing the P-M model, the image’s smoothness can be well controlled, and the edge detail information can be preserved while the residual noise information is removed. However, this method is only suitable for processing grayscale images and performs poorly when dealing with noise with unknown levels.
In reference [3], researchers used a color compensation method based on the designed attenuation matrix to improve the color distortion of the underwater image by compensating the red and green channels for the blue channel. In order to improve the local contrast of the image, they used a finite Bihistogram (DHIT) method based on the Rayleigh distribution. The proposed method can effectively enhance the details of the image and generate global and local contrast-enhanced images. Researchers have proposed a multi-scale unsharp mask strategy to improve the visual quality and sharpen the fused image further. These steps successfully improved the contrast of underwater and low-light images. However, it should be noted that adopting the DHIT method introduces a certain computational complexity, which may lead to the loss of some details.
In reference [4], in order to solve the problem of the high computational complexity of the Non-local Means (NLM) algorithm, researchers introduced moment invariant-based clustering and the Hidden Markov Model (HMM) for pre-classification. The proposed method exploits the Gaussian white noise dependence in the pixel wavelet transform domain to increase the number of candidate blocks for non-local means filtering, thereby improving the computational performance and denoising effect. However, this method contains multiple computational steps, such as clustering, wavelet transform, and Hidden Markov Model (HMM), and thus requires high computational resources when dealing with large images. In addition, this method is highly dependent on the data and can only suppress the additive white noise, which cannot adaptively deal with the noise of different scenes.
In reference [5], an image processing method based on minimum color loss and local adaptive contrast enhancement is proposed. The proposed method first combines the minimum loss principle with the maximum attenuation map to optimize the local color and details of the input image. Then, the contrast is adjusted by adaptive local mean and variance, and the color balance is performed further in CLELAB color space to enhance the contrast and details of the image. Although the proposed method performs well in improving image contrast and brightness and enhancing detailed texture information, it still has certain limitations when dealing with images under low light conditions, and the effect needs to be improved.
In reference [6], an image processing method based on a semi-soft threshold model is proposed. Firstly, the proposed method can classify sharp edges of noisy images while avoiding excessive smoothing. Then, the self-adjusting generative adversarial network (GAN) model through the scoring machine is used to improve the quality assessment ability of the GAN model by adjusting the model parameters adaptively to generate higher-quality samples. This method can adapt to the noise characteristics of different images and perform effective image reconstruction. However, this model approach has some challenges regarding training complexity, hyperparameter sensitivity, and data requirements.
As mentioned above, to deal with the problem of significant computational complexity and easy loss of detailed information in image processing, this paper proposes a fog noise image processing method based on an automatic color equalization algorithm and gradient P-M model from the image gradient. The working block diagram of the proposed method is shown in Fig 1. The contributions of this paper are highlighted as follows:
- This paper introduces an adaptive method, independent of large datasets, for processing low-light and fog-noise images with a single processor. Through extensive comparative experiments, the method’s superior performance has been validated. It effectively addresses issues of insufficient contrast and brightness in low-light fog noise images and blurring of detail while ensuring high image quality and structure with a fast processing speed.
- To mitigate the issue of noise amplification and loss of image detail during processing, the paper proposes a gradient-based P-M model method. The approach begins with threshold segmentation using the Otsu method to differentiate foreground from background. Subsequently, a gradient-based enhancement function is applied to accentuate the contrast between foreground and background, highlighting texture information and filling in gaps, thereby eliminating small-area noise.
The low-light fog noise image denoising algorithm proposed in this paper mainly involves the following three stages: preprocessing stage, enhancement stage, and post-processing stage.
- Preprocessing stage: Firstly, the input image is segmented into the RGB three channels, and then the color space is reconstructed in each channel to obtain the corresponding color reconstruction map. Then, the equalization filter is used to equalize the reconstruction map of each channel, which can improve the brightness and contrast of the image while suppressing the noise. Finally, the processed image was output by channel merging.
- Enhancement stage: The maximum between-class variance method(OTSU) performs threshold segmentation on the preprocessed image to determine an optimal threshold segmentation coefficient. Then, this threshold coefficient segments the image from the foreground and background. After that, different gradient-domain enhancements are performed on different segmentation regions to expand the difference between the foreground and background and highlight the image’s edge texture information.
- Post-processing stage: The gradient P-M model is established to denoise the gradient image in the enhancement stage, and the gradient descent method is used to solve the model and output the final result image. The proposed method can effectively suppress the noise and better retain the image’s essential features to improve the image’s quality.
2. Research methods
2.1 Automatic Color Equalization
Due to the influence of machines and environmental factors, the images acquired usually contain noise and are in low contrast. To solve these problems, Rizzi proposed the basic Automatic Color Equalization algorithm(ACE) based on Retinex and combined the theory of “gray world” and “white spot” [7, 8]. The proposed algorithm has remarkable results when dealing with high dynamic range images. By considering the position relationship between color and brightness, the adaptive filtering operation is carried out on local features to adjust the image brightness of local nonlinear characteristics [9]. In the brightness and contrast adjustment, the equalization filter dynamically adjusts the brightness to the appropriate range, thereby improving the image contrast. Low-light images usually have bright areas and shadow areas, so the brightness has a high dynamic range, so it is very suitable to use the automatic color equalization algorithm(ACE) to deal with low-light fog noise images [7]. The algorithm steps are shown as follows:
- Step 1: Convert the input low-light color image into RGB color space.
- Step 2: The color is reconstructed in RGB color space. The reconstruction function is shown in Eq (1):
(1) Where Rc(p) represents the reconstructed brightness of p points in the RGB channel. d(p, j) represents the distance between p and j points. Ic(p) − Ic(j) is the luminance difference between points p and j. r(*) is the brightness expression function, and the function is shown in Eq (2):
(2)
- Step 3: The equalization filter is used to correct the color difference of the reconstructed RGB color space to improve the spatial brightness distribution of the image. The equalization filter formula is given in Eq (3):
(3) Where qc represents the slope of the line segment of [(0, min[Rc(p)]), (255, max[Rc(p)])]. Rc(p) is the pixel value at point c after processing by the equalization filter. round() is a function that converts floating-point data to an integer. Through simple linear expansion, the effect is close to human perception [7].
- Step 4: The enhanced image is obtained by inverse transformation of the corrected image.
2.2 The Otsu’s thresholding method
Otsu’s thresholding algorithm is a widely used threshold segmentation method in image processing that can automatically determine the optimal threshold of the image. Based on the maximum between-class variance criterion, this method determines the optimal threshold by minimizing the between-class variance between background and foreground [10–12]. This method is robust, adaptive, and can effectively segment the image, so it is widely used in many fields.
When dealing with noisy images, the P-M model may introduce new noise, cause the loss of image detail information, and appear apparent gradient effects. To solve these problems, this paper uses Otsu’s algorithm to segment the image and split it into two parts: foreground and background. The foreground part is mainly composed of pixels with higher gray values, which usually carry the detailed texture information of the image. The background mainly comprises low grey pixels, often containing the image’s background, environment, and noise information. Through such segmentation processing, not only can the features of the image be better extracted and highlighted in the following enhancement processing, but the computational complexity of the image can also be reduced. The steps of Otsu’s algorithm are as follows:
- Grayscale conversion: Given that the input is a color image, Otsu’s method is only suitable for processing grayscale images. To preserve the details and contrast information of the image more effectively, the weighted average method [13] is used to convert the input color image into a grayscale image.
- Compute the histogram of the grayscale image: Calculate the histogram of the transformed image to obtain the number of pixels with gray value levels from 1 to m. The number of pixels of the gray value i is ni.
- Compute probability distribution for gray levels: Divide the number of pixels in each gray level by the total number of pixels(N) in the image to get each grey level’s probability distribution. The formula for calculating the probability distribution of each gray level is shown in Eq (4):
(4)
- Calculate the between-class variance: The between-class variance represents the degree of gray level difference between the background and foreground in the image. In the calculation process, the image’s gray level is first divided into two categories: background and foreground for each possible threshold T [12]. Next, the weights and means of the two classes are calculated, and the within-class variance is obtained. Through this index, the image’s distribution of different gray levels can be further understood, which provides strong support for subsequent processing and analysis. The within-class variance formula is given in Eq (5):
(5) Where ω0 is the weight of the foreground, ω1 is the weight of the background, μ0 is the mean of the foreground, μ1 is the mean of the background, and μ is the gray average of the whole image,
,
,
,
, μ = ω0μ0 + ω1μ1.
- Find the best threshold T: The best threshold T is found by iterating over all possible thresholds that maximize the variance between classes [13].
- Apply Threshold: The selected best threshold is used to segment the image into background and foreground [12, 13].
In Fig 2, Images (a) and (d) are grayscale images processed by the ACE algorithm. Images (b) and (e) are the segmented background maps. Images (c) and (f) are the segmented foreground maps. According to the Otsu’s algorithm, the image’s optimal threshold segmentation coefficients T are 123 and 118, respectively.
2.3 Gradient boosting function
When dealing with noisy images, the original P-M model [2] may encounter problems, such as enhancing noise, introducing new noise, and producing significant gradient effects. To solve these problems, we propose to use the enhancement function to optimize the image gradient and reduce the occurrence of these adverse effects. Since the sharpness of the image edge is directly proportional to the gradient, the texture and edge detail information of the image can be reflected in different degrees according to the amplitude of the gradient field. Therefore, by adjusting the enhancement function, we can better improve the edge information of the image and thus deal with noisy images more effectively. The enhancement function of the gradient field is given in Eq (6):
(6)
Where M is the maximum absolute value of gradient, that is, M = max(|∇I|). m is the minimum absolute value of gradient, that is, m = min(|∇I|). T is the determined segmentation threshold. |∇I| is the absolute gradient of the image. So the enhanced gradient field is calculated as shown in Eq (7):
(7)
The gradient maps of the four images before and after enhancement are shown in Fig 3. Images (a) to (d) are the original gradient maps and images (e) to (h) are the corresponding enhanced gradient maps.
As shown in Fig 3, the image gradient processed by the enhancement function shows more significant features, and the image’s edge contour also shows a more transparent and visible effect. In addition, applying the enhancement function can also improve the contrast of the image so that it presents a better visual effect and then highlights more detailed features.
This processing technique is significant for application scenarios such as image analysis, object detection, and recognition. Enhancing the image’s edge contour and gradient information can improve the algorithm’s accuracy and robustness to better deal with various challenges in practical scenes.
2.4 Gradient P-M model
The P-M anisotropic diffusion model based on partial differential equations was proposed by Perona and Malik in 1990 [2, 14–16] and applied to digital image processing. The model’s primary function is to suppress noise and protect the edge texture features of the image. By applying this model, the influence of noise can be effectively reduced while retaining essential details and features in the image, which provides a better foundation for subsequent image analysis and processing. The anisotropic diffusion equation is shown in Eq (8):
(8)
Where I(x, y, t) is the image function, ∇ is the gradient operator, div denotes the divergence, I0(x, y) denotes the initial image with noise, and c||∇I|| is the diffusion coefficient equation. The P-M model is anisotropic diffusion [2, 14, 16]. It can protect the image edge information during diffusion.
The diffusion coefficient equation is given in Eq (9):
(9)
Where ∇I is the image gradient, and k is the electric heating coefficient.
Before applying the gradient descent method to solve the model, the gradient of the image should be first processed using the methods described in Parts B and C. This has several advantages:
- Enhanced contrast: After implementing different levels of enhancement processing on the foreground and background, the contrast between the foreground and background can be strengthened, making the image more transparent and distinct.
- Removing noise: The use of thresholding techniques and gradient enhancement can effectively assist the application of gradient descent in model parameter optimization, which is crucial for promoting the rapid convergence of the nonlinear diffusion process of PM models. In addition, the enhanced image can also improve the pertinence of the gradient-based PM model when dealing with noise in different regions, which can significantly improve the denoising efficiency. These improved measures provide strong support for the performance improvement of the model and the optimization of the practical application effect.
- Edge enhancement: By using different gradient enhancement methods to differentiate the foreground and background, the contrast between the two can be increased, which helps the gradient-based PM model to identify and maintain these edges more accurately, reduce edge blur while denoising, and make the contour of the target object more visible.
According to the variational principle, the anisotropic diffusion Eq (8) is solved using gradient descent [17, 18]. Then, the minimized gradient descent flow function is given in Eq (10):
(10)
Where div is the divergence operator, ∇ is the gradient operator, t represents the iteration time, and ρ is the diffusion coefficient function of the image at time t, which is used to control the anisotropic characteristics of the diffusion process.
Therefore, the finite difference method [19, 20]] is used as the discrete differential operator to solve Eq (10), and the discrete expression of image enhancement is obtained as shown in Eq (11):
(11)
Where, ∇N(It) = Ix,y−1 − Ix,y, ∇S(It) = Ix,y+1 − Ix,y, ∇E(It) = Ix−1,y+1 − Ix,y, ∇W(It) = Ix+1,y − Ix,y is four divergence formulas, whose values are the partial derivatives of the current pixel in the directions of southeast, northwest, and northwest, respectively. cNx,y = exp(−‖∇N(It)‖2/k2), cSx,y = exp(−‖∇S(It)‖2/k2), cEx,y = exp(−‖∇E(It)‖2/k2), cWx,y = exp(−‖∇W(It)‖2/k2) is the thermal conductivity in the four directions of southeast, northwest, I is the current image. k is the thermal conductivity. φ is step size.
3. Experiment
3.1 Datasets
In this study, we collected low-light fog noise images from domestic and foreign photography websites. We integrated these images into the experimental data set of the model in this paper. For detailed information about the image data sets, please refer to lot address: https://github.com/WuyiLi1999/Low_illumination_fog_noise_images.git.
3.2 Experiment setup
3.2.1 Experimental environment.
The experimental environment of this paper: the system is a Windows 10 operating system, the processor is 2.50GHz Intel i5-7200U, and the memory size is 16G, the compiler is PyCharm2021.2.2, and the programming language environment is based on the 64-bit Python3.8 version.
3.2.2 Baseline methods.
To verify the performance of the proposed denoising algorithm, several noisy color images under natural conditions were selected for visual and quantitative comparison analysis with eight different denoising algorithms. According to the selected literature, we briefly outline the method steps and give the corresponding evaluation information, which is shown in Table 1 below:
Ref. [22]: By finding the local minimum in the image histogram, the image is divided into multiple regions, and a specific range of gray levels is assigned to each region. Then, the dynamic histogram equalization of each region is carried out to achieve the denoising effect.
Ref. [21]: Firstly, the multi-scale Retinex theory decomposes the image into reflection and illumination components. Then, the brightness enhancement function (BEF) and the improved Adaptive Contrast enhancement function (IACE) were used to adjust the brightness of the illumination. Then, the Gaussian Laplacian pyramid fuses the two processed illumination maps. Finally, the enhanced illumination and reflection map were fused at multiple scales to obtain the final enhanced image.
Ref. [23]: Based on Retinex theory, the image is enhanced in HSI color space. A linear stretch is applied to the S component. The improved SSR algorithm processes the I component (bilateral filter instead of the original Gaussian filter). Finally, the Sigmoid function is used for color restoration.
Ref. [24]: Firstly, the illuminated component is extracted by relative total variation, and multi-scale Retinex obtains the reflected component. Then, the processing is enhanced with histogram equalization, bilateral gamma function correction, and bilateral filter. The copied original image is enhanced by histogram equalization using weighted guided filtering. Finally, the results of the two processes were weighted and fused by the Artificial Bee Colony (ABC) algorithm.
Ref. [25]: Adaptive low-light image enhancement algorithm based on weighted least squares method. Firstly, the original image is converted to LAB color space, and the L channel is processed by multi-scale detail enhancement and smoothing. then, the weighted least squares smoothing operator extracts the detailed information in the LAB channel and fuses it with the smooth features. Finally, multi-scale Retinex is used to enhance the brightness.
Ref. [7]: Automatic color equalization algorithm based on edge suppression mechanism and Gaussian distribution function.
Ref. [26]: Use the Particle Swarm Optimization (PSO) algorithm to select the optimal gamma value and use this gamma value for gamma correction.
Ref. [27]: Firstly, adaptive color balance is applied to achieve uniform color distribution. Secondly, the HSV color space is transformed, and the Gaussian function and multi-scale Retinex theory are used to extract the illumination and reflection components. Then, high-light and low-light regions are divided according to the pixel mean. Finally, the improved gamma correction algorithm was used to enhance the low light area, and the Weber-Fechner law was used to enhance the high light area to achieve the overall optimization.
3.2.3 FOM merit evaluation index.
This paper uses the performance’s quantitative evaluation indexes to evaluate the performance in the average luminance (AL), information entropy(Entropy), peak signal-to-noise ratio(PSNR), structural similarity(SSIM), mean square error of peak(RMSE), mean absolute error(AME), and mean square error of structural similarity(SDME). The mathematical formula of each evaluation index is as follows:
(12)
(13)
(14)
(15)
(16)
(17)
(18)
Where N is the total number of pixels, I(x, y) is the pixel value of the image, p(x, y) is the probability of occurrence of each pixel in the image, MAX is the maximum possible value of the pixel value, MSE is the mean square error, μx and μy are the average brightness of images x and y respectively, σx and σy are the standard deviation of brightness of images x and y respectively, σxy is the brightness covariance of two images, C1 and C2 are constants (avoid the denominator being zero), I′(x, y) represents the pixel value of the enhanced image, S(x, y) is the structural information of the original image pixels, and S′(x, y) is the structural information of the enhanced image pixels.
3.2.4 Model parameter setting.
The P-M model has three key parameters: the number of iterations(T), the thermal conductivity(K), and the step size(φ). In order to ensure the accuracy and effectiveness of the experimental results, the parameters set by the P-M model are shown in Table 2:
3.3 Experimental results and analysis
Figs 4–6 are images processed by each denoising algorithm with waterfall, bench, and Tian ’anmen Square as benchmarks, respectively. Tables 3–5 show the quantitative index data of images processed by each denoising algorithm using waterfall, bench, and Tian ’anmen Square as the benchmark images, respectively.
According to the data in Figs 4–6, and Tables 3–5 show that the images processed by the method described in reference [22] has excellent performance in the average brightness index, so the color performance is relatively bright. However, this processing method may lead to over-enhancement, affecting the overall visual effect. In addition, some colors tend to have darker tones, and the problem of loss of details may occur, and discontinuities appear at the boundaries of the processed image.
According to the methods described in references [21, 23, 24, 25, 27], these methods are all based on multi-scale Retinex theory, with differences in preprocessing and processing methods for reflection and illumination components.
The method proposed in the literature [21] adopts the Brightness enhancement function (BEF) and improved Adaptive Contrast enhancement function (IACE), which significantly improves the overall brightness of the image. The texture rendering effect is excellent, especially with a close range, and the structural similarity with the original image is high. However, this method has the problem of stability, which may lead to over-whiteness of the image and loss of details when dealing with the perspective, and noise may be introduced in the fusion process, resulting in discontinuous boundaries.
The method proposed in reference [23] uses the bilateral filter to smooth the noise in the image effectively, and the visual effect is good. It also avoids the problem of discontinuous boundaries and preserves the structure of the image. However, the information entropy index of this method could be low, and less detailed information is retained.
The method proposed in reference [24] combines histogram equalization, bilateral gamma function correction, and bilateral filter to process reflection components. The processed image has a bright color, effectively removes noise, and reduces the appearance of halo and artifacts. Compared with the above methods [21, 23], the proposed method improves the information entropy and retains more detailed information. However, the processing effect of low-light images is unstable, there is a risk of excessive color distortion, and the difference from the original image is significant.
The method proposed in reference [25] uses the smoothing operator of the weighted least squares method for weighted fusion processing. This method reduces the phenomenon of brightness over-saturation and better preserves the image’s structure. The visual effects in the waterfall and bench images perform well. However, the noise reduction effect is not apparent in the bench image. The effect of this method is not stable in images of different scenes and needs to be further adjusted.
The method proposed in reference [27] uses gamma correction and Wender-Fechner law to deal with the illumination components’ low and high light areas, respectively. This method significantly improves the brightness and contrast of each region in the non-uniform illumination image and retains more details in the solid light area. However, this method will lose the structure of the image.
According to the research in the reference [7], automatic color equalization processing by applying an edge suppression mechanism and Gaussian distribution function can significantly improve the brightness and contrast of the image, thus providing a better overall visual effect. However, this method may suffer from oversaturation in the close-range region, resulting in color bias towards darker tones. In addition, the proposed method could perform better in structural similarity index, peak mean square error, and square absolute error value, and the structure difference between the proposed method and the original image is large, significantly impacting the image quality.
According to reference [26], the Particle Swarm Optimization (PSO) algorithm performs well in gamma correction, which can effectively smooth the noise, improve the image’s overall brightness, and better maintain the structure of the image. However, this method also has some shortcomings, such as the overall whiteness of the image and the loss of some detail information.
The processing method proposed in this paper performs well in visual and drying effects, and the image is bright in color, effectively reducing the excessive enhancement and gradient effect in the processing process. At the same time, the detailed information of the image is well preserved, and the critical texture information is also more prominent. After rigorous experimental comparison, the proposed method performs well in peak signal-to-noise ratio and ranks among the top three. However, in terms of indicators such as peak mean square error and absolute square error, the values are slightly lower in the middle. This shows that the proposed method can effectively reduce the difference with the original image and improve the quality of the reconstructed image when processing the image. In terms of structural similarity, the proposed method also achieved remarkable results. The mean square error of structural similarity is in the lower middle level, and the structural similarity index is among the best. This indicates that the processed image is more similar in structure to the original image. However, the structural similarity index has a low value when dealing with the Tiananmen smog image. This may be because the low-light yellowish haze image has more complex color and illumination changes, making it challenging to improve the evaluation index of the structural similarity index at the same time when enhancing denoising. However, this does not affect the excellent performance of the proposed method in most cases, which can ensure the quality and structure of the image.
3.4 Running time
The methods introduced in this paper include the automatic color equalization algorithm(ACE), Otsu’s thresholding method, and the gradient P-M model. In the automatic color equalization algorithm(ACE), color space conversion, equalization filter, and inverse color space conversion constitute the whole processing flow. In the phase of color space conversion, the time complexity of the calculation is O(MN). The filtering operation of the equalization filter involves the weighted sum of pixels and the comparison of pixel values, and its time complexity is related to the size of the filter(K*K) and the number of pixels in the image. The time complexity of the filter can be approximated as K2O(MN). Therefore, the total time complexity of the ACE algorithm is shown in Eq (19):
(19)
The Otsu’s thresholding algorithm is based on histogram statistics, and its time complexity is O(MN). In order to find the optimal threshold to maximize the between-class variance, the algorithm needs to iterate over all the thresholds. Therefore, the total time complexity of Otsu’s algorithm is shown in Eq (20):
(20)
The P-M model involves the calculation of the objective function, the calculation of the gradient, and the update of the parameters in each iteration. The computational complexity of the objective function and the gradient are O(MN), while the updating complexity of the parameters is O(1). The total time complexity of the gradient P-M model can be approximated as shown in Eq (21):
(21)
Therefore, the total time complexity of the proposed method is shown in Eq (22):
(22)
Where K is the size of the equalization filter, M represents the image’s height, N represents the image’s width, and L represents the number of gray levels of the image.
An algorithm running time evaluation metrics are significant for user experience, resource utilization and cost-effectiveness, scalability and adaptability, algorithm comparison and selection, etc. Fig 7 shows the average running time (in milliseconds) of 20 executions of different methods on images of three different resolutions in the experimental environment of this paper. According to the data in Fig 7, compared with the denoising algorithms in [7, 21, 24–27], the proposed method can process noisy images faster and output denoised image data. Compared with [22, 23], the processing speed is lower, but the quality of the output image is better than them.
3.5 Ablation experiment
In order to verify the influence of each part of the model on the experimental results, ablation experiments were performed on each part of the model method and the corresponding combination. The methods include 1) a single automatic color equalization algorithm; 2) a single P-M model; 3) Automatic color equalization +P-M model without gradient enhancement using the OTSU method; 4) The initial noise image is directly processed by OTSU’s gradient P-M model; 5) The method proposed in this paper. The ablation experimental results of methods 1-5 in waterfall and bench images are shown in Tables 6 and 7, respectively.
According to the data in Tables 6 and 7, the average brightness of the image processed by method 1 shows excellent performance. However, regarding information entropy value, the images processed by method 2 show apparent advantages.
Method 3 combines Method 1 and Method 2. Compared with the two methods alone, the average brightness of method 3 is increased by 60% and 134%, respectively, and the information entropy value is increased by 0.7% and 4.4%, respectively. However, there are some problems in the noise distortion degree and pixel difference effect of the image, which may introduce gradient noise in the processing process. Nevertheless, the image maintains an excellent structure to ensure the accurate transmission of basic visual information.
Based on Method 2, Method 4 further introduces Otsu’s algorithm and enhancement function. Compared with method 2, the average brightness of method 4 on the two images is increased by 37% and 102%, respectively, and the information entropy index is also increased by 1% and 5%, respectively. According to the data results of the last five evaluation indicators, the proposed method can more accurately present the details of the image structure and effectively improve the image quality. However, this method also has problems: the brightness structure will be lost in the processing.
Method 5, which further introduces the Otsu algorithm and the enhancement function based on Method 3, is proposed in this paper. Compared with method 3, the average brightness of method 5 on the two images is increased by 7% and 25%, and the information entropy value is increased by 1.8% and 4.9%, respectively. According to the peak signal-to-noise ratio, structural similarity index, structural similarity mean square error, and absolute square error data, the structural and perceptual quality of the image are significantly improved, and the brightness structure is more similar to the original image. However, it should be noted that part of the detailed structure may be lost during the processing. Considering the trade-off between different indicators, the improved scheme proposed in this paper can effectively improve the image contrast and retain the image details.
4. Conclusion
This study proposes a novel denoising method combining the Automatic Color equalization (ACE) algorithm and gradient P-M model (referred to as ACE-GPM for short), specifically for low-illumination hazy noisy images. This method first uses automatic color balance to enhance image contrast. Then, it applies OTSU threshold segmentation based on the most between-cluster variance method and threshold, leading to the image’s gradient field enhancement. Finally, the gradient of P—M removes the noise-denoising model, effectively reserving the detail of the image and texture information simultaneously. Experimental results show that the proposed method retains more detailed texture information while improving image contrast and removing noise. It also has significant advantages in running time and image quality evaluation index.
In addition, the critical parameters of the P-M model are accurately determined by manual iteration to ensure the convergence and stability of the algorithm. Although selecting these parameters is crucial to the efficiency of the gradient descent method, there is still room for further optimization, especially the adaptive parameter adjustment mechanism under different pixel densities and image sharpness conditions. Future research will explore the adaptive learning rate strategy in the deep learning environment to realize the adaptive optimization of the gradient P-M model parameters so as to enhance the generalization ability and practicability of the model and further improve its performance on various evaluation indicators.
References
- 1. XIANG S, WANG Y, DENG H, et al. Zero-shot Learning for Low-light Image Enhancement Based on Dual Iteration[J]. Journal of Electronics & Information Technology. 2022, 44(10): 3379–3388.
- 2. Khmag A. Natural digital image mixed noise removal using regularization Perona–Malik model and pulse coupled neural networks[J]. Soft Computing. 2023, 27(21): 15523–15532.
- 3. Zhang W, Wang Y, Li C. Underwater image enhancement by attenuated color channel correction and detail preserved contrast enhancement[J]. IEEE Journal of Oceanic Engineering 2022, 47(3): 718–735.
- 4. Khmag A, Al Haddad S A R, Ramlee R A, et al. Natural image noise removal using nonlocal means and hidden Markov models in transform domain[J]. The Visual Computer. 2018, 34: 1661–1675.
- 5. Zhang W, Jin S, Zhuang P, et al. Underwater image enhancement via piecewise color correction and dual prior optimized contrast enhancement[J]. IEEE Signal Processing Letters. 2023, 30: 229–233.
- 6. Khmag A. Additive Gaussian noise removal based on generative adversarial network model and semi-soft thresholding approach[J]. Multimedia Tools and Applications. 2023, 82(5): 7757–7777.
- 7.
Su J, Liu L. New automatic color equalization algorithm based on lateral inhibition mechanism[C]. Information Technology and Mechatronics Engineering Conference (ITOEC). 2018: 1592–1595.
- 8. Plutino A, Tarini M. Fast ACE (FACE): an error-bounded approximation of Automatic Color Equalization[J]. IEEE Transactions on Image Processing. 2023. pmid:37186534
- 9. Wang D, Liu Z, Gu X, et al. Feature extraction and segmentation of pavement distress using an improved hybrid task cascade network[J]. International Journal of Pavement Engineering. 2023, 24(1): 2266098.
- 10. Ning G. Two-dimensional Otsu multi-threshold image segmentation based on hybrid whale optimization algorithm[J]. Multimedia Tools and Applications. 2023, 82(10): 15007–15026.
- 11. Du Y, Yuan H, Jia K, et al. Research on Threshold Segmentation Method of Two-Dimensional Otsu Image Based on Improved Sparrow Search Algorithm[J]. IEEE Access. 2023.
- 12. Xiang S, Yu L, Chen C W. No-reference depth assessment based on edge misalignment errors for T+ D images[J]. IEEE Transactions on Image Processing. 2015, 25(3): 1479–1494.
- 13.
Quan Y, Sun J, Zhang Y, et al. The method of the road surface crack detection by the improved Otsu threshold[C]. International Conference on Mechatronics and Automation (ICMA). 2019: 1615–1620.
- 14. Raonic B, Molinaro R, De Ryck T, et al. Convolutional neural operators for robust and accurate learning of PDEs[J]. Advances in Neural Information Processing Systems. 2024, 36.
- 15. Zhang Q, Song X, Song S, et al. Finite-Time sliding mode control for singularly perturbed PDE systems[J]. Journal of the Franklin Institute. 2023, 360(2): 841–861.
- 16. Fučík Radek, Straka Robert. Equivalent finite difference and partial differential equations for the lattice Boltzmann method[J]. Computers & Mathematics with Applications. 2021,90:96–103.
- 17. Mei K, Hu B, Fei B, et al. Phase asymmetry ultrasound despeckling with fractional anisotropic diffusion and total variation[J]. IEEE Transactions on Image Processing. 2019,29:2845–2859. pmid:31751240
- 18.
Xie Z, Sato I, Sugiyama M. A diffusion theory for deep learning dynamics: Stochastic gradient descent exponentially favors flat minima[J]. arXiv preprint arXiv. 2002,03495.
- 19. Modanli M, Akgül A. On Solutions of Fractional order Telegraph partial differential equation by Crank-Nicholson finite difference method[J]. Applied Mathematics and Nonlinear Sciences. 2020,5(1):163–170.
- 20. Ureña F, Gavete L, Garcia A, et al. Solving second order non-linear parabolic PDEs using generalized finite difference method (GFDM)[J]. Journal of Computational and Applied Mathematics. 2020,363:1–21.
- 21. Pan X, Li C, Pan Z, et al. Low-Light Image Enhancement Method Based on Retinex Theory by Improving Illumination Map[J]. Applied Sciences. 2022, 12(10): 5257.
- 22.
Panse V, Gupta R. Medical image enhancement with brightness preserving based on local contrast stretching and global dynamic histogram equalization[C]. International Conference on Communication Systems and Network Technologies (CSNT). 2021. p. 164–170.
- 23.
Chen Z, Wang L, Wang C, et al. Fog image enhancement algorithm based on improved Retinex algorithm[C]. International Conference on Electronic Communication and Artificial Intelligence (IWECAI). IEEE. 2022: 196–199.
- 24. Sun Y, Zhao Z, Jiang D, et al. Low-illumination image enhancement algorithm based on improved multi-scale Retinex and ABC algorithm optimization[J]. Frontiers in Bioengineering and Biotechnology. 2022, 10:865820. pmid:35480971
- 25.
Zhao E, Gao J. An adaptive low-illumination image enhancement algorithm based on weighted least squares optimization[C]. Journal of Physics: Conference Series. IOP Publishing. 2022, 2181(1):012011.
- 26. Mahmood A, Khan SA, Hussain S, Almaghayreh EM. An adaptive image contrast enhancement technique for low-contrast images[J]. IEEE Access. 2019,7:161584–161593.
- 27. Ji X, Guo S, Zhang H, et al. Non-Uniform-Illumination Image Enhancement Algorithm Based on Retinex Theory[J]. Applied Sciences. 2023, 13(17):9535.