Traditional restoration methods

The restoration of historical landscapes and ancient artworks has long relied on traditional image restoration techniques, which primarily use rule-based methods to fill missing areas, remove scratches, repair damaged parts, and preserve the original artistic style as much as possible. Early restoration techniques mainly focused on interpolation-based methods, texture synthesis, and prior information-based approaches. These methods could restore the integrity of images to some extent but still had many limitations when dealing with complex damaged areas.

Interpolation-based methods were among the earliest techniques used for image restoration. They utilize information from surrounding pixels to fill damaged areas, minimizing visual discontinuities [21,22]. These methods are computationally efficient and perform well in cases of minor damage, but when applied to large missing areas, they tend to cause blurring or loss of local texture information. Additionally, gradient diffusion-based restoration methods attempt to propagate color and edge information within the image to fill missing areas, blending them with the surrounding environment [23–25]. However, these methods perform poorly when handling complex structures and struggle to accurately restore geometric details in images. To better preserve local textures in images, texture synthesis-based restoration methods were introduced. These methods analyze the texture features around damaged areas and generate texture structures matching the original image during the filling process, thereby improving the naturalness of restoration [26,27]. One representative method is patch-based inpainting, which synthesizes missing parts using small image patches. This approach is suitable for scenes with rich details and strong local repetition. However, when dealing with complex historical landscapes and irregularly shaped damaged areas, this method often fails to reconstruct missing information precisely, leading to inconsistencies or visual artifacts in the restored regions.

Another common category of restoration methods is probability model-based approaches, which use known image information to infer reasonable ways to fill missing parts. For example, some methods integrate distance fields to guide pixel filling, reducing structural distortion while maintaining smoothness in the restored areas [28,29]. Additionally, global optimization models have been applied to image restoration tasks by minimizing certain mathematical loss functions to ensure the continuity and rationality of the filled regions [30]. However, these methods often have high computational complexity in large-scale restoration tasks, making them difficult to apply to high-resolution historical landscape restoration. For different types of historical landscapes and artworks, traditional methods also vary accordingly. In the restoration of murals and calligraphy works, multi-layer structure segmentation methods are often employed to separately process overall structures and local details, making the restoration results more realistic [31–33]. In the restoration of traditional Chinese paintings, researchers have proposed damage-detection-based image restoration methods, where specific algorithms first identify damaged areas in the painting, followed by patch matching and texture synthesis to restore the image while maintaining stylistic consistency. Furthermore, to improve restoration accuracy, some methods incorporate spatial attention mechanisms to better balance local textures and overall layouts when dealing with large-scale damaged areas [34]. However, these methods still face adaptability issues when applied to highly complex painting styles and multi-layered artistic expressions.

Overall, traditional historical landscape restoration methods are effective for minor damage repair and regular texture filling but have limitations when dealing with large-scale damage, complex geometric structures, and highly artistic images. Additionally, restoration results from traditional methods often rely on manual parameter adjustments, making them unable to intelligently adapt to different types of historical landscapes. Therefore, exploring more efficient and intelligent restoration methods, particularly those integrating modern computer vision and deep learning technologies, has become a key research direction.

Image restoration based on generative adversarial networks (GANs)

In recent years, Generative Adversarial Networks (GANs) have become an essential tool for image restoration and have been widely applied in the restoration of artworks and historical landscapes. Compared with traditional methods, GANs leverage adversarial training between a generator and a discriminator to effectively learn both the global structure and local textures of images, making the restoration results more natural and coherent. Many researchers have proposed various GAN-based restoration methods tailored to different types of image damage to enhance restoration quality and adaptability.

Some researchers have introduced a GAN restoration method based on context encoding, which employs conditional generation to infer missing content using surrounding pixel information [35]. This method performs well in repairing regularly shaped damaged areas. However, since a single discriminator struggles to simultaneously focus on both global layout and local details, another class of methods has adopted a global-local discriminator architecture [36–38]. In this approach, the global discriminator ensures overall stylistic consistency, while the local discriminator focuses on the realism of details within the restored areas, thereby improving the coherence and visual quality of the restoration results. Furthermore, to enhance semantic understanding during the restoration process, some scholars have incorporated attention mechanisms [39,40]. By leveraging self-attention modules, these models improve their ability to capture long-range dependencies, ensuring that the restored regions align better with the global style of the original image. For instance, some researchers have introduced channel attention and spatial attention modules into the GAN generator to more accurately focus on the features of damaged areas, thereby achieving clearer detail recovery [41]. For complex structural restoration tasks, some researchers have proposed edge-guided generative models. These methods first use an edge detection network to predict structural information in the missing regions and then employ GANs for detailed filling, ensuring structural integrity in the restored images [42–44]. Additionally, some studies have introduced multi-stage generation approaches, where a coarse restoration is performed first, followed by progressive refinement to enhance restoration quality in large missing areas and improve overall visual consistency. In the field of cultural heritage and traditional art restoration, some researchers have developed GAN models specifically designed for artistic styles. For example, some studies have combined GANs with style transfer techniques to ensure that the restored images not only fill missing parts but also retain the original artwork’s colors and brushstroke characteristics [45,46]. Another category of methods incorporates Variational Autoencoders (VAEs) and Transformer architectures, utilizing multi-scale feature extraction and cross-region information interaction to better preserve artistic styles and enhance the model’s generalization capability [47].

Overall, GANs have demonstrated high flexibility and superiority in image restoration tasks. Their effectiveness has been validated in numerous studies, particularly in the restoration of artworks and historical landscapes. As GAN architectures continue to be optimized—through the integration of attention mechanisms, multi-stage generation, and edge-guided methods—their applications in image restoration will become even more promising.

Overall framework

The dual-branch channel attention fusion generative adversarial network (DCAF-GAN) proposed in this paper is designed to achieve high-quality historical landscape restoration. The overall framework consists of a generator and a discriminator, where the generator adopts a dual-branch structure to separately extract local texture information and global structural features. Through the channel attention fusion strategy, the complementary nature of different features is enhanced, ultimately generating high-quality restored images. The discriminator is built upon a deep convolutional structure based on ConvNeXt to improve the stability of adversarial training and optimize the learning capability of the generator. The architecture of the entire system is illustrated in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The overall architecture of the proposed DCAF-GAN model.

The generator consists of a texture encoder and a structure encoder, the texture encoder includes depthwise and pointwise convolutions, and the structure encoder contains dilated convolutions and pooling operations. The CAGF module processes multi-scale features. The decoder comprises upsampling and residual connections. The discriminator is based on the ConvNeXt architecture.

https://doi.org/10.1371/journal.pone.0334532.g001

In the generator, a dual-branch encoder is designed to extract features separately for texture details and structural information. The texture encoder employs depthwise separable convolution (DW Conv) to efficiently extract local details, enhancing the texture expressiveness of the restored regions. Meanwhile, the structure encoder utilizes dilated convolution (Dilated Conv) to expand the receptive field, capturing global contour information and ensuring the rationality of the overall layout. The extracted features are then fed into the CAGF module, which dynamically adjusts the contribution of texture and structural features through global pooling (GPA) and channel-adaptive weighting (CA), ensuring that the restored regions maintain stylistic consistency with the original image. Finally, the fused features are reconstructed by the decoder, which integrates nearest-neighbor interpolation, residual blocks, and self-attention to enhance detail restoration and global consistency. The discriminator adopts the ConvNeXt architecture, which, compared to traditional CNN-based discriminators, can more effectively capture complex features in high-resolution images, improving the ability to assess the authenticity of the restored results. Through adversarial training, the discriminator continuously enhances the restoration quality of the generator, making the generated historical landscape images more realistic and coherent. The overall framework follows a multi-stage training strategy. First, the dual-branch encoder is trained independently to ensure that texture and structural features are effectively extracted. Next, the channel attention fusion module is trained to optimize the interaction between features. Finally, adversarial training is introduced to further enhance the restoration effect. The architectural design ensures both the clarity of detail restoration and the rationality of global structure, achieving high-quality historical landscape restoration.

Generator

Dual-branch encoder and decoder.

The generator of DCAF-GAN adopts a dual-branch encoder structure, where the texture encoder focuses on extracting local high-frequency details to complete missing textures, while the structure encoder captures global contour information to ensure the overall consistency of the restored region. Given the input image I_in, which consists of missing regions M and known areas, the dual-branch encoder extracts features separately, and the decoder reconstructs the final restored image I_out.

Dual-branch Encoder

The dual-branch encoder consists of a texture encoder and a structure encoder, each learning different scale features to compensate for the limitations of a single-branch encoder. The texture encoder focuses on capturing fine-grained local features, while the structure encoder is responsible for preserving global structural information. The texture encoder employs depthwise separable convolution (DW Conv) to efficiently extract local features while reducing computational complexity. DW Conv decomposes standard convolution into depthwise convolution and pointwise convolution, allowing the model to efficiently capture local textures. After passing through L layers of DW Conv, the output of the texture encoder can be formulated as:

(1)

where and represent the convolution weights and biases of the i-th layer, denotes the convolution operation, and is the activation function (e.g., ReLU or GELU). This multi-layer convolutional design enables the network to effectively learn local patterns of missing areas and fill in missing textures.

The structure encoder utilizes dilated convolution (DC) to expand the receptive field and improve the ability to capture global structural information. The computation of dilated convolution is defined as:

(2)

where denotes the convolution kernel with dilation rate r_i, and represents the dilated convolution operation. When r > 1, empty spaces are inserted between convolution kernels, significantly expanding the receptive field:

(3)

where k is the convolution kernel size, and r is the dilation rate. Dilated convolution allows the model to effectively restore global structure information, making the reconstructed region more natural. The final output of the dual-branch encoder is represented as:

(4)

where E_t denotes texture features and E_s represents structural features. These features will be further refined in the CAGF module.

Decoder

The decoder maps the fused features back to the high-resolution image space and generates the final restored image I_out. To ensure efficient reconstruction, the decoder adopts nearest neighbor upsampling and residual blocks for fine-grained detail enhancement.

To progressively restore image resolution, the decoder first performs upsampling using nearest neighbor interpolation:

(5)

where represents the upsampling operation, computed as:

(6)

where s is the upsampling factor, and denotes the floor operation. Nearest neighbor interpolation preserves edge information while reducing artifacts.

Following upsampling, the decoder employs multiple residual blocks to reconstruct fine-grained details. Each residual block is formulated as:

(7)

where are convolution weights, are biases, and ReLU serves as the activation function. The residual connection stabilizes gradient propagation and enhances feature transfer. To further suppress artifacts and improve detail clarity, a convolution is introduced for channel compression:

(8)

where W_f and b_f are the convolution weights and biases.

The final restored image I_out should be as close as possible to the ground truth image I_gt, ensuring perceptual and structural consistency:

(9)

where represents the decoder network, which integrates local and global features to achieve high-quality historical landscape restoration.

This section introduced the design of the dual-branch encoder and decoder, where the texture encoder employs depthwise separable convolution to enhance local feature extraction, while the structure encoder utilizes dilated convolution to capture global contour information. The decoder combines nearest neighbor upsampling and residual blocks to ensure the generated restored image retains fine details and structural coherence. In the next section, we will describe the CAGF module, which further optimizes feature interactions to produce more natural restoration results.

Channel attention-guided fusion (CAGF).

The generator of DCAF-GAN employs a dual-branch encoder to extract local texture information and global structural information. However, directly concatenating or summing these two types of features may lead to feature redundancy or imbalance, which can degrade the restoration quality. To address this issue, we introduce the CAGF module, which dynamically adjusts the contributions of different features. This ensures that the restored region retains overall structural consistency while recovering detailed texture information, allowing for better integration of features. The CAGF module integrates global feature statistics, adaptive channel weighting, and feature compression to refine the fusion process.

In the CAGF module, the output features from the dual-branch encoder, denoted as f_hi (high-level features) and f_mi (low-level features), are first concatenated along the channel dimension:

(10)

where f_hi represents global structural information and f_mi represents local texture information. Since different regions of the image may require varying amounts of global and local information, a simple concatenation is not sufficient. To better balance the contributions of each feature type, a convolution is applied to transform the feature representation:

(11)

where represents a convolution operation, denotes batch normalization, and is the activation function. This operation normalizes the feature distribution before the attention mechanism is applied.

To further optimize feature selection, we employ a channel attention mechanism to adaptively adjust the importance of different channels. First, we extract global statistics from using global average pooling (GAP) and global max pooling (GMP):

(12)(13)

where H and W are the height and width of the feature map, respectively. captures the overall feature distribution, while emphasizes the most significant activations. These two statistics are concatenated and passed through a multilayer perceptron (MLP) to compute channel-wise attention weights:

(14)

where W₁ and W₂ are weight matrices of the MLP, represents the ReLU activation function, and denotes the sigmoid function. This process allows the network to learn the relative importance of different feature channels. The computed attention weights are then applied to the feature map through element-wise multiplication:

(15)

where denotes element-wise multiplication, ensuring that each channel is scaled according to its learned importance.

Finally, to preserve the original feature information, the attention-enhanced feature f_c is added back to the original transformed feature:

(16)

where is the final fused feature, which is fed into the decoder for reconstruction.

The CAGF module leverages channel attention to effectively regulate the contributions of different feature types. By dynamically adjusting feature weights, it enhances the model’s ability to focus on critical regions while suppressing redundant information. This fusion approach ensures that the restored image maintains structural coherence while preserving fine-grained texture details, improving the overall restoration quality.

Discriminator

The discriminator in DCAF-GAN adopts the ConvNeXt architecture to enhance the stability of adversarial training and improve the ability to distinguish between real and generated images. ConvNeXt builds upon traditional convolutional neural networks (CNNs) by introducing deeper convolutional layers and efficient normalization techniques, demonstrating superior performance in image classification and generative tasks. To enable the discriminator to effectively distinguish between generated and real images, a multi-layer convolutional structure is designed to capture both local details and global patterns, thereby improving the evaluation accuracy of restoration quality, as shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The proposed discriminator architecture consists of a ConvNeXt-based convolutional block, self-attention, and a fully connected layer with spectral normalization.

https://doi.org/10.1371/journal.pone.0334532.g002

The input to the discriminator consists of real images I_gt and generated images I_out, with the objective of learning the distribution differences between the two. These images are processed through multiple ConvNeXt blocks for feature extraction. The output of the l-th convolutional layer, denoted as f_l, is computed as:

(17)

where W_l and b_l represent the convolution kernel weights and biases, denotes the convolution operation, and is the nonlinear activation function (e.g., GELU or ReLU). Through multiple layers of ConvNeXt blocks, the discriminator can capture features across different scales, ranging from low-level textures to high-level semantics, progressively enhancing its discrimination ability.

To further strengthen the global feature modeling capability of the discriminator, self-attention is incorporated into the ConvNeXt structure. The self-attention mechanism is formulated as:

(18)

where Q, K, and V represent the query, key, and value matrices, respectively, and d_k is the feature dimension. By computing relationships between different regions in the image, self-attention enhances the discriminator’s ability to capture global consistency, making it more effective in distinguishing between real and restored images, especially in complex damaged areas.

Additionally, to further stabilize the discriminator, Spectral Normalization is applied to constrain the Lipschitz constant of the model, thereby improving training stability. The spectral normalization is computed as:

(19)

where W represents the weight matrix and is the largest singular value of W. This normalization technique prevents excessive optimization of the discriminator, enhances generalization, and contributes to more stable adversarial training.

Finally, the discriminator outputs a scalar D(I), representing the probability that the input image I is real. The discriminator is trained to minimize the binary cross-entropy loss function:

(20)

where is the real image distribution, is the generated image distribution, and D(I) is the discriminator’s output score.

In summary, incorporating the ConvNeXt structure allows the discriminator to effectively learn both local and global features while leveraging self-attention to improve discrimination in complex regions. Spectral normalization ensures stable training by preventing gradient explosion. This design ensures that the discriminator can effectively guide the generator during adversarial training, leading to improved realism in the restored images.

Loss functions

The loss function design in DCAF-GAN aims to enhance the realism and structural consistency of restored images while ensuring training stability. To achieve this, we employ a combination of adversarial loss, perceptual loss, structural loss, and content loss, optimizing the generator’s performance to produce more visually authentic restorations.

The adversarial loss is the core component of the generative adversarial network (GAN), encouraging the generator to produce images indistinguishable from real ones. It is defined as follows:

(21)

(22)

where D(I) represents the discriminator’s probability of classifying input I as real, is the distribution of real images, and represents the distribution of generated images. By optimizing this loss, the generator continuously improves the authenticity of restored images, while the discriminator learns to better distinguish between real and generated images.

The perceptual loss measures the similarity between generated and real images in high-level feature space. It extracts deep feature representations from a pre-trained VGG network and computes the Euclidean distance:

(23)

where denotes the feature map extracted from the l-th layer of the VGG network, and and represent the generated and real images, respectively. This loss helps the generator capture fine details and texture information, improving visual quality.

The structural loss employs the Structural Similarity Index (SSIM) to evaluate the structural consistency between the restored and ground truth images:

(24)

where measures the similarity between two images based on luminance, contrast, and structure components. By minimizing this loss, the generator preserves structural coherence and reduces unwanted artifacts in the restored image.

The content loss directly computes the L1 pixel-wise difference between the generated and real images:

(25)

This loss ensures that the restored images maintain overall color consistency and brightness, preventing unrealistic artifacts or color distortions.

Finally, the total loss function for the generator combines these four losses to balance adversarial learning, visual fidelity, and structural consistency:

(26)

where , , and are weighting coefficients for the corresponding loss terms. By appropriately tuning these weights, DCAF-GAN maintains both the authenticity of restored images and the preservation of fine details, ensuring natural and coherent restoration results.

Datasets

Hardware and software environment

This experiment is conducted on a high-performance computing platform to ensure the efficient training of DCAF-GAN and the generation of high-quality historical landscape restoration images. The hardware environment includes an NVIDIA A100 80GB or RTX 3090 24GB GPU, providing strong computational power, along with an Intel Xeon Gold 6226R @ 2.90GHz CPU and 256GB DDR4 memory to support large-scale data loading and model training. A 2TB NVMe SSD is used for high-speed data reading and writing. The software environment is based on the Ubuntu 20.04 LTS operating system, utilizing PyTorch 2.0 as the deep learning framework, with CUDA 11.8 and cuDNN 8.6 for GPU acceleration. OpenCV is used for image processing, NumPy and Matplotlib for data analysis, torchvision for dataset management, and tensorboard for training visualization. This configuration ensures efficient model training and the reproducibility of experiments.

Training parameters and optimization strategy

The training of DCAF-GAN adopts the Adam optimizer with an initial learning rate of 2e-4, which is dynamically adjusted using cosine annealing scheduling to optimize model convergence speed and stability. The batch size is set to 16, and the total number of training epochs is 200, ensuring that the model sufficiently learns restoration patterns and generates high-quality historical landscape reconstructions.

The training process is divided into two stages. In the first stage, the generator is trained to learn the fundamental ability to fill missing regions, with L1 loss and SSIM loss being the primary optimization objectives to ensure structural consistency. In the second stage, the discriminator is introduced for adversarial training, increasing the weights of perceptual loss and adversarial loss to enhance the realism and detail of the restored images. To stabilize training, the discriminator employs Spectral Normalization to prevent gradient explosion or vanishing. The overall training process is outlined in Algorithm 1.

Algorithm 1 Training procedure of DCAF-GAN

1: Initialize Generator G, Discriminator D with Spectral

  Normalization

2: for each epoch e in total epochs do

3:   for each batch in dataset do

4:    Sample masked images I_in and ground truth images I_gt

5:    Generate restored images

6:    if e < 100 then

7:     Compute L1 content loss:

8:     Compute structural loss:

9:     Compute total loss:

10:    else

11:     Compute L1 content loss:

12:     Compute structural loss:

13:     Compute perceptual loss:

14:     Compute adversarial loss:

15:     Compute total loss:

16:    end if

17:    Update generator G using Adam optimizer

18:    Compute discriminator loss:

19:    Update discriminator D using Adam optimizer

20:   end for

21:   Adjust learning rate using cosine annealing

22: end for

The discriminator adopts Spectral Normalization to stabilize training, preventing gradient explosion or vanishing. During the early stage of training, L1 and SSIM losses dominate to ensure fundamental structural restoration. In the later stages, perceptual loss (VGG) and adversarial loss are incorporated to enhance texture realism and fine details. This approach enables DCAF-GAN to generate high-fidelity historical landscape restorations with both global consistency and fine-grained textures.

Evaluation metrics

To comprehensively evaluate the performance of DCAF-GAN in historical landscape restoration tasks, several quantitative metrics are employed to assess the quality, structural consistency, and perceptual realism of the restored images.

First, the Structural Similarity Index (SSIM) is used to measure the similarity between the restored and ground truth images in terms of structural information. The SSIM value ranges from 0 to 1, where a higher value indicates better structural preservation. Peak Signal-to-Noise Ratio (PSNR) is also employed to evaluate image quality, with a higher PSNR value indicating lower distortion and better restoration fidelity. In addition, the Learned Perceptual Image Patch Similarity (LPIPS) metric is utilized, which is based on deep neural networks such as AlexNet or VGG to compute perceptual similarity in feature space. A lower LPIPS value suggests that the restored image is more visually similar to the ground truth. Furthermore, the Frechet Inception Distance (FID) is introduced to measure the distributional difference between generated and real images by computing the feature statistics from an Inception network. A lower FID score indicates that the restored images are closer to the real image distribution.

Additionally, to complement the quantitative metrics, a user perceptual study was conducted to evaluate the visual quality of the restored images from a human perspective. A group of 30 ordinary users was asked to rate the restored images based on clarity, color consistency, structural consistency, and visual appeal. These subjective ratings offer valuable insights into the visual realism of the restored images and help assess their suitability for cultural heritage restoration tasks. The results from this user study provide an additional layer of validation, ensuring that the model’s performance aligns with human visual perception.

The ratings were based on a five-point scale, as outlined in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Perceptual rating scale for user study.

https://doi.org/10.1371/journal.pone.0334532.t001

These evaluation metrics comprehensively assess the performance of the model from multiple perspectives, ensuring that the restored images achieve both high visual quality and structural accuracy.

Quantitative comparison analysis

To evaluate the effectiveness of DCAF-GAN in historical landscape restoration, we compare its performance with several state-of-the-art image restoration and inpainting models. The selected models include traditional Chinese painting restoration networks, super-resolution generative adversarial networks (GANs), and advanced image inpainting methods. Below is a brief introduction to each of the competing models:

SGRGAN (Sketch-Guided Restoration GAN) [48]: A restoration framework that utilizes sketch images as structural priors to restore traditional Chinese paintings while maintaining their unique artistic characteristics. It incorporates a Focal Block and BiSCCFormer Block to enhance the fusion of textural and structural features.

ConvSRGAN (Super-Resolution GAN for Chinese Paintings) [49]: A model designed for the super-resolution restoration of traditional Chinese landscape paintings. It employs an Enhanced Adaptive Residual Module and an Adaptive Deep Convolution Block to retain high-frequency details while preserving the painting’s artistic nuances.

NeXtSRGAN (ConvNeXt-based Super-Resolution GAN) [50]: A super-resolution model that integrates a ConvNeXt-based discriminator to improve realism and perceptual quality. It prioritizes visual fidelity over PSNR optimization, making it more suitable for complex restoration tasks.

EdgeConnect (Adversarial Edge Learning for Image Inpainting) [51]: A two-stage generative model that first predicts the missing edges using an edge generator, then reconstructs the missing regions via an image completion network, ensuring structural consistency.

DLP-GAN (Draw Modern Chinese Landscape Photos GAN) [52]: An unsupervised cross-domain image translation framework that translates traditional landscape paintings into modern photo-realistic images. It utilizes a dense-fusion module and asymmetric cycle mapping to balance realism and abstraction.

RestoreNet-Plus (Deep Learning-Based Optical Image Restoration) [53]: A deep-learning-based synthetic aperture imaging restoration network designed for optical imaging systems. It focuses on mitigating imaging blur, turbulence aberration, and noise to enhance restoration quality.

Table 2 presents the quantitative comparison between DCAF-GAN and other state-of-the-art models on the datasets. On the StreetView dataset, DCAF-GAN achieves a PSNR of 29.12 dB, significantly surpassing other methods, indicating its superior ability to restore image details while reducing noise. The SSIM score of 0.867 demonstrates that DCAF-GAN better preserves structural information compared to NeXtSRGAN (0.852) and RestoreNet-Plus (0.846). In terms of perceptual similarity, DCAF-GAN achieves an LPIPS score of 0.137, which is lower than EdgeConnect (0.158) and DLP-GAN (0.168), suggesting that the restored images are visually closer to the ground truth. Additionally, the FID value of 14.4 is significantly lower than other methods, indicating that the generated images exhibit higher overall quality and are more naturally aligned with real-world distributions. On the Places2 dataset, DCAF-GAN again demonstrates superior performance, achieving a PSNR of 28.6 dB, outperforming NeXtSRGAN (27.45 dB) and RestoreNet-Plus (26.95 dB), confirming its effectiveness in large-scale, complex scenes. The SSIM score of 0.854 further validates DCAF-GAN’s ability to maintain structural consistency across various damage types. In terms of perceptual similarity, DCAF-GAN achieves an LPIPS score of 0.143, lower than ConvSRGAN (0.176) and EdgeConnect (0.170), highlighting its capability in restoring fine details while ensuring high visual quality. The FID value of 15.3, the lowest among all methods, suggests that the restored images by DCAF-GAN maintain global coherence and closely resemble real-world image distributions. However, DCAF-GAN’s performance on larger occlusions and highly complex scenes may still be limited, as some blurring and artifacts remain in those regions, particularly in multi-layered and highly intricate structures. Overall, DCAF-GAN outperforms both datasets in PSNR, SSIM, LPIPS, and FID, demonstrating its effectiveness in historical landscape restoration. Its superior performance is attributed to the dual-branch encoder that captures both local textures and global structures, and the CAGF module, which balances feature types. Additionally, the optimized generator and discriminator architectures ensure high-quality restoration. The significant improvement in FID indicates better global consistency and perceptual realism. However, handling larger-scale occlusions and more complex scenarios remains a future challenge.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Quantitative comparison on StreetView and Places2 datasets.

https://doi.org/10.1371/journal.pone.0334532.t002

The experimental convergence curves, as shown in the Fig 3 illustrate the training process of DCAF-GAN on different datasets, including the variation trends of PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) over training epochs. From the figure, it is evident that the model converges rapidly in the early stages and gradually stabilizes, with noticeable differences in performance across datasets. Firstly, in subfigure (a), the training process of DCAF-GAN on the StreetView dataset is depicted. The PSNR curve starts from approximately 20, gradually increasing and stabilizing at around 28.7 after 150 epochs, indicating that the model progressively learns high-frequency details and improves restoration quality. Regarding SSIM, significant fluctuations are observed in the initial phase (first 50 epochs), reflecting instability in learning structural similarity. However, after 100 epochs, SSIM steadily increases and stabilizes at 0.95, demonstrating the model’s capability in restoring structural information effectively. Furthermore, the curves remain stable in the later training stages, suggesting that adversarial training (GAN) has reached an optimal generative effect on this dataset. In contrast, subfigure (b) presents the convergence behavior of DCAF-GAN on the Places2 dataset, where the performance is slightly inferior. The PSNR curve starts from approximately 20, gradually increasing to 28.2, which is slightly lower than that of the StreetView dataset. This suggests that the restoration task on this dataset is more challenging, particularly due to the increased scene complexity, including more occlusions and intricate details. Similarly, SSIM exhibits larger initial fluctuations and stabilizes at around 0.92 after 100 epochs, which is lower than that observed in the StreetView dataset. This implies that the model struggles more to restore globally consistent structures in Places2, possibly due to the dataset’s higher variability and more complex textures. From an overall perspective, the training curves of DCAF-GAN on both datasets display typical GAN training characteristics, where significant initial oscillations gradually stabilize. As adversarial training requires balancing the generator and discriminator, substantial fluctuations occur in the early training stages. However, thanks to the dual-branch CAGF and ConvNeXt-based discriminator, DCAF-GAN is able to stabilize training relatively quickly, entering a steady phase after 150 epochs while maintaining high PSNR and SSIM values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. PSNR and SSIM vs Epoch for DCAF-GAN during training.

(a) shows the performance on the StreetView dataset, and (b) shows the performance on the Places2 dataset.

https://doi.org/10.1371/journal.pone.0334532.g003

The experimental loss curves in the Fig 4 illustrate the training and testing loss trends of DCAF-GAN on the datasets. From the curves, it can be observed that the model experiences a rapid decrease in loss during the initial training phase, indicating its ability to quickly learn fundamental features. As training progresses, the loss gradually decreases and stabilizes, suggesting that the model has reached convergence. On the StreetView dataset, both training and testing losses are relatively low, and the gap between them is small, indicating strong generalization performance with minimal overfitting. In contrast, on the Places2 dataset, the testing loss is slightly higher than the training loss, and the gap between them is more pronounced. This is likely due to the increased complexity of the dataset, making it more challenging for the model to generalize well to unseen samples. Overall, DCAF-GAN achieves stable convergence on both datasets, and despite the increased difficulty in more complex scenarios, it maintains a favorable training trend, demonstrating its adaptability.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The training and testing loss curves for two datasets over 200 epochs.

https://doi.org/10.1371/journal.pone.0334532.g004

Computational cost analysis

Table 3 presents the computational cost comparison of DCAF-GAN with several benchmark models on the datasets. The results demonstrate that DCAF-GAN achieves significant advantages in computational efficiency and inference speed while maintaining a low parameter count and computational complexity. In terms of parameters, DCAF-GAN has only 24.4M (StreetView) and 24.6M (Places2), which is significantly lower than SGRGAN (85.7M/86.1M) and NeXtSRGAN (92.5M/93.0M), reducing the storage requirement by nearly 3-4 times. Compared to lighter models like EdgeConnect (64.3M/65.0M) and DLP-GAN (58.2M/59.0M), DCAF-GAN still exhibits a notable advantage, indicating that the model has undergone effective parameter optimization to minimize redundancy while maintaining high restoration quality. Regarding computational complexity, DCAF-GAN achieves only 19.7G FLOPs on the StreetView dataset and 19.9G FLOPs on the Places2 dataset. This represents more than a 50% reduction in computational cost compared to SGRGAN (42.3G/42.8G) and NeXtSRGAN (45.6G/46.2G). Even when compared to ConvSRGAN (38.1G/38.5G) and EdgeConnect (30.9G/31.5G), DCAF-GAN maintains the lowest computational cost, demonstrating its ability to process high-resolution images efficiently. These results suggest that DCAF-GAN is well-suited for large-scale data processing and real-time applications. For inference time, DCAF-GAN requires only 28.4ms on StreetView and 30.2ms on Places2, significantly faster than SGRGAN (92.5ms/95.2ms) and NeXtSRGAN (105.7ms/108.6ms), reducing the computation time by nearly three times. Even compared to more computationally efficient models like EdgeConnect (54.2ms/56.4ms) and RestoreNet-Plus (38.6ms/40.0ms), DCAF-GAN still maintains a faster inference speed. This is primarily due to the optimization of the dual-branch encoder and the channel attention-guided fusion module, which enhance computational efficiency when handling high-resolution images. Additionally, the use of a lightweight ConvNeXt-based discriminator reduces computational overhead, enabling high-quality restoration with a significantly lower inference cost. Overall, DCAF-GAN outperforms existing models in terms of parameters, computational complexity, and inference time, achieving a more lightweight and efficient design. These advantages make DCAF-GAN an optimal choice for historical landscape restoration and other applications requiring high-quality image restoration while maintaining computational efficiency. Particularly in tasks such as cultural heritage restoration and digital preservation, DCAF-GAN enables precise and efficient restoration with limited computational resources, enhancing its practicality in real-world applications.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Computational cost comparison on StreetView and Places2 datasets.

https://doi.org/10.1371/journal.pone.0334532.t003

Ablation study results and analysis

The ablation study aims to evaluate the contribution of each key module in DCAF-GAN to the historical landscape restoration task. Table 4 presents the results by removing different modules and conducting experiments on the dataset, changes in model performance can be observed. When the dual-branch encoder is removed, PSNR and SSIM significantly decrease, while LPIPS and FID increase, indicating that the absence of independent texture and structure encoding pathways severely affects the restoration capability. The PSNR on the StreetView dataset drops from 29.12 to 24.92, and on the Places2 dataset, it drops from 28.60 to 24.75. This demonstrates that the model struggles to effectively learn high-quality local details and global contours, leading to restoration results that deviate from the ground truth. The increase in FID further indicates a greater discrepancy between the generated images and real image distributions, resulting in unnatural image styles. These results confirm that the dual-branch encoder plays a crucial role in DCAF-GAN by separately extracting local texture information and global structural features, ensuring that the restored region aligns well with the overall scene. Further analysis of the texture encoder and structure encoder shows that removing the texture encoder leads to a slight decrease in PSNR and SSIM, but a more significant increase in LPIPS, suggesting that the absence of this module results in degraded texture quality in the restored regions, making local details appear blurry while maintaining overall contours. The LPIPS value on the Places2 dataset increases from 0.143 to 0.188, and on the StreetView dataset, it rises from 0.137 to 0.182, indicating a noticeable decline in perceptual quality. Removing the structure encoder results in a similar drop in PSNR, but a more significant decrease in SSIM, indicating that this module plays a crucial role in preserving the overall structure of the image. Comparatively, removing the texture encoder causes a greater increase in LPIPS, while removing the structure encoder leads to a more pronounced drop in SSIM. This further validates their complementary roles in DCAF-GAN, with the texture encoder focusing on local details and the structure encoder maintaining global layout consistency. When the channel attention-guided fusion module is removed, PSNR and SSIM decrease, while LPIPS and FID increase significantly. The FID value on the StreetView dataset increases from 14.4 to 17.8, and on the Places2 dataset, it increases from 15.3 to 18.9, indicating that the restored regions become less stylistically coherent with the original image. The channel attention-guided fusion module dynamically adjusts the contribution of texture and structure features, allowing the restored region to better match the style of the original image. Without this module, restored areas may appear inconsistent in style, with noticeable edges or mismatched colors. Finally, when the ConvNeXt-based discriminator is removed, PSNR remains relatively high, but FID deteriorates significantly. The FID value on the StreetView dataset increases from 14.4 to 16.5, and on the Places2 dataset, it increases from 15.3 to 17.1, indicating that the restored images deviate more from real image distributions. Although PSNR is not significantly affected, the absence of the discriminator reduces the visual consistency of the generated images, making the restorations appear less natural and realistic. The ConvNeXt-based discriminator, compared to traditional CNN discriminators, possesses a stronger feature extraction capability, allowing it to more effectively distinguish between real and generated images. This enhances the training of the generator and improves the realism of the restoration results. Consequently, removing this module does not significantly impact pixel-level losses but degrades overall perceptual quality and visual authenticity. Each module in DCAF-GAN is essential for historical landscape restoration. The dual-branch encoder ensures effective texture and structure extraction, with its removal causing the most significant drop in performance. The texture and structure encoders complement each other, while the channel attention-guided fusion module enhances feature interaction. The ConvNeXt-based discriminator improves perceptual quality, making restorations more realistic. Together, these components enable high-quality restoration, demonstrating DCAF-GAN’s potential in cultural heritage preservation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Ablation study results on StreetView and Places2 datasets.

https://doi.org/10.1371/journal.pone.0334532.t004

The above Fig 5 illustrates the restoration performance of DCAF-GAN under different loss weight combinations, evaluating the impact of each loss term on the final generated results. The leftmost column presents the input corrupted images, while the rightmost column shows the corresponding ground truth (GT). The intermediate subfigures (a)-(f) represent the results obtained with different settings of perceptual loss (), structural loss (), and content loss (). The comparison reveals that different loss weight combinations significantly affect the quality of the restored images. In (a), where perceptual loss dominates, the generated images exhibit rich textures and details, but minor structural inconsistencies may be observed. In (b), with a dominant structural loss, the restored images maintain a stable overall shape and contour, though some finer texture details may be missing. In (c), with content loss being the primary factor, the images show strong pixel-level consistency in color but tend to appear overly smooth, lacking realism. In contrast, (d) and (e) demonstrate a balanced integration of perceptual and structural or content loss, resulting in a better trade-off between structural integrity and texture fidelity. Finally, in (f), where all loss terms are equally weighted (), the restoration results achieve the best overall performance, closely resembling the ground truth in terms of global structure, fine details, and color consistency. The red boxes highlight the critical differences in the restored areas, demonstrating that models with stronger structural loss tend to preserve global contours better, while those with dominant perceptual loss emphasize finer details. The overall analysis suggests that carefully balancing different loss terms is crucial for achieving high-quality image restoration. Experimental results confirm that using the full loss combination () leads to optimal PSNR, SSIM, and visual performance, validating the effectiveness of DCAF-GAN in historical landscape restoration.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Qualitative comparison of restoration results under different loss weight settings.

The input corrupted images are shown on the left, and the ground truth (GT) images are shown on the right. (a)-(f) represent the results obtained using different combinations of perceptual loss (), structural loss (), and content loss (). The specific weight settings are as follows: (a) (perceptual loss dominant), (b) (structural loss dominant), (c) (content loss dominant), (d) (balanced perceptual and structural losses), (e) (balanced perceptual and content losses), (f) (full model with all loss terms equally weighted).

https://doi.org/10.1371/journal.pone.0334532.g005

Qualitative comparison analysis

The above Fig 6 presents the visualization analysis of DCAF-GAN in the task of historical landscape restoration. The figure includes the input corrupted images (Input), the restoration results generated by DCAF-GAN (DCAF-GAN), and the corresponding ground truth images (GT). From the comparison results, it is evident that DCAF-GAN can effectively restore the overall structure and fine details of various occluded regions. In the left-side examples, the input images contain significant occlusions, including both regular-shaped (rectangular missing parts) and irregular-shaped damages. The restoration results produced by DCAF-GAN exhibit high similarity to the ground truth in terms of color, texture, and architectural structure. Notably, the model successfully reconstructs high-frequency details, such as columns and walls, demonstrating strong generative capabilities. Additionally, the transitions between restored areas and surrounding regions are smooth, with no apparent boundary artifacts, indicating the model’s robust contextual awareness. The right-side examples further validate the generalization ability of DCAF-GAN across different historical architectural scenes. Even in cases with multiple occlusion blocks, the model effectively reconstructs building contours and fine details, ensuring high visual fidelity. However, in certain large missing regions, the restored details appear slightly blurred, which may result from limitations in global semantic reasoning. Overall, DCAF-GAN demonstrates its effectiveness in restoring historical landscapes across different types of damage. The model achieves superior performance in terms of texture preservation, structural consistency, and semantic coherence, highlighting its potential applications in heritage conservation and digital restoration.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Visualization results of DCAF-GAN for historical landscape restoration.

The figure compares the input corrupted images (Input), the restored images generated by DCAF-GAN (DCAF-GAN), and the corresponding ground truth images (GT).

https://doi.org/10.1371/journal.pone.0334532.g006

User perceptual quality comparison

The results from the user perceptual study are summarized in Table 5, which presents the average ratings of each model across various criteria, including clarity, color consistency, structural consistency, and visual appeal. The models compared include DCAF-GAN (ours), SGRGAN, ConvSRGAN, and EdgeConnect.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. User perceptual quality comparison (scores from 1 to 5).

https://doi.org/10.1371/journal.pone.0334532.t005

The results indicate that DCAF-GAN outperforms the other models across all evaluation metrics. Specifically, DCAF-GAN received the highest ratings in clarity (4.5), color consistency (4.7), and visual appeal (4.8). These results suggest that DCAF-GAN produces restored images that are perceived as clearer, more consistent in color, and more visually appealing compared to the other methods. In contrast, SGRGAN and EdgeConnect received lower scores in terms of clarity and visual appeal, with SGRGAN having particularly poor performance in these areas. ConvSRGAN performed well overall, particularly in color consistency (4.4) and visual appeal (4.5), but still lagged behind DCAF-GAN in most categories. Overall, DCAF-GAN achieved the highest overall rating (4.6), demonstrating its superior performance in restoring historical landscape images. The subjective ratings from the user study align with the objective evaluation metrics, confirming DCAF-GAN’s strong capabilities in producing high-quality, visually consistent restorations.