1.1 Background and motivation

Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Positron Emission Tomography (PET), and Single Photon Emission Computed Tomography (SPECT) each offers unique diagnostic capabilities within clinical practice. MRI provides excellent soft tissue contrast and high spatial resolution making it suitable for anatomical and structural assessment. CT excels in imaging dense tissues such as bone and calcification. On the other hand, PET and SPECT provide valuable functional and metabolic insights by visualizing physiological processes such as glucose metabolism or blood flow. Despite these strengths, no single modality offers a comprehensive diagnostic view, MRI and CT lack metabolic information, whereas PET and SPECT suffer from low spatial resolution and limited anatomical context [2,3].

These limitations have catalyzed the development of Multi-Modal Medical Imaging Fusion (MMIF), which aims to integrate complementary information from multiple imaging sources into a single, more informative representation [4]. MIF plays a vital role in enhancing diagnosis by integrating complementary anatomical and functional information from modalities such as MRI and PET. Accurate preservation of PET’s metabolic color gradients is crucial since these convey functional details relevant for tumor delineation and treatment planning [5,6].

Several classical multimodal fusion techniques, particularly multiscale decomposition frameworks such as wavelet transforms [7], contourlet [8] and shearlet transforms [9], and dual-tree complex wavelets [10], have achieved promising results in enhancing detail and contrast. More recently, sparse-based hybrid models [11] and deep learning-based models [12] have been widely explored for feature-guided integration to improve texture representation and visual interpretability. While these techniques have demonstrated strong detail enhancement, multimodal MRI–PET fusion additionally requires careful preservation of metabolic color information and proper handling of modality-specific noise characteristics.

Although recent hybrid and CNN-guided fusion techniques improve detail preservation, they often fail to retain PET chrominance information since color channels are altered or fused with luminance components during synthesis [13,14]. Furthermore, modality-specific noise characteristics such as Gaussian noise in MRI and Poisson photon noise in PET are rarely addressed jointly, potentially degrading clinical interpretability under realistic acquisition conditions [6,15].

To address these challenges, this work proposes a noise-aware and color-preserving fusion framework operating in the YUV domain. The luminance channel (Y) undergoes SVD-based decomposition and lightweight VGG feature enhancement, while PET chrominance channels (U/V) are preserved intact to maintain functional color fidelity. The objective of the proposed SVD-VGG hybrid approach is to achieve computationally efficient and noise-tolerant MRI–PET fusion, improving anatomical detail and PET interpretability for clinical workflows.

3.1 Preprocessing and noise modeling

In real-world clinical settings, medical images inherently suffer from noise introduced during acquisition and transmission. To evaluate the effectiveness and reliability of the proposed fusion and denoising pipeline under diverse noise conditions, this study employs a controlled synthetic noise simulation approach. Rather than relying on variable real clinical acquisitions with unknown and uncontrollable noise characteristics, synthetic noise is systematically added to clean baseline images from standard public datasets. This controlled-noise methodology is well-established in medical imaging research [46] as it enables: (1) ground-truth reference availability for quantitative evaluation, (2) systematic testing across defined and reproducible noise levels, and (3) direct comparison with state-of-the-art methods using identical noisy inputs, eliminating confounding factors from variable real acquisitions.

The source MRI and PET images used in this study are noise-free reference datasets obtained from the Harvard Medical Image Fusion Dataset repository. To simulate realistic clinical noise degradation, synthetic noise is externally applied post-acquisition. Specifically, Gaussian noise [47], which simulates scanner-induced statistical fluctuations commonly observed in MRI images, is synthetically generated and added to the clean MRI source. Meanwhile, PET images are subjected to Poisson noise (also known as photon noise) [47], which is synthetically generated to simulate the stochastic nature of radioactive decay and photon detection.

A hybrid denoising method is utilized to alleviate these distortions and maintain image integrity prior to fusion. Fig 2 visually depicts the entire workflow of these preprocessing activities. Bilateral filtering is employed in MRI images to enhance smoothness while preserving edges [48]. PET scans are subjected to a two-phase denoising procedure that combines Non-Local Means (NLM) filtering, which utilizes patch similarity, and guided filtering, which improves details through a guiding picture. This guarantees that the inputs to the fusion network are pristine, edge-preserving, and clinically dependable. In MMIF, particularly when amalgamating color-dense PET images with grayscale MRI, it is imperative to maintain both structural and chromatic information. To achieve this, PET images are converted from RGB to YUV color space, enabling the distinction of luminance (Y) and chrominance (U, V) components. This segregation facilitates precise processing, such as structural fusion in the Y channel, while preserving color accuracy via the U and V channels. The YUV color space enables separate handling of luminance (Y) and chrominance (U, V), aligning with our design where structural fusion is best applied on intensity (Y), while U and V preserve PET color fidelity. This separation facilitates modality-specific processing and reduces redundancy compared to RGB fusion, making it particularly well-suited for multimodal fusion involving color-sensitive modalities like PET.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Image pre-processing.

https://doi.org/10.1371/journal.pone.0340781.g002

Two kinds of synthetic noise were added to the input photos before fusion to improve the reliability of the fusion pipeline and evaluate performance under real-world noise situations. Fig 3 offers a visual description of the noise modelling and denoising pathway used on PET and MRI data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Noising and denoising module.

https://doi.org/10.1371/journal.pone.0340781.g003

Gaussian noise for MRI (MR_T1 and MR_T2):

(1)

Where I is the original image and is Gaussian noise with zero mean and variance . The variance parameter is a fundamental statistical measure of noise intensity, mathematically defined as the square of the standard deviation: . Therefore, directly increasing variance proportionally increases the level of noise degradation in images [46]. In accordance with standard denoising and noise-modeling practice, the impact of Gaussian noise on MRI images depends directly on its variance , where larger values induce proportionally stronger degradation of anatomical edges and fine-scale structure [46,47]. This behavior is extensively documented in denoising literature, where multiple noise levels, are routinely evaluated to study PSNR/SSIM degradation and denoising effectiveness [49]. Following this established methodology, we also examined several variance values to characterize noise sensitivity in our MRI–PET fusion pipeline.

Poisson Noise (Photon noise) for PET:

(2)

This reflects the statistical variation in photon emission and detection in PET imaging system [47].

3.1.1 Noise-variance sensitivity rationale.

In accordance with standard denoising and noise-modeling practice, the impact of Gaussian noise on MRI images depends directly on its variance , where larger values induce proportionally stronger degradation of anatomical edges and fine-scale structure. This behavior is extensively documented in denoising literature, where multiple noise levels (e.g., ) are routinely evaluated to study PSNR/SSIM degradation and denoising effectiveness [49]. Following this established methodology, we also examined several variance values to characterize noise sensitivity in our MRI–PET fusion pipeline. This analysis verifies that serves as a meaningful upper-noise condition representative of challenging but clinically plausible MRI acquisition scenarios, thereby justifying its selection for the main experiments.

Denoising: To store visual quality while preserving fine details and edges, a hybrid denoising strategy was applied:

MRI – Bilateral Filtering [48]: A non-linear, edge-preserving filter:

(3)

where:

I_d(x): denoised intensity at pixel x,

G_s, G_r are spatial and range Gaussian kernels,

Ω is the neighbourhood window,

W(x) is the normalization factor,

I(x) is the intensity of neighbouring pixel x.

This non-linear filter smooths the image while preserving edges using spatial and range Gaussian kernels [50,51].

PET – NLM + Guided Filtering:

NLM Filtering [50]:

(4)

where:

: denoised intensity at pixel x,

Ω: search window around x,

I(i): intensity value of neighbouring pixel i,

w(x, i): similarity-based weight between pixels x and i, defined as:

(5)

where:

P_x, P_i are image patches around pixels x and i,

h is the filtering strength,

Z(x) is a normalization term.

This method uses patch similarity to compute weighted averaging for denoising [50].

Guided Filtering [52]

(10)

where:

q_i: filtered image intensity at pixel i,

I_i: guidance image intensity at i,

a_k, b_k: local linear coefficients in window w_k.

These coefficients are computed by minimizing:

(6)

where:

p_i: input noisy image,

ε: regularization parameter.

Guided filtering enforces structural preservation using the input image as a reference guide [52].

3.2 Fusion in YUV color space

Explicitly convert RGB PET images into YUV color space to separate structural (Y) and chromatic (UV) information clearly:

Explicit RGB to YUV. The RGB to YUV color space conversion can be represented as:

(7)

Explicit YUV to RGB (Inverse clearly). The YUV to RGB conversion is given by:

(8)

The pre-processing step therefore not only converts the images into appropriate color space representation (RGB to YUV) but also enhances their quality through noise correction, setting a solid foundation for decomposition and feature extraction.

3.3 Feature extraction using trained VGG19

In this work, domain-specific feature are extracted from pre-processed MRI and PET images using a customized VGG19 network. Only the convolutional backbone VGG19 is kept and refined after the usual fully connected classification layers are eliminated. In order to maintain spatial dimensions, all of the convolutional layers in VGG19 [45] employ 3×3 kernels with padding and stride of 1. These kernels, often referred to as learnable filters, are initially trained using weights that were learned on ImageNet and subsequently adjusted through task-specific training. The number of kernels increases with network depth, from 64 in the first block to 512 in the following layers. This allows the network to collect increasingly complicated and abstract data that is pertinent to multimodal medical imaging. MaxPooling operations with 2 × 2 kernels and stride of 2 are applied after each set of convolutional layers. These learnable actions increase the receptive field and boost computing efficiency by halving spatial resolution while keeping the most prominent activations.

Each convolutional kernel generates a feature map that highlights regional patterns such as structural contours, edges, and textures. A more spatially rich and hierarchical representation of the input is obtained by consolidating the feature maps along the channel axis. VGG19 can effectively extract modality-specific and spatially aligned features from MRI and PET images because to this architecture. The choice of VGG19 as the backbone is motivated by its ability to provide dense low-level and mid-level texture representations that transfer well to multimodal brain MRI-PET data. Its convolutional blocks, initially trained on ImageNet, emphasize edges, contours, and local patterns, which are critical for preserving anatomical structures and metabolic gradients in fusion. Moreover, when used purely as a convolutional feature extractor without the fully connected classification head, VGG19 yields compact 512-channel feature maps at a fixed input resolution, resulting in a relatively low-dimensional feature representation that is computationally efficient for the subsequent SVD-based fusion stage. In contrast, lighter or residual backbones such as ResNet-variants [53] typically produce higher-dimensional, more semantically oriented feature embeddings that are better suited for classification tasks than for pixel-level medical image fusion, where fine anatomical detail and texture fidelity are paramount. By applying a set of data augmentation techniques during training, the feature extractor is further adapted for the medical imaging domain. Resized to 224 × 224 pixels, each image undergoes Gaussian blur, affine transformations, color jittering (hue, brightness, contrast, and saturation), and random horizontal flipping. In terms of mean and standard deviation, pixel intensities are normalized to zero mean and unit variance. L1 feature consistency loss, which promotes stability in the deep features taken from the original image and its augmented counterpart, is used to establish the training target. The network is trained over 10 epochs using the Adam optimizer, which has a learning rate of 1 × 10⁻⁵. The VGG19 feature extractor is trained on the 269 paired brain MRI–PET images from the Harvard Medical Image Fusion Dataset described in Sect 4, whereas the 94 T1-PET and 94 T2-PET pairs from the Harvard Medical School archive are strictly held out for testing and are never used during VGG19 training or hyperparameter selection. Although the medical training set is modest in size, overfitting is mitigated by three factors: (i) only the convolutional backbone of VGG19 is retained and initialized with ImageNet-pretrained weights, which substantially reduces the effective parameter search space compared with training from scratch; (ii) extensive domain-specific data augmentation (random horizontal flip, random rotation within , ColorJitter in brightness, contrast, hue, and saturation, and Gaussian blur) increases the diversity of the training samples and improves robustness to acquisition variability; and (iii) conservative optimization using a small learning rate (1 × 10⁻⁵), StepLR decay, and gradient clipping stabilizes training such that 10 epochs are sufficient to reach a practical performance plateau without observable overfitting in our experiments. The learning rate is gradually reduced throughout training using a StepLR scheduler with a step size of 3 and a decay factor of 0.7. Gradient clipping is used with a maximum norm of 1.0 to prevent gradient explosion. The extensive training architecture enables the network to learn robust and modality-specific semantic properties that are essential for effective MIF. While the L1 consistency loss does not directly encode fusion-task supervision, this design is deliberate for three reasons. First, domain-adaptive training on paired MRI-PET data with medical-specific augmentations (RandomRotation, ColorJitter, GaussianBlur) ensures that the VGG19 extractor learns modality-specific representations adapted to clinical imaging variations rather than generic patterns. Second, the hybrid architecture itself provides implicit fusion guidance: SVD handles structural decomposition with explicit rules, YUV space isolation protects PET chrominance, and the scalar gating mechanism constrains feature outputs to single-weight modulation. These architectural constraints channel self-consistency learning toward fusion-relevant extraction. Third, empirical validation through the ablation study shows that VGG-only fusion achieves an SSIM of 0.8777 for T1-PET, which is approximately three times higher than the SVD-only configuration (0.2943), confirming that self-consistency-trained features effectively facilitate fusion despite the absence of explicit supervision.

A detailed overview of the kernel configuration is presented in Table 2, while the spatial dimensions of feature maps across the network layers are visually illustrated in Fig 4, offering clear insight into the depth-wise and spatial progression of the learned representations. To improve feature generalization across modality variations, the VGG19 [45] model is trained using domain-specific augmentations. The training process is structured in the algorithm 1

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. VGG19 network training.

https://doi.org/10.1371/journal.pone.0340781.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Detailed kernel and feature map configuration in VGG19.

https://doi.org/10.1371/journal.pone.0340781.t002

This training ensures that the VGG19 feature extractor captures modality-specific semantic representations robust to noise and contrast variations. The next stage utilizes these learned features for guiding high-frequency fusion.

The image pixel values are normalized to the range [–1,1] using:

(9)

Data Augmentation: The training images undergo the following transformations:

• Resizing to
• Random horizontal flip (probability = 0.5)
• Random rotation within
• Color jitter (brightness, contrast, hue, saturation)
• Gaussian blur with

where denotes the feature extractor (VGG19), and , are the original and augmented images, respectively.

Optimizer: Adam optimizer is used with a learning rate of .

Learning Rate Scheduler: A StepLR scheduler with a step size of 3 and decay factor of 0.7 is applied.

Gradient Clipping: To prevent exploding gradients, the maximum gradient norm is clipped to 1.0.

Training Duration: The model is trained for 10 epochs.

Algorithm 1. Training of VGG19 feature extractor.

3.3.1 Feature consistency loss function.

To enforce semantic stability across augmentations, an L1-based feature consistency loss is employed:

(10)

where denotes the VGG19-based feature extractor, and are the original and augmented inputs respectively.

3.4 Image decomposition using SVD

Fig 5 shows the whole fusion pipeline comprising decomposition, feature-based fusion, and reconstruction. SVD decomposes both MRI and PET images into low-frequency (LF) and high-frequency (HF) components to isolate structural and detailed content. SVD is applied to the Y(luminance) channel of the PET image, extracted after converting RGB to YUV. For MRI, which is already grayscale, SVD is directly applied to the intensity values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Method architecture.

https://doi.org/10.1371/journal.pone.0340781.g005

SVD:

(11)

where:

I: input grayscale image,

U: orthonormal matrix (left singular vectors),

Σ: diagonal matrix of singular values,

V^T: orthonormal matrix (right singular vectors).

Component Separation:

(12)

(13)

where:

: retains only the top k singular values,

H: captures the discarded details. To determine whether truncated SVD provides any benefit, we evaluated several truncation ranks and compared them against the full decomposition. The results (Table 7) show that small ranks (e.g., k = 10–20) lead to substantial degradation in fusion quality, particularly in SSIM, due to loss of anatomical structure. Increasing the rank to k = 50–100 improves performance markedly; however, for , the performance becomes nearly identical to that of full SVD, with differences below 0.0002 in SSIM and below 0.02 dB in PSNR. This indicates that once the majority of singular energy is preserved, truncation offers negligible additional benefit. Given the minimal performance gain from tuning k beyond 100 and the added hyperparameter complexity, the full SVD is adopted in the proposed method for maximum stability, reproducibility, and elimination of rank-selection sensitivity. Importantly, the computational cost between k = 150 and full SVD is nearly identical, since both require reconstruction using a large portion of the singular spectrum. Therefore, the full decomposition is used throughout the pipeline.

3.5 Fusion strategy

After denoising and decomposition, a two-stage fusion approach is applied.

LF Fusion (L-components). LF components and contain structural information. An adaptive weighting strategy based on energy distribution is applied:

(14)

(15)

The fused LF component is given by:

(16)

where:

E_L = total energy of the LF component,

L(i, j) = intensity at pixel (i, j),

α and β = adaptive fusion weights,

, = energy values of MRI and PET components, respectively.

HF Fusion (HF-components). HF components and contain edge and texture details. Instead of direct averaging, deep features from the trained VGG19 model are used to weight fusion:

(17)

where:

H_f = fused HF component,

, = mean feature activations extracted from VGG19.

This approach leverages deep learning to enhance fine details in the fused image.

Image Reconstruction. After fusion, the final fused intensity image is reconstructed as:

(18)

where:

Y_f = final fused intensity image.

This is then recombined with the U and V channels (from PET YUV conversion) and transformed back to RGB using inverse YUV conversion, ensuring realistic color representation. The overall fusion pipeline is modularized into clearly defined steps encompassing preprocessing, decomposition, adaptive fusion, and reconstruction. The structured pseudocode is represented in algorithm 2

Algorithm 2. SVD-VGG fusion of MRI and PET images.

  Input: Paired MRI and PET images; trained feature extractor

  Output: Fused RGB image

1 Convert PET image to YUV and extract (Y, U, V)

2 Convert MRI to grayscale intensity

3 Decompose luminance channels using SVD:

4   ,

5 Compute adaptive weights using LF energy

6 Fuse LF components:

7 Extract deep features from HF components via

8 Generate fused HF based on feature modulation

9 Reconstruct fused luminance

10 Combine Y_f with original (U, V) and convert back to RGB

11 return Fused image

Complexity (code-level): pixelwise/SVD steps on the luminance image are at the working resolution; the VGG stage runs at a fixed input.

This modular design ensures clarity in execution and supports easy extension or replacement of individual stages. The fusion process retains structural, color, and contextual details effectively through this hybrid pipeline.

3.6 Performance metrics

The following standard measures are employed to explicitly evaluate and logically compare the quality and performance of image fusion:

Entropy (EN). EN measures the informational richness of the fused image. A higher EN value implies more detail and variation in pixel intensities [54].

(19)

where:

L: total number of gray levels (typically 256 for 8-bit images),

P_r(i): probability of occurrence of gray level i.

Standard Deviation (SD). SD evaluates the variation in intensity (contrast) within the image. A higher SD signifies higher contrast [54].

(20)

where:

M, N: number of rows and columns of the image,

f(i, j): pixel intensity at position (i, j),

μ: mean intensity of the image, given by:

(21)

Average Gradient (AG). AG assesses the sharpness and edge information in the image. A higher AG indicates clearer edges [54].

(22)

where:

: horizontal gradient,

: vertical gradient,

f(s, t): pixel value at local (s, t),

M, N: number of rows and columns of the image.

Mean Square Error (MSE). The MSE quantifies the average squared deviation of the fused image from the Ground Truth (GT) image. Lower MSE indicates better similarity [54].

(23)

where:

R_i: pixel value of the GT (reference) image at index i,

F_i: pixel value of the fused image at index i,

MN: total number of pixels.

Peak Signal-to-Noise Ratio (PSNR). PSNR assesses the reconstruction quality of the fused image in comparison to the GT. An increased PSNR signifies superior quality [54].

(24)

where:

: maximum possible pixel intensity (255 for 8-bit images),

: Mean Square Error.

Structural Similarity Index (SSIM). SSIM measures perceived image quality analyzing the structure, contrast, and brightness of the fused image with the GT image [54].

(25)

where:

, : mean pixel intensities of images I (original) and F (fused),

, : variances of I and F,

: covariance between I and F,

C₁, C₂: stabilizing constants.

Correlation Coefficient (CC). CC measures the linear correlation between the fused and GT images. Values approaching 1 signify greater similarity [54].

(26)

where:

R_i, F_i: pixel intensities of GT and fused images,

, : mean intensities of R and F,

, : standard deviations of R and F,

MN: total number of pixels.

Learned Perceptual Image Patch Similarity (LPIPS). LPIPS quantifies perceptual similarity [33] by comparing deep features from a pretrained convolutional neural network (e.g., VGG, AlexNet). Unlike pixel-wise metrics such as PSNR or SSIM, LPIPS better reflects human visual perception, making it a valuable metric for assessing image fusion quality in clinical settings [55,56].

LPIPS is defined as:

(27)

where denotes the feature map of image i at the l-th layer of a pretrained network, H_l, W_l are spatial dimensions, and w_l are learned channel-wise weights. The operator represents element-wise multiplication.

Multi-Reference Evaluation:

(28)

(29)

(30)

Lower LPIPS values indicate higher perceptual similarity. In our study, all three comparisons are reported to evaluate perceptual fidelity of the fused images.

3.7 Implementation environment and hardware specifications

Training was conducted on Google Colaboratory (Python 3.8, PyTorch 1.10) using an NVIDIA Tesla T4 GPU (16 GB GDDR6, 2,560 CUDA cores, 320 Tensor Cores), Intel Xeon CPU (2.3 GHz, 2 cores), and 12 GB RAM. The model was trained for 10 epochs with an initial learning rate of 1 × 10⁻⁵, a StepLR scheduler (step size = 3, ), and gradient clipping (max-norm = 1.0).

Inference was performed on a local HP Laptop 15s-ey2xxx (Windows 11 Home, AMD Ryzen 3 5300U @ 2.60 GHz, 4 cores, 8 GB DDR4 RAM) using Python 3.x with OpenCV 4.x, NumPy 1.x, Pandas 1.x, and LPIPS 0.1. The average end-to-end inference time for a single 224×224 MRI–PET pair was approximately 0.56 seconds, demonstrating real-time capability on standard CPU hardware.

3.8 Computational complexity analysis

The computational complexity of the proposed SVD–VGG fusion pipeline is analyzed by decomposing it into three main stages: luminance decomposition, deep feature extraction, and fusion–reconstruction.

Decomposition Phase: The Singular Value Decomposition (SVD) of an image matrix of size M × M has a theoretical time complexity of O(M³). In the proposed method, SVD is applied to fixed-resolution medical image slices of size 256 × 256, which are commonly used in slice-based multimodal medical image fusion. As a result, the SVD stage incurs a fixed per-image computational cost under the working resolution used in this study. Although this complexity is superlinear with respect to the number of pixels N = M², it remains practical and manageable for 2D fusion scenarios and avoids the need for iterative optimization, nuclear-norm minimization, or patch-wise decomposition.

Feature Extraction Phase: The VGG19-based feature extractor operates on inputs resized to a fixed spatial resolution of , independent of the original image dimensions. During inference, only the convolutional backbone of VGG19 is used, and no backpropagation or iterative refinement is performed. Consequently, the feature extraction stage incurs a constant inference cost per image pair, denoted as , where represents the fixed number of operations in a forward pass of the truncated VGG network at the specified resolution.

Fusion and Reconstruction Phase: The computation of adaptive fusion weights, deep-eq:img-reconstructionfeature-based modulation of high-frequency components, and final image reconstruction (Eqs 16 to 18) are all performed using pixel-wise operations. These steps therefore scale linearly with the number of pixels, resulting in a complexity of O(N).

Empirical Runtime: Despite the cubic theoretical complexity of SVD, the fixed working resolution ensures fast execution in practice. Experimental evaluation conducted on a standard CPU platform (AMD Ryzen 3 5300U) demonstrates an average end-to-end inference time of approximately 0.56 seconds per image pair (0.5827 s for T1–PET and 0.5619 s for T2–PET). These results confirm that the proposed SVD–VGG fusion framework is computationally efficient and suitable for practical clinical imaging workflows, without relying on heavy optimization procedures, large transformer models, or GPU acceleration.

3.9 Modular overview of proposed architecture

The proposed SVD-VGG framework integrates several submodules as outlined:

• Preprocessing: Noise modeling and denoising using hybrid filters.
• Color Transformation: RGB to YUV separation for structural (Y) and chromatic (UV) components.
• Decomposition: SVD applied to extract LF and HF components.
• Deep Feature Extraction: Trained VGG19 for HF enhancement.
• Fusion Strategy: LF fused using energy-adaptive weights; HF fused using semantic feature weighting.
• Reconstruction: Final image formed from fused Y and original U, V, followed by inverse YUV conversion.

4.1 Dataset

The experiments conducted in this study utilized a combination of publicly accessible brain imaging datasets to comprehensively evaluate the proposed SVD-VGG fusion framework. All images used in the study were uniformly resized to dimensions of pixels for consistent processing.

The training set was constructed using paired MRI and PET images obtained from the Harvard Medical Image Fusion Dataset repository, hosted on GitHub: https://github.com/xianming-gu/Havard-Medical-Image-Fusion-Datasets. This repository provided 269 paired MRI and PET images, which were used to train the customized VGG19 network with the application of various augmentation techniques to enhance robustness.

For testing and evaluation, a separate dataset was sourced from the Harvard Medical School’s publicly accessible archive: https://www.med.harvard.edu/aanlib/cases/caseNA/pb9.htm. From this source, a total of 94 PET images, 94 T1-weighted MR images, and 94 T2-weighted MR images were collected. Their corresponding ground truth (GT) fused images, comprising 94 T1-PET and 94 T2-PET pairs, are also obtained from the same Harvard dataset to facilitate ground truth-based metric evaluation and visual comparisons.

The training and testing datasets are obtained from two Harvard-affiliated repositories: the GitHub-based Harvard Medical Image Fusion Dataset and the Harvard AANLIB archive. Although a limited number of visually similar images exist across these sources (approximately 10% of the training data), they are independently curated, differently labeled, and lack explicit correspondence or paired indexing. Images from the GitHub repository are used exclusively to train the VGG19 feature extractor, while AANLIB images are used solely for fusion evaluation. Since the VGG19 model is trained without fusion targets or test-driven hyperparameter tuning, this partial overlap does not result in information leakage, and the reported results reflect genuine generalization of the proposed fusion strategy. No separate validation set or early stopping strategy was employed during VGG19 training. The feature extractor was trained for a fixed number of epochs using predetermined hyperparameters and a self-consistency loss, without any test-driven or validation-based hyperparameter tuning. The following subsections present a detailed evaluation of the proposed and comparative methods, beginning with quantitative metric analysis, followed by ablation study, visual assessment, histogram-based intensity distribution analysis, and energy behaviour evaluation.

4.2 Quantitative analysis

4.2.3 T2-PET fusion results.

Table 4 presents the core performance metrics across all methods of T2-PET fusion. The proposed SVD-VGG achieved:

• PSNR: 33.94 dB-(highest among all methods)-Superior reconstruction quality demonstrates consistent handling of different anatomical contrasts (T2 vs. T1), indicating robustness to modality variations.
• MSE: 0.000425 (lowest - highest reconstruction accuracy)-Achieves superior intensity fidelity compared to T1-PET, particularly important for preserving PET’s metabolic intensity gradients which encode functional information.
• SSIM: 0.9353 (among the highest)-Exceptional structural preservation across different MRI weightings, demonstrating that edge and anatomical detail preservation does not degrade with different modality contrasts.
• Correlation Coefficient (CC): 0.9715 (highest)-Maintains superior spatial alignment even with T2’s different contrast characteristics, indicating stable multimodal integration.
• Runtime: 0.5619 seconds per fusion (fastest)-Consistent sub-second performance regardless of MRI modality, supporting clinical feasibility across diverse imaging protocols.

The method achieved the best balance across all critical metrics while maintaining sub-second inference on standard CPU hardware, demonstrating robust, clinically applicable performance across anatomical variations. Figs 6 and 7 provide graphical representation of the GT-based metrics such as PSNR, MSE, SSIM, and CC and non-GT-based metrics such as EN, SD, and AG for both T1-PET and T2-PET fusion results respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Evaluation of GT-based metrics for T1-PET and T2-PET fusion: PSNR, MSE, SSIM and correlation-coefficient.

https://doi.org/10.1371/journal.pone.0340781.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Evaluation of non-GT-based metrics for T1-PET and T2-PET fusion: EN, SD, AG.

https://doi.org/10.1371/journal.pone.0340781.g007

4.2.4 LPIPS-based perceptual quality evaluation.

Beyond pixel-level and structural metrics, we incorporated the LPIPS (Learned Perceptual Image Patch Similarity) metric to assess perceptual fidelity aligned with human visual perception. This metric addresses a key limitation of traditional metrics (PSNR, SSIM): they often correlate poorly with human visual quality assessment and clinical diagnostic utility. LPIPS, trained on human perceptual similarity judgments, provides a psychophysical anchor for fusion quality. LPIPS values are bounded in [0,1], where lower values indicate greater perceptual similarity; in practice, differences on the order of 0.02–0.03 correspond to visible but modest perceptual improvements rather than large qualitative changes.

For medical imaging, where radiologist interpretation is the gold standard, perceptual alignment is as important as pixel-level accuracy. Table 5 summarizes LPIPS results:

• T1-PET: SVD-VGG achieved LPIPS = 0.0849, remaining highly competitive with the best performer (GeSeNet: 0.0645) while significantly outperforming ASFE (0.1928) and MATR (0.1050). This validates that the fused image maintains perceptual quality expected by radiologists for clinical assessment.
• T2-PET: SVD-VGG achieved LPIPS = 0.1016 (top three performers), demonstrating consistent perceptual quality across different anatomical contrasts.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. LPIPS scores for T1-PET and T2-PET fusion compared to ground truth.

https://doi.org/10.1371/journal.pone.0340781.t005

These results confirm that SVD-VGG maintains perceptual consistency in addition to structural and statistical fidelity, bridging the gap between mathematical metrics and clinical visual assessment.

Rationale for Secondary Metric Trade-offs: SVD-VGG achieved lower Entropy (4.88–4.95) and Average Gradient (0.024-0.043) compared to texture-focused methods (ASFE: EN 6.17, AG 0.044; GeSeNet: EN 5.45, AG 0.069). This reflects an intentional design choice: texture diversity and edge enhancement are deprioritized in favor of structural clarity and diagnostic accuracy. High EN/AG values often indicate over-enhancement of texture, potentially amplifying noise artifacts and reducing specificity. For PET-MRI fusion, where metabolic color gradients are diagnostic (not texture), texture maximization is clinically inappropriate and may reduce radiologist confidence in the result.

4.3 Statistical significance evaluation using ANOVA

A one-way ANOVA was performed to assess whether the performance differences among the seven fusion methods were statistically significant for each primary metric (PSNR, SSIM, CC and LPIPS). The ANOVA F-statistic is computed as:

(31)

Across both T1-PET and T2-PET experiments, all metrics yielded p-values < 0.05 Table 6, indicating that fusion method differences are statistically meaningful rather than random variations. The proposed SVD-VGG consistently ranked among the top performers in PSNR, SSIM and CC, confirming that its improvements are statistically significant and not due to noise or sampling variation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. One-way ANOVA p-values across fusion methods for T1-PET and T2-PET metrics.

https://doi.org/10.1371/journal.pone.0340781.t006

As shown in Table 6, all computed p-values fall well below the 0.05 threshold, indicating statistically significant performance differences between fusion algorithms for every evaluated metric. This supports the hypothesis that fusion quality varies meaningfully across methods, thereby justifying further comparative analysis. The one-way ANOVA results indicate that statistically significant differences exist among fusion methods; however, ANOVA alone does not specify which particular method pairs differ significantly.

T1-PET Box Plot Discussion. The box plots shown in Fig 8 for T1-PET metrics illustrate consistent performance for most fusion methods, with relatively compact interquartile ranges for metrics such as PSNR, SSIM, and CC. The proposed SVD-VGG method exhibits central tendencies closely aligned with or slightly above the median reference line in key metrics, indicating stable and effective fusion. In perceptual quality metrics like LPIPS, the proposed method performs competitively with limited dispersion, suggesting robustness in visual fidelity. While some methods exhibit a broader spread or more outliers, particularly in MSE and AG, SVD-VGG avoids extreme fluctuations, showcasing consistent behavior without erratic degradation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Boxplots with mean ± std for T1-PET fusion across 7 methods.

(a) Entropy, (b) Standard Deviation, (c) Average Gradient, (d) PSNR (GT), (e) MSE (GT), (f) SSIM (GT), (g) Correlation Coefficient (GT), (h) LPIPS (Ref) — perceptual similarity.

https://doi.org/10.1371/journal.pone.0340781.g008

T2-PET Box Plot Discussion. T2-PET box plots shown in Fig 9 reveal higher variability compared to T1-PET, especially in SD and AG, indicating a more challenging fusion scenario. Despite this, the proposed method maintains balanced performance, with median-aligned results and low variance in metrics like SSIM, MI, and CC. Notably, SVD-VGG shows fewer or no extreme outliers across all metrics, emphasizing its stability and resilience to input fluctuations. The method’s performance in perceptual fidelity (low LPIPS) further demonstrates its strength in preserving both anatomical and functional content, even in a noisier fusion context. These characteristics make it a clinically promising candidate despite not always achieving the absolute best metric score.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Boxplots with mean ± std for T2-PET fusion across 7 methods.

(a) Entropy, (b) Standard Deviation, (c) Average Gradient, (d) PSNR (GT), (e) MSE (GT), (f) SSIM (GT), (g) Correlation Coefficient (GT), (h) LPIPS (Ref) — perceptual similarity.

https://doi.org/10.1371/journal.pone.0340781.g009

4.4 Ablation study

To evaluate the individual contributions of each core module in the proposed SVD-VGG model, we conducted a comprehensive ablation study. This included thre below baseline configurations:

4.4.4 Effect of SVD truncation rank.

To further examine the influence of singular value preservation on fusion performance, an additional ablation experiment was conducted by varying the truncation rank k in the SVD decomposition. The proposed pipeline typically utilizes the full SVD (i.e., all singular values), but for analysis, we evaluated and compared these against the complete decomposition. Table 7 summarizes the resulting fusion quality for both T1-PET and T2-PET cases. The results reveal that very low ranks (k = 10–20) substantially degrade the quality of the fused image, as reflected by lower SSIM and higher entropy due to loss of essential anatomical content. Mid-range ranks (k = 50–100) improve performance but still exhibit noticeable deviations compared to the full-rank output. For , performance converges to that of the full SVD, with differences below 0.0002 in SSIM and below 0.02 dB in PSNR. This saturation trend indicates that once the dominant singular energy is retained, further truncation offers negligible benefit. Given the minimal performance gain from tuning k beyond 100 and the added hyperparameter complexity, the proposed method opts for the full SVD to ensure maximum structural preservation and reproducibility across modalities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Effect of SVD truncation rank k on fusion quality for T1–PET and T2–PET image pairs.

https://doi.org/10.1371/journal.pone.0340781.t007

4.4.5 Effect of noise levels and denoising pipeline.

To quantify the effect of acquisition noise on fusion behavior and to address the reviewer’s concern regarding noise justification, we extend our analysis by examining multiple Gaussian noise variances applied to MRI inputs. MRI images were degraded using , while PET images were corrupted using Poisson photon noise. Table 8 summarizes the corresponding fusion metrics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Fusion performance under varying Gaussian noise variances for MRI and Poisson noise for PET.

https://doi.org/10.1371/journal.pone.0340781.t008

Consistent with established denoising theory [49], increasing results in a monotonic decrease in PSNR and SSIM due to greater corruption of high-frequency structures, while entropy exhibits a slight upward trend from noise injection. The hybrid denoising step stabilizes luminance structure but introduces a controlled reduction in PSNR, which is expected when noise and edge components overlap in high-frequency domains. This ablation clarifies the role of the denoising module and validates the chosen noise parameters within the fusion pipeline.

4.4.6 Effect of color-space choice YUV Vs YCbCr.

To evaluate whether the choice of color space influences fusion quality, we compared YUV and YCbCr within the same SVD–VGG pipeline for both T1-PET and T2-PET pairs. Although both spaces separate luminance and chrominance, YUV provides a linear luminance representation that aligns more naturally with SVD-based decomposition. As shown in Table 9, the observed differences are extremely small (PSNR variations  dB and SSIM differences <0.0003), confirming that the fusion outcome is effectively invariant to the chosen color space. Therefore, YUV is adopted in the proposed method due to its simpler linear formulation and seamless integration with the SVD decomposition stage.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Ablation on color-space choice: Comparison between YUV and YCbCr for T1–PET and T2–PET fusion.

https://doi.org/10.1371/journal.pone.0340781.t009

4.4.7 Comparative observations.

Quantitative Metric Comparision

T1-PET: SVD-only achieved highest EN (6.73) and AG (0.1577), it shows the lowest SSIM (0.2943) and MSE is higher (0.001056), indicating texture richness but poor structural similarity. RGB-based fusion shows high EN (6.14) and SSIM (0.4814), but increased MSE, confirming it captures contrast well but lacks fine semantic guidance. VGG-only delivers strong SSIM (0.8776) with moderate EN (4.61), demonstrating semantic clarity with slightly reduced textural richness. The proposed SVD-VGG model exhibits the best SSIM (0.9130) and CC (0.9704), reflecting high semantic preservation and modality alignment.

T2-PET: SVD-only again results in higher EN (6.30) but very low SSIM (0.276), confirming lack of fine structure preservation. RGB-based fusion achieves moderate EN (5.81) and MSE (0.000739), but correlation drops (0.524), showing misalignment in details. VGG-only maintains high structural similarity SSIM (0.9066), confirming feature-based fusion’s effectiveness even without frequency decomposition. The proposed model achieves best PSNR (33.51 dB), lowest MSE (00047), and highest SSIM (0.9175), offering the most balanced performance.

A detailed summary of these metrics’ values across all configurations and fusion types is presented in Table 10.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Quantitative performance metrics of ablation study methods compared to the SVD-VGG (proposed) model for T1-PET and T2-PET fusion.

https://doi.org/10.1371/journal.pone.0340781.t010

4.4.8 Visual assessment.

A representative visual comparison of T1-PET fused outputs across these ablation variants that are RGB-SVD-VGG, SVD-Only, VGG-Only and the full proposed model is shown in Fig 10. The RGB fusion appears colorful but suffers from noticeable blur and reduced structure retention. The SVD-only output, through structurally rich, introduces noise and poor SSIM, while the proposed model clearly demonstrates superior edge retention, intensity balance, and anatomical fidelity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Visual comparison of T1-PET fused outputs across different ablation variants and SVD-VGG (proposed) model.

https://doi.org/10.1371/journal.pone.0340781.g010

The suggested approach employs YUV conversion, facilitating the segregation of structural (Y) and color (U, V) components, hence restricting extensive processing to a single channel. Conversely, the RGB-based system processes three whole channels, hence augmenting redundancy and computational demands. This design decision markedly decreases execution time, as evidenced by empirical data, enhancing the efficiency and scalability of the YUV-based SVD-VGG model for clinical applications.

4.5 Visual comparison of fusion outputs

Figs 11 and 12 present qualitative comparisons of the fused T1-PET and T2-PET images across all evaluated methods. The proposed SVD-VGG method preserves PET metabolic color patterns while enhancing MRI structural detail, particularly at cortical boundaries and soft-tissue regions. In contrast, texture-amplifying methods such as GeSeNet and MSRPAN introduce sharper but noisier edges, while ZLF and MATR tend to produce smoother images with reduced anatomical clarity. These visual differences are consistent with the quantitative trends reported earlier and highlight the balance achieved by the proposed method between structural fidelity and color preservation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Visual comparison of T1-PET fusion outputs alongside input modalities (MR-T1 and PET) across different methods.

https://doi.org/10.1371/journal.pone.0340781.g011

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Visual comparison of T2-PET fusion outputs alongside input modalities (MR-T2 and PET) across different methods.

https://doi.org/10.1371/journal.pone.0340781.g012

4.6 Histogram and energy-based intensity behavior

The histogram and energy analyses showing in Figs 13, 14, 15 and 16 provide supporting insight into how different fusion algorithms handle intensity distribution and contrast behavior. The proposed SVD–VGG method maintains stable PET metabolic intensity patterns while avoiding excessive contrast stretching, resulting in smoother and more clinically interpretable transitions across luminance levels.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Histogram representation of original input images: MR-T1, MR-T2, and PET.

https://doi.org/10.1371/journal.pone.0340781.g013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Histogram representations of T1-PET and T2-PET fused outputs generated by different fusion methods.

(a) ASFE (b) FATFusion (c) GeSeNet (d) MATR (e) MSRPAN (f) ZLF (g) SVD-VGG (proposed).

https://doi.org/10.1371/journal.pone.0340781.g014

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. Energy distribution of input modalities: MR-T1, MR-T2, and PET.

https://doi.org/10.1371/journal.pone.0340781.g015

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. Energy distribution plots of T1-PET and T2-PET fused outputs generated by different fusion methods: (a) ASFE (b) FATFusion (c) GeSeNet (d) MATR (e) MSRPAN (f) ZLF (g) SVD-VGG (proposed).

https://doi.org/10.1371/journal.pone.0340781.g016

Compared with methods that produce very high EN or AG values, which often correspond to over-amplified texture and may distort PET metabolic gradients the SVD–VGG fusion maintains balanced luminance–chrominance behavior and avoids artificial contrast inflation.

Because primary diagnostic conclusions are already well supported through PSNR, SSIM, CC, LPIPS, and runtime, the histogram and energy evaluations are included as secondary qualitative evidence to illustrate intensity stability rather than to drive performance claims. They reinforce that the proposed method enhances detail without disrupting functional color distribution.