https://orcid.org/0000-0002-7044-6783
Figures
Abstract
Brain tumor segmentation from MRI’s and PET has always been a challenging and time-consuming phase for radiologists, due to low sensitivity boundary region pixels in this image modality. Deep learning-based image segmentation is the hot research topic in recent days. Among all other deep learning models, U-Net-based variants are the most used models to segment medical images with respect to different modalities. In this paper, a Permutate version of the U-Net architecture was designed that precisely and automatically detects the boundaries of the tumour area and segments tumour regions from the fused image. There are two stages to the proposed work. In the first stage Principal component analysis (PCA) is used to fuse the MRI-PET images to enhance the fused image’s quality and improved interpretation. Later, a Permutate U-Net architecture is employed to precisely segment tumour region from the fused image. Further designed model performance is assessed using Dice Coefficient, intersection over union score (IoU) and accuracy with brain tumour segmentation challenge BraTS datasets of 2015, 2020 and 2021. Our proposed method demonstrates promising results that are superior to existing deep learning model and comparatively higher than the existing methods.
Citation: Karuna Y, Allapakam V, Priyanka S, Hussian SR, Nalwaya P, Saritha S (2025) Brain tumour segmentation in fused MRI-PET images with permutate U-Net framework. PLoS One 20(12): e0335952. https://doi.org/10.1371/journal.pone.0335952
Editor: Ananth JP, Dayananda Sagar University, INDIA
Received: May 9, 2025; Accepted: October 17, 2025; Published: December 4, 2025
Copyright: © 2025 Karuna et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data is publicly available data here: https://www.kaggle.com/datasets/mateuszbuda/lgg-mri-segmentation, https://www.med.harvard.edu/aanlib/,BraTS 2020 Dataset (Training+Validation)(kaggle.com), BRaTS 2021 Task 1 Dataset (kaggle.com).
Funding: The author(s) received no specific funding for this work.
1. Introduction
The brain is sophisticated bodily organ that manages a wide range of functions, including thought, movement, respiration and every bodily regulation process. It also coordinates the body’s responses to stimuli, enabling us to move, communicate, and interact with the world around us; thus, brain’s health plays a very crucial role in the proper regulations, functioning of overall body. In today’s world, it has been a noticeable change in habits and lifestyles, leading to health problems, including tumours treated as threats to mankind. Cancer was one of the foremost causes of mortality world-wide in 2020, approximately 10million deaths. A WHO report listed brain tumours as one of the most prevalent cancers and their incidence is rising. The International Association of Cancer Registries (IARC) estimates that over 24,000 people lose their lives to brain tumours each year [1–2].
Brain cancer is predicted to develop annual growth rate of 1.11% by 2030, according to a Delve Insight analysis. An estimate is that 40,000–50,000 persons in India receive a brain tumour diagnosis each year. Children make up approximately 20% of these [3]. The number could be far higher, according to experts, as many incidents, particularly in rural areas, go unreported.
The key to treating a brain tumour is spotting it accurately. The variety of distinct types of brain tumours makes diagnosis very difficult. A good classification process can provide suitable care, which in turn depends on the effective segmentation of the tumour. Medical images of different modalities are used for the detection of brain tumours. Different modalities provide unique and specific information, with some offering only structural information while others provide functional data. However, for proper diagnosis and analysis, more detailed information with improved imaging quality is key factor [4]. In most cases, MRI and PET images are the most frequently used, because the processes have no harmful effect on the human body and provide precise information for diagnosis.
MRI images are widely used in the detection of various medical conditions [5]. These images provide highly detailed and precise features of the internal organs, including the brain, without subjecting the patient to radiation injury, which helps in the early detection of the cause and intern helps the doctor plan and execute appropriate treatment. They are particularly helpful in identifying the kind, size, and location of brain tumours as well as other illnesses such as multiple sclerosis, stroke, and joint issues. Overall, MRI images are of great importance in the medical domain.
On the other hand, PET scans are generated by gamma rays that are emitted by a small radioactive material to determine the internal structures. PET images are widely used in the medical domain for the analysis of a variety of medical conditions [6]. PET scans can assess the metabolic activity of tumours and aid in the detection of cancer. PET scans reveal the biochemical activity of tissues and organs, enabling medical professionals to identify cellular abnormalities. For patient treatment planning and management, PET scans are very helpful in identifying tumour recurrence and metastasis.
Brain tumour identification and segmenting the tumour area rely heavily both on MRI and PET images, but due to low sensitive boundaries pixel regions of this image modality it is very hard to accurately segment it. Additionally, manual segmentation process is computationally intensive and time-consuming due to complex tumour shapes, which mainly depends on expertise. Shortage of experience radiologist also challenges issue need to be consider and traditional segmentation methods always results inaccurate segmentation.
Therefore, the contribution of this work is to combine both imaging modalities to generate fused output using appropriate fusion method that results in improved imaging quality in terms of both qualitatively and quantitatively [7–8]. And In addition, this research will look at various advances and improvements in the U-Net design as well as present techniques, all with the objective of emphasizing U-Net’s ongoing potential for improving brain tumor segmentation performance.
The rest of the paper is organized as follows: The literature survey is presented in Section II. The proposed methodology along with a) the Data Set b) the Pre-processing Steps c) the PCA algorithm for fusion and d) the Proposed U-Net model is described in section III. Evaluation metrics are discussed in section IV. Section V deals with the results part. Finally, discussion and conclusion are presented in Section VI
2. Literature survey
Brain tumour segmentation is always thought-provoking task due to low sensitivity boundary region pixels in this image modality. Conventional methods of Segmenting the tumour region always results inaccurate results and Traditional U-Net model lags in accurate segmentation. Several works carried out in past based on various machine & deep learning techniques that focus on segmentation of brain tumour. Here, reviews some of the works pertinent to the medical images, followed by the identification of their shortcomings.
In stark contrast to the conventional manual learning method, automatic segmentation based on deep learning (DL) techniques became very popular and CNN frameworks with U-net is most significant one.The benefit of this network structure is that it not only processes medical images successfully and objectively evaluates them but also helps to increase the accuracy of diagnoses. It can accurately segment the required features. The U-net network structure was used in this article to address a few issues with picture shadow and overlap for six different imaging systems. In addition to being a DL innovation has laid the groundwork for accurate pathological diagnosis by doctors [9].
Norouzi, Alireza, et.al explain the various modern image processing technique which are frequently employed in medical image analysis. They discuss four different segmentation techniques: the region-based approach, clustering, classification, and hybrid methods. For several medical images, it was shown that the fuzzy C-mean has greater accuracy. The hybrid approaches combine the region and boundary attributes. The graph-cut method offers the best N-dimensional segmentation results using cost function [10].
Hesamian, Mohammad Hesam, et.al offer a critical evaluation of popular approaches to medical image segmentation making use of deep learning techniques. They have discussed practical solutions to a range of problems [11].
The U-net design, one of the extensive DL architectures for image segmentation, employed in this study by Skourt, BrahimAit, et.al for lung CT image segmentation. The structural design is comprised of an encoder path that contracts to extract high-level information and a decoder path that expands symmetrically to restore the required information. This network outperforms many others and can be trained with a small number of images. The goal of this work was to segment lung nodules [12].
Dhana Chandra, Nameirakpam, et.al presented a segmentation model, using the unsupervised K-means clustering technique. A median filter is used to enhance the segmented image. They claim that the clustering technique has better segmentation accuracy. As future work, they suggested a morphological operation to enhance the attributes of output image [13]
Huang, Hong, et al. proposed a combined FCM clustering algorithm with rough-set theory. First, segmentation findings of FCM are used to build an attribute value table, and the picture is then divided into numerous small sections based on the indistinguishable relationship between the attributes. The next step is to determine the weight values for each attribute, which are then utilized as the foundation for calculating the differences across regions. Using an equivalent connection, each region’s similarity is assessed. According to the experimental findings, they propose better segmentation outcomes while reducing error rates when compared to the FCM method [14].
Chowdhary, et al have described a few image segmentations and feature extraction techniques that are relatively new in medical imaging, especially for mammography. If mammograms are utilized without improvement, the radiologists may make wrong judgments. Principal and independent component analysis are frequently employed for dimensionality reduction. Clustering is the best segmentation technique out of all of them, and it is employed in a variety of applications [15]
The hybrid approach suggested by Özyurt et.al makes use of neutrosophy and convolutional neural networks. During the classification stage, CNN collected the characteristics of the segmented brain images and categorized those using SVM and KNN classifiers. The results showed that CNN features have excellent classification performance across a range of classifiers. According to experimental findings, CNN features performed better at classification with SVM [16].
A depth variation analysis of the U-NET architecture for brain tumor segmentation was proposed by Jena, Biswajit, et al. in 2023 using the BraTS-2017 and BraTS-2019 datasets, which include High-Grade Glioma (HGG) and Low-Grade Glioma (LGG) MR Scan. An effective examination of the depth variation of the U-NET architecture—that is, after removing various layers—has been carried out in this work. The cross-validation replication cohort had very low dice coefficient of at least 0.8895 and as high as 0.8911, while the discovery cohort had a dice coefficient of at least 0.8866 and as high as 0.8887 [17].
In 2024, Lin, Shu-You, and Chun-Ling Lin presented a U-Net and Efficient Net v2 for brain tumour segmentation using the BraTS-2019 data set. In this work, the traditional U-net architecture is combined with an Efficient Net V2 encoder to enhance the model performance of the U-net architecture. The results of the experiment demonstrated that the model successfully segmented the tumor region with a minimal dice score of −0.0866, a dice similarity coefficient (DSC) of 0.9133, and a high accuracy of 0.9977 [18].
Using the Brats2019 dataset, Zhang, Yajie, et al. (2024) presented an attention mechanism to help the network focus on significant regions, overcoming the limitations of conventional U-net based segmentation techniques in handling very small targets and fuzzier boundaries. The proposed model makes use of multiscale feature fusion strategy to improve the efficacy of network segmentation at various scales. The experimental results demonstrate that the proposed methodology exhibits superior speed and efficiency. But the model results very low dice coefficients for the multiple branch TS-U-net model were 0.876, 0.868, and 0.814 in the tumour subregions of WT, TC, and ET, respectively [19].
Thus, there are various techniques available for detection of brain tumours. However, after going through the above works, a few drawbacks have been identified and mentioned below.
- The laborious process of manually segmenting with huge amount of data and shortage of experienced radiologist, always results in poor delineation.
- Delineation of tumour boundaries with accurate segmentation is very crucial and always a challenging task.
- Traditional U-Net model lags in accurate segmentation of region of interest of brain tumours with very low performance metric.
3. Proposed fusion based segmentation model
To addresses the above limitation, The proposed Permutate U-Net framework with additional layer in encoder and decoder are designed to perform segmentation task effectively from fused-MRI-PET images. The overall proposed configuration is categorized into two stages. Firstly, the fusion of MRI-PET images using PCA is taken up, The PCA-based fusion method combines the features of both MRI & PET images, resulting in more comprehensive and informative image with improved imaging quality for brain tumour segmentation.
Secondly, a modified U-Net for accurate segmentation from the resultant fused image is proposed. The block diagram of the proposed framework is indicated in Fig 1. The foremost motive of the work is to develop a custom U-Net model from scratch that can effectively segment tumour area from fused MRI and PET images.
The U-Net model processes the fused image to provide accurate and reliable results in segmenting any potential tumour. The flowchart of the proposed model is present in Fig 2. The dataset, pre-processing steps, image fusion using PCA, and the proposed U-Net model are present in Section 3. A-D respectively
First, the two inputs, MRI, and PET images, are taken, and these images are processed through an image registration process that involves aligning the co-ordinates of both the images. Then, these images are used for fusion. The PCA algorithm is used for fusing the MRI & PET images. Then, the fused image is given as an input to the modified U-Net model. This model is trained using the datasets mentioned in Section III.A. The proposed U-Net model uses a 5-level encoder and decoder layer configuration to obtain the segmented output image. Parameters, such as accuracy, dice-coefficient and IoU are used for evaluation purpose.
3.1 Dataset
In general, pre-made data sets are preferred when working with medical images. These data sets have been made available to the researchers after the images are labelled by experts. One of these is the BraTS (2015, 2020, and 2021) “Multimodal-Brain-Tumour-Segmentation Challenge” datasets. In the proposed work, to test and evaluate the model performance we consider the above publicly available data set, which has 110 patient scanned MRI and PET brain tumour images. We considered 88 images for training and 22 images for testing, input dimensions 256x256, we chose to make use of only images of tumours for training to make sure the network could find tumour features quickly. Next, we included all images in the network learning standardization.
3.2 Pre-Processing
Image pre-processing is an important phase when dealing with numerous datasets, since it improves the image quality and assists in identifying the finest features. To increase the eminence of the images, there exist various methods such as scaling, and intensity normalization are employed. To reduce noise, without losing precise information, noise filters are utilized. Below is a thorough explanation on image resizing, normalization, and augmentation which are some related pre-processing steps.
3.2.1 Image resizing.
Image resizing is a critical and most important pre-processing step in computer vision applications. Resizing is the process of converting all the images to a fixed standard or smaller images that accelerate the model training in machine learning and DL techniques. This process helps in facilitating the processes (comparison and analysis). All the images are resized to standard necessities of 256x256 to make the further process much easier. A 220x220 image size in Fig 3(a) has been resized to the required 256x256 size in Fig 3(b).
3.2.2 Normalization.
Normalization involves converting all the pixel values of an image into a range [0–1 by dividing them by 256 as we consider 8-bit images. (for 8-bit image it is 256, for 12- and 16-bit image it will be 4095 and 65535 respectively as the maximum value). It is sometimes called histogram stretching or contrast stretching. This adjustment helps to enhance the brightness and contrast, suitable for subsequent processes, as illustrated in Fig 4.
3.2.3 Augmentation.
It is an image pre-processing technique aimed at making the mode l more robust. It is defined as the process of generating or producing a new version of image from the given image to increase its diversity. It is mainly used for expansion of data sets artificially through techniques like rotation, brightness adjustment, zooming in/out, and various flipping and shifting in horizontal and vertical directions. It is also employed on an image to produce new variations of the same image, thus increasing the dataset’s diversity, and strengthening the model, as shown in Fig 5. After augmentation the size of the data set was raised to 110 from 92.
3.3 MRI-PET image fusion using PCA
In this section we present the PCA algorithm for dimensionality reduction followed by fusion of MRI & PET images.
Step-1: Let the images be [In] ∊ RW*H for n = 1,2…. N. Form a vector xn = vector (In) ∊ RD, where D = W*H is the dimension of the image. Form a matrix [X]= [x1, x2,..xN] ∊ RD*N.
Step-2: For each row of the matrix X, compute mean and standard deviation. From every element in row, subtract the mean then divide by standard deviation such that a normalized matrix [] is formed.
Step-3: Compute the covariance matrix ∊ RD*D. Perform Eigen Value Decomposition (EVD) of S such that S = U∑UT. Columns of U – Eigenvectors and ∑ is diagonal matrix with Eigen-values as the diagonal elements.
Step-4 Select the first K Eigen vectors of U corresponding to the K maximum Eigen values. Form a matrix.
B= [u1, u2, …. uK] ∊ RD*K. Compute, where
and
. The elements of each row of
are multiplied by the corresponding standard deviation and mean is added to restore the dynamic range of rows of
.
Step-5: Each column of is converted back to image format.
The above steps are repeated for both MRI & PET images. Finally, pixel wise fusion of reduced dimension images can be taken up. These fusion algorithmic steps are shown in Fig 6. The PSNR and SSIM values for the fused images using the PCA algorithm are found to be comparatively better than those of the other fusion algorithms. The qualitative and quantitative results of different fusion algorithms are shown in fig and Table-3 respectively.
3.4 Permutate U-Net framework for brain tumour segmentation
The Permutate U-Net architecture originated from traditionally available CNN network and is specially developed for medical images. A general CNN network takes an image as input and classifies it with a single label. However, in cases of medical images, it is not enough to classify whether there is an abnormality or not; it is also necessary to localize the region of the abnormality. This is where U-Net has the advantage over CNN. U-Net architecture performs classification on every pixel of the image, allowing it to localize and differentiate abnormal borders.
This paper proposes the use of a Permutate U-Net model to segment brain tumors automatically as shown in Fig 7. The Permutate U-Net model has been custom-designed with layers, which were specifically chosen through several experiments to achieve the desired results. The proposed U-Net model features 5 down sampling layers, 5 up sampling layers, and a middle layer, increasing the number of skip connections and filters. The down sampling layers consist of a combination of convolution, max pooling, and batch normalization, with activation performed after the convolution layer and batch normalization. Similarly, the up-sampling layers contain a combination of transpose convolution, concatenation, and activation and normalization after every concatenation. The use of activation functions and batch normalization play a significant role in improving the performance and steadiness of the model. This helps the model to learn relationships in a better way.
In this way, the model learns the parameters and over fitting is reduced. The model makes use of the ReLU and sigmoid activation functions due to the computational efficiency that they offer. The ReLU activation function speeds up the training by providing faster convergence and helps with non-linear mapping. Sigmoid is another activation function that is used.
The dimensions of the image are 256X256X3, and all the layers and hyper parameters are designed according to this input size to achieve the best possible results. The suggested U-Net model offers an effective and efficient method for accurately identifying and isolating tumors, representing a significant development in automatic brain tumor detection. The input image of size 256X256X3 passes through a series of five down sampling blocks, where the image undergoes a combination of two convolution layers followed by activation after every convolution and batch-normalization and a max-pooling layer.
A 3x3 filter, used for convolution, and 2x2 max pooling is performed with a stride of 1. The number of filters is doubled, starting from 64, 128, 256, and so on up to 1024. The filters provide depth to the model.
After passing through the five down sampling blocks, there is a middle layer with 2048 filters and a similar combination of two convolution layers, activation, and batch-normalization layers. The image is passed through this block (8x8x2048). Then, the image is sent through a combination of up sampling blocks. Five up sampling blocks are used in the model. These blocks have a combination of a transposed convolution layer, a concatenation layer, and two convolution layers. The size of the filters used in each block of decoder is reduced from 1024 to 64 in step 2. In the last up sampling block, convolution is performed three times, the activation function used is sigmoid, and the filter count is 1. The input from the middle leg to the decoder leg is 8x8x2048, and the output is 256x256x1. The review of the proposed model is accessible in Table 1.
4. Performance evaluation
The proposed model is evaluated with the following metrics, both fusion and segmentation. Fusion performance is analyzed using PSNR & SSIM [20]. These results are presented in Table 2 for different fusion techniques, PCA method of fusion is showing better results both for PSNR and SSIM. To assess the segmentation performance, we evaluate the dice-coefficient, IoU and the accuracy. Table 3. shows comparative results of segmentation parameters.
4.1 Peak-Signal-To-Noise-Ratio (PSNR)
For the qualitative analysis and evaluation of the fused image obtained from various algorithms, PSNR and SSIM are taken into consideration. PSNR is an image quality metric, defined as,
4.2 Structural-Similarity-Index-MeasureMENT (SSIM)
The other parameter used is SSIM. It stands for structural similarity index measures. It is a measure of how close the output image is to the input image. SSIM is a measure to quantify the similarity between two images. The SSIM index is a value [-1, 1]. Mathematically,
Whereas
- a, b – two images being compared.
- 𝜇a and 𝜇b – mean,
– variances, ab is the covariance of a and b respectively.
- Z1 and Z2- constants.
It is the similarity of the processed image with the input image.
4.3 Dice Coefficient (DC)
It is the measure of the check the overlapped regions between the result obtained from segmented image and ground-truth image, with a value range from 0 to 1, where a value of 0 – no overlap, and 1 -perfect overlap
5. Results analysis
The model was executed on Kaggle platform using Python 3, with a GPU utilized as an accelerator for the program. The system has 8GB RAM and a 256GB SSD and Corei5 processor 10th generations. Here we present the simulated outputs of both fused and segmented results.
5.1 Fusion results
Fig 8 shows the qualitative comparison of resultant fused MRI-PET Images, using various fusion techniques. Corresponding quantitative results are exhibited in Table 2.
The graphical representations of the above values for PSNR and SSIM of MRI & PET images are respectively presented in Fig 9. From the comparative analysis graph of various fusion algorithms, it is observed that the PCA based fusion has both better PSNR and SSIM values as compared to other fusion methods. PCA is showing PSNR values of 35.312 & 33.220 and SSIM values of 0.851 & 0.827 with respect to input MRI and PET images respectively. Table 2 shows the Qualitative Comparative Analysis of proposed fusion model with other fusion techniques that were presented. From the above table one can observe the various fused output that are obtained using different fusion algorithms. Among the resultant fused images from various fusion methods, the PCA based fused image provides high image quality. The improved image quality helps with good interpretation and helps in analyse the image at multiple resolutions. The resultant PCA based fused image has enhanced coarse and fine structures helps in effective detection and segmentation of tumours regions with improved performance metrics.
(a) PSNR w.r.t input images, (b) SSIM w.r.t input images.
5.2 Segmentation results
The proposed U-Net model’s segmentation results are reported both quantitatively and qualitatively. Visual analysis of the output segmented image is obtained after training and testing. Fig 10 shows the segmented image output when inputs are taken from the dataset randomly. Fig 11 shows the output results when the custom input is given to the model.
5.3 Analysis of parameters evaluation and segmentation results
The model was trained using 100 epochs. Since there had been no discernible improvement in Dice Coefficient, IoU, and Accuracy after 100 epochs, training was stopped at 100 epochs. From the results of training & validation, it can be observed that the model is not overfitting. We have implemented 5-fold cross-validation on the training data (from the BraTS dataset) to reduce the likelihood of overfitting and to ensure more reliable performance estimation. The averaged performance metrics like Dice-coefficient, IoU and accuracy across folds are reported in (Tables 5, 6, 7). Fig 12.a, b and c show graphical analysis of training and validation outcomes for the parameter:
(a) Dice Coefficient (b) IoU (c) Accuracy.
5.4 Comparative analysis with existing works
The performance analysis and obtained results are presented in Table 3. The brain tumour segmentation model employing fuzzy C-mean cluster [21] over MRI BRATS-2012 Data sets and obtained values of 0.750 and 0.66 for DC and IoU, respectively. An effective multi-scale 3D-CNN model [22] yields dice-coefficients of 0.85- and 0.88-pixel accuracy. A model for segmenting tumour regions in fused MRI-PET images based on morphological functions [23], with model segmentation accuracy of 0.992. Deep learning model incorporating FCNNs and CRFs for brain tumour segmentation [24] the model results moderate dc-0.84 and acc-0.82. CNN model [25] that employs brain image segmentation for MRI images and the model yields DC −0.774, IoU-0.80, and ACC-0.977.
A novel U-net Based Adversarial Networks model [26], the model yielding a mean accuracy of 0.94 and a high mean intersection over union of 0.89. For brain MRI image segmentation with minimal computing resources, SD-UNET: Stripping down U-net [27]. The model yields extremely high segmentation ACC-0.986 and moderate DC-0.827. An efficient wavelet-based image fusion using PET and MRI images [28], utilizing the SMO-DNN (weighted K-means algorithm). They obtained DC, IoU, and ACC values of 0.853, 0.75, and 0.930, respectively. A technique for segmenting brain MR images utilizing seeded regions and C-mean clustering has been reported by Al-Dabagh et al. [29] with an accuracy of 0.90 and an IoU of 0.82.
A precise and automated method of image segmentation with BTS-GAN utilizing conditional GAN (cGAN) in MRI scans. An average of 0.77 and 0.85 were obtained by proposed model in terms of IoU and Dice scores [30]. The segmentation of brain tumours employing the traditional U-Net architecture [31] for the BRATS −2018 MRI images yielded good accuracy, moderate DC, and low IoU with 0.992, 0.739, and 0.59 respectively. The proposed deep learning model attained DC-0.91, IoU-0.88 and ACC of 0.996, which are significantly greater than the previous result, which are presented in Table 3
5.6 Performance comparison of various U-Net with permutate U-Net model
The model’s performance was assessed with a variety of publicly accessible brain tumor segmentation challenge BraTS datasets from 2015, 2020, and 2021. Tables 4–6 present the findings of a comparison between the proposed model’s performance and those of existing deep learning architecture, for a variety of data sets. For various kinds of BraTS datasets, it is noticed that the proposed U-Net model is superior over existing models in terms of DC, IoU, and ACC performance metrics as shown in graphical representation of Figs 13–15– for various datasets.
6. Conclusion
Segmentation of tumor regions always involves crucial processes, especially in brain tumor segmentation is more difficult by boundary region pixels in MRI and PET brain imaging having low sensitivity. To extract more important features with improved imaging quality is a prominent research area in accurate characterization in medical image analysis, where image fusion plays a indispensable role. To fuse MRI and PET images, the PCA algorithm is used, as it has been shown to yield better results than other fusion algorithms. It is also observed that the resultant fused have improved resolution, good contrast with detailed-edge information as given in Table 2. The resultant fused image is considered to evaluate the Permutate U-Net model in effective segmentation.
The model is designed from scratch to suit the fused input images, and its hyper parameters are selected after a series of experiments to produce the best results. The U-Net model comprises a network of five down and up sampling layers with a middle layer. As the number of layers increases, the number of filters allows the model to learn more complex relations. The increased number of layers also resulted in more skip connections, enabling the model to learn even complex relations and avoid overfitting. Finally, the model effectively segments the tumor region with than existing models in terms of dice-coefficient, accuracy, and IoU of 0.91, 0.88 and 0.996 respectively. Father to prove superiority of the model, the resultant segmentation metrics of proposed model are compared by means of other existing DL architecture as shown in Tables 4–6 for various data sets.
Despite the modified U-Net architecture achieves promising results in brain tumour segmentation from fused MRI-PET images, certain limitations remain. Here the performance of model focusses mainly on quality of image fusion and the alignment between modalities which can introduce noise or artifacts affecting segmentation accuracy. Now a days many research focus on deep learning based fusion strategies for better image quality. So, there is scope for applying better deep learning for fusion strategies helps in improving the overall segmentation processes, which may enhance model robustness and clinical applicability.
References
- 1. Khan MSI, Rahman A, Debnath T, Karim MR, Nasir MK, Band SS, et al. Accurate brain tumor detection using deep convolutional neural network. Comput Struct Biotechnol J. 2022;20:4733–45. pmid:36147663
- 2. Gupta S, et al. Multi-class brain tumor classification with VGG models: An MRI-based analysis. In: 2023 International Conference on Next Generation Electronics (NEleX). 2023.
- 3.
Rajesh Para A, Punitha A, ChandraSekhar Reddy P. Gliomas analysis from nervous system using U-Net segmentation with VGG-19 net CNN architecture.
- 4. Nalwaya P, Allapakam V, Karuna Y, Saladi S. CT-PET Based Image Fusion Algorithm for Medical Applications. In: 2023 IEEE 12th International Conference on Communication Systems and Network Technologies (CSNT). 2023. 391–7.
- 5. Lohmann P, Kocher M, Ruge MI, Visser-Vandewalle V, Shah NJ, Fink GR, et al. PET/MRI Radiomics in Patients With Brain Metastases. Front Neurol. 2020;11:1. pmid:32116995
- 6. Ladefoged CN, Marner L, Hindsholm A, Law I, Højgaard L, Andersen FL. Deep learning based attenuation correction of PET/MRI in pediatric brain tumor patients: evaluation in a clinical setting. Frontiers in Neuroscience. 2019;12:1005.
- 7. Bhutto JA, Tian L, Du Q, Sun Z, Yu L, Tahir MF. CT and MRI Medical Image Fusion Using Noise-Removal and Contrast Enhancement Scheme with Convolutional Neural Network. Entropy (Basel). 2022;24(3):393. pmid:35327904
- 8. Huang B, Yang F, Yin M, Mo X, Zhong C. A Review of Multimodal Medical Image Fusion Techniques. Comput Math Methods Med. 2020;2020:8279342. pmid:32377226
- 9. Du G, Cao X, Liang J, Chen X, Zhan Y. Medical image segmentation based on U-net: A review. Journal of Imaging Science & Technology. 2020;64(2).
- 10. Norouzi A, Rahim MSM, Altameem A, Saba T, Rad AE, Rehman A, et al. Medical Image Segmentation Methods, Algorithms, and Applications. IETE Technical Review. 2014;31(3):199–213.
- 11. Hesamian MH, Jia W, He X, Kennedy P. Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. J Digit Imaging. 2019;32(4):582–96. pmid:31144149
- 12. Ait Skourt B, El Hassani A, Majda A. Lung CT Image Segmentation Using Deep Neural Networks. Procedia Computer Science. 2018;127:109–13.
- 13. Dhanachandra N, Manglem K, Chanu YJ. Image Segmentation Using K -means Clustering Algorithm and Subtractive Clustering Algorithm. Procedia Computer Science. 2015;54:764–71.
- 14. Huang H, Meng F, Zhou S, Jiang F, Manogaran G. Brain image segmentation based on FCM clustering algorithm and rough set. IEEE Access. 2019;7:12386–96.
- 15. Chowdhary CL, Acharjya DP. Segmentation and Feature Extraction in Medical Imaging: A Systematic Review. Procedia Computer Science. 2020;167:26–36.
- 16. Özyurt F, Sert E, Avci E, Dogantekin E. Brain tumor detection based on convolutional neural network with neutrosophic expert maximum fuzzy sure entropy. Measurement. 2019;147:106830.
- 17. Jena B, Jain S, Nayak GK, Saxena S. Analysis of depth variation of U-NET architecture for brain tumor segmentation. Multimed Tools Appl. 2022;82(7):10723–43.
- 18. Lin S-Y, Lin C-L. Brain tumor segmentation using U-Net in conjunction with EfficientNet. PeerJ Comput Sci. 2024;10:e1754. pmid:38196955
- 19. Zhang Y, Ngo HC, Zhang Y, Yusof NFA, Wang X. Imaging Segmentation of Brain Tumors Based on the Modified U-net Method. ITC. 2024;53(4):1074–87.
- 20. Allapakam V, Karuna Y. An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19. PloS One. 2024;19(10):e0309651.
- 21.
Baid U, Talbar S. Comparative study of k-means, gaussian mixture model, fuzzy c-means algorithms for brain tumor segmentation. InInternational Conference on Communication and Signal Processing 2016 (ICCASP 2016). Atlantis Press. 2016. 583–8.
- 22. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal. 2017;36:61–78. pmid:27865153
- 23. Selvapandian A, Manivannan K. Fusion based glioma brain tumor detection and segmentation using ANFIS classification. Computer methods and programs in biomedicine. 2018;166:33–8.
- 24. Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y. A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Med Image Anal. 2018;43:98–111. pmid:29040911
- 25.
Alkassar S, Abdullah MA, Jebur BA. Automatic brain tumour segmentation using fully convolution network and transfer learning. In2019 2nd international conference on electrical, communication, computer, power and control engineering (ICECCPCE). IEEE. 2019. p. 188–92.
- 26. Teki S, Varma M, Yadav A. Brain Tumour Segmentation Using U-net Based Adversarial Networks. TS. 2019;36(4):353–9.
- 27. Gadosey PK, Li Y, Adjei Agyekum E, Zhang T, Liu Z, Yamak PT, et al. SD-UNet: Stripping Down U-Net for Segmentation of Biomedical Images on Platforms with Low Computational Budgets. Diagnostics (Basel). 2020;10(2):110. pmid:32085469
- 28. Preethi S, Aishwarya P. An efficient wavelet-based image fusion for brain tumor detection and segmentation over PET and MRI image. Multimed Tools Appl. 2021;80(10):14789–806.
- 29. Nayef AL-Dabagh MZ. Automated tumor segmentation in MR brain image using fuzzy c-means clustering and seeded region methodology. IJ-AI. 2021;10(2):284.
- 30. Haq IU, Ali H, Wang HY, Cui L, Feng J. BTS-GAN: Computer-aided segmentation system for breast tumor using MRI and conditional adversarial networks. Engineering Science and Technology, an International Journal. 2022;36:101154.
- 31. Güvenç E, Ersoy M, Çetin G. Brain tumor segmentation on FLAIR MR images with U-Net. Mugla Journal of Science and Technology. 2023;9(1):34–41.
- 32. Raza R, Ijaz Bajwa U, Mehmood Y, Waqas Anwar M, Hassan Jamal M. dResU-Net: 3D deep residual U-Net based brain tumor segmentation from multimodal MRI. Biomedical Signal Processing and Control. 2023;79:103861.
- 33. Ahmad P, Qamar S, Shen L, Saeed A. Context aware 3D UNet for brain tumor segmentation. In: International MICCAI Brainlesion Workshop. 2020. 207–18.