https://orcid.org/0000-0002-3786-2363
Figures
Abstract
In recent years, various data-driven algorithms have been applied to the classification and staging of brain glioma MRI detection. However, the restricted availability of brain glioma MRI data in purely data-driven deep learning algorithms has presented challenges in extracting high-quality features and capturing their complex patterns. Moreover, the analysis methods designed for 2D data necessitate the selection of ideal tumor image slices, which does not align with practical clinical scenarios. Our research proposes an novel brain glioma staging model, Medical Cognition Embedded (MCE) model for 3D data. This model embeds knowledge characteristics into data-driven approaches to enhance the quality of feature extraction. Approach includes the following key components: (1) Deep feature extraction, drawing upon the imaging technical characteristics of different MRI sequences, has led to the design of two methods at both the algorithmic and strategic levels to mimic the learning process of real image interpretation by medical professionals during film reading; (2) We conduct an extensive Radiomics feature extraction, capturing relevant features such as texture, morphology, and grayscale distribution; (3) By referencing key points in radiological diagnosis, Radiomics feature experimental results, and the imaging characteristics of various MRI sequences, we manually create diagnostic features (Diag-Features). The efficacy of proposed methodology is rigorously evaluated on the publicly available BraTS2018 and BraTS2020 datasets. Comparing it to most well-known purely data-driven models, our method achieved higher accuracy, recall, and precision, reaching 96.14%, 93.4%, 97.06%, and 97.57%, 92.80%, 95.96%, respectively.
Citation: Xue H, Lu H, Wang Y, Li N, Wang G (2024) MCE: Medical Cognition Embedded in 3D MRI feature extraction for advancing glioma staging. PLoS ONE 19(5): e0304419. https://doi.org/10.1371/journal.pone.0304419
Editor: Pengfei Yan, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, CHINA
Received: January 14, 2024; Accepted: May 12, 2024; Published: May 31, 2024
Copyright: © 2024 Xue et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets used in this article are the brain tumor MRI datasets (BraTS2018 and BraTS2020) published by the Perelman School of Medicine at the University of Pennsylvania and can be found at: https://www.kaggle.com/datasets/anassbenfares/brats2018 and https://www.kaggle.com/datasets/awsaf49/brats20-dataset-training-validation.
Funding: This research is supported by the Fundamental Research Funds for the Central Universities, JLU (No. 93K172022K15), Industrial Technology Research and Development Special Project of Jilin Provincial Development and Reform Commission in 2023 (No. 2023C042-6), and Key Project of Science and Technology Research Plan of Jilin Provincial Department of Education in 2023 (No.JJKH20230763KJ). Moreover, we want to thank"Changchun Computing Center" and “Eco-Innovation Center" for providing inclusive computing power and technical support of MindSpore during the completion of this paper. There was no additional external funding received for this study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
According to the World Health Organization (WHO) [1], brain tumors pose a significant threat to global health and quality of life. Brain tumors are tumors that affect the central nervous system, characterized by abnormal proliferation of cells. They are categorized from Grade I to Grade IV based on their growth rate [2]. Specifically, Grade I and Grade II gliomas are classified as Low-Grade Gliomas (LGG), while Grade III and Grade IV gliomas classified as High-Grade Gliomas (HGG) [3]. Grade IV glioblastoma (GBM) is one of the most lethal and easily recurrent malignant solid tumors, accounting for 57% of all neuroglial tumors and 48% of primary central nervous system malignancies. The median survival period is usually no more than 2 years [4]. Brain Magnetic Resonance Imaging (MRI), as a non-invasive medical imaging technique, can clearly visualize the precise location, size, and morphology of tumors [5]. It is widely used for diagnosis and monitoring brain tumors. However, to achieve accurate assessment and staging of brain tumors through MRI images, radiologists must possess extensive experience and exceptional skills [6]. Furthermore, the training of radiologists in this specialized domain necessitates a substantial investment of time and financial resources. With the increasing volume of brain tumor image datasets, it becomes even more susceptible to subjective bias and human errors during the assessment [7, 8].
In recent years, deep learning algorithms such as Convolutional Neural Networks (CNN) have made revolutionary breakthroughs in the analysis of brain glioma MR images, providing new opportunities for the automated detection and classification of brain tumor MRI data [9]. These algorithms can learn complex features from MRI data and perform remarkably well on classification and staging tasks [10–12]. However, there are also some limitations: (1) Due to the scarcity of MRI data and the imbalance of categories, MRI analysis in data-driven deep learning algorithms lacks the guidance of prior knowledge and domain-specific rules. This can lead to excessive redundancy in automatically learned features and patterns, thereby affecting the model’s generalization ability [13]. (2) While deep features can effectively capture high-level abstract information in images, there are certain limitations in expressing specific features relevant to the staging of gliomas, such as texture, morphology, and grayscale distribution. The acquisition of texture and shape features requires meticulous texture analysis, contour extraction, and geometric measurements. (3) Researchers often observe that models mistakenly identify irrelevant features in images (e.g., artifacts, vignetting, background, etc.) as decision cues. In other words, models infer results based on non-relevant features and exhibit a certain degree of regularity [14]. This is a manifestation of the model’s excessive reliance on the specificity of training data.
To address the issues mentioned above, we propose a new method in this paper that incorporates external knowledge rules from the perspective of feature acquisition. We have designed the MCE model to enhance the quality of feature extraction. Additionally, given the limitation of 2D-based analysis methods necessitate the use of optimal tumor image slices, which may not align with clinical practice, our approach is thoughtfully tailored for 3D brain glioma MRI images. Specifically:
- Designing different modules allows the MCE model to capture and learn different patterns and modes present in the data, including the attention patterns of doctors, radiomic information, and diagnostic knowledge. The combination of multiple modules effectively reduces the random fluctuations caused by the high complexity of a single model, thus mitigating overfitting issues and enhancing the model’s generalization capability.
- Imitating the attention pattern of physicians during image interpretation—focusing on the characteristic regions of different sequence images. The paper designs, at both the model and strategy levels, to mimic the real process of physicians interpreting medical images, constructing two feature extraction models, Multi- Sequence Attention Neural Network (MSANN) and Clinical Learning Neural Network (CLNN).
- Introduced the Radiomics module, which explores information such as texture, morphology, statistical distribution, etc., in MRI images. This involves extracting and calculating specific features within the different sequence images to assist in enhancing the predictive performance of the model.
- By amalgamating insights derived from experimental results of Radiomics features, imaging characteristics of MRI sequences, and knowledge of neuroradiological diagnosis, we manually create four Diag-Features, with a focal emphasis on the Enhancing Tumor (ET), Necrotic and Non-Enhancing Tumor(NCR/NET) and ED(Edema). regions of brain gliomas. These features further enhance the model’s performance.
The first chapter of this paper introduces the background and innovative aspects of our work. In the second chapter, we present relevant works on glioma staging and the concept of knowledge embedding, while also discussing the role played by knowledge rules in feature extraction. The third chapter introduces the materials and methods used in this study. Our experimental results of the proposed modules are presented in the forth chapter. In the fifth chapter, the roles of each module in the paper are discussed. Finally, the paper concludes with a summary of the entire work in the sixth chapter.
Related works
The features used for brain glioma staging tasks are primarily divided into two categories: (1) Deep features refer to the design of Convolutional Neural Network (CNN)-based feature extraction networks to extract features from images, thereby improving classification performance through enhanced classifiers and training strategies. Deep features can be extracted end-to-end from raw images without the need for manually designing feature extractors. Highly abstract and semantically rich feature representations can effectively capture both local and global information in the images. Rehman et al. [15] designed a three- dimensional CNN architecture for extracting deep features from brain tumors, followed by feature selection and classification using a feedforward neural network; Narmatha et al. [16] combined fuzzy optimization and brainstorm optimization techniques for glioma classification tasks, iteratively optimizing the network structure through fuzzy execution; Sharif et al. [17] employed a hybrid segmentation histogram equalization and ant colony optimization method to enhance image contrast. This enriched data was then fed into a meticulously designed nine-layer CNN model for deep feature extraction. Subsequently, feature concatenation and optimization via differential evolution and firefly optimization were performed before training with multi-class support vector machines (MC-SVM); Deepa et al. [18] used CNN for feature extraction and used the extracted features as inputs for deep residual networks (DRN) for classification. The networks were trained using the combination of the Jaya algorithm, honey badger algorithm (HBA), and an integrated algorithm with a time concept (CJHBA) to achieve better classification performance; Bidkar et al. [19] proposed a deep belief network (DBN) based on Salp Water optimization for brain tumor classification, mining features using CNN, and then utilizing SWO-DBN technology to effectively classify brain tumors based on the extracted features. Although improvements in classifiers or training strategy have enhanced the overall model performance, the inherent challenge of small sample data has not been fundamentally resolved in terms of the quality of feature acquisition. (2) Manual features typically require experts to design feature calculation methods based on specific medical knowledge and task requirements, including statistical, morphological, and textural aspects (such as LBP, GLCM, etc.), for manual extraction. Each feature has its physical significance, thus manual features often exhibit good interpretability. These features are often combined with machine learning algorithms such as Support Vector Machine (SVM), Random Forest, etc. Jemimma et al. [20] used fractional probability fuzzy C-means (FRF-PFCM) for automated segmentation of brain glioma MRI data, followed by feature extraction using descriptors, empirical mode decomposition (EMD), local direction pattern (LDP), wavelet transform, etc. The resultant feature set was then input into a deep belief network based on whale cat swarm optimization (WCSO-DBN) for classification. Suárez-García et al. [21] extracted texture features from MRI sequences of brain gliomas and proposed an undersampling method to divide the data into different training subsets for extracting complementary information. They combined multiple linear regression (MLR) to create different classification models. Those methods are a form of feature engineering and, while the extracted features are more precise, they still need to improve the breadth (quantity) of features for small-sample glioma image data. To address the challenges of small-sample glioma imaging data, previous literature has mainly relied on techniques from computer vision, such as data augmentation, simplifying model complexity, expanding the sample space through generative techniques, and improving classifier algorithms. However, from the perspective of external information acquisition, these methods primarily focus on the optimizing target task within the given dataset and do not introduce new external information into the model.
Therefore, the key to addressing the problem of small-sample is how to extract higher-quality features from a limited dataset [22]. To achieve this beyond the given dataset, embedding knowledge characteristics to impose reasonable rules and constraints can help the model to better understand and utilize small-sample data. These methods can be broadly categorized into two directions [23]:
The first direction is embedding knowledge information applicable to wide range of basic image types and disease diagnoses—referred to as high-level knowledge. This encompasses strategies such as: (1) transferring feature extraction models from large-scale natural image datasets or other medical images: Wang et al. [24] utilized homologous breast cancer data and transfer learning methods to train a CNN structure. They obtained Deep Learning Features (DLF) by fine-tuning the fully connected layers of the CNN structure. Using Support Vector Machines (SVM), they constructed Deep Learning Radiomics Features (DLRS) and integrated clinical factors with DLRS, proposing a LN transfer status classifier. Saini et al. [25] introduced a novel deep neural network architecture based on VGG16. They transferred domain knowledge from a larger ImageNet object dataset to a smaller and imbalanced breast cancer dataset. (2) Emulating the training routines followed by radiologists by gradually increasing task difficulty through methods like Curriculum Learning (CL): Gracias et al. [26] applied the Self-paced Curriculum Learning strategy to 3D-CNN as a method that combines medical knowledge to enhance the performance of early Alzheimer’s disease diagnosis networks. Yang et al. [27] proposed severity-guided multiple instance curriculum learning, a curriculum that progresses from easy to difficult, utilizing image training models for the classification of pathological images. (3) Using attention mechanisms and other methods to mimic the specific regions or sequences that doctors focus on. Xie et al. [28] proposed a strategy called Domain Guided-CNN, aiming to incorporate edge information in breast ultrasound (BUS) images for cancer diagnosis tasks to mimic doctors’ diagnostic behavior. Zhou et al. [29] proposed a method called Non-Local Fourier Attention, which combines self-attention mechanism with Fourier transform to capture distant spatial dependencies in the frequency domain of MRI. Xie et al. [30] proposed a Multi-scale Efficient Network (MEN), which integrates various attention mechanisms to achieve comprehensive extraction of detailed features and semantic information in COVID-19 images through progressive learning.
The other direction is embedding knowledge information applicable to particular images or specific disease diagnoses—referred to as low-level knowledge. This involves strategies such as: (1) Incorporating special features that medical professionals focus on in certain disease diagnoses (including manually created features). These features are then integrated into the network as specific patches, a process known as feature-level fusion. Alternatively, distinct classifiers are individually trained and later weighted for fusion at the decision level, known as decision-level fusion [31]. Bogacsovics et al. [32] created lesion-related hand-crafted features from fundus images and combined them with deep features for diabetes classification tasks. Ranjbarzadeh et al. [33] proposed various encoding methods beneficial for detecting target textures to encode breast cancer X-ray images to eliminate the pectoral muscle portion. They then used local patches as input and concatenated all extracted features into a vertical vector to apply to fully connected layers for pixel-wise classification, achieving promising results. (2) Leveraging specific image feature calculation methods that focusing on texture, morphology, grayscale distribution, and other aspects to more effectively extract features to enhance the model’s prediction capabilities. Xiao et al. [34] extracted radiomic features from brain tumor MRI scans from 9 different regions of interest (ROI). They utilized LASSO and Principal Component Analysis (PCA) for feature selection and finally trained and predicted using Logistic Regression and Random Forest models to detect meningioma patients, achieving satisfactory results. Al-Mekhlafi et al. [35] combined deep features extracted from DenseNet-121 and ResNet-50 with handcrafted features such as GLCM, FCH, and DWT for the diagnosis of malignant lymphoma. Bagherian Kasgari et al. [36] proposed a robust and efficient brain tumor segmentation pipeline based on machine learning. Initially, they utilized the Zernike moment method to extract multiple feature maps from MRI, thereby generating different sets of edge feature maps. Simultaneously, they employed Local Directional Number Pattern (LDNP) for feature extraction. Finally, Enhanced Ant Lion Optimizer (EALO) was used to find the optimal feature map set for creating the best encoded images. Additionally, various imaging techniques exhibit differences in sensitivity to biological structures and lesions, spatial resolution, and the ability to capture tissue properties. This variability leads to unique focus areas, necessitating the consideration of differences among imaging technologies to ensure accurate diagnosis and analysis. In our survey, we found that there is limited research considering this factor, making it challenging to form effective references.
Materials and methods
In this study, we propose an approach to embed knowledge characteristics into the feature extraction stage, incorporating the learning process of radiologists during image interpretation, adiomics, the specificity of different MRI sequences [37, 38] (Table 1), and key points in neuroradiological diagnosis of brain gliomas into the model design. This is employed for the automatic classification of (GBM and non-GBM using patient MRI images. Considering that optimal tumor image slices may not always be guaranteed, our overall model strategy is based on 3D data design to better adapt to practical applications in tumor image analysis. The overall design of the model is as Fig 1:
- Step 1: Use 3D U-Net to segment the images and obtain masks for three lesion regions (NCR/NET, ET, ED).
- Step 2: After image preprocessing, input them into MSANN for training and calculate Score 1.
- Step 3: Input the MRI and its corresponding segmentation mask into CLNN for training and calculate Score 2.
- Step 4: Input the MRI and its corresponding segmentation mask into the Radiomics model for training and calculate Score 3.
- Step 5: Input the segmentation mask into the Diag-Feature for training and calculate Score 4.
- Step 6: Obtain the final result by averaging the scores obtained above through voting.
Data description and preprocessing
The experiments in this paper utilized two publicly available datasets from MICCAI, namely BraTS2018 (HGG = 210, LGG = 75) and BraTS2020 (HGG = 293, LGG = 76), the dataset can be obtained from http://www.braintumorsegmentation.org. It is worth noting that, unlike WHO grading, the BraTS dataset defines GBM as HGG, while I, II, III are LGG. Therefore, the essential task of this study is to differentiate between GBM and non-GBM. The dataset comprises three-dimensional MR images of brain gliomas (as shown in Figs 2 and 3), with each sample containing four MRI sequences: Flair, T1, T2, and T1ce. All images are co-registered to the same anatomical template after skull stripping. Additionally, the segmentation labels of the images are manually delineated by multiple evaluators, using MRI with the same resolution interpolation. The segmented labels include regions for ET, ED, and NCR/NET [39–41].
The first column composed of NCR/NET(red); the second column composed of NCR/NET(red) and ET(yellow); the third column composed of NCR/NET(red), ET(yellow) and ED(green).
The first column represents the flair; the second column represents the T1; the third column represents the T1CE; the fourth column represents the T2, the first row represents the transverse plane; the second row represents the coronal plane; the third row represents the sagittal plane.
Due to the BraTS dataset being collected from imaging across multiple medical centers, the voxel values have different scales, with a maximum range of 0-32767 and a minimum range of 0-111. To better adapt to subsequent processing, the images were subjected to Z-score normalization, and the original size of the 240 × 240 × 155 3D images was cropped to a resolution of 160 × 160 × 160.
Segmentation of MRI
To obtain spatial information of lesion areas and generate masks for subsequent feature extraction modules, a segmentation network based on 3D U-Net is used to segment the samples. 3D U-net is an extension of the traditional U-net [42]. Compared to other 2D-designed segmentation networks, it achieves a more balanced leading position in terms of segmentation accuracy, parameter count, and training time for the segmentation of three regions: Enhancing Tumor (ET), Tumor Core (TC, NCR/NET+ET), and Whole Tumor (WT, NCR/NET+ET+ED) [43]. 3D U-Net consists two fundamental components: an encoder and a decoder. The encoder gradually extracts feature representations from low- level to high-level, while the decoder reconstructs and fuses feature maps through upsampling and skip connections. Preprocessed MRI images are used as input, and traverse through a series of convolutional and pooling layers in both the encoder and decoder, progressively extract high-level feature representations. In the decoder part, upsampling operations gradually reconstruct feature maps, and skip connections are introduced to merge feature maps from the encoder and decoder stages, combining feature information from different levels. Finally, a 1 × 1 convolutional layer and a sigmoid activation function are applied to map the feature maps to binary segmentation masks. The generated segmentation masks not only provide spatial information for creating diagnostic features of brain gliomas but can also be used for subsequent CLNN extraction modules and radiomic feature extraction modules.
Simulating the image interpretation process of radiologists
Radiologists, when reviewing a patient’s MRI, typically concentrate their attention on characteristic regions of different sequences. Different MRI sequences provide diverse information about tissue structure, blood flow, metabolism, and function. These regions carry richer information during the disease diagnosis process compared to others (Table 1). The paper designs two methods, respectively, at the model and strategy levels to mimic the real process of doctors interpreting medical images. The former method utilizes a CNN network structure based on attention mechanisms, while the latter achieves it through the design of curriculum learning strategies.
Multi- Sequence Attention Neural Network.
Doctors typically do not analyze information from all images at once when reviewing medical images. Instead, they enhance their focus on specific regions in specific sequence MRI scans. Attention mechanisms are considered a means for humans to quickly select the most important information from a large amount of data using limited processing resources [44]. This aligns well with the goal we aim to achieve. Fig 4 illustrates the model using the Multi-Sequence Attention (MSA) mechanism to mimic the attention pattern of radiologists during image interpretation for deep feature extraction. As mentioned earlier, MRI images from different sequences reflect distinct tissue information and the differences between sequences need to be carefully considered. In our model input, we take MRI data from FLAIR, T1CE, and T2 sequences as three channels and feed them into the model. We utilize the Channel Attention Module (CAM) to learn the importance of each channel (sequence), enabling the model to focus more on channels that contribute most significantly to tumor classification. To ensure that the model pay attention to lesion areas within channels that provide more information for tumor classification, while also mitigating the impact of factors such as artifacts and vignetting, we introduce the Spatial Attention Module (SAM). Initially, convolutional and pooling layers are applied to each channel to extract low-level feature representations. We use adaptive max-pooling and average-pooling to generate a global description of the features. Subsequently, these descriptions are mapped to an attention weight vector through a fully connected layer. This vector reflects the importance of each spatial location and is used to weight the feature maps, giving higher attention to lesion areas.
The output and the convolution result of the initial input are concatenated as the input of the next layer. The result is then fused with the input convolution output and passed to the next layer. This process is designed to enable the model to focus on global information, which may include aspects such as tumor invasion and compression of normal tissues. The mapping between layers is accomplished using the module outlined in the left dashed box in Fig 4. The computation formula (1) is as follows:
(1)
where fx represents the input features, Conv denotes convolutional computation, CAM stands for channel attention weighting, and SAM represents spatial attention weighting, || represents the concatenation operation, ⊙ represents the dot product of two vectors. Finally, the features extracted from the linear layer are input into the classifier for training, obtaining Score 1.
Clinical Learning Neural Network.
Curriculum Learning is a strategy used to train models with a specific curriculum. The core idea is to design an effective curriculum that mimics the human learning process, typically progressing from easy to difficult. According to the complexity of tasks during radiologists’ image interpretation, the curriculum is set—designing a CL strategy consisting of two training rounds(Fig 5): in the first round of training (Epoch ≤ 50),the model is trained with the three MRI sequences (FLAIR/T1CE/T2) as learning labels. The model is trained to differentiate between the types of input image sequences, enabling it to focus on features relevant to the image sequences. In the second round(Epoch >50), the feature extraction model trained in the first round is transferred to the brain glioma classification task (GBM/non-GBM). Additionally, in this round, the weights ωMask, corresponding to the masks of relevant lesion areas, are multiplied in the network. This gradually directs the model’s attention towards the lesion areas. By setting a pre-task, the distinct imaging characteristics of MRI sequences and information from characteristic regions can participate in the glioma staging, making the learned features more reasonable and effectively alleviating overfitting issues in the model.
Acquiring ωMask: Input the segmentation results of U-Net into a 3D CNN with initialized weights (weights = 1, bias = 0), record the flattened one-dimensional vector as the mask. Set weights for the mask: on FLAIR, the weights for ED and ET regions are set to 1.5, while other regions are set to 1; on T1CE, the weights for ET and NCR/NET regions are 1.5, while other regions are 1; on T2, the weights for NCR/NET, ET, and ED regions are 1.5, while other regions are 1. Finally, input the weighted features into the classifier for training and computation, obtaining Score 2.
Radiomics
In this study, Radiomics methods are employed to extract features from brain glioma images. In medical imaging, there may be feature information that reflects lesions but goes beyond visual interpretation. Fully utilizing this information may enable the macroscopic decoding of the structural phenotypes of many lesions. Radiomics is a technique for quantitatively extracting features from medical images that can extract information such as texture, morphology, and grayscale distribution within images [45]. This information is then converted into numerical features. thereby constructing high-dimensional feature vectors for the analysis of brain glioma MRI.
Fig 6 illustrates the Radiomics model. Firstly, the original images are transformed using methods such as LoG, Wavelet, and Squared. Next, the segmentation mask is applied to both the transformed and original images to eliminate the background and retain different lesion areas related to the tumor. Subsequently, a series of feature extraction methods are employed for feature extraction and computation, such as Gray-Level Co-occurrence Matrix (GLCM), gray-level statistics, shape descriptors, etc. This includes specific aspects as follows [46]:
- Texture features, which include energy, entropy, contrast, and correlation, are used to describe the texture characteristics of brain tumor images;
- Morphological features, including volume, surface area, maximum diameter, and perimeter, are employed to describe the morphological characteristics of brain tumors;
- TGray-level statistical features, comprising mean, standard deviation, kurtosis, and skewness, are used to describe the grayscale distribution characteristics of brain tumors.
Finally, random forest classifiers were established for a total of six combinations, targeting two types of masks (TC and WT) and three sequences, to conduct training and computation for obtaining Score 3. It is important to note that the scores for each combination directly contribute to the weighted calculation of the final score 3.
Brain glioma diagnostic rules and diagnostic features
In brain glioma MRI diagnosis, grades I, II, and III of brain gliomas typically exhibit high signal intensity on T2 and FLAIR with relatively limited enhancement areas. In contrast, the GBM exhibit more extensive ET regions in the images, often accompanied by significant ED regions in the peripheral ET regions, and the central region may show distinct necrotic features–NCR/NET regions. Additionally, in the subsequent prediction results of Radiomics features, the features extracted from the T1CE show more prominent performance. This provides important guidance for feature selection. Taking into account the distinct characteristics of the T1CE sequence, we believe that the ET region may be more crucial for glioma classification. Based on the above rules, we consider the ratio of ET to ET+ED volume and the ratio of ET to NCR/NET+ET volume may be crucial for distinguishing GBM. We further transform these observed results into diagnostic features (Diag-Feature) F3 and F4. Additionally, to reduce linear redundancy among features, F3 and F4 are respectively subjected to nonlinear processing to obtain F1 and F2. The specific calculation formulas are as follows:
(2)
(3)
(4)
(5)
where VET represents the volume of the ET region, VNCR/NET represents the volume of the NCR/NET region, VED denotes the volume of the ED region, and N is a constant. We conducted T-test and Pearson Correlation Coefficient analyses on the four formulated features to ensure that the statistical correlation between these features and the labels is significant. Finally, we established a Logistic Regression model to train and predict Diag-Features, obtaining Score 4.
Prevent overfitting
Acquiring and annotating medical imaging data is often costly and time-consuming, resulting in relatively limited available data. Scarcity of data may lead to insufficient coverage of certain classes or features in the training set. This can prevent the model from capturing the true distribution of the data during the learning process, leading to the memorization of features from only a few samples. Additionally, extracted features typically have high dimensionality, requiring the model to learn complex features to distinguish between different pathological conditions. A lack of data can cause the model to overfit to noise or local features in the training data while ignoring the overall data distribution. In such cases, the model is prone to overfitting due to insufficient samples.
The proposed method aims to reduce overfitting from two perspectives: (1) From the overall perspective of the scheme, we employ a combination of four models to overcome the problem of model overfitting. At the design stage of the overall scheme, we intentionally allow different models to capture and learn different patterns and modes in the data. This diversity makes them prone to errors in certain situations but correct in others. When their predictions are combined, errors can be mutually offset, while the correct parts are retained, effectively reducing the random fluctuations caused by the high complexity of a single model. (2) From the perspective of individual modules, in the 3D U-Net, MSANN, and CLNN modules, we primarily mitigate model overfitting by reducing network depth and incorporating normalization layers and Dropout layers. In the Radiomics module, we use random forests for feature selection and modeling, primarily overcoming model overfitting by adjusting the number of decision trees in the random forest. In the Diag-Feature Model module, as the number of features used is small, we did not encounter severe overfitting issues.
Results
Implementation details
Training was conducted concurrently on two RTX 4090 GPUs, an Intel Xeon(R) Silver 4110 CPU @ 2.10GHz, and 32GB of RAM, utilizing the PyTorch framework. The proposed method was tested on the BraTS2018 and BraTS2020 datasets. We split the dataset into 80% for training and 20% for test. We used Ry Radiomics V3.0.1 [47] to design the Radiomics features extraction module, as configured in Table 2 of the PyRadiomics settings. If feasible in the future, our code will be made available as open source on https://github.com/Turing17/MCE_model.
All the experiments underwent 5 rounds of cross-validation. In the method, the hyperparameters for training MSANN were set as follows: the learning rate was 5 × 10−4, Epochs set to 30 when using BraTS2020 and 25 when using BraTS2018, Batch size was 2, loss parameter was CrossEntropyLoss, and optimizer was Adam. For training CLNN, the hyperparameters were set to a learning rate of 5 × 10−4, Epochs of 180, Batch size of 2, CrossEntropyLoss as the loss parameter, and Adam as the optimizer. In the 3D U-Net segmentation module, the hyperparameters are set as follows: the learning rate is 15 × 10−4, Epochs is 100, the optimizer is Adam, and the Batch size is 2. and the loss was defined as per Formula 6:
(6)
where voxels represents the total number of voxels, ∈ is a very small number; yic ∈ [0, 1] represents the one-hot encoding for voxel i belonging to class c; pic ∈ [0, 1] is the probability that voxel i is predicted as class c; N is the total number of voxels, and C is the number of classes (including the background).
Regarding the hyperparameter tuning of 3DU-Net, MSANN, and CLNN, the main objective is to use Dice and Accuracy as the target metrics, and adjust parameters such as learning rate, batch size, Dropout ratio, optimizer, etc., based on relevant parameter tuning experience. In the Radiomics module and the Diag-Feature Model module, grid search is primarily used to find the best hyperparameter combinations. These two types of adjustments are independent but both use Accuracy as the determining criterion.
The curves used in the experimental results figures have all undergone smoothing.
Evaluation index
To evaluate the performance of the proposed method, this paper adopts the commonly used Dice Similarity Coefficient (DSC) to assess the segmentation performance of the 3D U-Net:
(7)
Using Accuracy, Recall, and Precision to evaluate the classification performance of each model for GBM and non-GBM:
(8)
(9)
(10)
where TP represents the model’s correct prediction of positive class results. TN represents the model’s correct prediction of negative class results. FP represents the model’s incorrect prediction of negative class as positive results. FN represents the model’s incorrect prediction of positive class as negative results.
Segmentation
In the segmentation network based on the 3D U-Net, we reduced the network’s depth to mitigate overfitting in situations with limited data and to save training costs. We combined the MRI sequences of FLAIR, T1CE, and T2 into a tensor of size 3 × 160 × 160 × 160 and input it into the model for training. To facilitate efficient training, we further divided the input tensor into 5 smaller tensors of size 3 × 32 × 160 × 160 and used them for training. The segmentation results for each region are presented in Table 3.
Multi- Sequence Attention Neural Network
To validate that the designed attention mechanism module effectively focuses on lesion regions for better feature extraction, the following verifications were conducted: We used Gradient-weighted Class Activation Mapping (Grad-CAM) technique to visualize the specific image regions associated with predictions, providing an intuitive comparison of the regions the model pays attention to. From Fig 7, it can be seen that the network with the attention mechanism can more precisely focus on information-rich lesion regions, thereby effectively increasing the information density of feature extraction. As shown in Table 4, Figs 8 and 9, the comparative results of using MSANN and directly using a convolutional neural network on BraTS2018 and BraTS2020 are presented. The features extracted by MSANN contain more relevant information compared to the features extracted by the convolutional neural network without MSA mechanism, resulting in better classification accuracy.
The first column is original image, the second column is feature extraction using the Neural Network without MSA, the third column is feature extraction using MSANN, and the fourth column is ground truth segmentation labels.
Clinical Learning Neural Network
CLNN is trained in the first round (Epochs ≤ 50) with the sequences of MRI images as labels, and the best model is selected to transfer to the second round (Epochs > 50) of training. The specific training situation is shown in Figs 10 and 11. In the task of classifying MRI sequences, the Accuracy using the BraTS2018 and BraTS2020 datasets is 99.56% and 99.41%, respectively. We compared the results of two methods in glioma classification tasks: one using this curriculum learning strategy and the other directly executing the model for 50 rounds. CLNN achieved Accuracy, Recall, and Precision of 92.28%, 88.41%, 93.22% on BraTS2018, and 92.70%, 87.60%, 89.36% on BraTS2020, respectively, improving by 1.4%, 2.78%, 3.76% and 1.35%, 5.10%, 3.77%, respectively(Table 5, Figs 10 and 11).
The first row of images represents the first 50 epochs (task based on image sequences), and the second row of images represents the last 130 epochs (task based on glioma grades).
The first row of images represents the first 50 epochs (task based on image sequences), and the second row of images represents the last 130 epochs (task based on glioma grades).
Through comparison, it was observed that compared to CLNN, the model without using the CL strategy exhibited severe overfitting after 40 rounds of training. At the same time, the model trained with the CL strategy was able to converge to a better solution more quickly during training. This is because, in contrast to indiscriminately inputting various MRI sequences into the network, the CL strategy can help the model better adapt to different data distributions and input conditions, enhancing the model’s tolerance to noise and variations. Additionally, this strategy encourages the model to gradually learn complex feature representations rather than merely memorizing the training data, aiding the model in better understanding the essence of the problem rather than simply memorizing patterns in the training set.
Radiomics model
In the module for extracting Radiomics features, FLAIR, T1CE, and T2 sequences of MRI are used respectively as masking regions for TC and WT for imaging genomics feature extraction. After undergoing LoG, Wavelet, and Square transformations, each sequence obtained 1093 features, including texture features (GLCM, GLRLM, GLDM, NGTDM, GLSZM), first-order statistical features, and morphological features. Subsequently, the 6 sets of features are inputted into random forests for training and testing. In addition to using Accuracy, Recall, and Precision as evaluation metrics, we also conducted permutation tests on the predicted results of TC and WT regions to ensure that the features extracted from different regions have distinctiveness. The results are shown in Fig 12, Tables 6 and 7.
On both BraTS2018 and BraTS2020 datasets, the classification results based on features extracted from the T1CE sequence are higher than those from other sequences. This is because the imaging effect of the T1CE sequence on the ET and NCR/NET regions is superior, making it richer in information. This is crucial for the staging of gliomas, providing valuable guidance for the subsequent design of diagnostic feature computation methods. Meanwhile, it can be seen from the results of permutation tests that, except for the results on the T2 sequence of BraTS2020, the P-values of the other items are all less than 0.05, indicating significant differences. This indicates that selecting two regions for radiomics feature extraction in the experiment indeed contributes significantly to the prediction of GBM, without the occurrence of feature redundancy.
Diag-Feature model
To validate the relationship between the selected manual Diag-Feature and glioma staging, we conducted independent Pearson Correlation Coefficient and T-test analyses to assess the correlation between these four diagnostic features and the results. Pearson Correlation Coefficient was used to evaluate the correlation between each of the four diagnosis features and the results, as shown in Table 8. The results confirm that all four manual diagnostic features have P-values from the T-test calculations that are below 0.05, indicating significant statistical significance.
From the Pearson Correlation Coefficient and T-test results, it can be observed that features F1,F2 and F3 exhibit a high degree of correlation with glioma staging, while feature F4 demonstrates a moderate level of correlation. Table 9 displays the results of training the manual diagnostic features on a Logistic regression model.
Comparison experiment
In order to highlight the superiority of the proposed algorithm compared to purely data-driven algorithms, we compared the MCE model with some classic baseline models and conducted permutation tests to verify the reliability and stability of the results (Table 10). Meanwhile, we compared the proposed method with recent state-of-the-art methods experimented on BraTS2018 and BraTS2020 datasets (Table 11). From Tables 10 and 11, it can be seen that compared to classical baseline models, the proposed method has outstanding advantages, far surpassing baseline purely data-driven models. Moreover, the permutation test results of the predictive probabilities between the three baseline models and MCE are all less than 0.05, showing significant differences. Additionally, in comparison with recent studies, MCE has also achieved considerable success. The uniqueness of the MCE model lies in its focus on extracting high-quality features to enhance classification performance, rather than just improving classifiers and training strategies. The advantage of this approach is that it can better understand and utilize the information in the input data, achieving outstanding accuracy on the BraTS2018 and BraTS2020 datasets, reaching 96.14% and 97.57%, respectively, surpassing current mainstream classification methods. This is attributed to our improved feature extraction techniques, embedding knowledge features into the feature extraction methods, successfully obtaining high-quality features from small-scale data, effectively enhancing model performance and generalization capability.
Ablation experiment
In the final phase of our study, we determined the final prediction results by averaging the scores of classifiers trained on deep learning features, Radiomics features, and manual diagnostic features through ensemble voting. To validate the effectiveness of each extracted and manually created features, we conducted four sets of comparative experiments:
- Classification using deep features extracted by CLNN, Radiomics features and manually created diagnostic features (CLNN+Radiomcs+ Diag);
- Classification using deep features extracted by MSANN, Radiomics features and manually created diagnostic features (MSANN+Radiomcs +Diag);
- Classification using deep features extracted by MSANN and CLNN, anually created diagnostic features (CLNN+ MSANN+ Diag);
- Classification using deep features extracted by MSANN and CLNN, Radiomics features (CLNN+ MSANN+Radiomcs);
- All (MSANN+CLNN+Radiomcs+Diag).
We also conducted permutation tests on (1)(5), (2)(5), (3)(5), (4)(5) respectively to accurately demonstrate the effectiveness of the four modules. Table 12 and Fig 13 show the results of the four comparative experiments, and the experiments prove that the various features mined in the method have positive guidance for glioma staging.
Specifically, from the comparison between Experiment (1) and Experiment (5), it can be observed that the addition of MSANN has a greater impact on improvement in BraTS2020 than in BraTS2018, particularly in terms of accuracy. This could be attributed to the fact that MSANN is centered around deep learning, and issues related to the sample size may influence its effectiveness. Comparing Experiment (2) with Experiment (5), the integration of CLNN significantly enhances the overall performance of the model. Through a curriculum design with progressively challenging tasks and guided interpretation akin to radiological education, the model can better understand rule-based knowledge behind the tasks, leading to more rational feature extraction. Furthermore, from the comparison between Experiment (3) and Experiment (5), the inclusion of Radiomics features results in a substantial increase in accuracy. These features, encompassing texture, morphology, grayscale distribution, among others, prove to be highly beneficial for glioma staging. Lastly, the comparison between Experiment (4) and Experiment (5) shows that manually crafted diagnostic features predominantly driven by NCR/NET and ET region information play a crucial role in post-processing, aiding the model in achieving better classification results in glioma staging.
However, from the results of the permutation tests, it can be seen that the ablation experiments regarding MSANN and CLNN are not very ideal, as the P-values of both are greater than 0.05, indicating no significant differences. This is because their inherent accuracy in classification is much lower than that of Radiomics and Diag, which may result in lower classification contributions. In the ablation experiment results concerning Radiomics and Diag, the P-values of the permutation tests are all less than 0.05, showing significant differences. This demonstrates that the manually extracted features they provide contribute much more to predicting GBM than deep features. This also indirectly indicates that domain expertise is more directly and efficiently helpful for predicting GBM.
Discussion
The proposed method emphasizes improving the quality of feature extraction, and 4 feature extraction modules were designed for 3D MRI, including MSANN, CLNN, Radiomics Feature Extracter, and Diag-Feature Extracter. MSANN and CLNN were designed at both the model and strategy levels to mimic the radiologist’s image interpretation process in obtaining deep features. In MSANN, the incorporation of CAM and SAM allows the model to emulate the diagnostic attention of a radiologist by focusing on MRI sequences and lesion areas more relevant to the results. This involves assigning more biased weights to regions that are more relevant to the inference results and adaptively overlaying and emphasizing the weights of different image sequences. CLNN, leveraging curriculum learning strategy, mimics the sequential process of a radiologist’s image interpretation and attention to specific regions of different sequences, enabling the model to learn more reasonable features. Radiomics feature extraction from different sequences enables the model to capture information such as texture, morphology, and grayscale distribution specific to different lesion tissues, fully exploiting the roles of different image sequences. Guided by the feature extraction and classification results of imageomics, we found that the ET and NCR/NET regions contain more critical information for glioma staging. Therefore, we devised the Diag-Features primarily focus on these two regions, which has been well-validated through independent two-sample T-tests and Pearson Correlation Coefficient analysis. The success of the MCE model demonstrates the importance of feature extraction in medical image classification and how advanced feature extraction techniques can be utilized to enhance classification performance. This research provides valuable insights for the future development of the field of medical image analysis and offers potential methods and tools for further improving disease diagnosis and treatment.
It should be emphasized that this paper primarily focuses on the enhancement of feature extraction methods through the embedding of knowledge. The goal is to enable the model to incorporate reasonable external rules to guide the feature extraction process. Therefore, minimal selection and adjustment have been applied to the three types of feature classifiers, relying mainly on empirical and experimental insights. Additionally, this method incurs higher computational costs compared to other existing research, primarily due to its specialized design for 3D data. Nevertheless, in the context of glioma staging diagnosis, we believe that reliability should take precedence over considerations such as time and computational cost.
Conclusions
Medical imaging plays a crucial role in medical diagnosis, with continuous advancements propelled by innovations in image processing techniques. The emergence and evolution of deep learning technology have undoubtedly been one of the key drivers of advancements in medical image analysis in recent years. However, the scarcity of brain glioma MRI data makes it challenging for purely data-driven deep learning algorithms to extract high-quality features and capture their complex patterns, significantly affecting generalization capability. In order to extract higher-quality features from a limited number of samples, this paper introduces the MCE model, which incorporates external knowledge rules into the feature acquisition process. Furthermore, to address the issue of selecting the optimal slice when analyzing 2D slices, our model is meticulously designed based on 3D MRI data for brain imaging. We have designed two feature extraction models, MSANN and CLNN, to emulate the attention patterns of medical professionals when reviewing medical images. Additionally, we have designed an Radiomics features extraction model to capture specific information such as image texture, morphology, and grayscale distribution. Moreover, we have also designed manually created diagnostic features based on the results of Radiomics experiments, key points in radiological diagnosis, and imaging characteristics of the image sequences. Through these improvements in feature extraction, we have successfully achieved the high-quality features with increased information density. By introducing knowledge characteristics and external rules, model effectively incorporates the knowledge and expertise of medical professionals. Experimental results demonstrate that our proposed method outperforms purely data-driven models, achieving significant improvements in the classification and staging of brain gliomas. On the BraTS2018 and BraTS2020 datasets, our method achieves impressive Accuracy, Recall, and Precision rates of 96.14%, 93.44%, and 97.06%, as well as 97.57%, 92.80%, and 95.96%, respectively.
Acknowledgments
We want to thank “Changchun Computing Center” and “Eco-Innovation Center” for providing inclusive computing power and technical support of MindSpore during the completion of this paper.
References
- 1. Cosgrove N, Varešlija D, Keelan S, Elangovan A, Atkinson JM, Cocchiglia S, et al. Mapping molecular subtype specific alterations in breast cancer brain metastases identifies clinically relevant vulnerabilities. Nature communications. 2022;13(1):514. pmid:35082299
- 2. Louis DN, Ohgaki H, Wiestler OD, Cavenee WK, Burger PC, Jouvet A, et al. The 2007 WHO classification of tumours of the central nervous system. Acta neuropathologica. 2007;114:97–109. pmid:17618441
- 3. Gilanie G, Bajwa UI, Waraich MM, Anwar MW, Ullah H. An automated and risk free WHO grading of glioma from MRI images using CNN. Multimedia Tools and Applications. 2023;82(2):2857–2869.
- 4. Yang K, Wu Z, Zhang H, Zhang N, Wu W, Wang Z, et al. Glioma targeted therapy: insight into future of molecular approaches. Molecular Cancer. 2022;21(1):1–32. pmid:35135556
- 5. Guan Y, Aamir M, Rahman Z, Ali A, Abro WA, Dayo ZA, et al. A framework for efficient brain tumor classification using MRI images. Mathematical Biosciences and Engineering. 2021;18(5):5790–5815. pmid:34517512
- 6. Ranjbarzadeh R, Zarbakhsh P, Caputo A, Tirkolaee EB, Bendechache M. Brain tumor segmentation based on optimized convolutional neural network and improved chimp optimization algorithm. Computers in Biology and Medicine. 2024;168:107723. pmid:38000242
- 7. Bohlender S, Oksuz I, Mukhopadhyay A. A survey on shape-constraint deep learning for medical image segmentation. IEEE Reviews in Biomedical Engineering. 2021;16:225–240.
- 8. Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annual review of biomedical engineering. 2017;19:221–248. pmid:28301734
- 9. Haq EU, Jianjun H, Li K, Haq HU, Zhang T. An MRI-based deep learning approach for efficient classification of brain tumors. Journal of Ambient Intelligence and Humanized Computing. 2021; p. 1–22.
- 10. Coupet M, Urruty T, Leelanupab T, Naudin M, Bourdon P, Maloigne CF, et al. A multi-sequences MRI deep framework study applied to glioma classfication. Multimedia Tools and Applications. 2022;81(10):13563–13591. pmid:35250358
- 11. Angelin Jeba J, Nirmala Devi S, Meena M. Modified CNN architecture for efficient classification of glioma brain tumour. IETE Journal of Research. 2022; p. 1–14.
- 12. Tandel GS, Tiwari A, Kakde O. Performance enhancement of MRI-based brain tumor classification using suitable segmentation method and deep learning-based ensemble algorithm. Biomedical signal processing and control. 2022;78:104018.
- 13. Hyun CM, Kim KC, Cho HC, Choi JK, Seo JK. Framelet pooling aided deep learning network: the method to process high dimensional medical data. Machine Learning: Science and Technology. 2020;1(1):015009.
- 14.
Yan S, Yu Z, Zhang X, Mahapatra D, Chandra SS, Janda M, et al. Towards Trustable Skin Cancer Diagnosis via Rewriting Model’s Decision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023. p. 11568–11577.
- 15. Rehman A, Khan MA, Saba T, Mehmood Z, Tariq U, Ayesha N. Microscopic brain tumor detection and classification using 3D CNN and feature selection architecture. Microscopy Research and Technique. 2021;84(1):133–149. pmid:32959422
- 16. Narmatha C, Eljack SM, Tuka AARM, Manimurugan S, Mustafa M. A hybrid fuzzy brain-storm optimization algorithm for the classification of brain tumor MRI images. Journal of ambient intelligence and humanized computing. 2020; p. 1–9.
- 17. Sharif MI, Li JP, Khan MA, Kadry S, Tariq U. M3BTCNet: multi model brain tumor classification using metaheuristic deep neural network features optimization. Neural Computing and Applications. 2022; p. 1–16.
- 18. Deepa S, Janet J, Sumathi S, Ananth J. Hybrid Optimization Algorithm Enabled Deep Learning Approach Brain Tumor Segmentation and Classification Using MRI. Journal of Digital Imaging. 2023;36(3):847–868. pmid:36622465
- 19. Bidkar PS, Kumar R, Ghosh A. Segnet and salp water optimization-driven deep belief network for segmentation and classification of brain tumor. Gene Expression Patterns. 2022;45:119248. pmid:35667619
- 20. Jemimma T, Vetharaj YJ. Fractional probabilistic fuzzy clustering and optimization based brain tumor segmentation and classification. Multimedia Tools and Applications. 2022;81(13):17889–17918.
- 21. Suárez-García JG, Hernández-López JM, Moreno-Barbosa E, de Celis-Alonso B. A simple model for glioma grading based on texture analysis applied to conventional brain MRI. PloS one. 2020;15(5):e0228972. pmid:32413034
- 22. Akbarian S, Seyyed-Kalantari L, Khalvati F, Dolatabadi E. Evaluating knowledge transfer in the neural network for medical images. IEEE Access. 2023;11:85812–85821.
- 23. Xie X, Niu J, Liu X, Chen Z, Tang S, Yu S. A survey on incorporating domain knowledge into deep learning for medical image analysis. Medical Image Analysis. 2021;69:101985. pmid:33588117
- 24. Wang Y, Shao J, Wang P, Chen L, Ying M, Chai S, et al. Deep learning radiomics to predict regional lymph node staging for hilar cholangiocarcinoma. Frontiers in Oncology. 2021;11:721460. pmid:34765542
- 25. Saini M, Susan S. Vggin-net: Deep transfer network for imbalanced breast cancer dataset. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2022;20(1):752–762.
- 26.
Gracias C, Silveira M. Curriculum learning for early Alzheimer’s Disease diagnosis. In: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE; 2022. p. 4777–4780.
- 27. Yang M, Xie Z, Wang Z, Yuan Y, Zhang J. Su-micl: severity-guided multiple instance curriculum learning for histopathology image interpretable classification. IEEE Transactions on Medical Imaging. 2022;41(12):3533–3543. pmid:35786552
- 28. Xie XZ, Niu JW, Liu XF, Li QF, Wang Y, Han J, et al. DG-CNN: Introducing margin information into convolutional neural networks for breast cancer diagnosis in ultrasound images. Journal of Computer Science and Technology. 2022;37(2):277–294.
- 29. Zhou L, Zhu M, Xiong D, Ouyang L, Ouyang Y, Chen Z, et al. RNLFNet: Residual non-local Fourier network for undersampled MRI reconstruction. Biomedical Signal Processing and Control. 2023;83:104632.
- 30. Xie T, Wang Z, Li H, Wu P, Huang H, Zhang H, et al. Progressive attention integration-based multi-scale efficient network for medical imaging analysis with application to COVID-19 diagnosis. Computers in Biology and Medicine. 2023;159:106947. pmid:37099976
- 31. Algumaei AH, Algunaid RF, Rushdi MA, Yassine IA. Feature and decision-level fusion for schizophrenia detection based on resting-state fMRI data. Plos one. 2022;17(5):e0265300. pmid:35609033
- 32. Bogacsovics G, Toth J, Hajdu A, Harangi B. Enhancing CNNs through the use of hand-crafted features in automated fundus image classification. Biomedical Signal Processing and Control. 2022;76:103685.
- 33. Ranjbarzadeh R, Jafarzadeh Ghoushchi S, Tataei Sarshar N, Tirkolaee EB, Ali SS, Kumar T, et al. ME-CCNN: Multi-encoded images and a cascade convolutional neural network for breast tumor segmentation and recognition. Artificial Intelligence Review. 2023;56(9):10099–10136.
- 34. Xiao D, Zhao Z, Liu J, Wang X, Fu P, Le Grange JM, et al. Diagnosis of invasive meningioma based on brain-tumor interface radiomics features on brain MR images: a multicenter study. Frontiers in Oncology. 2021;11:708040. pmid:34504789
- 35. Al-Mekhlafi ZG, Senan EM, Mohammed BA, Alazmi M, Alayba AM, Alreshidi A, et al. Diagnosis of histopathological images to distinguish types of malignant lymphomas using hybrid techniques based on fusion features. Electronics. 2022;11(18):2865.
- 36.
Bagherian Kasgari A, Ranjbarzadeh R, Caputo A, Baseri Saadi S, Bendechache M. Brain tumor segmentation based on zernike moments, enhanced ant lion optimization, and convolutional neural network in MRI images. In: Metaheuristics and Optimization in Computer and Electrical Engineering: Volume 2: Hybrid and Improved Algorithms. Springer; 2023. p. 345–366.
- 37.
Díaz-Pernas FJ, Martínez-Zarzuela M, Antón-Rodríguez M, González-Ortega D. A deep learning approach for brain tumor classification and segmentation using a multiscale convolutional neural network. In: Healthcare. vol. 9. MDPI; 2021. p. 153.
- 38. Agravat RR, Raval MS. A survey and analysis on automated glioma brain tumor segmentation and overall patient survival prediction. Archives of Computational Methods in Engineering. 2021;28:4117–4152.
- 39. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby JS, et al. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific data. 2017;4(1):1–13. pmid:28872634
- 40. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE transactions on medical imaging. 2014;34(10):1993–2024. pmid:25494501
- 41.
Bakas S, Reyes M, Jakab A, Bauer S, Rempfler M, Crimi A, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:181102629. 2018.
- 42.
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer; 2015. p. 234–241.
- 43. Yousef R, Khan S, Gupta G, Siddiqui T, Albahlal BM, Alajlan SA, et al. U-Net-based models towards optimal MR brain image segmentation. Diagnostics. 2023;13(9):1624. pmid:37175015
- 44. Kasgari AB, Safavi S, Nouri M, Hou J, Sarshar NT, Ranjbarzadeh R. Point-of-interest preference model using an attention mechanism in a convolutional neural network. Bioengineering. 2023;10(4):495. pmid:37106681
- 45. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, Van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. European journal of cancer. 2012;48(4):441–446. pmid:22257792
- 46. Mayerhoefer ME, Materka A, Langs G, Häggström I, Szczypiński P, Gibbs P, et al. Introduction to radiomics. Journal of Nuclear Medicine. 2020;61(4):488–495. pmid:32060219
- 47. Van Griethuysen JJ, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational radiomics system to decode the radiographic phenotype. Cancer research. 2017;77(21):e104–e107. pmid:29092951