https://orcid.org/0009-0000-6639-0388
Figures
Abstract
Brain tumor is a common neurological surgical disease, where surgical resection is the primary treatment method. Neurosurgeons need to accurately determine the location of the tumor during tumor resection surgery, but existing clinical tumor identification technologies face numerous challenges, such as high equipment costs, long processing times, a certain degree of invasiveness, and insufficient image clarity. In this work, we propose a hyperspectral image detection algorithm based on the fusion of multiple features to maximize the determination of tumor boundaries. The algorithm establishes the machine learning models of Support Vector Machine (SVM) and Random Forest (RF) by integrating data features from optimal wavelengths, spectral indices, and textural features. Experimental results show that on different datasets, the classification accuracy of the three-feature fusion model is significantly higher than that of models using only two features or a single feature. Hyperspectral tumor image recognition can effectively help distinguish the tumors from the surrounding tissue, thereby enhancing the safety and thoroughness of tumor surgery.
Citation: Liu J, Zhang C, Xu J, Wang L, Pan M, Ma X, et al. (2026) Hyperspectral imaging for intraoperative brain tumor identification through fusion of spectral, textural, and spectral index features. PLoS One 21(2): e0340879. https://doi.org/10.1371/journal.pone.0340879
Editor: Longxiu Huang, Michigan State University, UNITED STATES OF AMERICA
Received: June 19, 2025; Accepted: December 29, 2025; Published: February 23, 2026
Copyright: © 2026 Liu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: This work was supported by the National Key Research and Development Program of China (Grant No. 2023YFF0715102), the Key Research and Development Plan in Shandong Province (Grant No. 2022CXGC010501), the National Natural Science Foundation of China (Grant No. 82227802), and the Taishan Young Scholar Program (Grant No. tsqn01909008). The funders provided financial support and general academic supervision for this study but had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Surgical resection is the conventional treatment for brain tumors, but it requires doctors to identify and remove tumor tissue as accurately as possible from normal brain tissue, otherwise it not only increases the risk of complications such as intracranial bleeding and infection but can also lead to tumor recurrence [1–3]. Conversely, excessive resection may alter the distribution of intracranial pressure, causing symptoms like headaches, nausea, vomiting, and even lead to severe consequences such as paralysis, loss of sensation, and speech disorders.Therefore, precise identification of tumor location during surgery is crucial for assisting doctors in resection, which can minimize damage to normal brain tissue [4]. Currently, the main technologies used in clinics to identify tumor locations are surgical navigation [5], intraoperative ultrasound [6,7], and intraoperative biopsy [8]. But they have their respective limitations. Surgical navigation systems are costly and rely on preoperative images, which may not adapt in real-time to anatomical changes that occur during surgery; intraoperative ultrasound provides non-standard images that differ from the standard images familiar to neurosurgeons; intraoperative biopsies can cause damage to the patient, are time-consuming, and carry the risk of false negatives.
Given the shortcomings of existing technologies, a cost-effective, accurate, and non-invasive technology is needed to develop for tumor resection [9]. Hyperspectral Imaging (HSI), as an emerging imaging technology, can generate medical images with one-dimensional spectral information and two-dimensional spatial information [10] by capturing the reflective or transmissive characteristics of tissue across multiple spectral bands. Benefiting from the advantage of broad spectral range and high resolution, HSI can provide detailed information for identifying tumor locations at the molecular and cellular levels. In recent years, the application of hyperspectral technology in cancer detection has become increasingly widespread [11]. For example, Huang Yi proposed a supervised least squares support vector machine segmentation method based on feature spectra for the segmentation of malignant melanocytes, achieving a segmentation accuracy of 85% in experimental samples of cutaneous melanoma. Leon and colleagues developed a dermatological HSI acquisition system and proposed a semi-supervised algorithm-based processing framework for the automatic identification and classification of PSL, achieving the diagnosed results with 87.5% of sensitivity and 100% of specificity. These studies indicate that the combination of hyperspectral imaging and machine learning technology is expected to provide an accurate technology for determining the boundaries of intraoperative brain tumors [12,13].
In this study, we proposed a hyperspectral imaging detection framework for brain tumor based on the fusion of multiple features to achieve precise judgment of tumor location and assist doctors in surgical resection. We first processed the data in three stages: using the DGWCR algorithm to select the optimal wavelengths (OWS), utilizing the Relief algorithm to choose appropriate spectral indices (SIS), and collecting textural features (TFS) through the IAPS algorithm. Then, we introduced different feature fusions into the SVM and RF detection models to achieve pixel-level detection of medical brain cancer hyperspectral data images. The main contributions of this study are as follows:
- (1) This is the first time that the classification performance of machine models has been enhanced through a three - feature fusion method.
- (2) The algorithm framework only requires 5 - pixel training, which solves the problem of the scarcity of medical datasets.
- (3) Compared with neural networks, the algorithm framework maintains high precision while having lower training costs.
Methods
Dataset
Ethics statement.
This study was conducted using a publicly available hyperspectral medical imaging dataset. All data were fully anonymized prior to release, and no personally identifiable information was included. The use of the dataset complied with the terms and conditions specified by the data provider. Therefore, informed consent and additional ethical approval were not required.
Patient tumor characteristics and image.
The experiments were conducted using data gathered between March 2015 and June 2016 from the European project "HELICoiD" (Hyperspectral Imaging for Cancer Detection) at the University of Las Palmas de Gran Canaria in Spain. The dataset [14] comprised in vivo Multispectral Hyperspectral Imaging System (MHSIS) images of human brains, obtained during neurosurgical operations at the University Hospital of Gran Canaria. Patients, aged 18 and above, diagnosed with either primary or secondary brain tumors, including those with Grade IV Glioblastoma Multiforme (GBM), were involved in this study.During the surgery, a craniotomy was performed to expose the brain’s surface, followed by the acquisition of HS images using a hyperspectral imaging system. Based on the guidance from the Image-Guided Surgery (IGS) system, rubber ring markers were strategically placed to delineate areas of tumor and normal tissue.
The datasets, labeled 8-01, 12-01, 12-02, and 20-01, consisted of four hyperspectral medical images from three GBM patients. The first three digits of each dataset identifier corresponded to the patient’s unique number, while the last two digits denoted the image sequence within that patient’s set. Leveraging the expertise of neurosurgeons, pixels within each MHSI image were categorized and labeled to produce a reference image, distinguishing four classes: Tumor Tissue (TT), Normal Tissue (NT), Blood Vessel (BV), and Background (BG). The Background class specifically referred to the non-biological, disinfected rubber rings used for tumor and healthy brain area identification. A detailed pixel count for each dataset was presented in Table 1, with color-coded labels for Normal Tissue in green, Tumor Tissue in red, Blood Vessel in blue, and Background in yellow.
For the model training, a random sampling method was employed, selecting five pixels per class to constitute the training set, thus streamlining the sampling process and ensuring a representative balance within the training samples. The remaining pixels were allocated to the test set for the evaluation of the trained model’s performance [15].
Image preprocessing.
The experiments utilized a Hyperspec VNIR series camera (HeadWall Photonics Inc., Fitchburg, MA, USA) for image generation. This camera is capable of capturing images within the spectral range of 400 to 1000 nm, with a sampling interval of 0.73 nm, as shown in Fig 1. The Hyperspectral Imaging system captures 826 spectral channels, covering the visible and near-infrared (VNIR) regions, with a spectral resolution of 2-3 nm. However, due to the lower performance of the HS sensor on certain channels, we excluded the extreme spectral channels, namely the first 56 and the last 126 channels, resulting in the use of 645 remaining spectral channels, as shown in Fig 1.
(a) Medical Dataset’s Spectral Information (b) Removal of noisy spectral bands.
Lighting conditions in the natural environment are constantly changing [16], and hyperspectral acquisition sensors respond differently to light, leading to significant environmental variations in the collected hyperspectral data [17]. To avoid the spectral non-uniformity of illumination devices and the issue of dark current generated by camera sensors, as well as to eliminate the impact of external lighting conditions, we use Eq 1 to calibrate the image reflectance on a per-pixel basis, where represents the white reference image obtained from a standard whiteboard, and
represents the dark reference image obtained with the camera shutter closed. By preprocessing to address the spectral non-uniformity of the illumination device and the dark current generated by the camera sensor, we remove noise from the spectral features and reduce the number of bands in the samples without losing relevant spectral information [18].
Feature extraction and fusion
Band selection.
Even after removing the interfering bands at the beginning and end of the spectral bands in hyperspectral images, due to the high correlation between bands in hyperspectral images, the processed medical hyperspectral data still suffers from data redundancy [19,20]. We decided to use a band selection algorithm to select more representative bands, retaining the useful information carried by the hyperspectral data, thereby reducing computational costs and time. Here, we chose the Data Gravitation and Weak Correlation-based Ranking algorithm [21] for band dimension reduction.
The DGWCR algorithm, compared to traditional algorithms, can compress the dimension without destroying the original data structure and does not require manual annotation of samples. As shown in Fig 2, the Connection Centre Evolution (CCE) [22] algorithm is used to determine the clustering centers, and then, based on the data gravitation and the similarity matrix containing entropy, the most representative bands are selected within each cluster according to the data gravitation. Within each cluster, other bands are ranked according to weak correlation. Finally, a global ranking of the bands is obtained based on the S-shaped ranking strategy, and the top 20 bands are selected as the data for band selection.
Spectral indices.
After brain tissue becomes infected with cancer, the physiological characteristics of the tissue, such as proteins, lipids, and water content, [10,23,24] undergo changes, which in turn alter the reflectance of the corresponding location in the hyperspectral image. Spectral indices are tools used to describe certain spectral features, and by combining the reflectance of several specific wavelengths, they help analyze and understand the manifestation of different material components. To enhance the differences between tumor tissue and normal tissue, we selected 38 spectral indices related to tissue color, tissue structure, tissue physiology, and tissue water content from relevant literature, as shown in Table 2. We then used the Relief algorithm to identify spectral indices that can significantly differentiate tumors from these 38 indices.
Relief algorithm is a feature weighting algorithm that assigns different weights to features based on their correlation with the class labels, and features with weights below a certain threshold are removed [25–27]. The Relief algorithm randomly selects a sample R from the training sample set each time, and then identifies k nearest neighbor samples (near Hits) from the same class as R, and k nearest neighbor samples (near Misses) from each different class of R. It then updates the weights of each feature according to Eq 2. Here, diff(A,,
) represents the difference in feature A between samples
and
(Eq 3).
is the sample from the i-th sampling,
is the j-th nearest neighbor of
, m is the number of samplings, k is the number of nearest neighbors,
is the probability of the occurrence of class C,
is the class to which the random sample R belongs,
is the probability of the occurrence of the random sample R, and Mj(C) indicates the j-th nearest neighbor sample that does not belong to Class(R) in class C.
We selected 70 hyperspectral images of tumors, of which 38 images contained tumors and 32 were images of normal tissue. We labeled whether they contained tumors with 0 and 1, which will serve as training labels. By averaging the reflectance of each band, we obtained the average reflectance spectrum of the hyperspectral image, and then calculated its 38 spectral indices from the spectral curve. By calculating the correlation between these 38 spectral indices and the presence of tumors, we obtained a weight diagram as shown in Fig 3. We selected the top 5 spectral indices most sensitive to tumors, RGI, PRI, NPCI, NRI, and mRESR, as the spectral index data.
Texture features.
Hyperspectral images contain not only rich spectral information but also important spatial information. Textural features can capture the spatial information in brain cancer, including the shape, size, and distribution of the cancer, which is crucial for distinguishing different tissues with similar spectral characteristics [28]. In hyperspectral imaging, due to the lower spatial resolution and a higher number of mixed pixels, detection relying solely on spectral information may not be accurate enough. Moreover, in some cases, different tissues may have similar spectral features, or the spectral characteristics of the same tissue may vary significantly under different conditions. Textural features provide an additional dimension of information that helps address these issues, leading to more accurate tumor identification. Furthermore, textural features are somewhat adaptable to changes in image acquisition conditions, such as lighting and shadows, making them useful for detection information under variable environmental conditions. Therefore, we use the Invariant Attribute Profiles (IAPS) algorithm to extract textural features, enhancing the model’s ability to recognize brain cancer tissue from a spatial dimension.
The IAPS algorithm processes spatial information by applying isotropic filters or convolution kernels to hyperspectral images to extract features that are invariant to translation and rotation. It then employs the Simple Linear Iterative Clustering (SLIC) superpixel segmentation [29,30] technique to divide the image into regions with similar characteristics, forming clusters. In the spectral frequency domain, features from the spatial domain are subjected to a Fourier transform, converting the image from the spatial domain to the frequency domain, and simultaneously transforming the Fourier-transformed image from the Cartesian coordinate system to the polar coordinate system. Under polar coordinates, by performing a polar Fourier transform on each pixel of the input image, we can obtain m amplitude features corresponding to m different Fourier orders with each pixel (Eq 4).
Eq 5) obtain absolute rotation-invariant features by removing the phase information.
To mitigate the loss of rich phase information, relative rotation-invariant features are developed by coupling the obtained additional convolutional Fourier representation with two adjacent convolution kernel radii (Eq 6).
We integrate the three components as shown in Eq 7.
Using isotropic triangular convolution kernels to aggregate the Polarimetric Weighted Fractal Features (PWFF) to obtain frequency-domain invariant features, and finally stacking the features extracted from the spatial and frequency domains to form the final Invariant Attribute Profiles (IAPs) feature representation. Through the acquisition of IAPs, a robust texture feature extraction can be provided for the model. In terms of textural features, the top 5 dimensions ranked by IAPS are selected as the textural feature data.
Feature fusion.
Feature fusion is a pivotal concept in the realms of machine learning and data analysis, with significant applications in image processing, computer vision, and medical image analysis. This process entails the amalgamation of features derived from various sources or stages of processing. The objective of feature fusion is to bolster the model’s capacity to generalize by harnessing the strengths of diverse features, thereby enhancing the model’s comprehension of the data and elevating the precision of classification, detection, or predictive tasks. We have implemented feature fusion in hyperspectral tumor detection, and to substantiate the notion that the integration of three distinct data modalities can augment the precision of medical hyperspectral detection, we conducted a comparative analysis of detection accuracy across seven feature configurations: three individual features (OWS,SIS,TFS), three combined pairs of these features (OWS+SIS, OWS+TFS, SIS+TFS), and the comprehensive fusion of all three features (ALL). The method of combination employed here involves a concatenation along the third dimension.
Classifier and evaluation
Classifier selection.
Currently popular medical hyperspectral detection models often come with built-in model selection and texture acquisition capabilities. To avoid the impact of complex algorithms on the fusion of the three types of data, we choose SVM (Support Vector Machine) and RF (Random Forest), two common and simple algorithms, to prevent the hyperspectral detection from gaining an advantage from the model itself. In hyperspectral image detection, SVM can effectively identify the spectral characteristics of different materials and achieve high-precision detection by selecting the appropriate kernel function and parameters. RF, as an ensemble learning method, has demonstrated its powerful capabilities in past medical hyperspectral image detections. It constructs multiple decision trees and aggregates their predictions, effectively handling high-dimensional data and capturing complex patterns within the data. In the experiment, cross-validation methods were used to find the optimal penalty parameter C and kernel function parameter gamma for SVM, and a random forest composed of 200 decision trees was used. This experiment was debugged in MATLAB 2020a and run on a personal computer with an Intel Core i7-12650H central processor and 16GB of RAM.
Evaluation metrics.
In this study, we employed three widely recognized standards to assess the performance of pixel-level detection in medical images: Overall Accuracy, Average Accuracy, and the Kappa coefficient [15]. These evaluation metrics provide us with an intuitive method for quantifying the results of our experiments and are frequently used in research within the relevant field. Overall Accuracy is a metric that quickly gives an overview of the model’s average performance across all categories. It measures the proportion of correctly detected samples out of the total samples, reflecting the detector’s overall performance on all categories. Average Accuracy takes into account the detection effect of each category, calculating the average of the accuracies of each category. AA is particularly important for dealing with imbalanced datasets because it can reveal the model’s performance differences across various categories, helping us identify the strengths and weaknesses of the model. The Kappa coefficient is a statistical measure used to assess the performance of a detector while considering the possibility of random detection. The value of the Kappa coefficient ranges from –1 to 1, with 1 indicating perfect agreement and 0 indicating that the detector’s performance is no different from random detection. The advantage of this coefficient is that it accounts for random agreement, thus providing an accurate assessment of the detector’s performance even in cases where the category distribution is uneven. By considering these three metrics comprehensively, we can evaluate the performance of medical image detection models in a holistic manner, ensuring that our research results are both scientifically sound and practically valuable.
Workflow of the study
In this study, we developed a machine learning algorithm for hyperspectral image (HSI) classification by fusing multiple types of features (Fig 4). The fusion integrates information from three complementary aspects: wavelength selection, disease indices, and texture features. Using this approach, the algorithm accurately delineates tumor locations and boundaries across different glioma HSIs. Moreover, the effectiveness and generalizability of the three-feature fusion in enhancing model accuracy were validated using two distinct machine learning models.
Results
Performance of the model on SVM
Our algorithm was implemented with SVM classifier across four datasets, conducting experiments to validate the efficacy of the three-feature fusion approach and comparing it with other feature combinations. To provide a more intuitive understanding of the classification effects under different feature inputs, we visualized the classification results for seven types of inputs. For the sake of brevity in the manuscript, only the classification effect diagrams for different inputs of dataset 8-01 were displayed.
As shown in Table 3, for the dataset 8-01, the three-feature fusion input achieved the best performance with OA, AA, and KA of 91.86%, 91.68%, and 0.8941, respectively. Among the single features, OWS achieved the best overall accuracy with 89.48%, and the three-feature fusion improved this by 2.4%. In the two-feature fusions, the three-feature fusion led with an increase of 0.3% over TFS+SIS, 5.79% over OWS+SIS, and 19.53% over OWS+TFS. The three-feature fusion also showed varying levels of improvement in AA and KA. As illustrated in Fig 5, Fig 5i, which represented the detection map for SVM with three-feature fusion input, more closely matched the ground truth compared to Fig 5b–5h. Only a small number of vascular pixels were misclassified as other categories in Fig 5i. The judgment on the tumor was completely consistent with Fig 5b.
(a) False color image. (b) Ground truth image. (c) OWS. (d) SIS. (e) TFS. (f) OWS + SIS. (g) OWS + TFS. (h) SIS + TFS. (i) All features combined.
The SVM detector achieved the best performance in terms of OA, AA, and KA with scores of 96.49%, 94.53%, and 0.9327 respectively, when using three feature fusion inputs in the 12-01 dataset. In the 12-02 dataset, the three-feature fusion reached the maximum values of OA, AA, and KA with 95.57%, 95.62%, and 0.9415 respectively. In the 20-01 dataset, the three-feature fusion obtained the maximum values of OA, AA, and KA with 95.95%, 93.85%, and 0.9469 respectively. The consistent performance across various datasets demonstrated the reliability of the three-feature fusion in enhancing tumor detection accuracy. As shown in Fig 6, the detection effect diagram with three-feature input showed minimal error when compared to the ground truth. In terms of tumor pixel detection, aside from some errors within the yellow circle in the upper right corner of Fig 6c, the results were consistent with the corresponding actual ground truth images.
(a, b, c) Images from patient 12-01. (d, e, f) Images from patient 12-02. (g, h, i) Images from patient 20-01.
Performance of the model on RF
To verify the detection accuracy and effectiveness on RF, we also conducted experiments on four Multi-Hyppectral Imaging (MHSI) datasets, using OA, AA, and KA as metrics for accuracy assessment, as shown in Table 4. Similarly, we only visualized the detection effect diagrams of the seven types of inputs on one dataset for comparison.
In the RF detector, the 8-01 dataset achieved the highest metrics in OA, AA, and KA with 99.21%, 99.28%, and 0.9897, respectively. For the 12-01 dataset, RF achieved a detection accuracy higher than other inputs with 90.95% OA, 0.8852 KA. For the 12-02 dataset, RF also outperformed other inputs with the same metrics of 89.56% OA, 91.34% AA, and 0.8616 KA. The detection effect diagrams for the three datasets were shown in Fig 7. On the 12-01 dataset, under the yellow rubber band in the upper right corner, only some tumor pixels were incorrectly detected as other tissues. On the 8-01 and 12-02 datasets, tumor pixels were perfectly detected by the model.
(a) False color image, (b) Ground truth image, (c) OWS, (d) SIS, (e) TFS, (f) OWS + SIS, (g) OWS + TFS, (h) SIS + TFS, (i) All features combined.
The detection accuracy of the 20-01 dataset on the RF was shown in Table 4. The RF led other inputs by an average of about 8% in the OA metric with 93.35%, by about 7% in the AA with 90.52%, and by about 0.1 in the KA with 0.9131. We trained the corresponding models for the seven types of inputs. Fig 8i showed the predicted detection diagram of the RF with three-feature fusion; it was consistent with Fig 8b except for some errors in the upper right rubber band and the middle blood vessel. Compared to the detection performance (Fig 8c–8h) under the other six inputs, the detection effect diagram with three-feature fusion showed superior performance in identifying the location of the tumor for Fig 8a.
(a, b, c) Images from the 8-01 dataset. (d, e, f) Images from the 12-01 dataset. (g, h, i) Images from the 12-02 dataset.
Summary of model classification performance
To validate the performance of a hyperspectral tumor image detection algorithm based on the fusion of multiple features, we conducted comparisons of seven different feature combinations on four medical hyperspectral datasets using two detectors. The evaluation was performed using three assessment metrics, and the statistical effects were shown in Fig 9.
Regarding the seven data combinations, it could be observed that the three basic single data inputs, particularly the optimal wavelength parts, yield the best detection effects, which was entirely reasonable. Spectral indices and textural features primarily served an auxiliary role; spectral indices aimed to enhance the data’s sensitivity to tumor tissues, while textural features provided additional spatial information. From Fig 9, it could be observed that the overall trend for the four datasets showed that the fusion of features performed better than a single input, with the three-feature fusion showing the most outstanding performance.
In terms of detectors, the performance of SVM and RF was not consistent across different datasets; sometimes RF outperforms, and other times SVM was better. We believed this was due to minor differences in the datasets. The purpose of using two detectors was to avoid the improvement in accuracy being specific to a particular detector; in practical applications, only one detector could be used.
Discussion
In this study, we are committed to developing a hyperspectral tumor image detection framework suitable for clinical use, to meet the need for precise identification of brain tumor locations. This framework significantly improves the accuracy of tumor identification by integrating three key data types: wavelength selection, spectral indices, and textural features, thereby achieving precise localization of tumors.
Hyperspectral imaging technology, with its outstanding capabilities, is increasingly valued in the medical field [9,11]. It not only extracts the spectral characteristics of each pixel but also provides detailed information about the different tissue components and their spatial distribution. At specific wavelengths, tissues with different pathological states exhibit different reflectance, absorption, and electromagnetic energy characteristics due to differences in chemical composition and physical properties, which are manifested through characteristic spectra. By carefully analyzing these spectral signals, we can achieve qualitative or quantitative detection of tissue conditions. Combined with the spatial distribution information provided by hyperspectral images, we can visualize various pathological states of tissues and diagnose the disease conditions of tissues. In recent years, the application of hyperspectral technology in the field of medical diagnosis has gradually increased [31]. For example, Song Nan and others successfully identified the spectral differences between gliomas and normal brain tissues by deeply analyzing and processing hyperspectral data, providing a new method for distinguishing between gliomas and normal brain tissues. Yi Weisong and others used a near-infrared hyperspectral imaging system to obtain hyperspectral images of gastric cancer tissues and normal tissues [32], extracted the characteristic spectra of tumors and normal tissues by chemometric methods, and performed spectral attribution. They further combined multiple target recognition algorithms to automatically extract gastric cancer tumor targets, and ultimately used image fusion technology to visually and intuitively display the recognition results, forming a method system for diagnosing gastric cancer tissues with hyperspectral images. These studies show that hyperspectral imaging technology is not only highly credible in tumor identification but also can provide more accurate and in-depth diagnostic information [33].
In the field of hyperspectral image detection, the application of deep learning has been very extensive [34–36], and many researchers have proposed many innovative and powerful frameworks, which have been effectively verified on remote sensing datasets [37–39]. For example, Li Ming-song fused the central pixel and neighboring pixels through the network to achieve detection [40], while Zheng Zhuo designed a lateral connection module to connect the encoder and decoder, thereby fusing the spatial details in the encoder and the semantic features in the decoder, [41] etc. In the field of hyperspectral medical image detection, although many innovations focus on the optimization of deep learning network architecture, such as introducing residual connections, attention mechanisms, encoding and decoding, and other advanced technologies to improve detection accuracy, our research focus has shifted to innovation at the data level. We are committed to exploring how to enhance the detection capabilities of the model through diversified processing and feature fusion of raw hyperspectral data [42,43]. This innovative method, which starts from the data, aims to provide new solutions for the precise detection of hyperspectral medical images by deepening data understanding and feature extraction. We try to process raw hyperspectral data in various ways and improve detection accuracy through feature fusion. Feature fusion is a machine learning technique that merges features from different sources to form a better feature representation, thereby enhancing model performance [44]. In the spectral analysis of medical images, optimal wavelength features correspond to the spectral absorption or reflection characteristics of tumor tissues [45], which can provide highly specific identification of the lesion area. Spectral indices can help enhancing the model’s sensitivity to tumor tissues [46], especially in distinguishing between tumors and normal tissues. Textural features reflect the spatial heterogeneity of tissue structure, providing important morphological information for the image [47], which helps to identify the boundaries and morphological characteristics of tumors. By combining the advantages of the three features, enhancing the model’s understanding of the data, and thus improving the model’s accuracy.
To select the appropriate bands for feature fusion, we used the DGWCR algorithm [21] to screen 20 representative bands from a large number of bands, forming the data basis for optimal wavelength. To enhance the model’s sensitivity to tumor tissue, we used the Relief algorithm [48] to select 5 spectral indices most related to tumor tissue from 38 spectral indices, as the data part of the spectral index. At the same time, to provide rich spatial texture information, we applied the IAPS algorithm [49] to calculate 5-dimensional regional texture features. By using band selection as the basis, texture features provide spatial auxiliary information, and spectral index data enhance the sensitivity to tumor tissue, we try to improve the performance of the detection model by splicing the three types of data. This method not only innovates the way of data processing but also is expected to achieve higher precision diagnosis in the field of hyperspectral medical image detection, bringing new perspectives and possibilities for clinical applications.
To verify the effect of the three-feature fusion on improving detection accuracy, we used two widely used detectors in brain cancer detection and analyzed them separately using three independent features and their fused features. In the early stage of the experiment, we conducted a detailed preprocessing of hyperspectral data to eliminate the impact of environmental and equipment factors. Based on the selected optimal wavelengths, spectral indices, and textural features, we constructed SVM and RF detection models [50,51] and detected tumor hyperspectral images using seven different input combinations and two detectors. We used standard evaluation indicators, including overall accuracy, average accuracy, and Kappa coefficient, to comprehensively evaluate the detection results. In addition, to intuitively display the detection effect, we visualized each detection result. The experimental results show that on four different datasets and two detectors, the detection method that fuses three features is superior to the other six single or two-way fusion input methods in all detection indicators, achieving excellent detection results. In the experimental design, we deliberately chose a variety of datasets to avoid the accidental results that a single dataset might bring; at the same time, we used two different detectors to exclude the impact of the detector’s own characteristics on the improvement of accuracy. These experimental designs ensure the reliability and universality of our results. Ultimately, the experimental results prove that under the combined action of optimal wavelengths, spectral indices, and textural features, the detection accuracy and performance of hyperspectral data have been significantly improved, and this comprehensive method provides a powerful auxiliary tool for doctors to identify tumor locations.
Although our research has achieved certain results, there are still some limitations. First, the datasets we used are public and only include patients with IV glioblastoma, which may lead to sample selection bias and limit the universal applicability of the research results. To overcome this limitation, we plan to include more data from patients with different types of tumors, different ages, and genders in future research to ensure the accuracy and wide applicability of the research. Secondly, on the algorithm level, although the DGWCR, Relief, and IAPS algorithms played an important role in this study, there is still room for further optimization. We consider replacing these algorithms with algorithms that can more efficiently extract effective data and speed up processing to improve the response speed and practicality of the entire framework. Overall, we are committed to building a powerful, universal, and easily promoted framework based on the fusion of optimal wavelengths, spectral indices, and textural features to achieve precise localization of tumor positions and assist doctors in tumor resection surgery. This study, as a high-quality study in the field of medical hyperspectral imaging and artificial intelligence, shows great clinical application potential. We believe that through continuous technological innovation and optimization, our research will bring new breakthroughs to the fields of medical image analysis and tumor diagnosis, providing patients with more accurate and personalized treatment plans.
References
- 1. Bindal RK, Sawaya R, Leavens ME, Lee JJ. Surgical treatment of multiple brain metastases. J Neurosurg. 1993;79(2):210–6. pmid:8331402
- 2. DeAngelis LM. Brain tumors. N Engl J Med. 2001;344(2):114–23. pmid:11150363
- 3. McFaline-Figueroa JR, Lee EQ. Brain Tumors. Am J Med. 2018;131(8):874–82. pmid:29371158
- 4. Raskind R, Weiss SR, Manning JJ, Wermuth RE. Survival after surgical excision of single metastatic brain tumors. Am J Roentgenol Radium Ther Nucl Med. 1971;111(2):323–8. pmid:5100218
- 5.
Figueras-Benítez G, Urbano L, Acero A, Huerta M, Castro M. Surgical navigation systems: a technological overview. In: VII International Conference on Electrical Engineering. 2014. p. 1–5.
- 6. Makuuchi M, Torzilli G, Machi J. History of intraoperative ultrasound. Ultrasound Med Biol. 1998;24(9):1229–42. pmid:10385947
- 7. Machi J, Oishi AJ, Furumoto NL, Oishi RH. Intraoperative ultrasound. Surg Clin North Am. 2004;84(4):1085–111, vi–i. pmid:15261754
- 8. AlHilli MM, Podratz KC, Dowdy SC, Bakkum-Gamez JN, Weaver AL, McGree ME, et al. Preoperative biopsy and intraoperative tumor diameter predict lymph node dissemination in endometrial cancer. Gynecol Oncol. 2013;128(2):294–9. pmid:23085458
- 9. Tsai C-L, Mukundan A, Chung C-S, Chen Y-H, Wang Y-K, Chen T-H, et al. Hyperspectral imaging combined with artificial intelligence in the early detection of esophageal cancer. Cancers (Basel). 2021;13(18):4593. pmid:34572819
- 10. Antequera T, Caballero D, Grassi S, Uttaro B, Perez-Palacios T. Evaluation of fresh meat quality by Hyperspectral Imaging (HSI), Nuclear Magnetic Resonance (NMR) and Magnetic Resonance Imaging (MRI): a review. Meat Sci. 2021;172:108340. pmid:33129061
- 11. Aboughaleb IH, Aref MH, El-Sharkawy YH. Hyperspectral imaging for diagnosis and detection of ex-vivo breast cancer. Photodiagnosis Photodyn Ther. 2020;31:101922. pmid:32726640
- 12. Khan MJ, Khan HS, Yousaf A, Khurshid K, Abbas A. Modern trends in hyperspectral image analysis: a review. IEEE Access. 2018;6:14118–29.
- 13. Amigo JM, Babamoradi H, Elcoroaristizabal S. Hyperspectral image analysis. A tutorial. Anal Chim Acta. 2015;896:34–51. pmid:26481986
- 14. Fabelo H, Ortega S, Szolna A, Bulters D, Pineiro JF, Kabwama S, et al. In-vivo hyperspectral human brain image database for brain cancer detection. IEEE Access. 2019;7:39098–116.
- 15. M H, M.N S. A review on evaluation metrics for data classification evaluations. IJDKP. 2015;5(2):01–11.
- 16. Tang Y, Song S, Gui S, Chao W, Cheng C, Qin R. Active and low-cost hyperspectral imaging for the spectral analysis of a low-light environment. Sensors (Basel). 2023;23(3):1437. pmid:36772477
- 17. Landgrebe D. Hyperspectral image data analysis. IEEE Signal Process Mag. 2002;19(1):17–28.
- 18. Mäkelä M, Geladi P, Rissanen M, Rautkari L, Dahl O. Hyperspectral near infrared image calibration and regression. Anal Chim Acta. 2020;1105:56–63. pmid:32138926
- 19. Tang C, Wang J, Zheng X, Liu X, Xie W, Li X, et al. Spatial and spectral structure preserved self-representation for unsupervised hyperspectral band selection. IEEE Trans Geosci Remote Sensing. 2023;61:1–13.
- 20. Ayna CO, Mdrafi R, Du Q, Gurbuz AC. Learning-based optimization of hyperspectral band selection for classification. Remote Sensing. 2023;15(18):4460.
- 21. Zhang C, Zhang Z, Yu D, Cheng Q, Shan S, Li M, et al. Unsupervised band selection of medical hyperspectral images guided by data gravitation and weak correlation. Comput Methods Programs Biomed. 2023;240:107721. pmid:37506601
- 22. Tang H, Zhang Y, Zhang P, Ji L. An automatic band selection algorithm based on connection Centre evolution. Remote Sensing Letters. 2023;14(4):323–33.
- 23. Day L, Cakebread JA, Loveday SM. Food proteins from animals and plants: differences in the nutritional and functional properties. Trends in Food Science & Technology. 2022;119:428–42.
- 24. Li X, Zheng S, Ma X, Cheng K, Wu G. Effects of dietary starch and lipid levels on the protein retention and growth of largemouth bass (Micropterus salmoides). Amino Acids. 2020;52(6–7):999–1016. pmid:32648068
- 25. Cui X, Li Y, Fan J, Wang T. A novel filter feature selection algorithm based on relief. Appl Intell. 2021;52(5):5063–81.
- 26. Malik H, Yadav AK. A novel hybrid approach based on relief algorithm and fuzzy reinforcement learning approach for predicting wind speed. Sustainable Energy Technologies and Assessments. 2021;43:100920.
- 27. Jin C, Kong X, Chang J, Cheng H, Liu X. Internal crack detection of castings: a study based on relief algorithm and Adaboost-SVM. Int J Adv Manuf Technol. 2020;108(9–10):3313–22.
- 28.
Bandara WGC, Patel VM. Hypertransformer: A textural and spectral feature fusion transformer for pansharpening. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. p. 1767–77.
- 29. Yin J, Wang T, Du Y, Liu X, Zhou L, Yang J. SLIC superpixel segmentation for polarimetric SAR images. IEEE Trans Geosci Remote Sensing. 2022;60:1–17.
- 30. Di S, Liao M, Zhao Y, Li Y, Zeng Y. Image superpixel segmentation based on hierarchical multi-level LI-SLIC. Optics & Laser Technology. 2021;135:106703.
- 31. Karim S, Qadir A, Farooq U, Shakir M, Laghari AA. Hyperspectral imaging: a review and trends towards medical imaging. Curr Med Imaging. 2022;19(5):417–27. pmid:35598236
- 32.
Yi W, Zhang J, Jiang H, Zhang N. Gastric cancer target detection using near-infrared hyperspectral imaging with chemometrics. In: SPIE Proceedings, 2014. 92301V. http://dx.doi.org/10.1117/12.2068821
- 33. Mangotra H, Srivastava S, Jaiswal G, Rani R, Sharma A. Hyperspectral imaging for early diagnosis of diseases: a review. Expert Systems. 2023;40(8).
- 34. Cheshkova AF. A review of hyperspectral image analysis techniques for plant disease detection and identif ication. Vavilovskii Zhurnal Genet Selektsii. 2022;26(2):202–13. pmid:35434482
- 35. Yan L, Zhao M, Wang X, Zhang Y, Chen J. Object detection in hyperspectral images. IEEE Signal Process Lett. 2021;28:508–12.
- 36. Soni A, Dixit Y, Reis MM, Brightwell G. Hyperspectral imaging and machine learning in food microbiology: developments and challenges in detection of bacterial, fungal, and viral contaminants. Compr Rev Food Sci Food Saf. 2022;21(4):3717–45. pmid:35686478
- 37. Seyrek EC, Uysal M. A comparative analysis of various activation functions and optimizers in a convolutional neural network for hyperspectral image classification. Multimed Tools Appl. 2023;83(18):53785–816.
- 38. Torun O, Yuksel SE, Erdem E, Imamoglu N, Erdem A. Hyperspectral image denoising via self-modulating convolutional neural networks. Signal Processing. 2024;214:109248.
- 39. Sahoo AR, Chakraborty P. Hybrid CNN Bi-LSTM neural network for hyperspectral image classification. arXiv preprint 2024.
- 40. Li M, Liu Y, Xue G, Huang Y, Yang G. Exploring the relationship between center and neighborhoods: central vector oriented self-similarity network for hyperspectral image classification. IEEE Trans Circuits Syst Video Technol. 2023;33(4):1979–93.
- 41. Zheng Z, Zhong Y, Ma A, Zhang L. FPGA: fast patch-free global learning framework for fully end-to-end hyperspectral image classification. IEEE Trans Geosci Remote Sensing. 2020;58(8):5612–26.
- 42. Liang M, Jiao L, Yang S, Liu F, Hou B, Chen H. Deep multiscale spectral-spatial feature fusion for hyperspectral images classification. IEEE J Sel Top Appl Earth Observations Remote Sensing. 2018;11(8):2911–24.
- 43.
Dobhal S. Performance analysis of high-resolution and hyperspectral data fusion for classification and linear feature extraction. ITC; 2008.
- 44. Kibriya H, Amin R, Alshehri AH, Masood M, Alshamrani SS, Alshehri A. A novel and effective brain tumor classification model using deep feature fusion and famous machine learning classifiers. Comput Intell Neurosci. 2022;2022:7897669. pmid:35378808
- 45. Huang Q, Zhu M, Xu Z, Kan R. A new framework for interval wavelength selection based on wavelength importance clustering. Anal Chim Acta. 2024;1326:343153. pmid:39260919
- 46. Mahajan GR, Das B, Murgaokar D, Herrmann I, Berger K, Sahoo RN, et al. Monitoring the foliar nutrients status of mango using spectroscopy-based spectral indices and PLSR-combined machine learning models. Remote Sensing. 2021;13(4):641.
- 47. Guo A, Huang W, Ye H, Dong Y, Ma H, Ren Y, et al. Identification of wheat yellow rust using spectral and texture features of hyperspectral images. Remote Sensing. 2020;12(9):1419.
- 48. Zhang J, Liu K, Yang X, Ju H, Xu S. Multi-label learning with relief-based label-specific feature selection. Appl Intell. 2023;53(15):18517–30.
- 49. Constantinescu AC, Wolters M, Moore A, MacPherson SE. A cluster-based approach to selecting representative stimuli from the International Affective Picture System (IAPS) database. Behav Res Methods. 2017;49(3):896–912. pmid:27287449
- 50. Tun W, Wong JK-W, Ling S-H. Hybrid random forest and support vector machine modeling for HVAC fault detection and diagnosis. Sensors (Basel). 2021;21(24):8163. pmid:34960257
- 51. Momade MH, Shahid S, Hainin MR bin, Nashwan MS, Tahir Umar A. Modelling labour productivity using SVM and RF: a comparative study on classifiers performance. International Journal of Construction Management. 2020;22(10):1924–34.