2.1 PCA-based feature extraction for HSI

PCA is a popular feature extraction and dimensionality reduction technique used for analyzing HSI data. Its objective is to retain crucial information while transforming the high-dimensional data into a lower-dimensional space. PCA achieves this by identifying principal components (PCs) that capture maximum variance in the data [52]. To implement PCA, each sample pixel’s spectral vector in the HSI data matrix is denoted as x_n = [x_n1x_n2…x_nF]^T, with n ∈ [1, S], and S = X × Y representing the spatial dimensions. The data matrix D is used to derive the zero-mean image I, where I_n = x_n − M, and . The covariance matrix is computed using eigen decomposition C = V E V^T, where E and V are eigenvalues and eigenvectors (PCs), respectively. A subset of q PCs, represented by matrix W, is selected based on descending eigenvalue rankings. Finally, the projection matrix Z of D is obtained as Z = W^T × I.

Seg-PCA implementation involves dividing the HSI data matrix into L subgroup datasets based on band correlations. Each subgroup captures local characteristics effectively by grouping strongly correlated bands together. Seg-PCA selectively reduces the number of spectral bands while preserving spatial dimensions. In Seg-PCA, the HSI data matrix D is divided into subgroups D_t, where t ∈ [1, 2, ..L]. Eigendecomposition is applied to each D_t to compute covariance matrices. The projection matrices of each subgroup are then combined to obtain the overall projection matrix.

2.2 mRMR based feature selection for HSI

mRMR is a widely used feature selection method in the analysis of HSI data. It aims to select the most relevant features related to the target variable while minimizing redundancy among them. The process involves calculating the Mutual Information (MI) between the target variable and each feature. The features are then sorted in descending order based on their MI scores, and they are added one by one to the subset until the desired number is achieved [20, 53]. To perform the MI-based feature selection, two input images, denoted as X and C, are required, along with their respective marginal probability distributions, represented as p(x) and p(c), as well as their joint probability distribution p(x, c). The MI, denoted as I(X, c), is calculated using these probability distributions in the following equation: (1)

In the context of HSI analysis, MI-based feature selection uses X as the input extracted spectral features and C as the corresponding ground truth image. The aim is to maximize the MI when X and C are equivalent. To avoid redundancy, the greedy approach of MI-based feature selection selects the (k + 1)^th feature while taking into account the previously selected Q features, using the following equation: (2) Here, β regulates redundancy’s significance in the mRMR criterion, and is the normalized Mutual Information defined as: (3)

To ensure adequate feature selection features with are eliminated, where ξ is a threshold representing the minimum level of relevance required for the target data. Also, the condition is used to verify if the chosen feature is unalike to the already selected ones, ensuring desirable selections.

2.3 Convolutional Neural Network (CNN)

Indeed, CNNs are a feedforward neural network that performs remarkably in large-scale image processing tasks. CNNs are specifically designed to handle image data efficiently, using their ability to learn hierarchical features from raw pixel values automatically [54–60]. It leverages the 2D image structure, recognizing pixel correlations in adjacent regions, and utilizes feature sharing through convolution. Thus, it detects local patterns and features in the input images, making them highly effective for tasks such as image classification, object detection, and image segmentation. Due to their ability to handle large-scale image data effectively, CNNs have become the backbone of numerous remotely sensed image applications and have significantly advanced the field of spectral image processing. The CNN architecture, comprising convolutional layers, activation functions (such as ReLU), pooling layers for dimensionality reduction, and fully connected layers, has revolutionized image processing tasks. Convolutional layers apply filters to capture local patterns in the input data, while activation functions introduce nonlinearity for better learning. Pooling layers reduce data size, preventing overfitting. Fully connected layers transmit features to the classifier for making predictions. The backpropagation algorithm optimizes the network’s parameters during training, leading to improved performance and accurate image processing results. CNN-based HSI classification methods encompass 2D and 3D CNNs. For instance, Fast and Compact 3D CNN utilize iPCA and 3D-2D CNNs for feature reduction and classification [41]. SpectralNET leverages factor analysis for preprocessing to reduce data dimension and a wavelet-based architecture for spatial-spectral feature extraction [39]. Similarly, HybridSN incorporates PCA for feature reduction, three 3D CNNs, and a 2D CNN for classification [38]. These represent the foundational, cutting-edge CNN models for HSI classification. However, the field has seen recent advancements in the form of fusion or hybrid models that build upon these aforementioned CNN-based frameworks. One such example is the Hybrid-2DNET [43], which introduces novel modifications to the SpectralNet architecture. By incorporating a multi-resolution mutual information-based feature selection process subsequent to the factor analysis stage, this model enhances classification accuracy. Similarly, there is the triple-path spectral-spatial network, TP-Net [44], which employs an innovative approach to elevate classification performance. TP-Net integrates two distinct interleaved attention mechanisms into its architecture. This strategic incorporation of attention mechanisms contributes to heightened discriminative capabilities, ultimately resulting in improved classification accuracy. These approaches address the challenges of limited training samples and high dimensionality in HSI data, offering effective solutions for accurate HSI classification.

3.1 Overview

The proposed approach for land cover object identification in HSIs, as illustrated in Fig 1, combines Seg-PCA with mRMR for feature reduction and employs a hybrid 3D-2D CNN architecture for effective spectral-spatial feature extraction. Initially, Seg-PCA is applied to reduce the dimensionality of the hyperspectral data and preserve critical spectral characteristics. Following this, mRMR is used to select the most informative features from the reduced data. The selected features are then processed through a 3D-2D CNN, which captures both spectral and spatial features for improved land cover classification. This network architecture integrates 3D convolutions to extract spectral features and 2D convolutions for spatial features, culminating in a classification layer with dense connections and dropout for robustness. By leveraging this combination of advanced techniques, the proposed method aims to significantly enhance land cover object classification performance in hyperspectral imagery.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Proposed spectral-spatial feature learning approach for HSI classification.

https://doi.org/10.1371/journal.pone.0313473.g001

3.2 Dataset description

In our experiments, we used three publicly available HSI datasets: Indian Pines (IP), University of Pavia (PU), and Salinas Scene (SA). The IP dataset contains images with a spatial dimension of 145 × 145 pixels and 224 spectral bands in the wavelength range of 400 to 2500 nm. We excluded 24 spectral bands corresponding to water absorption regions. The ground truth of the IP dataset provides labels for 16 land cover vegetation object classes [61]. The PU dataset consists of images with a spatial dimension of 610 × 340 pixels and 103 spectral bands in the wavelength range of 430 to 860 nm. The ground truth of the PU dataset classifies nine urban land cover object classes [62]. The SA dataset includes images with a spatial dimension of 512 × 217 pixels and 224 spectral bands in the wavelength range of 360 to 2500 nm. We removed 20 spectral bands affected by water absorption. The SA dataset consists of 16 different land cover object classes [63]. We selected these datasets for their diversity in spatial dimensions, spectral bands, and land-cover classes. This diversity enables us to thoroughly evaluate the performance and generalization capability of our proposed method across various HSI scenarios.

3.3 Seg-PCA-mRMR

While PCA is effective, it may overlook subtle information valuable for accurate HSI data analysis, particularly when the original HSI bands exhibit high correlation, resulting in a large covariance matrix and higher computational costs. To address this limitation, Seg-PCA has been proposed [48, 50] and presented [21, 49, 64, 65]. Seg-PCA is specifically designed for handling highly correlated blocks with low inter-correlations.

In Seg-PCA implementation, the HSI data matrix is divided into L subgroup datasets (L = 3 in this case) based on band correlations. Within each subgroup, strongly correlated bands are grouped together to capture local characteristics effectively. The sub-grouped data matrices are denoted as D_t, where t ∈ [1, 2, 3], and each D_t contains n_t consecutive bands. Eigendecomposition is applied to each D_t to compute covariance matrices. The overall projection matrix is obtained by sequentially combining the projection matrices of each subgroup’s D_t.

After applying Seg-PCA and feature extraction to the HSI dataset, we employ the mRMR feature selection process to enhance the feature set by identifying the most informative features. In the mRMR steps, we select the top b ranked PCs from the overall projected data, similar to our previous research with traditional machine learning [15]. The Seg-PCA-mRMR technique specifically reduces the number of spectral bands from F to Q, representing the extracted and selected PC dimensions for the reduced data. However, the spatial dimensions of the data matrix remain unchanged, with S = X × Y, where D = X × Y × F representing the overall input data cube. The modified input data after Seg-PCA-mRMR is represented by Z_Q = X × Y × Q, where Z_Q corresponds to the modified input data cube after applying Seg-PCA-mRMR. This additional step offers several advantages, such as improved computational efficiency, enhanced classification accuracy, and a deeper understanding of the underlying scientific insights present in the data. The Seg-PCA+mRMR algorithm (Algorithm 1 and 2) is employed as a preprocessing step in the proposed methodology for dimensionality reduction and feature selection. It divides the original hyperspectral data cube D into L subgroup datasets using Seg-PCA and selects the most informative principal components using mRMR.

Algorithm 1 Seg-PCA followed by mRMR Feature Selection

Require: HSI data cube D with dimensions X × Y × F

1: Divide D into L subgroup datasets based on band correlations

2: for t = 1 to L do

3:  Compute covariance matrix C_t for D_t

4:  Perform eigendecomposition on C_t to obtain eigenvalues and eigenvectors

5:  Select top b eigenvectors from C_t

6:  Concatenate selected eigenvectors from each subgroup

7: end for

8:  Apply mRMR feature selection on the concatenated eigenvectors (Algorithm 2)

9:  return Reduced data Z_Q with dimensions X × Y × Q

Algorithm 2 mRMR Feature Selection

Require: Concatenated eigenvectors Z from Seg-PCA

1: Calculate mutual information between each feature and class labels

2: Calculate redundancy and relevance scores for each feature

3: Rank features based on mRMR score (relevance—redundancy)

4: Select top b features with highest mRMR scores

5: return Selected features

3.4 Neighbourhood extraction process

To execute our 3D-2D CNN model, the HSI data cube is divided into small overlapping 3D patches. These patches, denoted as P_Q = s × s × Q, are created from the Seg-PCA-mRMR reduced data cube Z_Q and are centered at spatial location (i, j). Each patch covers a spatial extent of s × s and contains all Q PCs representing the selected spectral features. The total number of generated 3D patches, denoted as n, from Z_Q is determined by (X − s + 1) × (Y − s + 1), where X and Y represent the spatial dimensions of the data cube. As a result, the 3D patch located at (i, j), denoted as PQ, i, j, includes the width from i − (s − 1)/2 to i + (s − 1)/2, height from j − (s − 1)/2 to j + (s − 1)/2, and contains Q selected features of the Seg-PCA-mRMR reduced data cube Z_Q.

3.5 Spectral-spatial feature extraction with 3D-2D CNN

In our proposed spectral-spatial 3D-2D CNN architecture, we utilize three 3D convolution layers, one 2D convolution layer, and two fully connected layers. The 3D convolution operation involves convolving a 3D kernel with the input data to generate feature maps in the convolution layer. This process captures the spectral information present in multiple contiguous bands of the input layer. In the 3D convolution equation, the activation value at a spatial position (i, j, k) in the f-th feature map of the l-th layer, denoted as x^{i, j, k}l, f, is computed by applying the activation function ϕ to the sum of the bias parameter bl, f and the weighted sum of the input values from the previous layer using the kernel weights w_{l,f, α}. The kernel has a width of 2ξ + 1, a height of 2δ + 1, and a depth of 2η + 1 along with the spectral dimension, where ξ, δ, and η determine the kernel size. The weight parameter w_{l,f, α} represents the weight value for the f-th feature map of the l-th layer and the α-th spectral band. The computation of the activation function is carried out as follows: (4)

In the case of 2D-CNN, the input data is convolved with 2D kernels. The convolution operation involves computing the sum of the dot product between the input data and the kernel, which is strived over the input data to cover the full spatial dimension. The resulting convolved features are then passed through the activation function to introduce nonlinearity in the model. In the equation for 2D convolution, the activation value at spatial position (i, j) in the f-th feature map of the l-th layer, denoted as x^{i, j}l, f, is computed by applying the activation function ϕ to the sum of the bias parameter bl, f and the weighted sum of the input values from the previous layer using the kernel weights w_{l,f, α} as follows: (5)

The kernel has a width of 2ξ + 1 and a height of 2δ + 1, and it does not include the spectral dimension as in the 3D convolution equation.

Algorithm 3 3D-2D CNN for Spectral-spatial Feature Extraction

Require: Reduced data cube Z_Q with dimensions X × Y × Q

1: Divide Z_Q into overlapping 3D patches P_Q of size s × s × Q

2: Initialize CNN architecture with specified parameters

3: Perform 3D convolution with kernel size 3 × 3 × 5 and 8 kernels

4: Perform 3D convolution with kernel size 3 × 3 × 3 and 16 kernels

5: Perform 3D convolution with kernel size 3 × 3 × 3 and 32 kernels

6: Perform 2D convolution with kernel size 3 × 3 and 64 kernels

7: Apply ReLU activation function after each convolution operation

8: Flatten the output tensor to prepare for fully connected layers

9: Add two fully connected layers with ReLU activation functions.

10: Output softmax probabilities for classification

11: return Classification accuracy

The 3D-2D CNN algorithm (Algorithm 3) is utilized in the proposed spectral-spatial feature extraction process, where it plays a crucial role in extracting discriminative features from the reduced data cube Z_Q. These features are then utilized for subsequent classification tasks.

The parameters of the CNN, including bias and kernel weights, are trained using supervised approaches with gradient descent optimization. Conventional 2D CNNs operate on spatial dimensions only, limiting their ability to capture spectral information. On the other hand, 3D CNNs can extract both spatial and spectral features simultaneously, but at the cost of higher computational complexity. To leverage the feature learning capabilities of both 3D and 2D CNNs, our proposed spectral-spatial model combines them to extract different features and then fuses them for a more discriminative representation. The flattening layer is applied to preserve spatial information while reducing spectral dimensionality.

3.6 The model’s architecture and parameter details

The model’s architecture and parameter details are summarized in Table 1. The proposed model consists of convolutional layers with different kernel sizes and numbers of kernels, contributing to the extraction of different spatial and spectral features from the HSI data and enhancing the model’s discriminative power. The first 3D CNN has a kernel size of 3 × 3 × 5 and is composed of 8 kernels. The second 3D CNN has a kernel size of 3 × 3 × 3 and contains 16 kernels. The third 3D CNN has the same kernel size of 3 × 3 × 3 but contains 32 kernels. Finally, the 2D CNN has a kernel size of 3 × 3 and includes 64 kernels. The total trainable weight parameters for the proposed model is 4,824,816 for the Indian Pines (IP) dataset when Q = 10. These weights are randomly initialized and trained using the back-propagation algorithm with the Adam optimizer. The network was trained with a learning rate of 0.001, a decay rate of 0.000001, and mini-batch sizes of 256 over 100 epochs, without using batch normalization or data augmentation. The ReLU activation function is applied throughout the network, except in the output layer, as it efficiently captures non-linear relationships and aids in learning complex data representations. The output layer employs softmax activation for multi-class classification, ensuring the predicted probabilities for all classes sum to 1, which allows for accurate predictions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Layer-wise parameter details for the proposed model architecture with 25 × 25 window size based on IP dataset when Q = 10.

https://doi.org/10.1371/journal.pone.0313473.t001

4.1 Experiment design and parameter setup

We divided the labeled samples into training and test subsets using different ratios for each dataset. The ratios used were 5%-95%, 10%-90%, 15%-85%, 20%-80% and 25%-75%. For example, in the case of a 25%-75% ratio, 25% of the data were randomly assigned to the training group, and the remaining 75% was assigned to the testing group. During the CNN training process, 90% of the training samples were used to determine the weights and biases of each neuron, and the remaining 10% were utilized to determine overfitting and inform network design choices.

In our proposed model, we first apply Seg-PCA-mRMR for spectral feature reduction by segmenting the HSI datasets into three segments based on average correlation analysis in the segmented subgroups (Table 2). After Seg-PCA, we scale the data using the min-max scaler from the Python sklearn preprocessing library.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Details of datasets segmentation for Seg-PCA.

https://doi.org/10.1371/journal.pone.0313473.t002

We extracted 3D patches from the input volume with a spatial dimension of 25 × 25 and Q spectral features to ensure a fair comparison, where Q represents the number of input spectral features. In our experiments, we applied Seg-PCA-mRMR for spectral feature reduction and selected the top-ranked 10 features based on mRMR. The order of the selected features for different datasets is shown in Table 3. For the model training, we set the learning rate to 0.001, the decay rate to 0.000001, the batch size to 256, the window size to 25 × 25 and epochs to 100, based on the classification performance. The model were trained with Adam optimizer and categorical cross-entropy loss function.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Order of selected features by mRMR for different dataset.

https://doi.org/10.1371/journal.pone.0313473.t003

To assess classification performance, we employed four standard quantitative metrics: Overall Accuracy (OA), Average Accuracy (AA), Kappa Coefficient, and F1 score. OA represents the proportion of correctly classified samples out of the total test samples, while AA calculates the mean accuracy across all classes. The Kappa Coefficient measures agreement between predicted and actual classifications, and the F1 score, as the harmonic mean of precision and recall, provides a balanced indication of a model’s reliability by giving equal weight to both metrics. Additionally, we monitored the training loss and training accuracy during the training process. Finally, we produced classification maps for further visualization of the results and to validate the effectiveness of the proposed method.

4.2 Classification results and analysis

The classification results were evaluated using three metrics: OA, AA, and Cohen’s Kappa, as described in the experimental design. The accuracies were measured in percentages for both the proposed method and state-of-the-art methods, including SVM on original data, Segmented PCA + SVM, 2D-CNN, Fast 3D-CNN, SpectralNet, HybridSN, Hybrid-2DNet, TP-Net, S2PNet, and DS2FE. These state-of-the-art experiments are described as follows:

• 2D-CNN [25]: Applies randomized PCA for dimensionality reduction. It consists of two convolutional layers with filter counts of 30 and 90, respectively, followed by two fully connected layers with dropout rates of 25% and 50%, and an output layer.
• Fast 3D-CNN [41]: Utilizes incremental PCA for dimensionality reduction. The network comprises four convolutional layers with increasing filter counts and three fully connected layers.
• SpectralNet [39]: Applies factor analysis for dimensionality reduction and then executes a three-level Wavelet CNN decomposition.
• HybridSN [38]: Employs PCA for dimensionality reduction. It consists of three 3D convolution layers, one 2D convolution layer, and three fully connected layers.
• Hybrid-2DNet [43]: Utilizes factor analysis and mRMR-based feature selection followed by 2D-wavelet CNN in a four-level decomposition for reducing spectral and spatial dimensionalities. Additionally, batch normalization is applied.
• TP-Net [44]: Proposes a triple-path CNN with an attention mechanism to capture joint features.
• S2PNet [46]: An interactive learning approach used a multi-stage spectral purification module to reduce noise and spectral heterogeneity, while a global-local mutual guide module improved spatial feature discrimination.
• DS2FE [45]: Employs a vision-based transformer module that includes a multiscale feature extractor to capture joint spectral-spatial low-level and shallow features. For high-level semantic feature extraction, it uses a regional attention mechanism combined with a spatially gated module to enhance feature discrimination.

Table 4 displays the classification outcomes for ten spectral features across five different training-testing separations. The OA of the best-performing methods under various conditions is denoted in bold. The results indicate that our proposed method consistently outperforms other approaches in most scenarios.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Classification results for the proposed and state-of-the-art methods.

https://doi.org/10.1371/journal.pone.0313473.t004

Among the three datasets, the PU dataset exhibits the lowest number of categories, comprising only nine classes, thereby making it comparatively easier to classify than the other datasets. In contrast, the SA dataset features a larger spatial size and the maximum number of spectral bands, providing more discriminative information for classification. Consequently, the listed methods generally demonstrate higher classification performance on the SA dataset. The IP dataset, although having a relatively small spatial size, encompasses 16 categories, presenting a challenge for accurate classification and resulting in lower accuracy compared to the other datasets. The traditional SVM model applied directly to the original data yields inadequate classification performance. However, upon preprocessing the data using Seg-PCA and integrating it with SVM, there is a notable enhancement in classification accuracy.

The 2D-CNN and Fast 3D-CNN methods demonstrate lower classification performances, possibly attributable to their limited utilization of spectral information. Conversely, SpectralNet achieves moderate classification performances by incorporating both spectral and spatial features. HybridSN, which combines several 3D CNNs with a 2D CNN, is a simple and effective approach, yielding discriminative features. The Hybrid-2DNet method demonstrates a notable performance improvement over SpectralNet, highlighting the crucial role of feature selection. However, it does not consistently surpass HybridSN across all scenarios. In contrast, TP-Net consistently outperforms both SpectralNet and HybridSN in most cases, indicating its superior performance. S2PNet performs comparably to TP-Net, while DS2FE surpasses TP-Net in most instances due to its ability to handle joint spectral-spatial low-level and shallow features, along with high-level semantic feature extraction using a regional attention mechanism.

Indeed, the proposed approach capitalizes on the combined potential of spatial and spectral features within HSI data. Through the extraction of distinct image features from segmented subgroups’ PCs and their subsequent fusion, the network acquires a rich array of representative and differentiating features. This comprehensive feature set contributes to an enhanced classification performance overall. By leveraging both spatial and spectral information, and harnessing the benefits of feature fusion, the proposed method achieves an advanced level of classification accuracy.

Additionally, the class-wise classification accuracies in percentages are shown in Tables 5–7 with top 10 selected features and 20% training samples for all three experimented datasets. The results in Tables 4–7 demonstrate that the proposed methods offer the best OA of 99.61%, 100%, and 99.94% for IP, SA, and PU datasets, respectively. In comparison, the DS2FE, S2PNet, TP-Net, Hybrid-2DNet, HybridSN, and SpectralNet models achieve their optimal results with OAs of 98.97%, 98.92%, 98.71%, 98.58%, 99.49% and 98.51% for IP, 99.98%, 99.95%, 99.98%, 99.99%, 99.98% and 99.98% for SA, and 99.91%, 99.85%, 99.85%, 99.78%, 99.75% and 99.70% for PU datasets under the same spectral features conditions. These results indicate that the proposed method significantly improves the classification accuracy compared to the state-of-the-art models under the same spectral features and training sample conditions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Class-wise classification accuracies (%) of IP with 10 spectral features and 20% training samples.

https://doi.org/10.1371/journal.pone.0313473.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Class-wise classification accuracies (%) of SA with 10 spectral features and 20% training samples.

https://doi.org/10.1371/journal.pone.0313473.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Class-wise classification accuracies (%) of PU with 10 spectral features and 20% training samples.

https://doi.org/10.1371/journal.pone.0313473.t007

Fig 2 shows the overall accuracy performances of state-of-the-art methods with the proposed method as the number of training samples increases for different datasets. It is evident from the results in the tables and Fig 2 that the proposed methods consistently exhibit significantly better performance compared to SVM on original data and Seg-PCA + SVM methods for all three datasets. Similarly, our proposed methods clearly outperform the 2D-CNN, Fast 3D-CNN, and SpectralNet methods across all three datasets. Furthermore, in comparison to HybridSN, Hybrid-2DNet, DS2FE, S2PNet, and TP-Net the proposed methods demonstrate notable improvements. Overall, the results consistently indicate that the proposed methods outperform or surpass the performance of several benchmark methods across all three datasets. Moreover, as the number of training samples increases, all models show improved performance, except for when using 25% training samples, which sometimes exhibits lower accuracy compared to the performance with 20% training samples. This observation suggests that a larger training dataset allows the models to learn more representative and discriminative features, thus enhancing classification accuracy.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overall accuracy comparison of state-of-the-art methods with the proposed method as the number of training samples increases for (a) IP, (b) SA, and (c) PU datasets.

https://doi.org/10.1371/journal.pone.0313473.g002

Fig 3 illustrates the convergence of accuracy and loss for the proposed and state-of-the-art models on the IP dataset, using 20% of training samples over 100 epochs. Our method achieves convergence around the 18th epoch, demonstrating a relatively fast convergence rate compared to other models, except for S2PNet and DS2FE, which converge faster. This highlights the efficiency and effectiveness of our approach in capturing data patterns, resulting in improved classification performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3.

IP data, (a)-(i) Accuracy, and Loss convergence versus the number of epochs over top 10 selected features and 20% training samples, (a) 2D CNN, (b) Fast 3D-CNN, (c) SpectralNet, (d) HybridSN, (e) Hybrid-2DNet, (f) TP-Net, (g) S2PNet, (h) DS2FE and (i) Proposed Method.

https://doi.org/10.1371/journal.pone.0313473.g003

4.3 Classification maps

In addition to the quantitative evaluation, a visual assessment of classification maps is performed to validate the effectiveness of the proposed method. Figs 4–6 showcase the classification maps obtained by different methods for various HSIs. These maps are generated using 20% of the training samples for both the proposed and the other studied methods. Upon visual inspection, it becomes evident that the proposed method consistently outperforms the other approaches across all datasets. The classification maps produced by the proposed method exhibit higher clarity and fewer misclassified pixels, demonstrating its superior performance in capturing the complex spatial and spectral information present in hyperspectral imagery. The visually compelling results further validate the efficacy of our proposed method and reinforce its potential for accurate HSI classification tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

IP HSI (a) ground truth image, (b)-(h) classification maps for different SVM and CNN models with top 10 selected features and 20% training sample. (b) Original data + SVM, (c) Seg-PCA + SVM, (d) 2D CNN, (e) Fast 3D-CNN, (f) SpectralNet, (g) HybridSN, (h) Hybrid-2DNet, (i)TP-Net, (j) S2PNet, (k) DS2FE, and (l) Proposed Method.

https://doi.org/10.1371/journal.pone.0313473.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5.

SA HSI (a) ground truth image, (b)-(h) classification maps for different SVM and CNN models with top 10 selected features and 20% training sample. (b) Original data + SVM, (c) Seg-PCA + SVM, (d) 2D CNN, (e) Fast 3D-CNN, (f) SpectralNet, (g) HybridSN, (h) Hybrid-2DNet, (i)TP-Net, (j) S2PNet, (k) DS2FE, and (l) Proposed Method.

https://doi.org/10.1371/journal.pone.0313473.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6.

PU HSI (a) ground truth image, (b)-(h) classification maps for different SVM and CNN models with top 10 selected features and 20% training sample. (b) Original data + SVM, (c) Seg-PCA + SVM, (d) 2D CNN, (e) Fast 3D-CNN, (f) SpectralNet, (g) HybridSN, (h) Hybrid-2DNet, (i)TP-Net, (j) S2PNet, (k) DS2FE, and (l) Proposed Method.

https://doi.org/10.1371/journal.pone.0313473.g006

4.4 Ablation analysis

A comprehensive ablation analysis was conducted using the novel integrated approach (Seg-PCA + mRMR + 3D-2D CNN) across five distinct subgroups. The first experiment evaluated dataset segmentation before PCA, comparing the performance of conventional PCA (PCA + 3D-2D CNN) with Seg-PCA (Seg-PCA + 3D-2D CNN), without mRMR-based feature selection. The second experiment assessed the impact of mRMR-based feature selection on both conventional PCA and Seg-PCA. Compared to group 1, the efficacy of mRMR-based feature selection is clearly demonstrated, showing its impact on improving the model’s performance. The third experiment compared the deep 3D-2D model’s performance with traditional machine learning (SVM). In the fourth, the individual performance of 2D and 3D CNNs was evaluated, and the fifth experiment examined the influence of the number of 3D CNN layers in the 3D-2D CNN model.

The ablation analysis results, detailed in Table 8, demonstrate that both Seg-PCA and mRMR feature selection significantly improved classification accuracy compared to classical PCA and the absence of feature selection. The findings also determine the optimal number of 3D CNN layers and the advantage of deep 3D-2D CNN over traditional machine learning. Notably, combining Seg-PCA and mRMR produced a synergistic effect, greatly enhancing overall classification performance, underscoring the importance of these techniques both individually and together in the integrated framework.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Classification results for ablation analysis with 10 spectral features.

https://doi.org/10.1371/journal.pone.0313473.t008

4.5 Computation time analysis

All experiments were performed on Google Colaboratory using a setup consisting of 2 x vCPUs and a maximum of 25 (mostly 12) GB RAM. In order to ensure a fair and accurate comparison, we applied the same spatial dimension and spectral features extraction technique to obtain 3D patches from various input data. Throughout the experiments, we measured three types of computation times for both the different studies and our proposed method. These computation times are denoted as Pp for preprocessing, Tr for training, and Ts for testing. The results of these time measurements are presented in Table 9, with the time values given in seconds, minutes, and seconds, respectively. It is worth noting that our proposed model demonstrated higher computational efficiency compared to the fast 3D-CNN, SpectralNet, HybridSN, Hybrid-2DNet, TP-Net, DS2FE, and S2PNet, methods when considering the preprocessing, training, and testing times. However, fusion methods DS2FE, S2PNet, Hybrid-2DNet and TP-Net show better classification performances than SpectralNet, and HybridSN but face more computational complexity. Only 2D-CNN offers better computational efficiency than the proposed methods but gives lower classification accuracy.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Different stages computation time, prepossessing (Pp) time in seconds (s), training (Tr) time in minutes (m), testing (Ts) in seconds (s), for the proposed and state-of-the-art methods.

https://doi.org/10.1371/journal.pone.0313473.t009

4.6 Discussion

The experimental results highlight the challenges of limited training samples and high dimensionality in HSI classification, common in real-world applications where acquiring labeled data is difficult and resource-intensive. Despite these challenges, the proposed method effectively overcomes them by combining segmented principal component analysis (Seg-PCA) for feature extraction with 3D-2D CNNs for joint spectral-spatial feature learning. This hybrid approach addresses both the scalability and robustness issues, showing improved performance compared to traditional CNN-based methods. In addressing the limitations of traditional approaches, the integration of Seg-PCA with mRMR-based feature selection enhances the model’s ability to handle high-dimensional HSI data by reducing redundant information and focusing on key features. The incorporation of both 3D and 2D CNN layers balances computational efficiency with feature extraction depth, allowing the model to adapt effectively to variations in dataset characteristics. As a result, the proposed method achieves better classification accuracy and generalization across different datasets, while also mitigating the computational complexity of deep 3D CNNs.

This research demonstrating the model’s architecture remains efficient and adaptable as data complexity increases. Additionally, the robustness of the proposed method is evident in its generalizability across three widely used HSI datasets, outperforming state-of-the-art methods such as SVM, 2D-CNN, Fast 3D-CNN, SpectralNet, HybridSN, Hybrid-2DNet, TP-Net, 2PNet, and DS2FE. The results in tables and figures as well as classification maps indicate that the combination of Seg-PCA and 3D-2D CNNs significantly contributes to overcoming the limitations posed by traditional CNNs, particularly in handling high-dimensional data with limited training samples. In respect of computation time the proposed model offer relatively higher computational efficiency compared to the state of the art.

While our proposed method demonstrates significant improvements in HSI classification, it does have some limitations that offer avenues for future research. Despite the integration of both 3D and 2D CNN layers to balance computational efficiency, the overall computational cost remains relatively high, particularly when applied to large-scale datasets. The complexity could limit the scalability of the model for such real-time applications where large-scale datasets involve. Although the model generalizes well across the evaluated datasets, further validation on additional, more diverse datasets is necessary to confirm its adaptability to varying environments and spectral conditions. However, expanding the application of the proposed model to diverse fields such as precision agriculture, environmental monitoring, and urban mapping could validate its robustness and scalability across various real-world scenarios. Additionally, the model could be tailored for specific domain requirements, further improving its performance in specialized tasks.