1. Motivation

Various deep learning methods, such as the DNN, CNN, LSTM, and RNN, are incorporated into an NIDS. This approach involves neural networks with significant depth for optimal functionality [15]. However, it does not utilize complete domain understanding of the network traffic. Inspired by these research works based on the deep learning approach, we presented our hybrid model called MCL-FWA-BILSTM to tackle the imbalance class issue utilizing the two techniques of the BI-LSTM-based semantic base feature weights and the self-attention-based feature weights. Both techniques enhanced domain knowledge acquisition and mitigated the effects of class imbalance. Class imbalance in intrusion detection is not just a statistical challenge rather it reflects the asymmetric nature of cybersecurity threats where legitimate network activities vastly outnumber malicious ones. In Section 2, a scholarly analysis of the various intrusion detection system using machine learning and deep learning techniques in the literature is presented.

2. Challenging issues

Deep learning techniques enabled the network intrusion detection systems (NIDS) to enhance significantly, producing formidable methods of identifying and mitigating cyber threats. However, despite that significant progress, the NIDSs still have a big problem in detecting some attacks, such as low-traffic attacks, mainly because of the issues of class imbalance occurring inside data sets [16]. In actual network attack conditions, most of the incoming and outgoing network data traffic is a minority part of it. This imbalance causes a significant learning environment distortion for deep learning models, which, in turn, results in much better performance on majority classes than accurately identifying the minority attack traffic. Such an imbalance has dire consequences: the rate of false positives is increased, thus affecting the effectiveness of NIDS, which also drops because of the many undetected cases. The core of the issue lies in the nature of modern NIDS datasets, which aim to mirror real-world traffic patterns. Realism is critical to enhancing the effective development of NIDS, but at the same time, it also presents one major challenge related to traffic under-representation. For example, deep learning models trained on such data may need to learn more about the characteristics of these less frequent attacks, due to which there is a possibility for them to be omitted or misclassified. This is not a technical problem only; instead, it bears critical vulnerabilities that attackers might use. Hence, the complete detection of any form of attack traffic becomes a paramount object for genuinely effective NIDS. Despite such a crucial need, there needs to be more literature and practice on how to address class imbalance in NIDS. Prior art to improve the performance of NIDS was either naive to the subtleties of class imbalance or needed to provide solutions that curb the problem to a great level. This is actually quite an overlooking surprise, given that an ideal intrusion detection system should be very good at recognizing every form of attack traffic, regardless of its frequency within the dataset. If, in a nutshell, deep learning is the fuel of advancement for NIDS, then class imbalance issues pose a curse. It hampers their potential to detect low-traffic attacks and affects their overall performance. Hence, the most crucial aspect for solving this problem in the next wave of NIDS development is an enhanced effort to refine the deep learning approaches or innovate new strategies that can handle imbalanced data well. Ultimately, the capability sought is one of balanced detection that has the highest possible accuracy along with the lowest possible rate of false alarms across the full range of cyber threat coverage, thereby ensuring the most comprehensive protection of networks.

3. Contribution

A novel hybrid deep learning model MCL-FWA-BILSTM is proposed. This model is unique due to its integration of multiple techniques: MCL (Mean Convolutional Layers), CNN (Convolutional Neural Network), BI-LSTM (Bidirectional Long Short-Term Memory), and self-attention mechanisms. The combination of these techniques is novel, particularly in the context of network IDS. Further, the proposed model aims to classify the intrusion into binary and multiclass attack categories. These attack categories are mentioned in Tables 2 and 3. The detailed contribution is described below:

• Attack Classification and Feature Extraction: The model classifies attacks into specific categories using a combination of CNN-MCL for average convolutional processing and BI-LSTM layers. Self-attention-based feature weighting is employed to enhance feature extraction, which is crucial for dealing with imbalanced datasets.
• Optimal Random Forest for Imbalanced Datasets: A significant innovation is the use of an optimal random forest algorithm that adjusts the weights of decision trees, giving more importance to minority classes. This approach addresses the class imbalance problem, a common challenge in network security datasets.
• BI-LSTM for Error Reduction and Feature Integration: The BI-LSTM network minimizes errors and aids in feature integration. It also helps in extracting semantic features, improving the model’s performance on imbalanced datasets.
• Attention Mechanism for Detailed Feature Extraction: The attention mechanism in the BI-LSTM model identifies key features in packet sequence data, enhancing anomaly detection capabilities.
• Performance Evaluation: The model’s performance was evaluated using two benchmark datasets (UNSW-NB15 and NSL-KDD), which are standard in IDS research. Our approach outperformed baseline models and previous research efforts, indicating its effectiveness in real-world scenarios

4. Novelty

The novelty lies in the hybrid approach combining deep learning with an ensemble technique for feature selection. This approach is particularly effective for handling imbalanced datasets and improving predictions for minority classes. The proposed model comprises three key components, starting with the Convolutional Neural Network (CNN), which is responsible for the non-linear mapping of features. This step is crucial as it enhances the representation of the data by capturing higher-level features through the network’s layers. Following the CNN, the Bidirectional Long Short-Term Memory (BI-LSTM) network works to improve feature representation by addressing the issue of overlapping features. The BI-LSTM processes data in both forward and reverse directions, ensuring that all temporal dependencies are captured, thus reducing redundancy in feature representation. These refined features are then optimized by an attention layer, which focuses on the most relevant features for the classification task by adjusting hyperparameters. The attention mechanism selectively emphasizes important features and diminishes the less important ones, which can lead to a reduction in noise within the feature set. Finally, the Random Forest classifier learns from the enhanced features. It is an ensemble method that can handle class imbalance by constructing multiple decision trees and aggregating their predictions, which generally leads to improved accuracy and robustness against overfitting. In summary, the model is designed to improve feature quality, reduce noise, and handle class imbalance, with the ultimate goal of enhancing accuracy in the classification of the dataset. In the present study, we assessed a novel technique on two widely-used benchmark datasets (UNSW-NB15 and NSL-KDD) and compared it with both baseline models and with previous research endeavours in this field. The rationale behind selecting these particular datasets was their prevalence as IDS datasets, encompassing contemporary network attacks that meet real-world attack criteria. Our hybrid approach yielded better results than most of the other approaches considered, according to our experimental findings. Our experiments confirmed that the proposed approach outperformed most other approaches available in the open literature.

The organization of this research article is as follows: The second section provides materials and methods. It contains related work of IDS’s existing algorithms and current research gaps. It describes in detail the suggested deep learning based hybrid model methodology in depth that contains (i) workflow of the proposed architecture and the overview; (ii) details of the Mean Convolutional Layer and the related algorithm, (iii) the description of Multiple Convolutional Layers, Pooling, and bidirectional-LSTM (BI-LSTM), (iv) the attention layer description and the bidirectional-attention (BI-ATT) algorithm details, (v) a description of the machine learning classifier used in the study (namely, the random forest (RF) classifier), and (vi) a criterion for evaluating the model’s performance. In Section three, experimental setup, dataset employed in the present work, the experimental results and their explanations are presented. In the Section 4, discussions are given, and proposed model performance is compared with the previous research work. Finally, the present study is concluded in Section 5.

A. Related work

The present section reviews the most interesting and illustrative IDS research conducted in the last few years, especially on deep learning and machine learning. The application of deep learning technology in network intrusion detection systems (NIDS) has gained significant recognition owing to its exceptional ability to manage intricate and extensive datasets, as well as to extract the inherent features of traffic data. Consequently, such an application presents itself as a viable approach towards identifying security breaches. In recent years, many methods based on deep learning have been used to solve the intrusion classification problem. Following are few recent research works done in the field of ML and DL techniques and finally we present a summarized Table 1, in which we talk briefly about the DL-based approaches that have been developed recently and are available in the open literature.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Related studies on IDS using deep learning.

https://doi.org/10.1371/journal.pone.0302294.t001

In their research, Dutta et al. [25] present a hybrid network anomaly detection system consisting of two stages that utilize a Classical Autoencoder (CAE) and a deep neural network (DNN) for feature engineering and classification. The performance of the proposed model is evaluated on the UNSW-NB-15 dataset, resulting in an accuracy rate of 91.29%. Despite the significant advancements made to enhance NIDSs’ predictive abilities, recent studies have shed light on packet sampling techniques’ influence on NIDS models [26]. These investigations revealed that even minute sampling rates such as 1/100 and 1/1000 can substantially reduce Machine Learning (ML)-based NIDS systems’ performance. Aljbali et al. [27] presented a technique for detecting anomalies based on a bidirectional short-term memory (Bi-LSTM) algorithm. Experimenting with the UNSWNB15 dataset demonstrated that this approach outperformed other deep learning and ML models in terms of precision, recall, F1 score, and accuracy. The authors in [28] present a DL wireless intrusion detection (WIDS) approach that utilizes a feed-forward deep neural network (FFDNN). The FFDNN-WIDS scheme is equipped with an Extra Trees wrapper-based feature extraction module to produce an optimal input subset for the classifier. This study evaluated the performance of the FFDNN on the UNSW-NB15 dataset using binary and multiclass classification problems. Results revealed that the FFDNN network achieved high test accuracies of 87.10% (2-way) and 77.16% (10-way) for the UNSW-NB15 dataset. Tang et al. [29] introduced a deep stacking network (DSN) model that integrated the outputs of multiple classifiers to enhance the accuracy of intrusion detection systems. These authors claimed that a fusion approach with four classifiers improved the classification performance, yielding an overall accuracy rate of 86.8%. To achieve the best feature selection for NSLK-KDD, the authors in ref. [15] investigated the integration of the Long Short-Term Memory (LSTM) and the Genetic Algorithm (GA) approaches. The experimental outcomes reveal that their devised approach obtained a commendable accuracy level of 93.88%. Notably, no information was provided on the detection rate in this investigation.

B. Gaps in the previous research

This section outlines the several research gaps identified in the literature review. Earlier studies utilized dummy datasets for investigating class imbalance, but this increased the false information and, in real conditions, increased the error in the imbalanced class. In the literature, some studies on feature selection have decreased the amount of information and increased the number of false positives. Several studies have utilized efficient feature mapping but still, they need to map the backward and forward planes of features.

C. Proposed methodology

This study aimed to see how well the attention-based hybrid model works with deep learning techniques, specifically MCL, convolutional neural networks, and bidirectional long short-term memory networks, to classify network intrusions more accurately than with traditional techniques and classification using Random Forests. The NSL-KDD and UNSW-NB15 benchmark dataset served as a platform for experimentation. The workflow of the proposed architecture with hyperparameters to classify intrusions is shown in Fig 1 while the overall architecture of the hybrid MCL-FWA-BILSTM intrusion detection model is shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. A diagram showing the workflow of the proposed architecture with hyperparameters to classify intrusions.

https://doi.org/10.1371/journal.pone.0302294.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. A diagram showing the workflow of the proposed novel hybrid- MCL-FWA-BILSTM based architecture to classify intrusions.

https://doi.org/10.1371/journal.pone.0302294.g002

D. Brief description of workflow of the proposed architecture

The Fig 1 depicts a flowchart of a machine learning pipeline designed for classification tasks, likely in the context of an Intrusion Detection System (IDS) given the label "Input IDS dataset". The process begins with the input IDS dataset, which then undergoes preprocessing to prepare the data for feature mapping. Feature mapping appears to be handled by two neural network architectures: Convolutional Neural Network (CNN) and a Bidirectional Long Short-Term Memory network (BI-LSTM). The CNN is typically used for capturing spatial hierarchies in data, while BI-LSTM is adept at processing sequences and capturing long-term dependencies in either direction of a sequence. Following feature extraction, an attention layer is applied, which can help the model to focus on more relevant parts of the data for the task at hand. The attention mechanism’s performance can be influenced by hyperparameters (HP1), such as the number of layers in the neural network, the number of training epochs, the choice of activation function, and the amount of dropout in recurrent connections. These hyperparameters are crucial for optimizing the performance of the model. Finally, the output of the attention layer feeds into a Random Forest classification algorithm. Random Forest is an ensemble learning method that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes of the individual trees. This combination of neural networks with a traditional machine learning algorithm can leverage the strengths of both approaches to improve classification accuracy.

E. Overview of the proposed architecture

Fig 2 depicts an overview of the suggested hybrid MCL-FWA-BILSTM (Mean Convolutional Layer-Feature Weighted Attention BILSTM) classification architecture for intrusions. In Section 3, Tables 2 and 3 detail the various types of attacks on datasets and the number of characteristics associated with each class label. The methodology of this study focuses on several NIDS components, and in the following paragraphs, we present brief details of crucial steps. In the first step, a dataset is used as input. To begin with, data preprocessing is required. The data preprocessing operation was carried out one by one on both the datasets NSL-KDD and UNSW-NB15 (see details in Section 3, Table 2 and 3). The primary operations at this stage include the missing value process, one hot coding, label coding, feature transformation, feature scaling, and feature normalization. In the next step, this work has a novel CNN-MCL layer designed for feature extraction before applying it to the attention mechanism. The CNN-MCL mapping is implemented using the mean convolutional layer (MCL), which represents average convolutional processing. Additional convolutional layers (Convolutional 1 and Convolutional 2) are utilized as feature extractors. Fig 2 also depicts a pooling layer and a flattening step following the convolution phase. The feature vector is obtained once the layer is flattened. Then, in the following steps, features from the attention layer and the BI-LSTM are concatenated to produce the mapped features. In the following steps, CNN-MCL features are assigned semantic basis feature weights and self-attention-based feature weights for enhancing the expression ability of the traffic features, which are then combined as mapped features to increase the class’s learning with fewer instances due to the semantic meaning learned from the integrated features and to make an efficient decision regarding the unbalanced class. Feature attention weights balance this procedure in normal classes. Finally, these mapped features are fed to random forests for classification, and the model’s performance is evaluated.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Description of the NSL-KDD dataset attack categories.

https://doi.org/10.1371/journal.pone.0302294.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Attack types and their description in the UNSW-NB15 dataset.

https://doi.org/10.1371/journal.pone.0302294.t003

F. Mean Convolutional Layer (CNN-MCL)

The suggested method employs CNN-based layers to completely separate anomalies from normal data. The data is used for learning the changes occurring from abnormal data. As normal and abnormal events are generally identical in characteristics, the CNN is used to detect abnormality variation. The traditional CNNs have been used to detect attacks and feature learning for flow content or matrix content, so that the classifier is connected with training data instead of learning data differences. Though, the suggested technique was to carefully evaluate the content in order to learn the anomalous traces.

We developed a novel hybrid architecture using mean convolutional layer (MCL), which is commonly used in intrusion detection tasks [30]. The suggested layer aims to completely learn prediction error filters in order to replicate these actions. Therefore, the active prediction error fields are affiliated with the feature maps as low-level abnormal trace quantities. The CNN-MCL is responsible for being aligned in the opposite direction of the CNN intended to initiate the IDS tasks. This serves as a data storage mechanism, as prediction errors do not typically involve flow content, and it also provides the CNN low-level IDS features. The more layers a CNN has, the more capable it is of learning higher-level features. The CNN-MCL is described using the following Eq (1), in which L signifies the Lth CNN-MCL, k represents the kth convolutional filter inside a layer, and the core value of a convolutional filter is characterized as (C_x, C_y). In addition, the CNN must learn prediction error filters by enforcing significant constraints actively.

(1)

(2)

CNN-MCL predictions are recognized through a precise training phase. The filter weights are updated at every iteration using the Adam optimization algorithm during the following stage. By using CNN-MCL reinforcement, the updated filter weights are then integrated into the feasible set of prediction error filters, and a projection is executed on each training iteration. The central filter weight is set to a negative mean of middle values among all k filters inside the layer, and the remaining filter weights are normalized via Eq (2) above.

This methodology consists of two steps. The residual weights are partitioned by the sum of all filter weights, with the exception of the central value, after being multiplied by the mean value. The medians of all k filters are arranged to a negative mean value in layer L. Algorithm 1 below contains the pseudocode for this process:

Algorithm 1. CNN-MCL.

Input: Intrusion Dataset with features (F) and Labels

Output: Efficient Weight

Step 1. Initialize Random Weights (W) and  Set    i=1

Step 2. While i≤maximum iteration do

Do Feedforward pass

Update weight by Backpropagation using Adam

End While

(3)

  mean of central points of all filters of layer L

Step 4. Update the weights of layers using

(4)

Step 5. If Converge then exit

Step 6. Return W

Algorithm 1 gives efficient weights to features for improving the learning and its indirect impact on class imbalance. In the CNN-MCL algorithm, the iteration value depends on the training instances and the run-back propagation for reducing error. In the proposed approach shown in Fig 2, CNN-MCL filter Input Size N by 120, MCL layer N by 10 by 120, Conv1 N by 10 by 12 by 32, Conv2 5 by 5 by 16, flatten layer N by 56 after the weight is passed in the BI-ATT Algorithm 2 discussed in section J. Eq (3) in the CNN-MCL algorithm takes the mean of the values obtained by sigmoid mapping, with weights updated by Eq (4) and total weights also considered.

Algorithm 2. BI-ATT (Bidirectional Attention).

Input: Features and Weights

Output: Efficient mapped features

1. Input features F

2. 

3.  

4.  F_A<−by Eq(1)

5. F_AT<−by Eq (2)

6.  F_e<−F_BLConcatF_AT

7.  ReturnF_e

G. Multiple convolutional layers

The convolutional layer [31] is the most important component of a CNN. This layer applies multiple convolutional kernels to transform the feature maps (or input images) into new, unique feature maps. The deeper the network the more the layers might have a comprehensive range of vision and can capture global information [32]. So, as the number of convolutional layers increases, the scale of the convolutional features gradually gets bigger. A series of convolutional layers is used to learn higher-level prediction error features. As shown in Fig 2, each convolutional layer will learn the new representation of feature maps that the previous convolutional layer or lower-level features learned. The method described above shows how the Rectified Linear Unit (ReLu) activation function limits the range of data values at each stage of the network.

H. Pooling layer

The CNN used a max-pooled size of 3x3 with a stride of 2. The max-pooling layer has the largest value in the sliding window’s local neighbourhood. This layer tries to reduce the number of dimensions in the feature maps. The max-pooling layer lowers the cost of training and the risk of overfitting. The pooling layers keep the most representative feature and aid in subsampling and improving accuracy.

I. The Bidirectional Layer (BI-LSTM Layer)

During training, the BI-ISTM layer stores data in memory, sequentially represents long-distance correlation, and checks for correct gradient propagation. The LSTM model we use in our method is two-way, so it can be used to learn in both directions.

As stated in references [33,34] the BI-LSTM model is an improved form of the LSTM one. In order to extract coarse-grained features, the BI-LSTM model joins the forward LSTM model with the backward LSTM one. When new information is received, the LSTM model is trained to rewrite old content when new information is received. To perform the task first, it compares the contents of the innermost memory unit (Cu) via the input unit gate (I_gt), the forget unit gate (F_gt), and the output unit gate (O_gt) [35]. Information sent into an LSTM network is evaluated for usefulness in light of applicable rules, and outliers are forgotten using the mechanism of the forget gate as indicated in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The architecture of the BI-LSTM model.

https://doi.org/10.1371/journal.pone.0302294.g003

The hidden states of a BI-LSTM layer can be used in conjunction with an input arrangement z = z₀……..z_t at time t to yield the output sequence h = h₀……..h_t. The result of employing this forget unit gate can be derived as (5)

Here h_t−1 is the output of the hidden layer at time t₁ and z_t is the input at time t₂ corresponding to the forget gate (F_gt). The weights of the connections between the nodes are denoted by W, and the bias feature is denoted by b.

J. Attention layer

This layer works for the attention mechanism used in the proposed work. Today, an attention mechanism is a powerful tool for identifying essential information and achieving excellent results [36,37]. The feature-based attention mechanism is adopted to fully capture the genuinely significant features of the representation of network traffic.

The BI-LSTM is capable of utilizing information from both sides. The self-attention mechanism improves the attention mechanism, which lessens reliance on outside information and more effectively captures the internal correlation of data or features. Our model can pay more attention to the crucial features of the network intrusion dataset by using the self-attention mechanism. This mechanism can affect both the source’s and the target’s internal components. As a result, it can increase the learning features’ efficiency during training [38]. The feature-weighted attention (FWA) technique is used to assign weight values to features. It is incorporated into classification models to aid in focusing on the most relevant features for classification and to minimize the overfitting issue. The FWA-BILSTM model concentrates on unique features that aid in identifying changes in the input data. This improves the classification performance of the FWA-BILSTM model and aids in determining the alteration in an input.

The basic concept of the attention mechanism is to extract and signify the most significant information in the data. The attention mechanism is an automatic weight allotting scheme. In intrusion detection, the role of the attention mechanism would be to calculate the effects of each unit of network traffic, mostly due to the preceding unit of network traffic. The attention value for every unit of network traffic is determined using the following equation (6)

Where u_w is indeed the weight matrix and u_t is a matrix that acts as the implicit illustration of the CNN hidden state (h_t) at time t.

The BI-LSTM model’s packet vectors h_t are used in a nonlinear transformation to get its implicit representation μ_t which can be written as (7)

In this case, W_w denotes the weight matrix and b_w denotes the bias. After the determination of the attention probability distribution value for every instant, the following formula is used to calculate the feature vector v containing the network traffic information: (8)

Finally, we can use the following function Y to obtain the predicted label y: (9)

In Algorithm 2, we showed the non-linear mapping of features by the BI-LSTM RNN. This mapping improved class imbalance of IDs attacks, while the attention mechanism increased the reliability of features and the selection of an efficient mapping.

In Algorithm 2, F represents features and F_B and F_BL are weighted forward features and a combination of forwarded and backward feature mappings.

In , the Sigmoid Layer uses efficient weights that are obtained through the CNN-MCL approach.

The symbol W_cF_t denotes the initial weights obtained by the MCL layers and the CNN, and the symbol U_cF_t−1 denotes the previous layer non-linear mapping, while b represents the initial bias features.

In Algorithm 3, the proposed approach, which is called feature weighting by CNN-MCL starts with inputting weights Input to the BI-ATT algorithm, and, after feature mapping, it applies features or learning through the Random Forest algorithm by the bagging approach. After the learning phase, Algorithm 3 makes a classification model and analyzes performance metrics like accuracy, precision, recall, and F-score.

Algorithm 3. Proposed Approach (Attention-Feature Weighting).

Input: Dataset {features(F), Labels(X)}

Output: Classified Intrusion

1. Input Dataset and features

2. W_c<−CNN−MCL(F)

3. Fe<−BI−ATT(W_c,F)

4. C<−RandomForest(F_e,X)

5. AnalysisMetrics<−C(test)

K. Random Forest (RF)

The RF algorithm can be described as an ensemble of classification trees that use the results of the decision tree (DT) model to make predictions. Each tree has one vote for the most frequent class in the input data. The trees are trained to grow together into a forest using a method called bagging or bootstrapping aggregation [39]. In the RF algorithm, the best predictor for each node is chosen randomly at the node level. By bagging on bootstrap sets of training data, it would be possible to make a lot of decision trees. The average or mean output value from the different decision trees (DTs) are used to make the final prediction of the RF algorithm. We used the RF algorithm to look at the behaviour of the intrusion from a different point of view because the RF algorithm builds multiple decision trees and combines them to make a more accurate and stable prediction. Several studies [40–42] have shown that the RF algorithm is better than other traditional classifiers at spotting strange traffic.

The loss function is generated in terms of the following objective function F: (10) where, n denotes the given features with label instances and denotes the training loss function, which fits the data into the L norm of the leaf node by the following equation: (11)

All trees are assembled chronologically by using an additive learning process. Each newly added tree learns from its former tree and updates the prediction result by updating at the kth iteration. In this way, the input data are classified into two types: (a) normal data and (b) malicious or anomalous data.

L. Utilized metrics

Here, we use the most common metrics to measure how well the classifier model finds intrusions: precision, recall, false positive rate (FPR), false alarm rate (FAR), and F-score. The recall is also known as the Detection Rate (DR) in the intrusion detection problem. The detection rate (True Positive Fraction) generally represents the percentage of correctly classified malicious traffic. Similarly, the false positive rate (False Positive Fraction) shows the proportion of wrongly classified malicious traffic. Detection accuracy (DA) measures the system’s overall performance and demonstrates how well it can tell the difference between malicious and legitimate network traffic. In addition, the false alarm rate is the proportion of misclassified malicious and legitimate network traffic. The F-score is mainly utilized as a combination of Precision and Recall, which equals the Harmonic Mean of these two measures.

The Harmonic Mean is preferred to the Arithmetic Mean since the former punishes extreme values more severely. The F-score is used to evaluate the performance of every class of traffic. The performance of the proposed MCL-SA-BILSTM intrusion detection system is validated in terms of DR, Precision, FPR, and the F1-score. The definitions of these metrics are presented below: (12) (13) (14)

False Alarm Rate (FAR)—The false alarm rate represents the percentage of the normal and anomaly behaviours that are incorrectly classified.

(15)

(16)

(17)

The number of False Positives (FP) is the proportion of normal activities that are wrongly classified as an anomaly. The number of False Negatives (FN) is the proportion of anomalous behaviors that are incorrectly labelled as normal, whereas the number of True Negatives (TN) represents the proportion of normal behaviors that are accurately labelled as normal. Furthermore, the number of True Positives (TP) represents the proportion of anomalous behaviors that are accurately identified as an anomaly. The TP, TN, FP and FN are visualized in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Confusion matrix related to IDS for performance evaluation.

https://doi.org/10.1371/journal.pone.0302294.g004

The proposed approach consists of three main components. The first component is BILSTM, which is responsible for learning the sequence of features. Its length is set to 4 because LSTM units require four steps for error detection and correction, ensuring a fast learning rate by providing a value of 0.02. In our suggested approach, we set a hyperparameter Adam optimizer in the second stage, which involves hybridizing an attention layer and a mean CNN. The mean CNN consists of four layers with TANH and SoftMax activation functions. We next utilize a random forest algorithm, which employs boosting and bagging techniques, to tune the hyperparameters. The number of decision trees in the ensemble is determined through this process.

1. Dataset

We assessed the suggested framework of the present study using two datasets, namely NSL-KDD and UNSW-NB15.

2. Experimental results

In this section, we are going to analyze the performance of our MCL-FWA-BILSTM model in classifying network intrusion for the NSL-KDD and UNSW-NB15 datasets. Our assessment of the model outcome covers binary and multi-classification scenarios.

A. Multiclass experimental results for NSLKDD.

We tested the suggested approach on the NSL-KDD dataset to prove its efficacy. Table 4 lists the confusion matrix for all attack classes in the NSL-KDD dataset while Table 5 shows additional evaluations, FPR, DR, F-score Precision and accuracy. With relative values of 76952, 53360, and 14000, the classes Normal, DoS, and Prob were identified with extreme precision. By contrast, the R2L (3610) and U2R (186) classes exhibited significantly less precision than the other classes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Confusion matrix of NSL-KDD on the proposed approach.

https://doi.org/10.1371/journal.pone.0302294.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Performance metrics of NSL-KDD dataset using MCL-FWA-BILSTM model.

https://doi.org/10.1371/journal.pone.0302294.t005

The proposed model’s performance details and the performance of a multi-class classification against various types of attacks are presented in Table 5 and visualised in Fig 5. For each of the four attack categories, the precision, DR, and F-score metrics are separately computed. For DoS attacks, the suggested model showed excellent performance in terms of precision, detection rate, and F-score, with a performance metric of almost 1.0 for all, whereas for probe and normal attacks, the suggested scheme accomplished a performance metric approximately equivalent to 1.0, with a difference of roughly 0.2. For R2L, the model achieved a performance metric in terms of precision that amounted to 93.86%. Other metrics obtained are 98.23% for the detection rate and 96% for the F-score. On the other hand, for U2R, the precision, DR, and F-score obtained are 88.15%, 95.88%, and 91.85% respectively. Further, the proposed model obtained an overall accuracy of 99.88% for the multiclass classification.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. MCL-FWA-BILSTM model precision, DR, F-Score and FPR comparison on NSL-KDD dataset.

https://doi.org/10.1371/journal.pone.0302294.g005

B. Binary classification results of NSL-KDD.

The normal category has the value 0 and the remaining four categories (Dos, R2L, U2R, and Probe) are assigned the value 1. The assigned values are Normal = 0 and Attack = 1.

The proposed model’s performance for binary classification is represented in Fig 6. The MCL-FWA-BILSTM model accuracy and detection rate are observed as 99.67% and 99.28, respectively for NSL-KDD dataset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Binary classification accuracy and DR comparison on NSL-KDD.

https://doi.org/10.1371/journal.pone.0302294.g006

C. Multiclass experimental results for UNSW-NB15.

In order to illustrate the proposed method’s efficacy, we also conducted the experiment on the UNSW-NB15 dataset. The confusion matrix for the suggested technique on UNSW-NB15 is shown in Table 6, and additional evaluations are included in Table 7. The precision, detection rate (DR), and F-score values are highest for the normal and generic classes, exceeding 98% in nearly all instances. By contrast, the Back and Worms classes have the lowest precision values, owing to their relatively low number of examples. Consequently, the imbalance in the UNSW-NB15 dataset could have been considered one of the possible explanations for the performance disparity between different classes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Confusion matrix of UNSW-NB-15 dataset using MCL-FWA-BILSTM approach.

https://doi.org/10.1371/journal.pone.0302294.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Performance Metrics of the UNSW-NB15 dataset on the proposed approach.

https://doi.org/10.1371/journal.pone.0302294.t007

The MCL-FWA-BILSTM model’s performance details, as well as the performance of the multi-class classification against various types of attacks, are shown in Table 7. The Precision, DR, and F-score are computed separately for each of the nine attack categories in addition to the normal class. The proposed MCL-FWA-BILSTM model achieved performance metric values close to 1.0 for all Exp, Reco, and Fuzzy attacks, with a difference of about 0.3. The proposed MCL-FWA-BILSTM model performed exceptionally well regarding the precision, DR, and F-score metrics for the Normal and Generic classes, with all values approaching 1.0. The precision for Back, DoS, Shell, and Worm is between 0.6 and 0.8, while DR is between 0.8 and 1.0, and the F-score is between 0.7 and 0.9.The Fig 7 represents the visualization of performance metrics of all the nine attack classes. The Fig 8, represents the accuracy of the binary classification and the Fig 9 represents the comparison of the binary and multiclass classification results.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. MCL-FWA-BILSTM model precision, DR, F-Score and FPR comparison on UNSW-NB15.

https://doi.org/10.1371/journal.pone.0302294.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Comparison of accuracy and DR on UNSW-NB15.

https://doi.org/10.1371/journal.pone.0302294.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. MCL-FWA-BILSTM binary and multiclass accuracy comparison.

https://doi.org/10.1371/journal.pone.0302294.g009

A. Comparison of MCL-FWA-BILSTM model with the state of the art for Binary Classification

To objectively evaluate the accuracy and differentiation of the MCL-FWA-BILSTM model, we compared our model with some recent related works presented by several other researchers, shown in Table 8 and Fig 10, available in open sources. Experimental results confirmed that the proposed approach outperforms most other methods concerning binary classification accuracy. The results showed that the proposed model (99.67%) performed better than other existing techniques in the NSL-KDD datasets. For the UNSW-NB15 dataset, the MCL-FWA-BILSTM model performed optimally (99.56%) in comparison with the existing methods, as shown in Fig 10. The experimental results showed that the performance of MCL-FWA-BILSTM model is superior to that of models based on traditional machine learning methods and other deep learning methods in binary-class classification using the same datasets, NSL-KDD and UNSW-NB15, for network traffic classification. Further, some researchers validated their work using the NSL-KDD benchmark dataset. While few researchers validated their work using the UNSW-NB15 benchmark dataset only. The comparison of all those model results with the proposed hybrid model MCL-FWA-BILSTM visualization is shown in Table 9. It is evident that the proposed model outperformed almost all the previous models.

B. Comparison with the state of the art for multiclass classification

To assess the accuracy and differentiation of MCL-FWA-BILSTM model, we conducted a comparative analysis with various related works from open-source materials. The experimental findings validate that our approach surpasses most other methods in terms of multiclass classification accuracy for both datasets examined. In particular, MCL-FWA-BILSTM model achieved excellent results (99.88%) on the NSL-KDD dataset compared to other methods, while it did the best (99.45%) on the UNSW-NB15 dataset compared to other methods as shown in Table 10. Previous researchers in this field used the NSL-KDD and UNSW-NB15 datasets to identify network traffic. The obtained results demonstrated that the MCL-FWA-BILSTM model outperformed traditional machine learning models and other deep learning techniques for multi-class classification using these datasets. Fig 11 depicted the results of comparing accuracy across various studies conducted on the NSL-KDD and UNSW-NB15 datasets. Some researchers validated their work using the NSL-KDD benchmark dataset, while others used only UNSW-NB15, the comparison of MCL-FWA-BILSTM model and others shown in Table 11.

C. DR and FPR comparison for multiclass classification on UNSW-NB15

Fig 12 illustrates the performance of the proposed method on the UNSW-NB15 dataset with respect to the detection rate and FPR. The model obtained an excellent detection rate for Gene, with an FPR of 0.13. Worms have the lowest FPR value, but their detection rate is very low in comparison to most other attacks. As discussed in the above section only a high DR value does not ensure the effectiveness of the model unless its FPR is low. It is evident from the results, as shown in the Fig 12, that the MCL-FWA-BILSTM model achieved the lowest FPR and the highest DR in most of the cases, which confirms the model’s efficiency. The detection rate of each of the ten classes is plotted against the FPR in Fig 12. The highest DR is 99.75% for the generic class, and the lowest DR obtained is 80.97% for the shell category. While the model obtained FPR with a maximum value of 1.14% for exploits and a minimum value of 0.02% for worms. The result obtained by the model MCL-FWA-BILSTM showed that it can classify Nor, Back, DoS, Exp, Shell, Gene, Reco, and Wrm very accurately. The performance of the MCL-FWA-BILSTM model is comparatively better than any state-of-the-art model, as shown in Table 12. The MCL-FWA-BILSTM model achieved best performance in multiclass classification, with an accuracy of 99.45%.

D. F-score, DR and FPR comparison for multiclass classification on UNSW-NB15using MCL-FWA-BILSTM approach

The F-score for multiclass UNSW-NB15 dataset showed excellent performance above 90% for Normal, Fuzzy, Exploits, Generic, and Reconnaissance as shown in Fig 13. While for analysis, Dos and shellcode the achieved F-score values found above 80%. On the other hand, for the backdoor and worms, the F-score value found below the 80% but the DR value of Backdoor found 95.26% and also the FPR was small as 0.09. Further it is observed from the Fig 13, worms attained the DR value as 81.48% and the FPR as 0.02. So, we can conclude that even for all the 10 classes, MCL-FWA-BILSTM model showed excellent performance concerning the F-score, DR and FPR.

E. Minority class detection from UNSW-NB15

In UNSW-NB15 dataset, the worms, shellcode, analysis and the backdoor belong to minority traffic. The worms attacks account for 0.07%, shellcode 0.59%, backdoor 0.9%, and the analysis attacks account for 1.05%. Nonetheless, by means of our suggested approach, MCL-FWA-BILSTM, this model managed to identify such attacks with a considerably elevated level of detection rate as shown in Fig 14. It is evident from Fig 14, in relation to analysis attacks, the model attained a highest detection rate of 97.79%, and for the shellcode, the least value of DR was found 80.97%. Moreover, the total accuracy score amounted to an impressive 99.45%. This indicates that the model effectively improved the imbalance issue and the model’s performance.

F. Comparison of MCL-FWA-BILSTM approach with the state of art considering DR and FPR and Accuracy using UNSW-NB15

It is evident from the comparison as shown in Table 12A and 12B the MCL-FWA-BILSTM model performed exceptionally well on all metrics, including DR, FPR, and accuracy. Further, it is more obvious from the results each category (DR and FPR) shown in Table 12B outperformed all other models. This makes the proposed model preferred for usage in intrusion detection systems.

G. DR and FPR comparison for NSL-KDD

Fig 15 illustrates the performance of the proposed model on the NSL-KDD dataset with respect to the detection rate and FPR. The model obtained the highest detection rate for DOS and the lowest value for FPR. It is observed experimentally that a model is considered effective if its detection rate(DR) is high and its FPR is low. It is evident from the results, as shown in Fig 15, that the proposed model achieved the lowest FPR, which confirms the model’s robustness and efficiency. On the other hand, the model performance is lowest in terms of DR for the U2R, and the FPR value is comparatively large for the U2R, among others. Examining the plot of class DR and FPR in Fig 15, the best detection rate (DR) is 99.93% for DoS, and the best false positive rate is 0.02% for DoS and U2R. The least DR is 95.88 for U2R, and the highest FPR value is 0.16 for R2L. It is very clear that the model is able to classify all the classes very accurately.

H. F-score, DR and FPR comparison for multiclass classification on NSL-KDD using MCL-FWA-BILSTM approach

It is evident from the Fig 16, the F-score for multiclass using NSL-KDD dataset, showed excellent performance above 90% for all five classes in terms of F-score as well as DR. Further, FPR value also considerably low for all the five classes. So, we can say the MCL-FWA-BILSTM model achieved excellent classification performance concerning the F-score, DR and FPR related to all attack class of NSL-KDD.

I. Minority class detection from NSL-KDD

In NSL-KDD, the minority traffic is the R2L and U2R attacks. The R2L attacks account for 0.08%, while the U2R attacks account for 2.61%. Nonetheless, by means of our suggested approach, MCL-FWA-BILSTM, we managed to identify such attacks with a considerably elevated level of detection rate as shown in Fig 17. For example, in relation to R2L attacks, we attained a detection rate of 98%, and for the U2R attempts, this rate was as high as 96%. Moreover, the total accuracy score amounted to an impressive 99.88%. This indicates that the model effectively improved the imbalance issue and the model’s performance.

J. Comparison of MCL-FWA-BILSTM approach with the state of art considering DR and FPR results on NSL-KDD

We compared the indicators of our method with others state of art models in Table 13. The proposed MCL-FWA-BILSTM method outperformed all other deep learning-based models in terms of DR and also in terms of FPR. Based on the evaluation, results obtained from NSL-KDD dataset, the MCL-FWA-BILSTM model outperformed other approach in nearly all aspects. Specifically, when examining the two categories with adequate samples, we observed that our method’s detection rate surpasses 95%, while other techniques typically exhibit rates below 85%. Regarding the FPR metric, we note that this metric also outperformed all other models except the FPRs of U2R, which are slightly inferior to U2R RNN [12].

K. Comparison of MCL-FWA-BILSTM approach with the existing MCL and FWA-BILSTM model

In the Fig 18, bar chart illustrates a comparative analysis of three different models’ performance on multiclass classification tasks across two datasets, NSL-KDD and UNSW-NB15. The vertical axis represents classification accuracy percentages, while the horizontal axis identifies the models: MCL [69], FWA-BILSTM [70], and proposed MCL-FWA-BILSTM method. For the NSL-KDD dataset, MCL [69] shows lower performance, while the proposed MCL-FWA-BILSTM method achieves the highest accuracy, closely followed by FWA-BILSTM. In contrast, for the UNSW-NB15 dataset, the proposed MCL-FWA-BILSTM method still leads with the highest accuracy, but it shows a significant improvement over its performance on NSL-KDD, surpassing FWA-BILSTM. This visual representation emphasizes the superiority of the proposed MCL-FWA-BILSTM method in multiclass classification accuracy for both datasets.

Based on the outcomes, it is evident that the MCL-FWA-BILSTM hybrid deep learning model outperformed other existing techniques owing to its superior performance. In fact, it utilized the novel hybrid mechanism to enhance the classification performance. When utilizing the UNSW-NB15 dataset, the proposed model achieved the highest accuracies of 99.56% and 99.45% in the cases of binary and multiclass classification, respectively. Also, for the NSL-KDD, the achieved accuracies for binary and multiclass are 99.67% and 99.88%, respectively. Further, this model showed enhanced DR and FPR values in comparison to other deep learning machine learning methods as shown in Tables 12 and 13.

L. Reason for the improved results

In intrusion detection, primary challenge data or situations of class imbalance are detected, and in real-time, new attacks that have not been detected previously are detected; how can this be resolved? Although the results indicate improvement, the reason for this improvement is discussed in this section. The proposed approach involves the transmission of features via two networks. The first is the attention mechanism. Layers of attention refine the features based on their relative weight. The second section uses BI-LSTM to decide how to use learning features and how much weight to give them based on semantic relationships. In both processes, the proposed approach obtains the most efficient weight of components and aids in parameter learning. During training, learning parameters determines whether a feature is significant and should receive domain or class weights. If so, the learning phase using ensemble learning improves. So far, this section has talked about why learning gets better. Now we will discuss the impact of our new algorithm on challenges. First, we deal with the class imbalance; our algorithm enhances learning in the presence of class imbalance due to the semantic weight of its features. If we have some instances, it is capable of detecting additional instances. The second challenge in real-time is that if a new sample arrives, the attention mechanism must be capable of performing a weighted attack rather than an attack on the class. So, it will be easy for the mechanism to figure out its binary type.

M. Statistical analysis

Table 14, based on nine approaches or measurements presents descriptive statistics for two datasets–NSLKDD and UNSWNB15. Both of these mean figures reflect the average value acquired from the nine approaches. NSLKDD has a mean of about 85.91, which is higher than that of UNSWNB15, which has a mean of around 77.13, smaller std. err. (Stand Error) indicate high precision in representing a sample’s average as the population’s one. In relation to UNSWNB15, it is essential to note that NSLKDD has a standard error of about 2.16, while for UNSWNB15, it is about 4.22, thus indicating a more precise estimation of its MEAN. The 95% and 99.9% confidence intervals(CI) give an estimate within which the true mean of the population will lie, assuming we have taken a representative sample. NSLKDD shows both a higher mean and a more precise estimation of that mean (as indicated by the more minor standard error and narrower confidence intervals). The interval between 80.92 and 90.90 contains the actual population mean for NSLKDD with a relatively confident level of likelihood, though with significant probability errors. In contrast, in the case of UNSWNB15, this interval holds between 67.40 and 86.85 because these CIs are more extensive due to big standard errors. The wider range interval (55.87–98.39) accounts for 0.01% margin error at most for UNWNB15’s estimated means with such a high degree certainty level as usually happens when raising confidence level beyond lower limits where narrower ones can be realized. Moreover, it should be noted that when compared to other measures, NSLKDD demonstrates broader ranges due to more minor standard errors. Thus, we can say that if the data completeness approach is used in evaluation studies, then NSLKDD would yield better results than KDDCUP99.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 14. MCL-FWA-BILSTM and other existing approaches for multiclass classification in both datasets.

https://doi.org/10.1371/journal.pone.0302294.t014