https://orcid.org/0000-0002-3014-609X
Figures
Abstract
Intrusion detection plays a significant role in the provision of information security. The most critical element is the ability to precisely identify different types of intrusions into the network. However, the detection of intrusions poses a important challenge, as many new types of intrusion are now generated by cyber-attackers every day. A robust system is still elusive, despite the various strategies that have been proposed in recent years. Hence, a novel deep-learning-based architecture for detecting intrusions into a computer network is proposed in this paper. The aim is to construct a hybrid system that enhances the efficiency and accuracy of intrusion detection. The main contribution of our work is a novel deep learning-based hybrid architecture in which PSO is used for hyperparameter optimisation and three well-known pre-trained network models are combined in an optimised way. The suggested method involves six key stages: data gathering, pre-processing, deep neural network (DNN) architecture design, optimisation of hyperparameters, training, and evaluation of the trained DNN. To verify the superiority of the suggested method over alternative state-of-the-art schemes, it was evaluated on the KDDCUP’99, NSL-KDD and UNSW-NB15 datasets. Our empirical findings show that the proposed model successfully and correctly classifies different types of attacks with 82.44%, 90.42% and 93.55% accuracy values obtained on UNSW-B15, NSL-KDD and KDDCUP’99 datasets, respectively, and outperforms alternative schemes in the literature.
Citation: Yılmaz AA (2025) A novel deep learning-based framework with particle swarm optimisation for intrusion detection in computer networks. PLoS ONE 20(2): e0316253. https://doi.org/10.1371/journal.pone.0316253
Editor: Raman Singh, University of the West of Scotland, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: September 2, 2024; Accepted: December 9, 2024; Published: February 12, 2025
Copyright: © 2025 Abdullah Asım Yılmaz. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The raw code used in this study is publicly available on GitHub at https://github.com/abasimyilmaz/DL-Based_IDS_Framework_with_PSO and Zenodo at https://doi.org/10.5281/zenodo.11212077. The study utilized three publicly accessible datasets: the KDDCUP’99 dataset, available at https://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html; the NSL-KDD dataset, available at https://www.kaggle.com/datasets/hassan06/nslkdd; and the UNSW-NB15 dataset, available at https://research.unsw.edu.au/projects/unsw-nb15-dataset.
Funding: The author(s) received no specific funding for this work.
Competing interests: The author have declared that no competing interests exist.
Introduction
In the realm of network security research, network intrusion detection systems (IDSs) have great importance. Through the use of active protection technology, these systems can identify signs of intrusion and swiftly respond by taking the necessary measures to halt such intrusions. These measures may include issuing warnings to users or implementing other relevant safeguards [1]. The various intrusion detection methods in the literature primarily fall into two types: anomaly-based IDS (AIDS) and signature-based IDS (SIDS) [2]. Anomaly-based detection entails formulating a model that describes the customary behaviour of network traffic; any observed activity that diverges from this model is then deemed to be an intrusion. This type of method is effective at identifying previously unknown attacks [3]. Signature-based systems operate primarily on the basis of identified intrusion attacks; in scenarios where attack signatures are unknown, the identification of anomalous network activity is predominantly accomplished through the use of anomaly-based systems [4]. The development of a strong and efficient network IDS remains a key challenge for researchers in network security. Despite considerable advancements in IDS technology, a considerable number of solutions still rely on less effective signature-based methods, rather than employing the more capable anomaly detection techniques [5]. To overcome the challenges of creating a versatile and effective network IDS to handle unforeseen and unpredictable attacks, one possible solution is to implement a deep-learning-based approach [6]. When applying machine learning (ML) in practical applications, experts in the field have typically designed the features used to describe samples, and specialised knowledge is then necessary to process the data effectively. The quality of these features plays a critical role in the model’s generalisation performance, and the design of good features is a challenging and error-prone task. In contrast, deep learning techniques are specifically designed to learn superior feature representations through the analysis of vast quantities of unlabelled data, which can then be applied to the classification process [7]. Deep learning has been used for concerted efforts in this direction, and represents a distinct subfield of ML [8].
Over the past few years, significant numbers of researchers have adopted deep learning techniques across diverse fields, such as graph-based applications [9], hand gesture recognition [10], enhancement of driving safety [11, 12], identification of human actions [13], recognition of facial emotions [14], speech recognition [15], natural language processing [16], malware classification [17–19], medical applications [20], and object detection [21]. The achievements of deep learning techniques have attracted the interest of scholars in the field of intrusion detection, and research in this field has been undertaken to tackle the shortcomings of existing intrusion detection techniques and to elevate the overall effectiveness of models [22]. However, the development of a resilient system remains challenging. Robust systems are of the utmost importance for reducing feature spaces and computational complexity, surmounting the constraints imposed by time, resources, and hardware, and augmenting the accuracy rates of IDSs. Hence, this study presents a new framework for detecting intrusions that exploits the capabilities of deep learning methods. The objective is to build a hybrid system that improves intrusion detection’s effectiveness and precision.
In this study, intrusion data were first collected from the KDDCUP’99 [23], NSL-KDD [24] and UNSW-NB15 [25] datasets, and were pre-processed, which involved data coding, standardisation and conversion. When the pre-processing stage was complete, the convolution layers of the proposed hybrid architecture were applied to extract high-level features from the intrusion data, and high-level features were obtained. At the parameter optimisation stage, the parameters were fine-tuned for optimal performance. Finally, classification was conducted by applying the optimised parameters to the features that were extracted. In essence, the proposed approach combines multiple extensive deep learning models, based on the transfer learning technique, to create a hybrid model. In the stages described above, the rectified linear unit (ReLU) function and several hidden layers were applied. Moreover, parameter optimisation for these deep learning algorithms was carried out with particle swarm optimisation (PSO). The outcomes of the test phase demonstrated that the proposed approach efficiently extracted distinctive features for each category of attack, thus enabling effective classification. The empirical results also indicated that the proposed deep learning method achieved superior accuracy in terms of classifying distinct attack categories compared to existing methods in the literature.
This study makes five major contributions to the literature. Firstly, it introduces a novel hybrid deep-learning-based intrusion detection method. Secondly, the PSO method is used to optimise the hyperparameters, thus enhancing the performance of the model. Thirdly, the method is evaluated on three extensive intrusion detection datasets to ensure its effectiveness and applicability. Fourthly, the proposed approach incorporates a new hash layer that combines three pre-trained models, which distinguishes it from conventional deep learning approaches that use only one model. Lastly, the method can significantly reduce the feature space and achieves superior accuracy compared to other known methods, representing a valuable advancement in intrusion detection research.
The remaining sections of this work are structured as follows. Section 2 presents background information about intrusion detection and a concise overview of pertinent studies in this field. Section 3 provides a detailed explanation of the proposed framework. A discussion and some experimental results are given in Section 4, with an explanation of the intrusion detection datasets used. The paper is concluded with a concise summary and a discussion of potential future work.
Related work
In this section, we present some necessary background information and review the studies of IDSs in the literature that are relevant to our study to enable the reader to gain an understanding of the concepts that underpin the model proposed in this paper.
IDSs can be classified as detection or deployment techniques, and a framework depicting this taxonomy is provided in Fig 1. An IDS can be subclassified as a SIDS or an AIDS from a detection-based perspective, whereas from a deployment- based perspective, it can be classified as a network-based IDS (NIDS), hybrid IDS, or host-based IDS (HIDS) [26–28]. Among these classifications, Machine learning-based, Deep learning-based and Hybrid IDSs, which are relevant to our study, will be explained in detail with examples in the following subsections. Table 1 summarizes various deep leraning (DL), ML, and Hybrid approaches for intrusion detection systems with various datasets. Among these classifications, Machine learning-based, Deep learning-based and Hybrid IDSs, which are relevant to our study, will be explained in detail with examples in the following subsections. In addition to the examples in the subsections, Table 1 summarizes various DL, ML, and Hybrid technique for intrusion detection systems with various datasets. The dataset utilized to test the system, the author and year of publication, and the DL, ML, and hybrid IDS techniques are listed in Table 1. When reviewing the studies in the literature presented in Table 1, it is evident that systems integrating various DL and ML techniques have been developed. The KDDCUP’99, NSL-KDD, and CICID2017 datasets—which are often used in the literature—are used in the tests of the created systems.
Machine learning-based IDS
ML models can be divided into two primary types, unsupervised and supervised [39], as depicted in Fig 2. In unsupervised learning, labels are not attached to the training data passed to the system. Examples of supervised learning include K-means clustering, reinforcement learning, and principal component analysis. In supervised learning, the training data passed to the algorithm contain the desired solutions, which are known as labels. Examples of supervised learning methods include decision trees (DTs), linear regression, SVM, and k-nearest neighbours (KNN). Bayesian classification, SVM and similar ML algorithms are commonly employed in the existing literature for IDSs. ML intrusion detection systems are based on five stages, as shown in Fig 3. The initial step is to collect data using monitoring tools, and the second involves pre-processing the gathered data. In the third stage, dimensionality reduction is carried out to obtain more accurate results. The model is then trained, and finally, results are obtained by testing the trained model.
Singh et al. [40] developed an intrusion detection technique based upon the online sequential extreme learning machine. Their goal was to address the challenges faced by IDSs, such as low detection rates, high false alarm rates, and the need to handle large amounts of data. The suggested approach employed an ensemble of feature selection techniques including filtered, correlation, and consistency-based methods, in order to eliminate irrelevant features. Furthermore, this method leveraged alpha profiling to alleviate the problem of time complexity, while beta profiling was employed to reduce the size of the training dataset. The empirical results showed a detection time of 2.43 s and a false positive rate of 1.74%, with high accuracy, on the binary class NSL-KDD dataset.
Costa et al. [41] conducted research on the application of ML techniques in the context of IoT security, with a specific focus on intrusion detection. The aim of their study was to provide an overview of recent works in the field and their contributions. Various papers in the literature were reviewed, and the challenges faced in IoT environments were highlighted. The use of ML and deep learning algorithms for intrusion detection was emphasised, and their promising results were discussed. The article also mentioned the importance of addressing false positive rates in intrusion detection, while considering the trade-off between improved recognition rates and increased computational burden. Overall, these authors concluded that intrusion detection in the context of IoT remains a challenge, but highlighted the ongoing efforts in terms of developing optimised security protocols to protect data while minimising energy consumption.
A comprehensive article by Buczak and Guven [42] in 2016 surveyed various ML algorithms, such as SVM, intended for the purpose of anomaly detection. The article delved into the intricacies of ML and DL algorithms, examined the difficulties associated with employing ML and DL in the realm of cybersecurity, and put forth a number of suggestions and recommendations. In another study in the same year, Aburomman et al. [43] introduced a unique approach for constructing ensembles of classifiers (SVM and KNN) using weights generated by PSO. The authors emphasised that the use of weights generated by metaheuristic techniques could lead to enhanced accuracy in IDSs. Although SVM is considered a cutting-edge ML algorithm, its performance relies heavily on the careful selection of appropriate parameters. Enache et al. [44] introduced an IDS in which the information gain was used to select relevant features, and which included an SVM classifier. The parameters for the SVM were determined using the artificial bee colony (ABC) and PSO swarm intelligence algorithms. The experimental results showed that the optimised IDS model achieved a high accuracy rate when fine-tuned using PSO or ABC on the NSL-KDD dataset.
Deep learning-based IDS
Deep learning models are typically created using deep neural network (DNN) structures. A DNN is an artificial neural network with many layers, and learning is performed via the connections between these layers. Deep learning models learn using multiple layers of these DNN structures, and attempts to learn the complex relationships in the dataset [45].
Recurrent neural networks (RNNs) and CNNs represent two of the most frequently used types of DNN models. CNNs are specifically designed to accept raw data as input, thereby eliminating the need for feature extraction or image reconstruction. In addition, they have a relatively low number of parameters and require only a small amount of data. Due to these advantages, CNNs have proven to be exceptionally successful in performing tasks related to image recognition [46], natural language processing and so on. In the case of certain network traffic protocols, CNNs have shown a capacity for performing well due to their ability to be trained quickly [47].
Fan and Ling-Zhi [48] achieved a high level of accuracy in feature extraction by employing a multilayer CNN architecture in which a convolution layer was linked to the sampling layer. When applied to the KDDCUP’99 dataset, this model demonstrated superior performance compared to conventional detection algorithms such as SVM. However, CNNs are unable to consider timing knowledge in a specific traffic scenario, and are limited to analysing a single input package. In a real-world attack scenario, a single packet of traffic may appear as normal data, and may be identified as malicious traffic only when a large number of packets is transmitted concurrently or within a brief time frame. CNNs are inadequate in this case, as they could lead to a large number of missed alarms. In contrast, RNNs are frequently used to analyse sequential data, and long short-term memory (LSTM), a type of RNN, has performed well in applications that require the analysis of sequential data, such as natural language processing [48] Kim et al. [49] evaluated the performance of various algorithms, including a Bayesian approach, SVM, KNN, a product-based neural network, a generalised regression neural network, and the LSTM-RNN network, on the KDDCUP’99 dataset. The LSTM-RNN network performed better than the other methods. Hassan et al. [50] introduced a combined CNN and weight-dropped LSTM (WDLSTM) deep learning approach for detecting network intrusions, and conducted experiments on the UNSW-NB15 dataset. The authors of the work employed WDLSTM to maintain long-term dependencies between the features in order to prevent overfitting in recurrent connections and CNN to derive meaningful features from the massive IDS data. On the UNSW-NB15 dataset, the performance of the suggested hybrid method was compared with that of conventional methods and it is obtained satisfactory performance.
Hybrid IDS
Hybrid intelligent systems have been created to overcome the limitations of traditional IDSs, which often struggle with low detection rates for new attacks and a high incidence of false positive alerts. A hybrid approach involves merging both misuse-based and anomaly-based techniques to achieve a more comprehensive and effective detection capability [51]. An HIDS combines both anomaly and signature-based IDSs to strike a balance between storage and computing costs while minimising false positive alarms. This approach has become increasingly popular due to its efficient detection capabilities and ease of use [52]. An HIDS can enhance the accuracy of intrusion detection by integrating multiple detection models, and typically consists of two main components: the first component receives and processes unclassified data, while the second component analyses the processed data and identifies potential intrusion activities. The ultimate goal is to achieve superior detection performance through the combined efforts of these two components [53]. There are three main categories of HIDSs: clustered models with a single hybrid, integrated hybrids and cascaded hybrids. Each of these categories represents a distinct approach to combining multiple detection models for improved intrusion detection capabilities [54].
Issa et al. [37] proposes a novel deep learning-based Intrusion Detection System (IDS) for detecting Distributed Denial- of-Service (DDoS) attacks by hybridizing Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks. The model leverages CNN for automatic feature extraction and LSTM for sequence prediction, creating a seven-layer architecture to enhance detection performance. The model is tested using the NSL-KDD dataset, and it achieves superior performance metrics, a high accuracy rate, which surpasses traditional CNN and LSTM models as well as other state-of-the-art approaches. The results demonstrate the model’s effectiveness in improving detection accuracy and reliability for DDoS attack scenarios. Alghayadh et al. [55] developed a hybrid intrusion detection model specifically for enhancing smart home security. The model consisted of two different components, each serving a different purpose: in the first module, ML algorithms such as KNN, DT, XGBoost and random forest were used to enable real-time intrusion detection, whereas in the second module, the misuse intrusion detection technique was applied to identify known attack patterns. To evaluate the performance of this model, the researchers conducted tests using both the CSE-CIC-IDS2018 and NSL-KDD datasets. The results demonstrated the model’s exceptional ability to detect network intrusion and user-based anomalies in the context of smart homes.
Saheed et al. [56] introduces a hybrid feature selection approach combining the Bat metaheuristic algorithm with the Residue Number System (RNS) to enhance intrusion detection systems (IDS). The Bat algorithm is used to identify significant features by optimizing search spaces, while RNS improves computational speed and reduces complexity through its modular arithmetic properties. Principal Component Analysis (PCA) is applied for feature extraction, and classification is performed using Naïve Bayes (NB) and K-Nearest Neighbors (KNN) algorithms. Experimental evaluations on the NSL-KDD dataset demonstrated the superior performance of the Bat-RNS+PCA+KNN model, achieving a detection high accuracy, precision and F-Score values.
Abdulganiyu et al. [57] developed XIDINTFL-VAE model to detect minority class intrusions to detect minority class intrusions in highly imbalanced network traffic. This framework combines Class-Wise Focal Loss (CWFL) with a Variational AutoEncoder (VAE) and integrates XGBoost as the classifier. The CWFL-VAE generates synthetic data tailored to the minority classes, improving the detection capabilities for rare and critical attacks. XGBoost is used for robust classification, benefiting from its ability to handle structured data and imbalances. Experiments were conducted on the NSL-KDD and CSE-CIC-IDS2018 datasets, demonstrating significant improvements over traditional methods such as SMOTE and ADASYN, as well as existing classifiers like Logistic Regression and Random Forest. The XIDINTFL-VAE achieved high performance across metrics, with obtained high precision, F1 score and recall values, effectively balancing detection accuracy and minimizing false alarms.
Saheed et al. [58] proposes the IoT-Defender framework, a lightweight intrusion detection system (IDS) designed to enhance IoT network security using edge computing. IoT-Defender integrates a Modified Genetic Algorithm (MGA) for optimal feature selection and a Fine-Tuned Long Short-Term Memory (LSTM) model for anomaly detection. The framework employs MGA to identify the most relevant features from IoT-specific datasets (BoT-IoT, UNSW-NB15, and N-BaIoT) and uses a genetic algorithm to fine-tune the LSTM architecture, optimizing parameters like the number of hidden layers and learning rates. The system also incorporates a focal loss function to address class imbalance in IoT traffic data. Experimental results on edge devices such as the Raspberry Pi 4 show that IoT-Defender has a 2.56% It shows that it achieves a low false alarm rate and achieves high accuracy and detection rate.
Even though a lot of research has been done on intrusion detection, it is still quite challenging to detect different kinds of network intrusions effectively in the field of cybersecurity. Cyberattacks are also growing and changing at exponential rates, and many of these modern cyberattacks cannot yet be detected by any reliable system or technique. Additionally, traditional and contemporary IDSs are no longer able to differentiate sophisticated intrusions from typical network traffic. Therefore, a novel hybrid deep learning architecture for the effective detection of many intrusion types has been developed in this study. The proposed architecture is a hybrid of three pre-trained networks based on transfer learning. Our work’s primary contribution is a novel deep learning-based hybrid architecture that combines three well-known pre-trained network models in an optimized manner while using PSO for hyperparameter optimization.
Proposed method
A novel framework for intrusion detection is introduced in this section. Our framework leverages deep learning algorithms and incorporates a hybrid DNN architecture that supports hyperparameter optimisation. The most important contribution of this framework is its unique hash architecture, which effectively combines three commonly used pre-trained network models and enables hyperparameter optimisation. The methodology of our system consists of five main phases, as shown in Fig 4. First, intrusion detection data are collected from two extensive datasets. The data are then subjected to pre-processing, as described in depth in the subsection on data pre-processing. Thirdly, high- and low-level intrusion detection features are extracted using pre-trained networks in the feature extractor phase, which also involves performing hyperparameter optimisation. At this stage, the ResNet50 [59], GoogLeNet [60] and AlexNet [61] deep learning architectures are used for feature extraction/selection process, while the PSO method is used for hyperparameter optimization. Next, the proposed architecture is trained using the optimised hyperparameters obtained through PSO together with the intrusion detection datasets in the fully connected (FC) layers. Finally, the output phase involves classification using a SoftMax classifier. To assess the performance of the proposed method, we carried out experiments on three extensive intrusion datasets. The results demonstrated that our method achieved high accuracy in terms of classifying intrusion detection data and outperformed existing state-of-the-art methods. We provide a detailed explanation of the performance results in Section 4.
The remainder of this section is structured as follows. We describe the pre-processing of the data, give an overview of the suggested model, and discuss the schemes in the literature that were used in our method. In the data pre-processing subsection, we describe how the intrusion detection data are converted into visual images for identification using a CNN. The next subsection provides a detailed description of the proposed intrusion detection framework. Lastly, we explain the deep learning architectures employed in our method and discuss the literature pertaining to PSO.
Data pre-processing
In our scheme, a pre-processing method designed by Li et al. [62], in which a CNN is used, is applied to define an image representation of the intrusion detection data. The aim is to convert intrusion data into a visual image format. To accomplish this, different types of features are mapped into a binary vector space, which is then transformed into an image.
In the intrusion data attributes, there are three symbolic datatypes: the protocol type, flag, and service. These features are encoded using a one-hot encoder, resulting in binary vectors. For instance, the protocol type, with values icmp, udp, and tcp, is transformed into binary vectors with three dimensions, (100, 010, 100), as illustrated in Fig 5(a).
In this method, a standard scaler min-max normalisation technique is applied to manage the continuous features in intrusion data consisting of integer and floating point types. The purpose is here to transform the continuous data to the range [0, 1]. This approach can ensure that the values of the features are rescaled proportionally to fit the desired range. Min-max normalisation was carried out using the formulation in Eq (1)):
(1)
where wmax, wmin, and x represent the maximum value of the feature, the minimum value and the numeric feature value respectively, whereas x new represent value after normalisation process in equation. The scaled continuous value is divided into 0 intervals after the normalisation step. The order number of intervals are then encoded into 10 binary vectors using a one-hot encoder, as shown in Fig 5(b).
Overview of the proposed method for intrusion detection
Our approach provides an optimised framework for the task of intrusion detection, since it is built as a hybrid DNN. The suggested structure, depicted Fig 4, comprises six phases: collection of intrusion data, pre-processing, creation of the DNN architecture, fine-tuning of the hyperparameters, training, and assessment. A flowchart of the system is shown in Fig 6 to give a more thorough illustration of these five processes. Feature extraction is done using the pre-trained networks that were used in the parameter tuning and pre-training steps. At the training stage, the last layer is a SoftMax classifier, while the first three layers are FC layers that carry out learning operations.
Intrusion data are converted into a binary vector following this pre-processing step. The acquired binary vector is then converted into an 8 × 8 greyscale image with zero padding for any empty pixels. Fig 7 displays samples of visualised intrusion image frames that were produced as a result of these processes.
In the first step, samples of intrusion data are collected from the KDDCUP’99 [23], NSL-KDD [24] and UNSW-NB15 [25] datasets. A comprehensive description of these intrusion datasets is provided in Section 4. The intrusion data then undergo a pre-processing step, the specifics of which are detailed in the previous subsection.
The architecture of the suggested DNN is then created. At this point, a process is first carried out to determine an suitable deep learning architecture. A hash model is constructed by combining pre-trained architectures, since preliminary experiments indicated that a hash model produced a higher total precision [38]. This hash module includes the ResNet50, GoogLeNet and AlexNet architectures. Transfer learning is then applied. Adapting a model trained on one task to do a different but related task is known as transfer learning, and this is carried out by stripping the architecture of the pre-trained model down to a specific layer and then adding new layers that are relevant to the current problem. In this way, the medium- and low-level feature extraction layers of the pre-trained model are transferred to the new model, and high-level feature extraction is accomplished using the layers that are added to the architecture to reflect the relevant classification problem. Transfer learning methods have been extensively used to tackle challenges related to classification processes, such as large dataset sizes, model complexity, time constraints and hardware resource limits. In view of these challenges, transfer learning is adopted in the proposed architecture.
Next, our deep-learning-based architecture undergoes hyperparameter tuning in order to optimise the parameters. The automated process of adjusting deep learning or ML model hyperparameters through optimisation approaches is known as hyperparameter tuning [63]. PSO, a widely used metaheuristic optimisation method, is applied to tune the hyperparameters in our approach, as this has a low time complexity and can support a variety of hyperparameters.
After hyperparameter tuning has been carried out, training is conducted in order to achieve a high accuracy rate. In this stage, public comprehensive datasets mentioned above and the PSO method, which is a metaheuristic method and whose details are explained in the Proposed Method and Prior Schemes Used in the Suggested Method Sections, were used to minimize the problem of class imbalance in the training data and obtain high accuracy values. In addition, 20% of the intrusion data are used for testing, 10% for validation, and 70% for training.
Finally, an empirical analysis is carried out by passing comprehensive datasets as input to the trained model. A detailed explanation of the empirical analysis and the results from the model are provided in Section 4.
In summary, PSO is used for hyperparameter optimisation of the transfer learning-based model, and three pre-trained networks are combined using an equal weighting operation to produce feature vectors. The stages of the process are as follows. Firstly, intrusion data are gathered. Secondly, pre-processing is applied to the gathered data. The third step involves hyperparameter tuning, and the pre-training operation is then conducted. The ImageNet dataset [64] is used to train the Resnet50, GoogLeNet and AlexNet architectures in this step. Following this, 6144-dimensional combined feature vectors are generated by merging the features acquired from the ResNet50, AlexNet, and GoogLeNet models. These features are extracted from the final FC layers as 2048-dimensional vectors. To normalise the 6144-dimensional combined feature vectors, they are transmitted from the FC layers to the SoftMax layer. At this point, there are 18 outputs in the SoftMax layer, representing the 18 different types of intrusion, while the FC layers consist of 6144 nodes.
Prior schemes used in the suggested method
The schemes in the literature that were used to design the proposed intrusion detection architecture are reviewed in this section. In this section, we introduce three CNN architectures and the PSO algorithms used as a hyperparameter tuning method.
Particle swarm optimisation.
PSO is an algorithm for optimisation that draws inspiration from the collective behaviour of flocks of birds and fish schools. It aims to find the best solution to a problem based on the cooperation and communication of a group of potential solutions called particles [65] The particles in PSO represent potential solutions that move through a multidimensional search space. Each particle’s velocity determines the direction and magnitude of its movement, and its position represents a solution. Particles explore the search space by adjusting their velocities and positions based upon the best experience of the entire group and their own experience [66]. The algorithm begins by initialising particles with random velocities and positions. In each iterative step, the velocities and positions of the particles are updated based upon the best position found by all particles in the group and their own best positions. This update is affected by the collective behaviour of the group and individual experiences [67]. The velocity update process in PSO consists of two main parts, known as the social and cognitive constituents. The cognitive step guides particles towards their best positions, while the social step pulls them towards the best position found by any particle in the group. These components strike a balance between exploitation and exploration, allowing for effective searches of the solution space [68]. The PSO algorithm repeats until a termination criterion is met, such as convergence to a solution, reaching a desired fitness value or reaching a maximum number of iterations. The final positions of the particles reflect optimal solutions, or the best solution discovered [69].
CNN arhitectures.
ResNet-50 [59] is a CNN with a depth of 50 layers. The goal of the ResNet model is to resolve the problem of a decrease in performance with CNNs, which is done by adding shortcuts called residual learning blocks between the layers. These blocks, illustrated in Fig 8(a), can be considered the core structural components of ResNet. In a residual block, the path that results from adding the input x directly to the network’s output is referred to as a shortcut or jump link. The ResNet50 architecture, as shown in Fig 8(a), has 25.6 million parameters, and includes a SoftMax layer, two pooling operations, an FC layer and five convolutional blocks.
GoogLeNet [60] is a CNN with seven million process parameters and a depth of 27 layers, including pooling layers. It is also known as Inception, and is a deep CNN architecture developed by researchers at Google. It includes Inception modules, which use multiple convolutional filter sizes in parallel to capture features at different scales. This module consists of a shortcut branch and few deeper branches, and it provides the width item in the model to be obtained. The architecture of GoogLeNet [57], illustrated in Fig 9, aims to strike a balance between depth and computational efficiency by reducing the number of parameters. The architecture consists of four main sections: the output classifier, stacked, stem, and Inception auxiliary classifiers modules. In addition, this architectur e has 144 layers, including FC, convolution, max-pooling, output, SoftMax, input layers and ReLUs, and it contains nine Inception modules.
AlexNet [61] is a groundbreaking deep CNN architecture that played a crucial role in popularising deep learning for computer vision tasks. Developed by Alex Krizhevsky et al., it achieved remarkable success by winning the ImageNet Large Scale Visual Recognition Challenge in 2012 [70] This network, as shown in Fig 10, consists of seven ReLUs, two normalisation layers and three pooling layers following the convolution layers. In addition, it includes a SoftMax layer and FC layers for classification and learning.
Experimental results and discussion
The results of our experiments, an evaluation of the proposed model, the details of the implementation and a description of the datasets are presented in this section. The Python scripting language was used to implement our model, and experiments were performed in a Linux environment on a personal computer with 128 GB of RAM and Intel Core i9 12950HX processor with a speed of 5.2 GHz. The test, training and validation data were chosen randomly from the datasets, and evaluation operations were carried out separately. A total of 10% of the data were used for validation, 20% for testing, and 70% for training. Without GPU support, the training process was carried out for about 50 hours and stopped at 100 epochs. The hyperparameters selected for the experiments are shown in Table 2.
Benchmark datasets
The popular datasets that were used to test the performance of the proposed method are described in detail in this subsection. A detailed summary of the datasets and the classes of attack within each is given in Table 3. The details of the testing and training datasets used for the experiments are displayed in Tables 4 and 5.
KDDCUP’99 dataset.
KDDCUP’99 [23] is a widely used benchmark dataset for intrusion detection research. It was created as part of the 1999 Knowledge Discovery and Data Mining (KDD) Cup competition organised by the National Science Foundation and DARPA. This dataset contains a large number of network traffic records, both normal and malicious, which are used to train and evaluate IDSs [70]. In dataset, each record is characterized by 41 parameters and labeled either as normal traffic or an attack of a specific type.
The attacks in this dataset are classified into four primary categories: U2R (user to root), R2L (root to local), probe (probing attacks), DoS (denial of service). The training dataset also includes 24 specific types of intrusions with an additional 14 attacks in the testing dataset, which includes the aforementioned four main types of attack.
NSL-KDD dataset.
NSL-KDD [24] is a publicly available dataset that was created by building upon the KDDCUP’99 dataset. Important problems that can significantly impact the accuracy of intrusion detection and lead to a false assessment of AIDS were identified through a statistical analysis of the KDDCUP’99 dataset, with large numbers of duplicate packets being the primary issue. The creators of the NSL-KDD dataset addressed these problems in the KDD-CUP dataset by removing duplicate records from the training and testing sets and increasing the proportion of minority samples in the testing set. As discussed above, the KDDCUP’99 dataset has 41 features, and intrusion attacks are categorised into four types: U2R, R2L, Probe, and DoS.
UNSW-NB15 dataset.
UNSW-NB15 [25] was produced by the Australian Centre for Cyber Security, and includes roughly two million records with 49 features, extracted using Bro-IDS, Argus tools and some newly developed algorithms. This dataset includes nine types of intrusion attack: exploits, shellcode, DoS, reconnaissance, backdoor, generic, fuzzers, worms and port scans.
Results and discussion
The performance of DNN models is assessed based on evaluation metrics that are of crucial importance when evaluating classification processes. These metrics distinguish between model results and measure the performance of the classification model [71]. The classification performance of the proposed method was assessed using the F1-score, accuracy, specificity, sensitivity, the area under the curve of a receiver operating characteristic (ROC-AUC) metrics, and the results are presented in this subsection. The formulae in Table 6 were used to calculate these metrics, where TN and FN refer to true negative and false negative, respectively, while TP and FP refer to true positive and false positive.
The results for the sensitivity, specificity, F1-score, and accuracy metrics on the individual datasets for the ResNet152, GoogLeNet, and AlexNet models as well as the suggested network are shown in Fig 11. It can be seen that our model performs better than the other three DNN models. In addition, whereas the performances of the three DNNs differ significantly on the three datasets, our model produces similar performance results for all three, showing that our network outperforms the other three deep neural networks in terms of performance and robustness.
Following the models’ training over 100 epochs, the ROC-AUC curves were plotted, as seen in Fig 12, respectively. Plotting the true positive rate (TPR) against the false positive rate (FPR) at various classification thresholds yields the ROC-AUC curve. Test ROC-AUC values for the three models were 96.6%, 96.9%, and 97.4%, respectively.
Confusion matrices were also used to investigate the performance for various types of intrusion attacks. The confusion matrices for five intrusion attack types (U2R, R2L, Probe, DoS and normal traffic) of the ResNet50, GoogLeNet, AlexNet and proposed deep neural network models are shown in Fig 13 for the NSL-KDD dataset. Here, accuracy rates are shown for each type of intrusion attack through the use of confusion matrices. It can be seen from Fig 13(d) that the proposed approach produces better results than the other models for all types of intrusion apart from U2R. In addition, compared to the other network models, ResNet50, shown in Fig 7(c), achieves superior detection of the U2R attack. In this case, the Probe and R2L intrusion attack types were readily detected by each network.
Finally, a comparison was performed against state-of-the-art intrusion detection methods in order to assess the performance of our approach. The values for accuracy obtained on KDD CUP’99, NSL-KDD and UNSW-NB15 datasets for these state-of- the-art schemes and the proposed model are given in Table 7. We note that the suggested model is more accurate and efficient than the alternatives based on the higher value of accuracy.
Conclusion
Effective detection of various types of network intrusion continues to be an extremely difficult task in the area of cyber-security, despite the fact that a great deal of research has been done on intrusion detection. Cyberattacks are also increasing and evolving at exponential rates, and there is currently no method that can identify many of these contemporary cyber-related attacks. Complicated cyberattacks can no longer be distinguished from normal network traffic by traditional IDSs. Hence, this paper has presented a novel deep learning architecture for the efficient detection of several types of intrusion. The proposed architecture is a hybrid of three pre-trained networks based on transfer learning. Initially, intrusion data were gathered from three exhaustive datasets, and were pre-processed and subjected to data coding, standardisation and conversion processes. Next, high-level features were obtained using pre-trained networks. Hyperparameter adjustment was then carried out using PSO. Finally, training of the model was carried out using a supervised learning method.
Our work’s primary contribution is a novel deep learning-based hybrid architecture that combines three well-known pre-trained network models in an optimized manner while using PSO for hyperparameter optimization. The proposed deep learning technique was assessed on UNSW-B15, NSL-KDD and KDDCUP’99 datasets, and the performance of the proposed hybrid model was initially compared with each model in isolation. The findings demonstrated that our approach could successfully classify intrusion with high values for the F1-score, accuracy, recall and precision metrics. Next, our model was evaluated on the NSL-KDD dataset using confusion matrices, and it was shown to generate better results than the alternative models for all types of intrusion apart from U2R. Finally, a comparison between our architecture and state-of-the-art alternatives was performed, and results revealed that the proposed method was superior to the other methods. In future work, we plan to compare the performances of more models on different databases and employ other metaheuristic techniques for hyperparameter optimisation.
Acknowledgments
I would like to express our deepest gratitude to my mother Serap YILMAZ and my wife Süheyla YILMAZ for their unwavering support and help.
References
- 1. Zhang C, Zhang X, Wang H, Sun G, Chen W, Liu J. Comparative research on network intrusion detection methods based on machine learning. Computers and Security. 2022;121:102861.
- 2.
Hsu YF, He Z, Tarutani Y, Matsuoka M. Toward an online network intrusion detection system based on ensemble learning. In: Proceedings of the 2019 IEEE 12th International Conference on Cloud Computing. IEEE; 2019. pp. 174–178.
- 3. Jiang H, He Z, Ye G, Zhang H. Network intrusion detection based on pso-xgboost model. IEEE Access. 2020;8:58392–58401.
- 4. Devarakonda A, Sharma N, Saha P, Ramya S. Network intrusion detection: a comparative study of four classifiers using the NSL-KDD and KDDCUP’99 datasets. In: Journal of Physics: Conference Series. vol. 2161; 2022. pp. 012043.
- 5. Shone N, Ngoc TN, Phai VD, Shi Q. A deep learning approach to network intrusion detection. IEEE Transactions on Emerg Top Comput Intell. 2018;2(1):41–50.
- 6.
Niyaz Q, Sun W, Javaid A, Alam M. A deep learning approach for network intrusion detection system. In: Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies; 2018. pp. 21–26.
- 7.
Zhang C, Zhang X, Wang H, Sun G, Chen W, Liu J. A deep learning approach for network intrusion detection based on nsl-kdd dataset. In: Proceedings of the 2019 IEEE 13th International Conference on Anti-counterfeiting, Security, and Identification. IEEE; 2019. pp. 41–45.
- 8.
Shinde PP, Shah SA. A review of machine learning and deep learning applications. In: Proceedings of the 4th International Conference on Computing Communication Control and Automation. IEEE; 2018. pp. 1–6.
- 9.
Cao ND, Kipf T. Molgan: An implicit generative model for small molecular graphs. arXiv preprint. 2018;.
- 10. Yilmaz AA. A novel hyperparameter optimization aided hand gesture recognition framework based on deep learning algorithms. Traitement du Signal. 2022;39:823–833.
- 11. Wang D. Intelligent detection of vehicle driving safety based on deep learning. Wirel Commun Mob Comput. 2022; pp. 1–11.
- 12.
Yilmaz AA, Guzel MS, Askerzade I, Bostanci E. A vehicle detection approach using deep learning methodologies. In: Proceedings of International Conference on Theoretical and Applied Computer Science and Engineering; 2018. pp. 64–71.
- 13. Yilmaz AA, Guzel MS, Bostanci E, Askerzade I. A novel action recognition framework based on deep-learning and genetic algorithms. IEEE Access. 2020;8:100631–100644.
- 14.
Yilmaz AA, Guzel MS, Askerzade I, Bostanci E. A hybrid facial emotion recognition framework using deep learning methodologies. USA: Nova Science Publishers; 2020.
- 15. Maas AL, Hannun AY, Ng AY. Building dnn acoustic models for large vocabulary speech recognition. Comput Speech Lang. 2017;41:195–213.
- 16.
Schwenk H. Continuous space translation models for phrase-based statistical machine translation. In: Proceedings of 24th International Conference on Computational Linguistics (COLIN); 2012. pp. 1071–1080.
- 17. Aslan O, Yilmaz AA. A new malware classification framework based on deep learning algorithms. IEEE Access. 2021;9:87936–87951.
- 18.
Yılmaz AA, Bagdat B. A comparative study on various intrusion detection techniques using machine learning and deep learning algorithms. In: Proceedings of 2023 11th International Congress of Academic Research; 2023. pp. 386–395.
- 19. Yılmaz AA. A machine learning-based framework using the particle swarm optimization algorithm for credit card fraud detection. Commun Fac Sci Univ Ankara Ser A2-A3 Phys Sci Eng. 2023;66:82–94.
- 20.
Yılmaz AA. A novel deep learning-based model for breast cancer detection. In: Proceedings of 2023 1st International Congress on Solutions in Science; 2023.
- 21. Gu B, Ge R, Chen Y, Luo L, Coatrieux G. Automatic and robust object detection in x-ray baggage inspection using deep convolutional neural networks. IEEE Transactions on Industrial Electronics. 2021;68:10248–10257.
- 22. Aslan O, Aktuğ SS, Ozkan Okay M, Yilmaz AA, Akin E. A comprehensive review of cybersecurity vulnerabilities, threats, attacks, and solutions. Electronics. 2023;12:1333.
- 23.
Bay S. The UCI KDD Archive;. Available from: http://kdd.ics.uci.edu/.
- 24.
Tavallaee M, Bagheri E, Lu W, Ghorbani AA. A detailed analysis of the KDD Cup 99 data set. In: Proceedings of the IEEE Symposium on Computational Intelligence for Security and Defense Applications; 2009. pp. 1–6.
- 25.
Moustafa N, Slay J. UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network dataset). In: Proceedings of the Military Communications and Information Systems Conference; 2015. pp. 1–6.
- 26. Ahmad Z, Shahid Khan A, Wai Shiang C, Abdullah J, Ahmad F. Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Transactions on Emerging Telecommunications Technologies. 2021;32:1–29.
- 27.
Mukkamala S, Janoski G, Sung A. Intrusion detection using neural networks and support vector machines. In: Proceedings of the 2002 International Joint Conference on Neural Networks; 2002. pp. 1702–1707.
- 28. Verwoerd T, Hunt R. Intrusion detection techniques and approaches. Computer Communications. 2002;25:1356–1365.
- 29. Ravale U, Marathe N, Padiya P. Feature selection based hybrid anomaly intrusion detection system using K-means and RBF kernel function. In: Procedia Computer Science. vol. 45; 2015. pp. 428–435.
- 30. Al-Yaseen WL, Othman ZA, Nazri MZA. Multi level hybrid support vector machine and extreme learning machine based on modified K-means for intrusion detection system. Expert Systems with Applications. 2017;67:296–303.
- 31. Yang C. Anomaly network traffic detection algorithm based on information entropy measurement under the cloud computing environment. Cluster Computing. 2018; pp. 1–9.
- 32.
Farahnakian F, Heikkonen J. A deep auto-encoder based approach for intrusion detection system. In: Proceedings of the 2018 20th International Conference on Advanced Communication Technology (ICACT); 2018. pp. 178–183.
- 33. Gao X, Shan C, Hu C, Niu Z, Liu Z. An adaptive ensemble machine learning model for intrusion detection. IEEE Access. 2019;7:82512–82521.
- 34.
Krishna A, Lal A, Mathewkutty AJ, Jacob DS, Hari M. Intrusion detection and prevention system using deep learning. In: Proceedings of the 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC); 2020. pp. 273–278.
- 35.
Nayyar S, Arora S, Singh M. Recurrent neural network-based intrusion detection system. In: Proceedings of the 2020 International Conference on Communication and Signal Processing (ICCSP); 2020. pp. 0136–0140.
- 36.
Chen L, Kuang X, Xu A, Suo S, Yang Y. A novel network intrusion detection system based on CNN. In: Proceedings of the 2020 Eighth International Conference on Advanced Cloud and Big Data (CBD); 2020. pp. 243–247.
- 37. Issa ASA, Albayrak Z. DDOS attack intrusion detection system based on hybridization of CNN and LSTM. Acta Polytechnica Hungarica. 2023;20:1–9.
- 38. Qazi EUH, Faheem MH, Zia T. HDLNIDS: Hybrid deep learning-based network intrusion detection system. Applied Sciences. 2023;13:4921.
- 39.
Yilmaz AA. Intrusion detection in computer networks using optimized machine learning algorithms. In: Proceedings of the 2022 3rd International Informatics and Software Engineering Conference; 2022. pp. 1–5.
- 40. Singh R, Kumar H, Singla RK. An intrusion detection system using network traffic profiling and online sequential extreme learning machine. Expert Systems with Applications. 2015;42:8609–8624.
- 41. Costa KAP, Papa JP, Lisboa CO, Munoz R, Albuquerque VHC. Anomaly detection using machine learning techniques. Computer Networks. 2019;151:147–157.
- 42. Buczak AL, Guven E. A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys and Tutorials. 2016;18:1153–1175.
- 43. Aburomman A, Bin Ibne Reaz AM. A novel SVM-KNN-PSO ensemble method for intrusion detection system. Applied Soft Computing. 2016;46:360–372.
- 44.
Enache AC, Patriciu VV. Intrusions detection based on support vector machine optimized with swarm intelligence. In: Proceedings of the 9th IEEE International Symposium on Applied Computational Intelligence and Informatics; 2014. pp. 153–158.
- 45.
Miikkulainen Rea. Artificial intelligence in the age of neural networks and brain computing. Academic Press; 2018.
- 46. Wang Z. The applications of deep learning on traffic identification. BlackHat USA. 2015;24:1–10.
- 47. Sun Pea. DL-IDS: Extracting features using CNN-LSTM hybrid network for intrusion detection system. Security and Communication Networks. 2020;2020.
- 48. Fan J, Ling-zhi K. Intrusion detection algorithm based on convolutional neural network. Beijing Institute of Technology. 2017;.
- 49.
Kim J, Kim J, Thu HLT, Kim H. Long short-term memory recurrent neural network classifier for intrusion detection. In: Proceedings of the 2016 International Conference on Platform Technology and Service; 2016. pp. 1–5.
- 50. Hassan MM, Gumaei A, Ahmed A, Alrubaian M, Fortino G. A hybrid deep learning model for efficient intrusion detection in big data environment. Information Sciences. 2020;513:386–396.
- 51. Kim G, Lee S, Kim S. A novel hybrid intrusion detection method integrating anomaly detection with misuse detection. Expert Systems with Applications. 2014;41:1690–1700.
- 52. Smys S, Basar A, Wang H. Hybrid intrusion detection system for Internet of Things (IoT). Journal of ISMAC. 2020;2:190–199.
- 53. Khari M, Karar A. Analysis on intrusion detection by machine learning techniques: A review. International Journal of Advanced Research in Computer Science and Software Engineering. 2013;3:1–4.
- 54. Maseno EM, Wang Z, Xing H. A systematic review on hybrid intrusion detection system. Security and Communication Networks. 2022;2022.
- 55. Alghayadh F, Debnath D. A hybrid intrusion detection system for smart home security based on machine learning and user behavior. Advances in Internet of Things. 2021;11:10–25.
- 56. Saheed YK, Kehinde TO, Ayobami Raji M, Baba UA. Feature selection in intrusion detection systems: A new hybrid fusion of Bat algorithm and Residue Number System. Journal of Information and Telecommunication. 2023;8:189–207.
- 57. Abdulganiyu OH, Tchakoucht TA, Saheed YK. XIDINTFL-VAE: XGBoost-based intrusion detection of imbalance network traffic via class-wise focal loss variational autoencoder. Journal of Supercomputing. 2025;81:16.
- 58. Saheed YK, Abdulganiyu OH, Tchakoucht TA. Modified genetic algorithm and fine-tuned long short-term memory network for intrusion detection in the Internet of Things networks with edge capabilities. Applied Soft Computing. 2024; pp. 111434.
- 59.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. pp. 770–778.
- 60.
Szegedy Cea. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015. pp. 1–9.
- 61.
Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks. In: Proceedings of the International Conference on Neural Information Processing Systems; 2012. pp. 1097–1105.
- 62.
Kim H, Benson T, Akella A, Feamster N. The evolution of network configuration: A tale of two campuses. In: Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference IMC’11; 2011. pp. 499–514.
- 63.
Yang L, Shami A. A transfer learning and optimized CNN based intrusion detection system for internet of vehicles. In: Proceedings of the IEEE International Conference on Communications; 2022. pp. 2774–2779.
- 64.
Dataset TI. The ImageNet Dataset;. http://www.image-net.org/.
- 65. Khan SU, Yang S, Wang L, Liu L. A modified particle swarm optimization algorithm for global optimizations of inverse problems. IEEE Transactions on Magnetics. 2016;52:1–4.
- 66. Selvi V, Umarani R. Comparative analysis of ant colony and particle swarm optimization techniques. International Journal of Computer Applications. 2010;5:1–6.
- 67.
Shi Y, Eberhart RC. Empirical study of particle swarm optimization. In: Proceedings of the 1999 Congress on Evolutionary Computation; 1999. pp. 1945–1950.
- 68.
Krohling RA. Gaussian swarm: A novel particle swarm optimization algorithm. In: Proceedings of the IEEE Conference on Cybernetics and Intelligent Systems; 2004. pp. 372–376.
- 69. Bai Q. Analysis of particle swarm optimization algorithm. Computer and Information Science. 2010;3:180.
- 70. Russakovsky Oea. ImageNet large scale visual recognition challenge. International Journal of Computer Vision. 2015;115:211–252.
- 71. Saranya T, Sridevi S, Deisy C, Chung TD, Khan MKAA. Performance analysis of machine learning algorithms in intrusion detection system: A review. In: Procedia Computer Science. vol. 171; 2020. pp. 1251–1260.
- 72. Vinayakumar Rea. Deep learning approach for intelligent intrusion detection system. IEEE Access. 2019;7:41525–41550.
- 73.
Vinayakumar R, Soman KP, Poornachandran P, Akarsh S. In: Application of deep learning architectures for cyber security. Springer; 2019. pp. 125–160.
- 74.
Li Z, Qin Z, Huang K, Yang X, Ye S. Intrusion detection using convolutional neural networks for representation learning. In: Neural Information Processing. Springer; 2017. pp. 858–866.
- 75. Yin C, Zhu Y, Fei J, He X. A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access. 2017;5:21954–21961.
- 76. Ogundokun ROea. An enhanced intrusion detection system using particle swarm optimization feature extraction technique. In: Procedia Computer Science. vol. 193; 2021. pp. 504–512.