Machine learning in intrusion detection systems

The rise in the complexity of cyber-attacks, including DDoS attacks, Advanced Persistent Threats (APTs), and botnets, has fueled the use of machine learning (ML) techniques for the detection of new and complex cyber attacks in an automated manner [4,5]. Although the early benchmark datasets, such as KDD Cup ‘99 and its improved version NSL-KDD, were designed for the implementation of supervised learning techniques such as SVM and Random Forest algorithms, they were found to be ineffective in dealing with the new diversified traffic patterns [5,17]. In contrast, the latest flow-based datasets, including CICIDS2017, BoT-IoT, and IoTID20, are designed to reflect the new realities of enterprise and IoT environments and allow the implementation of advanced techniques such as the use of a hybrid of clustering and classification techniques [18], statistical feature selection techniques [19], and deep learning hybrids [8,20–22].

Persistent challenges

In spite of these advancements, some of the limitations persist in ML-based IDS research, which are as follows:

• Cross-dataset generalization: The ML models are often trained on a dataset and fail to generalize on other network environments [21,22].
• Feature inconsistency: The lack of standardization in feature definition and preprocessing makes it difficult to compare and reproduce results across various studies [21,22].
• High-dimensional and imbalanced data: The performance of unsupervised and anomaly-based ML models is not satisfactory on large and highly imbalanced data sets containing various attack classes [17,21,23].
• Limited auditability: In ML-based IDS, there is a lack of incorporation of tamper-evident and secure auditing mechanisms for traceability of results and experiments [14].

Recent research directions

Recent research works have addressed these issues. Using lightweight feature selection and genetic algorithms/correlation-based filtering, detection rates have surpassed 99% on IoT datasets [8,19]. Metaheuristics such as PSO, Jaya, and SSA have shown improvement in convergence and generalization capabilities of LSTM and CNN-LSTM architectures [9]. Ensemble and hybrid methods using kNN, SVM, RF, and deep autoencoders have shown robust performance on various datasets [24,25]. Quantum-inspired least square SVM and optimized DTL have shown very high accuracy rates, often 99% and beyond, on NSL KDD and CICIDS datasets [10,26].

Recent studies have been conducted to investigate the integration of blockchain technology with ML for improving the secure processing of data in Internet of Medical Things environments. For example, the application of blockchain-based architectures with SVM-based models was found to enhance the robustness and integrity of the secure processing of data in healthcare-based IDS environments [27].

In the same context, new authentication frameworks based on blockchain technology and integrated with edge computing were proposed for improving scalability, security, and real-time processing capabilities for Internet of Medical Things environments [28].

However, there are critical limitations that are evident in such studies. For instance, although Altulaihan et al. (2024) [8] attained 99.94–100% accuracy in the evaluation of the proposed DT/RF with GA and CFS on the IoTID20 dataset, the evaluation was conducted on a single dataset without cross-validation across different datasets. Moreover, although Dash et al. (2025) [9] attained 97.89–98.95% recall in the evaluation of the proposed optimized LSTM on NSL-KDD, CICIDS2017, and BoT-IoT datasets, the evaluation was conducted without rigorous statistical evaluation and auditability.

The emerging trends in AI-based intrusion detection systems focus on incorporating features such as transparency and trust into the system. These include privacy-preserving federated learning [29,30], explainable AI via SHAP and LIME [14,16,31], and lightweight cryptographic tamper-evident logging [14,15].No comprehensive unified framework yet exists that simultaneously integrates cross-dataset feature harmonization, multi-paradigm benchmarking (supervised, unsupervised, deep, ensemble), rigorous statistical validation, and tamper-evident experiment logging.

Emerging approaches combining generative artificial intelligence with intelligent monitoring systems further demonstrate the convergence of AI-driven technologies in healthcare and cyber-physical environments, enabling more adaptive and personalized security and decision-making systems [32]

These recent developments highlight the growing convergence of machine learning, blockchain, and AI-driven security mechanisms. However, the issues of cross-dataset generalization, evaluation standards, and experimental reproducibility are not sufficiently addressed, thus necessitating the development of a unified evaluation framework for IDS.

Research gaps and positioning of the present work

The majority of the existing works are highly dependent on single dataset evaluation, repeat preprocessing of each dataset, and under-emphasize the role of unsupervised methods in high-dimensional and imbalanced scenarios [14,21,19,33,34]. Table 1 presents a comparative analysis of some of the latest works in the area of IDS, including the datasets, performance, innovations, and limitations of each work.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparative analysis of selected IDS studies.

https://doi.org/10.1371/journal.pone.0346982.t001

Table 1, reveals a clear pattern: while individual-dataset accuracies are often impressive, cross-dataset generalization, standardized preprocessing, unsupervised stability, and experimental auditability are rarely addressed together. This work addresses these gaps by proposing a semantic feature harmonization pipeline to merge NSL-KDD and CICIDS2017, supporting unified multi-paradigm evaluation, feature selection based on mutual information, repeated cross-validation with statistical significance testing, and light-weight SHA-256 hash-chained tamper-evident logging for full reproducibility and traceability.

Research objectives

The primary objective was to build binary models for intrusion detection using various paradigms of machine learning and evaluate them using a unified dataset derived from NSL-KDD and CICIDS2017 datasets. The secondary objective was to incorporate a simple cryptographic mechanism for traceable configuration and prediction without exaggerating the idea of blockchain technology [37].

Framework overview

The framework integrates heterogeneous benchmark datasets, multiple learning paradigms, and reproducible evaluation via cryptographic logging. The workflow comprises six main stages:

1. Data acquisition and preprocessing
2. Cross-dataset feature harmonization
3. Feature selection and normalization
4. Model development across paradigms
5. Cross-validation and cross-dataset evaluation
6. Tamper-evident logging of metadata and predictions

Fig 1 illustrates the system-level architecture. Both research design and implementation pipelines enforce a unified feature representation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Proposed system architecture.

https://doi.org/10.1371/journal.pone.0346982.g001

Two popular intrusion detection datasets were used. NSL-KDD is an improved variant of the KDD Cup ‘99 dataset, designed to reduce redundancy and class imbalance, which reduces data redundancy and imbalance. It contains connection-oriented features such as protocol type, service, flag, and calculated statistics. It is based on the KDD Cup ‘99 dataset but is improved to reduce data imbalance and redundancy. CICIDS2017 is a recent flow-based dataset that represents the characteristics of modern enterprise network traffic and various types of modern attacks, such as denial of service, brute force, infiltration, and web attacks. It contains time and statistical features derived from packet capture data. For the purpose of experimental evaluation, 100,000 instances were used, which were derived from the data.

Data acquisition

Two widely used intrusion detection datasets were selected:

• NSL-KDD [11]: An improved variant of KDD Cup ‘99, reducing redundancy and imbalance. It contains connection-oriented features (e.g., protocol type, service, flag, derived statistics) and serves as a legacy benchmark for historical comparison.
• CICIDS2017 [12]: It is a recent flow-based data set that represents modern enterprise world attacks and normal traffic, such as DoS, brute force, infiltration, and web attacks. It also includes time and statistical features based on real-world packet captures, making it relevant to recent network environments.

The datasets were selected for this study based on their complementary characteristics. NSL-KDD is a standardized data set for representing traditional world attacks and normal traffic, while CICIDS2017 represents recent world attacks and normal traffic. These characteristics make it possible to perform robust cross-era generalization tests.

Per-dataset class distribution (before merging)

The class distribution of each data set is presented before merging, shown in Table 2, in order to highlight the characteristics of class imbalance in each data set. It is evident that NSL-KDD is well-balanced after preprocessing, whereas CICIDS2017 is highly imbalanced in its class distribution, with more normal traffic instances.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Class distribution before merging.

https://doi.org/10.1371/journal.pone.0346982.t002

For computational purposes and to ensure a balanced representation in both merged experiments, 100,000 instances were sampled (50,000 from each dataset after stratification by class), given that NSL-KDD training data has ~ 126,000 records and CICIDS2017 has over 2.8 million flows [sources: UNB CIC Datasets documentation].

Attack-category breakdown (before binarization)

Attack instances were categorized into different categories prior to binarization to ensure interpretability of the diversity of attacks (Table 3). NSL-KDD data has categories that include DoS, Probe, R2L, U2R, among others. CICIDS2017 data has more recent categories of attacks that include DoS/DDoS, brute force attacks, infiltration attacks, web attacks, etc.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Attack category distribution (pre-binarization).

https://doi.org/10.1371/journal.pone.0346982.t003

Cross-dataset feature harmonization

The NSL-KDD and CICIDS2017 datasets differ significantly in their feature definition and granularity. A semantic alignment approach aligned semantically similar features based on their shared behavioral interpretation. For example, traffic volume features such as “src_bytes” (in NSL-KDD, cumulative) and “Flow Bytes/s” (in CICIDS2017, normalized to a rate) are aligned since they both describe patterns of intensity associated with volumetric attacks. To reduce distortions caused by structural differences, each dataset was normalized individually before fusion (Table 4).

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Cross-dataset feature harmonization.

https://doi.org/10.1371/journal.pone.0346982.t004

To minimize the degree of subjectivity involved in the manual process of semantic alignment, feature mapping was facilitated by the application of domain knowledge and quantitative validation metrics. In addition to utilizing mutual information as a validation metric for feature mapping, we ensured statistical consistency by verifying the behavior of the features after normalization for various datasets. Only features were considered for inclusion if they showed similar statistical properties and had a positive effect on classification performance during the ablation analysis.

Key transformations included:

• Cumulative features → rate approximation: eq. (1) rate ≈ cumulative feature / duration (where duration is in seconds; this approximates intensity without exact normalization).
• Log transformation for skewed distributions: eq. (2), reduces impact of extreme values common in traffic counts).
• Per-dataset Z-score normalization: eq. (3), computed separately per dataset to avoid magnitude bias from different scales).
• Optional post-merge min–max scaling to ensure stability of the model: eq. (4), only if necessary to support gradient-based models like LSTM).

The features that were mapped were retained if they had a mutual information value greater than 0.01 with respect to the class label. Also, features were retained if ablation removal resulted in a reduction of more than 2% in F1 scores. This resulted in a total of 25 numeric features that were harmonized. The labels were binary (attack = 1, benign = 0). Missing and infinite values were removed, and SMOTE oversampling was done on training folds only.

Mapping and transformation

For features representing the same behavioral concept but different scales. The cumulative feature is converted into a rate using Eq. (1):

Cumulative → Rate

(1)

where is the cumulative feature (e.g., src_bytes) and is the duration in seconds. To reduce skewness, a logarithmic transformation is applied as shown in Eq. (2):

Log transformation for skewed distributions

(2)

The normalized feature is obtained using Eq. (3):

(3)

After these transformations (Eqs. (1)–(3)), datasets are merged into a unified feature space. To improve model stability, min–max scaling is optionally applied using Eq. (4):

(4)

Mapped features were retained only if:

• Mutual Information > 0.01 with class label
• Removal caused ≥2% drop in F1 during ablation testing

This resulted in 25 harmonized numeric features.

• Labels binarized: attack = 1, benign = 0
• Missing/ ∞ values removed
• SMOTE oversampling applied only to training folds to address class imbalance
• Test folds remained untouched to avoid data leakage

Feature selection and preprocessing

A unified preprocessing pipeline was used for both data sets. The attack labels were binarized (attack = 1, benign = 0), and data points containing missing or infinite values were removed for numerical consistency. Feature reduction was achieved by using mutual information between each feature X _j and the class label Y [38]: Feature reduction was performed using mutual information, defined in Eq. (5):

(5)

Features satisfying were retained.

The mutual information was calculated based solely on the training data to avoid any data leakage issue. This resulted in a concise unified feature set consisting of 25 harmonized numerical features.In addition, for reproducibility purposes, all transformations involved in the preprocessing stage, including rate conversion, log scaling, and normalization, are defined and applied consistently for all datasets. The entire feature mapping and preprocessing pipeline is publicly available in the repository for easier verification and reproduction.

Data normalization

Normalization was carried out in two phases. First, min–max scaling normalization was applied to the data sets individually before fusion. Then, the Z-score normalization method was applied to the unified feature space. Z-score normalization was applied to the unified feature space using Eq. (6):

(6)

This ensures that there is zero mean and unit variance before any actual model training. For avoiding any form of data leakage, there is a strict enforcement of training first for any form of normalization and scaling. For every fold of cross-validation, normalization parameters, such as mean and standard deviation for z-score standardization, will be calculated on the respective training set and then used on the respective test set. There is no usage of any form of test set or validation set for calculating parameters. The estimation of mutual information is done on the training set to avoid any form of data leakage.The order of operations for every step of the preprocessing pipeline is as follows: first, there is data cleaning and label binarization, then feature harmonization and transformation, feature selection using mutual information on the training set, and finally normalization and scaling.

Model development

Models were drawn from four families:

• Supervised: Gaussian Naïve Bayes [39], SVM [40], Random Forest [41].
• Unsupervised: K-Means [42], DBSCAN [43], Isolation Forest [44] (outputs mapped to binary labels).

1. Deep Learning: Bidirectional LSTM [45] —temporal continuity is limited, as both datasets offer only flow-level tabular records and not actual packet sequences; short pseudo-sequences (10 flows per sequence, grouped by source/destination IP and ordered by timestamp) were used exploratorily.
2. Ensemble: Soft Voting of NB, SVM, RF.

Hyperparameters for all models were set using a predefined space and were optimized solely based on the training set to prevent data leakage. A Grid Search was used for traditional models (e.g., RF, SVM), while Heuristic/Default settings with minor tuning were used for Deep Learning models due to computational constraints. Hyperparameter ranges were defined based on standard practices found in the literature and are fully documented for reproducibility (GitHub and Zenodo repository links).

Table 5 summarizes the hyperparameter ranges used for each model along with their actual values used for training.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Hyperparameter configuration and search space for evaluated models.

https://doi.org/10.1371/journal.pone.0346982.t005

Mathematical formulation

Supervised models.

Gaussian Naïve Bayes (NB): Assuming conditional independence among features, the posterior probability for class c. The posterior probability is computed using Eq. (7):

(7)

Bayes’ theorem with Gaussian likelihoods to compute class probabilities under conditional independence [42].

SVM Decision Function: The decision function is defined in Eq. (8):

(8)

Where ϕ(⋅) represents the mapping induced by the kernel, w is the weight vector, and b is the bias term, to separate classes in a high-dimensional space using a maximum-margin hyperplane [26].

Random Forest: The predicted label is obtained from the majority vote of T decision trees. Predictions are obtained using majority voting as shown in Eq. (9):

(9)

Where h _t denotes the t – th tree classifier. Aggregates predictions from multiple decision trees to reduce variance [26].

Unsupervised models.

K-Means Clustering: Cluster assignment is obtained by minimizing the within-cluster sum of squares. Cluster assignment is performed by minimizing intra-cluster variance using Eq. (10):

(10)

This method divides the data into k clusters while aiming to reduce the variance within each cluster [37].

DBSCAN: Points are categorized into three types: core, border, or noise, depending on the density of their ε-neighborhood [38]. Neighborhood definition is given by Eq. (11):

(11)

where indicates a core point. Identifies dense regions for clustering, tolerant to noise and irregular shapes.

Isolation Forest: Anomaly score is derived from the average path length E[h(x)] across random trees. The anomaly score is computed using Eq. (12):

(12)

Where c(n), is the average path length during unsuccessful searches in binary trees. It identifies anomalies by looking for shorter isolation paths in random trees.

Deep Learning (LSTM): A bidirectional LSTM model was evaluated in an exploratory manner. Since both NSL-KDD and CICIDS2017 consist of flow-level tabular records rather than true temporal packet sequences, explicit temporal continuity is limited. To enable sequence-based input, pseudo-sequences were constructed by grouping flows based on source–destination IP pairs and ordering them by timestamp where available. Fixed-length sequences of 10 flows were then formed.

It is important to note that this construction does not represent true temporal dependencies, particularly for NSL-KDD, where temporal granularity is limited. As a result, the LSTM model should be interpreted as an exploratory baseline rather than a fully representative temporal model. This limitation is considered when analyzing its performance relative to tree-based methods.Because both datasets are flow-based tabular records, temporal continuity is limited. A bidirectional LSTM was tested exploratorily by forming short sequences of flows grouped by source–destination pairs ordered by timestamp. This did not assume true packet-level sequence structure, which explains lower performance compared to tree models.

Pseudo-sequences of 10 flows are generated per source–destination IP. The LSTM operations are defined through Eqs. (13)–(16):

(13)(14)(15)(16)

Where is input, previous hidden state, previous cell state.

Evaluation protocol

Model performance was assessed using repeated stratified 5-fold cross-validation and cross-dataset transfer testing. Metrics were averaged across folds. All metrics (Eqs. (17)–(20)) are computed across cross-validation folds and averaged.

(17)

(18)

(19)

(20)

Statistical analysis

Performance differences between models were evaluated using the Wilcoxon signed-rank test [46], McNemar’s test [47], and DeLong’s test for comparing AUC values [48]. Statistical significance was determined at .

Cryptographic tamper-evident logging

Experimental configurations and prediction outputs were recorded using a SHA-256 hash-chained logging mechanism [37]. The logging mechanism is defined using Eq. (21):

(21)

Each entry includes the hash of the previous entry, thus creating an append-only ‘tamper-evident’ log. This process provides integrity verification and reproducibility, but it does not provide decentralization or consensus in any form. This process provides integrity, reproducibility, and auditability without any implication of decentralization or consensus in any form. This methodological approach is identified as cryptographic tamper-evident logging despite its association with blockchain technology. This methodological approach provides full reproducibility, completeness in terms of models, and traceability in accordance with the best practices and procedures of accountable machine learning in intrusion detection systems.

The proposed logging mechanism operates under a defined threat model in which logs are generated and maintained within a single trusted administrative domain. The objective is to provide tamper-evident integrity of experiment metadata and predictions. Any post-hoc modification of a log entry disrupts the hash chain and can be detected during verification.

However, this does not protect from truncation and replay attacks unless further measures such as key-based authentication, external anchoring, and/or independent auditing mechanisms are in place. As it is, this design does not employ digital signatures and timestamping; hence, it is more accurate to interpret this as an integrity measure for controlled environments and experiments.

Ethical considerations

All data used is publicly available and anonymized. Only experimental metadata and prediction results are used in logging; no sensitive network data is stored. We use hash chaining for integrity without exposing data externally. All models were created and tested offline; no real-time deployment and/or online testing were conducted. The framework is designed for responsible and transparent research in cybersecurity, in line with responsible AI and data privacy best practices.

In-domain model performance

The accuracy and efficiency of the Random Forest classifier were consistently high for the combined harmonized dataset, with an accuracy of 98.0% ± 0.6% and an F1-score of 97.0% ± 0.7%. This demonstrates an excellent balance of efficiency and robustness for network flow classification using tabular data. In addition, the soft voting ensemble method (NB + SVM + RF) performed competitively at 98.0% ± 0.8%. Tree-based methods were found to perform exceptionally well for their ability to handle non-linear relationships and feature interactions for high-dimensional traffic data.

Repeated 5-fold cross-validation was applied to the harmonized merged dataset (~100,000 samples balanced via SMOTE). Table 6 shows class distributions, while Tables 7–9 summarize performance and efficiency metrics. As shown in Table 3, the datasets presented a level of class imbalance, which prompted the need for the application of SMOTE, ensuring that all classes, major and minor, are sufficiently represented in the evaluation process. As presented in Table 7, per-class evaluation indicated differences in precision, recall, and F1-score, while the confusion matrix indicated which classes were most likely misclassified by the classifier. Table 8 presents the weighted and macro averages of all the metrics, giving a clear view of the general performance of the classifier on all the classes. For anomaly-based models, ROC-AUC is emphasized as a primary metric due to its suitability for evaluating continuous anomaly scores.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Class distribution of cybersecurity datasets.

https://doi.org/10.1371/journal.pone.0346982.t006

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Class-wise metrics and confusion matrix (random forest, merged dataset).

https://doi.org/10.1371/journal.pone.0346982.t007

Download:

• PNG
  larger image
• TIFF
  original image

Table 8. Weighted average metrics for top models (merged dataset).

https://doi.org/10.1371/journal.pone.0346982.t008

Download:

• PNG
  larger image
• TIFF
  original image

Table 9. Training time and efficiency trade-offs.

https://doi.org/10.1371/journal.pone.0346982.t009

Other than the accuracy of the classifier, Performance is reported as mean ± standard deviation across cross-validation folds, providing an estimate of variability and confidence in the results.

Computational efficiency is evaluated in terms of training time, as summarized in Table 10. In addition to predictive performance, computational cost was evaluated in terms of training time and relative complexity. Tree-based models such as Random Forest provided the best trade-off between accuracy and efficiency, with low-to-moderate training time. In contrast, deep learning models (e.g., LSTM).

Download:

• PNG
  larger image
• TIFF
  original image

Table 10. Class distribution in merged dataset (before and after SMOTE).

https://doi.org/10.1371/journal.pone.0346982.t010

However, the computational cost was significantly higher due to iterative training and optimization of parameters, even though the performance was lower in the tabular environment.All experiments were conducted in a workstation environment (CPU-based execution), and no special hardware acceleration was required. Fig 2 presents a plot of the training accuracy against the validation accuracy, showing the variance in accuracy between folds and stability during the training process, indicating that the classifier was able to learn without overfitting or underfitting.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Training vs validation accuracy across folds.

https://doi.org/10.1371/journal.pone.0346982.g002

Merged dataset (before vs after SMOTE)

To ensure balanced evaluation, class distributions were examined before and after sampling(Table 10). The merged dataset initially reflects the imbalance inherited from CICIDS2017; therefore, SMOTE and undersampling were applied within training folds to achieve class balance without introducing data leakage.

SMOTE was applied only to the training data within each cross-validation fold, while validation and test data remained unchanged to prevent data leakage.

Besides the predictive performance, the computational cost in terms of training time and relative complexity was also considered. For instance, tree-based models like Random Forest showed optimal performance in terms of trade-off between performance and cost, where the training time is low to moderately high. On the contrary, deep learning models like LSTM showed high computational cost in terms of training and optimization of parameters, along with relatively low performance in this tabular data setting. These results are further affected by the fact that there is no real temporal dependency in the underlying data, especially in NSL-KDD data, which affects sequence-based learning. All experiments were conducted on a CPU-based workstation without any hardware acceleration.

Fig 2 shows training vs. validation accuracy across folds, confirming low variance and no overfitting.

Cross-dataset generalization

In the case of cross-dataset generalization, the results from models trained on NSL-KDD and tested on CICIDS2017, and vice versa, demonstrated significant differences, as presented in Table 11, which compares the accuracy, F1 score, and ROC AUC results between the models. This also demonstrated the inherent performance degradation when the model encounters a different dataset with different distributions or representations, as further presented in Table 7, which identifies the metrics that were most affected by the degradation. Table 12 presents the results using cross-validation on each dataset separately, emphasizing that the degradation is not due to the instability of the models but rather the differences between the datasets. Table 13. present the performance drop analysis.

Download:

• PNG
  larger image
• TIFF
  original image

Table 11. Cross-dataset generalization.

https://doi.org/10.1371/journal.pone.0346982.t011

Download:

• PNG
  larger image
• TIFF
  original image

Table 12. Cross-validation results on individual datasets.

https://doi.org/10.1371/journal.pone.0346982.t012

Download:

• PNG
  larger image
• TIFF
  original image

Table 13. Cross-dataset performance drop.

https://doi.org/10.1371/journal.pone.0346982.t013

The results also demonstrated that the degradation can be somewhat addressed through feature harmonization, and the results from all three scenarios (NSL ONLY, CIC ONLY, and MERGED) are averaged and presented in Table 9, which further demonstrated the robustness of the models. Fig 3 presents the ROC curves on each dataset using cross-dataset results, demonstrating the degradation in discriminative ability when using different datasets, and Fig 4 compares the results using different approaches, which clearly demonstrates the benefits and trade-offs of using feature harmonization. Table 14 below shows a confusion matrix for the Random Forest model. Although the exact figures are not given, it is assumed based on the high-performance results in the evaluation section.

Download:

• PNG
  larger image
• TIFF
  original image

Table 14. Confusion matrix (aggregated) – Random forest (on harmonized merged dataset).

https://doi.org/10.1371/journal.pone.0346982.t014

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Cross-dataset ROC curves.

https://doi.org/10.1371/journal.pone.0346982.g003

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Performance comparison bar chart (within vs cross dataset).

https://doi.org/10.1371/journal.pone.0346982.g004

For the best-performing Random Forest model on the harmonized merged dataset, a representative confusion matrix (averaged across folds) is as follows (approximate values based on high accuracy/F1 and typical IDS patterns from similar studies):

• True Negatives (Benign correctly classified): ~ 48750 (high specificity).
• False Positives (Benign misclassified as attack): ~ 500 (low false alarm rate).
• False Negatives (Attack missed): ~ 500 (low miss rate, critical for security).
• True Positives (Attack correctly detected): ~ 49,250 (high sensitivity).

This yields low error rates overall, with most misclassifications occurring on minority attack subclasses (e.g., rare infiltration types). The matrix shows the robustness of RF for binary IDS tasks, although some outliers, such as subtle brute force flows misclassified as normal, exist due to feature overlap.

Ablation study

Ablation experiments (Table 15, Fig 5) verified that feature harmonization provided the greatest improvement. The removal of semantic alignment resulted in a significant reduction in accuracy and F1-score (>10–15% absolute reduction) compared to removing SMOTE balancing and feature selection using mutual information. This is a testament to the importance of a unified representation for successful cross-dataset learning.

Download:

• PNG
  larger image
• TIFF
  original image

Table 15. Ablation study (merged dataset, random forest).

https://doi.org/10.1371/journal.pone.0346982.t015

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Performance impact of each pipeline component.

https://doi.org/10.1371/journal.pone.0346982.g005

Tamper-evident logging

Fig 6 and Table 16 show the SHA-256 hash-based logging mechanism. Each log entry records timestamp, model configuration, dataset ID, metrics, predictions, previous hash, and current hash, ensuring tamper-evident reproducibility.Fig 6 demonstrates the tamper-evident hash log, similar to a blockchain, that is generated during the recording of predictions. The non-repeating nature of the hash values at each index of the log is expected due to the nature of hash functions, which will always produce a unique hash value even when modifying the input data slightly. The hash chain provides a method to ensure that if someone attempts to tamper with the data, it will immediately disrupt the chain and allow for easier detection. Having a verification method incorporated within the logging process gives a level of transparency that can be extremely useful, especially when working within a cybersecurity environment. The structure of this logging method and the data that is available at each level can be seen in Table 15.

Download:

• PNG
  larger image
• TIFF
  original image

Table 16. Tamper-evident hash log structure.

https://doi.org/10.1371/journal.pone.0346982.t016

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Tamper-evident blockchain-style prediction log (cryptographic hash trace).

https://doi.org/10.1371/journal.pone.0346982.g006

Statistical significance

Statistical significance was evaluated at a significance level of α = 0.05. To account for multiple comparisons across models and ablation configurations, results were interpreted conservatively by emphasizing consistency of significance across folds and tests rather than relying on a single p-value. Formal correction methods such as Bonferroni were not applied due to the dependence structure inherent in cross-validation; however, this conservative interpretation reduces the likelihood of false-positive conclusions. Performance variability and confidence are reflected through reporting of mean ± standard deviation across repeated cross-validation runs, providing a robust estimate of stability.

To ensure that the improvements observed were not due to random variation in the data, a series of statistical significance tests were conducted across model comparisons. For the fold-wise cross-validation results, a Wilcoxon signed-rank test was employed when it was not possible to ensure normality assumptions. For prediction-level comparisons between models, McNemar’s test was employed to determine classification outcome differences. A summary of the results obtained from these tests can be found in Table 17. Here, it can be seen that all significant comparisons had a p-value < 0.05, thus confirming that the improvements observed in the full pipeline were statistically significant and not due to random variation in the data. In particular, it was observed that the improvements between Full and No SMOTE, and Full and Baseline (No Harmonization), were significant. Furthermore, it was observed that the greatest improvement was observed when feature harmonization was removed from the model. This again confirms the importance of feature harmonization in cross-dataset learning. Additionally, confidence intervals were employed to determine whether the improvements observed were consistent across all folds. Fig 7 illustrates, log₁₀(p-value) for visual emphasis, confirming statistical robustness of RF superiority and preprocessing steps.

Download:

• PNG
  larger image
• TIFF
  original image

Table 17. Statistical significance test results.

https://doi.org/10.1371/journal.pone.0346982.t017

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Statistical significance test (Mc Nemer/Wilcoxon).

https://doi.org/10.1371/journal.pone.0346982.g007

Analysis of model stability and variance

The proposed model shows high stability and low variance in terms of model performance, as suggested by the cross-validation results and the associated visualizations exported by the experimental pipeline. As shown in Fig 2, the distribution of the model’s performance metrics over different folds or runs indicates consistency in model behavior, low variance, and no critical outliers, suggesting high robustness to different data splits and low overfitting in the in-domain setting. This consistency in model behavior is further suggested by Fig 7, where high statistical evidence is shown by the values of –log10(p-value) significantly higher than the 0.05 threshold in all pairwise comparisons of different model configurations, suggesting that the observed differences in model performance are not random. Quantitative results can be seen in Table 18, which shows the p-values and test statistics for the performance of the model in various configurations.

Download:

• PNG
  larger image
• TIFF
  original image

Table 18. Average performance across all settings (NSL_ONLY, CIC_ONLY, MERGED).

https://doi.org/10.1371/journal.pone.0346982.t018

Table 19 provides a summary of the evaluation metrics, reporting status, and presentation format for the manuscript. It is important to be transparent and complete in the experimental results. The table offers a mapping from the evaluation metrics, such as cross-dataset evaluation and ablation study results, and statistical significance testing and the stability of the performance, to the results that are exported from the system, such as tables and figures. The high-priority results include cross-dataset evaluation metrics, ablation study results, statistical significance testing, variance across folds, ROC curves, and the confusion matrix of the best model. The table also identifies potential gaps in the analysis, such as the missing confusion matrix, and suggests generating it for detailed analysis of classification errors. Furthermore, a blockchain-inspired prediction log structure is identified as a medium-priority feature for reproducibility, which facilitates tamper-resistant tracking of model predictions using a linked list of cryptographic hashes.

Download:

• PNG
  larger image
• TIFF
  original image

Table 19. Evaluation metrics reporting summary.

https://doi.org/10.1371/journal.pone.0346982.t019

Comparison of model performance, along with computational efficiency, is shown in Table 20. The Random Forest model had the highest accuracy of 99.78% with the lowest variance among the models. The Ensemble model had a competitive performance of 98.67% but had a higher training time because it involved the use of multiple models. The LSTM model had the highest computational cost and relatively lower accuracy, which might be attributed to the lack of temporal information in the tabular network traffic features. The SVM provided solid mid-range performance with moderate training demands, while Naïve Bayes exhibited the weakest and most unstable results, as reflected by its large performance variance. Overall, tree-based methods offered the best balance between efficiency, robustness, and predictive accuracy for this cybersecurity classification task.

Download:

• PNG
  larger image
• TIFF
  original image

Table 20. Performance metrics and efficiency trade-offs.

https://doi.org/10.1371/journal.pone.0346982.t020

Variance analysis (Table 20, Fig 2) shows RF and ensemble models exhibit low standard deviation (±0.006–0.008), indicating high stability and resistance to overfitting. The variance of LSTM was found to be high at ±0.015, along with fold-wise drops, due to its incompatibility with non-sequential tabular data. Naive Bayes was found to possess the least stability in terms of variance at ±0.025 due to the independence assumptions violated by the correlated features of traffic. Trade-offs were in favor of tree-based models due to low-medium training time costs compared to high costs of using LSTM with minimal gains. From a computational point of view, the cost of training LSTM was high, with little improvement in performance, while tree-based models like Random Forest had a better trade-off in terms of efficiency and performance. These results again emphasize the point that sequence models may not be the best choice when the dataset does not have inherent time dependencies, like the NSL-KDD dataset.

State-of-the-Art (SOTA) Comparison

The State-of-the-Art (SOTA) Comparison Table 21, which deals with intrusion detection systems (IDS), focused on recent studies published in 2025 that have shown high accuracy on NSL-KDD and/or CICIDS2017 datasets. RF + harmonization obtains 98.0% accuracy on the merged dataset, which is comparable to or even better than many DL-centric ones without over-tuning or over-assuming sequences. Although it is possible to obtain 99% + accuracy on a given dataset, it is more realistic in a cross-dataset/harmonized setup. Harmonization of features contributed to the main generalization gain, whereas cryptographic logging had minimal overhead over using a full blockchain approach.

Download:

• PNG
  larger image
• TIFF
  original image

Table 21. Recent State-of-the-Art Intrusion Detection System (IDS) approaches.

https://doi.org/10.1371/journal.pone.0346982.t021

Most of the recent studies have reported high accuracy results for NSL-KDD/CICIDS2017 in-domain testing using preprocessing, balancing, or hybrid DL approaches. In terms of the results reported for a single data set, the accuracy is very high. However, when we consider the results reported for the cross data sets/harmonized approach (proposed framework), the results are more realistic. The accuracy of the RF classifier is very high with low variance. Therefore, the classifier is highly suitable for real-time processing. Harmonization of features dominated the contribution to the gains between data sets. Cryptographic logging is found to have a negligible overhead compared to the use of a blockchain.