https://orcid.org/0009-0005-1072-9422
Figures
Abstract
AdaBoost, a classical boosting ensemble algorithm, is widely applied for its strong classification performance. However, its standard exponential loss is highly sensitive to outliers, prone to overfitting, and inherently biased toward the majority class under class-imbalanced settings, degrading overall performance. To address these limitations, ASWBoost, a robust boosting algorithm, is proposed by introducing a parameter to modify the exponential loss. New update rules for base classifier weights and training sample probability distributions are derived, enabling adaptive adjustment of sample weights. The training error upper bound of the proposed algorithm is theoretically proven, and the impact of
on its convergence properties is rigorously analyzed. Extensive experiments on synthetic and OpenML datasets demonstrate robust performance across Balanced Accuracy, G-mean, Macro-F1, and AUC. Critically, statistical analyses confirm that ASWBoost significantly outperforms AdaBoost, CS-AdaBoost, GBDT, SMOTEBoost, and RUSBoost, with overall performance comparable to XGBoost.
Citation: Meng F, Yan M, Liu H, Xu W, Li H (2026) ASWBoost: Classification algorithm for noisy and imbalanced data based on parametric exponential loss. PLoS One 21(8): e0356889. https://doi.org/10.1371/journal.pone.0356889
Editor: Kaywan Othman Ahmed, Tishk International University, IRAQ
Received: November 18, 2025; Accepted: August 10, 2026; Published: August 27, 2026
Copyright: © 2026 Meng et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets analyzed in this study are publicly available from the OpenML repository (https://www.openml.org). To ensure full reproducibility, the specific OpenML Data IDs and direct URLs for the 12 datasets utilized in our experiments are provided below: Mammography (ID: 310): https://www.openml.org/d/310 Diabetes (ID: 37): https://www.openml.org/d/37 Breast Cancer (ID: 15): https://www.openml.org/d/15 ILPD (ID: 1480): https://www.openml.org/d/1480 Leukemia (ID: 1104): https://www.openml.org/d/1104 SPECT heart (ID: 42): https://www.openml.org/d/42 Hypothyroid (ID: 2): https://www.openml.org/d/2 Cardiotocography (ID: 1466): https://www.openml.org/d/1466 Transfusion (ID: 1464): https://www.openml.org/d/1464 Heart Failure (ID: 42720): https://www.openml.org/d/42720 EEG Eye State (ID: 1471): https://www.openml.org/d/1471 Dermatology (ID: 35): https://www.openml.org/d/35 No specific ethical approval or patient consent was required for this study as the data are open-source and fully anonymized.
Funding: This work was supported by the Cross-integration Innovation Team of Modern Applied Mathematics and Life Sciences in Yunnan Province, China (Grant No. 202405AS350003). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Within the framework of Boosting ensemble learning [1], AdaBoost [2] can boost weak classifiers into a strong classifier without prior knowledge. Its superior classification performance has attracted widespread attention in the machine learning community. This ensemble mechanism provides AdaBoost with robust generalization capabilities [3], demonstrating highly competitive classification performance across various critical domains, including text classification [4], feature selection [5], object recognition [6], spam filtering [7] and face recognition [8].
However, because AdaBoost relies on an exponential loss function, it is highly sensitive to datasets containing noise [9]. Therefore, enhancing its noise tolerance is of significant importance for addressing practical application problems. Specifically, the AdaBoost algorithm optimizes the expected risk primarily by minimizing the exponential loss function, [10], and fitting an additive model in a forward stagewise manner within the function space [11]. When noise is present in the training data, AdaBoost assigns excessively high weights to hard-to-classify noisy samples, inevitably leading to model overfitting [12]. Dietterich et al. [13] compared three ensemble algorithms—AdaBoost, Random Forest, and Bagging—and their results indicated that while AdaBoost consistently outperforms the other two in noise-free environments, its performance degrades significantly and falls behind both Random Forest and Bagging when noisy samples are introduced into the data. To mitigate this issue, Friedman et al. [10] proposed a series of representative AdaBoost variants that utilize adaptive Newton steps to optimize different loss functions. This approach ensures a smoother decrease in the training error of the optimal base learner, preventing excessive weight increases for misclassified samples and thereby reducing the algorithm’s sensitivity to noise. Furthermore, Kalai et al. [14] investigated robust base classifiers and boosting algorithms, finding that although robust classifiers converge faster than AdaBoost in the presence of label noise, their overall robustness remains insufficient. Consequently, Freund et al. [15] introduced a non-convex loss function incorporating a time variable based on the concept of Brownian motion, defined as
. Although this enhanced the robust capabilities of the algorithm, the non-convexity sacrifices the desirable properties of convex optimization, resulting in unstable training and high sensitivity to hyperparameters. The SPLBoost algorithm proposed by Wang et al. [16] integrated a self-paced learning mechanism into the AdaBoost framework, effectively enhancing the algorithm’s robustness. Its loss function is formulated as
. However, this method requires complex explicit regularization terms for sample selection, leading to high computational costs. Additionally, multiple subsequent studies [17] have continuously focused on improving the robustness of boosting algorithms in noisy environments. Cao et al. [18] proposed a noise-detection based AdaBoost (ND-AdaBoost) that utilizes a k-nearest neighbor pre-filtering mechanism to identify and penalize mislabeled data. While effective, it relies on external distance metrics and separate preprocessing steps rather than an inherently robust loss function. More recently, Xing et al. [19] introduced a bounded exponential loss function to improve the AdaBoost ensemble of One-Class Support Vector Machines (OCSVMs). Although this bounded loss significantly reduces the negative impact of outliers, it is specifically tailored for one-class novelty detection and alters the convexity properties, complicating its direct application to standard multi-class imbalanced classification tasks.
AdaBoost also exhibits significant limitations when processing class-imbalanced datasets [20–22]. The primary reason for this substantial degradation in classification performance is that, while iteratively adjusting sample weights to focus on misclassified samples, the algorithm fails to account for the uneven distribution of class labels. Consequently, the algorithm struggles to effectively learn the features of the minority class, ultimately resulting in decision boundaries that are biased toward the majority class. To address this, Wang et al. [23] employed a synergistic mechanism of feature selection and weighted non-negative matrix factorization, enhancing AdaBoost’s ability to handle misclassified samples. Xiao et al. [24] integrated clustering algorithms to partition the dataset into distinct clusters, training weak classifiers separately before ensembling them, which effectively improved overall performance. The “AdaImC” algorithm proposed by Bei et al. [25] combined AdaBoost with cost-sensitive learning to enhance predictive accuracy. Kang et al. [26] fused AdaBoost with stacked DoG feature extraction, improving recognition accuracy and robustness in complex environments by extracting common features. While these studies primarily approach the issue from the perspectives of feature selection and data preprocessing, their generalization capabilities remain limited when the intrinsic distribution of the data changes. Among these approaches, cost-sensitive learning has proven effective for handling imbalanced classification or asymmetric misclassification costs. However, the loss function of Cost-Sensitive Boosting proposed by Fan et al. [27], defined as , relies on a static cost matrix, making it susceptible to overfitting on minority class outliers. The AWABoost algorithm proposed by Wang et al. [28] applies a heuristic adjustment solely during the weight update step using
. By inserting a parameter control function into the weight update process, it limits excessive weight growth from a procedural standpoint. This method relies heavily on empirical procedural constraints and lacks rigorous theoretical derivation based on objective function consistency, resulting in insufficient theoretical completeness.
To overcome the limitations, this paper proposes a robust boosting algorithm named ASWBoost. The algorithm strictly adheres to the framework of forward stagewise additive modeling, discarding heuristic fixed cost matrices and complex explicit regularization terms. By introducing a parameter to improve the traditional exponential loss function, formulated as
, it ensures the theoretical completeness and robustness of the algorithm in both noisy and imbalanced environments. Furthermore, it theoretically preserves the desirable properties of convex optimization. Through the parameter
, the algorithm achieves adaptive adjustment of sample weights, thereby striking an effective balance between enhancing the feature learning of minority classes and suppressing the interference of noisy samples.
The main contributions of this paper are as follows:
- Improvement of the Exponential Loss: The traditional exponential loss function is modified by introducing a parameter
.
- Derivation of Updating Mechanisms: Novel updating formulas for both the weights of the base classifiers and the probability distribution of the training samples are derived, enabling parameterized adjustment of sample weights.
- Proof of the Error Upper Bound:The upper bound on the training error for the proposed algorithm is rigorously proven, guaranteeing its convergence.
- Empirical Validation: Extensive experiments on a synthetic dataset and various OpenML medical datasets demonstrate that ASWBoost outperforms standard AdaBoost and other variant algorithms in terms of Balanced Accuracy, G-mean, Macro-F1, and AUC.
2. Preliminaries
2.1. The AdaBoost Algorithm
The core mechanism of the AdaBoost algorithm lies in training a series of weak classifiers by continuously adjusting sample weights. Specifically, in each iteration, it increases the weights of misclassified samples and decreases the weights of correctly classified ones. This strategy forces subsequent classifiers to focus more on hard-to-learn samples, thereby progressively improving the classification performance of the overall ensemble model.
Algorithm 1. AdaBoost
Input: A set of training samples with labels , a weak learning algorithm, and the maximum number of iterations M.
Initialize: The weights of the training samples: .
For m = 1 to M:
(a) Learn a base classifier using the training set with weight distribution
.
(b) Calculate the classification error rate of :
.
(c) Calculate the coefficient of :
.
(d) Update the weight distribution :
. Where,
is a normalization constant, and
.
Output: Final classifier
2.2. Standard exponential loss and margin
From a statistical perspective, the AdaBoost algorithm can be interpreted as a forward stagewise additive modeling process that minimizes an underlying exponential loss function [10]. For a binary classification task, the performance of the ensemble classifier on a specific training instance
is quantitatively evaluated by the classification margin, defined as
. The sign of the margin indicates the correctness of the classification, while its absolute value
represents the confidence level of the prediction [12]. The ultimate objective of standard AdaBoost is to minimize the empirical exponential loss over the entire training dataset:
At the m-th iteration, minimizing this objective intrinsically translates into the weight updating rule . While the strict convexity of the exponential loss guarantees fast convergence and a global optimum in clean datasets, it becomes a critical vulnerability in practical environments contaminated with noise. As delineated by the mathematical property of the exponential function,
assigns an unbounded, exponentially growing penalty to instances with large negative margins
. In real-world datasets, these hard-to-classify instances are typically outliers or mislabeled noise. Consequently, the standard exponential loss forces the subsequent base learners to disproportionately focus on these anomalous samples, causing the decision boundary to severely overfit the noise and thereby degrading the generalization performance. This intrinsic mathematical limitation underscores the theoretical imperative for reshaping the exponential loss function, which constitutes the core motivation behind the proposed ASWBoost algorithm.
3. Theoretical Analysis of ASWBoost
3.1. Parameterized exponential loss function
The ASWBoost algorithm can be formalized as a specific instance of forward stagewise additive modeling. Specifically, the final model is an additive model composed of base classifiers, and it is optimized using a modified version of the traditional exponential loss function. ASWBoost employs an additive ensemble approach, constructing the final model through the linear superposition of multiple base classifiers.
Theorem 1: The ASWBoost algorithm is modeled using the forward stagewise additive algorithm. It is derived by iteratively training each weak classifier, which serves as a basis function in the linear combination, through the minimization of a modified exponential loss function.
Proof: In terms of constructing an additive model, the ASWBoost algorithm acts as a concrete implementation of the forward stagewise algorithm. The final ensemble strong classifier is exactly the additive model obtained when selecting weak classifiers as basis functions within the forward stagewise framework. Furthermore, the step-by-step process of training weak classifiers in ASWBoost corresponds perfectly to the sequential learning of basis functions in the forward stagewise algorithm. The loss function of the ASWBoost algorithm is formulated as follows:
3.2. Update formulae for classifier and sample weights
Assume that after m-1 iterations, the forward stagewise algorithm has obtained :
In the m-th iteration, the algorithm computes to update
as:
The objective is to find and
that minimize the loss of
on the training dataset:
Since depends neither on
nor on
, it acts as a constant within the current minimization step. However,
depends on
and changes with each iteration. Minimizing Equation (5) yields the optimal base classifier
and its coefficient
for the ASWBoost algorithm. The detailed derivation consists of two main steps:
First, we solve for . For any
, the optimal
that minimizes the objective is:
Where, .The classifier
is the base classifier
in the ASWBoost algorithm, as it is the base classifier that minimizes the weighted classification error rate on the training data in the m-th round.
Second, we solve for . The objective function is:
Based on Equation (7), we solve for the optimal as follows:
Applying Fermat’s theorem, we take the derivative of the objective function with respect to and set it to zero:
Simplifying this yields the optimal :
Where is the weighted classification error rate:
Thus, we have derived the optimal , which is identical to the
defined in the ASWBoost algorithm. From Equation (4) and
, we can further derive the sample weight update rule:
Given the normalization factor is:
We obtain the normalized weight distribution:
The complete training procedure of the ASWBoost algorithm is outlined in Algorithm 2.
Algorithm 2. ASWBoost
Input: A set of training samples with labels , a weak learning algorithm, and the maximum number of iterations M,
: noise handling,
: imbalance handling.
Initialize: the weight of the training samples:
For m = 1 to M:
(a) Learn a base classifier using the training set with weight distribution .
(b) Calculate the classification error rate of :
.
(c) Calculate the coefficient of :
.
(d) Update the weight distribution :
Where, is a normalization constant, and
.
Output: Final classifier .
3.3. Training error upper bound analysis
Theorem 2: (Training Error Bound of ASWBoost): The training error bound of the final classifier produced by the ASWBoost algorithm is given by:
Proof: When , it implies
, and consequently
. This directly establishes the first half of the inequality. Using the definitions from previous derivations, the second half regarding
can be deduced as follows:
This proves the upper bound of the training error for the modified exponential loss function. In each iteration, selecting a with optimal classification performance ensures that
is minimized. Consequently, a smaller total training error bound is achieved, leading to a final classifier with minimized empirical error.
Theorem 3: (Impact of on the Training Error Bound): Compared to the standard AdaBoost algorithm, when
, ASWBoost is mathematically equivalent to AdaBoost. When
, the upper bound of the training error for the proposed algorithm is slightly larger, but it still decreases at an exponential rate. When
, the upper bound of the training error is smaller than that of AdaBoost, indicating superior convergence properties.
Proof: From the definition of , we have:
Let .
- a). When
, the algorithm degenerates into the standard AdaBoost algorithm:
Using the Taylor expansion inequalities and
at
, we obtain
. Consequently:
. This demonstrates that the training error of the AdaBoost algorithm decreases exponentially.
- b). When
, we have
. If we were to use the standard coefficient
, the upper bound of ASWBoost would be strictly larger than standard AdaBoost. However, utilizing the optimal coefficient derived for the proposed algorithm,
, the training error still decreases exponentially.
Proof: When , the true upper bound of the training error is:
For the function , its Taylor expansion at
is:
The Taylor series of the exponential function serves as an upper bound for
, meaning
. This leads to:
This proves that even for , the training error of ASWBoost continues to decrease at an exponential rate.
- c). When
:
When is negative, the upper bound of the training error for the proposed algorithm is strictly smaller than that of standard AdaBoost, resulting in superior convergence.
3.4. Explanation of the ASWBoost Algorithm
The sample weight updating mechanism in ASWBoost is critical, as it shapes the training data distribution for subsequent iterations. For a binary classification task, the updating rule can be explicitly expressed as:
Building upon the foundation of AdaBoost, ASWBoost retains the core strategy of increasing the weight of misclassified samples and decreasing the weight of correctly classified ones. However, the magnitude of this weight adjustment is governed by the parameter . This mechanism does not alter the underlying statistical distribution of the dataset; rather, it provides a new weight distribution in each iteration, guiding subsequent weak classifiers to focus on hard-to-classify samples.
The parameter plays a pivotal role in this process. When
, ASWBoost completely degenerates into standard AdaBoost. When
, the magnitude of weight reduction for correctly classified samples remains consistent, but the penalty (weight increase) for misclassified samples is amplified. This explicitly directs the algorithm to focus more heavily on the minority class in extremely imbalanced datasets. Conversely, when
, the weight increase for misclassified samples is attenuated, resulting in a more conservative updating scheme. This constraint effectively prevents the model from overfitting when dealing with noisy data, as the weights of hard-to-classify noisy samples are prevented from growing exponentially. The distinct weight updating mechanisms in noisy and imbalanced environments are illustrated in Fig 1.
3.5. Connection and distinction with existing robust boosting frameworks
To fully articulate the theoretical positioning of ASWBoost, it is essential to compare its mathematical mechanisms with recent advanced robust boosting frameworks, notably ND-AdaBoost [18] and BELF-AEOCSVMs [19]. All three algorithms fundamentally recognize that standard AdaBoost’s exponential loss function leads to overfitting by consistently assigning excessively high weights to hard-to-learn instances. However, their theoretical solutions differ significantly:
- (1). Implicit vs. Explicit Noise Handling: ND-AdaBoost introduces an explicit noise-detection function
into the loss function, requiring external algorithms (such as k-NN or EM) to assign a noise label at each iteration. While this explicitly categorizes instances into four types, it incurs substantial computational overhead. In contrast, ASWBoost introduces the parameter
to reshape the curvature of the loss function. This allows the algorithm to implicitly limit the weight penalty for large-margin misclassified instances without relying on any external distance metrics, thereby maintaining algorithmic efficiency.
- (2). Convexity and Optimization: BELF-AEOCSVMs replaces the traditional loss with a bounded exponential loss function
. While highly effective at neutralizing outliers, this bounded function is mathematically smooth but nonconvex, which complicates optimization and necessitates complex numerical methods like the Newton-Raphson approach. ASWBoost strictly preserves the desirable convexity of the exponential loss. By ensuring convexity, ASWBoost guarantees that the forward stagewise additive modeling (Theorem 1) converges stably to the global optimum, ensuring rigorous theoretical error bounds (Theorem 2) for standard decision trees.
Thus, ASWBoost acts as a theoretical bridge: it achieves the robust outlier-suppression capabilities seen in bounded methods and noise-detection methods, while retaining the mathematical simplicity and convexity of standard AdaBoost.
4. Experimental results and analysis
In this section, we evaluate the classification performance of the proposed ASWBoost algorithm on a synthetic dataset and 12 OpenML datasets. Furthermore, a Friedman test with post-hoc analysis is employed to determine whether statistically significant differences exist between ASWBoost and the baseline algorithms. To ensure the robustness of the experimental results, a stratified cross-validation framework is adopted. The outer loop utilizes a five-fold cross-validation, while the inner loop performs automatic optimization of the hyperparameter via nested grid search and three-fold cross-validation. All experiments are conducted using a fixed random seed of 42 to guarantee reproducibility.
4.1. Evaluation metrics
To comprehensively evaluate the model’s performance on imbalanced data and prevent biases dominated by the majority class, we utilize Balanced Accuracy, G-mean, Macro-F1, and Area Under the Curve (AUC) as evaluation metrics [29].
Balanced Accuracy is the arithmetic mean of the recall for each class:
G-mean comprehensively considers the model’s discriminative ability across all classes, achieving a high score only when the model performs well on all classes:
Macro-F1 treats each class equally regardless of its frequency of occurrence, calculating the F1 score for each class and then averaging them:
AUC represents the probability that the model ranks a randomly chosen positive instance higher than a randomly chosen negative instance. Its value ranges from 0.5 to 1, with values closer to 1 indicating stronger discriminative ability.
4.2. Performance on synthetic datasets
The synthetic dataset is generated using the make_classification function in Python to simulate common data noise, feature redundancy, and class imbalance found in real-world scenarios. The dataset comprises 1,000 samples and 40 features, among which 25 are informative features, 10 are redundant features (random linear combinations of the informative features), and the remaining 5 are noise features drawn from a standard normal distribution N (0,1). The inter-class separability is controlled by setting class_sep = 1.5.
The proposed ASWBoost algorithm is compared against standard AdaBoost, Cost-Sensitive AdaBoost (CS-AdaBoost), RobustBoost, GBDT, and XGBoost on this synthetic dataset. To evaluate the algorithm’s robustness in noisy environments, varying degrees of label noise are simulated by randomly flipping 10%, 20%, and 30% of the class labels. A grid search is employed to determine the optimal parameter and to investigate its relationship with the performance of ASWBoost. Fig 2 illustrates the comparative experimental results across different noise conditions.
(Figure Captions: Experimental performance of six algorithms across 10%, 20%, and 30% noise levels in terms of Balanced Accuracy, G-mean, Macro-F1, and AUC. The optimal values for are −0.082, −0.045, and −0.026, respectively).
As shown in the experimental results in Fig 2, the performance of AdaBoost and CS-AdaBoost degrades significantly in high-noise environments. This is primarily attributed to the weight allocation mechanism of the traditional exponential loss function for misclassified samples. During the iterative process, hard-to-classify noisy samples are assigned increasingly larger weights, leading the base learners to overfit the noise. In contrast, XGBoost and GBDT employ log-loss functions with second-order approximation optimization and incorporate regularization terms, making them more robust to higher ratios of label contamination. RobustBoost improves noise tolerance to some extent by designing a robust loss function; however, it still exhibits limitations in filtering high-dimensional noise features. ASWBoost preserves the convexity of the traditional exponential loss while mitigating the penalty intensity for small-margin samples by introducing the parameter . This effectively suppresses the abnormal expansion of weights for noisy samples during iterations. The experiments validate our theoretical premise: the optimal negative values of
obtained through grid search alter the curvature of the exponential loss, restrict the excessive growth of weights for misclassified noisy samples, and thereby enhance the model’s robustness against outliers.
To verify the generalization performance of ASWBoost on class-imbalanced data, three highly imbalanced datasets with negative class proportions of 90%, 80%, and 70% are constructed based on the synthetic data for experimental evaluation. The results are illustrated in Fig 3.
(Figure Captions: Experimental performance of six algorithms across imbalance ratios of 7:3, 8:2, and 9:1, evaluated by Balanced Accuracy, G-mean, Macro-F1, and AUC. The optimal values of for these ratios are 0.5, 0.5, and 0.6, respectively).
The experiments demonstrate that as the degree of imbalance intensifies, the performance of all models degrades to varying degrees. However, ASWBoost exhibits significant robustness in terms of G-mean and Balanced Accuracy. Particularly in highly imbalanced scenarios (8:2 and 9:1), it substantially outperforms standard AdaBoost and its variants, proving its superiority in balancing minority class recall and majority class precision. Although CS-AdaBoost increases attention to the minority class via a cost-sensitive factor, it fails to fundamentally mitigate the overfitting risks induced by exponential weight growth. GBDT and XGBoost show strong competitiveness under mild imbalance; however, under extreme imbalance, their global gradient optimization inevitably shifts toward the majority class, reflected by a noticeable decline in G-mean (which captures minority class recognition capability). When , ASWBoost no longer blindly pursues absolute correctness for every sample as standard AdaBoost does. Instead, through its margin-based weight updating mechanism, it emphasizes enhancing minority class recall, avoids misclassifying them as noise, and thereby strengthens the model’s generalization capacity.
4.3. Complexity analysis
Let be the number of training samples,
be the feature dimension,
be the number of iterations, and the training complexity of the weak classifier be
. The computational complexity of AdaBoost can be formalized as
[30], which primarily stems from conducting approximately
sequential iterations. Each iteration involves core steps such as training the weak classifier
, calculating the weighted error rate
, and updating the sample weights
. Therefore, the total time complexity is
. The proposed ASWBoost algorithm retains all the computational steps and data dependencies. The introduced parameter
merely serves as a constant factor in the weight coefficient calculations and does not alter the asymptotic complexity order of the algorithm. Consequently, ASWBoost remains consistent with standard AdaBoost in both time and space complexity, with its space complexity also bounded by
.
To empirically validate the theoretical analysis, we conducted runtime experiments comparing ASWBoost with the baseline models on the synthetic dataset. The results are shown in Table 1.
Table 1 presents the average runtime and standard deviation of 5-fold cross-validation for each algorithm under identical experimental settings. The results indicate that the runtime of ASWBoost is significantly higher than that of the other algorithms. This is primarily attributed to the external hyperparameter tuning overhead. The experiment employed a nested cross-validation strategy with grid search to precisely optimize , thereby increasing the time cost of model training and evaluation. However, considering the enhanced robustness and generalization performance on noisy and imbalanced data, this increase in runtime is entirely justifiable and acceptable.
4.4. Performance on OpenML Datasets
To further validate the effectiveness of the ASWBoost algorithm in handling class-imbalanced data under real-world medical scenarios, comprehensive experiments were conducted on 12 datasets sourced from the OpenML repository (https://www.openml.org). As detailed in Table 2, these datasets encompass a diverse range of sample sizes, feature dimensions, and severe imbalance levels. For multi-class datasets (e.g., Hypothyroid, Cardiotocography, and Dermatology), the selected target class was treated as the positive class, while all remaining classes were aggregated into the negative class, effectively transforming them into highly imbalanced binary classification tasks.
During the data preprocessing stage, all features were standardized to ensure a mean of 0 and a standard deviation of 1. Regarding experimental parameter settings, the search space for the hyperparameter was strictly constrained between (−1, 1). The baseline models were implemented using the Scikit-learn library with default configurations, and the number of iterations was uniformly set to 50 for all boosting algorithms. To accurately reflect the characteristics and the exact imbalance severity of the data, Table 2 explicitly outlines the original number of classes, the specific number of instances in both the majority and minority classes, and the resulting imbalance ratio. All reported results in the subsequent analyses represent the averages derived from 5-fold cross-validation to ensure maximum reliability.
The real-world medical datasets detailed in Table 2 provide a rigorous testbed for evaluating algorithmic robustness under complex distributions. Specifically, the nature of the class imbalance originates from the epidemiological rarity of positive disease instances (e.g., Mammography, Hypothyroid) relative to healthy baselines, which inherently biases standard classifiers toward the majority class. Concurrently, the nature of the noise is dual-faceted: it comprises feature noise from instrumental artifacts (e.g., EEG sensor inaccuracies) and label noise driven by subjective diagnostic discrepancies. Although Macro-F1 was evaluated on synthetic data, it was omitted in real-world experiments because its inherent reliance on precision renders it excessively volatile and potentially misleading under extreme class skews. Consequently, we exclusively prioritized AUC, Balanced Accuracy, and G-mean to yield a more robust, prevalence-independent assessment of the model’s true discriminative capability in severely imbalanced medical scenarios.
Fig 4 presents the AUC comparisons across the 12 OpenML datasets. Overall, ASWBoost achieved optimal or near-optimal performance on the vast majority of the datasets. This indicates that ASWBoost, through its adaptive sample weighting mechanism, effectively overcomes the inherent flaws of standard AdaBoost, namely its sensitivity to noise and insufficient focus on minority classes.
Specifically, on highly imbalanced datasets such as Mammography and Hypothyroid, standard AdaBoost’s performance was restricted due to majority class dominance, while RUSBoost suffered fluctuations due to information loss from under-sampling. In contrast, ASWBoost effectively mitigated the interference from majority class samples via its adaptive weight adjustment mechanism. Its AUC values were significantly higher than those of CS-AdaBoost and SMOTEBoost, demonstrating its robustness under extreme distributions. Compared to GBDT and XGBoost, ASWBoost exhibited stronger competitiveness on datasets like Pima Indians Diabetes, Heart Failure, and Indian Liver Patient. This demonstrates that ASWBoost not only addresses the imbalance issue but also broadens the model’s generalization boundaries through a weighting strategy based on sample difficulty and distribution, effectively preventing overfitting.
Fig 5 further evaluates the decision bias and overall recall capability of the models on imbalanced datasets. The experimental results show that ASWBoost achieved the highest Balanced Accuracy on the majority of the datasets, validating its significant effectiveness in alleviating class imbalance bias.
ASWBoost secured the highest Balanced Accuracy on most datasets, including Mammography, Indian Liver Patient, Leukemia, and Heart Failure. Compared to the oversampling SMOTEBoost and the under-sampling RUSBoost, ASWBoost performed much better and more stably on datasets like Hypothyroid and Blood Transfusion. This suggests that the adaptive weighting mechanism of ASWBoost can preserve original information more effectively than simply altering the data distribution, thereby enhancing the discriminative power of the classification hyperplane. When compared against strong ensemble learners like XGBoost and GBDT, ASWBoost also demonstrated exceptional competitiveness. This further confirms that the algorithm successfully balances the classification accuracy of both minority and majority classes by optimizing the sample weight distribution.
Fig 6 further presents the G-mean results, a metric that evaluates balanced predictive performance across both majority and minority classes.
Unlike overall accuracy, G-mean strictly penalizes models biased toward the majority class. As illustrated, ASWBoost consistently achieves the highest G-mean scores across most datasets, demonstrating particularly pronounced advantages on severely imbalanced sets such as Mammography, Hypothyroid, and Leukemia. By suppressing outlier interference and adaptively emphasizing hard-to-learn minority instances, ASWBoost substantially outperforms CS-AdaBoost, SMOTEBoost, and RUSBoost, while remaining highly competitive with GBDT and XGBoost. These findings confirm that the parameterized exponential loss mechanism maintains an optimal trade-off between minority-class recall and majority-class recall without artificially altering the underlying data distribution.
4.5. Statistical significance analysis
To rigorously evaluate the statistical significance of the performance differences among the compared algorithms across the 12 OpenML datasets, we conducted non-parametric statistical tests following the authoritative guidelines by Demšar [31]. Addressing the common limitation of relying on a single metric, this validation was comprehensively extended across all primary evaluation metrics: AUC, Balanced Accuracy, and G-mean.
As a non-parametric method, the Friedman test operates on the average ranks of the algorithms rather than their raw performance values, rendering it inherently robust to varying data distributions and incommensurate metric scales. For each metric, we ranked the algorithms on each of the
datasets (assigning a rank of 1 to the best-performing algorithm and 7 to the worst, with ties receiving average ranks). The exact average ranks for all algorithms across the three metrics are explicitly summarized in Table 3.
The Friedman test evaluates the null hypothesis that all algorithms perform equivalently. The initial Friedman statistic
is defined as:
Because the statistic is known to be undesirably conservative, we utilized the corrected Iman-Davenport statistic
, which is distributed according to an F-distribution with
and
degrees of freedom:
For AUC, Balanced Accuracy, and G-mean, the computed statistics are 5.45, 5.74, and 6.67, respectively. All values significantly exceed the critical value of
at the significance level
. Consequently, the null hypothesis
is firmly rejected, indicating statistically significant performance differences among the algorithms across all metrics. To ascertain the specific superiorities of the proposed algorithm, we proceeded with the Bonferroni-Dunn post-hoc test [31]. This specific test is strictly appropriate for our scenario, where multiple baseline classifiers are systematically compared against a single control classifier (ASWBoost). The performance of a baseline algorithm is considered significantly different from the control algorithm if their average ranks differ by at least the Critical Difference (CD), formulated as:
Given algorithms,
datasets, and a stringent significance level of
, the critical value
, yielding a rigorous CD threshold of 2.111.
Based on this mathematical threshold, we draw the following statistically backed conclusions from Table 3:
- (1). For G-mean and Balanced Accuracy: ASWBoost consistently achieves the lowest average rank (1.500). The rank differences between ASWBoost and standard AdaBoost, CS-AdaBoost, GBDT, SMOTEBoost, and RUSBoost all strictly exceed the CD threshold of 2.111. This statistically confirms that ASWBoost significantly outperforms these five algorithms at
. XGBoost is the only evaluated algorithm that demonstrates statistically comparable performance to ASWBoost on these two comprehensive metrics.
- (2). For AUC: ASWBoost ranks highly, aligning with advanced gradient-based frameworks (GBDT and XGBoost). Statistical testing verifies that ASWBoost significantly outperforms CS-AdaBoost, SMOTEBoost, and RUSBoost, while maintaining statistically comparable performance to standard AdaBoost, GBDT, and XGBoost.
In Fig 7, the average ranks of all algorithms are plotted on the horizontal axis, with lower ranks positioned to the right. Algorithms connected by a thick solid line exhibit no statistically significant performance difference.
The analysis confirms that the performance of ASWBoost is significantly superior to AdaBoost, CS-AdaBoost, GBDT, SMOTEBoost, and RUSBoost. Although the average rank of ASWBoost is better than that of XGBoost, under the current statistical testing framework, the performance of the two algorithms is considered comparable.
5. Conclusion
To address the performance degradation of the AdaBoost algorithm in environments characterized by data noise and class imbalance, this paper proposes a robust boosting algorithm named ASWBoost. By introducing the parameter to improve the traditional exponential loss function, the algorithm achieves adaptive adjustment of sample weights while preserving the desirable properties of convex optimization. The optimal parameter for different datasets is determined via grid search. Experimental results on a synthetic dataset and 12 real-world OpenML datasets demonstrate that ASWBoost outperforms the baseline models across evaluation metrics including Balanced Accuracy, G-mean, Macro-F1, and AUC. Furthermore, the Friedman test and Nemenyi post-hoc analysis validate the statistically significant advantages of ASWBoost.
However, the determination of the current parameter relies on grid search and lacks prior data information, which leads to lower tuning efficiency and limited interpretability. Future research will explore dynamic parameter adjustment strategies based on data distribution and noise estimation. Additionally, extending ASWBoost to multi-class classification and regression tasks will be investigated to further validate the generalization capabilities of the algorithm.
Acknowledgments
We thank the OpenML public repository (https://www.openml.org) for providing the free datasets used in our experiments. We also acknowledge the School of Mathematics at Yunnan Normal University and the Yunnan Key Laboratory of Modern Analytical Mathematics and Its Applications for their support. In addition, we thank the editors and anonymous reviewers of PLoS One for their invaluable feedback and helpful comments during the revision of our manuscript.
References
- 1. Kearns M, Valiant L. Cryptographic limitations on learning Boolean formulae and finite automata. JACM. 1994;41(1):67–95.
- 2. Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci. 1997;55(1):119–39.
- 3.
Schapire RE. Explaining AdaBoost. Empirical inference: festschrift in honor of Vladimir N. Vapnik. Berlin, Heidelberg: Springer Berlin Heidelberg. 2013. 37–52.
- 4. Shi L, Ma X, Xi L, Duan Q, Zhao J. Rough set and ensemble learning based semi-supervised algorithm for text classification. Expert Systems with Applications. 2011;38(5):6300–6.
- 5. Koutanaei FN, Sajedi H, Khanbabaei M. A hybrid data mining model of feature selection algorithms and ensemble learning classifiers for credit scoring. Journal of Retailing and Consumer Services. 2015;27:11–23.
- 6.
Ma S, Du T. Improved AdaBoost face detection. In: 2010 International Conference on Measuring Technology and Mechatronics Automation, 2010. 434–7.
- 7. Chai BR, Gu WC, Niu ZY, Zhou HJ, Wang KS. A filtering method against junk mail using Boosting algorithm. Trans Beijing Inst Technol. 2013;33(1):79–83.
- 8. Ranftl A, Alonso‐Fernandez F, Karlsson S, Bigun J. Real‐time AdaBoost cascade face tracker based on likelihood map and optical flow. IET Biometrics. 2017;6(6):468–77.
- 9. Khoshgoftaar TM, Van Hulse J, Napolitano A. Comparing boosting and bagging techniques with noisy and imbalanced data. IEEE Trans Syst, Man, Cybern A. 2011;41(3):552–68.
- 10. Friedman J, Hastie T, Tibshirani R. Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann Stat. 2000;28(2):337–407.
- 11. Hastie TJ. Generalized additive models. Statistical Models in S. Routledge. 2017. 249–307.
- 12. Schapire RE, Freund Y. Boosting: foundations and algorithms. Kybernetes. 2013;42(1):164–6.
- 13. Dietterich TG. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Mach Learn. 2000;40(2):139–57.
- 14.
Kalai A, Servedio RA. Boosting in the presence of noise. In: Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, 2003. 195–205. https://doi.org/10.1145/780542.780573
- 15. Freund Y. A more robust boosting algorithm. 2009.
- 16. Wang K, Wang Y, Zhao Q, Meng D, Liao X, Xu Z. SPLBoost: an improved robust boosting algorithm based on self-paced learning. IEEE Trans Cybern. 2021;51(3):1556–70. pmid:31880577
- 17. Zhang CX, Zhang JS, Zhang GY. An efficient modified boosting method for solving classification problems. J Comput Appl Math. 2008;214(2):381–92.
- 18. Cao J, Kwong S, Wang R. A noise-detection based AdaBoost algorithm for mislabeled data. Pattern Recognition. 2012;45(12):4451–65.
- 19. Xing H-J, Liu W-T, Wang X-Z. Bounded exponential loss function based AdaBoost ensemble of OCSVMs. Pattern Recognition. 2024;148:110191.
- 20. Fachrie M, Musdholifah A, Pulungan R. Effectiveness of data resampling and ensemble learning in multiclass imbalance learning. Artif Intell Rev. 2025;58(12):368.
- 21. Wang W, Sun D. The improved AdaBoost algorithms for imbalanced data classification. Inf Sci. 2021;563:358–74.
- 22. Khan AA, Chaudhari O, Chandra R. A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation. Expert Systems with Applications. 2024;244:122778.
- 23. Wang Y, Feng L, Zhu J, Li Y, Chen F. Improved AdaBoost algorithm using misclassified samples oriented feature selection and weighted non-negative matrix factorization. Neurocomputing. 2022;508:153–69.
- 24. Xiao H, Xiao Z, Wang Y. Ensemble classification based on supervised clustering for credit scoring. Applied Soft Computing. 2016;43:73–86.
- 25. Bei H, Wang Y, Ren Z, Jiang S, Li K, Wang W. A statistical approach to CS-AdaBoost for imbalanced data classification. Math Probl Eng. 2021;2021(1):3165589.
- 26. Kang HH, Lee SW, You SH, Ahn CK. Novel vehicle detection system based on stacked DoG kernel and AdaBoost. PLoS One. 2018;13(3):e0193733. pmid:29513727
- 27.
Fan W, Stolfo SJ, Zhang J, Chan PK. AdaCost: misclassification cost-sensitive boosting. In: Proceedings of the 16th International Conference on Machine Learning (ICML), 1999.
- 28. Wang S, Liu W, Yang S, Huang H. An optimized AdaBoost algorithm with atherosclerosis diagnostic applications: adaptive weight-adjustable boosting. J Supercomput. 2024;80(9):13187–216.
- 29. Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Inf Process Manag. 2009;45(4):427–37.
- 30.
Arunachalam S, Maity R. Quantum boosting. In: International Conference on Machine Learning. PMLR. 2020. 377–287.
- 31. Demšar J. Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res. 2006;7:1–30.