Predictive features.

Predicting learning achievement involves leveraging various learner features, which can be categorized into demographic information and online behavioral data. Demographic information typically includes factors such as gender, age, and educational background, while online behavioral data encompasses metrics like video consumption, time spent on course materials, and participation in online activities.

Studies such as [16] have leveraged demographic factors like age, gender, and prior academic performance to forecast learning outcomes. These approaches employed interpretable machine learning techniques to identify factors contributing to poor performance and integrated rule-based risk models to enhance prediction accuracy.

Another study [17] focused on basic learner information to predict learning achievement. This research presented a two-stage predictive model development process aimed at improving recognition accuracy and supporting educators in implementing diverse teaching practices to enhance student learning outcomes.

Additionally, research by [18] incorporated features like gender, age, and residential status into a nonlinear State Space Model tailored for predicting student dropout. This model emphasized the evolving latent state of students in open and distance education settings, highlighting the importance of ongoing student status monitoring.

However, these studies often rely on static input features that do not account for dynamic learning processes, serving more as early warning systems prior to actual learning engagement. They often fail to capture the behaviors of learners during their learning activity, thereby limiting prediction accuracy.

In contrast, online interaction data emerged as a critical predictor of learning achievement, offering insights into individual learning quality [19, 20]. For instance, [21] used Logistic Regression (LR) to extract features from learners’ interactions with video lectures and assignments, predicting learning performance based on these behaviors. Similarly, [22] employed Random Forest (RF) to model learning achievements using online clickstream data.

Most studies within the MOOCs context have prioritized behavioral data for predicting learning outcomes [23, 24]. However, both behavioral data and demographic information are crucial predictors of learning achievement. Therefore, this paper integrates demographic information and online behavioral data to predict learning achievement.

Prediction models.

The development of learning achievement prediction models involves a diverse range of machine learning methods extensively studied by researchers. These models can be categorized into two primary groups: traditional machine learning algorithms and ensemble learning methods.

Traditional machine learning algorithms. Marbouti [25] employed LR, Decision Trees (DT), Naive Bayes (NB), K-Nearest Neighbor (KNN), Artificial Neural Networks (ANN), and Support Vector Machines (SVM) to develop a robust model aimed at identifying learners at risk of failure. Their study highlighted the challenge of accurately identifying both successful and unsuccessful learners across different algorithms.

Howard [26] systematically compared the performance of common prediction algorithms, including RF, SVM, ANN, and KNN, among others. Their findings indicated that RF yielded the most effective results.

In another investigation [27], DT, LR, NB, and RF algorithms were evaluated to recommend an optimal choice for predicting dropout. The study concluded that straightforward algorithms could achieve reliable accuracy in identifying predictors of dropout.

Similarly, a study [28] utilized RF, SVM, DT, NB, KNN, and LR for classification tasks, with RF demonstrating superior performance in detecting students susceptible to dropout, achieving an accuracy rate of 94.14%.

These studies underscore the efficacy of machine learning in addressing student learning achievement prediction. However, the choice of prediction algorithm depends on the perspective of data observation, mining objectives, and research context, making it challenging to determine a universally superior algorithm for online learning achievement prediction.

The objective of employing machine learning technology in building learning achievement prediction models is to achieve high accuracy, strong generalization ability, and robustness through rigorous training. Nonetheless, in practical applications, machine learning models often exhibit biases that hinder them from fully meeting operational requirements.

Ensemble learning methods. Ensemble learning stands as a critical approach in machine learning, combining multiple weak learners to form a robust model with improved accuracy and generalization capabilities [29, 30]. Methods like Bagging, Boosting, and Stacking are particularly effective in improving model performance and addressing both classification and regression tasks.

Bagging and Boosting are homogeneous ensemble methods, which rely on using the same base learning algorithm across multiple iterations. Conversely, Stacking is a heterogeneous ensemble approach that employs diverse base learners in parallel, combining their outputs through a meta learner to generate the final prediction [31]. This approach increases model diversity and enhances generalization, offering distinct advantages over homogeneous ensemble methods.

Ensemble learning algorithms have broad applications across various fields. For example, recent models like Deepstacked-AVPs, iAFPs-Mv-BiTCN, pAVP_PSSMDWT-EnC, iACP-GAEnsC, and CACP, developed by Akbar et al. [32–36], significantly enhance peptide identification and prediction by integrating advanced feature selection techniques and optimized algorithms. These models have proven to be highly valuable in pharmaceutical design and research. Similarly, Ullah et al. [37] developed the DeepAVP-TPPred model, which improves antiviral peptide prediction using a novel binary tree growth algorithm.

These examples highlight the wide-ranging applicability and impact of ensemble learning methods across different domains, demonstrating their effectiveness in addressing complex problems through model integration and optimization.

Data preprocessing.

Data preprocessing is a critical step in building predictive models for learning achievement, involving tasks such as data cleaning and transformation. This section discusses two key aspects of data preprocessing: addressing class imbalance and performing data transformation.

In learning achievement prediction research, student grades are central indicators of academic performance and serve as targets for both regression and classification tasks. However, datasets used for modeling often exhibit class imbalance, with students achieving extremely low or high grades representing only a small portion of the overall data. The class imbalance issue can severely affect the predictive performance of the learning achievement prediction.

Many studies in the field proceed with modeling based on imbalanced datasets without directly addressing this issue. For example, Al-Musharraf et al. [38] categorized course grades into five classes for learning achievement prediction but observed a disproportionately small number of students in the highest performing class (Class A). Only a few researchers have explored resampling strategies to mitigate the impact of data imbalance on predictive models. For instance, Romero et al. [39] applied random oversampling to rebalance their dataset and assessed the performance of predictive models before and after resampling. Their findings revealed that while resampling improved the performance of some algorithms, its effects varied across different models.

Therefore, identifying the optimal sampling strategy to enhance the effectiveness of predictive models, particularly in the context of the unique characteristics of educational data, remains an area that requires further research and exploration.

Data analysis.

Research context and participants. This study utilized data sourced from XuetangX (https://www.XuetangX.com), encompassing 59,581 learners across six courses: Circuit Principles (I) (courseid: TsinghuaX-20220332_1X-_), Circuit Principles (II) (courseid: TsinghuaX-20220332_2X-_), Data Structures (courseid: TsinghuaX-30240184_1X-_), History of Chinese Architecture (courseid: TsinghuaX/80000901_1X_), Financial Analysis and Decision (Fall 2013) (courseid: TsinghuaX-80512073X-_), and Financial Analysis and Decision (Spring 2014) (courseid: TsinghuaX-80512073_2014_1X-_). Four of these courses were conducted from October 10, 2013, to January 2014, and two from March to June 2014, each spanning a duration of 10 weeks.

A demographic analysis of the learners, depicted in Fig 2, revealed that 67% of participants were male and 33% were female, with ages primarily ranging between 20 and 50 years old. The educational background distribution included 28,605 learners with a bachelor’s degree, 9,159 with a master’s degree, 7,094 with an associate’s degree, 4,662 with a high school diploma, and 994 with a doctoral degree. The data utilized in this study comprised both learner demographic information and behavioral data collected during the courses.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Statistics of participants.

https://doi.org/10.1371/journal.pone.0312124.g002

A comprehensive overview of the learner data is summarized in Table 1. For the prediction phase, data from all 59,581 learners were utilized, among whom 3,155 learners obtained a pass certificate, while the remaining 56,426 learners did not receive a certificate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Details of the dataset.

https://doi.org/10.1371/journal.pone.0312124.t001

In this study, the total MOOC score ranges from 0 to 100. Learners who score between 60 and 100 receive a passing certificate, while those scoring below 60 do not receive a certificate. The relationship between learning achievement and certificate attainment is shown in Table 2. Among the learners in this dataset, 3,155 (5.29%) received a passing certificate, while 56,426 (94.77%) did not. The certificate attainment rate of 5.29% is consistent with typical MOOC patterns, which generally range between 3.5% and 7.3%.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Learning achievement.

https://doi.org/10.1371/journal.pone.0312124.t002

Feature histogram. Regarding the learning behavioral data, histograms were generated to depict the frequency distributions of watch counts and the number of posts. These histograms specifically illustrate the frequency of video clicks and forum postings by learners in each course, as shown in Fig 3. From these figures, it is evident that learners in certain courses, such as History of Chinese Architecture (courseid: TsinghuaX/80000901_1X_), Financial Analysis and Decision (Spring 2014) (courseid: TsinghuaX-80512073_2014_1X-_), and Financial Analysis and Decision (Fall 2013) (courseid: TsinghuaX-80512073X-_), demonstrate higher levels of engagement in forum activities. This observation underscores their active participation in discussions within these courses.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Feature frequency distribution histogram.

https://doi.org/10.1371/journal.pone.0312124.g003

Correlation analysis. Correlation analysis uses statistical indicators to quantify the degree of linear association between continuous variables. Common methods include creating scatter plots, constructing scatter plot matrices, and calculating correlation coefficients. In bivariate correlation analysis, the Pearson correlation coefficient, Spearman correlation coefficient, and Kendall’s tau coefficient are commonly used. This paper utilizes the Pearson correlation coefficient to evaluate the strength of the relationships between dependent and independent variables. The Pearson correlation coefficient is given by Eq (1): (1)

The Pearson correlation coefficient was employed to analyze the raw data, which included eight independent variables and one dependent variable related to learning achievement prediction. To visually represent the degree of correlation between these variables, a heatmap of the correlation coefficient matrix was generated using Python. This heatmap uses color intensity to illustrate the strength of correlations, as shown in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Pearson correlation heatmap.

https://doi.org/10.1371/journal.pone.0312124.g004

Data resampling. Addressing data imbalance is a common challenge in classification tasks, particularly with real-world datasets like those from online learning environments. This issue often leads to skewed distributions, where one class (e.g., dropout students) is significantly underrepresented compared to another (e.g., students who complete their studies). Such imbalances can distort predictive models, causing them to disproportionately favor the dominant class while neglecting instances of the minority class.

Traditional evaluation metrics like accuracy can be misleading in imbalanced datasets because high accuracy may mask the model’s inability to effectively predict the minority class. In dropout prediction tasks, for instance, where the number of students completing their studies is relatively small compared to those who drop out, models may incorrectly favor predicting dropout, thereby compromising overall performance and generalization ability.

To address these challenges, various resampling techniques are employed [42, 43]. Oversampling involves increasing the number of minority class samples by duplicating or synthetically generating new ones, with methods like SMOTE (Synthetic Minority Over-sampling Technique) [44] and ADASYN (Adaptive Synthetic Sampling Approach) [45] being particularly popular. In contrast, undersampling involves randomly removing samples from the majority class to balance the dataset, with techniques like Tomek links and NearMiss commonly used [46]. Mixed sampling combines both oversampling and undersampling to balance class distributions more effectively [47].

Despite the availability of these resampling methods, their application and effectiveness in educational contexts, particularly in predicting student dropout, remain underexplored. It is essential to determine which resampling approach is best suited for educational datasets characterized by class imbalance. This study aims to fill these gaps by evaluating various resampling techniques to enhance the predictive performance of dropout prediction models in academic settings.

Learning achievement prediction based on stacking ensemble learning.

Given the intricate relationships within educational data and the unique strengths of different algorithms, this study employs ensemble learning techniques to improve the accuracy of learning achievement predictions. Specifically, we utilize the stacking ensemble approach, which combines multiple algorithms to enhance overall model performance.

The stacking algorithm uses a hierarchical blending strategy, where various base learners are integrated through a meta learner to boost model accuracy. To reduce overfitting, we select logistic regression as the meta learner. The stacking model consists of two layers: the first layer includes heterogeneous base learners, and the second layer involves the meta learner. The training set is divided using k-fold cross-validation (CV), where each base learner’s predictions are used as inputs for the meta learner, ultimately leading to the final prediction.

Unlike homogeneous ensemble methods that rely on similar base learners, stacking uses diverse learners in parallel, enhancing model diversity and generalization. This makes it particularly effective for predicting learning outcomes in varied educational settings.

Originally introduced by David H. Wolpert in 1992 [48], stacking differs from Bagging and Boosting by combining the outputs of diverse base learners through a meta learner rather than using identical base models. This method, illustrated in Fig 5, enhances model robustness and flexibility, making it a powerful tool in ensemble learning. The process involves splitting the training set into k folds, training base learners on k-1 folds, and using these predictions to train the meta learner. For this study, k was set to 5. The implementation of a two-layer stacking framework involves the following four steps:

• Divide the Training Set: Split the training set into 5 folds using 5-fold cross-validation.
• Train Base Learners: Train the base learners on 4 folds and predict the 5-th fold. Repeat this process for each fold, then concatenate the predictions to form a new training set while retaining the original labels.
• Train Meta Learner: Train the meta learner on the new training set constructed from the base learners’ predictions.
• Predict New Test Set: Use the trained meta learner to predict the new test set samples, yielding the final predictions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Flowchart of the stacking method.

https://doi.org/10.1371/journal.pone.0312124.g005

Explanation of prediction results based on SHAP theory.

In the realm of predicting learning achievement, understanding the inner workings of machine learning algorithms is crucial for meaningful interpretation of their predictions. Existing algorithms, often perceived as “black boxes”, require interpretability analysis to clarify how they arrive at their predictions [49]. This is particularly important in educational contexts, where transparency and insights into prediction outcomes are essential for developing effective intervention strategies.

While traditional ensemble learning models excel in ranking feature importance, they often lack in providing detailed insights into how each feature contributes to individual prediction outcomes. To address this limitation, this study integrates SHAP theory with ensemble learning algorithms. SHAP theory enables comprehensive global importance analysis of features, identifying key predictors that significantly influence learning achievement predictions.

Beyond merely ranking feature importance, SHAP theory elucidates the directional impact of input features on prediction outcomes. It quantifies both positive and negative correlations between features and prediction results, offering nuanced insights into the interactions among features and their respective influences on learning achievement predictions. This analytical approach not only enhances the reliability of the prediction model but also provides novel perspectives for designing targeted teaching interventions tailored to individual student needs.

By integrating SHAP theory with ensemble learning, this study aims to bridge the gap between predictive accuracy and interpretability, thereby empowering educators and researchers with actionable insights to foster student success in educational settings.

The proposed model.

To enhance predictive accuracy in learning achievement prediction, this study employs ensemble learning by integrating multiple machine learning algorithms through a stacking approach. Six diverse algorithms have been selected: K-Nearest Neighbors (KNN), Naive Bayes (NB), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), eXtreme Gradient Boosting (XGBoost), and Multi-Layer Perceptron (MLP). These algorithms were chosen for their effectiveness in handling classification tasks and their ability to complement each other’s strengths within an ensemble framework. Below is a brief overview of each algorithm:

• K-Nearest Neighbors (KNN): KNN is a simple yet effective classification algorithm that assigns a new data point to the most common category among its K nearest neighbors, determined by Euclidean distance.
• Naive Bayes (NB): NB is a probabilistic classifier based on Bayes’ theorem and assumes conditional independence among features. It calculates the posterior probability of each class given the input features and predicts the class with the highest probability.
• Random Forest (RF): RF is a Bagging ensemble learning method that constructs multiple decision trees and aggregates their predictions through voting. It reduces overfitting by randomly selecting features and samples during tree construction.
• Gradient Boosting Decision Tree (GBDT): GBDT builds decision trees sequentially, with each tree correcting the errors of its predecessor. It combines the strengths of boosting and decision trees, achieving high accuracy but requiring careful parameter tuning.
• eXtreme Gradient Boosting (XGBoost): XGBoost is an optimized implementation of gradient boosting that enhances performance and computational speed. It uses a more regularized model to control overfitting and is known for its efficiency in handling large datasets.
• Multi-Layer Perceptron (MLP): MLP is a type of neural network consisting of multiple layers, including input, hidden, and output layers. It learns complex patterns in data through forward propagation and backpropagation of errors, requiring substantial computational resources and data.

The stacking ensemble learning approach integrates diverse algorithms into a hierarchical framework. In the first layer, each base learner (KNN, NB, RF, GBDT, XGBoost, MLP) independently processes the input data and generates predictions. These predictions are then passed to the second layer, where a meta learner (LR in this study) aggregates them to produce the final prediction. The pseudocode for the proposed model is outlined below.

Algorithm 1 Pseudocode of the Stacking Ensemble Learning Model

Require: Training set D = {(x₁, y₁), (x₂, y₂), …, (x_n, y_n)}; Base learning algorithms ; Meta learning algorithm ;

Ensure: Trained ensemble model H(x)

 1: Phase 1: Training Base Learners

 2: for t = 1, 2, …, T do

 3:  Train the t-th base learner h_t on the full dataset D:

 4:  ;

 5: end for

 6: Phase 2: Generating Meta-Features

 7: Initialize the meta-training set D′ = ∅;

 8: for i = 1, 2, …, n do     ▹ For each training instance

 9:  Initialize meta-feature vector z_i = ∅;

 10:  for t = 1, 2, …, T do      ▹ For each base learner

 11:   Compute prediction z_it = h_t(x_i);

 12:   Append z_it to z_i;

 13:  end for

 14:  Add the meta-feature vector and the true label to meta-training set:

 15:  D′ = D′ ∪ (z_i, y_i);

 16: end for

 17: Phase 3: Training Meta Learner

 18: Train the meta-learner h′ on the meta-training set D′:

 19: ;

 20: Phase 4: Making Predictions with the Ensemble Model

 21: Define the final ensemble model as:

 22: return H(x) = h′(h₁(x), h₂(x), …, h_T(x)).

Model performance evaluation.

This study conducts a comparative analysis of a learning achievement prediction method based on stacking ensemble learning compare with six independent machine learning models, underscoring the advantages of the stacking ensemble approach. The six base learning models utilized are KNN, NB, RF, GBDT, XGBoost, and MLP, with Logistic Regression (LR) serving as the meta model to form a Stacking classifier. To address the challenges posed by imbalanced datasets during the experiments, various sampling techniques were employed.

Detailed results and insights derived from the classifier performance are presented in Table 4. Overall, the models demonstrated satisfactory accuracy in predicting student learning achievements, with most achieving approximately 0.9 on the test set. Notably, the stacking ensemble learning models outperformed the six independent machine learning models. Specifically, the ensemble learning model utilizing the OneSidedSelection resampling strategy achieved an accuracy of 0.8520 (compared to NB: 0.6751, KNN: 0.7882, RF: 0.8395, GBDT: 0.8282, XGBoost: 0.8265, and MLP: 0.8048), surpassing all other independent models. This model also demonstrated superior performance metrics, with a higher F1 (0.8597) and precision (0.9853) compared to the independent models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Learning achievement prediction results.

https://doi.org/10.1371/journal.pone.0312124.t004

Receiver Operating Characteristic (ROC) curves are essential tools for assessing the predictive performance of models, particularly in the context of predicting learning achievement. As shown in Fig 6, our proposed ensemble learning model achieves an impressive Area Under the Curve (AUC) of 0.9953, outperforming both the XGBoost and MLP models. The ROC curve’s diagonal line serves as a reference, marking the distinction between true positives and false negatives [50]. This analysis highlights the superior accuracy and performance of stacking ensemble learning models in educational contexts and demonstrates the effectiveness of the stacking ensemble approach compared to individual machine learning models. The findings provide valuable insights into the benefits of ensemble techniques for predicting learning outcomes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Receiver operating characteristic curve.

https://doi.org/10.1371/journal.pone.0312124.g006

Baseline approaches. In addition to comparing against the individual machine learning models that make up the Stacking ensemble (i.e., KNN, NB, RF, GBDT, XGBoost, and MLP), we also conducted comparative experiments with state-of-the-art models (i.e., Song et al. [51], Liu et al. [52], Zerkouk et al. [53]). Below is a brief overview of the baselines:

• Song et al. [51]: This study employs a variant of the Grey Wolf Optimization (GWO) algorithm to optimize the weights and biases of Multi-Layer Perceptron (MLP) models for predicting student achievement.
• Liu et al. [52]: This approach integrates Bi-LSTM with attention mechanisms and LightGBM to predict MOOCs dropouts by effectively modeling both time series and general information features.
• Zerkouk et al. [53]: This model uses XGBoost in combination with logistic regression to develop a binary classification framework that accurately predicts student dropout by analyzing socio-demographic and behavioral data.

The results, as summarized in Table 5 using the XuetangX dataset, consistently demonstrate that our proposed model outperforms the other models across key performance metrics, achieving the highest precision (0.8520), recall (0.8676), F1 (0.8597), and accuracy (0.9853). While the other models show effectiveness in specific areas, they each have limitations that impact their overall performance. For instance, the model in [52] does not adequately address data imbalance, primarily focusing on video features while overlooking critical factors such as student profiles. Similarly, the model in [53] relies exclusively on the XGBoost algorithm, limiting its adaptability by not leveraging the potential benefits of ensemble methods. Additionally, [51] confines its approach to using an MLP model for predicting student performance, thereby missing the advantages of integrating multiple algorithms. Furthermore, none of these models sufficiently tackle the crucial issue of model interpretability, which is vital for enhancing educational outcomes and aiding informed decision-making.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of results with the existing model.

https://doi.org/10.1371/journal.pone.0312124.t005

Our proposed stacking ensemble model effectively overcomes these limitations by integrating the strengths of various models. By employing a meta learner, such as Logistic Regression (LR), to aggregate the outputs of multiple base learners, the model successfully balances the simplicity of linear models with the complexity of non-linear ones, resulting in superior overall performance. Additionally, our approach directly addresses the issue of data imbalance, ensuring more accurate and reliable predictions. We also emphasize model interpretability, which is crucial for deriving actionable insights in educational settings. This strategic integration enables our method to outperform individual models, providing more dependable and precise predictions of learning outcomes.

Model explanation based on SHAP.

This section presents the results of the interpretability analysis conducted using the SHAP method. To illustrate the specific influence of each feature on the prediction of learning achievement, a SHAP summary plot for each feature is introduced (Fig 7).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Model explanation through SHAP.

https://doi.org/10.1371/journal.pone.0312124.g007

In Fig 7, the vertical axis represents the Shapley value for each feature, while the horizontal axis shows the distribution of these values across the samples. Each point on the graph corresponds to a sample, with the color gradient (ranging from blue to red) indicating the feature’s value. The middle axis at a SHAP value of 0 signifies minimal feature importance, where points tend to cluster. The vertical order of the features reflects their relative importance, with the most influential features listed at the top, descending in significance toward the bottom.

The SHAP summary graph for each feature serves two primary purposes: demonstrating global feature importance and illustrating how feature values influence predictions of learning achievement. As shown in Fig 7, quiz score, number of posts, watch time, number of quizzes, and watch counts emerge as the top five influential factors in predicting learning achievement. For instance, a higher quiz score is strongly associated with a greater likelihood of course completion. Similarly, the number of quizzes, as indicated by the number of completed tests, shows a positive correlation with learning performance [25, 54, 55]. The number of posts ranks second, with Fig 7 confirming that increased posting activity enhances completion probabilities, aligning with previous studies [56, 57]. Additionally, watch time and watch counts, which reflect the duration and frequency of video viewing, demonstrate that greater engagement with video content is positively correlated with student success in courses. These findings highlight the critical role of active participation and focused engagement in shaping learning outcomes, as revealed through SHAP analysis.

To evaluate the importance of SHAP-identified high-rank features on model performance, we conducted a comprehensive ablation study. This involved systematically removing each high-rank feature, retraining the model, and analyzing the impact on key performance metrics, including precision, recall, F1, and accuracy. The findings, as shown in Table 6, highlight the crucial role these features play in maintaining the model’s predictive accuracy and generalization capability.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Experimental results after removing high rank features.

https://doi.org/10.1371/journal.pone.0312124.t006

The results clearly demonstrate that SHAP-based high-rank features are vital for the model’s performance. The removal of any top-ranked feature leads to a noticeable decline in all metrics, underscoring their indispensability. For instance, the elimination of critical features like quiz score and the number of posts results in significant performance drops, emphasizing their importance in sustaining predictive accuracy. On the other hand, removing features like age and gender had minimal or even positive effects, suggesting that these factors might contribute to unnecessary model complexity.

Overall, the study confirms that SHAP-identified high-rank features are essential for ensuring the model’s robustness and high predictive accuracy. Their retention is necessary for achieving effective generalization in educational data mining models.

Beyond the technical findings, these insights have practical applications for educators. By focusing on the top-ranked features, educators can design more targeted instructional materials, develop personalized learning plans, and optimize learning activities. For example, if student engagement with interactive content is identified as a key factor, incorporating more quizzes and simulations into lessons could enhance learning outcomes. Additionally, SHAP analysis can inform feedback and assessment strategies, ensuring that feedback is both timely and aligned with the most impactful factors on student success.

The insights from this study can also guide professional development for educators, helping them incorporate these findings into their teaching practices. For instance, workshops on creating engaging content or managing discussion forums can leverage these insights to improve educational practices.

In summary, integrating SHAP-based feature analysis into educational settings not only enhances the practical relevance of our study but also demonstrates its broader impact, enabling educators to create more effective, personalized, and engaging learning environments tailored to their students’ needs.