2.1 Customer churn prediction

In recent years, for the problem of customer churn prediction, various scholars have used a combination of machine learning and data mining to analyze the real causes of customer churn and build appropriate churn prediction models to retain existing customers, among which support vector machine (SVM), decision tree, logistic regression and ensemble learning are widely used in this field for prediction and classification problems. Table 1 summarizes the literature review.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The summary of the literature review.

https://doi.org/10.1371/journal.pone.0289724.t001

Literature [7] proposes the use of Kernel SVM to construct a churn prediction model for a telecom company, in addition, resampling techniques such as Tomek Link and ENN are used on the dataset to deal with unbalanced data. Literature [8] analyzes the sales of a retail store on Friday using a classification model and proposes a machine learning based technique to predict the age group of Friday shoppers for comparison. Machine learning algorithms such as decision trees, KNN, Naïve Bayes and neural networks were used to determine the most appropriate algorithm and the final decision tree achieved an accuracy of 88.22%. Literature [9] addresses the problem of class imbalance and focuses on comparing the performance of six oversampling solutions for handling CIP, demonstrating that MTDF and genetic algorithm based rule generation have the best overall prediction performance. In literature [10], a credit default prediction model was developed using GBDT and the K-means SMOTE oversampling method was used to address the imbalance in the data set, while the original hypothesis was rejected with a p-value < 0.001 using one-way analysis of variance, confirming the statistical significance of the improved performance of the proposed model. Literature [11] uses univariate techniques for feature selection in the customer churn domain and uses a grid search approach to select the optimal hyperparameters for the optimal model GDBT, demonstrating the benefits of applying data transformation methods and feature selection when training an optimized CCP model. Literature [12] proposes a default prediction model based on decision tree model using XGBoost model in integrated learning for accurate prediction of customer default in P2P lending, and also applies feature ranking based on learning model to P2P lending credit data with hyperparameter optimization for individual classifiers. Literature [13] uses the XGBoost algorithm in ensemble learning to predict customer churn in telecommunications and uses the LIME model to interpret the classifier prediction results with a maximum accuracy of 85%. The literature [14] identifies XGBoost as the churn prediction model. To improve the interpretability and transparency of the model, a new method is also proposed to calculate the Shapley values of the possible combinations of features to explain which features are the most relevant ones for the model to be highly interpretable, transparent and explainable to potential customers. In the literature [15], the Improvised-XGBoost churn prediction model was constructed for customer churn prediction in the telecommunication industry, and the loss function of the model was optimized, and the experimental results showed that the model performed better in terms of accuracy and precision, and had better results for some classes of sample prediction. In the literature [16], the K-fold cross-validation method was used to build the churn prediction model, and several common churn prediction algorithms were selected, including SVM, logistic regression, and decision tree, etc. Finally, it was concluded that the Adaboost model and the XGBoost model could achieve better results in terms of AUC with better classification performance.

Although the above studies have contributed to customer churn prediction, most of the current studies on customer churn prediction have used ensemble learning methods to construct customer churn models, for example, in the literature [10–16], ensemble learning has been used to construct the corresponding models. The ensemble learning approach, as a black box model with high complexity, cannot justify the prediction results of the models used. In the case of customer churn, the model can only determine the importance of the feature that influences customer churn, but cannot account for the positivity or negativity of the feature that influences customer churn on the prediction results. The SHAP algorithm is one of the latest interpretable methods that can explain both locally for a single sample and globally for the whole sample. Although the literature [13, 14] also used the same interpretable method to explain and analyze the constructed customer churn prediction model, they did not use innovative methods to optimize the model parameters, which resulted in no significant improvement in the model’s AUC and F1 results. In this study, instead of using grid search and random search for parameter optimization for the selected best model XGBoost, the GA-XGBoost model was constructed using genetic algorithm for parameter optimization. The experimental comparison results prove that the genetic algorithm is more effective compared to the traditional parameter optimization method, and the SHAP interpretable framework is used to transform the black box model of ensemble learning into an easy-to-understand visual mapping, which is convenient for bank staff to conduct in-depth research on customer churn.

2.2 Interpretability in data science

Some studies have already used SHAP interpretation methods in the fields of medicine, finance, and sports. For example, Yi Ming et al [17] proposed an interpretable model of public value in government new media that integrates XGBoost and SHAP model to explain and analyze the articles and comments under "Today’s Headline", and obtained that the article topic type, public value type, article length, content form, etc. are important features that influence the consensus of articles in government headlines. The analysis shows that article type, public value type, article length, and content format are important features that influence the consensus of articles in government headlines. Chen, W. H. et al [18] interpreted the pre-lending overdue identification of Internet finance through the SHAP interpretation framework, visualized from the variable perspective and the sample perspective, respectively, to achieve the effect expressed by the XGBoost model. Yan Luo et al [19] demonstrated that the use of SHAP interpretable models helped clinicians to identify patients at risk of AKI in the ICU more accurately and quickly, and to provide better treatment to patients. AB Parsa et al [20] fused XGBoost and SHAP models to predict and characterize traffic accidents in real time. And X Wen et al [21] fused LightGBM and SHAP models to predict and characterize traffic accidents. Y Meng et al [22] used SHAP visualization to predict and characterize the usefulness of online reviews. Bin Liao et al [23] integrated the SHAP framework to analyze the important factors affecting player value at different positions based on the football player value prediction model to provide decision support for scenarios such as player value assessment, comparative value analysis, and player training strategy development.

In summary, since there are few studies in the field of bank customer churn using the SHAP method, this paper uses the SHAP method to analyze the interpretability of bank customer churn prediction results based on the bank customer dataset on the Kaggle platform, and explains the learning process of the model in terms of both local and global interpretation.

3.1 Technical route

This paper integrates SHAP interpretation framework and GA-XGBoost model to construct bank card customer churn prediction, which is mainly divided into the following steps: 1.Obtain public bank card customer churn dataset from Kaggle official website; 2.Perform data preprocessing, data cleaning, data conversion and other operations on the original dataset; 3.Deal with the problem of uneven distribution of datasets, including oversampling, undersampling, resampling and other methods for comparison; 4. Compare and verify the effect of six machine learning models and select the best model, perform genetic algorithm tuning on the best model, and then compare and verify the model effect; 5. Global and local interpretation of the best model prediction results using the SHAP framework. The technical route of this study is shown in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Technical route.

https://doi.org/10.1371/journal.pone.0289724.g001

3.2 Related theories

3.2.1 GA-XGBoost model construction.

In this paper, a bank customer churn model is constructed using a combination of genetic algorithm (GA) and XGBoost algorithm. XGBoost (Extreme Gradient Boosting) is a new method based on GBDT algorithm proposed by Chen in 2016 [24]. It is used as a boosting tree model to improve the traditional GBDT model, i.e., regularization and second-order Taylor expansion. The objective function of XGBoost for constructing bank customer churn model can be composed of loss function and regularization term. The specific customer churn prediction model objective function is shown in Eq (1), where represents the loss function of the predicted result and the true result; Ω represents the regularization term of the objective function, which is also represented as a penalty term to the complexity of the model [25].

(1)

The objective function obtained by second-order expansion using Taylor’s formula is shown in Eq (2), where g_i is the first-order derivative of x_i and h_i is the second-order derivative of x_i.

(2)

Derivative of ω and making the derivative of the derivative zero gives the ω that minimizes the objective function, as shown in Eq (3).

(3)

L(φ)_min is the minimum value of the objective function after the derivative of ω. The smaller the value of the objective function is, the better the tree model is, corresponding to the minimum value of the objective function as shown in Eq (4).

(4)

The results of the split nodes of the tree model are counted as shown in Eq (5).

(5)

Since the data set used in this paper is large and the XGBoost model has many parameters, the convergence is slow and the parameters have a great influence on the prediction results, and the traditional grid search method for parameter optimization has the problems of low accuracy, long computation time, and avoiding falling into local optimum when adjusting the parameters. Therefore, a new method of parameter tuning by genetic algorithm is used in this study. When performing hyper-parameter optimization, genetic algorithms can jump out of local extremes and use individual fitness functions to optimize the optimal parameter population, and continuously reduce the search area during the optimization process. Therefore, this paper proposes a GA-XGBoost model that combines genetic algorithm and XGBoost model, using the global search capability of genetic algorithm to achieve the optimal selection of XGBoost parameters, and using AUC as the fitness function to achieve the index adjustment. The flow of genetic algorithm is shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Genetic algorithm flow chart.

https://doi.org/10.1371/journal.pone.0289724.g002

The GA-XGBoost optimal hyperparameter combination is the optimal number of chromosomes that the genetic algorithm can obtain when the number of iterations reaches the termination condition. In predicting bank customer churn, the three parameters of n_estimators, learning_rate and max_depth were optimized and initialized using the genetic algorithm. The genetic algorithm was used to obtain the next generation population and the best population was obtained by replicating, crossing and mutating in the parent population and then replacing the inferior population according to the fitness of the offspring population.

The steps for optimizing the XGBoost model based on genetic algorithm are specified as follows.

1. Initialize the XGBoost model parameters θ.
2. Continuous parameter tuning and training using genetic algorithm.
3. Use the AUC index as the fitness function, and select the optimal parameters according to the fitness.
4. Determine whether the maximum number of iterations is reached, if so, output the current training parameter value, otherwise return to (2).
5. Output the optimal parameter values and test the results in the test set.

3.2.2 SHAP interpretation framework.

Based on Section 3.2.1, the GA-XGBoost algorithm can train a bank’s customer churn prediction model with high accuracy, but due to the high complexity of the XGBoost model, as a black-box model, the interpretability is poor and the model cannot analyze how the sample eigenvalues affect the final prediction results. The bank wanted to understand the relationship between sample features and customer churn outcomes, and needed to analyze the correlation between feature importance and predicted churn outcomes. To solve the problem of poor interpretability of the GA-XGBoost model, the SHAP framework is introduced to analyze the reliability interpretation of the GA-XGBoost model prediction results.

SHAP (Shapley additive explanations) is an explanatory framework for explaining black box models, proposed by Lundberg in 2017 [26]. Using the optimal Shapley value in game theory, the model gives a prediction value for each prediction sample, and the SHAP algorithm can calculate the Shapley value of each feature as a measure of the contribution of each feature to the overall prediction result. SHAP interprets the prediction value of the model in two ways, the Shapley value of each input feature and the SHAP algorithm’s analysis of each feature in terms of its importance in the overall sample [27].

The SHAP framework has powerful visualization capabilities, with the ability to show the interpretation of model predictions, and is widely used to interpret more complex classification and regression models. The traditional feature importance ranking method can only determine the importance of a feature and cannot explain how the feature affects the prediction results, while the main advantage of the SHAP value is that it can explain and analyze the degree of influence of each feature and also reflect the positive and negative influence of each feature. The results of the explanatory model of this framework are consistent with human intuition [28].

For GA-XGBoost model for customer churn classification, the model output is a probability value. Suppose x_i is the i-th customer sample, where the j-th feature of x_i is x_ij, y_i is the model′s prediction of whether the sample is churned or not, y_base is the mean value of all sample target variables, and f(x_ij) contribution of the j-th feature in the i-th sample to the final prediction y_i, the SHAP value is calculated as shown in Eq (6).

(6)

For the SHAP value of each feature, when f(x_ij)>0, it means that the feature has a boosting effect on the prediction result, and when f(x_ij)<0, it means that the feature makes the contribution lower. the biggest advantage of the SHAP framework is that it can correctly reflect the impact of each feature and the positivity and negativity of the impact.

3.3 Experimental data and its preprocessing

To test the effectiveness of the proposed method, the Credit Card Customer dataset, which consists of personal information, behavioural information and transaction information of bank customers, is taken from the Kaggle website. The specific dataset URL is https://www.kaggle.com/datasets/sakshigoyal7/credit-card-customers. The details of the characteristic variables in the dataset are shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Details of feature variables of the data set.

https://doi.org/10.1371/journal.pone.0289724.t002

Fig 3 shows the variable correlation heat map, where the feature correlation heat map is plotted to explore and analyze the features that influence bank customer churn. The lighter color of the heat map indicates the lower correlation of two features, and the darker color indicates the higher correlation of two features. From the feature correlation heat map, we can analyze that the correlation between Avg_Utilization_Ratio and the average value of credit limit, total turnover balance, and available credit limit in the past year are -0.48, 0.63, and -0.54, respectively, and the correlation coefficient between them is large, so we decide to delete the "Avg_Utilization_Ratio" feature.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Heat map of variable correlation.

https://doi.org/10.1371/journal.pone.0289724.g003

In this paper, One-Hot unique hot coding is introduced to process the category-type variables, and Marital_Status, Education_Level, and Card_Category are assigned to the coding. After the unique hot coding process, all the features of the data set are continuous type variables, and then this paper uses the Z-score normalization method to normalize the original data so that the feature data conform to the Gaussian normal distribution.

The chi-square test, which is a free distribution test, does not exist for specific parameters and the assumption of overall normal distribution, and the mean and variance cannot be calculated. For the bank customer dataset we used, we set H₀: the feature has no effect on customer churn results; H₁: the feature has an effect on customer churn results. Through the experimental results in Table 3, we can conclude that the p-values of Gender, Income_Category, and Education_Level are almost less than 0.05, and the original hypothesis H₀ is rejected, that is These features have a significant effect on customer churn results. p-value indicates the probability of mutual independence when the sample row variables and column variables are drawn from the overall, and since the probability value of p is small, we reject the original hypothesis that the features and customer churn results are independent of each other.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Chi-square test for selected features.

https://doi.org/10.1371/journal.pone.0289724.t003

3.4 Experimental environment and evaluation index

In the experiments, the algorithm is programmed in Python 3.6 language using XGBoost, Imbalance, and Scikit-learn toolkit.

In the area of bank customer churn, due to the imbalance of dichotomous data in the dataset, the learning metrics for evaluating bank customer churn were chosen to focus the evaluation metrics on positive cases, where accuracy, recall, precision, and F1 score metrics were obtained from the confusion matrix, as shown in Table 4. In Table 4, TP indicates actual customer churn and predicted customer churn; FN indicates actual customer churn but predicted customer non-churn; FP indicates actual customer non-churn but predicted customer churn; TN indicates actual customer non-churn and predicted customer non-churn.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Confusion matrix.

https://doi.org/10.1371/journal.pone.0289724.t004

In the bank customer churn prediction problem, the evaluation metrics used based on the confusion matrix are accuracy, precision, recall, and F1-score, respectively. the specific formulas are shown in the following order.

(7)

(8)

(9)

(10)

The AUC value is the area under the ROC curve, which is a curve plotted with TPR as the vertical axis and FPR as the horizontal axis. Where TPR = TP/(TP+TN) and FPR = FP/(FP+TN), the AUC value is suitable for the overall evaluation of the bank customer churn prediction model, and the closer the value is to 1, the better the performance of the bank customer churn prediction model.

4.1 Unbalanced data set processing

The distribution of whether the data set in Fig 4 is churned or not can be analyzed to show that the percentage of churned customers is low, and the ratio of online customers to churned customers is about 5:1. The distribution of the data set is extremely unbalanced, customer data is easily predicted as majority class samples, which leads to poor prediction results. Using the XGBoost model for prediction, although the accuracy rate can reach 90%, the recall rate is only 38%, and the value of AUC is 0.69, indicating that the model classification effect is weak, and it easily predicts customer data as majority class samples, leading to poor prediction results. To solve the data imbalance problem, this paper uses multiple resampling algorithms to achieve balanced processing by changing the distribution of data. In the experiments, the ADASYN and SMOTE algorithms of the oversampling method and the SMOTEENN algorithm of the hybrid sampling method are used, and the data processed by different sampling algorithms are substituted into the XGBoost model for training, and the results are evaluated by multiple evaluation metrics, and finally the algorithm with the best effect is selected to deal with the data imbalance. A comparison of the metrics of different sampling algorithms in the XGBoost model is shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Distribution of bank customer churn label.

https://doi.org/10.1371/journal.pone.0289724.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Performance comparison of different adoption algorithms in XGBoost model.

https://doi.org/10.1371/journal.pone.0289724.t005

The SMOTE algorithm increases the number of minority class samples by manually synthesizing new samples, and its main method is to generate new samples by linear interpolation among the minority class samples. The ADASYN algorithm, on the other hand, synthesizes a new minority class sample by interpolating a certain proportion of the minority class samples. Compared to SMOTE, the ADASYN method places more emphasis on the problem of discrimination difficulty of minority class samples, and more minority class samples can be generated around the positive class sample data. The SMOTEENN algorithm is a composite sampling technique combining the SMOTE and ENN algorithms, which first increases the minority class samples by linear interpolation using the SMOTE method, and then the majority class samples are removed by the ENN algorithm. The noisy data in the samples that are different from their majority K-nearest neighbor sample classes are removed by the ENN algorithm [29].

The experimental comparison shows that the evaluation metrics of several sampling algorithms outperform the unprocessed data samples, so the use of sampling algorithms to balance the data set is suitable for modeling bank customer churn. Comparing the three sampling algorithms, the SMOTEENN algorithm of the hybrid sampling method performs best with an AUC of 0.99, an accuracy of 0.96, and a recall of 0.92, which is better than the ADASYN and SMOTE algorithms of the oversampling method. Therefore, the best algorithm, SMOTEENN, is used to balance the bank customer data set.

4.2 Comparison of experimental results

In this paper, the SMOTEENN algorithm is used in combination with a machine learning classification algorithm. The XGBoost model is selected to compare the performance with five classification models, namely LightGBM, DecisionTree, KNN, GDBT and ExtraTrees, and the comparison results are shown in Table 6 and Fig 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Comparison results of each classification model.

https://doi.org/10.1371/journal.pone.0289724.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Comparison results of different model.

https://doi.org/10.1371/journal.pone.0289724.t006

Comparing the results in Table 6 and Fig 5, it can be seen that XGBoost model for predicting customer churn has the highest accuracy, precision, F1 value, and AUC of 0.9608, 0.8488, 0.8830, and 0.9902, respectively, among the six models, and only has a slightly lower recall than LightGBM’s 0.9242. The XGBoost algorithm stores the input data as a block structure, and the block structure greatly reduces the computation and improves the prediction speed when predicting the input data. In addition, XGBoost can also perform second-order Taylor expansion on the objective function and add regularization terms, which can prevent the overfitting phenomenon. Compared with other comparative algorithms, XGBoost algorithm is more stable and the prediction results are more accurate than other algorithms.

4.3 Model hyperparameter optimization

The XGBoost model has various parameter types, and three parameters, n_estimators, learning_rate, and max_depth, which have a large impact on the model, are selected for tuning in this paper. The parameters are optimized by genetic algorithm, and AUC is used as the fitness function of genetic algorithm. The obtained tuning results are shown in Table 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Results of genetic algorithm tuning parameters.

https://doi.org/10.1371/journal.pone.0289724.t007

According to the characteristics of the genetic algorithm and the range of XGBoost parameters, the tuning parameters of the genetic algorithm are set as follows: the population size is 50, the maximum number of evolutionary generations is 10, the log information is recorded every second time, the judgment threshold of single-objective optimization into stagnation is 1e-6, and the maximum upper limit of the evolutionary stagnation counter is 10. As shown in Figs 6 and 7, after 10 iterations, the population optimal single objective function value converges to the population average single objective function value and searches for the optimal fitness function AUC of 0.9912. The final approximate optimal solution of GA-XGBoost is n_estimators = 265, learning_rate = 0.097, max_depth = 5. 5. The results of the XGBoost model after genetic algorithm tuning are shown in Table 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Iteration diagram of genetic algorithm.

https://doi.org/10.1371/journal.pone.0289724.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. GA-XGBoost optimization process diagram.

https://doi.org/10.1371/journal.pone.0289724.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Comparison of GA-XGBoost with XGBoost and LightGBM test results.

https://doi.org/10.1371/journal.pone.0289724.t008

According to the comparison of model tuning indexes shown in Table 8, GA-XGBoost is the model after genetic algorithm tuning. The experimental results show that the GA-XGBoost model has improved in each evaluation index, and the precision, recall, F1 value, accuracy, and AUC are optimal, which are 0.8760, 0.9262, 0.9004, 0.9671, and 0.9912, respectively. The XGBoost model after parameter optimization using genetic algorithm performs optimally in terms of generalization performance and time consumption due to the proper combination of parameters.

4.4 Model performance analysis

The GA-XGBoost model optimized with XGBoost hyperparameters by genetic algorithm was used in the test set to predict bank customer churn, and the confusion matrix of the test set prediction results is shown in Fig 8. In the test set, the actual number of churned customers was 488, and the GA-XGBoost model judged that 452 customers were churned and 36 were misjudged, with a recall rate of 0.9262; the actual number of unchurned customers was 2551, and the GA-XGBoost model judged that 2487 customers were unchurned and 64 were misjudged, with an accuracy rate of 0.8760. The area under the ROC curve of GA-XGBoost is 0.9912, as shown in Fig 9. The GA-XGBoost model metrics are good by performance analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. GA-XGBoost confusion matrix.

https://doi.org/10.1371/journal.pone.0289724.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. ROC curve of GA-XGBoost.

https://doi.org/10.1371/journal.pone.0289724.g009

4.5 Model comparison analysis

Finally, after the bank customer dataset was balanced by SMOTEENN, we built seven different machine learning models of LightGBM, DecisionTree, KNN, GBDT, ExtraTrees, XGBoost, and GA-XGBoost, respectively. The model comparison results are shown in Fig 10 and Table 9, where the precision, recall, F1 score, accuracy, and AUC of GA-XGBoost are 0.8760, 0.9262, 0.9004, 0.9671, and 0.9912, respectively, and each evaluation index is the highest among the seven different models. Through the experimental comparison, GA-XGBoost is the most effective in predicting bank customer churn, which can effectively help commercial banks to deal with customer churn in advance and make corresponding retention plans in real life.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. ROC curve of models.

https://doi.org/10.1371/journal.pone.0289724.g010

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Comparison of models test results.

https://doi.org/10.1371/journal.pone.0289724.t009

4.6 SHAP-based model interpretation

In this paper, we use the SHAP method to explain the learning process of the model, which is the degree of contribution of each feature to improve the prediction ability of the model. It is a more direct representation of the degree of influence of the features on the model. The focus of this chapter is on the explanatory analysis of the model prediction results of bank customer churn based on the SHAP framework.

4.6.1 Global explanation.

The results of the GA-XGBoost model analysis using the SHAP method are shown in the summary plot of the feature analysis in Fig 11, where each row represents a feature and the horizontal axis is the SHAP value of the model, i.e., the impact of the model output. A point represents a sample, and a red color indicates a larger feature value and a blue color indicates a smaller feature value. Fig 11 shows that Total_Trans_Ct is an important feature that is basically negatively correlated with the probability of churn. When the total number of transactions in the last year is low, the probability of churning is higher. On the other hand, when the total number of transactions in the last year is high, the probability of churn gradually decreases because the bank has multiple transactions with its customers and thus the level of trust increases. This characteristic of Total_Relationship_Count also has a significant impact on churn. The red dot area is mainly concentrated in the area where SHAP is less than zero, and the analysis shows that the more customers hold bank products, the less likely they are to churn. Total_Trans_Amt is positively correlated with SHAP, i.e. the higher the value, the higher the likelihood of churn. By analyzing the results of SHAP value, the total amount of transactions in the last year, due to the increase in the amount, customers may choose other products with higher returns and risks, resulting in lower sales of bank products.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Summary chart of SHAP feature analysis.

https://doi.org/10.1371/journal.pone.0289724.g011

Fig 12 shows the SHAP summary plot of the GA-XGBoost model feature importance ranking. This summary plot combines the importance of the features with the impact of the features. The position of the Y-axis is determined by the features that affect customer churn, and the position of the X-axis is determined by the Shapley value of each feature. From the figure, we can see that the three most important features in the model are Total_Trans_Ct, Total_Trans_Amt, and Total_Revolving_Bal. The Total_Trans_Ct feature has the greatest impact on the mode. The higher the value of this characteristic, the higher the Shapley value and the lower the probability of customer churn. The analysis of these key variables indicates that the number of transactions and the amount of transactions by users in the bank can be more effective in helping banks predict user churn. In addition, the results of the feature importance ranking found that the features of customer gender, presence of partners, and credit limit are low in influencing customer churn, and the bank can appropriately reduce the analysis of these features for influencing customer churn.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. SHAP summary plot of feature importance ranking of GA-XGBoost model.

https://doi.org/10.1371/journal.pone.0289724.g012

According to the feature importance ranking of SHAP in Fig 12 and the feature importance ranking of GA-XGBoost model in Fig 13, it can be seen that the ranking feature order is not exactly the same, and it can be concluded that the most critical factor influencing customer churn is Total_Trans_Ct, and both ranking algorithms rank the feature of Total_Trans_Ct as the factor influencing customer churn the most. In addition, both ranking algorithms rank Total_Trans_Ct, Total_Trans_Amt, Total_Revolving_Bal, and Total_Relationship_Count as the most important factors influencing customer churn. Through the comparative analysis of the characteristic importance ranking, the existing customers of the bank are less likely to churn the more bank products they hold and the more total transactions they make. Banks can launch specific financial products for this category of high-value users in the future customer relationship maintenance process, strengthen ties with them and involve this group in various banking operations and activities of the bank.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. GA-XGBoost feature importance order diagram.

https://doi.org/10.1371/journal.pone.0289724.g013

4.6.2 Partial explanation.

For each sample that is predicted to be lost or not, the GA-XGBoost model produces a predicted value, and the SHAP value is the value obtained from the sample features. As shown in Fig 14, for this sample of non-churning users, the values of Total_Trans_Ct and Total_Revolving_Bal are small and blue, indicating that these features reduce the SHAP value for this sample. A total of 87 transactions in the last year and a total working balance of $1,595 make it more likely that this user will continue to use the bank’s products. However, it is also worth noting that the number of products held by the customer and the age of the customer also affect the likelihood of churn for this user. Since the number of products held by this user is 2 and his age is 47, the user can be considered a middle-aged customer who frequently uses the bank’s transactions but is new to the bank’s financial products according to the user profile.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Predicted effect of SHAP feature analysis for non-churning customers.

https://doi.org/10.1371/journal.pone.0289724.g014

As shown in Fig 15, for this sample of churned customers, the values of Total_Trans_Amt, Total_Trans_Ct, and Income-Category are large and red, indicating that these characteristics increase the SHAP value of this sample and play a positive role in the churn results. The bank can develop appropriate retention strategies for middle-aged customers with low income, such as offering products with high benefits and high cost performance, to attract them to participate in the bank’s business, strengthen the relationship with them, and increase their loyalty to the bank. This group of customers can be attracted to participate in the bank’s services, strengthen the relationship with customers, enhance their loyalty to the bank, and optimize marketing costs and resources.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. Effect of SHAP feature analysis for predicted churn customers.

https://doi.org/10.1371/journal.pone.0289724.g015

Since Total_Trans_Amt is strongly correlated with Total_Trans_Ct, Fig 16 shows the effect of Total_Trans_Ct and Total_Trans_Amt on predicting whether a customer will churn. The Shapley value is mostly positive until the Total_Trans_Ct is 60, and after the Total_Trans_Ct is 60, the Shapley value gradually decreases and becomes negative. The analysis of the SHAP values shows that the main effect of Total_Trans_Ct is negative, i.e. the probability of causing customer churn is lower when this characteristic is larger. The red dots appear more often when the SHAP value of Total_Trans_Ct is low, indicating a positive correlation between Total_Trans_Ct and Total_Trans_Amt.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. SHAP feature dependence plot of Total_Trans_Ct and Total_Trans_Amt on model impact.

https://doi.org/10.1371/journal.pone.0289724.g016

The SHAP waterfall plot of the churn prediction model for the 2nd user sample is shown in Fig 17, which illustrates how each feature drives the model output from the baseline value to the actual output value. In the figure, E[f(X)] is the average of all sample SHAPs with a value of 0.024; f(X) is the actual input SHAP value of the model in the current sample with a value of -8.547. By comparing f(X) < E[f(X)], it can be determined that the sample of users is an unchurned customer. The color of the SHAP value indicates the positive and negative impact of the sample on the predicted outcome, with blue indicating characteristics that decrease the predicted churn and red indicating characteristics that increase the predicted churn. In this user sample, Total_Trans_Ct is the most influential feature with a score of 87, causing a decrease in SHAP of 6.87. The second most important feature is Total_Revolving_Bal, which causes a decrease in SHAP of 0.98. In addition, the number of products owned by the customer is two and the age of the customer is 47, which increases the possibility of churning, but since the total SHAP value is negative, the model judges the user as a non-churning customer.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 17. Churn prediction SHAP waterfall for the 2nd user sample.

https://doi.org/10.1371/journal.pone.0289724.g017