Data description

Data was collected over 10 years (2011–2021) from patients at Imam Khomeini Hospital in Tehran, Iran. The dataset initially comprised 67 features and 6403 records, including prescription details and dates related to various conditions. The full list of variables and their corresponding details is provided in Supplementary S1 Table. All patients in this dataset were diagnosed with T2DM, based on criteria from the American Diabetes Association (ADA) [23]. Diagnostic tests included fasting blood sugar (FBS), HbA1c, the 2-hour postprandial glucose test (2HPP), and the glucose tolerance test (GTT). Patients were classified as having T2DM if they exhibited HbA1c levels ≥ 6.5%, FBS levels ≥ 126 mg/dl, or 2HPP levels ≥ 200 mg/dl.

Participants were non-pregnant, without serious liver-related diseases, and over 20 years old. To ensure accurate fatty liver diagnosis, only patients who had undergone abdominal ultrasonography were included, reducing the dataset to 3964 records.

This study was approved by the Research Ethics Committee of Tehran University of Medical Sciences (Approval ID: IR.TUMS.IKHC.REC.1400.221). Written informed consent was obtained from all participants prior to their inclusion in the study. All data were anonymized prior to analysis, and the study adhered to the ethical standards for research involving human subjects. The dataset was accessed for research purposes on 18 Feburary 2023. All data were fully anonymized prior to access, and none of the authors had access to personally identifiable information at any stage during or after data collection.

Data cleaning

Initially, irrelevant features were removed based on expert opinion to focus on relevant clinical variables. Acceptable ranges for each variable were determined, and values outside these ranges were treated as missing. Features with more than 45% missing values and rows with more than 50% missing values or duplicates were excluded. This resulted in 3769 records and 31 variables, plus the target variable (MAFLD). The robustness of these thresholds was further assessed through a sensitivity analysis using alternative cutoffs (30% and 60%), which demonstrated no material differences in cohort characteristics across scenarios (all standardized mean difference (SMD) < 0.01, Kolmogorov–Smirnov (KS) p = 1.00; see Supplementary S2 Table).

The features included demographic characteristics: sex, age, height, weight, waist circumference, hip circumference (Hip), and body mass index (BMI); medical history: diabetic duration (DDM), retinopathy (Retino), hypertension (HTN), cerebrovascular accident (CVA), and coronary artery disease (CAD). CAD was defined as having percutaneous coronary intervention (PCI), coronary artery bypass graft (CABG), or angioplasty. Additionally, features based on medical test results included: platelet count (PLT), diastolic blood pressure (DBP), C-reactive protein (CRP), vitamin D (VitD), insulin, total cholesterol (Chl), HDL cholesterol, LDL cholesterol, triglycerides (TG), aspartate aminotransferase (AST), ALT, alkaline phosphatase (ALKP), homeostatic model assessment for insulin resistance (HOMA), smoking status, fasting blood sugar (FBS), 2-hour postprandial blood sugar (2HPP), HbA1c, creatinine (Cr), and uric acid (UA).

Machine learning methods

Eight machine learning algorithms were implemented and compared for MAFLD prediction: Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), Gradient Boosting (GB), XGBoost, LightGBM, and K-Nearest Neighbors (KNN). These models were selected to represent a range of linear, tree-based, and ensemble approaches commonly used in biomedical prediction tasks.

Model performance was assessed using accuracy, precision, recall (sensitivity), F1-score, and the area under the receiver operating characteristic curve (AUC). In addition, the R² (coefficient of determination) was used to evaluate the goodness of fit for imputation models during the preprocessing stage. For each classifier, performance metrics were computed as the mean and 95% confidence interval (CI) across the five cross-validation folds.

Descriptions of the models and the mathematical definitions and formulae for all evaluation metrics (Accuracy, Precision, Recall, F1, AUC, and R²) are provided in Supplementary S3 and S4 Tables.

Handling missing values

After cleaning the dataset, 25 out of 31 variables had missing values which are shown in Fig 2. Features without missing values included sex, age, height, weight, diastolic blood pressure, and BMI.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Percentage of missing values for each variable with missing data.

https://doi.org/10.1371/journal.pone.0339580.g002

Various approaches exist for imputing missing data. Some methods are univariate, meaning they rely solely on the values within each feature independently to estimate missing values. Examples include the Mean and Median imputation techniques, where missing values in a feature are replaced by the mean or median of the observed values in that same feature. In contrast, multivariate methods leverage the values of other variables to predict missing data [24]. Selecting an appropriate imputation method requires an understanding of the nature of missing data, as the relationship between missingness and observed values influences the choice of approach. Based on the literature, missing data can be categorized into three types [25]:

Missing Completely at Random (MCAR): In this category the missingness is entirely random, and every record has an equal chance of having missing data in any variable.

Missing at Random (MAR): The probability of a missing value in a variable depends on other observed variables but is independent of the actual missing value.

Missing Not at Random (MNAR): The missingness is related to the unobserved value itself. For example, older individuals may be less likely to report their age.

In this study, data collection was conducted systematically by medical staff, meaning that the values of the records themselves were unlikely to influence the missingness mechanism. Consequently, the probability of MNAR is low. Instead, there remains a possibility that the missingness follows the MAR pattern, which can be statistically tested.

To determine the missingness mechanism in the dataset, Little’s MCAR test was applied. This statistical test first identifies distinct missingness patterns, i.e., which features had missing values simultaneously and then assesses whether the patterns occur randomly using a chi-square test [26]. The results (chi-square value = 6,987, degrees of freedom = 3,832, p-value ≈ 0) rejected the MCAR hypothesis, indicating that the data are MAR. This finding suggests a relationship between the probability of missingness and the values of other observed variables. Since the missing data mechanism was assumed to be MAR, imputations were conditioned on other observed variables. Therefore, the marginal distributions of variables may differ after imputation, which is expected and does not indicate bias under the MAR assumption.

Based on this result, univariate methods may not be suitable, as they do not account for interdependencies between variables. Instead, multivariate imputation methods, which consider relationships among multiple variables, were considered more appropriate for handling missing values in this study.

Types of missing values

Given that the missing data were MAR, multivariate imputation methods were employed to estimate the missing values more accurately. Three popular techniques were utilized:

1. Multiple Imputation by Chained Equations (MICE): MICE is an iterative method for handling missing data by generating multiple imputations for each missing value. It uses a series of predictive models to estimate missing values based on other observed variables in the dataset. A key advantage of MICE is its flexibility in model selection, allowing users to specify different machine learning algorithms for each variable during the imputation process, making it adaptable to various data characteristics [24].
2. K-Nearest Neighbors (KNN) Imputer: The KNN imputer estimates missing values by identifying the ‘k’ instances in the dataset that are most like the instance with missing data, based on a defined distance metric (e.g., Euclidean distance). For a given missing value, the algorithm imputes it using the mean (for continuous data) or mode (for categorical data) of these ‘k’ nearest neighbors. This method leverages the similarity between instances to provide accurate imputations [27].
3. Missforest: Missforest is a non-parametric imputation method that utilizes random forests to predict missing values. It can handle both continuous and categorical data and is capable of capturing complex interactions between variables without requiring explicit model specification. The algorithm iteratively trains a random forest model on the observed data to predict missing values, updating the imputed dataset until convergence is achieved [28].

Missing value simulation

Since the actual missing values in the dataset are unknown, an effective approach for evaluating different imputation methods is to simulate missingness patterns [25]. The results of Little’s MCAR test indicate that the MAR hypothesis is rejected, meaning there is a relationship between the probability of missingness and the values of other features. Therefore, a simulation process was designed to replicate this pattern.

To conduct this process, a subset of the dataset where all values were available was first selected, ensuring that no missing values existed initially. Within this subset, null values were artificially introduced, and various imputation methods were then applied to fill in the missing data. Since the original values were known, the imputed results could be compared against the actual values, allowing for the determination of the most effective imputation method.

To ensure that the simulated missing values closely resembled real missingness patterns, observed relationships between missing and non-missing values in the full dataset were used. The process involved selecting one column at a time as the target column, where missing values were labeled as “1” and observed values as “0.” A classifier was then trained using the remaining columns as input features to predict the probability of missingness in the target column. This process was repeated for all columns containing missing values. Multiple classifiers were trained for each column, and the best-performing model was chosen based on its ability to predict missing data. As a result, each column with missing values was assigned an optimal classifier that best modeled its missingness pattern. For instance, in the dataset with 31 columns, 25 columns contained missing values. Each of these 25 columns was iteratively selected as the target column, where missing and observed values were encoded as “1” and “0,” respectively. The remaining 30 columns were used as input features for training classifiers to predict missingness probability. By the end of this step, 25 trained classifiers had been obtained, one for each column with missing values.

Next, these trained classifiers were applied to the complete dataset (which initially had no missing values) to simulate missingness. Again, each column was chosen as the target, and its corresponding trained classifier was used to predict the probability of missingness for each record based on the values of other columns. To ensure that the missing values reflected real-world distributions, the missingness ratio observed in the full dataset was replicated. For example, if a column originally had 20% missing values, then in the simulated subset, the top 20% of records with the highest predicted missingness probability were artificially set to null.

By repeating this process for all columns, a simulated missing dataset was generated that closely mirrored the real missingness patterns, providing a robust foundation for evaluating different imputation methods.

The process has been presented in Fig 3. To implement the described method on our dataset, the data was divided into two halves. The first half was used to extract a fully observed subset by removing all rows containing missing values; this ensured a complete dataset for simulating missingness. The second half was used to develop models trained to predict the probability of missingness based on observed values in other columns. To ensure the rigor and stability of the results, the entire simulation procedure, from data splitting through imputation evaluation, was repeated five times using different random seeds.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The process of simulating missingness in the dataset.

https://doi.org/10.1371/journal.pone.0339580.g003

Prediction of probability of missingness

Eight classifiers were chosen for this task, including four basic models and four ensemble models: LG, KNN, SVM, DT, RF, ET, AdaBoost, and XGBoost.

Hyperparameter tuning was conducted using Grid Search with cross-validation to optimize each classifier’s performance. Grid Search cross-validation (CV) is a systematic approach to hyperparameter tuning in which a predefined grid of hyperparameter values is exhaustively searched [29]. Each classifier was trained and tuned for columns with missing values, and the best classifier was selected based on the F1. The F1-score was chosen due to the imbalance in missingness across columns, ensuring unbiased classification.

Evaluating imputation methods

To simulate missingness in the second half of the dataset, the best classifier was used for each variable to predict the probability of missing data. Nine imputation methods were evaluated, including different implementations of MICE with estimators such as Ridge Regression, SVM, DT, RF, ET, AdaBoost, and XGBoost, along with KNN and missforest. The imputation methods were assessed by calculating R² for continuous variables and Accuracy for binary variables. A weighted average was computed for each metric, considering the proportion of missing values in each variable. This provided an overall performance measure for each imputation method, aiding in identifying the most effective one. The complete hyperparameter grids and the best-performing configurations of all imputation models, are provided in Supplementary S5 Table.

Outlier detection

After data imputation, outlier detection was performed to eliminate noise from incorrectly imported data while retaining valuable outlier values. Outliers were identified after imputation, as this ensured that true missingness patterns did not affect the detection process.

Despite the typical use of statistical approaches in medical studies [30], given the unknown distribution of the dataset, a density-based method, Local Outlier Factor (LOF), was used. LOF detects outliers by assessing the local density of each data point relative to the densities of its neighboring points. It computes a score that reflects how isolated a point is relative to its surrounding points; a significantly lower local density suggests that the point is an outlier [31]. This method detected 29 records as outliers, which were removed due to apparent data issues.

Balancing the dataset

After outlier removal, 3740 records remained, 1827 (48.8%) in class 1 (MAFLD) and 1913 (51.1%) in class 0 (non-MAFLD), indicating a slight imbalance. Using an unbalanced dataset can bias the machine learning models toward the majority class. Therefore, the dataset was balanced by randomly removing 86 rows from class 0, resulting in a final dataset of 3654 records.

Training and testing procedure

The dataset was first divided into independent training (80%) and test (20%) subsets. All model development steps were performed exclusively on the training data. Within the training subset, a stratified 5-fold cross-validation was used to conduct hyperparameter tuning through grid search. After obtaining the optimal hyperparameters, feature selection was carried out on the entire training set. The trained models were then re-evaluated within the training data using 5-fold cross-validation to ensure robust estimation. The independent test set was kept completely unseen during these steps and was used only for the final evaluation and comparison of models.

Feature selection

To identify the most relevant variables for predicting MAFLD, a diverse set of feature selection methods was intentionally chosen to encompass different approaches, including filter-based (SelectKBest), wrapper-based (Recursive Feature Elimination with Cross-Validation), feature extraction (Principal Component Analysis), and a metaheuristic-based approach (Genetic Algorithm). This selection was motivated by the goal of exploring and comparing distinct methodologies, ranging from statistical filtering to iterative optimization, to ensure a comprehensive evaluation of feature importance and enhance the robustness of the predictive model.

Select K Best: Select K Best is a univariate feature selection technique that ranks features based on their individual relationship with the target variable using a specified scoring function. Common scoring metrics include the ANOVA F-test for numerical input features and mutual information for categorical variables. The top K features with the highest scores are retained for further analysis. This method is computationally efficient and widely used in preliminary feature selection steps [32].

Principal Component Analysis (PCA): PCA is an unsupervised technique that reduces the dimensionality of the input data by projecting the original features onto a new set of uncorrelated axes, referred to as principal components. These components are ranked by the amount of variance they capture in the dataset, with the first few components preserving most of the information. PCA is particularly useful for addressing multicollinearity, reducing noise, and improving model efficiency, though it may lead to a loss of interpretability since transformed features are linear combinations of the original ones [33].

Recursive Feature Elimination with Cross-Validation (RFECV): RFECV is a wrapper-based feature selection method that iteratively eliminates the least important features, optimizing model performance through cross-validation. In each iteration, a machine learning algorithm is trained, and based on metrics like coefficient weights the feature with the lowest importance is eliminated. These iterations continue until the best possible subset of features is selected. RFECV incorporates cross-validation to ensure robustness, making it a reliable method for selecting highly predictive feature [34].

Genetic Algorithm (GA) for Feature Selection: GA is a metaheuristic optimization technique inspired by natural selection. It encodes potential solutions (feature subsets) as chromosomes, which undergo evolutionary processes such as crossover (combining parts of different solutions) and mutation (introducing small random changes). Over successive generations, individuals with higher fitness, determined by an evaluation function measuring model performance, are more likely to be selected. This iterative process leads to the discovery of an optimal or near-optimal feature subset [35].

K-fold cross-validation was used for training. For K Best and PCA, tuning involved varying the number of output features. The Genetic Algorithm’s hyperparameters were optimized using the Taguchi method, a design-of-experiments approach that systematically evaluates parameter combinations to identify robust settings with minimal computational cost [36]. Python 3, along with numpy, pandas, scikit-learn, SciPy, XGBoost, and lightgbm, was employed for data processing and classification.

Classification

After completing the preprocessing steps, several machine learning algorithms were applied to classify patients with and without MAFLD. The baseline models included LG, KNN, SVM, and DT. To enhance performance, ensemble methods, which combine multiple models, were employed. The DT model was selected as the base model due to its interpretability and frequent use in ensemble methods. While Decision Trees are prone to overfitting, performance evaluation using cross-validation indicated that overfitting was not a major concern in this case, potentially due to appropriate hyperparameter tuning. Consequently, the focus was placed on boosting methods rather than bagging, as boosting can enhance predictive accuracy by sequentially improving weak learners, whereas bagging primarily alleviates overfitting by reducing variance. [37]. Therefore, ET was chosen as a bagging method, while GB, XGBoost, and LightGBM were selected as boosting methods.

The complete hyperparameter grids and the best-performing configurations of all classification models, are provided in Supplementary S6 Table.

To ensure the robustness and reproducibility of the classification results, the entire model training and evaluation process was repeated ten times under different random seeds. The reported results represent the mean and 95% confidence intervals computed across these ten independent runs.

Variables

After data cleaning, 31 features remained, as shown in Table 1 (continuous variables) and Table 2 (binary variables). The Shapiro-Wilk test indicated that none of the continuous variables followed a normal distribution; Consequently, the Kruskal-Wallis test was used to assess the significance of the relationship between MAFLD and each continuous variable. For binary variables, the chi-square test was employed to evaluate their significance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Details and characteristics of continuous variables for patients with MAFLD and patients without MAFLD.

https://doi.org/10.1371/journal.pone.0339580.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Details and characteristics of binary variables for patients with MAFLD and patients without MAFLD.

https://doi.org/10.1371/journal.pone.0339580.t002

Evaluation of imputation methods

After simulating the missingness in a complete subset of the dataset, the performance of imputation methods was evaluated. This process was repeated five times, and the mean results across these runs are presented in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Performance comparison of different imputation methods for binary and continues variables (mean results over five runs).

https://doi.org/10.1371/journal.pone.0339580.g004

For continuous features, AdaBoost achieved the highest performance. For binary features, all imputers yielded comparable results, with Ridge having the highest accuracy along with Adaboost (with 0.001 difference). Therefore, AdaBoost was selected as the final imputation method, and all subsequent analyses were performed on the dataset imputed using this approach.

Feature selection and classification

The accuracy of different feature selection strategies compared with baseline models using all features is summarized in Table 3. Results are reported as mean accuracy ± standard deviation (SD) and 95% confidence intervals (CI) across five cross-validation folds. The bolded values indicate the highest mean accuracy for each classifier.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Accuracy and number of selected features for each classifier with various feature selection methods.

https://doi.org/10.1371/journal.pone.0339580.t003

Among them, XGBoost without Overall, the ensemble models demonstrated very similar performance, with only marginal differences across algorithms and feature selection approaches. feature selection achieved the highest overall accuracy (0.806 ± 0.014) and was therefore identified as the best-performing model. The Gradient Boosting and LightGBM models followed closely, showing only slight differences in performance. For these two models, the versions incorporating feature selection methods, K Best for Gradient Boosting and Genetic Algorithm for LightGBM, yielded nearly identical results to their corresponding models without feature selection. Given these minimal gaps, the feature-selected versions were considered representative of their best performance configurations

Using the preferred feature selection method for each classifier, the mean of outcomes for the ensemble methods, which outperformed the basic models, are illustrated in Fig 5. The figure presents a comparison of various metrics, including Accuracy, Recall, Precision, F1-score, and the AUC.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Comparison of performance metrics for ensemble methods.

https://doi.org/10.1371/journal.pone.0339580.g005

Feature importance

Fig 6 illustrates the top ten features with the highest mean importance scores across the three best-performing models, XGBoost, LightGBM, and Gradient Boosting. The bar plot shows, for each of these top features, their relative importance within each model. The full values are presented in Supplementary S7 Table.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Top ten features with the highest mean importance across XGBoost, LightGBM, and Gradient Boosting models.

https://doi.org/10.1371/journal.pone.0339580.g006

To further analyze the data, a SHapley Additive exPlanations (SHAP) plot using the XG Boost model is presented in Fig 7. SHAP plots, derived from SHAP values, illustrate the contribution of each feature to the prediction (e.g., class membership probability) and how these contributions deviate from the mean prediction across all data points [38].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. SHapley Additive exPlanations value plot of different features in XGBoost classifier.

https://doi.org/10.1371/journal.pone.0339580.g007