Datasets

We utilize two real-world medical datasets to ensure the validity of our findings. The use of these real-world medical datasets ensures that our data is representative of regular clinical practices and incorporates clinical data from laboratory tests on suspected COVID-19 patients. This allows us to capture the relationships between the laboratory test outcomes, COVID-19 diagnosis, and patient deterioration.

Einstein Data4u: As our first dataset, we used an open-source dataset made publicly available by the Hospital Israelita Albert Einstein in São Paulo, Brazil, through Kaggle [16]. The dataset was anonymized by the publisher in accordance with international privacy guidelines, ensuring that no personally identifiable information is accessible. Therefore, additional ethical approval was not required for its use in this research. The data were accessed and downloaded in April 2022 for the development of a COVID-19 screening and triage pipeline. This dataset was utilized to evaluate the influence of missing data imputation methods. In total, 1591 patient records were used in this study, which consist of anonymized data from patients tested for COVID-19 using RT-PCR (positive or negative), admission decisions for positive cases (regular ward, semi-intensive (high-dependency) or intensive care unit), and 106 laboratory tests collected during their visit to the emergency room. The data are characterized by a significant skew in outputs (approximately 9% of the samples are positive for COVID-19 tests) and a large number of missing laboratory test values, with 77.25% of the samples missing at least one laboratory test result. The dataset comprised 5644 records. A significant proportion (71.81%; n = 4053) lacked values for the 20 specific laboratory tests selected for this analysis. These records were excluded from our study to ensure a consistent feature matrix for the evaluation of imputation strategies.

MIMIC-IV v3.1 [17]: Medical Information Mart for Intensive Care (MIMIC)-IV is a large and well-established open dataset of patients in emergency or intensive care. This dataset contains de-identified data of over 265,000 patients between 2008−2022. For privacy purposes, the admission time of the patients has been shifted. The patient data include 381 lab tests, ICU stay information, and diagnoses, but there is no COVID-19 test in the features. Therefore, we only considered the data of patients who had been diagnosed with COVID-19. The dataset contains 3,620 patients diagnosed with COVID-19. Some patients (665) were diagnosed with COVID-19 more than once; however, we only considered the first record for each patient in this study to reduce potential correlation between the records. We also removed patient records that had no values for any of the laboratory tests considered in the analysis. After excluding these patient records, the final MIMIC-IV dataset contained 3,337 samples.

Data analysis

The AI/ML pipeline for COVID-19 screening, triage, and the comparative analysis of imputation methods is illustrated in Fig 1. The pipeline consists of four stages: pre-processing, pruning, imputation, and modeling, allowing a systematic assessment of predictive integrity across different imputation strategies. In the pre-processing step, input data, including COVID-19 tests, laboratory tests, and patient deterioration data, are cleaned to remove empty records and to transform the features into numerical values. In the feature pruning step, the dimensionality of the data is reduced by selecting representative features and removing dependent ones. This process was tailored to the specific characteristics of each dataset (see Section Feature Analysis). The imputation step fills in the missing data by applying one of the techniques considered in the study: no imputation (baseline), single imputation, multiple imputation (KNN, MICE), Matrix factorization–based imputation (SoftImpute), or Random forest–based imputation (MissForest). Finally, in the modeling step, different classifiers are trained and evaluated to predict COVID-19 diagnosis and patient deterioration. The classifiers include logistic regression (LR), random forest (RF), support vector machine (SVM), k-nearest neighbors (KNN), XGBoost, and LightGBM. These steps are described in detail in the following sections.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Missing data imputation pipeline through feature pruning and missing value imputation.

https://doi.org/10.1371/journal.pone.0320105.g001

Feature analysis

Medical practices and procedures govern which medical tests and procedures a specific patient undergoes. As these vary among patients, there are certain commonalities but also variations across patients. Accounting for these variations in procedures can be used to alleviate data sparsity. Specifically, the availability of a particular test variable depends on three interlinked factors: (i) demand for specific test procedures, i.e., whether a patient is required to undergo a specific laboratory test; (ii) relatedness, i.e., several tests monitor factors that are closely correlated with another test and it may suffice to use one of the tests; and (iii) redundancy, i.e., tests may measure the same aspect and be candidates for replacement. We take advantage of these three criteria to reduce the feature space and minimize biases and inaccuracies in the analysis.

Feature selection. In this step, we first removed the features with a large number of missing values. These features impose higher complexity for imputation and increase the sparsity of the modeling. Most of these removed features are missing completely at random (MCAR) or the underlying reason for missing values is difficult to deduce (i.e., MNAR) and have over 90% of values missing without any distinguishable patterns. In total, 55 continuous and 18 discrete features were removed from the Data4u dataset. For the MIMIC-IV dataset, 281 continuous features were excluded, ensuring that only the most significant and independent laboratory markers were retained for analysis.

For the Data4u dataset, the 33 retained features are categorized as follows: (i) 14 hematological parameters (complete blood count), which have demonstrated efficacy in detecting COVID-19 during the second week of symptom onset [18], alongside rapid antigen tests for influenza A and B; (ii) 17 variables representing various viral pathogen families; and (iii) the COVID-19 test result and subsequent hospital admission decision. Pathogens belonging to identical families were combined into unified features to reduce the dimensionality of the input space. Based on the assumption that pathogen absence reflects a non-infected state, we utilized binary encoding to represent each family. A value of 1 was assigned if at least one test within the family yielded a positive result, and 0 if all constituent tests were negative. This domain-driven aggregation produced the following six features:

1. Adenoviridae: Adenovirus
2. Coronaviridae: Coronavirusnl63, Coronavirus hku1, Coronavirus229e and Coronavirusoc43
3. Orthomyxoviridae: Influenza a, Influenza b and Inf a h1n1 2009
4. Paramyxoviridae: Parainfluenza 1, Parainfluenza 2, Parainfluenza 3 and Parainfluenza 4
5. Picornaviridae: Rhinovirus/Enterovirus
6. Pneumoviridae: Chlamydophila pneumoniae, Metapneumovirus and Respiratory syncytial virus

Two additional rapid test features were removed from the feature space as they are already included in the Orthomyxoviridae virus family. Finally, we removed hematocrit from the feature space as it has a high correlation with hemoglobin [19]. Since both parameters reflect red blood mass, using only one of these features suffices to avoid multicollinearity and clinical redundancy. We retained hemoglobin as it demonstrated higher data completeness within the cohort. The selected features for Data4u are shown in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Einstein Data4u Dataset construction.

https://doi.org/10.1371/journal.pone.0320105.t001

For the MIMIC-IV dataset, we retained a feature space consistent with the Data4u dataset, with the exception of viral pathogen family indicators, which were unavailable in this cohort. We augmented the feature set with additional clinical variables previously validated for coronavirus mortality prediction within the MIMIC data [20]. Table 2 shows the selected features of the MIMIC-IV dataset.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. MIMIC-IV extracted dataset construction.

https://doi.org/10.1371/journal.pone.0320105.t002

Labels. For the Data4u dataset, we used two labels for prediction. The first label is COVID-19 diagnosis, which is encoded as a binary variable where 1 indicates having COVID-19 disease and 0 indicates not having the disease. The second label is potential patient deterioration, derived from the admission to semi-intensive care units and admission to intensive care units features. The label value is 0 if both admission to semi-intensive care units and admission to intensive care units are 0; otherwise, it is 1. Note that this implies that we do not model the extent of deterioration, only whether the patient’s condition deteriorates or not. This means that patients admitted to the regular ward (but not semi-intensive (high-dependency) or intensive care) are not considered to have a deteriorated condition. This was done to improve the potential of AI/ML algorithms to learn robust dependencies from the data rather than fit on noise. Indeed, the number of patients admitted to semi-intensive or intensive care in the data is relatively small for modeling purposes.

For the MIMIC-IV dataset, we specifically targeted patient deterioration. This label was derived from ICU admission data: assigned a value of 1 if the patient was admitted to the ICU during the same hospital stay as the COVID-19 diagnosis, and 0 otherwise. Similar to the Data4u dataset, we considered admission to the ICU as the primary indicator of deterioration, regardless of the duration of stay.

Missing data handling

The optimal way to handle missing data depends on the mechanism that results in the missing values, with the two main approaches being excluding the records with missing entries and replacing the missing entries using imputation [21]. The decision on whether to impute or not depends on the rate of missing values and the underlying mechanism that causes the missing entries. Generally, missing data can be a result of one of three processes: missing completely at random (MCAR), missing not at random (MNAR), or missing at random (MAR) [22]. MCAR is a result of random errors, such as equipment failures, whereas MNAR occurs when the mechanism causing the missing entries has no discernible patterns. MAR is the most common type of missing data and is defined as missing data that is dependent on observed values. Most of the missing data in our case results from one or more laboratory tests being omitted at the hospital depending on the medical analysis, and thus most of the missing entries in the data considered in this paper are MAR in nature. MCAR data can be deleted without risking adding bias to the results whereas imputation is recommended for missing values of MAR or MNAR types [23].

We separately assess how the profile of missing features affects the performance of imputation techniques and the performance of the AI/ML techniques used for predicting COVID-19 status and outcomes. The profile of missing features is characterized by the composition of the features that contain missing values (categorical, numerical, or mixed), the rate of missing values, and the pattern of the missing features. The pattern refers to the likelihood of data being missed together (togetherness) or independently (randomness). This pattern is motivated by the observation that certain combinations of laboratory tests either miss consistently or are present together. For example, features related to blood characteristics require a sample of blood from the patient and if this sample is not taken then all related features are necessarily missing. We verified the existence of this pattern by performing a pairwise correlation analysis of the presence and absence of features. All feature pairs with perfect correlation were merged into the togetherness pattern whereas others were encoded as random. Fig 2 illustrates the missingness patterns of features and their correlation values for the MIMIC-IV dataset. The subset of features that have very high correlation (more than 90) are considered in one together subset. The patterns are only used for comparing the imputation performance of imputation methods.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Heatmap of features co-occurrence of missingness (MIMIC-IV dataset).

https://doi.org/10.1371/journal.pone.0320105.g002

We assess six approaches for handling missing values in this study: (i) no imputation (baseline), where records with missing entries are excluded; (ii) single imputation using a random draw from the observed values; (iii) multiple imputation using KNN, which estimates missing entries from the k most similar samples; (iv) multiple imputation using MICE (multiple imputation by chained equations), which iteratively models each variable conditional on the others to generate multiple completed datasets and combines estimates; (v) SoftImpute, a low-rank matrix completion method that leverages global structure via nuclear-norm regularization; and (vi) MissForest, an iterative random-forest imputation method that accommodates mixed data types and nonlinear relationships. KNN and MICE are commonly used methods for imputing data [24] and they have been shown to perform well consistently, particularly for larger datasets [25]. The diversity of these methods allows us to compare various imputation methods and assess accuracy in imputing clinical data and evaluate their impact on ML predictions and clinical implications.

The imputation techniques are evaluated using simulated datasets that are realistic but that reflect different profiles of missing data. As a baseline, we consider the complete data, and hence the evaluation is limited to the 362 samples in Data4u dataset and 317 samples in MIMIC-IV COVID dataset that contain no missing values. Our focus on complete records is driven by the need to establish a controlled simulation environment to systematically investigate the impact of various rates of missingness on the performance of ML models. This approach allows us to synthetically inject missingness into the dataset and evaluate the models under different missingness patterns. We create 86 copies of these data by considering different levels of missing rate (5% − 90%, 1% increment) of three feature types (categorical, numerical, categorical+numerical) following the pipeline depicted in Fig 3. Missing rates of 5% and 90% are selected as a lower bound and an upper bound of the missing rate to examine ML models performance under low and extreme missingness. In practice, low missingness rate (e.g., ≈5%) is often considered manageable, whereas very high missingness (e.g., ≈90%) is typically excluded as they cannot be meaningfully imputed due to lacking sufficient observed information to estimate missing values and can severely limit the reliability of imputation and downstream analyses [23,26]. Data is randomly dropped following either the togetherness pattern or the randomness pattern. In total, 516 datasets (3 feature types x 86 missing rates x 2 missingness patterns) were created in total, each represents a unique missingness profile. These datasets are referred to as simulated datasets. The five imputation techniques (single, KNN, MICE, SoftImpute, and MissForest) were run on each simulated dataset 500 times and the mean absolute error (MAE) of the values was computed using Equation 1 and compared to the baseline, i.e., the complete data.

(1)

where N is the total samples, represents actual values and denotes the values predicted by imputation. MAE is a widely used, interpretable metric that measures the average absolute error between imputed and true values in the complete dataset to evaluate value-level imputation accuracy. Moreover, MAE is more robust to large outliers compared to other metrics such as RMSE, thus provides a conservative assessment of average errors across repeated imputations [27].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Pipeline generating datasets with different missingness profiles.

https://doi.org/10.1371/journal.pone.0320105.g003

Infection and patient deterioration prediction

Classifiers were trained and tested for detecting COVID-19 infections and predicting patient deterioration from laboratory tests. As described in the previous section, for the Data4u dataset, the targets were encoded as positive (1) or negative (0) depending on the value of the RT-PCR test (Task 1) or whether the patient was admitted to semi-intensive ward or intensive care (Task 2). Both tasks were carried out on two datasets, one where all entries with missing values were excluded (i.e., complete) and one where entries with missing values were included; see Table 1. The latter was imputed using MICE and SoftImpute for Data4u and MIMIC-IV, respectively as they yielded the lowest error in clinical settings for corresponding datasets; see Section Imputation performance.

For data from the MIMIC-IV dataset, we only have Task 2 and the target was encoded as patient admitted to intensive care (1) or not (0).

Six different classification algorithms were considered: logistic regression (LR), random forest (RF), support vector machine (SVM), k-nearest neighbors (KNN), XGBoost, and LightGBM models. These six learning models were selected for their performance on tabular data [28] and offering a degree of interpretability with feature ascription methods [29]. Deep learning was omitted due to the number of entries being insufficient for reliably validating overfitting.

Evaluation metrics. The performance of the learning models is compared using accuracy, balanced accuracy, sensitivity, specificity, and the area under the curve (AUC). Sensitivity and specificity represent the percentage of individuals being correctly identified as positive or negative for the condition of interest, i.e., COVID-19 infection or patient deterioration, respectively. Sensitivity determines whether the test is satisfactory compared to a reference standard whereas specificity is important in establishing a patient tested positive on the screening test do or do not actually possess the condition [30]. Since the proposed imputation and AI/ML pipeline emphasizes both aspects, we calculated the average of sensitivity and specificity scores, i.e., balanced accuracy [13], and included it as a performance metric. Lastly, AUC depicts the degree of separability between classified classes, i.e., patients with and without COVID-19, or patients being admitted to semi-/intensive care units or not.

The training and test data were obtained using a stratified 5-fold cross-validation technique to balance the bias and variance of the small dataset during training. To address the imbalanced class problem, the synthetic minority oversampling technique (SMOTE) was used to increase the minority class samples in the training dataset so that the model can generalize well on new unseen data. After applying SMOTE method, the dataset become more balanced to differentiate between the target classes.

To understand the impact of features on the predictive performance of the different models, we used Shapley Additive Explanations (SHAP) to estimate the average marginal contribution of each feature and used this to assess the importance of different features. The Shapley value originates from coalitional game theory and is a popular method for estimating the overall contribution of a feature for a given prediction [31]. The higher the absolute Shapley value of a feature, the more significant the contribution of a feature is on the predictions given by a model. The contributions of features can be visualized using a summary plot where the x-axis corresponds to the Shapley values and the y-axis to the different features.

Implementation

All algorithms and experiments were implemented in Python (version 3.8). NumPy [32] was used for numerical data processing, Pandas [33] for data processing, the scikit-learn library [34] for data processing, imputation and the classifier implementation, MissForest for MissForest imputation, fancyimpute for SoftImpute imputation, and Matplotlib [35] for data visualization. We used the default hyperparameters of the scikit-learn library for our ML methods, as our goal was to investigate the effect of various imputation methods on the ML model performance. S1 Table in the Supporting information shows the hyperparameters of the ML methods used in this study.

Imputation performance

In this section, we compare the performance of various imputation methods for different rates of missing for both of our datasets.

Einstein Data4u. Fig 4 depicts the Mean Absolute Errors (MAE) of data imputation using simple imputation, multiple imputation KNN, multiple imputation MICE, SoftImpute, and MissForest at 86 levels of missing rates (from 5% to 90% at 1% increments) of different feature types and patterns of missing data, following the simulated data generation pipeline detailed in Section Missing data handling. 95% Confidence Intervals (CIs) for these MAE values were computed and are reported as part of the figures.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Mean absolute errors of data imputation in relation with missing rates (5%–90%) and missingness patterns (randomness: a-c, togetherness: d-f) (The average CI for all the plots were smaller than [−0.01, 0.01] and they are not shown in this figure.).

https://doi.org/10.1371/journal.pone.0320105.g004

As seen from the figure, imputation using MICE yields the lowest errors compared to KNN and simple imputation techniques for typical clinical scenarios (missingness < 25%). However, the performance landscape shifts at higher sparsity levels. While MICE is dominant in low-missingness numerical settings, SoftImpute closely follows MICE for these rates and eventually outperforms all other methods as missingness increases beyond 30%.

Simple imputation, SoftImpute, and MissForest are best suited for missing categorical data or data having a high percentage of missing numerical data that are correlated. KNN imputation works better than simple imputation when the missing numerical data is uncorrelated, i.e., the values are randomly missing across multiple rows. When the missing data pattern is random, multiple imputation using MICE yields the lowest error for numerical data. For higher sparsity, and more complex settings, such as categorical or mixed features, MICE remains competitive but techniques like SoftImpute and MissForest often yield the strongest performance.

MIMIC-IV. Fig 5 demonstrates the performance of imputation methods on the MIMIC-IV dataset. As shown in the figure, for this data, SoftImpute consistently outperforms other methods, followed by MissForest. SoftImpute is a matrix-completion method that leverages the global covariance structure of the data via nuclear-norm regularization, unlike KNN which relies on local neighborhoods. This characteristic allows SoftImpute to effectively capture inherent relationships between features even at high rates of missingness. MissForest, being based on random forests, captures non-linear complex relationships between features but is computationally more intensive. Simple imputation ignores feature correlations and thus yields higher error. As missingness increases, the performance of KNN and MICE degrades more rapidly than SoftImpute, suggesting they are less robust to extreme sparsity in this specific dataset and that the best performing methods can depend on the data characteristics, as well as specifics of the processing performed on the data.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Mean absolute errors of data imputation in relation with missing rates (5%–90%) and missingness patterns for MIMIC-IV dataset (randomness: a, togetherness: b) (The average CI for all the plots were smaller than [−0.01, 0.01] and they are not shown in this figure.).

https://doi.org/10.1371/journal.pone.0320105.g005

General Findings. When analyzed across both datasets, most imputation techniques target data that is missing at random (MAR), and both single and multiple imputation also fare best in these cases with our data. KNN imputation seems best suited for data that is missing completely at random (MCAR). Overall, the results indicate that imputation is important for increasing data coverage, but that it is also important to assess the patterns of missing data and achieving the best possible result may require combining different imputation strategies.

Statistical Validation. To verify the statistical significance of the main findings, we conducted pairwise statistical comparisons using Welch’s two-sample t-test on the reconstruction errors (MAE) across the 500 simulation folds. For the Einstein Data4u cohort, advanced multivariate methods (MICE and SoftImpute) consistently yielded significantly lower errors (p < 0.05) than baseline methods across low and moderate missingness rates. As expected, at extreme levels of data sparsity (typically exceeding 50% to 60%), the performance gap began to narrow, with differences occasionally becoming statistically non-significant (). Under the “together” missingness pattern, this convergence usually occurred at lower sparsity thresholds than under the random missingness pattern. This convergence reflects the theoretical limits of imputation. When the majority of data or larger blocks of related clinical features are missing, underlying joint distributions cannot be reliably modeled, causing advanced techniques to perform similarly to simpler baselines. However, within clinically realistic sparsity ranges, the statistical advantage of multivariate methods remained robust. For the MIMIC-IV dataset, predictive differences between multivariate and univariate imputation consistently demonstrated statistical significance (p < 0.05) across all evaluated missingness rates, with a single exception (KNN vs. MissForest at 22% missingness, p = 0.166).

Prediction of COVID-19 infections and patient deterioration

The predictive performance for COVID-19 infections and patient deterioration on the Einstein Data4u dataset is summarized in Table 3. The performance was assessed by comparing predictions from a complete dataset (where records lacking any laboratory test were removed) to those from a dataset imputed using MICE. As shown in Table 3, for the Data4u dataset, all models demonstrate high accuracy and balanced accuracy, reaching up to 97% and 90% for COVID-19 diagnosis, and up to 96% and 82% for predicting patient deterioration. The consistent improvement of these performance metrics on the imputed data demonstrates the effectiveness of the proposed pipeline.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Results for predicting COVID-19 infection and for predicting patient deterioration (Data4u Dataset). The best value for each score is denoted in bold.

https://doi.org/10.1371/journal.pone.0320105.t003

Infection detection. Overall, for the incomplete dataset, the probability of correctly predicting non-infected COVID-19 cases (specificity) is between 96% and 99% across models, and the probability of correctly predicting COVID-19 infections (sensitivity) reaches up to 81%. While specificity remains comparable across datasets, the sensitivity is significantly lower on the complete dataset. All models show strong predictive performance on the incomplete dataset. Minimum values for accuracy, balanced accuracy, and AUC are 0.92, 0.76, and 0.89, respectively. On the complete dataset, these baselines drop to 0.88 (accuracy), 0.66 (balanced accuracy), and 0.86 (AUC).

In terms of algorithms, RF and KNN achieve the best performance, followed closely by SVM and LR. XGBoost and LightGBM also perform well, though their AUCs are comparatively lower than RF or KNN for this specific task. More importantly, the results show a high improvement in correctly detecting COVID-19 infection cases (sensitivity) when using the imputed dataset. RF shows a 22% increase in sensitivity compared to the complete case, while SVM shows a 26% increase. These results suggest that patients testing positive with our pipeline are highly likely to be infected, allowing for the prioritization of RT-PCR testing and providing a simple method to identify early infections prior to symptom onset.

Patient deterioration prediction. As shown in Table 3, SVM is the best performing model for predicting patient deterioration, followed by LR, RF, and KNN. LightGBM and XGBoost also achieve high specificity and AUC. However, their balanced accuracies are lower than the other models. The sensitivity score increases by 4% with SVM (sensitivity = 0.65), by 5% with LightGBM and XGBoost (sensitivity = 0.68), and by 9% with LR (sensitivity = 0.62) when prediction is performed on the imputed dataset. Failing to detect the need for semi-/intensive care units for 9% of patients can rapidly burden the healthcare system and, in the worst case, lead to loss of life. While the sensitivity of the KNN and RF models remains practically unchanged, their AUC scores increase when data is imputed, indicating a better ability to distinguish patients likely to deteriorate. The specificity of all models is very high (minimum 0.94) regardless of whether the data is imputed or not. In other words, any positive prediction for deterioration should be treated as a high-priority admission to semi-/intensive care, as the likelihood of false positives is minimal. This highlights the potential of using our approach for triaging COVID-19 patients based on simpler laboratory tests.

Patient deterioration prediction on MIMIC-IV dataset. On the MIMIC-IV COVID cohort, the trend reverses compared to the Data4u dataset. Models trained and evaluated on the complete dataset consistently outperform models trained on incomplete datasets with imputed data (Table 4). RF achieves the highest performance on complete cases, with AUC, balanced accuracy, and sensitivity values of 0.99, 0.96, and 0.98, respectively. The performance improvements, however, should be interpreted with caution as they suggest that the missingness mechanism in MIMIC-IV is Informative (Missing Not At Random – MNAR) or that the complete dataset represents a biased sub-population. Indeed, as shown in Table 4, the complete dataset contains only 36 records of the non-deteriorate class, whereas in the incomplete dataset, the non-deteriorate class is twice the size of the deteriorate class. This implies that complete case analysis filters for a specific subset of patients (likely those heavily monitored due to severity), introducing selection bias. While imputation yields lower raw metrics than this (biased) baseline, it allows analyzing broader and more representative patient cohort. These results emphasize that while imputation is beneficial for increasing coverage, its impact on predictive metrics depends heavily on the underlying missingness mechanism (MAR vs. MNAR) and population characteristics.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Results for predicting patient deterioration (MIMIC-IV COVID Dataset). The best value for each score is denoted in bold.

https://doi.org/10.1371/journal.pone.0320105.t004

Feature importance

We analyzed feature importance using Shapley Additive Explanations (SHAP) to understand how imputation affects the clinical factors driving model predictions.

Data4u Dataset. The feature importance for predicting COVID-19 diagnosis using the best-performing model (RF) is shown for the complete and imputed datasets in Fig 6a and 6b, respectively. Similarly, Fig 6c and 6d display the feature importance for predicting patient deterioration using the best-performing model (SVM). In these SHAP summary plots, each horizontal bar corresponds to a specific feature. The length of the bar indicates the magnitude of the SHAP value (impact on prediction), while the color represents the feature value (red for high, blue for low).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Impact of the features on predicting COVID-19 diagnosis (a, b) and patient deterioration (c, d).

https://doi.org/10.1371/journal.pone.0320105.g006

For COVID-19 diagnosis (Figs 6a, b), the feature ranking is consistent across both complete and imputed datasets. Blood variables—specifically leukocytes, platelets, monocytes, and eosinophils—consistently emerge as the most critical indicators for detecting the presence of the virus. For patient deterioration (Figs 6c, d), imputation brings subtle but important shifts. While key inflammatory markers remain important, the imputed model (SVM) places greater weight on a broader set of viral family coinfections compared to the complete-case model, suggesting that imputation recovers signals from sparser viral panel data.

MIMIC-IV Dataset. Fig 7 illustrates the shift in feature importance for patient deterioration in the MIMIC-IV cohort. The model trained on the complete dataset (Fig 7a) relies heavily on pCO2 (partial pressure of carbon dioxide) and specific blood inflammation parameters. While highly predictive for the subset of patients with these tests, this view is limited. In contrast, the model trained on the imputed dataset (Fig 7b) reveals a more comprehensive clinical picture. Indicators of lung function (pH, pO2, pCO2) alongside broader markers of inflammation and immunity emerge as the top predictors. This demonstrates that even when imputation does not drastically improve predictive metrics (as seen in the previous section), it fundamentally alters the clinical narrative. By enabling the inclusion of patients with partial data, imputation reduces selection bias and unmasks risk factors related to respiratory failure that are obscured in the smaller, complete-case cohort.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Impact of the features on predicting patient deterioration for MIMIC-IV dataset.

https://doi.org/10.1371/journal.pone.0320105.g007

Imputation and model performance

Handling missing data in clinical datasets is a significant challenge, as excluding records with missing values can introduce biases and lead to models with poor generalizability. Our study systematically investigates the effects of various imputation methods on AI/ML model performance, demonstrating that the choice of imputation strategy significantly influences the performance of ML models for detecting COVID-19 infection cases or patient deterioration. Specifically, we found that a lower number of infection cases are detected when missing values are excluded, which could result in many patients requiring admission to semi-intensive (high-dependency) or intensive care units being overlooked.

Our results on the Einstein Data4u dataset indicate that the accuracy, balanced accuracy, sensitivity, specificity, and AUC of different models for predicting both COVID-19 infections and patient deterioration are distinctly higher when using imputed data compared to a complete dataset. Conversely, for the MIMIC-IV dataset, the absolute performance metrics are numerically higher with the complete dataset, which is consistent with selection bias. The complete subset in MIMIC-IV represents a distinct, heavily monitored patient population (where tests are ordered due to clinical severity), whereas the imputed dataset captures the broader, more representative cohort. This demonstrates that complete-case analysis may yield higher apparent accuracy but is limited to a biased sub-population; in contrast, imputation enables decision-making for the general patient population, even if the inclusion of noisier data results in more conservative raw metrics.

In our experiments, multivariate imputation methods (MICE and SoftImpute) resulted both in the lowest reconstruction error (MAE) and the highest sensitivity. A lower MAE indicates that the imputed values closely mirror the true underlying clinical data. By minimizing the absolute error during reconstruction, multivariate methods like MICE and SoftImpute better preserve the natural distributions, variance, and multivariate correlations present in laboratory tests. Consequently, the link between lower MAE and predictive utility stems from highly accurate data reconstruction, which is essential for ensemble tree-based classifiers (Random Forest, XGBoost) that rely on the continuous data spread to identify optimal decision boundaries as destroying this variance prevents effective class separation. The same dependency extends to models using hyperplanes or distance metrics (SVM, KNN), where artificial mean-clustering skews the feature space topology.

Clinical Implications

The clinical implications of our findings are central to the operational utility of healthcare AI. For providers, our study provides a framework for selecting imputation methods that maximize diagnostic sensitivity—the primary requirement for early screening. Policymakers can leverage these results to inform public health strategies, enabling timely intervention in resource-limited settings where RT-PCR availability is constrained.

The AI/ML models for detecting infections leverage established laboratory procedures instead of requiring specialized tests, facilitating timely interventions. Early population screening using laboratory tests enables medical authorities to understand disease prevalence and severity, allowing for effective prioritization of public health interventions. Additionally, demand prediction helps identify the needs of different hospital wards, alerting authorities to potential shortages of semi-intensive or intensive care unit beds in advance.

Feature importance and its variability

The analysis of feature importance indicated that certain features, such as blood variables (leukocytes, platelets, monocytes, eosinophils), consistently play a crucial role in detecting COVID-19 infections. However, the importance of some features varies significantly depending on whether the data are imputed. For example, high levels of Pneumoviridae or low levels of Bordetella pertussis indicate a likely infection when the data are imputed, but these features do not contribute to prediction accuracy when trained on the complete dataset.

Similarly, in predicting patient deterioration, the significance of features changes after imputation. For the Einstein Data4u dataset, high values of leukocytes and Picornaviridae indicate a higher likelihood of deterioration, which is less apparent in the complete dataset. In the MIMIC-IV dataset, markers related to respiratory function (e.g., pH and pO2) showed higher importance in the imputed dataset compared to the complete dataset, suggesting that restricting analysis to complete cases may obscure critical respiratory signals present in the broader population.

Comparison with previous work

While previous studies have established the feasibility of COVID-19 screening using laboratory data, they often present a static view of clinical predictors based on complete-case analysis or simple imputation. For example, a logistic regression model using complete blood count data and sex was proposed by Joshi et al. to predict SARS-CoV-2 PCR positivity and improve testing resource allocation [14]. In turn, Brinati et al. showed that machine learning models based on routine blood tests can achieve performance comparable to rRT-PCR while enabling interpretable and accessible COVID-19 screening tools [13]. Our work extends these findings by demonstrating that clinical insights are tied to the imputation strategy employed. Specifically, while foundational models commonly highlight leukocytes and platelets as top predictors – a finding we also replicate – our systematic approach reveals that the inclusion of imputed viral panel data unmasks additional predictors (e.g., specific pathogen coinfections) that are entirely obscured in complete-case cohorts.

In addition, our comparison between the Data4u and MIMIC-IV datasets provides insights into performance differences between complete-case and imputed analyses. Although imputation is often recommended to enhance performance, generalizability, and reduce bias, these benefits depend on the plausibility of the assumed missing data mechanism (e.g., MAR) and correct model specification. Our results on the MIMIC-IV cohort indicate that, in intensive care settings, higher apparent predictive metrics in complete cases may reflect selection bias. By shifting emphasis from pure accuracy to the validity and transportability of the clinical narrative, we offer a more realistic framework for deploying AI/ML in clinical environments, where data are seldom missing at random.

Limitations and future work

While our study provides valuable insights, it is not without limitations. First, the presented pipeline aims to provide valid analytic estimation in the presence of missing data while preserving relationships among features, rather than estimating the precise value for each variable. Second, we focused on classical and ensemble machine learning models (RF, SVM, XGBoost) and did not include deep learning architectures. This was a deliberate choice driven by the limited sample size of the primary cohort, where deep models are prone to overfitting.

Third, our patient deterioration target combines admissions to semi-intensive and intensive care units. While necessary to achieve sufficient sample sizes for robust machine learning, we acknowledge that this binarization oversimplifies clinically heterogeneous outcomes. Patients may require intensive care for a numerous of physiological reasons, e.g., isolated respiratory failure versus systemic septic shock, and the use of a binary target loses information on time-to-event dynamics. At the same time, this abstraction does not invalidate our methodological conclusions. The demonstrated capability of multivariate imputation to reconstruct underlying data distributions and recover latent prognostic signals from sparse laboratory tests remains robust, regardless of the target’s clinical heterogeneity. Nevertheless, our pipeline should be viewed primarily as an early-warning triage tool rather than a granular physiological tracker.

Fourth, our simulation study relies on MAR/MCAR assumptions. While the comparison with MIMIC-IV highlights the impact of informative missingness (MNAR), synthetic metrics cannot fully capture the complexity of real-world clinical bias where test ordering is tied to unobserved health states. Finally, the results are based on specific geographic regions and clinical protocols. Future work should validate this approach on larger, multi-center datasets to further assess generalizability across diverse patient populations.

Ethical considerations

Imputation in biomedical research introduces ethical concerns about algorithmic fairness and transparency. If imputation models are trained on datasets that underrepresent specific demographics, they may systematically introduce biases that lead to inequitable clinical decisions for those groups. Similarly, automated data filling can create a false sense of certainty in clinical high-stakes scenarios. To mitigate these risks, our pipeline utilizes Explainable AI (SHAP) methods to ensure that imputed predictors are physiologically plausible and transparent to the clinician. We argue that imputation should be viewed as a tool to improve population-level representation rather than a replacement for primary data collection, and its deployment must be accompanied by rigorous bias auditing.

Guidelines

The results demonstrated that both missing data and the selection of imputation techniques for handling it affect the performance of AI/ML models trained on patient records. In the following, we provide a summary of the key guidelines on how to best handle missing data to improve the quality of the data and AI/ML model performance.

Step 1. Pruning the features.

• Feature-demand filtering: The first step is to understand the nature and reason for missing data and to classify features with missing values as Missing Completely at Random (MCAR), Missing Not at Random (MNAR), or Missing at Random (MAR). The features classified as MCAR can be omitted without the risk of adding bias [25]. For features classified as MAR, those having missing rates between 5% and 90% should be retained. However, caution is needed for MNAR features (e.g., tests not ordered for healthy patients); excluding them may remove valuable clinical signals, as seen in our MIMIC-IV analysis.
• Feature-relatedness filtering: Dependencies between features can degrade imputation performance and hence it is important to identify features (i.e., test procedures) that are closely related and to merge them. This can be done using hierarchical clustering to produce a clinically meaningful division of the data. Crucially, this step must be performed either using established domain knowledge (e.g., viral taxonomy) or nested within the cross-validation loop to prevent data leakage from the test set.
• Feature-redundancy filtering: Features that are highly correlated among one another contain significant redundancy and can again degrade imputation performance. As a result, only one of the redundant features should be selected. For example, in the case of blood tests, the information of the red blood cells can be obtained from hematocrit and hemoglobin test results, so selecting the one with the better feature profile suffices.

Step 2. Identifying the missingness profile. Features should be associated with a missingness profile that comprises the composition of the features that contain missing values, the rate of missing values, and the pattern of the missing features. In the context of personal medical records, the missingness profile can be determined based on the medical practices and procedures that govern which medical tests and procedures a specific patient undergoes.

Step 3. Identifying the best imputation technique(s). Imputation generally improves performance, but the imputation technique should be chosen based on the missingness profile identified in Step 2.

• Low to Moderate Missingness (< 25%): For datasets with typical clinical missingness rates, MICE yields the lowest error and is the recommended default for tabular medical data.
• High Missingness (> 30%): For datasets with high sparsity, matrix-completion methods like SoftImpute are more robust than iterative regression methods. They leverage global covariance structures to recover signals where local neighbors (KNN) or regression targets (MICE) become unreliable.
• Mixed Data Types: For datasets with complex non-linear relationships or mixed categorical/numerical data, tree-based methods like MissForest offer a strong alternative, though at a higher computational cost.