Expert-augmented automated machine learning optimizes hemodynamic predictors of spinal cord injury outcome

Austin Chou; Abel Torres-Espin; Nikos Kyritsis; J. Russell Huie; Sarah Khatry; Jeremy Funk; Jennifer Hay; Andrew Lofgreen; Rajiv Shah; Chandler McCann; Lisa U. Pascual; Edilberto Amorim; Philip R. Weinstein; Geoffrey T. Manley; Sanjay S. Dhall; Jonathan Z. Pan; Jacqueline C. Bresnahan; Michael S. Beattie; William D. Whetstone; Adam R. Ferguson; the TRACK-SCI Investigators

doi:10.1371/journal.pone.0265254

Abstract

Artificial intelligence and machine learning (AI/ML) is becoming increasingly more accessible to biomedical researchers with significant potential to transform biomedicine through optimization of highly-accurate predictive models and enabling better understanding of disease biology. Automated machine learning (AutoML) in particular is positioned to democratize artificial intelligence (AI) by reducing the amount of human input and ML expertise needed. However, successful translation of AI/ML in biomedicine requires moving beyond optimizing only for prediction accuracy and towards establishing reproducible clinical and biological inferences. This is especially challenging for clinical studies on rare disorders where the smaller patient cohorts and corresponding sample size is an obstacle for reproducible modeling results. Here, we present a model-agnostic framework to reinforce AutoML using strategies and tools of explainable and reproducible AI, including novel metrics to assess model reproducibility. The framework enables clinicians to interpret AutoML-generated models for clinical and biological verifiability and consequently integrate domain expertise during model development. We applied the framework towards spinal cord injury prognostication to optimize the intraoperative hemodynamic range during injury-related surgery and additionally identified a strong detrimental relationship between intraoperative hypertension and patient outcome. Furthermore, our analysis captured how evolving clinical practices such as faster time-to-surgery and blood pressure management affect clinical model development. Altogether, we illustrate how expert-augmented AutoML improves inferential reproducibility for biomedical discovery and can ultimately build trust in AI processes towards effective clinical integration.

Citation: Chou A, Torres-Espin A, Kyritsis N, Huie JR, Khatry S, Funk J, et al. (2022) Expert-augmented automated machine learning optimizes hemodynamic predictors of spinal cord injury outcome. PLoS ONE 17(4): e0265254. https://doi.org/10.1371/journal.pone.0265254

Editor: Antal Nógrádi, Szegedi Tudomanyegyetem, HUNGARY

Received: December 13, 2021; Accepted: February 25, 2022; Published: April 7, 2022

Copyright: © 2022 Chou et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The training and validation datasets analyzed in the current study will be published and made publicly available in the Open Data Commons for Spinal Cord Injury (ODC-SCI.org) repository concurrent with the publication of the manuscript. The associated dataset DOI(s) directing to the published datasets will be provided and this section updated prior to manuscript publication.

Funding: This work was supported by research grants: Department of Defense/Congressionally Directed Medical Research Programs/Spinal Cord Injury Research Program SC150198 (MSB) and SC190233 (MSB); Craig H. Neilsen Foundation SCI - Center of Excellence Award (MSB); National Institute of Health/National Institute of Neurological Disorders and Stroke R01NS088475 (ARF) and UH3NS106899 (ARF); US Department of Veterans Affairs 1I01RX002245 (ARF); US Department of Veterans Affairs I01RX002787 (ARF); Craig H. Neilsen Foundation (ARF); Wings for Life Foundation (ARF); National Institute of Health/National Institute of Neurological Disorders and Stroke National Research Service Award F32NS117728 (AC). Authors AC, SK, JF, JH, AL, RS, and CM are current or former employees of DataRobot and own shares of the company. Authors SK, JF, JH, AL, RS and CM received support in the form of salaries from DataRobot. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. DoD/CDMR/SCIRP: https://cdmrp.army.mil/scirp/default Craig H. Neilsen: https://chnfoundation.org/ NIH/NINDS: https://www.ninds.nih.gov/ US VA: https://www.research.va.gov/ Wings for Life: https://www.wingsforlife.com/us.

Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: AC, SK, JF, JH, AL, RS, and CM are current or former employees of DataRobot and own shares of the company. Access to the DataRobot Automated Machine Learning platform was awarded through application and selection by the DataRobot AI for Good program. DataRobot affiliated authors provided editorial contributions during the preparation of the manuscript. All other authors have declared that they have no competing interests.

Introduction

Automated machine learning (AutoML) is a rapidly-developing ML subfield focused on automating model optimization processes including algorithm selection, feature engineering, and hyperparameter tuning [1, 2]. AutoML applications produce high-performance models across diverse sophisticated algorithms and preprocessing methodologies while reducing the overall need for human input and modeling expertise [3, 4]. Correspondingly, AutoML is lowering the technical and knowledge barrier impeding ML democratization for various domains including biomedicine [5–9]. With the growing popularity of artificial intelligence and machine learning (AI/ML) in clinical research [10–13] and the increasing breadth, depth, and accessibility of clinical health data [14], AutoML stands to exponentially accelerate clinical ML applications by empowering scientists and clinicians to train and leverage powerful models [5]. However, clinical utility requires ML models to be interpretable for biological mechanisms, verifiable by clinicians, and methodologically and inferentially reproducible [15, 16]. Achieving reproducibility is further complicated by the fact that clinical datasets often have small sample sizes relative to the number of variables collected which can result in unstable model behavior [17, 18]. This is especially true for rare diseases and disorders with smaller patient populations, and translation of AutoML from computer-to-clinic thus necessitates additional approaches beyond maximizing prediction accuracy with “black box” algorithms. Here, we developed a modeling framework that incorporates explainable and reproducible AI strategies to predict spinal cord injury (SCI) patient outcome. Furthermore, we demonstrate how we can improve the inferential reproducibility of ML, augment the process with clinical knowledge, and effectively leverage AutoML for model optimization.

While SCIs have a comparatively small patient population—about 17,900 new cases a year and 296,000 patients with chronic disabilities in the US—SCIs are highly debilitating and result in chronic motor, sensory, and autonomic impairment including paralysis [19]. The severity is reflected by the total societal cost which is estimated to exceed $267 billion [20]. Alongside the relatively limited sample size of available clinical SCI datasets, the variety of SCI characteristics presents significant challenges for reproducible identification of patient outcome predictors despite the volume of data collected throughout patient hospitalization and treatment [20]. Various prognostic models for SCI outcome have been developed with algorithms ranging from logistic regression to extreme gradient boosted (XGB) trees and convolutional neural networks [12, 21, 22]. While such studies bear potential for informing clinical care, algorithm selection in many SCI ML studies have primarily depended on the researchers’ familiarity with specific ML algorithms, and prediction accuracy remains the primary metric for comparing models [23, 24]. Moreover, deciphering the relationships between outcome and predictors is still difficult with complex algorithms, ultimately dampening clinician enthusiasm about applying ML tools and results given the inability to interpret and verify such models [25].

In our AutoML application for SCI patient prognosis, we demonstrate:

A framework for reproducible and explainable modeling that implements (1) a repeated cross-validation strategy, (2) performance precision and feature instability metrics and analyses, (3) model interpretation with permutation feature importance (pFI) and partial dependence plots (PDPs), (4) stabilized backward feature reduction, and (5) model validation with population similarity analysis (Fig 1). In particular, repeated cross-validation allows for model aggregation to account for modeling variability and improve the inferential reproducibility of the results.
The importance of integrating domain expertise. We highlight how stabilized pFI and PDPs, useful model-agnostic explainable AI tools, enable biomedical researchers to draw robust inferences regarding the relationship between clinical variables and outcome. Furthermore, we illustrate how augmenting feature selection with domain expertise can improve model performance beyond deploying ML naively.
Additional analyses to interpret model validity. Many clinical ML studies with smaller sample sizes depend on newly collected data for external validation. Investigating the population similarity between training and validation cohorts provides meaningful information about model generalizability beyond validation performance and can capture evolving clinical practices that inevitably affects clinical ML implementation.

Download:

Fig 1. A framework for applying Automated Machine Learning (AutoML) for reproducible inferences in biomedical research.

After data is curated, we perform a cyclical model development process utilizing AutoML to optimize an array of models. Reproducible and explainable AI tools and strategies can be applied to ultimately draw clinical and biological inferences from the models and allow for integration of domain expertise. Critically for clinical modeling, we also include a feature reduction component to achieve a more parsimonious model. The final models are then validated with external validation data along with population similarity analysis for further clinical contextualization. By applying this framework, models produced by AutoML can be stabilized and interpreted for inferential reproducibility and clinical verifiability.

https://doi.org/10.1371/journal.pone.0265254.g001

By applying this framework to SCI, we identified actionable intraoperative mean arterial pressure (MAP) thresholds for hypertension and hypotension associated with worse patient outcome. Additionally, our analysis revealed underlying evolutions of clinical practices, such as reducing time-to-surgery and hypotension management, that invariably affects model validation efforts in SCI research. Altogether, we present methods to bolster the interpretability, reproducibility, and trustworthiness of clinical ML especially as AI/ML and AutoML becomes increasingly accessible to biomedical researchers.

Results

AutoML model generation

We applied an AutoML platform to investigate clinical predictors of SCI patient outcome from intraoperative and acute hospitalization records collected between 2005–2011 and curated by the Transforming Research and Clinical Knowledge for SCI (TRACK-SCI) program, one of the largest SCI patient registries in the US [20]. We selected 46 variables (i.e. features) as predictors from de-identified data of 74 patients (S1 Table). Of these, 16 features were summary statistics (i.e. mean, standard deviation, skew, and kurtosis) derived from timeseries data capturing heart rate, systolic blood pressure, diastolic blood pressure, and mean arterial pressure (MAP) during SCI surgery. As intraoperative hypertension and hypertension have been shown to be detrimental to SCI outcome [26, 27], we also calculated the time each patient spent outside of previously-established upper (104 mmHg) or lower (76 mmHg) MAP thresholds during surgery (time_MAP_Avg_above_104 and time_MAP_Avg_below_76, respectively) [28]. We defined the prediction target as whether the patient’s ASIA Impairment Scale (AIS) score, a common SCI outcome assessment [29], improved between time of hospital admission and time of hospital discharge.

To account for potential instability in model optimization, we applied a repeated 10-fold cross-validation strategy with 25 repetitions where each had a unique partitioning arrangement (i.e. 25 projects) [30, 31]. We then aggregated the results for analysis. AutoML generated 80–90 blueprints: unique combinations of data preprocessing methods and ML algorithms. From these, the platform fully optimized 30–40 models, 15 of which had better mean performance (lower LogLoss and higher AUC) than the benchmark majority class classifier model (Fig 2A and 2B).

Download:

Fig 2. AutoML generated 15 models that performed better than the Majority Class Classifier model.

Each model consisted of automatically implemented preprocessing steps and algorithms. Models were assigned names according to the algorithm and encoded by a unique color. Blueprints of the same algorithm class are numbered for identification across both (A) LogLoss and (B) Area Under Curve (AUC) plots. Two models were selected for additional analysis: BP_log (blue box) and BP_XGB (green box). Aggregating across 25 projects (unique partitioning arrangements of the dataset), BP_log had an average performance of 0.67 ± 0.01 LogLoss and 0.68 ± 0.02 AUC; BP_XGB had an average performance of 0.68 ± 0.01 LogLoss and 0.67 ± 0.02 AUC. (C) BP_log consisted of a regularized logistic regression (L2) algorithm with a notable quintile spline transformation preprocessing step for numeric variables. (D) BP_XGB implemented an eXtreme Gradient Boosted (XGB) trees classifier with unsupervised learning features, which refers to the TensorFlow Variational Autoencoder preprocessing step for categorical variables.

https://doi.org/10.1371/journal.pone.0265254.g002

For the purposes of illustration, we selected two high performance blueprints for further interpretation and validation. The first was a L2 regularized logistic regression model with a spline transformation of numeric variables during data preprocessing (BP_log; Fig 2C). BP_log had the best overall performance by LogLoss (0.67 ± 0.01) and was a top performer by AUC (0.68 ± 0.02). To apply our framework to a highly complex model use case and given the popularity of XGB in biomedical ML research, we also examined the “eXtreme gradient boosted trees classifier with unsupervised learning features” blueprint with the best LogLoss performance in its class (BP_XGB; LogLoss = 0.68 ± 0.01; AUC = 0.67 ± 0.02). Importantly, XGB trees have gained popularity in biomedical ML research. BP_XGB specifically includes a TensorFlow Variational Autoencoder preprocessing step [32] (Fig 2D) as the “unsupervised learning feature”, exemplifying the availability of sophisticated methodologies through AutoML platforms.

Feature importance

We utilized a permutation-based approach [33] to quantify feature importance (pFI) for BP_log and BP_XGB. Notably, pFI values from individual models varied significantly; we accordingly aggregated pFI across the 25 projects for more robust comparisons. While the order of features by importance for BP_log and BP_XGB were different, we observed that many of the high importance features for both models were the timeseries summary statistics (S1 Fig). Interestingly, time_MAP_Avg_above_104 and time_MAP_Avg_below_76 were the most important features for BP_log (S1A Fig) but were 11^th and 18^th in rank respectively for BP_XGB (S1B Fig).

Performance precision

We observed that different partitioning arrangements resulted in varying model performances even with the same blueprint. We sought to determine whether aggregating the results of 25 projects significantly improved the precision of model performances (i.e. performance precision) towards reproducible comparisons between blueprints. In addition to the confidence intervals (CI) for each blueprint, we additionally defined and calculated the standardized performance CI width: a measure of the performance precision relative to the model’s mean performance (see Methods for formula). For example, the standardized performance CI width allows us to identify the number of projects needed to obtain a CI width that is within 5% of the blueprint performance. From the 25 projects in our primary repeated cross-validation workflow, we observed standardized CI widths of 1.92% by LogLoss and 2.88% by AUC for BP_log and 1.55% by LogLoss and 2.73% by AUC for BP_XGB.

To investigate how performance precision changes with the number of projects aggregated, we ran 150 projects with BP_log and BP_XGB and performed a sampling analysis (see Methods). For BP_log, we observed that aggregating 25 projects more than halved the expected standardized performance CI width when compared to only 2 projects: by LogLoss, 5.22 ± 0.24% vs 1.85 ± 0.01% for 2 projects vs 25 projects respectively (S2A Fig) and by AUC, 8.03 ± 0.38% vs 2.79 ± 0.02% respectively (S2B Fig). Analysis of BP_XGB performance precision generated similar results: by LogLoss, 5.42 ± 0.32% vs 2.06 ± 0.04% for 2 projects vs 25 projects respectively (S2C Fig) and by AUC, 8.36 ± 0.38% vs 2.84 ± 0.02% respectively (S2D Fig). The performance precision analysis further highlights the variability in model performances by different partitioning arrangements even if the blueprint and training dataset are unchanged.

Feature instability

We similarly observed that different partitioning arrangements resulted in pFI variability (i.e. feature instability). Given two different pFI lists—for example, from two different modeling projects or multi-project aggregates with corresponding averaged pFI—we can quantify the differences between them by calculating the feature rank instability (FRI) (see Methods for formula). The FRI value sums the difference in the pFI-based ranking for each feature shared between any two pFI lists. Higher FRI indicates more dissimilarity in ranking and thus more feature instability.

Similar to the performance precision analysis, we performed a sampling analysis with 150 projects to determine the relationship between number of projects aggregated and feature instability for BP_log and BP_XGB (see Methods). We observed extremely high FRI when the number of aggregated projects is small, suggesting that pFI can differ significantly from one project to another. When we increased the number of aggregated projects towards 150, FRI decreased towards 0, indicating that pFI ranking can be stabilized with sufficient project aggregation. At 25 projects, BP_log and BP_XGB had average FRI values of 13.03 ± 0.34 and 11.65 ± 0.33 respectively (Fig 3A and 3B). This approximately amounted to a 93% decrease in instability for both BP_log and BP_XGB as compared to when only aggregating across 2 projects.

Download:

Fig 3. Feature rank instability (FRI) analysis as a function of number of projects aggregated.

As the number of projects increased, FRI decreased (i.e. pFI ranking became more stable). (A, B) Expected FRI calculated for all 46 features. BP_log had an average FRI of 174.40 ± 2.14 with 2-project aggregation and 13.03 ± 0.34 with 25-project aggregation (A). Similarly, BP_XGB started with an average FRI of 153.83 ± 3.06 that decreased to 11.65 ± 0.33 at 25 projects (B). (C, D) Focusing only on the bottom five features by pFI to calculate FRI, BP_log had an average FRI of 20.41 ± 0.75 with 2-project aggregation and decreased to 0.96 ± 0.08 with 25-project aggregation (C). Similarly, BP_XGB started with an average FRI of 7.77 ± 0.37 and decreased to 0.56 ± 0.06 for the bottom five features with 25-project aggregation (D).

https://doi.org/10.1371/journal.pone.0265254.g003

Automated feature reduction

A component of AutoML is automating feature reduction (i.e. variable selection) to obtain a more parsimonious feature list [34, 35]. This is particularly important for clinical models since clinical features often outnumber the observations in biomedical datasets, increasing the danger of model overfitting [36]. We employed an iterative backward wrapper approach utilizing pFI to determine and remove the lowest-importance features at each step (see Methods).

Importantly, we needed to ascertain the stability of features to be eliminated. Because our approach initially removes five features at a time, we accordingly applied our feature instability analysis to just the five lowest-importance features. With 25 projects, we found that BP_log had a FRI value of 0.96 ± 0.08 (Fig 3C) and BP_XGB had a FRI value of 0.56 ± 0.06 (Fig 3D) for the bottom five features. In contrast, the bottom five features cumulatively shifted at least eight ranks on average if we aggregated only two projects. By aggregating across 25 projects, we can be confident that the least important features are reliably the lowest ranked.

The best-performing parsimonious BP_log had an average LogLoss of 0.55 ± 0.02 and only nine retained features (Fig 4A). Of these, the highest pFI features included time_MAP_Avg_above_104, time_MAP_Avg_below_76, and the MRI BASIC score (MRI_1_BASIC_Score), a neuroimaging score for injury severity collected upon hospital admission (Fig 4B) [37]. The corresponding mean AUC (0.83 ± 0.02) was also close to the maximum AUC of the feature-reduced models (S3A Fig).

Download:

Fig 4. Applying an iterative backward feature reduction process to identify parsimonious feature lists that maximize model performance.

The process was performed first by removing the lowest five features by feature importance (step size = 5) and then repeated with step size = 1 within the feature list size range that contained the best performance. (A) For BP_log, the step size was reduced starting at 16 features with the best performance observed with the 9-feature parsimonious feature list (LogLoss = 0.55 ± 0.02). (B) The corresponding pFI of the 9-feature parsimonious BP_log model showed that the MRI BASIC score and the time patients spent outside of the MAP thresholds were the most important features. The remaining features included other intraoperative timeseries-derived features and the time between hospitalization and surgery (Time_to_OR_a). (C) The feature reduction for BP_XGB was expanded to always preserve the two MAP threshold features. The step size was reduced to one starting at 16 features with the best performance observed with the 11-feature parsimonious feature list (LogLoss = 0.48 ± 0.02). (D) The corresponding pFI for the parsimonious BP_XGB model showed that the AIS score at admission (AIS_ad) was the most important feature. Non-timeseries-derived features included Cervical_Injury, Vertebral_Artery_Injury, and TBI_Present. The time_MAP_Avg_above_104 and time_MAP_Avg_below_76 features were ranked 7^th and 9^th respectively.

https://doi.org/10.1371/journal.pone.0265254.g004

Interestingly, initial feature reduction with BP_XGB removed the time_MAP_Avg_below_76 feature. Given that clinical experts and SCI literature emphasize the correlation between hypotension and worse patient outcome [28], we tested if preserving time_MAP_Avg_below_76 during feature reduction would produce better parsimonious model performance. The resulting parsimonious BP_XGB model included 11 features, had an average LogLoss of 0.48 ± 0.02 (Fig 4C), and was close to the maximum AUC observed (0.87 ± 0.01) (S3B Fig). Notably, this final performance was better than the best parsimonious model when time_MAP_Avg_below_76 had been eliminated (LogLoss = 0.52 ± 0.02; AUC = 0.87 ± 0.01). Furthermore, time_MAP_Avg_below_76 did not end up as the lowest-ranked feature in the final parsimonious feature list despite requiring user guidance to prevent elimination (Fig 4D). Since time_MAP_Avg_below_76 exhibited collinearity with the other surgical timeseries-derived features, the handling of collinear features by the XGB algorithm is likely why time_MAP_Avg_below_76 was dropped without user intervention. Indeed, we observed higher FRI at each feature reduction step for BP_XGB as compared to BP_log corresponding with larger pFI changes for BP_XGB as collinear features are eliminated (S4 Fig). Additionally of interest, the most important feature of the parsimonious BP_XGB feature list was the AIS score at admission (AIS_ad) which provides similar context for initial injury severity as the MRI BASIC score [37].

Feature interpretation

For additional interpretability, we utilized partial dependence plots (PDPs) to quantify the relationship of each individual features’ values to the model’s prediction [38–40]. PDPs are a model-agnostic approach and thus can be applied regardless of the preprocessing steps or algorithm implemented. We aggregated the partial dependence for each feature for BP_log and BP_XGB across the 25 projects (S5 and S6 Figs).

The PDPs of the initial injury severity features–MRI_1_BASIC_Score for BP_log and AIS_ad for BP_XGB–captured nuances between the two (Fig 5A and 5B). We observed that a BASIC score of 4, which corresponds to severe injuries with notable hemorrhage, reduced the probability of patient improvement. Similarly, patients classified as AIS A (i.e. complete, severe SCIs) had the lowest probability of outcome improvement. Both PDPs thus conveyed that the most severe SCI cases are unlikely to see improvement by the time of hospital discharge. However, the two scores had different effects for mild injuries: for BP_log, low BASIC scores (0–3) all increased the probability of improvement whereas AIS D reduced the likelihood of improvement for BP_XGB. This underscores the difference in sensitivity and granularity between the AIS and BASIC scores. AIS D broadly encapsulates mild SCIs and is effectively a ceiling on the scale since improvement requires full recovery, which is uncommon. BASIC scores of 0–2 cover a range of functionally mild-to-moderate SCIs; indeed, BASIC and AIS scores do not correlate 1:1 [37]. Overall, the results suggested that patients with moderate SCIs (AIS B and C; BASIC 2) have the highest likelihood of outcome improvement.

Download:

Fig 5. Partial dependence plots (PDPs) for features of interest help interpret how features affect model prediction of BP_log and BP_XGB.

(A) For BP_log, an MRI BASIC score of 4 resulted in lower prediction of improved outcome. A MRI BASIC score of 0–3 increased prediction of better outcome with a MRI BASIC score of 2 leading to the highest probability of improvement. (B) For BP_XGB, an AIS score of A or D at admission resulted in lower probability of patient improvement. AIS scores of B and C both led to higher probability of improvement with AIS score C resulting in the highest probability. (C) For BP_log and (D) BP_XGB, if a patient’s MAP exceeded an upper threshold of 104 mmHg for more than 50–75 minutes, the predicted probability of improvement decreased significantly. (E) For BP_log and (F) BP_XGB, if a patient’s MAP fell below a lower threshold of 76 mmHg for more than 100–150 minutes, the predicted probability of improvement decreased significantly. Notably, BP_XGB PDP for both time_MAP_Avg_above_104 and time_MAP_Avg_below_76 exhibited a rebound in predicted improvement probability at extreme upper values that was absent on the BP_log PDPs.

https://doi.org/10.1371/journal.pone.0265254.g005

PDPs of time_MAP_Avg_above_104 and time_MAP_Avg_below_76 revealed that the models predicted worse outcome if a patient exceeded 104 mmHg by more than 70 minutes (Fig 5C and 5D) or dropped below 76 mmHg for more than 150 minutes (Fig 5E and 5F). BP_log and BP_XGB produced similar time_MAP_Avg_above_104 and time_MAP_Avg_below_76 PDPs, though we observed that BP_XGB predicted relatively better outcome for patients at the extreme upper range of time (>115 min for time_MAP_Avg_above_104 and >200 min for time_MAP_Avg_below_76). This is likely due to the dataset having fewer patients at the extreme ranges rather than a true clinical effect. Critically, BP_log implemented a spline transformation (Fig 2C) and quintiled the continuous variables; all patients exceeding 70 or 150 minutes outside the upper or lower thresholds respectively were categorized to the same quintile. Accordingly, BP_log would not produce the PDP rebound observed with BP_XGB.

MAP threshold validation

We previously found that time outside the MAP range of 76–104 mmHg was associated with lower probability of AIS improvement as determined by LASSO logistic regression models testing different MAP ranges while expanding the lower and upper MAP thresholds simultaneously [28]. To validate the MAP thresholds, we started with the best-performing parsimonious feature lists, removed the MAP threshold features, and then swept through various lower threshold (70–85 mmHg) or upper threshold (95–115 mmHg) features separately to identify how the threshold affected prediction.

With BP_log, we observed the best performances with lower thresholds of 74–76 or 79 mmHg and with upper thresholds at 103–105 mmHg by LogLoss (Fig 6A). With BP_XGB, we observed the best performances at lower thresholds of 74–76 mmHg and at upper thresholds at 103–104 mmHg (Fig 6B). The results were similarly reflected with AUC (S7 Fig), corroborating the thresholds of 76 and 104 mmHg for predicting patient outcome. Our analysis furthermore revealed that the time spent above the upper threshold improved the predictive performance of the models significantly more, highlighting intraoperative hypertension as an important correlate and a potential factor for worse SCI recovery.

Download:

Fig 6. LogLoss performance plots for investigating different lower and upper MAP thresholds using best-performing parsimonious BP_log and BP_XGB models.

(A) With BP_log, we observe that the lower threshold values of 74, 75, 76, and 79 mmHg performed the best of the lower thresholds. The upper threshold values of 103, 104, and 105 mmHg performed the best of the upper thresholds. Notably, the best-performing upper threshold feature (104 mmHg) resulted in a larger improvement to model performance compared to the best-performing lower threshold feature (79 mmHg). (B) With BP_XGB, the values of 74, 75, and 76 mmHg performed the best of the lower thresholds, and the values of 103 and 104 performed the best of the upper thresholds. Similar to BP_log, the best-performing upper threshold feature (104 mmHg) resulted in a larger improvement to model performance compared to the best-performing lower threshold feature (76 mmHg).

https://doi.org/10.1371/journal.pone.0265254.g006

Model validation

We visualized how each step of the workflow improves AUC and corresponding receiver operating characteristic (ROC) curves. Starting with BP_log and all our predictors except MAP threshold features, we achieved an average AUC of 0.63 ± 0.019 (S8A Fig). Inclusion of MAP threshold features (with 76 and 104 mmHg thresholds) improved AUC to 0.68 ± 0.02 (S8B Fig), underscoring the importance of intraoperative MAP regulation. The largest improvement to model performance occurred after feature reduction to a more parsimonious feature list (AUC 0.84 ± 0.02) (S8C Fig). Adjusting the MAP thresholds produced miniscule improvements to AUC (AUC 0.85 ± 0.02) (S8D Fig). Lastly, by repeating the process with BP_XGB, prediction performance was improved to AUC 0.87 ± 0.01 (S8E Fig). Altogether, we can improve model performance by adjusting the feature list and model by leveraging the AutoML workflow (S8F Fig). Importantly, the ROC curves from each of the 25 projects notably differ, emphasizing how varying the dataset partitioning can produce significant model variability.

Critically, the greatest obstacle for translating predictive models into the clinic is the validity of the models on novel data. While many ML scenarios implement a holdout partition from the original dataset for validation, clinical datasets often have relatively small sample sizes where such practices would result in underfitted models, especially for medical fields with smaller patient populations such as SCI [41]. Clinical model validity is thus often assessed with an external validation dataset collected from a new cohort of patients. Here, we obtained additional data for external validation from a prospective (2015 onward) TRACK-SCI cohort of 59 patients. Of these, 14 patients improved in outcome while the remainder 45 did not.

We used the parsimonious BP_log and BP_XGB models to predict the probability of AIS improvement of the validation cohort. We aggregated these values across the 25 projects and generated plots and confusion matrices using the mean predictions and best F1 thresholds calculated by the AutoML platform. The 9-feature parsimonious BP_log model correctly predicted 13 of the 14 patients who improved but only 15 of the 45 patients who did not (Fig 7A; S9A Fig). The 11-feature parsimonious BP_XGB model correctly predicted only 9 of the 14 patients with AIS improvement and 14 of the 45 patients without (Fig 7B; S9B Fig). While BP_XGB has higher predictive accuracy on the training dataset, the model did not perform as well on novel data as the BP_log model.

Download:

Fig 7. Model validation confusion matrices and clustering analysis to demonstrate differences in patient population between training and validation datasets.

Validation predictions were scored by comparing the average predicted probability of each validation sample against the average best F1 threshold for the corresponding model. (A) The best parsimonious BP_log model correctly predicted 13 of the 14 true positives (i.e. patient improved in outcome) and 15 of the 45 true negatives. (B) The best parsimonious BP_XGB model correctly predicted 9 of the 14 true positives and 14 of the 45 true negatives. (C) UMAP and HDB clustering analysis on the combined training and validation data produced six clusters of patients. Notably, Clusters 1 and 2 showed high representation in the training cohort and low representation in the validation cohort. Conversely, Cluster 3 showed low and high representation in the training and validation cohorts respectively. Clusters 3, 5, and 6 have no discernable differences between cohorts.

https://doi.org/10.1371/journal.pone.0265254.g007

We hypothesized that the poor validation performance was due to data drift where the validation patient population no longer resembled that of the training dataset. We accordingly performed population similarity analysis, starting with population stability index (PSI) assessment for each of the parsimonious features. PSI broadly reflects the differences in value distribution between the two cohorts [42]. We observed that most of the features exhibited significant (PSI > 0.25) or moderate (PSI > 0.1) drift between training and validation datasets, and only TBI_Present and Vertebral_Artery_Injury features could be considered to not have drifted (PSI < 0.1) (S2 Table). The PSI results overall suggested that the training and validation populations are dissimilar, thus resulting in poor model performance during validation.

To investigate clinical trends underlying data drift, we clustered all the patients by the raw feature values of the 15 parsimonious features via UMAP and HDB clustering. We observed notable differences in cluster representation: in particular, the validation cohort was only sparsely represented in Clusters 1 and 2 as compared to the training cohort (Fig 7C). We summarized the distribution of values within each cluster to better understand the subpopulation characteristics and found that Cluster 1 was defined by extremely high Time_to_OR and Cluster 2 by extremely high time_MAP_Avg_below_76 (S3 Table). In discussion with clinical experts, we found that this corresponded with shifting clinical practices to reduce time-to-surgery and prevent intraoperative hypotension [43, 44]. Population similarity analysis thus provided critical insight into the differences between training and validation populations while demonstrating the crucial need to validate predictive models and corresponding conclusions before translating findings to the clinic.

Overall, we illustrate a framework to augment AutoML for interpretability, reproducibility, and validity. The process presents opportunities to integrate methods for explainable and reproducible AI that are essential for biological inferences and evidence-based clinical practice. Augmenting AutoML to generate verifiable machine intelligence will be critical to building confidence and trust in powerful AI tools towards advancing biomedical research and precision medicine.

Discussion

As AutoML become increasingly efficient at optimizing an ever-growing repertoire of data preprocessing methods and algorithms, biomedical researchers will have more opportunities to leverage AI/ML to transform their field [4, 6, 9, 45]. Importantly, there remains a parallel, fundamental need for better evaluation and interpretation of AutoML-derived models for healthcare where trust in ML requires reproducibly identifying and validating biomedical inferences before translation to patient care [46]. Here, we demonstrate a framework to improve the reproducibility of ML and AutoML workflows by implementing a repeated cross-validation strategy, assessing metrics of model stability, directly interrogating and augmenting the models with clinical knowledge, and applying population similarity analysis to better contextualize the findings and generalizability of the models (Box 1; Fig 1). By incorporating such methods and strategies for model interpretability and inferential reproducibility, biomedical experts will be empowered to fully leverage their domain expertise into ML processes to construct more trustworthy models and build confidence towards translatable AI/ML applications.

Box 1. Key highlights of the framework for reproducible, interpretable AutoML application in biomedicine

Perform modeling with a repeated cross-validation strategy. Aggregating across models mitigates spurious findings due to model variance from implicit modeling parameters that can lead to model instability with smaller clinical datasets, such as partitioning arrangement.
Stabilize pFI and PDPs for model interpretation by aggregating repeated cross-validation models to improve inferential reproducibility.
Characterize performance precision (metrics: performance CI; standardized performance CI width) for more robust model comparisons. Performance precision analysis can be further applied to achieve a target precision in modeling processes.
Characterize feature instability (metric: pFI CI; feature rank instability) to capture pFI variability. Feature instability analysis can be further applied to stabilize pFI-dependent processes such as feature reduction.
Integrate domain expertise throughout; combining domain expertise and model-driven feature selection can improve final parsimonious model performance. Any inferences drawn from modeling should be verified with clinical expertise.
Investigate data drift (i.e. population similarity analysis) between the training and external validation datasets during model validation. PSI and clustering analysis can uncover clinical differences between the cohorts and the underlying evolution of clinical practices that further inform model validation results.

Repeated cross-validation is an easily applied strategy to reinforce the reproducibility of ML research and has been well-established for handling variation in partitioning [47, 48]. Yet despite the focus on confidence intervals, significance, and inference in biomedical research, repeated cross-validation is not commonly implemented by clinical ML studies. This is likely due to the associated computational costs, resulting in clinical ML researchers optimizing and drawing conclusions from a single model instead. We show that different dataset partitions affected model performance and pFI values; the practice of only relying on a single optimization can thus lead to false assumptions that one algorithm is definitively better at prediction or that specific features are definitively more important. Additionally, combining repeated optimizations can help stabilize subsequent processes; 25 projects was the critical aggregation threshold for our backward feature reduction approach such that the bottom 5 features by pFI would be reproducibly ranked as the least important. Naturally, selecting the number of repetitions ultimately depends on the dataset, the specific model, and the research question or ML process of interest. Nevertheless, a repeated k-fold cross-validation strategy to report confidence intervals, performance precision, and feature instability can help contextualize the reproducibility of results and mitigate spurious conclusions whether applying AutoML or a single model blueprint.

The variability we observed also reflects the concept of underspecification in ML: that even with the same model blueprint and training data, different optimizations can produce divergent solutions [49]. Underspecification highlights how seemingly arbitrary modeling choices—such as implicit modeling parameters and data partitioning—can lead to models with high accuracy on training data that then fail to perform on novel data. While this emphasizes the need to validate models, underspecification also underscores an ongoing demand for additional model evaluation metrics. Repeated cross-validation strategies along with performance precision and feature instability analyses can be applied to characterize and control for underspecification factors of model optimization. This can be further extended to deployment applications: bagging models to account for underspecification can improve model validity, and precision and instability analyses can help estimate the effective number of models for ensembling. Moreover, the predictions of the individual models underlying an ensemble can provide useful context about model precision to users and ultimately enhance trustworthiness and adoption for clinical decision-making.

A major challenge in clinical modeling is obtaining external validation data, especially for rarer diseases and disorders [20, 41]. Inconsistent data collection methods and standards further exacerbate the difficulties. Many clinical ML studies consequently produce accurate models that remain unvalidated or are later shown to underperform on novel data [50–52]. We obtained data from a recent patient cohort enrolled in the TRACK-SCI prospective study and observed that despite the high performance of the final BP_log and BP_XGB models on training data, both performed poorly during validation. Population similarity analysis by PSI revealed that almost every feature exhibited significant data drift, indicating differences between the two patient cohorts. We reviewed the PSI and cluster analysis results with TRACK-SCI clinicians who validated that the observed changes corresponded with evolving guidelines including moving SCI patients into surgery sooner and improving blood pressure management to avoid hypotension [43, 44]. Indeed, both practices were implemented during the prospective era, and the overarching findings illustrate the fact that biomedical data inevitably shifts with ever-updating clinical practices [25]. Furthermore, while all of our data was collected at UCSF, the phenomenon of data drift can apply to data from different medical centers; clinical ML studies commonly utilize data from only a single source, and resultant models fail to generalize. Thus beyond simply validating models with novel data, the analytical framework for successful clinical ML application should also contrast the training and validation populations with quantitative metrics alongside domain expertise to identify critical clinical context to inform model validity.

Understanding population similarity can also provide avenues for improving clinical models: a simplistic approach could be to retrain the model with more recent or representative data while balancing dataset size and population similarity. Researchers could also directly incorporate sources of data drift and population dissimilarity into the modeling strategy, such as via cross-hospital validation, to produce the most generalizable models and inferences [53]. A third possibility is to combine data across time and centers to obtain a larger sample size, thus allowing for a holdout partition that better mimics the training dataset. With careful, balanced representation of patients and feature values in training and holdout partitions, researchers could identify common predictive features that are generalizable across a broader clinical population. Specific to SCI, harmonizing data across multiple clinical programs could help reinforce future ML studies with a more comprehensive patient dataset. Most importantly, once a model is deployed into clinic, monitoring for data drift will be necessary to determine whether the model will need to be retrained to keep up with changing medical practices.

Especially for datasets with a small observations-to-features ratio as is common in clinical research of rare disorders, parsimony can improve performance by decreasing multicollinearity and removing low-signal features while improving interpretability [54]. We deployed an iterative wrapper approach [35]: backward feature selection based on recalculated pFI rankings to optimize LogLoss. While the process is model-agnostic, the results are affected by the characteristics of the underlying algorithms; the final parsimonious feature lists differed between BP_log and BP_XGB, and BP_XGB exhibited in greater pFI instability between each feature reduction step. This can be attributed to how regularized regression and tree-based models handle multicollinearity—a common trait of clinical datasets—which affects pFI as colinear features are eliminated. Critically, there is no definitive approach to feature selection; clinical verification of the final parsimonious feature list is necessary given that the preserved features depend heavily on the implemented reduction method and models [34, 35]. More broadly, pFI has been shown to overestimate the importance of colinear features, especially for tree-based models [55]; future work should refine the framework by employing techniques that better address feature collinearity, such as Shapley additive explanations or accumulated local effects plots [56, 57].

We allowed for preselection of features by domain experts during feature reduction, and because the process inherently optimizes model accuracy, we can determine if preservation of expert-selected features improves or undermines the final model performance. Here, by preserving the time a patient spent below 76 mmHg MAP, we improved the maximum performance for the parsimonious BP_XGB model. The process can also be applied to test other hypotheses, including ones where deliberate experimentation might be difficult or impossible. For example, whether a SCI patient receives specific treatments is a matter of clinical care rather than experimental design, but we can compare the impact of excluding or including the treatment feature on the final model accuracy to glean a relationship between treatment and outcome. Such findings would provide further insight to the efficacy of clinical practices that may be difficult to test experimentally and highlight areas for future, targeted clinical research. Importantly, removed variables are not necessarily unimportant or uninformative; feature selection ultimately reflects the representation of samples and the limitations of the dataset. Continual validation and updating of the parsimonious feature list alongside the model are critical for maintaining and improving clinical models. This aligns with the budding concept of expert-augmented AI wherein the interaction between human expertise and ML leads to better models.

Our results also provide more granularity on how intraoperative MAP thresholds relate to outcome: the time patients spent outside the upper MAP threshold contributed more to model prediction than the time spent below the lower threshold. Additionally, the corresponding PDPs reveal that the critical out-of-threshold duration for worse outcome is shorter for hypertension (> ~70 min) than for hypotension (> ~150 min). Previous blood pressure management studies have primarily focused on hypotension as a contributor to worse patient outcome [58–60] despite the increased risk of cardiovascular and cerebrovascular complications as a result of hypertension [61]. Notably, the importance of perioperative hypertension for SCI outcome has been observed both clinically and preclinically [26–28, 62]; this is the first analysis to suggest that hypertension is more predictive of worse outcome than hypotension, thus proposing that careful MAP management should strive to avoid hypertension while minimizing hypotension. Future prospective clinical studies should extend the verifiability of the findings throughout broader SCI patient care such as during treatment in the emergency room and intensive care unit. Moreover, the illustrated framework can be similarly applied to investigate other modifiable components of clinical care as well as identify predictors of other patient outcome metrics including chronic recovery.

Ultimately, there remains significant untapped potential for AI-driven impact on clinical practices, precision medicine, and general patient care, especially for rarer diseases and disorders. However, the challenges of achieving inferential reproducibility with AI/ML dictate the need for parallel development of explainable and reproducible AI methods that augment powerful processes such as AutoML. By unboxing even the most complex models and directly empowering clinical experts to guide the modeling process, we can build an essential culture of trust for AI in biomedicine towards a bidirectional relationship where clinicians inform AI development and AI applications effectively support clinical decision-making to improve patient care.

Materials and methods

Datasets

The data were collected and de-identified by the Transforming Research and Clinical Knowledge for Spinal Cord Injury (TRACK-SCI) program [20] and contains clinical variables (i.e. features) collected during acute hospitalization and SCI-related surgery. The training dataset consisted of 74 clinical records collected between 2005–2011. After implementing our AutoML workflow, we obtained a second dataset for model validation from TRACK-SCI consisting of 59 clinical records collected after 2015. Protocols for data collection and extraction were approved by the Institutional Research Board (IRB) at the University of California, San Francisco under the protocol numbers 11–07639 and 11–06997.

Of note, 18 of the 47 features in the full feature list were derived from time-series data for intraoperative heart rate, systolic blood pressure, diastolic blood pressure, and mean arterial pressure (MAP). Each set of time-series data was summarized as mean, standard deviation, skew, and kurtosis features for each individual patient. Additionally, the total time each patient spent above or below a MAP threshold (starting with 104 and 76 mmHg respectively) was derived from the time-series MAP data. The prediction target AIS_is_improved was derived from the patients’ AIS scores as assessed by clinicians using the International Standards for Neurological Classification of SCI (ISNCSCI) exam at the time of hospital admission and discharge. Specifically, AIS_is_improved was assigned a value of 0 for no improvement or a value of 1 if the patient’s AIS score improved between admission (AIS_ad) and discharge (AIS_dis). Notably, the training data included 39 patients who improved in AIS score while the validation dataset included 14 such patients. The AIS_dis feature was excluded for model training since it was used to derive the prediction target and would cause target leakage. The remaining 46 features used for modeling are listed in S1 Table.

AutoML model generation and feature importance

Among various available implementations of AutoML, we utilized the DataRobot commercial AutoML platform for our workflow [5]. Access and application of the platform was done primarily through the API in Python. Performance values (LogLoss and Area Under Curve; AUC), mutual information between predictors, permutation feature importance (pFI), partial dependence plot (PDP) values, receiver operating characteristic (ROC) values, model validation predictions, estimated best F1 thresholds, and population stability index (PSI) values were downloaded and then analyzed and graphed in R with the tidyverse package [63, 64].

The training data was uploaded to the platform which generates a new project instance (i.e. project). Each project encapsulates a specific set of modeling inputs and parameters such as the training data, the type of ML problem (i.e. regression vs classification), partitioning strategy, and others that define the initial state of the AutoML process. For our projects, we assigned distinct random seeds which specifically affected the unique partitioning arrangement of the data for modeling. Given that the AutoML platform requires a minimum of 100 observations to perform automated classification modeling, we accounted for the small number of records in our dataset by duplicating the entry for each patient, thus doubling the dataset from 74 to 148 records. Importantly, we specified a 10-fold cross-validation strategy while ensuring that each of the ten partitions had at least one representation of each of the possible prediction target values and that duplicated records were always partitioned together.

For the first round of modeling, we included 46 features as predictors (S1 Table). The AutoML platform generated 80–90 possible configurations of data preprocessing steps and algorithms (i.e. blueprints, with each blueprint being assigned a unique identification number). Blueprints range from simple (e.g. BP_log: regularized logistic regression model with a spline transformation preprocessing step) to complex (e.g. BP_XGB: extreme gradient boosted tree with a modified TensorFlow Variational Autoencoder preprocessing step), and the platform automatically performs data preprocessing and algorithm-specific optimization to maximize the final model performance. To identify the best-performing models, the platform first trained the blueprints on a small subset of the dataset and selected the top performing blueprints according to their validation LogLoss accuracy. The blueprint selection process was repeated with a larger subset of data for a second round of selection. The remaining blueprints, numbering between 30–40 total, were then optimized on the full dataset, and cross-validation accuracy was calculated.

To characterize the stability of the modeling process, we applied the strategy of repeated 10-fold cross-validation with 25 repetitions. Each repetition corresponded to a project with a unique random seed that determined the unique arrangement of data in the partitions. Each project implemented the same blueprints for AutoML; we accordingly aggregated the performances for each blueprint across all 25 projects by calculating the mean and 95% confidence interval for the corresponding cross-validation performances (i.e. performance). We arranged the models according to mean performance and plotted those that outperformed the Majority Class Classifier benchmark model which simply predicts every patient as having improved—the majority class of the AIS_is_improved target.

Permutation feature importance (pFI; also termed feature impact on the platform) was calculated through a permutation-based approach by the AutoML platform [33]. In brief, the values of a single feature were permuted, and the resulting loss in LogLoss accuracy was calculated. The permutation and performance loss assessment were repeated multiple times to generate an average accuracy loss. This process was performed on every feature individually. The platform further normalized the pFI values to the maximum pFI value observed; pFI in this study thus refers to the normalized values. We aggregated the pFI values across all 25 projects to calculate the mean pFI and 95% confidence interval for each feature. The features were then arranged from highest to lowest pFI for visualization.

Performance precision analysis

To characterize the relationship between number of aggregated projects and the precision (i.e. variability) of model performance as a result of different partitioning arrangements, we created 150 projects, each with a unique random seed and corresponding partitioning arrangement. We optimized both BP_log and BP_XGB in each project with the 46 features using the AutoML platform. We collected the resulting cross-validation LogLoss and AUC performance values for all 150 projects. We then performed the following sampling analysis:

Randomly sample one project.
Randomly sample another project without replacement. The newly-sampled project and any previously-sampled projects form the current project aggregate.
Calculate the standardized performance CI width with the current project aggregate.
Repeat steps 2–3 until all 150 projects have been aggregated.
Perform steps 1–4 1000 times.
Calculate the expected (i.e. mean) standardized performance CI width with corresponding 95% confidence intervals for each number-of-projects-aggregated.

Standardized performance CI width was calculated with the following formula:

The results were visualized with an emphasis on the 25-project point. The process was repeated for BP_log and BP_XGB as well as for LogLoss and AUC metrics.

Feature instability analysis

To characterize the relationship between number of aggregated projects and feature instability, we used the same 150 projects as in the performance precision analysis. For each instance of BP_log and BP_XGB, we calculated the normalized pFI values. To obtain the pFI ranks, we ordered and ranked the pFI values for each specific instance of the model and project from highest to lowest. As a metric for feature instability, we calculated the feature rank instability (FRI): where p and q represent two different pFI lists, i is the ith feature, and f is the total number of features.

We then performed a sampling analysis as follows:

Randomly sample one project.
Randomly sample another project without replacement. The newly sampled project and any previously sampled projects form the current project aggregate.
Rank the features according to the mean pFI values in the current project aggregate.
Calculate FRI between the pFI lists from the previous project aggregate and the current project aggregate. If only two projects have been aggregated, FRI is calculated between the pFI list from the first sampled project and the pFI list from the current project aggregate.
Repeat steps 2–3 until all 150 projects have been aggregated.
Perform steps 1–5 1000 times.
Calculate the expected (i.e. mean) FRI with corresponding 95% confidence intervals for each number-of-projects-aggregated.

The results were visualized with an emphasis on the 25-project point.

We also performed the sampling analysis on just the ranking of the bottom five features by pFI (i.e. least important features). In this case, the entire feature list was ranked as before, but the FRI was only calculated for the five least important features based on the aggregate with fewer projects (i.e. when comparing 3-project aggregate vs 4-project aggregate, we considered the bottom 5 features from the pFI values of the 3-project aggregate).

To investigate the feature instability during feature reduction for BP_log and BP_XGB, we applied the FRI quantification to compare the feature list before and after each reduction step. Specifically, we calculated FRI for the features that remained after elimination. For example, at feature list size = 41, we calculate the FRI for the 41 features by comparing their rankings between the 46-feature model (before reduction) and the 41-feature model (after reduction).