Fig 1.
PRISMA diagram of search process and selection of eligible records.
Table 1.
Cohort origin statistics.
Table 2.
Summarized characteristics of included studies.
Table 3.
Summarized AUC/C-stat. results of new models in the included studies.
Table 4.
External validations of risk models found in literature.
Fig 2.
Variables were counted as those used in any final developed model in a study. We summarize by studies due to variation in the number of developed models per study. Variables used by only one single study were either merged with similar ones, or grouped as”Other” within its category. Note: Variable information from five studies were excluded as they did not report complete information, meaning variable information from 45 studies developing new models are included here. ‘BMI’: Body Mass Index, ‘BP’: Blood Pressure,’ Chol.’: Cholesterol, ‘HDL’: High-density lipoprotein, ‘LDL’: Low-density lipoprotein, ‘Misc’: Miscellaneous, ‘SNPs’: Single nucleotide polymorphisms.
Fig 3.
ROB assessment using short form PROBAST.
PROBAST assessment summarized per study. In short, domains were: 1) Outcome assessment, 2) EPV, 3) Continuous predictors handling, 4) Missing data management, 5) Univariable selection of predictors, and 6) Correction for overfitting/optimism. See Venema et al. [12] for more details. Domains 3, 5 and 6 were not applicable for external validations. Six studies had remarks that were only valid for some of the reported results, e.g., due to the Events Per Variable (EPV) criteria being less strict for external validations or different methods used on some of the developed models. These were marked with a mixed “High/Low” symbol on relevant domain or overall assessment. ‘Dev.’: Developed models, ‘Ext. val.’: External validations, ‘NA’: Not applicable.
Fig 4.
ROB assessment using long form PROBAST.
PROBAST assessment for studies marked potentially low risk of bias using short form PROBAST. ‘An.’: Analysis, ‘App.’: Applicability, ‘DEV’: Developed models, ‘NI’: No information, ‘Out.’: Outcome, ‘Part.’: Participants, ‘Pred.’: Predictors, ‘ROB’: Risk of bias, ‘VAL’: External validations.
Table 5.
Summarized estimates for AUC/C-stat. and heterogeneity found in meta-analyses.
Table 6.
Study characteristics for model results included in the respective meta-analyses.
Fig 5.
Forest plot of traditional models.
The 95% prediction interval for new models extending from the summary diamond on the bottom line was calculated as (0.660–0.865). ‘*’: The result was obtained from a risk score, or nomogram, developed using that method. ‘CI’: Confidence interval, ‘Reg.’: Regression.
Fig 6.
The 95% prediction interval for new models extending from the summary diamond on the bottom line was calculated as (0.547–0.943). ‘Method 1 + Method 2’: Ensemble of Method 1 and 2. ‘Method 1 into Method 2’: Outputs from Method 1 were used as inputs to Method 2. ‘BN’: Bayes Network, ‘CI’: Confidence interval, ‘DT’: Decision tree, ‘GBM’: Gradient Boosting Machines, ‘KNN’: K-Nearest Neighbor, ‘LSTM NN’: Long Short-Term Memory Neural Net, ‘LWNB’: Locally Weighted Naïve Bayes’, ‘MLP NN’: Multi-Layer Perceptron Neural Net, ‘NB’: Naïve Bayes, ‘Reg.’: Regression, ‘RF’: Random Forest, ‘SVM’: Support Vector Machines, ‘XGBoost’: eXtreme Gradient Boosting.
Fig 7.
Forest plot for external validations of the Framingham risk model.
The 95% prediction interval extending from the summary diamond was estimated as (0.571–0.883). ‘CI’: Confidence interval.