https://orcid.org/0000-0002-0463-6909
Figures
Abstract
Background
Identification of community-dwelling older adults at risk of unplanned hospitalizations is of importance to facilitate preventive interventions. Our objective was to review and appraise the methodological quality and predictive performance of prediction models for predicting unplanned hospitalizations in community-dwelling older adults
Methods and findings
We searched MEDLINE, EMBASE and CINAHL from August 2013 to January 2021. Additionally, we checked references of the identified articles for the inclusion of relevant publications and added studies from two previous reviews that fulfilled the eligibility criteria. We included prospective and retrospective studies with any follow-up period that recruited adults aged 65 and over and developed a prediction model predicting unplanned hospitalizations. We included models with at least one (internal or external) validation cohort. The models had to be intended to be used in a primary care setting. Two authors independently assessed studies for inclusion and undertook data extraction following recommendations of the CHARMS checklist, while quality assessment was performed using the PROBAST tool. A total of 19 studies met the inclusion criteria. Prediction horizon ranged from 4.5 months to 4 years. Most frequently included variables were specific medical diagnoses (n = 11), previous hospital admission (n = 11), age (n = 11), and sex or gender (n = 8). Predictive performance in terms of area under the curve ranged from 0.61 to 0.78. Models developed to predict potentially preventable hospitalizations tended to have better predictive performance than models predicting hospitalizations in general. Overall, risk of bias was high, predominantly in the analysis domain.
Conclusions
Models developed to predict preventable hospitalizations tended to have better predictive performance than models to predict all-cause hospitalizations. There is however substantial room for improvement on the reporting and analysis of studies. We recommend better adherence to the TRIPOD guidelines.
Citation: Klunder JH, Panneman SL, Wallace E, de Vries R, Joling KJ, Maarsingh OR, et al. (2022) Prediction models for the prediction of unplanned hospital admissions in community-dwelling older adults: A systematic review. PLoS ONE 17(9): e0275116. https://doi.org/10.1371/journal.pone.0275116
Editor: Dong Keon Yon, Kyung Hee University School of Medicine, REPUBLIC OF KOREA
Received: December 4, 2021; Accepted: September 11, 2022; Published: September 23, 2022
Copyright: © 2022 Klunder et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Background
In the Netherlands, approximately one in five older adults is admitted to hospital each year [1]. Moreover, hospital admission rates in ED patients aged 65 years and older are twice as high as those in ED patients aged <65 years [2]. When hospitalized, older adults are at high risk of experiencing adverse events such as hospital-associated infections and delirium, causing lengthy hospital stays [3, 4]. In addition, hospitalizations pose a significant risk to the functional ability of older adults, whereas 30% of older patients experiences loss of independence in activities of daily living (ADL) after hospital admission [5].
Older adults account for a large proportion of hospitalized adults, which is likely to increase with the aging population, causing overcrowding of emergency departments (EDs) and hospital wards [6, 7]. Overcrowded EDs have been described as a global health problem having negative effects on patients (e.g. treatment delay), healthcare staff (e.g. stress) and the healthcare system (e.g. increased length of stay in ED as well as in hospital wards) [8]. Taking into account that a large proportion of hospitalizations and ED visits in older adults is considered preventable [9], it seems crucial to timely identify older adults at risk of hospitalization to assess possible preventive measures. This would not only increase patient’s health and quality of life, but also relieve pressure on secondary and tertiary care, resulting in a decrease in overall health care costs [10].
Prediction models can be used to identify community-dwelling older adults at risk for unplanned hospital admissions. By defining and combining important predictors of future emergency care use, preventive interventions can be targeted at high risk individuals [11]. Several prediction models for the prediction of unplanned hospitalizations have been developed and two systematic reviews on this topic have previously been published. However, these reviews included studies in adults of all ages or only included easy to apply case-finding instruments [12, 13]. Furthermore, these reviews were published over seven years ago. In an era of personalized and precision medicine, interest in and the number of prediction models have grown rapidly [14, 15]. Moreover, with the emergence of big data, attention has grown towards different modelling techniques beside traditional regression methods, such as machine learning (ML). Despite guidelines as the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) [16], quality of methodology and reporting of clinical prediction model studies is however often insufficient [17, 18].
We carried out a systematic review of validated prediction models for predicting unplanned hospital admissions in community-dwelling older adults (≥65 years). Our objective was to describe characteristics of the models’ development, the predictors included in the final models, the predictive performance, and to appraise methodological quality of the included models.
Methods
This review is reported according to the Preferred Reported Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement [19]. The study protocol has been registered on the International Prospective Register of Systematic Reviews (PROSPERO, registration number: CRD42020207305).
Search strategy, study selection and data-extraction
We conducted systematic searches in the bibliographic databases PubMed, Embase.com and CINAHL (Ebsco) in January 2021, in collaboration with a medical information specialist. The following terms were used (including synonyms and closely related words) as index terms or free-text words: "Hospital admission", "Patient admission", "Unplanned", "Aged", "Older adults", "Prediction". We applied a validated search filter for finding clinical prediction model studies [20]. The full search strategies are provided in S1 File.
As previously mentioned, two systematic reviews on this topic have been published. Wallace et al. carried out a systematic literature search in February 2014 on risk prediction models to predict emergency admissions in community-dwelling adults [13]. O’Caoimh et al. reviewed short case-finding instruments, published up and until November 2014, for community-dwelling older adults (> 50 years) at risk for multiple adverse outcomes, of which hospitalization was one [12]. To provide a complete overview of available prediction models our search was restricted to August 2013 through January 2021 and we added the models described in the previous reviews that fulfilled the eligibility criteria of this systematic review.
The references of the identified articles were searched for relevant publications. Duplicate articles were excluded.
Studies were included if they met the following criteria:
- Population: community-dwelling older adults, aged 65 years and over
- Intervention: prognostic prediction models derived from retrospective or prospective cohort studies and containing at least one validation cohort
- Comparator: not applicable
- Outcome: one or more unplanned hospitalizations (defined as unplanned overnight stay in hospital). Studies that had admission to the ED as part of their outcome of interest (i.e. combined endpoints) were also included
- Timing: admission to hospital within any time period
- Setting: prediction models intended to be used in primary care
We excluded studies if the prediction models:
- were contingent on an index hospital admission or ED visit (i.e. readmission models)
- studied hospitalizations for specific conditions (e.g. falls or congestive heart failure) as primary outcome
- were intended to be used in the ED
- were developed in specific populations (e.g. patients in palliative care or with psychiatric conditions), with the exception of participants with sensory impairments, because of high prevalence in the older population [21]
Studies that assessed risk factors only and did not build a prediction model, studies that were not developed to specifically predict unplanned hospitalizations, such as models that identify frailty, and studies published in languages other than English, Dutch, German, French, Italian and Spanish were also excluded.
All records were deduplicated in Endnote v9.1, and consequently exported to the Rayyan web app for title and abstract screening and study selection [22]. After study selection, data extraction was performed using a standardized form following the recommendations of the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS; S2 File) [23]. Both selection and data extraction phases were independently conducted by two reviewers (JK and SP). Any disagreements were resolved through a consensus procedure or by third review (OM, KJ, HvH). Additional data were sought from authors, when necessary.
Due to heterogeneity of the prediction models, meta-analysis was not possible. We therefore narratively summarized each unique prediction model on study population, predictors, number of outcomes and predictive performance. For clarity reasons, regression models and machine learning models were presented separately. Predictive performance was assessed as model discrimination using the area under the ROC curve (AUC) with 95% confidence intervals. Higher AUC values indicate better discriminatory ability. An AUC of 0.7–0.8 reflects fair discrimination, whereas a model with AUC ≥ 0.8 represents good discrimination [24].
Methodological quality assessment
The Prediction model Risk of Bias ASsessment Tool (PROBAST; S2 File) was used to assess risk of bias and applicability, of which the latter addresses whether the primary study matches the review question [25]. PROBAST rates study methodology and applicability to the review question as being at “high”, “low” or “unclear” risk of bias based on a predetermined set of questions and scoring guide [26].
In addition, we calculated the number of events per variable (EPV) for each model. The number of EPV is the number of outcome events divided by the number of candidate predictors assessed in the multivariable modelling [27]. Studies with an EPV <10 are generally subject to overfitting, therefore an EPV of >20 is recommended. Prediction models developed using ML techniques often require higher EPVs (often >200) to minimize overfitting [26].
Results
Study selection
The literature searches yielded a total of 16,098 citations (Fig 1). After removing duplicates 8,820 references remained. Additionally, twenty-three articles were identified by checking the reference lists of relevant studies. Full texts were retrieved for 170 studies of which ten met all inclusion criteria. Additionally, a total of nine studies were included from the previously published systematic reviews (Tables 1 and 2).
Description of included studies
Of the 19 studies included, the majority were developed in the United States (n = 10) [28–37] and two in Italy [38, 39]. The other studies were developed in the United Kingdom [40], Ireland [41], Canada [42], Sweden [43], Spain [44], Taiwan [45], and South-Korea [46]. Twelve studies included participants aged ≥65 years [29, 33, 35–39, 41, 42, 44–46], the remaining studies used a higher age as inclusion criterion with 81 years [30] as the highest minimum age for inclusion. Total sample sizes ranged from 150 [34] to 1,095,613 [39] participants. Two studies were developed in patients receiving home or community care [34, 41], and one study developed a prediction model in older adults with a vision and/or hearing impairment [29].
Eight studies developed their model using administrative or electronic medical record data [31, 32, 34, 39, 43–46]. Eight studies used survey data to develop their model [28–30, 33, 36, 38, 40, 41], and three models were developed using both [35, 37, 42].
Various outcomes were assessed in the development of the prediction models. Two studies validated their models for more than one outcome (i.e. unplanned hospitalizations and potentially preventable hospitalizations, separately) [37, 39]. Two models predicted a combined endpoint of any hospitalization or ED visit [34, 36]. Fourteen studies assessed unplanned hospitalizations as single endpoint [29–33, 37–45], two studies predicted multiple hospitalizations within a specific time period [28, 35], and three studies presented a model for potentially preventable hospitalizations [37, 39, 46]. Two out of these three studies defined admissions as potentially preventable based on the principal diagnosis on admission [37, 46]. The third study did not report its definition for preventable admissions [39]. The prediction horizon ranged from 4.5 months [30] to 4 years [28]. The majority of studies (n = 12) were developed to predict the outcome within 12 months [29, 31, 32, 36, 37, 39–41, 43–46].
Variables used in prediction models
The number of predictors included in the final model ranged from 3 [36] to 38 [43]. The variables most frequently included in the final models were previous hospital admission (n = 11) [28, 29, 31, 35, 36, 38, 40, 42–44, 46], age (n = 11) [28, 31–33, 37–39, 43–46] and sex or gender(n = 8) [28, 32, 33, 35, 37, 38, 43, 44] (Table 3). Twelve studies included one or more specific diseases in the final model, of which cardiovascular diseases (e.g. coronary artery disease, heart failure, or hypertension) were most frequently included (n = 11) [28–30, 33, 36, 39, 40, 42–45]. The most frequently included cardiovascular predictor was ischemic heart disease (n = 7) [28, 29, 33, 39, 42–44]. Diabetes was included in seven models [28, 30, 33, 35, 42–45]. Other frequently included medical diagnoses were cancer (n = 4) [29, 33, 43, 45] and COPD or respiratory problems (n = 4) [33, 39, 44, 45]. Six studies included a multimorbidity measure, either defined as the Charlson Comorbidity Index or a disease count, in the final model [31, 32, 36, 37, 44, 46]. Living arrangement (mostly defined as living alone) was considered for inclusion in seven models [28, 29, 33, 35, 36, 40, 42], and was retained in one model [42]. This model defined living arrangement as living with a spouse.
Predictive accuracy of the models
Two studies analyzed predictive performance of the same prediction model for two different outcomes [37, 39]. One study did not report its predictive performance [42].
Eighteen studies reported an AUC, ranging from 0.61 to 0.78 after validation. The models published after 2014 tended to perform better; median AUC was 0.72 (range 0.64–0.78) (n = 9), whereas the median AUC from the models in the previous reviews was 0.67 (range 0.61–0.76) (n = 9). Models developed using survey data had median AUC of 0.67 (range 0.61–0.72) (n = 8), the median AUC of models developed with administrative data was 0.73 (range 0.64–0.78) (n = 8). Studies that used both data sources are not included in this count.
The models developed for a specific type of hospitalization (i.e. preventable hospitalization or fall with hospitalization) (n = 3), tended to perform better than the models for all-cause hospitalization (n = 17), with a median AUC of 0.78 (range 0.74–0.78) versus 0.69 (range 0.61–0.76), respectively. The two models that assessed AUCs for both outcomes (i.e. Tarekegn et al. and Wu et al. [37, 39]) were included in calculations of both medians with its corresponding AUC and were thus counted twice.
Methodological quality
Overall, the methodological quality of included studies was low (Table 4). Risk of bias was either high or unclear in all studies, predominantly due to bias or insufficient reporting in the analysis domain. More specifically, the handling of missing data was not reported or performed inappropriately in ten studies [29, 31, 33, 36, 37, 40–42, 44, 45], eight studies selected predictors based on univariable analyses [30, 31, 33, 35, 40, 42, 43, 45], and five studies solely handled a split-sample procedure for internal validation [28, 30, 31, 33, 45]. Whereas almost all studies (except one [42]) reported model performance in terms of discrimination, only five sufficiently evaluated calibration [28–30, 38, 45]. Four studies only reported results of the Hosmer-Lemeshow test as a single calibration measure [31, 35, 40, 44].
The median EPV was 60 and ranged from 8 [36] to 2003 [45] (n = 15). Two studies reported an EPV <20 [36, 41]. In four studies the EPV could not be computed because data on the number of events or the number of candidate predictors were not reported [30, 33, 34, 42]. The models published after 2014 had a higher EPV (median = 129 (range 27–2003)) than the older models (median = 46 (range 8–64)).
Concern for applicability was high in three studies, because the study population or study outcome did not fully match the review question: one study only included older adults with a sensory impairment [29], one study excluded older adults with a hospital admission <6 months prior to the index date [45], and one study evaluated preventable hospital admissions as only outcome [46].
Discussion
This systematic review identified 19 prediction models to predict unplanned hospital admissions in community-dwelling older adults. With our search strategy we built on a review by Wallace et al. on the same topic, however focusing the study population to adults aged 65 years and over. In total we identified 19 prediction models, of which the current review added 10 new prediction models that were not included in the previous reviews. The new models had higher predictive accuracy than the older models. This might be explained by the fact that new models had larger samples of the development cohort and also higher EPVs than the older models. Both are recommended by the TRIPOD guidelines, published in 2015 [16], to improve predictive accuracy and methodological quality. Moreover, the new models used administrative or clinical record data more often for the development of their model. Consistent with Wallace et al., we found that models developed using administrative or clinical record data had higher predictive accuracy than those developed using self-report data. Of the 10 new prediction models, eight used administrative data for development of their model.
To potentially improve predictive accuracy, Wallace et al. suggested to consider nonmedical factors (e.g. social support and functional status) [13]. Despite this recommendation, these variables were rarely evaluated for inclusion in the latest studies. We found that predictors most frequently included in the final models were medical diagnoses (specifically heart disease), prior hospitalizations, age, and sex, which is in line with Wallace’s findings. These risk factors seem to have more impact in the prediction of unplanned admissions than nonmedical factors, considering the relatively high beta-coefficients of these variables in most models (data not shown). Also, chronic diseases and health care use variables are probably more readily available in large routine care data, whereas nonmedical factors are rarely assessed in a systematic way.
Overall, reporting of methodology and findings was often inappropriate or lacked relevant information, risk of bias was therefore either unclear or high in all models. Moreover, despite the publication of the TRIPOD guidelines in 2015, only one [29] out of seven studies published after 2015 reported their study according to the TRIPOD checklist. The majority of studies showed high risk of bias in the analysis domain. Mainly because of univariable analyses as selection method or inappropriate handling of missing data.
Strengths and limitations
The aging population across the globe and increasing interest in personalized medicine makes this review topical. We added a substantial number of prediction models to the previous systematic reviews on this topic. Furthermore, we conducted a thorough search strategy using a validated search filter and assessed data using tools specifically designed for systematic reviews of prognostic studies. However, there are some limitations. First, care must be taken with directly comparing the prediction models because of heterogeneity in study characteristics (e.g. study populations, and selection of candidate predictors) and study outcomes. Since models perform differently in other populations, comparison of predictive performance can only be performed when these models are validated in the same sample. Further, by limiting our inclusion criteria to participants aged 65 and over, we excluded potential prediction models developed in participants with younger age. For example, the DIVERT scale, a tool to predict emergency department visits, was developed in home care clients aged ≥50 years. Even though reported AUCs are a little over 0.6 after geographical validation, targeted application of the risk score has shown its clinical added value for cardiorespiratory management and reduction of hospitalizations in home care recipients [47]. Last, while in principle CHARMS and PROBAST are relevant for prediction model studies using ML, they predominantly focus on regression-based modelling and some unique aspects of ML methods are not captured [48]. This complicated the critical appraisal of the ML study and therefore risk of bias was unclear. Necessity for guidelines for reporting and critical appraisal of prediction model studies using ML has been addressed and PROBAST-ML (as well as TRIPOD-ML) has been announced [48]. Until then, it is recommended to use TRIPOD, CHARMS and PROBAST as benchmark for the development of prediction model studies rather than none [49].
Implications for future research
Our findings provide a proper basis of prediction models on hospitalizations in older people. Knowing that prediction models often perform worse in new populations, external validation studies are needed to assess generalizability across different countries and healthcare systems. Moreover, models that underperform in external samples should not be discarded and studies should assess the possibility of updating existing models by recalibrating, adjusting weights or considering additional predictors [50, 51]. This way, data of the original development model is not wasted. However, updating of a prediction model is only recommended provided that the initial model was appropriately developed and demonstrated promising accuracy [51]. Most prediction models in this review are poorly reported and all are at either high or unclear risk of bias, which makes updating of the existing models more complicated and we therefore cannot recommend one specific model.
Moreover, while recalibration and adjusting weights only affect a model’s calibration, adding (previously missed) important predictors should be considered to improve a model’s discrimination [51]. As mentioned above, nonmedical factors remain under researched in the prediction of hospital admissions in older adults. Taking into account the influence of nonmedical factors on unscheduled secondary care use [52, 53], these variables may contribute to a better discriminative ability of the model.
Last, for both development studies and validation studies we advise to fully report all modelling steps and analysis in sufficient detail according to the TRIPOD guidelines [16]. The TRIPOD guidelines have been developed to improve the reporting of studies developing, validating, and updating prognostic models and to maximize transparency and reproducibility. More specifically, for example, predictive performance should not only be evaluated in terms of discrimination, but also in terms of calibration. Regarding calibration, it is recommended to include a calibration plot or table in addition to the p-value of the Hosmer-Lemeshow test. Furthermore, variables or participants with missing data should not simply be omitted, multiple imputation is recommended as the preferred method for handling of missing data to decrease bias [26].
Implications for future practice
Our study found that the models to predict preventable hospitalizations tended to have better predictive ability than models for all-cause hospitalizations. Preventable admissions reflect admissions for conditions that could have been managed with timely and effective treatment by outpatient primary care (e.g. pneumonia, congestive heart failure, and COPD, often also referred to as ambulatory care sensitive conditions (ACSCs)) [54]. Interventions targeted at older adults with ACSCs provide a window of opportunity for prevention of admissions. Possibly even more so if targeted at persons with additional important risk factors (e.g. recent hospitalization, polypharmacy and/or multimorbidity). In consequence, reduction of the incidence of preventable admissions could substantially lower healthcare costs, and improve health outcomes and older adult’s quality of life [11].
There is however limited evidence for effective preventive interventions to reduce preventable admissions in general [54]. High continuity of care with a general practitioner is associated with lower rates of hospital admissions [55]. Furthermore, several targeted interventions have shown to be effective in patients with specific diseases, such as self-management in patients with COPD and heart failure, and telemedicine in patients with heart failure [11]. Focusing on these targeted interventions may have a beneficial impact on the reduction of hospital admissions in community-dwelling older adults [54].
Conclusion
The prediction models developed to predict preventable hospitalizations tended to perform better than models predicting all-cause hospitalizations. Focusing on enhancing primary care management of conditions related to these preventable admissions may have a beneficial effect on health care quality.
To improve predictive accuracy of prediction models the use of administrative data sources is recommended as well as incorporation of important variables, i.e. age, prior hospitalization and multimorbidity. The impact of nonmedical factors remains unresearched. Moreover, future researchers are recommended to follow the TRIPOD guidelines for prediction model studies, as methodological quality of reporting and analyses of the included studies was low.
Acknowledgments
The authors thank Stichting Preventie, Vroegdiagnostiek en eHealth for their support.
References
- 1. Netherlands CBfS. Health and Healthcare utilization 2021 [cited 2022 April 30]. Available from: https://opendata.cbs.nl/#/CBS/nl/dataset/83005NED/table?dl=672AF.
- 2. Berben S, Bloemhoff A, Habets K, Liefers J, Hensens C, van Grunsven P, et al. [Care contacts of elderly patients in the emergency care pathway: a retrospective cohort study]. Ned Tijdschr Geneeskd. 2019;163:D3523. Epub 2019/07/31.
- 3. Long SJ, Brown KF, Ames D, Vincent C. What is known about adverse events in older medical hospital inpatients? A systematic review of the literature. Int J Qual Health Care. 2013;25(5):542–54. Epub 2013/08/09. pmid:23925507
- 4. Hoogervorst-Schilp J, Langelaan M, Spreeuwenberg P, de Bruijne MC, Wagner C. Excess length of stay and economic consequences of adverse events in Dutch hospital patients. BMC Health Serv Res. 2015;15:531. Epub 2015/12/03. pmid:26626729
- 5. Loyd C, Markland A, Zhang Y, Fowler M, Harper S, Wright N, et al. Prevalence of Hospital-Associated Disability in Older Adults: A Meta-analysis. J Am Med Dir Assoc. 2020;21(4):455–61.e5. Epub 2019/11/18. pmid:31734122
- 6. Legramante JM, Morciano L, Lucaroni F, Gilardi F, Caredda E, Pesaresi A, et al. Frequent Use of Emergency Departments by the Elderly Population When Continuing Care Is Not Well Established. PloS one. 2016;11(12):e0165939. Epub 2016/12/16. pmid:27973563; PubMed Central PMCID: PMC5156362.
- 7. Šteinmiller J, Routasalo P, Suominen T. Older people in the emergency department: a literature review. Int J Older People Nurs. 2015;10(4):284–305. Epub 2015/07/18. pmid:26183883
- 8. Morley C, Unwin M, Peterson G, Stankovich J, Kinsman L. Emergency department crowding: A systematic review of causes, consequences and solutions. PloS one. 2018;13(8):e0203316. Epub 2018/08/31. pmid:30161242
- 9. Kozak L, Hall M, Owings M. Trends in avoidable hospitalizations, 1980–1998. Health Aff (Millwood). 2001;20(2):225–32. Epub 2001/03/23. pmid:11260947
- 10. Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. The Milbank quarterly. 2005;83(3):457–502. Epub 2005/10/06. pmid:16202000
- 11. Purdy S, Paranjothy S, Huntley A, Thomas R, Mann M, Huws D, et al. Interventions to reduce unplanned hospital admission: a series of systematic reviews. Final Report. 2012.
- 12. O’Caoimh R, Cornally N, Weathers E, O’Sullivan R, Fitzgerald C, Orfila F, et al. Risk prediction in the community: A systematic review of case-finding instruments that predict adverse healthcare outcomes in community-dwelling older adults. Maturitas. 2015;82(1):3–21. Epub 2015/04/14. pmid:25866212
- 13. Wallace E, Stuart E, Vaughan N, Bennett K, Fahey T, Smith S. Risk prediction models to predict emergency hospital admission in community-dwelling adults: a systematic review. Medical care. 2014;52(8):751–65. Epub 2014/07/16. pmid:25023919
- 14. Hemingway H. Prognosis research: why is Dr. Lydgate still waiting? Journal of clinical epidemiology. 2006;59(12):1229–38. Epub 2006/11/14. pmid:17098565
- 15. Riley R, Ridley G, Williams K, Altman D, Hayden J, de Vet H. Prognosis research: toward evidence-based results and a Cochrane methods group. Journal of clinical epidemiology. 2007;60(8):863–5. Epub 2007/07/04. pmid:17606185
- 16. Moons K, Altman D, Reitsma J, Ioannidis J, Macaskill P, Steyerberg E, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Annals of internal medicine. 2015;162(1):W1–73. Epub 2015/01/07. pmid:25560730
- 17. Bouwmeester W, Zuithoff N, Mallett S, Geerlings M, Vergouwe Y, Steyerberg E, et al. Reporting and Methods in Clinical Prediction Research: A Systematic Review. PLoS medicine. 2012;9(5):e1001221. pmid:22629234
- 18. Heus P, Damen J, Pajouheshnia R, Scholten R, Reitsma J, Collins G, et al. Poor reporting of multivariable prediction model studies: towards a targeted implementation strategy of the TRIPOD statement. BMC Medicine. 2018;16(1):120. pmid:30021577
- 19. Lee SW, Koo MJ. PRISMA 2020 statement and guidelines for systematic review and meta-analysis articles, and their underlying mathematics: Life Cycle Committee Recommendations. Life Cycle. 2022;2:e9.
- 20. Geersing G, Bouwmeester W, Zuithoff P, Spijker R, Leeflang M, Moons K. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PloS one. 2012;7(2):e32844. Epub 2012/03/07. pmid:22393453
- 21. Swenor B, Ramulu P, Willis J, Friedman D, Lin F. The prevalence of concurrent hearing and vision impairment in the United States. JAMA internal medicine. 2013;173(4):312–3. Epub 2013/01/23. pmid:23338042
- 22. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Systematic reviews. 2016;5(1):210. Epub 2016/12/07. pmid:27919275
- 23. Moons K, de Groot J, Bouwmeester W, Vergouwe Y, Mallett S, Altman D, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS medicine. 2014;11(10):e1001744. Epub 2014/10/15. pmid:25314315
- 24.
Iezzoni L. Risk adjustment for measuring health care outcomes: Health Administration Press Chicago; 2003.
- 25. Wolff R, Moons K, Riley R, Whiting P, Westwood M, Collins G, et al. PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Annals of internal medicine. 2019;170(1):51–8. Epub 2019/01/01. pmid:30596875
- 26. Moons K, Wolff R, Riley R, Whiting P, Westwood M, Collins G, et al. PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration. Annals of internal medicine. 2019;170(1):W1–w33. Epub 2019/01/01. pmid:30596876
- 27. Austin P, Steyerberg E. Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Statistical methods in medical research. 2017;26(2):796–808. Epub 2014/11/21. pmid:25411322
- 28. Boult C, Dowd B, McCaffrey D, Boult L, Hernandez R, Krulewitch H. Screening elders for risk of hospital admission. Journal of the American Geriatrics Society. 1993;41(8):811–7. Epub 1993/08/01. pmid:8340558
- 29. Deardorff W, Sloane R, Pavon J, Hastings S, Whitson H. Hospitalization Risk Among Older Adults with Sensory Impairments: Development of a Prognostic Model. Journal of the American Geriatrics Society. 2020;68(11):2650–5. pmid:32852787
- 30. Freedman J, Beck A, Robertson B, Calonge B, Gade G. Using a mailed survey to predict hospital admission among patients older than 80. Journal of the American Geriatrics Society. 1996;44(6):689–92. Epub 1996/06/01. pmid:8642161
- 31. Inouye SK, Zhang Y, Jones RN, Shi P, Cupples LA, Calderon HN, et al. Risk factors for hospitalization among community-dwelling primary care older patients: development and validation of a predictive model. Medical care. 2008;46(7):726–31. Epub 2008/06/27. pmid:18580392
- 32. Kan H, Kharrazi H, Leff B, Boyd C, Davison A, Chang H, et al. Defining and Assessing Geriatric Risk Factors and Associated Health Care Utilization Among Older Adults Using Claims and Electronic Health Records. Medical care. 2018;56(3):233–9. pmid:29438193
- 33. Kurichi J, Bogner H, Streim J, Xie D, Kwong P, Saliba D, et al. Predicting 3-year mortality and admission to acute-care hospitals, skilled nursing facilities, and long-term care facilities in Medicare beneficiaries. Arch Gerontol Geriatr. 2017;73:248–56. pmid:28863353
- 34. Mishra A, Skubic M, Popescu M, Lane K, Rantz M, Despins L, et al. Tracking personalized functional health in older adults using geriatric assessments. BMC Med Inform Decis Mak. 2020;20(1):270. pmid:33081769
- 35. Reuben D, Keeler E, Seeman T, Sewall A, Hirsch S, Guralnik J. Development of a method to identify seniors at high risk for high hospital utilization. Medical care. 2002;40(9):782–93. Epub 2002/09/10. pmid:12218769
- 36. Shelton P, Sager MA, Schraeder C. The community assessment risk screen (CARS): identifying elderly persons at risk for hospitalization or emergency department visit. Am J Manag Care. 2000;6(8):925–33. Epub 2001/02/24. pmid:11186504
- 37. Wu S, Mulcahy J, Kasper J, Kan H, Weiner J. Comparing Survey-Based Frailty Assessment to Medicare Claims in Predicting Health Outcomes and Utilization in Medicare Beneficiaries. J Aging Health. 2020;32(7):764–77. pmid:31148491
- 38. Mazzaglia G, Roti L, Corsini G, Colombini A, Maciocco G, Marchionni N, et al. Screening of older community-dwelling people at risk for death and hospitalization: the Assistenza Socio-Sanitaria in Italia project. Journal of the American Geriatrics Society. 2007;55(12):1955–60. Epub 2007/10/20. pmid:17944891
- 39. Tarekegn A, Ricceri F, Costa G, Ferracin E, Giacobini M. Predictive Modeling for Frailty Conditions in Elderly People: Machine Learning Approaches. JMIR Med Inform. 2020;8(6):e16678. pmid:32442149
- 40. Lyon D, Lancaster GA, Taylor S, Dowrick C, Chellaswamy H. Predicting the likelihood of emergency admission to hospital of older people: development and validation of the Emergency Admission Risk Likelihood Index (EARLI). Fam Pract. 2007;24(2):158–67. Epub 2007/01/11. pmid:17210987
- 41. O’Caoimh R, Gao Y, Svendrovski A, Healy E, O’Connell E, O’Keeffe G, et al. The Risk Instrument for Screening in the Community (RISC): a new instrument for predicting risk of adverse outcomes in community dwelling older adults. BMC Geriatr. 2015;15:92. pmid:26224138
- 42. Roos N, Roos L, Mossey J, Havens B. Using administrative data to predict important health outcomes. Entry to hospital, nursing home, and death. Medical care. 1988;26(3):221–39. Epub 1988/03/01. pmid:3352323
- 43. Marcusson J, Nord M, Dong H, Lyth J. Clinically useful prediction of hospital admissions in an older population. BMC Geriatr. 2020;20(1):95. pmid:32143637
- 44. López-Aguilà S, Contel JC, Farré J, Campuzano JL, Rajmil L. Predictive model for emergency hospital admission and 6-month readmission. Am J Manag Care. 2011;17(9):e348–57. Epub 2011/09/10. pmid:21902442
- 45. Lin K, Chen P, Huang L, Mao H, Chan D. Predicting Inpatient Readmission and Outpatient Admission in Elderly: A Population-Based Cohort Study. Medicine (Baltimore). 2016;95(16):e3484. pmid:27100455
- 46. Kim H, Jung Y, Kim K, Park J. Prospectively identifying older adults at risk for potentially avoidable hospitalizations in Korea using population-based data. Int J Qual Health Care. 2019;31(8):620–6. pmid:30462246
- 47. Costa A, Schumacher C, Jones A, Dash D, Campbell G, Junek M, et al. DIVERT-Collaboration Action Research and Evaluation (CARE) Trial Protocol: a multiprovincial pragmatic cluster randomised trial of cardiorespiratory management in home care. BMJ open. 2019;9(12):e030301. Epub 2019/12/18. pmid:31843821
- 48. Andaur Navarro C, Damen J, Takada T, Nijman S, Dhiman P, Ma J, et al. Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques. BMJ open. 2020;10(11):e038832. Epub 2020/11/13. pmid:33177137
- 49. Andaur Navarro C, Damen J, Takada T, Nijman S, Dhiman P, Ma J, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ. 2021;375:n2281. pmid:34670780
- 50. Janssen K, Moons K, Kalkman C, Grobbee D, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. Journal of clinical epidemiology. 2008;61(1):76–86. Epub 2007/12/18. pmid:18083464
- 51. Janssen K, Vergouwe Y, Kalkman C, Grobbee D, Moons K. A simple method to adjust clinical prediction models to local circumstances. Can J Anaesth. 2009;56(3):194–201. Epub 2009/02/28. pmid:19247740
- 52. Huntley A, Lasserson D, Wye L, Morris R, Checkland K, England H, et al. Which features of primary care affect unscheduled secondary care use? A systematic review. BMJ open. 2014;4(5):e004746. Epub 2014/05/27. pmid:24860000
- 53. Longman J, Johnston J, Ewald D, Gilliland A, Burke M, Mutonga T, et al. What could prevent chronic condition admissions assessed as preventable in rural and metropolitan contexts? An analysis of clinicians’ perspectives from the DaPPHne study. PloS one. 2021;16(1):e0244313. Epub 2021/01/08. pmid:33411717
- 54. Purdy S, Huntley A. Predicting and preventing avoidable hospital admissions: a review. The journal of the Royal College of Physicians of Edinburgh. 2013;43(4):340–4. Epub 2013/12/19. pmid:24350320
- 55. Barker I, Steventon A, Deeny S. Association between continuity of care in general practice and hospital admissions for ambulatory care sensitive conditions: cross sectional study of routinely collected, person level data. BMJ. 2017;356:j84. Epub 2017/02/06. pmid:28148478