https://orcid.org/0000-0002-4118-6038
Figures
Abstract
Background
Although hip fractures are commonly associated with functional decline, increased morbidity, and mortality, accurate models for both short- and long-term prediction that do not rely on in-person assessment remain lacking. The aim was to develop and validate a high-performing clinical decision support tool, that can be used for population screening without the need for patient assessment, for predicting hip fracture risk.
Methods and findings
All individuals aged ≥50 years living in Sweden at baseline (randomly set between 2011 and 2013, N = 3,542,647), who had not been prescribed osteoporosis medication within the previous 2 years, were included and followed through the end of 2021. During follow-up, 142,327 individuals sustained a hip fracture. Using a broad unconditional approach, 139,980 variables encompassing diagnoses, medications, procedures, demographics and socioeconomic data with multiple historic windows and level of detail were defined. The dataset was divided into discovery (25%), development (65%) and holdout (10%) cohorts. The risk of hip fracture was evaluated using traditional Cox models, as well as machine learning methods XGBoost and DeepSurv. The developed clinical support tool FRACTURE-ML based on DeepSurv using 2,500 predictors yielded an area under the curve (AUC) of 0.89 (95% CI 0.88, 0.89) at year 1, 0.88 (95% CI 0.87,0.88) at two years, and a reduced model with 35 predictors yielded similar AUCs, 0.87 at 2 years and 0.85 at 5 years. Traditional Cox models with 35 and 400 predictors reached similar AUCs. Both the DeepSurv and the Cox models performed excellently at the individual level based on calibration plot analysis. Screening using the in Sweden advocated fracture liaison services (FLS) secondary prevention approach (recent fracture), resulted in an AUC of 0.55 (95% CI 0.54, 0.55) at two years. For 2-year prediction, FRACTURE-ML, which could be used as a complementary approach for primary prevention, identified nearly seven times more persons at risk (sensitivity 0.84 (95% CI 0.82, 0.85) versus 0.12 (95% CI 0.11, 0.13) than the FLS approach, with limited reduction in specificity (0.79 (95% CI 0.79, 0.79) versus 0.98 (95% CI 0.98, 0.98), respectively). The lack of external validation and implementation studies represents a limitation, as such studies are needed to establish the clinical usefulness of FRACTURE-ML.
Author summary
Why was this study done?
- Hip fractures are common in older adults and often lead to serious outcomes such as disability, illness, and increased risk of death.
- Existing tools to predict fracture risk usually require patient-provided information, making large-scale screening difficult.
- There is a need for an accurate and simple way to identify people at high risk without requiring direct patient assessment.
What did the researchers do and find?
- The model identified about seven times more high-risk individuals than current clinical practice, while maintaining relatively high precision.
- They developed a prediction model (FRACTURE-ML) using routinely collected health data, which showed high discriminative performance (AUC of up to 0.89) and excellent calibration.
- The researchers analyzed data from over 3.5 million individuals aged 50 years and older in Sweden and followed them for up to 10 years.
What do these findings mean?
- The lack of external validation and implementation studies represents a limitation, as such studies are needed to establish the clinical usefulness of FRACTURE-ML.
- This approach could help target preventive measures more efficiently and potentially reduce the number of hip fractures.
- The findings show that it is possible to predict hip fracture risk at the population level without direct patient interaction.
Citation: Axelsson KF, Litsne H, Konstantinou K, Khalid H, Pivodic A, Lorentzon M (2026) A clinical decision support tool for accurate hip fracture prediction: A nationwide cohort study. PLoS Med 23(8): e1005190. https://doi.org/10.1371/journal.pmed.1005190
Academic Editor: Jonathan P. Evans, University of Exeter, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: December 5, 2025; Accepted: July 14, 2026; Published: August 27, 2026
Copyright: © 2026 Axelsson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be made publicly available for ethical and legal reasons. Such information is subject to legal restrictions according to national legislation. Specifically, in Sweden confidentiality regarding personal information in studies is regulated in the Public Access to Information and Secrecy Act (SFS 2009:400). The data underlying the results of this study might be made available upon request, after an assessment of confidentiality. There is thus a possibility to apply to get access to certain public documents that an authority holds. In this case, the University of Gothenburg is the specific authority that is responsible for the integrity of the documents with research data. Questions regarding such issues can be directed to the head of the Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden. Contact information can be obtained from medicin@gu.se. Code used to generate results in this analysis can be found at https://github.com/RajaHussnain/FRACTURE-ML and https://doi.org/10.5281/zenodo.21070637.
Funding: The study was funded by the Swedish Research Council (Dnr 2023-01976 to ML), the Sahlgrenska University Hospital (ALFGBG-997803/1006873 to KFA; ALFGBG-1006860 to ML), the Gothenburg Society of Medicine (GLS-999015/1022125 to KA), King Gustav V:s and Queen Victoria’s foundation (2023-2024 to ML), the Swedish Society of Medicine (SLS-985867 to KA) and the Skaraborg Research Institute (23-1058 to KA). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: ML has received lecture or consulting fees from Astellas, Amgen, UCB Pharma, Medison Pharma, Sandoz, Gedeon Richter, Jansen-Cilag, Medac, Pharmacosmos, Parexel International, and Crinetics, all outside the submitted work. KK, HK and AP received consultancy fees for performing analyses. KFA and HL have no competing interests.
Abbreviations: AUC, area under the curve; BMD, bone mineral density; FLSs, Fracture Liaison Services; FRACTURE-ML, Fracture Risk Assessment and Classification Using Real-world Evidence and Machine Learning; NAISS, National Academic Infrastructure; NPV, negative predictive values; PPV,, Positive Predictive Value;; ROC, receiver-operating characteristic curves; TRIPOD+AI, Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis using machine learning or Artificial Intelligence; UPPMAX, Uppsala Multidisciplinary Centre for Advanced Computational Science
Introduction
Hip fractures are associated with significant morbidity, disability, mortality and substantial healthcare costs [1–4]. Therefore, it is of utmost importance to identify individuals at high risk of hip fracture in order to initiate effective preventive measures, such as treatment with osteoporosis medications and fall prevention [5–8]. As recently reviewed in JAMA, the US Preventive Services Task Force concluded that screening for osteoporosis to prevent osteoporotic fractures in postmenopausal women has moderate net benefit [9].
Several risk tools to evaluate the risk of hip fracture are available and include FRAX, QFracture and CFracture, but these algorithms are dependent on patient interaction due to their implementation of body mass index (BMI) and lifestyle information on smoking and alcohol in the algorithms [9–12]. A risk model sourced on Danish registry data and developed using traditional statistical methods had high accuracy, but did not consider competing risk of death [13,14]. Accurate risk algorithms based on electronic health records alone would allow cost-efficient population screening and an opportunity for prevention without a large impact on the healthcare system.
Patients sustaining a first fracture have an increased risk of recurrent fracture, especially during the first 2 years following the index fracture [15–17]. Therefore, structured secondary prevention programs known as Fracture Liaison Services (FLSs) targeting patients with recent fractures have been recommended and launched worldwide [18,19], resulting in higher rates of bone mineral density (BMD) testing, treatment initiation, improved medication adherence, and a decrease in the risk of recurrent fracture [20,21]. The Swedish National Board of Health and Welfare has identified the establishment of FLSs as a top national priority [22]. Although implementation is strongly advocated in most Western countries, and known to be important for those with osteoporotic fracture, general primary screening for high fracture risk is not.
We hypothesized that machine learning algorithms and traditional statistical models applied to national electronic health record data from more than 3.5 million individuals aged 50 years or older could be used to develop and validate Fracture Risk Assessment and Classification Using Real-world Evidence and Machine Learning (FRACTURE-ML), a clinical decision support tool capable of accurately predicting both short- and long-term hip fracture risk.
Methods
Study design
This nationwide cohort study used multiple national registers in Sweden to develop a clinical decision support tool for hip fracture. A dataset of all Swedish men and women born 1981 or earlier and alive in 2005 was used. All individuals were assigned a random baseline date between 2011 and 2013. Only those aged 50 years or older and alive at baseline were included in the study population. Furthermore, individuals with recent (≤2 years) osteoporosis medication were excluded, rendering a population size of 3,542,647 individuals. A broad and unconditional approach was used to identify and create 139,980 baseline variables and possible predictors (Fig 1 and extended methods in S1 Appendix). Incident hip fracture was defined as fractures of the femoral head, neck, trochanter or subtrochanteric part of the femur accompanied with a code for surgical procedure and censored for death, emigration and end of study (December 31st, 2021). The whole dataset was divided into three separate cohorts for (i) discovery (25%), (ii) development (65%) and (iii) holdout (10%). The risk of hip fracture was evaluated yearly up to 10 years using three different methods: Cox regression, deep learning generalization of Cox regression (DeepSurv) with and without considering competing risks, and XGBoost for survival analysis. Ethical approval was obtained from the Swedish Ethical Review Authority (2022-00796-01). The requirement for informed consent was waived due to the use of anonymized participant data. This study is reported as per the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis using machine learning or Artificial Intelligence (TRIPOD+AI) guideline (S1 Checklist).
Brief overview including data sources, variables, method workflow and follow-up. Hip fx acc (n) denotes the number of hip fractures accumulated after 2, 5 and 10 years follow-up. *DeepSurv Comp is a DeepSurv model not censored for death corresponding to a competing risk model.
Data sources
Information regarding fractures and comorbidities was retrieved from The National Patient Register which includes hospital-based diagnoses from both inpatient and outpatient visits and uses ICD-10 since 1997. Socioeconomic data were retrieved from the LISA Register (Statistics Sweden), and date of death from Swedish Cause of Death Register (The National Board of Health and Welfare). Medication data was collected from The Swedish Prescribed Drug Register, starting July 1st, 2005. In Sweden, every resident receives a personal ID number at birth or immigration, allowing linkage across registries.
Data partitioning and sample size justification
The total cohort was divided into discovery, development, and holdout sets which is essential for building reliable and generalizable models. The discovery set was used to explore patterns and to generate hypotheses, while the development set allowed for model training and tuning without biasing results. The holdout set, kept separate, provided an unbiased evaluation of how the model performed on new, unseen data. This separation helped to prevent overfitting, ensured that the findings were not artifacts of a specific dataset, and increased the confidence that the results will replicate in real-world applications. Sample size calculations for the development and holdout cohorts were performed using the three criteria of Riley and colleagues [23,24]. As a result, the development cohort consisted of 2,302,022 participants (65% of study population), the holdout cohort of 354,261 participants (10% of study population) and the discovery cohort of 886,362 participants (25% of study population) (for extended methods, see S1 Appendix).
The discovery phase
Starting with 139,980 variables, age and sex adjusted univariable Cox models were fitted to eliminate insignificant predictors. The Benjamini–Hochberg [25] stepdown procedure was used as a multiple testing adjustment and variables with large standard errors were filtered out. Remaining candidate variables were evaluated using both multivariable Lasso Cox, and XGBoost in clinically adjacent groups, and gradually merged with other significant variables (S1 Fig). The 2500 most important variables were selected for model development.
The model development phase
Subsampling techniques were used to deal with data imbalance and optimize the predictive performance of the models (DeepSurv, XGBoost, Cox). The tradeoff between predictive performance and model complexity was also investigated. The models were adequately tuned. The following models were evaluated: DeepSurv (2500 and 35 variables), DeepSurv competing (35 variables) [26], XGBoost (2500 and 35 variables) and Cox (400 and 35 variables). To avoid overfitting of the model, the loss from the internal validation set, a dataset from the development cohort used for model hyperparameter optimization was tracked, i.e., the partial likelihood, for each epoch, and stopped training when the validation loss started to increase (S2 Fig). To correct for an observed consistent overestimation of risk, the predicted probabilities from the DeepSurv and Cox models were recalibrated changing the intercept and slope of the calibration curves to minimize the sum of the squared errors (S3 Fig). The XGBoost model did not allow for individual probability calculations and could therefore not be calibrated.
Validation in the holdout cohort
For prediction, the hip fracture probabilities from the discovery cohort were used to select Youden cutoffs. The models were evaluated for different time periods in the holdout cohort, and results are presented with receiver-operating characteristic curves (ROC), area under the curve (AUC), sensitivity, specificity, positive (PPV) and negative predictive values (NPV), calibration plots, and confusion matrix parameters. Finally, the predictive power of risk factors was investigated using permutation importance with change in AUC as statistic, i.e., variable importance was defined as how much AUC would have changed if the variable had no predictive value.
Clinical implications
A comparison of the best models to FLS represented by any fracture the past year, was made. Furthermore, to quantify the clinical usefulness of the developed models, net benefit was calculated for different thresholds and models, and then compared to screen all, screen none and screening using FLS. To further demonstrate the clinical implications, hip fracture risk within 2 years for selected health profiles and age groups, as predicted by the model for women and men were presented. Two scenarios—one prioritizing high sensitivity (80%) and the other high specificity (80%)—were presented to illustrate effects of algorithm implementation.
Statistical analyses
Descriptive baseline statistics for the most common diagnoses, medications, medical and surgical procedures, socioeconomical and previous fracture history variables are presented in terms of counts with percentage for categorical variables and averages with standard deviations (SD) for continuous variables. Statistical analyses were performed using Python version 3.11 using libraries such as Pycox, Scikit-Learn, and XGBoost, and R software version 4.4 for sample size calculation and some visualizations. Computations were performed using distributed data processing and distributed training across multiple compute nodes in a high-performance environment (Bianca) hosted by Uppsala Multidisciplinary Centre for Advanced Computational Science (UPPMAX) and National Academic Infrastructure (NAISS) for sensitive personal data (Extended methods in S1 Appendix). No large language models have been used for the creation of this manuscript.
Results
Study population
A total of 3,542,647 persons were included in the study. During a median (interquartile range) follow-up of 9.04 (8.16, 9.98) years there were a total of 142,327 hip fractures and 754,051 (21.3%) deaths. Patients with incident hip fractures were generally older, more frequently women, had more prescribed medications, and greater comorbidity than those without (Tables 1 and S1). The discovery, development and holdout cohorts were well balanced in terms of baseline characteristics (S2 Table).
Model performance
All tested models were able to efficiently discriminate persons with and without incident hip fracture. The full DeepSurv model using 2500 variables resulted in an AUC of 0.89 (95% CI 0.88, 0.89) at year 1 (Fig 2A), which gradually declined with longer follow-up time to 0.88 (95% CI 0.87, 0.88) at year 2, 0.85 (95% CI 0.85, 0.86) at year 5 and 0.82 (95% CI 0.81, 0.82) at year 10. The sensitivity and specificity at year 1 was 0.83 (95% CI 0.81, 0.85) and 0.81 (95% CI 0.81, 0.81), respectively, and remained high year 2, 0.84 (95% CI 0.82, 0.85) and 0.79 (95% CI 0.79, 0.79), year 5 0.83 (95% CI 0.83, 0.84) and 0.75 (95% CI 0.75, 0.75)), and year 10 0.81 (95% CI 0.81, 0.81) and 0.71 (95% CI 0.71, 0.71) (S3 Table), respectively. The DeepSurv model with 35 variables performed similarly well with slightly lower 1-year AUC of 0.88 (95% CI 0.87, 0.89) (Fig 2B).
Deep Learning model (DeepSurv) performance in holdout cohort (n = 354,261) for (A) the full model using 2,500 variables and (B) a reduced model with 35 variables. Key confusion matrix parameters and ROC curves presented graphically for different follow-up periods. The most important variables in the DeepSurv 35 prediction model are presented with variable importance (C), representing change in AUC when randomly permutated, i.e., how much AUC would have changed if the variable had no predictive value. Red dot denotes negative association. See S1 Appendix Extended Methods for variable details and characteristics.
The XGBoost model performed similarly well and yielded an AUC of 0.89 (95% CI 0.88, 0.89) using the 2500 variable model at year 1 and resulted in slightly lower AUC values in the reduced model containing 35 variables. The Cox regression prediction results were nearly equivalent reaching an AUC of 0.87 (95% CI 0.87, 0.88) at year 1. Performance metrics for all models are presented in detail in S3 Table.
Model parameters
In the reduced DeepSurv model with 35 variables, the key variables listed in the order of importance were age, sex, number of prescriptions the past year, marital status and alcohol related psychiatric disorder, with all 35 parameters listed in Fig 2C and variable characteristics provided in S4 Table.
Calibration
Calibration plots for DeepSurv 2500, DeepSurv 35, Cox 35 and Cox 400 were used to compare the predicted probabilities of hip fracture at 2, 5 and 10 years against the observed events, to determine model performance on an individual level. Both Cox models (35 and 400) and the DeepSurv (2500 and 35) models generated predicted probabilities highly aligned with observed probabilities regardless of risk level, especially 2 years using the DeepSurv 35 model (Figs 3A–3C and S4).
Calibration plots for DeepSurv 35 were used to compare the predicted probabilities of hip fracture at 2 (A), 5 (B) and 10 (C) years with the observed events in the holdout cohort (N = 354,261). Thresholds for high-sensitivity (80%) and high-specificity (80%) are marked with red and green, respectively. For calibration plots with other methods, number of variables and follow-up time, see S4 Fig. Net benefit plots for hip fracture probabilities at 2 (D), 5 (E), and 10 (F) years for DeepSurv 35, DeepSurv 2,500, Cox 35, Cox 400 and FLS. Net benefit was calculated as (TP − FP × PB/(1 − PB))/N where TP = true positives, FP = false positives, PB = threshold probability, N = observations using the holdout cohort (N = 351,261). Default strategies screen all and screen none are represented as dark gray-dotted lines. Threshold probabilities representing 80% sensitivity (red) and 80% specificity (blue) are marked for DeepSurv 35.
Mortality and competing risk
In a DeepSurv model with the 35 most important variables not censoring for death to assess the impact of competing risk of death, the model still performed well with AUC after 1, 2, 5 and 10 years of 0.88 (95% CI 0.87, 0.89); 0.87 (95% CI 0.87, 0.88); 0.85 (95% CI 0.85, 0.85) and 0.81 (95% CI 0.81, 0.81), respectively. All confusion matrices and performance parameters are presented in S3 Table.
Comparison with the fracture liaison services approach
To assess the viability of the DeepSurv35 model compared to the FLS approach (based on the presence of a fracture in the past year and frequently used in clinical practice), the hip fracture probability at 2, 5 and 10 years were compared (S5 Fig). The FLS model resulted in an AUC of 0.55 (95% CI 0.54, 0.55) at two years (S3 Table). In comparison, the DeepSurv35 model at 2 years identified nearly 7 times as many patients at risk (sensitivity 0.83 (95% CI 0.82, 0.84) versus 0.12 (95% CI 0.11, 0.13) respectively, with both models having high specificity (0.78 (95% CI 0.78, 0.78) vs. 0.98 (95% CI 0.98, 0.98)) (S5 Fig and S3 Table).
Decision curve analyses
The net benefit, i.e., the trade-off between true positives and false positives, for different threshold probabilities to show that the model is better than screening all or none, is presented for DeepSurv 2500, DeepSurv 35, Cox 400, Cox 35 and compared to the FLS approach. All models showed added net benefit compared to FLS and screen all/none within the relevant threshold intervals. (Fig 3D–3F).
Clinical implications
To illustrate clinical implications and predicted probabilities of the model for selected health profiles, ranging from a healthy person to those with multiple risk factors, the probabilities for hip fracture risk within 2 years are presented for men and women per age groups (Fig 4A and 4B). Probability thresholds dependent on either high (80%) sensitivity or high (80%) specificity were used. For example, using an 80% sensitivity probability threshold on 739,352 women 65−79 years old (the total population), 344,201 would be classified as high risk, of which 4,513 would suffer a hip fracture (true positives), whilst 1,118 patients would sustain a hip fracture but have probabilities below the threshold (false negatives). In comparison, using an 80% specificity threshold probability, would identify 126,820 women of which 2,870 true positives and 2,761 false negatives (Fig 4). An alternative illustration of clinical implication using precision-recall curves is presented with predictions for 2, 5 and 10 years (S6 Fig). Two cases are exemplified (S5 Table), one with 80% recall (sensitivity), and one with maximum precision (positive predicted value). For instance, for the five-year prediction with actual hip fracture incidence of 2.3%, at an 80% sensitivity, the models yielded precisions ranging between 6.7% and 7.9% (S5 Table). Alternatively, maximal precisions for the models ranged from 11.7% to 12.3% at which the corresponding sensitivities ranged from 25.0% to 38.5% (S5 Table).
Hip fracture risk within 2 years for selected health profiles and age groups, as predicted by the model (DeepSurv 35) for women (A) and men (B). Visualization of thresholds at 80% sensitivity and 80% specificity for women (C) and men (D) with corresponding numbers identified.
Discussion
In this nationwide study of 3,542,647 adults in Sweden aged 50 years and older, an unconditional approach using 139,980 variables was applied to develop and validate, in an untouched holdout cohort, a deep-learning hip fracture prediction model based entirely on national registry data without patient assessment. The best model (FRACTURE-ML), using DeepSurv with 2500 variables, showed excellent performance with a 1-year AUC of 0.89 and sustained accuracy at 5 years (AUC 0.85), representing the most accurate registry-based hip fracture model to date. High performance persisted with a reduced 35-variable model (1-year AUC 0.88), supporting clinical implementation. Cox models performed similarly, showing near-equivalent calibration and discrimination, and all DeepSurv and Cox models demonstrated substantial net benefit across thresholds from 80% sensitivity to 80% specificity.
The fracture prediction tool FREM (fracture risk evaluation model) was developed using Danish hospital diagnoses (n = 2,495,339) to predict major osteoporotic fractures and hip fracture. The FREM tool demonstrated high performance in 1-year prediction of hip fractures, particularly in women for which AUC reached 0.87, with slightly worse performance in men with an AUC of 0.85 [13]. Limitations included short-term prediction, no use of machine learning methods for prediction, input variables consisted exclusively on diagnoses, consideration of competing risk of death was not made, and external validation revealed consistent overestimation of hip fracture prediction [14]. Another large study used retrospective electronic health record data from over 1,000,000 individuals and applied a deep learning approach to identify patients with high immediate risk of fracture [27]. While the model achieved an AUC of 0.81 which demonstrated the potential of machine learning methods applied to very large datasets, the utilized black-box approach prevented model interpretability, and without knowledge of clinically relevant factors for fracture risk, and how these are implemented in the model development, credibility for clinical implementation may be questionable. Highly accurate prediction with AUC of 0.89 and 0.91 for women and 0.86 and 0.89 for men was obtained for hip fracture prediction using QFracture and CFracture, respectively, in analyses based on data from general practices in England and Wales, including both diagnosis, treatments and measurements of body mass index and lifestyle factors such as smoking and alcohol use [11,12]. FRAX without BMD yielded somewhat lower AUCs, 0.85 for women and 0.82 for men, using the same cohort data [11]. Although effective, these algorithms are dependent on physical measurements for weight and height, requiring patient interaction, and may not work sufficiently solely on register-based information, and while these variables might have improved FRACTURE-ML even further, they are not available in nationally collected data in Sweden and were therefore not included.
General screening for osteoporosis using the FRAX-tool has been demonstrated to reduce hip fracture numbers in a pooled analysis of randomized trials, but screening is not implemented in clinical practice in most countries [28]. Secondary prevention strategies including coordinator-based FLSs are recommended and implemented in many countries globally [18]. Since FLS is the most commonly used screening tool currently implemented to reduce hip fracture rates and is advocated by Swedish authorities with top priority [22], this approach was used as a comparator to the FRACTURE-ML clinical decision support tool. Although this comparison demonstrated that the proposed hip fracture prediction algorithm identified approximately 7-fold more patients at risk within 2 years and more than 12-fold more at risk within 10 years, with only a marginal reduction in specificity, these different approaches should not be seen as competitive, but rather complementary. Thus, screening for primary prevention using FRACTURE-ML could be used in addition to the already established FLS approach [19] to identify additional risk individuals and prevent a greater proportion of hip fractures than using FLS alone.
A key issue is to select appropriate probability thresholds, which may likely depend on availability of healthcare resources and willingness to invest in follow-up interventions after screening, which will likely vary across the 21 autonomous healthcare regions in Sweden. In our simulation of implementation (Fig 4) in the Swedish population, we either used a high sensitivity or a high specificity scenario, both at 80%, to illustrate the screening consequences. In women, the 80% sensitivity approach would require intervention in 43–64% of the population, depending on age group, whilst the 80% specificity approach would only require intervention in 16%–17%, at the expense of missing more patients with hip fractures. To find an optimal balance which would reduce the number of patients identified for intervention and the numbers of hip fractures missed, a proposed approach would be to implement a system with a variable threshold based on higher specificity in the younger ages and higher sensitivity at older ages.
FRACTURE-ML could be introduced for the primary prevention of hip fracture through a screening-based approach. One potential implementation would be to deploy the model via a national service integrated with registry data needed for FRACTURE-ML calculations. With appropriate consent, patients could access their FRACTURE-ML risk score through a health-service mobile application, an online form, or via a healthcare provider. Individuals identified as having elevated risk could then be offered further evaluation in line with clinical practice, including assessment of physical function and balance, as well as a bone mineral density measurement of the hip and lumbar spine using dual x-ray absorptiometry [29]. These assessments would inform appropriate interventions, such as fall prevention strategies and osteoporosis treatment.
Depending on healthcare setting and reimbursement willingness, different approaches to implement FRACTURE-ML could be used. For example, over a 5-year horizon in a population of 100,000 individuals, applying a FRACTURE-ML screening strategy with 80% sensitivity, which is normally recommended [30], would yield an estimated 2,300 hip fractures, based on the 2.3% 5-year incidence. Of these, 1,840 cases would be correctly identified (80% sensitivity), while 20% (n = 460) would remain undetected compared with universal screening. However, only 24,533 individuals would be flagged for further evaluation, corresponding to an approximate 75.5% reduction in the number of individuals requiring further investigation.
The present study has several strengths. It utilized the so far largest cohort ever used to develop a prediction model for hip fracture, using the most comprehensive dataset with a variety of variables routinely available from a large selection of registers, with different levels of detail and historic windows, unconditionally testing a very large number of predictor variables, using machine learning methods, as well as traditional statistical methods. Hip fractures are identified with high accuracy in Swedish registers with PPV of 95% [31]. The resulting AUC of 0.89 is the highest yet achieved for prediction algorithms for hip fracture for models developed on very large datasets without more granular data on BMD, anthropometrics and lifestyle factors which must be obtained by direct patient interaction. This particular tool would allow population-based screening since risk estimates can be provided for the entire population 50 years and older. Furthermore, the algorithm maintained high performance for longer time periods, and after considering the competing risk of death, which currently available automated data-mining-based algorithms have not investigated. Implementation will be facilitated by excellent model performance across different risk levels, by using the simplified 35-variable version requiring less computational effort, and, most importantly, a resource-efficient collection of risk information without the need for unnecessary patient examinations. The statistical reporting is comprehensive with (i) AUC confirming adequate ranking, (ii) precision-recall analyses to assess error rates, (iii) calibration curves to verify that the predicted risks are numerically accurate and (iv) net benefit calculations to show that the model improves real decision-making compared to doing nothing or everything.
Study limitations include lack of patient interview/questionnaire data on lifestyle factors of importance, such as smoking, exercise and alcohol intake. The accuracy and generalizability of FRACTURE-ML must be further validated in other countries. Thus, both external validation in other cohorts and studies testing the implementation of FRACTURE-ML are necessary to successfully evaluate the clinical utility of the tool.
In conclusion, the FRACTURE-ML clinical decision support tool showed excellent hip fracture predictive performance, with an AUC of 0.89 and strong individual-level calibration. Its high accuracy persisted over longer follow-up and after accounting for competing mortality, making it the most accurate hip fracture model tested to date. Implementing FRACTURE-ML could substantially improve efficient identification of high-risk patients, enabling targeted treatment and reducing hip fracture rates.
Supporting information
S1 Checklist. Tripod AI.
Completed TRIPOD+AI reporting checklist. The checklist was reproduced from Collins GS, Moons KGM, Dhiman P, and colleagues. The official checklist is available from https://www.tripod-statement.org/wp-content/uploads/2019/12/TRIPODAI_checklist.pdf. The checklist is licensed under the Creative Commons Attribution 4.0 International Licence (CC BY 4.0; https://creativecommons.org/licenses/by/4.0/). The original checklist wording was not modified; manuscript locations and study-specific responses were added by the authors.
https://doi.org/10.1371/journal.pmed.1005190.s001
(PDF)
S1 Appendix. Extended methods.
Extended information on study design, data sources, detailed variable descriptions, size of cohorts, discovery phase with overview of variable filtering process, development phase, validation in the holdout cohort, net benefit, clinical implications and statistical analyses
https://doi.org/10.1371/journal.pmed.1005190.s002
(DOCX)
S1 Fig. Discovery phase - overview of variable filtering process.
Clinically adjacent means variables from same sources with similar codes were analyzed together. The 2% cut-off was selected as it represents the same variable importance as previous fracture last year.
https://doi.org/10.1371/journal.pmed.1005190.s004
(DOCX)
S2 Fig. Loss curves for DeepSurv.
The loss functions for training versus validation of DeepSurv 2500 and DeepSurv 35 are presented below. Early stopping was used to monitor the validation loss and prevent potential overfitting of the models.
https://doi.org/10.1371/journal.pmed.1005190.s005
(PDF)
S3 Fig. Calibration plots for hip fracture probabilities in the development cohort.
Calibration plots were used to compare the predicted probabilities of hip fracture at 2, 5 and 10 years against the observed events. Original probabilities as well as recalibrated probabilities are presented, with their respective thresholds for high-sensitivity (80% in red) and high-specificity (80% in green). Comparisons were made for Cox 35 and Cox 400 (Panel A), DeepSurv 35 and DeepSurv 2500 (Panel B) and DeepSurv 35 competing risk (Panel C).
https://doi.org/10.1371/journal.pmed.1005190.s006
(DOCX)
S4 Fig. Calibration plots for hip fracture probabilities in the holdout cohort.
Calibration plots were used to compare the predicted probabilities of hip fracture at 2, 5 and 10 years against the observed events in the holdout cohort, with their respective thresholds for high-sensitivity (80% in red) and high-specificity (80% in green). Comparisons were made for Cox 35 and Cox 400 (Panel A), DeepSurv 2500 and DeepSurv 35 competing (Panel B).
https://doi.org/10.1371/journal.pmed.1005190.s007
(DOCX)
S5 Fig. Comparison with the current clinical practice - fracture liaison services.
Comparison of hip fracture prediction at 2, 5, and 10 years of the DeepSurv35 and 2500 models to current clinical practice, i.e., Fracture Liaison Service (FLS), represented by any fracture the past year.
https://doi.org/10.1371/journal.pmed.1005190.s008
(DOCX)
S6 Fig. Precision-recall curves for 2, 5 and 10 years prediction.
Presentation of precision (=positive predictive value) versus recall (= sensitivity) for hip fracture prediction at 2, 5 and 10 years for Cox 35, Cox 400, DeepSurv 35 and DeepSurv 2500. Calculated in the hold-out cohort (N = 354,261). For comparison, hip fracture incidences at 2, 5 and 10 years were 1.0%, 2.3% and 4.0%, respectively.
https://doi.org/10.1371/journal.pmed.1005190.s009
(DOCX)
S1 Table. Cohort characteristics per incident hip fracture (extended data).
The table contains a selection of variables from the 139,980 list of candidates (5-year history). SMD = standardized mean difference.
https://doi.org/10.1371/journal.pmed.1005190.s010
(DOCX)
S2 Table. Cohort characteristics per discovery/development/holdout.
The table contains a selection of variables from the 139,980 list of candidates split per discovery, development and holdout.
https://doi.org/10.1371/journal.pmed.1005190.s011
(DOCX)
S3 Table. Performance summary of different prediction models after 1, 2, 5 and 10 years.
AUC, Area Under Curve; PPV, Positive Predictive Value; NPV, Negative Predictive Value; TN, True Negatives; FP, False Positives; FN, False Negatives; TP, True Positive
https://doi.org/10.1371/journal.pmed.1005190.s012
(DOCX)
S4 Table. Cohort characteristics for the top 35 variables.
Cohort characteristics for the 35 most important variables in DeepSurv prediction model. Continuous variables presented with mean ± SD and categoric variables with n (%). Variable importance represents change in AUC (x 1,000,000,000) when randomly permutated, i.e., how much AUC would have changed if the variable had no predictive value. * = missing values treated as separate categories (marital status 3.1%, sickness benefits 3.1% and education level 4.2%)
https://doi.org/10.1371/journal.pmed.1005190.s013
(DOCX)
S5 Table. Values from precision-recall curves for 2, 5 and 10 years prediction.
Performance metrics in hold-out cohort (N = 354,261) for hip fracture prediction at 2, 5 and 10 years for Cox 35, Cox 400, DeepSurv 35 and DeepSurv 2500 at 80% recall (sensitivity) and maximum precision (positive predictive value). TP, True Positives; FP, False Positives; TN, True Negatives; FN, False Negatives.
https://doi.org/10.1371/journal.pmed.1005190.s014
(DOCX)
Acknowledgments
The authors wish to acknowledge all the individuals that contributed with their register data and all register officials for their comprehensive work in making the data available for this study.
References
- 1. Lorentzon M, Johansson H, Harvey NC, Liu E, Vandenput L, McCloskey EV, et al. Osteoporosis and fractures in women: the burden of disease. Climacteric. 2022;25(1):4–10. pmid:34319208
- 2. Lawrence VA, Hilsenbeck SG, Noveck H, Poses RM, Carson JL. Medical complications and outcomes after hip fracture repair. Arch Intern Med. 2002;162(18):2053–7. pmid:12374513
- 3. James SL, Lucchesi LR, Bisignano C, Castle CD, Dingels ZV, Fox JT, et al. The global burden of falls: global, regional and national estimates of morbidity and mortality from the Global Burden of Disease Study 2017. Inj Prev. 2020;26(Supp 1):i3–11. pmid:31941758
- 4. Dyer SM, Crotty M, Fairhall N, Magaziner J, Beaupre LA, Cameron ID, et al. A critical review of the long-term disability outcomes following hip fracture. BMC Geriatr. 2016;16(1):158. pmid:27590604
- 5. Lorentzon M. Treating osteoporosis to prevent fractures: current concepts and future developments. J Intern Med. 2019;285(4):381–94. pmid:30657216
- 6. Cummings SR, San Martin J, McClung MR, Siris ES, Eastell R, Reid IR, et al. Denosumab for prevention of fractures in postmenopausal women with osteoporosis. N Engl J Med. 2009;361(8):756–65. pmid:19671655
- 7. Black DM, Delmas PD, Eastell R, Reid IR, Boonen S, Cauley JA, et al. Once-yearly zoledronic acid for treatment of postmenopausal osteoporosis. N Engl J Med. 2007;356(18):1809–22. pmid:17476007
- 8. Black DM, Cummings SR, Karpf DB, Cauley JA, Thompson DE, Nevitt MC, et al. Randomised trial of effect of alendronate on risk of fracture in women with existing vertebral fractures. The Lancet. 1996;348(9041):1535–41.
- 9. US Preventive Services Task Force, Nicholson WK, Silverstein M, Wong JB, Chelmow D, Coker TR, et al. Screening for osteoporosis to prevent fractures: US Preventive Services Task Force Recommendation Statement. JAMA. 2025;333(6):498–508. pmid:39808425
- 10. McCloskey EV, Harvey NC, Johansson H, Lorentzon M, Liu E, Vandenput L, et al. Fracture risk assessment by the FRAX model. Climacteric. 2022;25(1):22–8. pmid:34319212
- 11. Hippisley-Cox J, Coupland C. Predicting risk of osteoporotic fracture in men and women in England and Wales: prospective derivation and validation of QFractureScores. BMJ. 2009;339:b4229. pmid:19926696
- 12. Livingstone SJ, Guthrie B, McMinn M, Eke C, Donnan PT, Morales DR. Derivation and validation of the CFracture competing risk fracture prediction tool compared with QFracture in older people and people with comorbidity: a population cohort study. Lancet Healthy Longev. 2023;4(1):e43–53. pmid:36610448
- 13. Rubin KH, Möller S, Holmberg T, Bliddal M, Søndergaard J, Abrahamsen B. A New Fracture Risk Assessment Tool (FREM) Based on Public Health Registries. J Bone Miner Res. 2018;33(11):1967–79. pmid:29924428
- 14. Möller S, Skjødt MK, Yan L, Abrahamsen B, Lix LM, McCloskey EV, et al. Prediction of imminent fracture risk in Canadian women and men aged 45 years or older: external validation of the Fracture Risk Evaluation Model (FREM). Osteoporos Int. 2022;33(1):57–66. pmid:34596704
- 15. Klotzbuecher CM, Ross PD, Landsman PB, Abbott TA 3rd, Berger M. Patients with prior fractures have an increased risk of future fractures: a summary of the literature and statistical synthesis. J Bone Miner Res. 2000;15(4):721–39. pmid:10780864
- 16. Johansson H, Siggeirsdóttir K, Harvey NC, Odén A, Gudnason V, McCloskey E, et al. Imminent risk of fracture after fracture. Osteoporos Int. 2017;28(3):775–80. pmid:28028554
- 17. PubMed 36970835.Axelsson KF, Litsne H, Lorentzon M. The importance of recent prevalent fracture site for imminent risk of fracture – a retrospective, nationwide cohort study of older Swedish men and women. J Bone Miner Res. 2023;38(6):851–9. pmid:36970835
- 18. Ganda K. Fracture liaison services: past, present and future. Osteoporos Int. 2021;32(8):1461–4.
- 19. Akesson K, Marsh D, Mitchell PJ, McLellan AR, Stenmark J, Pierroz DD, et al. Capture the fracture: a best practice framework and global campaign to break the fragility fracture cycle. Osteoporos Int. 2013;24(8):2135–52. pmid:23589162
- 20. Axelsson KF, Jacobsson R, Lund D, Lorentzon M. Effectiveness of a minimal resource fracture liaison service. Osteoporos Int. 2016;27(11):3165–75. pmid:27230521
- 21. Axelsson KF, Johansson H, Lundh D, Möller M, Lorentzon M. Association between recurrent fracture risk and implementation of fracture Liaison Services in Four Swedish Hospitals: a cohort study. J Bone Miner Res. 2020;35(7):1216–23.
- 22. The National Board of Health and Welfare. The national guidelines for musculoskeletal diseases. 2021. Available from: https://www.socialstyrelsen.se/publikationer/nationella-riktlinjer-for-rorelseorganens-sjukdomar--reumatoid-artrit-axial-spondylartrit-psoriasisartrit-artros-och-osteoporos--stod-for-styrning-och-ledning-2021-2021-1-7137
- 23. Riley RD, Ensor J, Snell KIE, Harrell FE Jr, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441. pmid:32188600
- 24. Riley RD, Snell KIE, Archer L, Ensor J, Debray TPA, van Calster B, et al. Evaluation of clinical prediction models (part 3): calculating the sample size required for an external validation study. BMJ. 2024;384:e074821. pmid:38253388
- 25. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B Stat Methodol. 1995;57(1):289–300.
- 26. Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Methodol. 2018;18(1):24. pmid:29482517
- 27. Almog YA, Rai A, Zhang P, Moulaison A, Powell R, Mishra A, et al. Deep learning with electronic health records for short-term fracture risk identification: crystal bone algorithm development and validation. J Med Internet Res. 2020;22(10):e22550. pmid:32956069
- 28. McCloskey EV, Chotiyarnwong P, Harvey NC, Lorentzon M, Kanis JA, International Osteoporosis Foundation Epidemiology/Quality of Life Working Group. Population screening for fracture risk in postmenopausal women – a logical step in reducing the osteoporotic fracture burden?. Osteoporos Int. 2022;33(8):1631–7. pmid:35763073
- 29. Hoong CWS, Saul D, Khosla S, Sfeir JG. Advances in the management of osteoporosis. BMJ. 2025;390:e081250. pmid:40738610
- 30. Power M, Fell G, Wright M. Principles for high-quality, high-value testing. Evid Based Med. 2013;18(1):5–10. pmid:22740357
- 31. Ludvigsson JF, Andersson E, Ekbom A, Feychting M, Kim J-L, Reuterwall C, et al. External review and validation of the Swedish national inpatient register. BMC Public Health. 2011;11:450. pmid:21658213