Literature review

We carried out a literature review, searching Medline, PubMed and Cochrane libraries for papers concerning falls interventions, cost of falls interventions, and falls risk modelling. For falls interventions, the terms accidental falls/fractures/hip fractures/hip injuries AND accident prevention were searched. For the cost of falls and interventions, the terms accidental falls/ fractures/hip fractures/hip injuries AND accident prevention AND cost and costs analysis were used. For risk modelling, the terms accidental falls/fractures/hip fractures/hip injuries AND risk modelling/risk assessment/risk were searched. All searches were limited to English language full text articles and ‘aged 65 and over’.

Study Population

Data for this study were collected as part of an integrated care pilot (ICP) study [10]. These data were routinely collected information from 130 general practice databases across North West London. We set the index date to January 31^st 2013, so that we had a year of data before, to build the predictive model, and an after period during which we could predict falls. The total registered population was 656902 patients, of whom 74751 met the eligibility criteria of age 65 years or over, and being registered with their general practice for a year prior to and following the index date. As data were extracted for the whole practice population in each of the 130 practices, there were no exclusions that might have resulted in selection bias being applied. These general practice data were collated with hospital data from three acute providers delivering secondary care to this population. Data collection was performed over a 2.5 year period between September 2011 and February 2014.

Risk Factors

Many potential risk factors and predictors for falls were identified for inclusion in this study. These were selected from variables extracted as part of the Integrated Care Pilot. Variables were selected if there was evidence suggesting that there was an association with or had the potential to predict falls. Patient factors associated with an increased risk of falling and therefore considered for the model included increasing age [1,2,11,12], female sex [11,13], living alone and disability [14], poor mobility and balance [2,12,15], weight loss/low body weight [12,14], and poor vision [12,16]. Smoking status, vulnerability, and deprivation were also considered as potential risk factors.

Recent healthcare utilisation is a recognised risk factor for falling. Therefore, recent inpatient episode, recent non-elective admission (and the number of non-elective admissions), recent outpatient visit, and a recent accident and emergency attendance (along with the number of attendances) were considered as potential predictors of falls. Previous fracture [12] and previous fall [1,13,16–18] also have a strong evidence base for predicting further falls.

Chronic disease in itself is a well-recognised risk factor for falling, and the risk increases with increasing number of co-morbidities [12]. Specific chronic diseases that have been associated with falling are Chronic Obstructive Pulmonary Disease (COPD), cardiovascular disease, diabetes, previous stroke, thyroid disease and arthritis [2,12,13,16], along with mental illnesses such as depression and cognitive impairment [12,15]. Other diseases considered in this model are asthma, any cancer, epilepsy, multiple sclerosis, osteoporosis, and urinary tract infection.

There are strong associations between patient medications and falls risk. Specific drugs with a firm evidence base are benzodiazepines [13,19–21], psychotropic drugs [12,14,20–22], antidepressants [12,13,19], and some cardiovascular medications including digoxin, class 1A anti-arrhythmic drugs, ACE-inhibitors, and diuretics [12,23]. Further to these specific drugs, polypharmacy (usually defined as 4 or more regular medications) has also been frequently implicated as a risk factor for falls [12,13,16,23,24]. In addition, statins have also been considered in the model due to their association with proximal myopathy and therefore leg weakness.

Definition of Primary Outcome

The primary outcome measure for this study was the occurrence of a fall or a fracture or both during the study period. This was determined by the presence of codes for these outcomes in the general practice or hospital data. A full list of codes selected for both falls and fractures can be found in S1 and S2 Tables. Fractures were used as a surrogate marker for falls where a fall itself was not documented, as over 90% of fractures occur as a result of a fall [1].

Cost Analysis

To undertake our cost analysis, the cost of a fall, cost of a falls intervention, and the efficacy of falls interventions were required. The cost of a fall was taken as the difference between the health utilisation, calculated using the Payment by Results methodology [25], for those that fell and those that did not, in the year following the index date. Other values for the cost of a fall were also considered, and the results of these can be found in S4 Table.

The cost of a falls intervention was taken as the mean cost from the literature of multifactorial falls interventions [26–28], as this is the intervention recommended by The National Institute for Health and Care Excellence (NICE) [29]. A recent Cochrane review found multifactorial falls interventions to produce a relative rate reduction of 24% [30], therefore this value was used to represent the efficacy of a falls intervention.

Statistical Analysis

Multilevel logistic regression analysis was performed to identify predictors of falls. The strongest predictors were used to generate a risk model. Multilevel analysis was performed in order to adjust for variation between populations based on general practice. A two level model was used with patients nested with their general practice. The random effect was the general practice, which may reflect quality of care but might be a proxy for other factors, such as deprivation and other characteristics of the locality not included in the model. For each of the variables, the strength of the association and impact on the Bayesian information criterion value was calculated, and variables without a statistically significant association (p<0.05) were removed from the final risk model. Odds ratios (OR) with 95% confidence intervals were calculated for each of the variables included in the final risk profile model (Table 1).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Risk profile model for occurrence of falls, obtained by multi-level logistical regression.

https://doi.org/10.1371/journal.pone.0159365.t001

The predictive accuracy of the model was expressed using the area under the curve (AUC) of the receiver operating characteristics (ROC) curve, sensitivity and specificity, and Youden’s index (sensitivity + specificity—1). Further validation studies were also carried out, including generating a Hosmer-Lemeshow plot for goodness-of-fit across different sub-populations, and Variance Inflation Factor (VIF), to assess for co-linearity. All statistical tests were carried out using R (Version 3.1.1), and the ‘lme4’ package was used to produce the model. “lme4” can be obtained from the Comprehensive R Archive Network (CRAN, https://cran.r-project.org/web/packages/lme4/index.html).

For the cost analysis, a Monte Carlo Model (MCM) simulation was used to generate a distribution around the key variables for the cost analysis; the cost of a fall, the cost of a falls intervention, and the effectiveness of a falls intervention [31]. The MCM simulation allowed for us to adjust for uncertainty in the estimates of these variables, by sampling from this distribution. A matrix was produced containing 100,000 values within 95% confidence intervals around the key variables for the cost analysis. Savings were calculated as the cost of the falls intervention in the intervened population minus the cost of the potential falls that were prevented in the intervened group. Using the matrix generated by MCM simulation, we produced the mean and confidence range, of the net savings with cut-off risk values at intervals of 0.01.

Ethical Considerations

This study formed part of the evaluation of the ICP. The data used were those recorded for routine care. This evaluation was retrospective and did not impact on any healthcare decisions made between doctor and patient. The data used were anonymised and no patient identifiable information was used. This investigation was approved by the ICP steering group.

Study Population

Data was collected on 74751 individuals over a 2.5 year period. During the follow up period 4941 subjects (6.6%) presented to their GP or to hospital with either a fall or a fracture or both. There were 40249 females (54%) and 34502 males (46%) in the study population. This greater proportion of females is reflective of the general elderly population and likely to reflect differences in life expectancy between genders [32]. Prevalence of falls was greater in females (8.3%) compared to males (5.2%). All of the population were aged 65 and over. The distribution of subjects among the age groups was: 21700 in 65–69, 16101 in 70–74, 12469 in 75–79, 8972 in 80–84, 5323 in 85–89, 2522 in 90–94, and 7664 in 95+.

Risk Model

Multilevel logistic regression analysis was performed to identify predictors of falls. The strongest predictors were used to generate a risk model, and variables without a statistically significant association (p<0.05) were removed from the final risk model in a stepwise fashion. Tests of validity were carried out on the risk model. A Hosmer-Lemeshow plot, for goodness-of-fit, indicated that the model was well calibrated. The ROC curve demonstrated a high level of predictive accuracy, with an area under the curve of 0.87 (Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Receiver operating characteristics curve of the falls risk prediction model.

AUC = 0.87.

https://doi.org/10.1371/journal.pone.0159365.g001

Tests for co-linearity were also carried out, which confirmed that there was no significant correlation between any variables in the model, with no variables having a Variance Inflation Factor greater than 3. Female sex, increasing age, an inpatient episode in the previous 1 month, outpatient visits within the previous 1 and 12 months, polypharmacy, asthma, COPD, depression, mental illness, previous stroke, previous urinary tract infection, non-elective hospital admission in the previous 12 months, GP coding of a previous fracture, osteoporosis, and a previous fall were all identified as significant risk factors contributing to our risk model (Table 1). The level of risk (Odds Ratio) increased with increasing age, with the exception of the over 95 age group. The level of risk also increased with increasing number of outpatient visits in the previous 12 months, and with increasing number of drugs (polypharmacy).

Further to this, risk was higher for a fracture or a fall occurring in the previous 6 months, as opposed to any past history of a fracture or a fall. An inpatient episode in the previous 2 months, and an A&E investigation in the previous 3 months, were both protective for falls. A total of 116 variables were considered in construction of the risk model. A full list of variables that were excluded due to lack of a significant association with a fall or fracture can be found in S3 Table. Notable variables excluded from the model, that based on existing evidence would have been expected to be risk factors, included: deprivation (Index of multiple deprivation score), disability [14] and vulnerability.

Chronic disease in itself [12] was also excluded, along with and specific chronic diseases such as cardiovascular disease, diabetes, thyroid disease and arthritis, cognitive impairment, any cancer, epilepsy and multiple sclerosis [12,15]. Although polypharmacy was a significant risk factor in our model, specific drugs with very strong evidence as falls risk factors, such as psychotropic drugs [12,14,20–22] including benzodiazepines[13,19–21] and antidepressants[12,13,19], and cardiovascular medications including statins, digoxin, class 1A anti-arrhythmic drugs, ACE-inhibitors, and diuretics [12,23] were all excluded from the model due to lack of a significant association.

To consider our risk model as a predictive tool, diagnostic and predictive values using risk cut-off values were calculated for intervals of 0.01, and of 0.05 (Table 2). The cut-off with the highest combination of sensitivity and specificity was a risk cut-off value of 0.07 (Fig 2). This cut-off produced a sensitivity of 81% and specificity of 78%. The Negative predictive value at this cut-off was 98% and positive predictive value was 21%.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Table showing cost calculations for varying risk level cut off values.

https://doi.org/10.1371/journal.pone.0159365.t002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Sensitivity and specificity of our risk model for cut off values at intervals of 0.05.

The cut-off with the highest combination of sensitivity and specificity was a risk cut-off value of 0.07.

https://doi.org/10.1371/journal.pone.0159365.g002

Cost Analysis

If no cut-off value is used, and 100% of the population is sent to a falls intervention, at a cost of £592 per patient, the total cost of intervening in our study population would be £44.2 million. 585 falls would be prevented, producing a saving of £3.6 million. This results in an overall cost of £40.6 million (95% Confidence Interval £24.9 million, £56.2 million) (Table 2). As the cut-off value is increased, the trend of costs exceeding savings continues until a cut-off risk value of p = 0.27. At this value, 1376 individuals (1.8% of the population) would be sent to the falls intervention, at a cost of £0.81 million. 127 falls would be prevented, producing a saving of £0.83 million. This would result in an overall net saving of £15889 (95% CI -£306289, £344415) (Fig 3). As the cut-off value increases further, net savings continue to increase, to a maximum of £92810 (95% CI £2677, £189932), at p = 0.53. Beyond a cut-off of p = 0.53 net savings begin to fall, however never below 0.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. A graph demonstrating the potential savings (or costs) to be made by intervening at varying levels of risk.

The mean (solid line), upper (dotted line) and lower (dashed line) 95% confidence intervals for net savings are represented along with bars for the cost of referral and savings from prevented falls.

https://doi.org/10.1371/journal.pone.0159365.g003

Principal Findings

Our model performs better than any other existing model or assessment tool to assess risk of falling. The AUC of the ROC curve is 0.87. The maximum combination of specificity and sensitivity is achieved at a risk cut-off of ≥0.07. At this cut-off, sensitivity is 0.78 and specificity is 0.81, creating a Youden’s index of 0.59, positive predictive value (PPV) is 0.23 and negative predictive value (NPV) is 0.98.

The next nearest contender, a prospective cohort study by Stalenhoef et al gathered data over a 36 week period and produced a risk model with an AUC of the ROC curve of 0.79 [33]. This risk model was produced from data on 311 patients, versus the 74751 participants in our study. Our model runs on routine data from primary and secondary care and performs better than our previous falls risk model produced from primary care data alone [8]. Of the final 29 variables forming our risk model, 10 of these are variables obtained from hospital data, and endpoints were collated from both general practice and hospital data. Our previous falls model produced a maximum sensitivity and specificity at a cut-off score of 9, sensitivity was 0.68 and specificity was 0.60, with an AUC of the ROC curve of 0.72, versus the 0.78, 0.81 and 0.87 respectively of the model presented in this paper.

Our falls risk model can be applied in a way that saves health services costs. Cost analysis was successfully applied to elucidate those patients in whom cost savings could be made through referral to a falls prevention service. At low cut-offs, there are considerably large net costs, because the costs of referring a large proportion of the population to a falls prevention service outweigh the savings made by preventing relatively few falls (Fig 2). At a cut-off value of p = 0.27, the savings from falls prevented exceed the costs of referral. At this cut off 1.8% of the population would be referred and the prevalence of falls among this population is 38%. Net savings increase as the cut-off increases beyond 0.27, to a peak at p = 0.53. Beyond 0.53, net savings fall, but never back below 0, into net costs. This means that above a risk score of 0.27, savings can be made by referral to a falls prevention service.

Implications of findings

In optimum use, this model will identify the top 22% most at risk of falling within a population with a sensitivity of 0.78, a specificity of 0.81, PPV of 0.23 and NPV of 0.98 at a cut-off value of ≥0.07. Based on our data, this 22% of the population would include 77% of all falls. Due to our model being produced from routine healthcare data, it could be implemented across healthcare services within the UK or adapted to run in systems with similar health data. We have provided the full list of variables considered for the model and code lists used to produce these variables in the supporting documents.

Our model could be integrated into a general practice or integrated care database to identify high risk patients at an individual population level. Through use of the findings of our cost-analysis, users could identify those patients in whom savings could be made through referral to a falls prevention service. Further savings can be attributed to the very minimal screening time and costs compared to conventional screening methods which cost approximately £165 per patient [26].

We recommend that a cut-off value of 0.27 is used to guide referral to falls prevention services: the value at which savings from prevented falls outweigh the costs of referral. At this value the most at risk 1.8% of the over 65 population would be referred. In a moderate sized general practice of 10,000 patients this would result in referral of 31 individuals [32]. Service providers may however prefer to use a cut-off that produces the maximum level of savings, which would be achieved at a higher cut-off risk value of 0.53. This would result in referral of the most at risk 0.45% of the over 65 population (8 individuals from a population of 10,000).

Comparison to the literature

A literature review of existing falls risk models and tools (S5 Table), was unable to yield any models with superior predictive capabilities (AUC of the ROC curve) to our risk model. The most frequently used tool in falls services in the UK is the Falls Risk Assessment Tool (FRAT) [6]. A study of the FRAT did not provide the AUC, but instead presented Youden’s Index as a predictive measure. The maximum Youden’s Index was 0.39, for a cut-off of ≥2 [5], compared to 0.56 at a cut-off of 0.07 in our model. In addition to the superior predictive capabilities of our model, entire populations could be screened in minutes, versus a time consuming individual assessment required in other tools such as FRAT.

While several studies have investigated the cost-effectiveness of multifactorial falls interventions, in the UK [26] and abroad [27,34–36], no studies (of any type of falls intervention) could be found that apply cost-analysis at varying levels of risk, or that aim to identify those in whom the intervention would be cost effective, as has been carried out in our research; although one study did identify that an intervention was more cost effective in the higher risk population [37]. The majority of these studies show multifactorial interventions to not be cost effective. However, as highlighted by our research, this does not mean that they would not be cost effective, if the correct population were referred.

Limitations

Although also a clear strength, one limitation of this study is that the database was derived from routinely collected general practice and hospital data. The general practice data was derived from Read Codes recorded during consultations and is therefore reliant on accurate code selection and on hospital attendances with falls/fractures being communicated. Hospital data was derived from ICD-10 codes recorded by hospital coders and is therefore reliant on accurate code selection and on the hospital doctors having documented that a fall/fracture had occurred. We have minimised this effect by combining falls from both general practice and hospital data in the analysis. Not all elderly people who fall will go to their general practice or to the hospital. Medical attention will likely only be sought if there are multiple falls or the mechanism of the fall or the resultant injuries are significant enough to require medical assistance. A Cochrane review has shown that only 20% of falls result in injury significant enough to require medical attention [30]. This means that a significant proportion of falls will not be included in the data, which is of particular importance given that previous falls have consistently been shown to be a predictor of subsequent falls [4]. A potential benefit of minor falls not being represented is that the falling population included within our risk model are those who have required medical attention resulting from a fall, and are therefore likely to represent a higher risk population that would benefit more from intervention. A further, unavoidable limitation of the study is that the inclusion criteria stipulated that subjects must be registered with their general practice for the entire study period. This unfortunately excludes those subjects that died during the study period. As such, any falls occurring in this population have not been included in the analysis. Our MCM simulation assumed that the various cost and benefits of a fall were uncorrelated, e.g. there was no correlation between the cost of an intervention and its chance of success. This assumption may be unrealistic; however there is no prior literature that could be used to inform this model. Further limitations are that the model was developed with data from a single area, albeit with a population of 656902, and it has not been validated on an external population.

Further research

Further work is needed to validate this risk model in other populations. A randomised controlled trial also needs to be performed, in which the risk model is applied to a general practice population to identify high risk individuals for referral to a falls prevention service, to elucidate whether the application of the model can firstly reduce the number of falls or fractures occurring in this population, and secondly produce savings through the prevention of these falls.

The model in its current iteration is produced from integrated care data. Since integrated care is currently a limited concept, most general practices are likely to have access to only general practice records. Therefore a version of the risk model with general practice only variables will need to be produced. This will improve the ease of use and widen its application possibilities. An Excel spreadsheet illustrating the model has been produced and is available.