https://orcid.org/0000-0002-3790-9205
Figures
Abstract
Background
Health systems are under pressure to maintain services within limited resources. The Evidence-Based Interventions (EBI) programme published a first list of guidelines in 2019, which aimed to reduce inappropriate use of interventions within the NHS in England, reducing potential harm and optimising the use of limited resources. Seventeen procedures were selected in the first round, published in April 2019.
Methods
We evaluated changes in the trends for each procedure after its inclusion in the EBI’s first list of guidelines using interrupted time series analysis. We explored whether there was any evidence of spill-over effects onto related or substitute procedures, as well as exploring changes in geographical variation following the publication of national guidance.
Results
Most procedures were experiencing downward trends in the years prior to the launch of EBI. We found no evidence of a trend change in any of the 17 procedures following the introduction of the guidance. No evidence of spill-over increases in substitute or related procedures was found. Geographic variation in the number of procedures performed across English CCGs remained at similar levels before and after EBI.
Conclusions
The EBI programme had little success in its aim to further reduce the use of the 17 procedures it deemed inappropriate in all or certain circumstances. Most procedure rates were already decreasing before EBI and all continued with a similar trend afterwards. Geographical variation in the number of procedures remained at a similar level post EBI. De-adoption of inappropriate care is essential in maintaining health systems across the world. However, further research is needed to explore context specific enablers and barriers to effective identification and de-adoption of such inappropriate health care to support future de-adoption endeavours.
Citation: Glynn J, Jones T, Bell M, Blazeby J, Burton C, Conefrey C, et al. (2023) Did the evidence-based intervention (EBI) programme reduce inappropriate procedures, lessen unwarranted variation or lead to spill-over effects in the National Health Service? PLoS ONE 18(9): e0290996. https://doi.org/10.1371/journal.pone.0290996
Editor: Dirceu Henrique Paulo Mabunda, University Eduardo Mondlane, MOZAMBIQUE
Received: November 2, 2022; Accepted: August 20, 2023; Published: September 1, 2023
Copyright: © 2023 Glynn et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This study is based in part on data from the Hospital Episode Statistics (HES) obtained from NHS Digital (previously the Health and Social Care Information Centre), re-used with the permission of The Health & Social Care Information Centre, and is not publicly available. The data are provided by patients and collected by the NHS as part of their care and support. HES data can be accessed via NHS Digital (https://digital.nhs.uk/services/data-access-request-service-dars). The authors confirm that others would be able to access or request these data in the same manner as themselves. The authors also confirm that they did not have any special access or request privileges that others would not have.
Funding: This project is funded by the NIHR Health and Social Care Delivery Research Programme (NIHR130547). TJ’s time is supported by the National Institute for Health and Care Research Applied Research Collaboration West (NIHR ARC West). The views expressed in this article are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Health systems are under pressure to maintain services and introduce novel interventions within limited budgets. Before the COVID-19 pandemic, there were approximately 170 million elective operations globally each year [1]. However, some established elective procedures may be inappropriate or considered ‘low value’ in certain circumstances. ‘Low value’ is defined as interventions where the costs outweigh the benefits in some or all patients [2, 3]. This may be the result of more effective procedures entering the health system or research identifying procedures that offer little or no benefit relative to alternatives including conservative management or no intervention. For example, research has found that several common elective orthopaedic procedures lack high quality evidence of benefit [4]. Across the world it is common for surgical procedures to have lower regulation for adoption compared to other health interventions such as pharmaceuticals and medical implants which may result in over adoption of “low value” care [5, 6]. De-adoption is the process of stopping or reducing an existing clinical practice. International efforts such as the ‘Choosing Wisely’ initiative actively promote the identification and de-adoption of unnecessary medical tests, treatments, and procedures [7]. This is essential for the financial sustainability of healthcare systems. Inappropriate use of procedures has opportunity costs, diverting resources from other, more effective, care. However, attempts to reduce established healthcare interventions often fail [8–10]. In the English National Health Service (NHS), local health budget holders (known, until 1st July 2022, as clinical commissioning groups) developed evidence-based, but often divergent, policies regulating access to surgical interventions for their populations [11]. This is likely to be one factor resulting in variation in geographical access to surgery in England [12]. In 2019, NHS England and the Academy of Medical Royal Colleges (AoMRC) launched the Evidence-Based Interventions (EBI) Programme [13]. The EBI programme aimed to identify interventions that are being used in inappropriate circumstances and should either be stopped or limited to patients who would benefit most. Inappropriate here meaning the risks and costs of surgery outweigh the benefits. The EBI programme’s primary aim was to reduce the use of inappropriate health care interventions, thereby reducing potential harm and optimising the use of limited resources [13]. However, the additional objectives of reducing unwarranted geographical variation and encouraging shared patient-clinician decision making have become more prominent as the EBI programme has evolved [14]. Seventeen surgical procedures were initially identified in April 2019; of these, EBI recommended that four (category 1 procedures) should not be routinely funded and the remaining 13 (category 2 procedures) should only be offered to patients who meet specific clinical criteria. The EBI programme targeted a reduction in category 1 procedures to “near zero” and approximately a 65% reduction in category 2 procedures by April 2020, a total reduction of approximately 128,000 procedures per year [13]. The list of 17 procedures were selected in an iterative process from a much larger list collated by ’expert working groups’ utilising clinical evidence from bodies such as NICE, the ‘Choosing Wisely’ programme, academic studies and local CCG work. The procedures are presented in Table 1. The full details of the procedures, clinical criteria, and the selection process, have been published in detail elsewhere [13]. Alongside the publication of list one as statutory guidance, the EBI programme also set individual reduction targets for each CCG. Monitoring of progress towards these targets was made available through an online dashboard. The programme also recommended that reimbursement tariff payments be reduced to zero for category one procedures and that all category two procedures require a prior approval process by their CCG [13].
Anderson et al. [15] provided a robust initial evaluation of the EBI list one guidance using a single difference-in-differences analysis aggregating 16 of the 17 procedures. They concluded that the EBI programme did not accelerate ‘disinvestment’ when analysing the programme as a whole. Disinvestment in their work was defined as the withdrawal of health care resources from existing practices deemed to deliver no or low health gain for their cost [15]. Our work extends this research by: (1) evaluating whether the EBI programme successfully promoted de-adoption in any of the 17 individual surgical procedures by analysing procedures individually; (2) evaluating whether the EBI programme reduced geographic variation in procedure rates across England; and (3) exploring potential ‘spill-over’ effects on related or substitute procedures. Given expected restriction in therapeutic options, ‘spill-over’ effects refer to any concurrent increases or decreases in the use of other similar procedures for the same medical conditions.
Methods
Data sources
All data were extracted from the Hospital Episode Statistics Admitted Patient Care (HES-APC) or Outpatient (HES-OP) datasets [16]. HES-APC is a routinely collected dataset that records all episodes of care delivered to patients admitted to acute hospitals in England. HES presents data as patient episodes of care referring to periods of care under a single consultant. Therefore, an individual patient may have more than one episode of care within the dataset we analyse, but their procedure would not be counted twice. HES-APC records up to 24 procedures per episode captured as Office of Population Censuses and Surveys (OPCS-4) codes and up to 20 diagnoses defined by International Classification of Disease (ICD-10) codes which we used to identify eligible episodes of care [17, 18]. These diagnosis and procedure codes are used to reimburse hospital trusts, or private care funded by the NHS, for treatments provided. Therefore, they are considered accurate and are widely used in research [16]. HES-OP captures all specialist outpatient consultations at acute hospitals in England. Increasingly, minor procedures are being performed in outpatient clinics rather than involving a hospital admission. Therefore, we included HES-OP procedures if they represented more than 10% of the total performed interventions. Primary diagnosis is poorly recorded in HES-OP (>95% missing), therefore we assumed the same percentage of EBI defined, related, and cancer procedures (explained below) present in the inpatient dataset. Both HES datasets include care at NHS hospitals and NHS-funded patients treated in independent sector hospitals.
Eligible episodes
Hospital episodes were eligible if they took place between 1st April 2010 and 31st March 2020. We used the EBI programme’s definition for each procedure, which was based on the relevant procedure code(s) being recorded as the primary procedure and combined with specific diagnosis codes identifying the targeted patient group [13].
To explore ‘spill-over’ effects we extracted episodes containing procedures that were potentially ‘related to’ or ‘substitutes for’ the EBI procedures. Related procedures were defined as an episode of care that had the targeted EBI primary procedure code(s) but with a non-EBI diagnosis (i.e. a type of within unit spill-over in the INTENTS framework [19]). For example, the EBI procedure knee arthroscopy relates to the treatment of osteoarthritis, this procedure is also commonly undertaken for ‘internal derangement of the knee’, these procedures are not target by EBI and therefore are captured as related procedures in our analyses. Substitute procedures were defined as different primary procedures which could be used as an alternative to the EBI procedure to treat the same patient group (i.e. a type of between unit spill-over [19]). For example, increased use of more invasive procedures to remove haemorrhoids (e.g. rubber band ligation). To identify potential substitute procedures, using existing contacts, we met with 11 specialist surgeons with expertise in one or more of the 17 EBI procedures. EBI procedures, related procedures, and substitute procedure episodes were mutually exclusive categories. However, appropriate or measurable substitute procedures were not identified for all EBI procedures, and related procedures were only analysed if they constituted a significant (>10%) proportion of the total number of the related EBI procedure performed. All codes for data extraction are available in the S1 File.
Statistical analysis
All analyses were completed in Stata version 16 or 17 statistical software.
Trend analyses.
Individual hospital episodes were aggregated into monthly admission counts and used as the dependent variable in the regression models, assuming that episodes with the same admission date and patient ID represent a single admission. A dummy variable categorised the month as either before (pre-April 2019) or after EBI. We controlled for the seasonality of procedure rates by including two dummy variables for the summer (June, July, August) and winter months (November, December, January). The model was offset using population estimates, provided by the Office for National Statistics (ONS), to control for changes in population size over the study period [20].
For each procedure, we generated scatter plots of total procedures per month over the 10-year period. From visual inspection of the scatter plots we decided to limit the regression analyses to the 2 years prior and 11 months after the EBI programme launch (i.e. 1st April 2018 to 29th February 2020). This captured the immediate pre-intervention trend and excluded longer-term, often non-linear, trends in the 9 years before EBI. We also excluded procedures after 29th February 2020, given the onset of the COVID-19 pandemic hugely diminished procedure rates form March 2020 onwards.
We used an interrupted time series (ITS) analysis utilising segmented Poisson regression models to compare trends in pre- and post-EBI procedure rates. This was implemented in Stata using a GLM regression with a Poisson family and log link function. We hypothesised a gradual rather than immediate change in procedure rates (i.e. a change in the slope rather than a step change in procedure rates) following the implementation of the EBI programme. Visual inspection of scatter plots supported this hypothesis. To account for autocorrelation in the regression models we used Newey-West standard errors, assuming a maximum lag of 2 months [21, 22].
Geographic variation analysis.
In the geographic variation analyses we clustered hospital admissions by CCG (using April 2019 boundaries) based on patient residence captured in HES (i.e. lower-super output areas (LSOAs)) [23]. Indirectly age-sex standardised procedure rates per 100,000 population were calculated for each CCG for the two financial years before the EBI programme (2017/18 and 2018/19) and the 11 months after the EBI programme (March 2019 to February 2020) [24]. The national age-sex specific rates for each time period were applied to the age-sex specific CCG population to calculate expected CCG procedure counts. Expected and observed procedures were aggregated by EBI category (i.e. category 1 and 2). Variation in the ratio of observed to expected procedure counts by CCG in each time period were estimated using the systematic component of variance (SCV) [25–27]. The SCV statistic (equation1 in S1 File) indicates the amount of variation between CCGs after adjusting for chance variation. We generated 95% confidence intervals around the SCV values using bootstrapping and the percentile method [28].
Sensitivity analysis.
We assessed the impact of the EBI programme’s list one consultation period (July 2018- April 2019). To do this we explored trends changes at the release of the consultation document for the EBI programme in July 2018, and whether this was followed by any further change once fully implemented in April 2019. Secondly, we assessed if there was any difference in results using a longer run-in period (36-months) before the EBI programme. Thirdly, we assessed whether there was an immediate step change in procedure rates for category 1 EBI procedures.
Results
Procedure counts
Aggregating across the 17 EBI procedures, there were 447,227 procedures in 2017/18, 433,159 in 2018/19 (3.2% reduction) and 403,739 in 2019/20 (6.8% reduction, including March 2020). By far the most frequently performed procedure was the removal of benign skin lesions with 655,219 from April 2017 to April 2020. The least frequent procedure was snoring surgery with 1,836 procedures over the 3 years, and the median procedure was grommets for glue ear in children with 31,020 procedures over the three years.
For the majority of the procedures, rates were clearly declining across the 10 years of data before the publication of the first EBI guidance. The gradient and timing of these declines varied between procedures (Figs 1–17 in S1 File) and in a minority of procedures, such as Dupuytren’s contracture release and the removal of benign skin lesions, there was no obvious decline before the EBI programme. There were no procedures displaying a consistent increase in rates before the EBI programme.
The regression analyses showed no significant changes in procedure rate trends after the introduction of the EBI programme for all 17 procedures (Table 1). For example, tonsillectomy rates (Fig 1) steadily declined from 50 per 100,000 in 2017/18 to 46 per 100,000 in 2018/19 and 37 per 100,000 in the 11 months after EBI. Across all category one (not to be routinely commissioned) and category two procedures (commissioned only when specific criteria are met), there were 43 and 327 procedures per 100,000 in 2017/18 respectively, falling to 32 and 298 per 100,000 in 2018/19 and 23 and 259 per 100,000 in the 11 months after EBI (excision of benign skin lesions are excluded given very large numbers).
Related and spillover procedures.
There was little evidence of spillover effects on related and substitute procedures (Table 1). In general, trend changes in these procedures were not significant and had point estimates in the same direction as the point estimate for the associated EBI procedure. Related procedure rates for knee arthroscopy for osteoarthritis, breast reduction surgery and chalazion removal (see S1 File for details on procedures and codes) were significantly higher post-EBI than would have been predicted based on pre-EBI trends. The remaining related procedures showed no significantly different trend following the publication of the guidance. The six procedures with identified inpatient or outpatient substitute procedures showed no evidence of spill-over effects (Table 1).
Geographic variation analyses
Variation in procedure rates between CCGs was consistently higher in category one compared to category two procedures between April 2016 and Feb 2020 (Table 2). Both category one and two procedures showed no evidence of reduced geographical variation post EBI. In fact, geographic variation (measured by SCV scores) was higher following EBI for both category one and two procedures, although confidence intervals were overlapping and wide (Table 2). There was little discernible pattern of higher procedure rates in particular regions of the country (Fig 2). In the 11 months after the launch of the EBI programme, the top 25% of CCGs, by adjusted procedure rates, were providing 3,808 more category 1 procedures and 26,209 more category 2 procedures than expected based on the age and sex distribution of their population.
Discussion
Main findings
We found little evidence that the EBI programme achieved its aim to reduce the number of surgical interventions through publishing nationwide guidance on appropriate use. The majority of the selected procedures exhibited declining procedure rates in the years before the publication of the EBI criteria and all pre-existing trends continued after the publication of the EBI guidelines in April 2019. Given that the EBI programme had limited additional impact on already declining trends, it is unsurprising that there was limited evidence of a spillover effect on related or substitute procedures. We also found no evidence that the publication of national guidance reduced variation in the number of procedures performed between CCGs compared to pre-EBI variation.
Comparisons with similar studies.
In accordance with Anderson et al. [15], we found no evidence of an impact of the EBI programmes list one on procedure rate trends [15]. Furthermore, we demonstrated that no individual procedure showed an associated reduction in procedure rates following EBI. Adding further to Anderson et al. [15], we explored whether the EBI programme had reduced geographical variation, even if overall levels continued to fall at similar rates. Potentially, national guidance could have resulted in some localities relaxing stricter pre-existing clinical policies to match the EBI guidelines while others were tightening or implementing guidelines for the first time. We also explored ‘spill-over’ effects hypothesising the number of related or substitute procedures may increase as access to EBI defined procedures is restricted. These are important considerations for the EBI, and other, programmes as there could be a warranted or unwarranted increase in the use of more expensive or resource intensive treatment options, or inappropriate use of procedures beyond their evidence-based indication. We used an ITS analysis in contrast to the difference in differences method used by Anderson et al. [15]. Difference in differences require a comparison group of similar procedures not affected by EBI. Anderson et al. [15] use EBI list two interventions as a comparison group not target by EBI at the time of analysis. However, there are important difference between the list one and list two procedures. List two included procedures that were not already reducing and less widely agreed upon as ‘low value’ compared to list one. Therefore, in the absence of a clear comparator, we decided the ITS would be an appropriate approach and the assumption of a continuing trend to be plausible.
De-adoption of medical procedures presents different challenges to the adoption of novel health care [29]. Programmes such as ‘Choosing Wisely’ and the National Institute of Health and Care Excellence (NICE) “do not do” recommendations have similarly struggled to implement changes into practice [30, 31]. A number of reviews have explored the barriers and enablers to the success of de-adoption programmes such as EBI [29–32]. Incorporating strategies to promote de-adoption amongst the relevant clinicians has been associated with successful de-adoption. Examples of such strategies included making changes to clinical documentation, computer alerts, and education, mainly in the context of primary care [30–32]. Applying multiple strategies is associated with greater success and there is currently little evidence that patient level approaches to de-adoption, such as cost sharing, are successful, although these approaches have not been explored in much published work [31].
Strengths and limitations
Our study is the first to explore whether the EBI programme had ‘spill-over’ effects on other types of NHS surgical care not specifically targeted by the programme, and to examine geographical variation before and after the programme. By examining each of the procedures individually, we were able to explore whether or not the EBI programme led to successful de-adoption from some procedures while making little headway with others.
The EBI list one procedures are predominantly undertaken in an inpatient or outpatient hospital setting which is captured by the routine HES datasets. As the HES datasets are mandatory and include all hospital care provided for NHS-funded patients our results should provide an accurate reflection of the impact of the EBI programme on its intended target, publicly funded healthcare. Some of the most minimally invasive procedures could have been provided in a primary care setting. However, this is not expected to be common and unlikely to mask any substantial effect of the EBI programme on procedure rates.
Our analyses were limited to 11 months post-EBI due to the emergence of the COVID-19 pandemic in March 2020. Therefore, we were not able to assess whether the EBI programme had any longer-term impact on procedure counts or variation beyond 11 months. There may be some delay between EBI publication and impact due to patients being placed on waiting lists for surgery pre-EBI but not receiving surgery until post-EBI, and for guidance to be adopted into practice. Before COVID-19 waiting times were increasing, although the majority of patients did not wait more than 18 weeks for elective treatment [33]. ITS is susceptible to bias due to unmeasured external shocks, many of these will be specific to one procedure (e.g. publication of a paper supporting or questioning efficacy). If it occurs, the bias could either inflate or deflate the apparent effect of EBI. If EBI were generally effective, and yet (as we have observed) procedure rate trends did not change post-EBI, that would imply either one general external shock post-EBI that has counteracted the effect of EBI or seventeen specific external shocks have had the same counterbalancing effect for each procedure. Whilst we cannot exclude these possibilities, a simpler explanation and one that is in line with previous research [15]. using different methods, is that the EBI programme did not achieve its objectives to reduce unnecessary healthcare or geographical disparities in access to care.
We were not able to explore spill-over effects elsewhere in the health system. For example, restricted access to NHS surgical care may increase the proportion of patients who seek surgical care through the private sector. Other patients may receive more non-surgical care for their symptoms through NHS primary and community care providers. HES Outpatient data has very poorly captured diagnoses, with less than 95% of interactions having a diagnosis code. Therefore, we had to estimate diagnoses and exclusions based on percentages observed in the HES-APC dataset, which may be inaccurate. Our variation analyses adjusted for the key variables of population size, age and sex, but there may be other factors such as amount of private provision that may be associated with the number of procedures within CCG areas.
Implications for policy
The majority of procedures included in EBI’s list one guidelines were already on a downward trajectory prior to their publication. Many of the procedures had prominent randomised clinical trials providing evidence of limited effectiveness and cost-effectiveness of these interventions in certain patient groups prior to the launch of list one [34, 35]. It is plausible that commissioners and clinicians were already aware of, and acting on, this evidence. Also, in many CCGs procedure counts were already low limiting the scope for the additional impact for the EBI programme. It may therefore be counterproductive to begin de-adoption programmes for procedures with already existing downward trends. This is particularly the case if recent evidence has questioned the efficacy of established procedures and if many local commissioning polices already restrict access. A simple monitoring of rates may be more appropriate in these cases. Identification of new areas for de-adoption that extend beyond the ‘usual suspects’ may have resulted in greater impacts. Questions remain around what constitutes an effective ‘starting point’ for the identification of candidates for de-adoption, particularly given that de-adoption initiatives themselves have resource implications. Previous rigorously developed de-adoption initiatives, such as the ‘Sustainability in Health care by Allocating Resources Effectively’ (SHARE) disinvestment programme in Australia, have also struggled to show impact: of 19 procedures identified for potential disinvestment, only one was taken forward for potential implementation. Reasons for rejecting candidate procedures revolved around lack of local relevance, or later realisation that de-adoption efforts had been already initiated or even concluded [36, 37]. There is an argument for future de-adoption identification and prioritisation exercises to pay explicit attention to whether candidates have a history of de-adoption attempts. If only to learn from barriers or facilitators to activity reduction, and better assess whether further injection of resource is worthwhile relative to pursuit of other, novel candidates. De-adoption initiatives that seek to reduce activity, whether local (e.g. SHARE) or national (e.g. the EBI programme), need to consider the cost and resources required to run such programmes.
It is also apparent that, at least in the short term, publishing statutory national guidelines does not automatically result in standardisations in care, as variation in procedure rates, taking account of population sizes, age and sex, were not reduced. This highlights the important role of evidence-based implementation in the success of programmes like EBI.
Future research
This work is part of a mixed methods analyses of the EBI programme. We are conducting concurrent qualitative work with clinicians, patients and commissioners and will explore the delivery, impact, and acceptability of the EBI programme’s list one and two guidance. Building on the findings of this work, this qualitative analyses will be able to shed light on why little differences in procedure trends were found following EBI, including barriers to the implantation of these policies. We will also be expanding our analyses of ‘spill-over’ effects by exploring the impact of EBI on the wider treatment pathway including the impact within primary care. It is plausible that restriction in the use of surgical treatment for these conditions will have impacts on consultations, treatments, and referrals within primary care.
Since 2019 the EBI programme has published a second list of recommendations including 31 interventions. This subsequent lists go beyond elective surgery to include screening, diagnostic tests, changes to referral pathways, and increasing the prominence of shared decision making, that could result in fewer interventions. We will extend our analyses to examine the impact of the EBI programme’s second list of procedures. It is plausible that different approaches or strategies work in different contexts, and this is why applying multiple strategies appears to have a better chance of success. Understanding what strategies work in which contexts would be helpful in targeting interventions [32]. Our project aims to synthesis findings from our quantitative and qualitative work and present these to stakeholders including patients, clinicians, commissioners, and national policy leads with a view to co-producing recommendations to optimise future de-adoption efforts.
Conclusion
The EBI programme’s first set of national guidelines on appropriate criteria for elective surgical care has had little success in their aim of further reducing unnecessary healthcare or minimising geographical disparities in access to care. The overall number of procedures fell, however pre-existing downward trends mean we cannot attribute this to the EBI programme. De-adoption of low-value care is essential in maintaining health systems across the world. We need to better understand the enablers and barriers to effective de-adoption to support future de-adoption endeavours.
Acknowledgments
We thank the specialist surgeons and consultants that identified potential substitute secondary care procedures.
References
- 1. COVID Surg Collaborative. Elective surgery cancellations due to the COVID-19 pandemic: global predictive modelling to inform surgical recovery plans. British Journal of Surgery. 2020;107(11):1440–9. Epub 20200613. pmid:32395848; PubMed Central PMCID: PMC7272903.
- 2. Niven DJ, Mrklas KJ, Holodinsky JK, Straus SE, Hemmelgarn BR, Jeffs LP, et al. Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med. 2015;13:255. Epub 20151006. pmid:26444862; PubMed Central PMCID: PMC4596285.
- 3. Verkerk EW, Tanke MAC, Kool RB, van Dulmen SA, Westert GP. Limit, lean or listen? A typology of low-value care that gives direction in de-implementation. IInternational Journal for Quality in Health Care. 2018;30(9):736–9. pmid:29741672; PubMed Central PMCID: PMC6307334.
- 4. Blom AW, Donovan RL, Beswick AD, Whitehouse MR, Kunutsor SK. Common elective orthopaedic procedures and their clinical effectiveness: umbrella review of level 1 evidence. BMJ. 2021;374:n1511. Epub 20210707. pmid:34233885; PubMed Central PMCID: PMC8262448.
- 5. Darrow JJ. Explaining the absence of surgical procedure regulation. Cornell J Law Public Policy. 2017;27(1):189–206. pmid:29239595.
- 6. Friebel R, Hauck K, Aylin P, Steventon A. National trends in emergency readmission rates: a longitudinal analysis of administrative data for England between 2006 and 2016. BMJ Open. 2018;8(3):e020325. Epub 20180312. pmid:29530912; PubMed Central PMCID: PMC5857687.
- 7. Levinson W, Kallewaard M, Bhatia RS, Wolfson D, Shortt S, Kerr EA, et al. ’Choosing Wisely’: a growing international campaign. BMJ Quality & Safety. 2015;24(2):167–74. Epub 20141231. pmid:25552584.
- 8. Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007;4:23. Epub 20071031. pmid:17973993; PubMed Central PMCID: PMC2174492.
- 9. Haas M, Hall J, Viney R, Gallego G. Breaking up is hard to do: why disinvestment in medical technology is harder than investment. Aust Health Rev. 2012;36(2):148–52. pmid:22624634.
- 10. Rooshenas L, Owen-Smith A, Hollingworth W, Badrinath P, Beynon C, Donovan JL. "I won’t call it rationing…": an ethnographic study of healthcare disinvestment in theory and practice. Soc Sci Med. 2015;128:273–81. Epub 20150114. pmid:25635374.
- 11. Rooshenas L, Ijaz S, Richards A, Realpe A, Savovic J, Jones T, et al. Variations in policies for accessing elective musculoskeletal procedures in the English National Health Service: A documentary analysis. J Health Serv Res Policy. 2022:13558196221091518. Epub 20220515. pmid:35574682.
- 12.
Hollingworth W, Rooshenas L, Busby J, Hine CE, Badrinath P, Whiting PF, et al. Using clinical practice variations as a method for commissioners and clinicians to identify and prioritise opportunities for disinvestment in health care: a cross-sectional study, systematic reviews and qualitative study. Health Services and Delivery Research. Southampton (UK)2015.
- 13.
NHS England in partnership with NHS Clinical Commissioners; the Academy of Medical Royal Colleges; NHS Improvement; and the National Institute for Health and Care Excellence. Evidence-Bases Interventions: Guidance for CCGs 2018. Available from: https://www.aomrc.org.uk/ebi/wp-content/uploads/2021/05/ebi-statutory-guidance.pdf.
- 14.
The Academy of Medical Royal Colleges on behalf of the Evidence-based Interventions Programme Board. Evidence-based Interventions List 3 clinical guidance Proposals 2022 [cited 2022 November]. Available from: https://www.aomrc.org.uk/ebi/wp-content/uploads/2022/01/EBI_List_3_cinical_guidance_Proposals.pdf.
- 15. Anderson M, Molloy A, Maynou L, Kyriopoulos I, McGuire A, Mossialos E. Evaluation of the NHS England evidence-based interventions programme: a difference-in-difference analysis. BMJ Qual Saf. 2022. Epub 20220407. pmid:35393354.
- 16. Herbert A, Wijlaars L, Zylbersztejn A, Cromwell D, Hardelid P. Data Resource Profile: Hospital Episode Statistics Admitted Patient Care (HES APC). Int J Epidemiol. 2017;46(4):1093–i. pmid:28338941; PubMed Central PMCID: PMC5837677.
- 17.
NHS Digital. National Clinical Coding Standards OPCS-4 (2021) 2021. Available from: https://classbrowser.nhs.uk/ref_books/OPCS-4.9_NCCS-2021.pdf.
- 18.
World Health Organization. International statistical classification of diseases and related health problems: tenth revision, 2nd ed 2004. Available from: https://apps.who.int/iris/handle/10665/42980.
- 19. Francetic I, Meacock R, Elliott J, Kristensen SR, Britteon P, Lugo-Palacios DG, et al. Framework for identification and measurement of spillover effects in policy implementation: intended non-intended targeted non-targeted spillovers (INTENTS). Implement Sci Commun. 2022;3(1):30. Epub 20220314. pmid:35287757; PubMed Central PMCID: PMC8919154.
- 20.
Office for National Statistics. United Kingdom population mid-year estimate 2022. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/timeseries/ukpop/pop.
- 21. Bottomley C, Scott JAG, Isham V. Analysing Interrupted Time Series with a Control. Epidemiological Methods. 2019.
- 22. Dorward J, Khubone T, Gate K, Ngobese H, Sookrajh Y, Mkhize S, et al. The impact of the COVID-19 lockdown on HIV care in 65 South African primary care clinics: an interrupted time series analysis. Lancet HIV. 2021;8(3):e158–e65. Epub 20210204. pmid:33549166; PubMed Central PMCID: PMC8011055.
- 23.
Office for National Statistics. Clinical Commissioning Group (April 2019) Bounderies EN BFC. 2019.
- 24. Standardization: a classic epidemiological method for the comparison of rates. Epidemiol Bull. 2002;23(3):9–12. pmid:12608346.
- 25. Appleby J, Raleigh V, Frosini F, Bevan G, Gao H, T L. Variations in health care: the good, the bad and the inexplicable 2011. 2011 [2022]. Available from: https://www.kingsfund.org.uk/sites/default/files/field/field_publication_file/Variations-in-health-care-good-bad-inexplicable-report-The-Kings-Fund-April-2011.pdf.
- 26. McPherson K, Wennberg JE, Hovind OB, Clifford P. Small-area variations in the use of common surgical procedures: an international comparison of New England, England, and Norway. N Engl J Med. 1982;307(21):1310–4. pmid:7133068.
- 27. Diehr P, Cain K, Connell F, E V. What Is Too Much Variation—the Null Hypothesis in Small-Area Analysis. Health Serv Res. 1990;24(6):741–71. pmid:2312306
- 28. Ibanez B, Librero J, Bernal-Delgado E, Peiro S, Lopez-Valcarcel BG, Martinez N, et al. Is there much variation in variation? Revisiting statistics of small area variation in health services research. BMC Health Serv Res. 2009;9:60. Epub 20090402. pmid:19341469; PubMed Central PMCID: PMC2676262.
- 29. Norton WE, Chambers DA. Unpacking the complexities of de-implementing inappropriate health interventions. Implement Sci. 2020;15(1):2. Epub 20200109. pmid:31915032; PubMed Central PMCID: PMC6950868.
- 30. Chambers JD, Salem MN, D’Cruz BN, Subedi P, Kamal-Bahl SJ, Neumann PJ. A Review of Empirical Analyses of Disinvestment Initiatives. Value Health. 2017;20(7):909–18. Epub 20170516. pmid:28712620.
- 31. Cliff BQ, Avancena ALV, Hirth RA, Lee SD. The Impact of Choosing Wisely Interventions on Low-Value Medical Services: A Systematic Review. Milbank Q. 2021;99(4):1024–58. Epub 20210817. pmid:34402553; PubMed Central PMCID: PMC8718584.
- 32. Burton CR, Williams L, Bucknall T, Fisher D, Hall B, Harris G, et al. Theory and practical guidance for effective de-implementation of practices across health and care services: a realist synthesis. Health Services and Delivery Research 2021;9(2).
- 33.
Nuffield Trust. Elective (planned) treatment waiting times 2022. Available from: https://www.nuffieldtrust.org.uk/resource/treatment-waiting-times.
- 34. Beard DJ, Rees JL, Cook JA, Rombach I, Cooper C, Merritt N, et al. Arthroscopic subacromial decompression for subacromial shoulder pain (CSAW): a multicentre, pragmatic, parallel group, placebo-controlled, three-group, randomised surgical trial. Lancet. 2018;391(10118):329–38. Epub 20171120. pmid:29169668; PubMed Central PMCID: PMC5803129.
- 35. Chesterton LS, Blagojevic-Bucknall M, Burton C, Dziedzic KS, Davenport G, Jowett SM, et al. The clinical and cost-effectiveness of corticosteroid injection versus night splints for carpal tunnel syndrome (INSTINCTS trial): an open-label, parallel group, randomised controlled trial. Lancet. 2018;392(10156):1423–33. pmid:30343858; PubMed Central PMCID: PMC6196880.
- 36. Harris C, Allen K, Brooke V, Dyer T, Waller C, King R, et al. Sustainability in Health care by Allocating Resources Effectively (SHARE) 6: investigating methods to identify, prioritise, implement and evaluate disinvestment projects in a local healthcare setting. BMC Health Serv Res. 2017;17(1):370. Epub 20170525. pmid:28545430; PubMed Central PMCID: PMC5445482.
- 37. Harris C, Allen K, Ramsey W, King R, Green S. Sustainability in Health care by Allocating Resources Effectively (SHARE) 11: reporting outcomes of an evidence-driven approach to disinvestment in a local healthcare setting. BMC Health Serv Res. 2018;18(1):386. Epub 20180530. pmid:29843702; PubMed Central PMCID: PMC5975394.