The Bristol heart inquiry in the United Kingdom (UK) highlighted the lack of standards for evaluating surgical performance and quality. In 2009, the World Health Organisation (WHO) proposed six standardised metrics for surgical surveillance. This is the first study to collect and analyse such metrics from a cohort of National Health Service (NHS) Trusts in England, helping to determine their feasibility and utility in measuring surgical performance, its impact on public health and mortality, and for tracking surgical trends over time.
Freedom of Information Act 2000 (FOI) requests for WHO standardised surgical metrics were made to 36 NHS Trusts in England during July to November 2010. Additional data on Hospital Standardised Mortality Ratio (HSMR), Patient Safety Score and Abdominal Aortic Aneurysm (AAA) volume and mortality was obtained from Dr Foster Health and The Guardian Newspaper. Analysis was performed using mixed-effect logistic regression.
30/36 trusts responded (83%). During 2005–9, 5.4 million operations were performed with a 24.2% increase in annual number of operations. This rising volume within hospitals was associated with lower mortality ratios. A 10% increase in operative volume was associated with a lower day of surgery death rate (DDR OR = 0.94, p = 0.056) and post-operative inpatient 30-day mortality (PDR30 OR = 0.93, p = 0.001). For every 10,000 more operations that an NHS Trust does, a 4% drop in PDR30 mortality was achieved. A 10% increase in the volume of elective AAAs was associated with lower elective AAA (OR = 0.96, p = 0.032) and emergency AAA (OR = 0.95, p = 0.009) PDR30 mortality. Lower DDR mortality was noted for emergency AAA mortality (OR = 0.95, p = 0.025) but not elective AAAs (OR = 0.97, p = 0.116).
Standarised surgical metrics can provide policy makers and commissioners with valuable summary data on surgical performance allowing for statistical process control of a complex intervention. This study has shown their collection is feasible albeit using FOI and the first to show a statistically significant volume-outcome relationship for surgery as a whole within hospitals. It adds weight to the argument that patients are safer in larger hospitals or those that become larger by growing their patient base. Together with other measures, such metrics can help build a picture of surgical surveillance in the UK and potentially lead us to safer surgery.
Citation: Agha R (2012) Towards National Surgical Surveillance in the UK – A Pilot Study. PLoS ONE 7(12): e47969. https://doi.org/10.1371/journal.pone.0047969
Editor: John E. Mendelson, California Pacific Medicial Center Research Institute, United States of America
Received: April 19, 2012; Accepted: September 24, 2012; Published: December 11, 2012
Copyright: © 2012 Agha. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: No funding was sought or obtained. The study was supported by NICE as part of their scholarship programme. NICE helped provide links with data providers and general support/mentorship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The author has declared that no competing interests exist.
Public health surveillance has long relied on standardised metrics to quantify disease burden in a population, track mortality rates and guide health system programming, assessment and investment. For over half a century, such standard metrics have included; maternal mortality, infant mortality and life expectancy. Vaccination rates and treatment coverage for specific infectious diseases (e.g. Human Immunodeficiency Virus infection) are also being added to this growing pool. Cardiac surgeons have led the way in the open and transparent publication of named surgeon mortality results . However, there are currently no standardised metrics for surgical surveillance globally . Such metrics would allow for the assessment of the safety of perioperative care and surgical performance.
Globally an estimated 234 million major surgical operations are performed annually . This volume of procedures is thought to result in seven million complications and one million deaths - double the number of annual maternal deaths and resulting in 164 m disability adjusted life years (DALYs) . The average American will now undergo 9.2 surgical procedures in a lifetime and modern surgery includes everything from coronary artery bypass grafting to joint replacement and transplantation .
The public inquiry into children's heart surgery at the Bristol Royal Infirmary in the UK  drew attention to the lack of standards for evaluating and tracking surgical performance in the NHS and for assessing the quality of care. The need to monitor standards and benchmark outcomes in healthcare has been underscored more recently by several inquiries in the UK including; the Shipman inquiry , the Mid-Staffordshire inquiry  and reports into Basildon and Thurrock hospitals by the UK hospital regulator - the Care Quality Commission (CQC) .
In the UK, over the last 10 years surgical waiting lists have come down and the focus has shifted towards quality and safety of care . This focus is being shared more globally and in 2007, the World Health Organisation (WHO) launched an initiative: “Safe Surgery Saves lives”. One aspect of the programme was to develop standardised measures for surveillance of the volume of surgical care and its effect on public health outcomes over time. A technical working group consisting of experts in epidemiology, global health and surgical outcomes from around the world came together to develop standardised metrics for assessing surgical services . The group proposed six as follows (see Table 1.)
These standardised surgical metrics provide structure, process and outcome measures for evaluating healthcare . The WHO states that such data when combined with other routinely collected data could provide a baseline that can then be used to monitor surgical services on an annual basis. Thus allowing policy makers to quantify demand for surgical services, surgical safety, identify access barriers, track mortality rates, benchmark outcomes, provide an early warning system for poor performance and potentially assess the effects of new interventions. It would also provide Government, the NHS and healthcare authorities with a rich dataset that can be used to programme and reshape health services nationally.
The objective of this study was to retrospectively gather and analyse these standardised surgical metrics from a cohort of NHS Hospitals in England. This study would help assess the practical feasibility of gathering such metrics and help determine their utility in measuring surgical performance and its impact on public health and mortality, and for tracking surgical trends over time.
A sample of 36 NHS Trusts in England, representing 23% of Acute and Foundation NHS Trusts, were approached in late 2010 using a Freedom of Information (FOI) Act 2000  request for yearly data on the six standardised surgical metrics for the period 2005–9. NHS Trusts are in effect Public Sector Corporations consisting of one or more hospitals responsible for delivering care on behalf of the English NHS . NHS Trusts are designed to serve the need of the local population and have variable budgets and catchment areas. The Freedom of Information (FOI) Act was passed on 30 November 2000 in the UK and was part of the Government's commitment to greater openness in the public sector. It gives a general right of access to all types of recorded information held by public authorities with full access granted in January 2005. Information released under FOI undergoes a number of internal checks for accuracy prior to release.
Half were randomly selected (via a mouse scroll wheel technique on the NHS Choices website: http://www.nhs.uk/Pages/HomePage.aspx) and the other half were based on a convenience sample which had been previously approached (unsuccessfully with an 11% response rate) for data by a polite letter. Following this poor response it was felt that FOI would be a more robust approach for data. For the purposes of uniformity and in keeping with local practice, ‘Accredited Surgeons and Anaesthetists’ in the WHO definition was changed to Consultant Surgeons and Anaesthetists respectively. Consultant Surgeons and Anaesthetists in the UK are those that have passed fellowship or ‘exit’ exams of the established professional body; the Royal College of Surgeons and Royal College of Anaesthetists respectively, have successfully completed their training/residency with the award of a certificate of completion of training and been appointed to a substantive role as a Consultant to a specific NHS Trust for independent delivery of care to patients (the equivalent of an Attending in the USA).
Upon receipt of the dataset from the NHS trust, the information was sent back to the Medical Director for the same trust for them to check the data and verify its accuracy. Whilst Hospital Episode Statistics (HES) could have been used to provide mortality and operative volume data, they would not have been able to provide all the data (e.g. staffing and operating rooms) and there are ongoing concerns about HES data accuracy , . Hence a single request was thought to be more reliable and efficient and would make verification by the Medical Director simpler and more consistent.
Additional data were added from the UK broadsheet newspaper The Guardian and its ‘Safety in numbers for hospital patients’ FOI investigation on Abdominal Aortic Aneurysm (AAA) mortality . This provided useful raw data that the author could analyse and incorporate into this study rather than duplicating prior efforts. Hospital Standardised Mortality Ratio (HSMR) data together with Patient Safety Score (PSS) for 2008/9 for the same NHS trusts was sourced from Dr Foster Health (a provider of healthcare information) . The PSS relates to the statistical combination through z-scoring of 13 equally weighted indicators from the patient safety domain of Dr Foster's quality accounts, giving an overall measure of each hospital . The result is a score between 0 and 100, with 100 being the best. Dr Foster is a joint venture between the NHS Information Centre for health and social care and Dr Foster Holdings LLP, it has a code of conduct that prohibits political bias and is independently monitored. All data was stored and basic analysis performed in Excel® 2007 database (Microsoft, Redmond, WA, USA) under strict password controlled access.
All statistical models were developed in STATA v11.2 (available at www.stata.com, published by StataCorp LP, Texas, USA) and used mixed-effect logistic regression  incorporating both fixed and random effects  with p<0.05 considered significant. Models were developed to longitudinally analyse changes in the standardised surgical metrics across different hospitals and within hospitals over the five year period 2005–9. HSMR and PSS models were cross-sectional and used only 2008 data. In each case DDR was the outcome variable and the effect of increasing either HSMR or PSS by 10% was modelled. Both models contain a random effect for hospital. The capacity model (looking at volume of operations, number of surgeons and anaesthetists) uses five years of data with DDR again as the outcome. The independent effects of 10% changes in the following were modelled; volume of operations, surgeons per 10,000 operations, anaesthetists per 10,000 operations and rooms per 10,000 operations.
The models were also adjusted for year as a categorical variable. The hospital specific model includes a random intercept for hospital and a random slope for year (continuous). This model can be considered as modelling the within hospital change. The population average model does not contain any random effects but uses a sandwich estimator for standard errors and associated p-values and confidence intervals to allow for the clustering of repeated observations within hospitals. This can be considered as the between hospital effect.
The elective AAA and emergency AAA models use three years of data. For both models death rate is the outcome. The effect of a 10% increase in DDR adjusting for volume of elective AAA and year (categorical) was modelled. As above for the hospital specific models, this also includes a random intercept for hospital and a random slope for year (continuous) and the population average models use a sandwich estimator. In order to model a 10% change, the variable in question was log transformed and then divided by log (1.1) (natural logs in both cases). This transformed variable was used in the regression and the resulting odds ratio corresponded to a 10% change.
30 out of 36 NHS trusts approached responded to the FOI request with 27 providing a complete data set and three providing partial data (83% response rate). One NHS trust refused to provide data and the other five were unable to provide data despite multiple requests and reminders. Data validation letters were sent to 27 Medical Directors with a response received from 14. Of these 14, nine (64%) confirmed the data as accurate, four (29%) made adjustments to the mortality data and one (7%) refused to confirm the data.
WHO Standardised Surgical Metrics
Over the five-year period, 5.4 million operations were conducted and there was a 24.2% increase in the volume of operations over this period (figure 1 and table 2). This ranged for individual hospitals from −19.8% to +105%.
The data did not show evidence to support a consistent relationship between staffing levels; Consultant Surgeons (DDR OR = 1.00, CI = 0.94–1.06, p = 0.993), Consultant Anaesthetists (DDR OR = 1.00, CI = 0.94–1.07, p = 0.929) or number of operating rooms (DDR OR = 1.00, CI = 0.95–1.07, p = 0.897) and DDR and PDR30 outcomes (figure 2).
Cross-sectionally for any particular year, there was a range of DDRs and PDR30s across the hospitals sampled (see figures 3 and 4). These ratios consistently increase in some hospitals and decrease in others over the five-year period under study. Figure 4 illustrates how outliers can be identified outside of the 95% and 99% confidence intervals.
A 10% increase in volume of operations within and across hospitals over the five-year period was associated with lower DDR (OR = 0.94, CI = 0.87–1.00, p = 0.056) and PDR30 outcomes (OR = 0.93, CI = 0.90–0.97, p = 0.001). For every 10,000 more operations that an NHS Trust does a 4% drop in PDR30 mortality was achieved.
HSMR and PSS
There was no evidence to support a link between 2008 HSMR and surgical mortality DDR (OR = 0.95, CI = 0.73–1.23, p = 0.684) and PDR30 (OR = 0.93, CI = 0.72–1.20, p = 0.573) outcomes (modelled using a 10 point increase in HSMR). However, there was a significant association between 2008 Patient Safety Score (PSS) and surgical mortality DDR (OR = 1.18, CI = 1.03–1.35, p = 0.017) and PDR30 (OR = 1.19, CI = 1.05–1.34, p = 0.005) outcomes (modelled on a 10 point increase in PSS).
AAA Data Analysis
A 10% increase in the volume of elective AAAs (2006–8) was associated with lower elective AAA (OR = 0.96, CI = 0.92–1.00, p = 0.032) and emergency AAA (OR = 0.95, CI = 0.91–0.99, p = 0.009) PDR30 mortality. Lower DDR mortality was noted for emergency AAA mortality (OR = 0.95, CI = 0.92–0.99, p = 0.025) but not elective AAAs (OR = 0.97, CI = 0.93–1.01, p = 0.116).
To the author's knowledge at the time of writing, this is the first study to collect the WHO standardised surgical metrics in the UK. Gathering such data is feasible and potentially useful but was found to be difficult without resorting to use of FOI. The demand upon surgical services increased by 24.2% over the period 2005–9 with each NHS Trust carrying out an average of 7,400 more operations in 2009 than in 2005. This trend for increasing demand is consistent with the work of Wieser et al who have shown how demand is increasing globally  and this is supported by the work of others . Over this period staffing levels have risen in concert with the greater volume of operations. The relatively small increase in operating theatres in the face of this rise in volume is probably the result of better theatre utilisation and possible waiting list initiatives, such as elective theatre lists at weekends .
The finding that a 10% increase in the volume of elective AAAs was associated with lower elective and emergency AAA mortality is consistent with previous volume-outcome data already reported for specific surgical procedures like elective or urgent open AAA repair ,  elective endovascular AAA repair  and carotid endarterectomy , . Such data has led increasingly to the call for a national reconfiguration of vascular services into a centralised ‘hub and spoke’ model of care ,  with all major arterial interventions to be performed in high volume specialist centres where all vascular surgeons present exceed minimum case volumes criteria . Holt et al have also found that high volume centres are more likely to operate on ruptured AAAs , perhaps due to the greater number and confidence of the operating surgeons and support staff, access to the latest technology and intensive therapy unit facilities . Awopetu et al found that higher-volume centres were associated with reduced amputation and mortality rates post lower limb vascular surgery although they did point to significant heterogeneity within their data .
The result that a 10% increase in operative volume (across all surgical specialties) within and across hospitals was associated with lower DDR and PDR30, shows that volume-outcome relationships may extend across the surgical service as a whole, not just for vascular surgery. To the author's knowledge this is the first study to show such an effect. Such data again point to the need for high volume centres and the reconfiguration of surgical services on a national basis, particularly for those operations that involve significant morbidity and mortality. However, mortality and patient safety are not the only considerations in this debate, with patient preferences, travel times and low volume centres which perform well also providing policy makers with much food for thought . It is no surprise that such reconfigurations have been put on hold by the government .
Such data can help to determine outliers and may prompt further study of potential underlying reasons  especially when there is a consistent longitudinal trend. This is facilitated by the strong signal to noise ratio that high risk surgery provides .
Indeed, Poloniecki et al  examined false alarm rates and mortality at a regional cardiothoracic centre and advised gathering such data on an ongoing basis. They called for the introduction of hospital mortality monitoring groups to routinely monitor and chart all deaths in a hospital by specialty.
Aylin et al  did similar work in primary care where they studied 1,009 family physicians. Of these, 33 (including Harold Shipman) crossed the alarm threshold designed to detect a two standard deviation increase in standardised mortality, with a 97% successful detection and a 5% false-alarm rate. It may be plausible to look at such a system at the hospital or NHS Trust level in a prospective manner using statistical process control methodology. It could also complement and be a useful addition to the broad corpora of data that is already collected on a routine basis by agencies like the Care Quality Commission, National Patient Safety Agency, the Department of Health, the NHS, the National Audit office and the office for National Statistics.
There was no evidence of a link between HSMR and surgical mortality. In the UK, Canada, the Netherlands and the United States, HSMR has been used for many years within organisations to monitor performance and response to various quality and safety programs . It has also been used by a variety of stakeholders to compare performance between hospitals . The data demonstrates that HSMR should not be used as a surrogate of surgical performance or quality per se between hospitals. This may be a reflection of how HSMR is calculated and weighted. The variation in non-surgical death and surgical case-mix appears to be too large to see any direct relationship between HSMR and surgical mortality. However, increasing PSS was paradoxically linked with an increase in surgical mortality. This may reflect how this scoring is weighted more towards medical patients who outnumber surgical patients and who are more strongly represented in overall mortality figures. PSS has been abandoned by Dr Foster as a metric in its most recent hospital guides .
There are several limitations to this study. The sample was not fully randomised and had initially been designed to give a spread of NHS trusts across England. The mortality data was not case-mix adjusted and there is likely to be significant variation here across England. However this is much less likely to vary significantly for the same institution over the five year period for which data was collected, hence the impact on longitudinal analyses will be limited. The author agrees with the statement by Weiser et al  when proposing the metrics, that these are “not metrics of quality but rather of the effect of surgery on public health and mortality, and for tracking surgical trends over time.”
It should be noted that some NHS trusts did not have such data readily available requiring additional time to locate, summate and verify it and 10% of NHS Trusts who responded could not provide a complete set of data, missing key operational information like number of operations performed in a year and mortality rates. This may reflect the fact that such surveillance data is not routinely collected at present. This culture needs to change. The recent report; The Higher Risk General Surgical Patient  by the Royal College of Surgeons of England and the UK Department of Health advocated a national audit of outcome as one of its key recommendations and that this could be used to address high variations in outcome across the country. Since 2001, when surveillance and mandatory reporting began for Methicillin-resistant Staphylococcus Aureus (MRSA) and Clostridium difficle infections across the NHS, their rates have continued to fall . Recent data shows how outcomes in cardiac surgery have improved since outcome data has been published .
WHO Standardised surgical metrics together with other measures can help build a picture of surgical surveillance in the UK, which could be repeated on an annual basis and help build an increasingly rich corpora of surveillance. The policy implications of such types of research are important. Potentially, such an exercise could provide the NHS with rich annualised data on a national level to assess surgical volume, safety, identify barriers to access, provide a baseline to benchmark and track changes over time. Such surveillance has been shown to be of value in the recent metal on metal prosthesis scandal and could have been useful in detecting the high rupture rates of Poly Implant Prothese (PIP) breast implants earlier , . Furthermore, the establishment of such a baseline will provide a means by which the efficacy and cost-effectiveness of policy interventions can be evaluated (e.g. the effect of rationalising services to higher volume centres of excellence). This data can be mined and utilised in health services research aimed at improving surgical services and will help to guide the NHS and national health care organisations in resource allocation and the programming of services.
Standarised surgical metrics could provide policy makers and commissioners with valuable summary data on surgical performance allowing for statistical process control of a complex intervention. This is the first study to shown how their collection is feasible albeit using FOI and the first to show a statistically significant volume-outcome relationship for surgery as a whole within hospitals. It adds weight to the argument that patients are safer in larger hospitals or those that grow their patient base significantly as shown by previous work in Vascular Surgery.
Such metrics when expanded and potentially combined with other data sources could provide a data armamentarium and foundation for building a ‘performance mosaic’ for surgical services in England to help monitor demand, benchmark standards, guide rationalisation of services and identify gaps in resource allocation or safety and quality. Future work in this area includes analysis of WHO metrics data in other countries, further investigation of the link between overall surgical volume and mortality outcomes with deeper statistical process control analysis at the NHS trust level. This may have an impact on the high volume rationalisation of healthcare services taking place in the UK and in other countries for trauma and major surgery.
The author would like to thank Professor Martin Roland for his mentoring and support and Dr Gary Abel for his assistance with the statistical analysis, both at the Cambridge Centre for Health Services Research, University of Cambridge. Thanks to Mr Barry White and Dr Peter Littlejohns at the National Institute for Health and Clinical Excellence for their mentoring during the scholarship period. Thanks also go to Dr Gareth Goodier, Chief Executive Officer of Cambridge University Hospitals NHS Foundation Trust for his support of this project.
Conceived and designed the experiments: RA. Performed the experiments: RA. Analyzed the data: RA. Contributed reagents/materials/analysis tools: RA. Wrote the paper: RA.
- 1. Bridgewater B (2005) Adult Cardiac Surgeons of North West England (2005) Mortality data in adult cardiac surgery for named surgeons: retrospective examination of prospectively collected data on coronary artery surgery and aortic valve replacement. BMJ 5;330 (7490) 506–10.
- 2. Weiser TG, Makary MA, Haynes AB, Dziekan G, Berry WR, et al. (2009) Standardised metrics for global surgical surveillance. Lancet 374 (9695) 1113–7.
- 3. Weiser TG, Regenbogen SE, Thompson KD, Haynes AB, Lipsitz SR, et al. (2008) An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet 372: 139–44.
- 4. Lee P, Regenbogen S, Gawande AA. How many surgical procedures will Americans experience in an average lifetime?: Evidence from three states [online]. Available: http://www.mcacs.org/abstracts/2008/P15.cgi. Accessed 2012 Aug 19.
- 5. Kennedy I (2001) Learning from Bristol: the report of the public inquiry into children's heart surgery at the Bristol Royal Infirmary 1984–1995.
- 6. Shipman Inquiry. Sixth Report - Shipman: The Final Report. Published 27 January 2005 [online]. Available: http://www.shipman-inquiry.org.uk/finalreport.asp. Accessed 2012 Jun 6.
- 7. The Mid Staffordshire NHS Foundation Trust Inquiry [online]. Available: http://www.midstaffsinquiry.com. Accessed 2012 Jun 6.
- 8. Care Quality Commission. CQC tells Basildon and Thurrock University Hospitals NHS Foundation Trust further improvement needed or enforcement will follow [online]. Available: http://www.cqc.org.uk/media/cqc-tells-basildon-and-thurrock-university-hospitals-nhs-foundation-trust-further-improvement. Accessed 2012 Jun 6.
- 9. Keogh B (2010) Keynote speech. Presented at BaSIS: Building Safety Improvement Skills Course, NHS Institute for Innovation and Improvement, Marriot Hotel, Central London, 3 March 2010. Course details Available: http://www.institute.nhs.uk/safer_care/safer_care/basis%3A_building_safety_improvement_skills.html. Accessed 2012 Jun 6.
- 10. Donabedian A (1966) Evaluating the quality of medical care. Milbank Mem Fund Q 44: 166–203.
- 11. Information Commissioner's Office. Freedom of Information Act 2000 [online]. Available: http://www.ico.gov.uk/for_organisations/freedom_of_information.aspx. Accessed 2012 Jun 6.
- 12. NHS Choices. About the NHS [online]. Available: http://www.nhs.uk/NHSEngland/thenhs/about/Pages/authoritiesandtrusts.aspx. Accessed 2012 Jun 6.
- 13. Head RF, Byrom A, Ellison GT (2008) A qualitative exploration of the production of Hospital Episode Statistics in a Guernsey hospital: implications for regional comparisons of UK health data. Health Serv Manage Res 21 (3) 178–84.
- 14. Patrick H, Sims A, Burn J, Bousfield D, Colechin E, et al. (2006) Monitoring the use and outcomes of new devices and procedures: how does coding affect what Hospital Episode Statistics contribute? Lessons from 12 emerging procedures. J Public Health (Oxf) 2012 Jul 17.
- 15. Guardian Newspaper. Safety in numbers for hospital patients [online]. Available: http://www.guardian.co.uk/society/2010/jun/13/specialist-surgery-increases-survival. Accessed 2012 Jun 6.
- 16. Dr Foster. The Doctor Foster Hospital Guide 2009: How Safe is Your Hospital, Edited by Bedford Z and Kafetz A, 2009 [online]. Available: http://www.drfosterhealth.co.uk/docs/hospital-guide-2009.pdf. Accessed 2012 Jun 6.
- 17. Dr Foster. Quality accounts – methodology [online]. Available: http://www.drfosterhealth.co.uk/quality-accounts/methodology.aspx. Accessed 2012 Jun 6.
- 18. Stoddard GJ. Biostatistics and Epidemiology Using Stata: A Course Manual. Salt Lake City, UT: University of Utah School of Medicine. Chapter 2-1 [online]. Available: http://www.ccts.utah.edu/biostats/?pageId=5385. Accessed 2012 Jun 6.
- 19. Rabe-Hesketh S, Skrondal A (2008) Multilevel and Longitudinal Modeling Using Stata, 2nd ed, College Station, Tx, Stata Press.
- 20. Kang WL, Balakrishnan A (2006) Operative workload of a paediatric otolaryngology department over a five-year period. Singapore Med J 47 (7) 614–6.
- 21. NHS Institute for Innovation and Improvement. The Productive Operating Theatre [online]. Available: http://www.institute.nhs.uk/quality_and_value/productivity_series/the_productive_operating_theatre.html. Accessed 2012 Jun 6.
- 22. Holt PJ, Poloniecki JD, Loftus IM, Michaels JA, Thompson MM (2005) Epidemiological study of the relationship between volume and outcome after abdominal aortic aneurysm surgery in the UK from 2000 to 2005. Br J Surg 2007;94 (4) 441–8.
- 23. Young EL, Holt PJ, Poloniecki JD, Loftus IM, Thompson MM (2007) Meta-analysis and systematic review of the relationship between surgeon annual caseload and mortality for elective open abdominal aortic aneurysm repairs. J Vasc Surg 46 (6) 1287–94.
- 24. Holt PJ, Poloniecki JD, Khalid U, Hinchliffe RJ, Loftus IM, et al. (2009) Effect of endovascular aneurysm repair on the volume-outcome relationship in aneurysm repair. Circ Cardiovasc Qual Outcomes 2 (6) 624–32.
- 25. Holt PJ, Poloniecki JD, Loftus IM, Thompson MM (2007) Meta-analysis and systematic review of the relationship between hospital volume and outcome following carotid endarterectomy. Eur J Vasc Endovasc Surg 33 (6) 645–51.
- 26. Holt PJ, Poloniecki JD, Loftus IM, Thompson MM (2007) The relationship between hospital case volume and outcome from carotid endartectomy in England from 2000 to 2005. Eur J Vasc Endovasc Surg 34 (6) 646–54.
- 27. Holt PJ, Poloniecki JD, Hinchliffe RJ, Loftus IM, Thompson MM (2008) Model for the reconfiguration of specialized vascular services. Br J Surg 95 (12) 1469–74.
- 28. Karthikesalingam A, Hinchliffe RJ, Poloniecki JD, Loftus IM, Thompson MM, et al. (2010) Centralization harnessing volume-outcome relationships in vascular surgery and aortic aneurysm care should not focus solely on threshold operative caseload. Vasc Endovascular Surg. 44 (7) 556–9.
- 29. Thompson M, Holt P, Loftus I (2011) Part one: All major arterial interventions should now be performed in high volume centres–abdominal aortic aneurysms. Eur J Vasc Endovasc Surg 42 (4) 411–4.
- 30. Holt PJ, Karthikesalingam A, Poloniecki JD, Hinchliffe RJ, Loftus IM, et al. (2010) Propensity scored analysis of outcomes after ruptured abdominal aortic aneurysm. Br J Surg 97 (4) 496–503.
- 31. Karthikesalingam A, Hinchliffe RJ, Poloniecki JD, Loftus IM, Thompson MM, et al. (2010) Centralization harnessing volume-outcome relationships in vascular surgery and aortic aneurysm care should not focus solely on threshold operative caseload. Vasc Endovascular Surg 44 (7) 556–9.
- 32. Awopetu AI, Moxey P, Hinchliffe RJ, Jones KG, Thompson MM, et al. (2010) Systematic review and meta-analysis of the relationship between hospital volume and outcome for lower limb arterial surgery. Br J Surg 97 (6) 797–803.
- 33. Thompson M, Holt P, Loftus I, Forbes TL (2011) Debate: whether abdominal aortic aneurysm surgery should be centralized at higher-volume centers. J Vasc Surg 54 (4) 1208–14.
- 34. Boseley S, Zorlu G, Evans R (2010) Huge disparity in NHS death rates revealed. Guardian Newspaper, 13 June 2010. Available: http://www.guardian.co.uk/society/2010/jun/13/nhs-death-rates-huge-disparity. Accessed 2012 Jun 6.
- 35. Holt PJ, Poloniecki JD, Thompson MM (2008) How to improve surgical outcomes. BMJ 336 (7650) 900–1.
- 36. Ettinger WH, Hylka SM, Phillips RA, Harrison LH Jr, Cyr JA, et al. (2008) When things go wrong: the impact of being a statistical outlier in publicly reported coronary artery bypass graft surgery mortality data. Am J Med Qual 23 (2) 90–5.
- 37. Poloniecki J, Sismanidis C, Bland M, Jones P (2004) Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups. BMJ 328 (7436) 375.
- 38. Aylin P, Best N, Bottle A, Marshall C (2003) Following Shipman: a pilot system for monitoring mortality rates in primary care. The Lancet 362 (9382) 485–91.
- 39. Scott IA, Brand CA, Phelps GE, Barker AL, Cameron PA (2011) Using hospital standardised mortality ratios to assess quality of care — proceed with extreme caution. MJA 194 (12) 645–648.
- 40. Dr Foster. Inside your hospital: Dr Foster Hospital Guide 2001–2011 [online]. Available: http://drfosterintelligence.co.uk/wp-content/uploads/2011/11/Hospital_Guide_2011.pdf. Accessed 2012 Oct 10.
- 41. Dr Foster. Hospital Guide 2011 [online]. Available: http://drfosterintelligence.co.uk/wp-content/uploads/2011/11/Hospital_Guide_2011.pdf. Accessed 2012 Jun 6.
- 42. The Royal College of Surgeons of England and the Department of Health. The Higher Risk General Surgical Patient: Towards Improved Care for a Forgotten Group [online]. Available: http://www.rcseng.ac.uk/publications/docs/higher-risk-surgical-patient. Accessed 2012 Jun 6.
- 43. Department of Health. New objectives set to reduce MRSA and C Difficile, published 13 January 2012 [online]. Available: http://www.dh.gov.uk/health/2012/01/mrsa-cdifficile-objectives. Accessed 2012 Jun 6.
- 44. Bridgewater B, Grayson AD, Brooks N, Grotte G, Fabri BM, et al. (2007) North West Quality Improvement Programme in Cardiac Interventions (2007) Has the publication of cardiac surgery outcome data been associated with changes in practice in northwest England: an analysis of 25,730 patients undergoing CABG surgery under 30 surgeons over eight years. Heart 2007 Jun;93 (6) 744–8.
- 45. Smith AJ, Dieppe P, Vernon K, Porter M, Blom AW (2012) National Joint Registry of England and Wales (2012) Failure rates of stemmed metal-on-metal hip replacements: analysis of data from the National Joint Registry of England and Wales. Lancet 379 (9822) 1199–204.
- 46. Department of Health. Poly Implant Prothèse (PIP) silicone breast implants: Review of the actions of the Medicines and Healthcare products Regulatory Agency (MHRA) and Department of Health [online]. Available: http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_134044. Accessed 2012 Aug 14.