Introduction

Clinical and health services research continually produces new evidence that can benefit patients. However, this evidence does not reliably find its way into everyday patient care [1]. This gap between evidence and clinical practice is a strategically important problem for policy-makers, healthcare systems and research funders because it limits the health, social and economic impacts of research [2].

There are particular implementation challenges specific to primary care that are not encountered in other settings. These include rising workloads and demands upon practice: Hobbs et al. have shown substantial increases in both the number and duration of practice consultations in UK primary care over the period 2007–14 [3]. This needs to be considered alongside the complex management of escalating numbers of ageing and multimorbid patients [4], and rising public expectations [5], all in the context of limited practice organisational capacity and continual reforms of general practices [6, 7].

These system strains are compounded by multiple, often competing implementation priorities. In 2012, we identified 107 National Institute for Health and Care Excellence (NICE) clinical guidelines relevant to primary care, as well as 114 specific quality indicators. Together, these yielded a total of 2365 clinical practice recommendations [8]. Several recent studies have identified the problems that may arise when attempting to apply guidelines to patients with multiple morbidities [e.g. 9, 10, 11]. These contextual factors go some way towards explaining the well-documented variations in the delivery of primary care that are consistently observed in the United Kingdom (UK) and internationally [12, 13]. In England, there is evidence that the overall standard of care is improving [14, 15] but the variation between general practices remains [e.g. 16, 17, 18].

‘Big data’ offers considerable promise in healthcare research [19, 20]. One potential opportunity is in the measurement of quality of care delivered across populations. A prime example is the NHS Atlas of Variation in Healthcare [21], which illustrates large geographical variations across England in the level and quality of care in several clinical areas, including diabetes, stroke and cancer. The magnitude of these differences cannot be easily explained by population and casemix factors. Indeed, the Foreword to the Atlas notes that the variation caused by “the idiosyncratic practices of clinicians and of healthcare organisations” represents unwarranted variation (p.15). Variation per se is not necessarily inappropriate; however, the variation examined here is considered ‘inappropriate’ as it means that many patients are generally not receiving recommended care. This is distinct from variation in care where there is not a recommended standard.

Enhanced delivery of care can have substantial implications. For example, it has been estimated that in England more than 7000 strokes per year could be prevented, and 2100 lives saved, with increased adherence to guidelines on the management of atrial fibrillation and use of recommended therapy [22].

Clearly, quality improvement initiatives cannot focus on all clinical practice recommendations at once; there is a need to identify and prioritise those with the potential for the most positive impact for patients. In earlier work [8], we identified a set of evidence-based quality indicators (QIs) based on criteria including: burden of illness (e.g. prevalence, severity); potential for significant patient benefit (e.g. longevity, quality of life); scope for improvement upon current levels of achievement; the extent to which following a recommendation is directly within the control of individual practice teams; and the feasibility of measurement using routinely collected data. This work yielded a set of QIs that we termed ‘high impact’.

We examined achievement against a set of these QIs in a sample of practices in a region of the UK. We aimed to examine the extent to which variations in achievement to high impact indicators can be explained using routinely collected data.

Methods

Variables

All clinical data were routinely recorded by practices and amenable to remote extraction. Many of the variables were linked to conditions included in the UK Quality and Outcomes Framework (QOF) year 2012/13. QOF is a pay-for-performance scheme whereby general practices are remunerated according to achievement of targets reflecting the quality of care delivered by the practice [23]. Practice data collection for QOF operates on an annual cyclical basis from 1 April to 31 March. QOF has driven consistent electronic recording of data in general practice and practices routinely record data for incentivised conditions on investigations, prescriptions and intermediate outcomes, such as blood pressure, or glycated haemoglobin [24].

Our dependent variables were drawn from a set of eighteen ‘high impact’ QIs developed by an earlier, multidisciplinary consensus process [8]. The list was reduced to seven indicators (Table 1) following initial data collection and analysis. The level of scope for improvement and potential for intervention were of primary importance in the selection process. Other indicators were rejected due to concerns about the reliability of routinely collected data or uncertainty about the ongoing use of the clinical indicator.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Details of the clinical indicators included in the study.

https://doi.org/10.1371/journal.pone.0177949.t001

Four indicators focused on processes of care (e.g. prescribing or testing) and three on clinical outcomes (the achievement of recommended targets for the intermediate health outcomes of blood pressure, cholesterol and glycaemic control). Several indicator sets consisted of multiple individual recommendations that were then pooled to permit analysis of achievement. Two of these were composites, assessing the degree to which all recommended care processes or treatment targets were achieved for individual patients (e.g. blood pressure, cholesterol and glycaemic values within recommended ranges for patients with Type 2 diabetes [indicator 5: diabetes control]). Others were formed from combinations of individual recommendations, but could not be considered composites in the truest sense as they did not necessarily all apply to a single individual. For example, the risky prescribing indicator (indicator 2) combined nine individual instances of prescribing that might be considered high-risk; six concerned gastro-intestinal risks, two renal risks and one heart failure. These have been detailed elsewhere [25, 26]. Individual patients may have been exposed to one or more of the prescribing risks and the combined measure gave an overall signal of practice risky prescribing. This latter indicator was scored in the opposing direction to the others: lower values here generally suggest safer, more desirable practice, while higher values were desirable on all other indicators. When producing combined indicators, all indicators were equally weighted on the basis that they had all been through a priority-setting process [8] and were considered clinically important.

Independent variables included patient- and practice-level characteristics we hypothesised or recognised from earlier work to be associated with practice performance [27, 28]. At the patient level we examined demographic (age, gender and ethnicity), and illness variables. Illness was assessed by a measure of comorbidity: the number of QOF disease registers on which a patient appeared.

Practice-level variables comprised the number of practice partners (which served as a proxy of practice size), the number of salaried family physicians, and training status—all recorded as at April 2014. We used practice-level Index of Multiple Deprivation (IMD) scores. This measures area deprivation and is determined for each patient on the list, where available, and then averaged over the practice. Overall achievement in the QOF clinical domain (2012–13) was used as a proxy measure for overall quality of care. By providing a wide-ranging and objective measure of items recommended by professionals and patients, it is generally accepted that the QOF offers a good snapshot of quality [29]. Practice prescribing costs over the period were also collected, measured as cost per Age, Sex and Temporary Resident Prescribing Unit (ASTRO-PU) over the year 2012–13.

Two further practice-level variables were included: patient satisfaction (the proportion reporting that they would recommend the practice to others) and practice accessibility (the proportion reporting that they were able to speak with a family physician/nurse within 48 hours of approach).

Results

Participants

Eighty nine practices (78.1% of those approached) shared patient data. Of the 114 practices originally approached, 22 opted out of data sharing: one had closed, one had merged and it transpired that another provided care for an atypical population. Practices which declined participation only differed from participating practices in having a smaller mean number of family physicians (5 vs. 3.6; p = 0.05). The total number of patients in the denominator for each indicator ranged from 4,773 (anticoagulation in atrial fibrillation and risk of stroke) to 77,587 (blood pressure control in hypertension). Patient demography is summarised in Table 2. Practice size was indicated by the number of practice partners (mean [M] 3.7, standard deviation [sd] 2.3) and salaried family physicians (M = 1.3, sd = 1.8). Mean practice-aggregated Index of Multiple Deprivation score was 31.2 (sd = 11.9). Mean QOF 2012/13 performance across practices was 637.4 (sd = 27.6) and 20.2% of the sample were registered as training practices. Mean practice prescribing costs (total Net Ingredient Cost per 1000 ASTRO-PUs [Age, Sex and Temporary Resident Prescribing Unit] were £50.4 (sd = 9.5). Patient reported satisfaction (M = 76.5%, sd = 13.4) and practice accessibility (M = 53.5%, sd = 14.4) were obtained.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Patient characteristics for cross-sectional analysis of associations with achievement on selected quality indicators.

https://doi.org/10.1371/journal.pone.0177949.t002

Achievement of indicators

Median practice achievement of the indicators under consideration ranged from 43.2% (diabetes outcomes) to 74.2% (blood pressure control in chronic kidney disease (CKD)) (Tables 3 and 4). Median achievement of the risky prescribing indicator was 8.7%, but this was scored such that a low score was indicative of fewer instances of risky prescribing and was therefore desirable. Considerable between-practice variation in achievement existed on all indicators: the difference between the highest and lowest achievers was 26.3 percentage points for risky prescribing and 100 percentage points for anticoagulation in atrial fibrillation. The variation in achievement of the indicator examining blood pressure in hypertension is presented as an example (Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Practice achievement and 95% confidence intervals for blood pressure control in patients with hypertension.

https://doi.org/10.1371/journal.pone.0177949.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Practice and patient characteristics significantly associated with process indicators in cross-sectional analysis.

https://doi.org/10.1371/journal.pone.0177949.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Patient and practice characteristics significantly associated with outcome indicators in cross-sectional analysis.

https://doi.org/10.1371/journal.pone.0177949.t004

Discussion

We found marked variations between general practices in the achievement of clinically important quality indicators. The odds of patients receiving recommended care or achieving recommended treatment targets varied between two- and over ten-fold by indicator according to the practice attended. These marked variations were partly explained by a range of routinely available practice and patient variables; it is likely that much variation is related to clinical and organisational behaviours.

We have demonstrated inappropriate variations for a set of ‘high impact’, evidence-based indicators judged important by clinicians and patients [8]. We did anticipate suboptimal performance given that scope for improvement was an initial selection criterion for the indicators, although we did not know the actual extent of variation. Our findings are generally consistent with previous studies, in England [15, 17, 18] and worldwide [12, 13, 33]. Three of our indicators (diabetes processes and control, anticoagulation in atrial fibrillation) show similar variations in the most recent NHS Atlas of Variation [21]. These levels of inappropriate variation (i.e. unexplained by patient morbidity or preferences) are all the more striking given that several indicators were derived from QOF indicators and NICE guidelines [8]; whilst such initiatives might be necessary for quality improvement they are insufficient by themselves to effectively target inappropriate variations.

One main study strength was the use of opt-out recruitment, whereby sampled practices had to actively decline sharing of their data. This method can produce a larger sample whilst avoiding some of the biases often associated with opt-in recruitment, and thus may be more representative of typical general practices [31, 34].

Our study had several limitations. First, we considered quality of care from a single, technical perspective, i.e. achievement against selected clinical indicators. Other, important aspects of care include interpersonal elements and practice accessibility, and variation has been demonstrated here also (e.g. [13, 35]). Nevertheless, the indicators were the outcome of a rigorous consensus process [8] and we are confident of their importance and relevance to both clinicians and patients.

Second, the sample was restricted to practices using a specific computerised patient record system (SystmOne). Patient record systems are associated with small differences in achievement of incentivised targets [36]. We judged that this by itself was not a major limitation to generalisability as regional coverage was high (over 80% of practices at the time of recruitment).

Third, our study was limited to one geographical area, albeit one with practice characteristics broadly similar to English averages. Moreover, our opt-out approach to practice data sharing has helped ensure that the sample covers a typical range of practices and is likely to have reduced selection bias.

Fourth, routinely collected electronic data offers an efficient but flawed means of gathering large-scale information. Detailed checking of patient records was not possible and it was evident that coding errors were present. In particular, patient ethnicity was poorly recorded: 41% of the patient population in the sampled practices were recorded as ‘British or mixed British’, with a further 14% as ‘unknown’ or ‘ethnic category not stated’. This is a recognised problem with medical records [37] and made analysis of the variable and interpretation of its impact practically meaningless. The quality of research using ‘big data’ is highly dependent upon the quality of data entered, and the extent to which this represents care delivered.

Fifth, the use of combined indicators can mask varying performance between individual component indicators. Giving equal weighting to indicators can also be contentious [38]. However, there is no single agreed method of combining indicators and our methods were similar to those utilised elsewhere [12, 15, 17, 21, 33].

Sixth, we utilised a rather crude measure of deprivation (practice-averaged IMD) that did not consider deprivation at the individual patient level and is likely to have masked differences in variation across practice populations. For reasons of patient confidentiality, we were unable to utilise more precise measures of deprivation.

Given that we were assessing a range of process and outcome indicators concerning different patient populations and clinical behaviours, we would naturally expect different patterns of associations. Nevertheless, we did identify some shared associations, particularly around age, and gender and comorbidity.

Men were generally more likely than women to achieve indicators which is consistent with previous evidence that men tend to receive better care for cardiovascular disease and diabetes [39]. Females, however, were more likely to achieve recommended blood pressure targets in hypertension, and less likely to be the recipient of a risky prescribing combination.

Age had a more varied relationship with our indicators. Older individuals were more likely to receive diabetes processes of care and achieve recommended blood pressure targets. A potential explanation is that older patients with diabetes and hypertension may visit their practice more frequently and thus be exposed to more opportunities for the management of long term conditions. By contrast, two indicators more closely related to prescribing behaviour were less likely to be achieved in older patients. The oldest patients were less likely to receive anticoagulation for stroke prevention in atrial fibrillation. Concerns about frailty and bleeding risk may be relevant here, despite evidence of the benefits of this treatment for elderly patients [40]. Older patients were also less likely to receive recommended treatment for secondary prevention of MI. It may be that prescribing of the four different recommended medications (angiotensin-converting-enzyme inhibitor or angiotensin receptor blocker; aspirin or alternative antiplatelet; beta-blocker; and statin) is less straightforward than in those who are relatively young with fewer comorbidities and contraindications.

Risky prescribing was more likely in patients aged 40–79 years. Short-term prescription of an NSAID without gastro-protection, for example, may be considered less of a risk in younger, otherwise healthy patients. Risky prescribing was also significantly less likely in patients with more comorbidities (i.e. on at least three QOF registers). This result might be considered counter-intuitive if more comorbidities typically means being prescribed more medications, and thus at greater chance of exposure to risky prescription combinations [41]. However, a plausible explanation may be offered for our findings. As suggested in relation to the diabetes results, having more comorbidities—and thus the potential for hazardous medical combinations—may alert health professionals to the risks associated with NSAID prescribing. A study in primary care in Scotland similarly found that risky prescribing was less likely in patients with increasing numbers of repeat medications (as an indication of overall morbidity) [42].

Having a higher number of comorbidities was positively associated with achievement of the diabetes processes, diabetes control, and hypertension target indicators, together with reduced risky prescribing. This finding is consistent with other evidence of a positive relationship between the number of chronic conditions and quality of care [e.g. 43, 44]. The precise nature of the comorbid conditions in question is likely to be an important factor in such a relationship: there is a growing body of evidence to suggest that the presence of diabetes is associated with better preventive treatment of cardiovascular disease [45]. It is perhaps surprising, therefore, to find an inverse relationship between comorbidity and achievement of blood pressure targets in CKD.

Practice-level characteristics showed few associations with indicator achievement and wider evidence for their importance is inconsistent. For example, a systematic review of the impact of practice size upon quality of care suggested that larger practices were associated with improved care, but the evidence was mixed [28]. No association was found between deprivation (measured at the level of primary care organisations) and achievement of diabetes processes or outcomes in the NHS Atlas of Variation [21]. Notably, we found no associations with practice training status although earlier work has suggested an association with features of good quality care [46].

We have conducted parallel, qualitative work exploring health professionals’ views about the determinants of adherence to several of the indicators analysed here [47] shedding some light on the present findings. Interviewees recognised the importance of clinical recommendations from the population perspective, but were also clear that rigid adherence may not be desirable for individual patients (e.g. where someone was already prescribed multiple medications). Attention to the needs of individual patients was considered a general influence across clinical conditions and featured prominently in interviews. The desire to meet these needs (which may be perceived by the clinician rather than actual) was described as a strong driver of prescribing practice, in deciding whether to adhere to recommended targets in diabetes, and in considering whether to prescribe anticoagulation for stroke prevention in atrial fibrillation. Of course, 100% adherence would not be achievable, nor appropriate, but the beliefs of clinical staff may contribute to the observed differences between practices as well as those associated with patient age, gender and comorbidity.

A further possible factor is the role played by organisational culture. Focusing upon the diabetes control indicator, a recent systematic review of qualitative studies [48] offers potential explanations for the recognised between-practice variation. Several of the identified themes concerned professional and organisational components, including resource constraints, deficits in knowledge and skills and uncertainties about professional roles. Thus, higher-performing practices may be those with better structured management systems, access to specialist teams, and shared awareness of guideline recommendations. Elsewhere, the degree of variation observed is suggested to be related “predominantly to the ways in which services for people with diabetes are organised” [21, p.105]. Our finding that practices with higher QOF performance and those with better reported accessibility were more likely to achieve this indicator is consistent with the view that organisation and culture play an important role in performance.

Our work has three messages for the improvement of care and further research. First, the consistent, substantial variation between practices emphasises that this remains an important issue, with variation existing at similar levels to that assessed almost two decades ago [13]. It is concerning that the likelihood of receiving recommended care at different practices can vary by such magnitude. There remains a need to better understand the reasons behind these differences and develop interventions to narrow the gap between the lowest and highest performers. A small number of studies have examined variation at multiple levels, i.e. that occurring between practices and that between practitioners within practices [42]. This has shown how the role played by different types of variation differs by the outcomes under observation. Unfortunately, this level of analysis was not possible in the current study as individual clinicians could not be identified.

Second, the modest but significant associations between achievement and specific patient characteristics have implications for improvement strategies. Better performance on the diabetes, hypertension and risky prescribing indicators was associated with comorbidity. Thus, greater attention could be focused upon patients who are (relatively) healthier and consequently less likely to attend the practice. We notably did find an association between practice accessibility and achievement of the diabetes control indicator.

Third, the associations between patient and practice variables and indicator achievement suggests the importance of clinical and organisational behaviours. Qualitative research has shown that health professionals often believe that practice performance is heavily influenced by local casemix or demographic variables [47]. Research such as our own can combat such potentially erroneous perceptions. The potential importance of organisational culture is recognised [35, 49]. Our recent qualitative work also recognised the importance of internal practice norms and ways of working, e.g. shared understanding of professional roles and the delegation of responsibilities for different tasks and management decisions [47]. Partly due to difficulty of measurement, organisational culture or climate is often not considered but warrants further investigation. Further work is needed to elucidate the reasons for practice-level variation and to design and evaluate interventions targeting practice teams to encourage consistent delivery of evidence-based clinical care.

Several interventions to promote guideline implementation (e.g. audit and feedback, educational outreach) have modest if variable effects [50, 51]. We are presently conducting a cluster randomised controlled trial of an intervention to improve achievement of a subset of high impact indicators by targeting identified barriers to change [52].

Acknowledgments

The ASPIRE programme team comprises Susan Clamp¹, Amanda Farrin², Suzanne Hartley², Claire Hulme¹, Rebecca Lawton^3,4, Rosie McEachan⁴, Martin Rathfelder⁵, Judith Richardson⁶, Vicky Ward¹, and Ian Watt⁷, in addition to the named authors. The ASPIRE programme team can be contacted via Robbie Foy (r.foy@leeds.ac.uk). We would like to thank Helen Swapp and Joanne Staniforth for their clinical input and advice on the data searches; and Chris Jackson for completing the data extraction.

¹ Leeds Institute of Health Sciences, University of Leeds

² Leeds Institute of Clinical Trials Research, University of Leeds

³ School of Psychology, University of Leeds

⁴ Bradford Institute for Health Research, Bradford Royal Infirmary

⁵ Chairperson, ASPIRE Patient and Public Involvement Panel, Manchester

⁶ National Institute for Health and Care Excellence

⁷ Department of Health Sciences, University of York