Study design

The EPOCH (Evaluating a Point of Care device in Heart healthcare) trial is a pragmatic, cluster randomised controlled trial in general practice. We incorporated a practice interview and questionnaire at the end of the trial to assist in interpreting the findings. The outcome of interest was completion of CVD risk assessments. Intervention practices were trained to use a cobas b 101 POC device in addition to usual care and the control practices conducted usual care alone.

Setting and participants

All 38 general practices in the Northland region were invited to participate in the trial. Patients enrolled in these practices were included if they were 35–79 years old, had no previous history of CVD (including previous stroke, angina, myocardial infarction, peripheral vascular disease, cardiac bypass or other coronary or arterial procedures) and met national guideline criteria for an initial CVD risk assessment or follow-up assessment[7], as determined by their GP. The recommended frequency of follow-up CVD risk assessments varies from five yearly to annually (people with diabetes, those previously found to have high CVD risk [5-year CVD risk ≥15%], or people taking CVD medications).

Practices were randomised centrally by a University of Auckland biostatistician using a computer-generated list, with stratification by location (rural vs urban) and size of practice (small, [< two full time equivalent {FTE} doctors], or large [≥2 FTE doctors]). Study investigators, practice and PHO staff were unable to intervene in the randomisation procedure or allocation. At the end of the trial, the control practices were offered a POC device if desired.

Sample size

Sample size calculations (standard two-sample t-test with normal approximation for a binomial distribution) drew on estimates of the number of people eligible for a CVD risk assessment (first or a follow-up assessment) and projections of regional CVD risk assessments for a baseline rate of 70% in the control group (personal communication, C Wiltshire, Enigma Solutions Ltd) and an effect size of 15% (an absolute increase to 85%) in the POC group. The effect size estimate was based on a study set in Canadian community pharmacies where POC testing for cholesterol risk management led to a 10% increase.[15] Assuming an intraclass correlation coefficient of 0.025 (from three New Zealand cluster randomized controlled trials[16]) a 5% two-sided level of significant test and power of 90%, we estimated 1000 individual patient risk assessments would be required (on average 50 patients per practice) to establish superiority of the intervention over control conditions.

Intervention training and quality assurance

We implemented a comprehensive general practice internal quality control programme (IQC) according to the requirements of the ISO 22870: 2006(E) standard and the NZ Best Practice Guidelines for Point-of-care Testing[17] together with external quality assurance(EQA) from two New Zealand laboratories (Canterbury Health Laboratory [CHL] and Northland District Health Board Laboratory [NDHBL]). Both laboratories are accredited by International Accreditation New Zealand on the medical testing standards ISO 15189: 2012 and ISO 22870:2006(E) for POC testing. Members of each general practice team who would use the cobas b 101 device were trained by staff from Roche Diagnostics NZ Ltd for approximately 1.5 to 2 hours on finger prick testing, capillary sampling, use of the device, routine maintenance and ongoing quality requirements. IQC included paired testing of samples to ensure that staff were achieving results on the POC device comparable to standard laboratory testing. The cobas b 101 machine was electronically linked to the practice computer systems to transfer results directly to the patient’s own electronic health record. General practice staff were instructed to use the POC device solely for CVD risk assessments (and not for monitoring of glycaemic control or lipids).

Technical problems

Prior to commencing the trial, CHL tested several batches of cobas discs against laboratory analysers and found results of acceptable standard (unpublished data). EPOCH began on 12 August 2013 with practice training. However, on 29 August 2013 the trial was halted following an assay precision problem identified with the batch of cobas b 101 HbA_1c discs being used in the trial. The investigators notified all practices to stop using the POC device and informed the ethics committee. The 22 individuals who had been tested for HbA_1c with the POC device were advised by their practices to have a laboratory HbA_1c test. Roche Diagnostics International Ltd instituted a global recall and identified a manufacturing fault. Three months later, a new batch of HbA_1c discs became available. This batch was then validated in two independent New Zealand laboratories (LabTests Ltd and CHL). On the basis of these results and data from NDHBL, it was determined that it was safe to resume the trial following retraining and recertification of the 10 practices. After ethical approval, the trial restarted in May 2014. The practices were required to continue monthly paired, IQC tests using two levels of the manufacturer’s quality control solutions. In addition three EQA survey samples with known concentrations of HbA_1c and lipids were distributed by NDHBL for analysis on the POC device at each practice over the course of the trial and three additional EQA surveys occurred for external accreditation. The results confirmed good performance of the Roche cobas b101 system using Clinical Laboratory Standards Institute EP-5 and EP-9 protocols.[18] The IQC data showed CV levels of 5.2% and 4.2% for the level 1 and level 2 quality control solutions. Reference laboratory CV’s for level 1 and level 2 quality control solutions at different HbA_1c levels varied between 2.0% and 4.9%.

However, by the time the trial restarted, the two PHOs in Northland had documented CVD risk assessment completions for approximately 80% of their eligible enrolled populations (personal communication C Wiltshire, Enigma Solutions Ltd.). Therefore we revised the analysis plan and adopted a non-inferiority hypothesis (that the proportion of CVD risk assessment completions in the POC practices would be no less than that in the control practices) at the restart. Our revised sample size was much the same as previously calculated.

Outcomes and follow-up

The primary outcome was a completed CVD risk assessment conducted for eligible patients in the 20 participating practices. A completed risk assessment was defined as containing all co-variates necessary to calculate a CVD risk assessment score and to screen for diabetes. Two study periods were defined: eight months prior to the trial period (01 September 2013 to 30 April 2014) and 12 months after intervention (01 May 2014 to 30 June 2015). Secondary outcome measures included; incomplete CVD risk assessment that had been electronically saved (e.g. awaiting patient CVD risk factor data such as blood tests); and time (in days) to complete CVD risk assessment. We measured differences in completion rate by practice location (urban/rural), practice size (large/small), age group, gender, Māori/non Māori, and deprivation (NZDep) quintile. The NZDep Index Score, a measure assigned to a patient’s area of residence based on nine variables from the Census reflecting eight dimensions of relative deprivation of census tracts,[19] ranges from 1 to 10, but for these analyses they were aggregated into quintiles (1 to 5, from least to most deprived).

Data collection

All practices in Northland have a web-based CVD risk assessment and management decision support system (called “PREDICT”) integrated within their electronic health records (EHR).[20,21] When a GP or practice nurse opens a PREDICT on-line form within a patient’s EHR, the software automatically pulls in relevant clinical and demographic data from the EHR. This can then be checked and any missing or incorrect fields can be updated by the clinician. The following variables are required fields for calculating a patient’s 5-year CVD risk: date of birth, sex, ethnicity, prior diagnosis of CVD, prior diagnosis of diabetes (type 1, type 2 or type unknown), atrial fibrillation, a self-reported family history of premature ischaemic CVD, smoking status, systolic and diastolic blood pressure (mean of two measures), and TC/HDL (one measure). Ethnicity was defined according to a national prioritisation protocol[22] in the following order; Māori, Pacific, Indian and NZ European combined with other ethnicities (Euro/Other). Family history of premature CVD was defined as a self-reported familial history of ischaemic heart disease or ischaemic stroke occurring in a father or brother before 55 years of age, or a mother or sister before 65 years of age. Screening for diabetes (either HbA_1c or fasting glucose was made a compulsory field after national guideline updates in 2009.[23] Practices with the POC device could either use the device or the community laboratory to provide TC/HDL or HbA_1c results for their patients.

Once a completed form is submitted by the clinician to the web server, a risk score is computed and returned to the clinician. This score and the patient risk profile are stored both in the EHR and on the centrally hosted secure webserver. If screening blood tests or other patient data are not available the assessment can be ‘parked‘(i.e. saved on the webserver and in the EHR until all data are subsequently available).

Blinding

While it was not possible to blind practices to having the POC device or not, all study investigators (except GH) were blinded to the practice allocation until the end of the trial when one investigator visited the practices, interviewed staff and administered the questionnaire. GH developed and supervised the quality assurance programme, set QA testing protocols and fed back results to practices. Other than this study staff had no influence, or involvement in, how the device was used in the practice, or the collection or recording of clinical data. All study data collection was conducted through the PREDICT programme as part of routine clinical practice. At the end of the trial de-identified data from complete and incomplete assessments from the participating practices were extracted and sent to the investigators for analyses.

Practice interviews and questionnaires

Two short questionnaires were designed for intervention and control practices to assess current CVD risk assessment procedures, resources for blood testing within practices and the real (intervention) or expected (control) experience of POC testing. At the end of the trial, one investigator (SW) conducted face-to-face interviews or telephone calls to administer the POC- or control-specific questionnaires with the key person/s who led the CVD risk assessment performance in each practice Informant comments were written down and checked for accuracy and completeness with the participants at the time of the interview. Data were de-identified to ensure confidentiality. Braun and Clarke’s approach to thematic analysis was used to generate initial codes, collate codes into potential themes and refine the identified themes and categories into a coherent patterm.[24]

Statistical analysis

Statistical analyses were performed on the intention to treat principle using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA). All statistical tests were two-tailed at the 5% significance level. Continuous variables were presented as mean and standard deviation with categorical variables presented as frequency and percentage. Generalized linear mixed models were used in treatment evaluations, adjusting for the random effect of clustering and important baseline prognostic factors. The point estimate and 95% confidence interval were reported as appropriate to the link function for a continuous (identity) or categorical (logit) outcome variable. The intra cluster correlation (ICC) coefficient was calculated on the primary outcomes using the observed trial data.

We have reported the RCT component of the study according to the CONSORT statement, and have included the qualitative results so as to provide a richer understanding of the important contextual factors.[25]

Ethics statement

Written consent was obtained by all participating practices. The EPOCH study was initially approved by the Northern B Health and Disability Ethics Committee (Reference 13/NTB/79). The committee was informed of the trial being halted in August 2013 and subsequently gave approval to recommence the trial in April 2014.

Main outcomes

Having a POC testing device within the practices made no difference to the completion of CVD risk assessments. During the trial period 99.5% of CVD risk assessment templates in both groups had complete data (Table 2). Furthermore, given that over 99% of the CVD risk assessments were completed on the same day, there were no statistically significant differences in the other secondary outcomes (incomplete CVD risk assessments, time [in days] to complete CVD risk assessment or differences by size of practice or rurality or by patient characteristics).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Data completion in CVD risk assessments in the 12 months post randomisation.

https://doi.org/10.1371/journal.pone.0174504.t002

Questionnaire and interview findings

Practice interviews took place at the end of the trial, 13 face-to-face and six via telephone call. There were 10 informants (eight practice nurses, one practice manager and one practice assistant) from the nine POC practices and 10 informants (nine practice nurses, one practice manager) from the 10 control practices. Three overall themes emerged; i) the importance of context, specifically usual policy, procedures, staff time and resources ii) expected usefulness of the POC device from the control group versus actual ease of use and usefulness from POC group and iii) how POC testing might fit into practices in the future.

The context of CVD risk assessment in both POC and control practices

In this study involving 19 practices, 90% of the eligible patients had already had a CVD risk assessment by the time the POC device was re-introduced.[6] The emphasis was on undertaking a CVD risk assessment for all patients in the practice population within five years, rather than ensuring population groups with high CVD risk had the recommended frequency of monitoring. In most practices, patients with long term conditions or multi-morbidity were being seen at no charge (due to PHO-funded initiatives) by the practice nurse and blood tests arranged prior to or at the time of the visit. In addition, four out of nine POC and five out of 10 control practices routinely collected blood samples on their patients on-site.

One of the major barriers for practices to reach their targets for CVD risk assessments was difficulty getting the last 10% of eligible people to attend. If these patients did come in they often required a full set of laboratory investigations, not just HbA_1c and TC/HDL. The time taken to perform venepuncture (intervention and control practices) was estimated as being around 15 minutes—similar to that reported by the practices for use of the cobas b 101 machine.

Expected usefulness of POC device for CVD risk assessment (control practices)

Many (8/10) of the control nurses were keen to use the POC device once the trial was completed.

“Especially if people can't get to the laboratory. Catch them while they are in the practice. Hard to reach group especially. At the moment patient goes to laboratory, has blood taken, fax / report results, then nurses have to ring on the phone and it takes a lot of time. If patient has POC test, they are right there in front of you. All done and dusted". (Control Practice nurse)

“rather than nurses spending two hours in the afternoon trying to contact patients with blood test results—then unsure if patient understands what the nurse is saying. Much quicker and then finished face to face.” (Control Practice nurse)

Actual ease of use and usefulness (POC practices)

The most common concern (8/10 POC respondents) about ease of use was that POC testing was not a good use of their time. Work flow changed as a result of having the device as nurses needed to get a sample at the start of a patient consultation. Nurses reported feeling so time-pressured with their workload that having to wait by the machine and the sequential nature of the analysing (one test taking five minutes then the next test taking five minutes) was a negative aspect they had not anticipated.

“It was the waiting 5 minutes—we can do a lot of things in 5 minutes but we were tied to the machine” (POC Practice nurse)

Additional time was required to conduct the monthly quality assurance measures that, if not done, would lock them out from further testing. The other practical limitation for practices was finding a space for the POC device. Usually it was kept in one of the nurse’s consultation rooms which precluded others from using it. The POC device was seen as an additional tool rather than an opportunity to review practice work flow and systems for CVD risk assessment. The present time was more highly valued than the future time gained by the immediacy of getting a result and starting discussions on heart health. Follow-up with patients (from both POC and control groups) to inform them about their laboratory blood tests results and CVD risk assessment typically would take an additional 15–30 minutes depending on the patient CVD risk profile.

The two intervention practices that found the device most useful had purposely changed their systems and allocated extra staff time and space for the POC testing.

“We decided to run a nurse clinic with dedicated time…. This worked much better than fitting things in opportunistically- working on the floor nurses are stressed and stretched and the time it took to do the cobas test was really disruptive.”” (POC Practice nurse)

Several nurses commented that opportunistic screening using POC testing would be most useful for younger people (e.g. <40 years) and men.

“particularly younger ones, 45-55yrs, Māori men loved it—instant result” (POC Practice nurse)

Seven out of ten respondents volunteered that the finger prick was better from a patient perspective than venepuncture and seemed to smooth the path to engagement.

“its good for some people who are hard to get to the clinic- we use it as a first point of engagement—after that we would take full bloods” (POC Practice nurse)

Furthermore the ability to get results almost immediately meant that discussion about the meaning of the CVD risk score could happen without delay.

“We loved the tool as we got results back immediately and so could start the conversation” (POC Practice nurse)

Place of POC testing for HbA_1c and TC/HDL in the future

Most practices (8/10 control and 7/9 POC) indicated that the place of POC testing in the practice was monitoring (rather than screening), especially for patients with diabetes in nurse- led chronic disease clinics. However, the majority of respondents (7/9 POC practices and 5/10 control practices) reported that they would not conduct POC testing if the cost of the consumables (the lipid and HbA_1c discs) was borne by the practice.

Conclusion

The introduction of a POC testing device in urban and rural New Zealand general practices was not associated with increased completion of electronic CVD risk assessment templates. An external policy change exerted a far greater influence on practitioner behaviour.