The effect of electronic monitoring feedback on medication adherence and clinical outcomes: A systematic review

Objective This study aims to assess the efficacy of Electronic Monitoring Feedback (EMF) as an intervention to improve medication adherence (i.e. dose- or full adherence) and clinical outcomes in adult patients. Methods A systematic search was performed in Medline, EMBASE, PsycINFO and Web of Science and reported according to the PRISMA guidelines. Randomised controlled trials (RCTs) comparing EMF with usual care were identified to systematically summarise the evidence for use of EMF in improving medication adherence and clinical outcomes. The GRADE approach was used to assess the quality of the body of evidence. Results Of 9,993 initially-identified studies, ten studies (four of high-quality and six of low-quality) were included. The sample size of the studies included varied from 18 to 205 patients. Four of the six studies (66.7%) reported a significant positive effect of EMF on mean dose adherence levels, whereas a significant positive effect of EMF on mean full adherence levels was found in all of the included studies (100%, five out of five of the studies included). A significant positive effect of EMF on clinical outcomes was reported in one of the seven studies included. The overall effect of EMF on mean dose- and full adherence was positive and the overall effect of EMF on clinical outcomes was inconclusive. Conclusion Considering the positive effect of EMF on medication adherence, EMF might be a promising intervention to enhance medication adherence. However, the effect of EMF on clinical outcomes was inconclusive. Prior to implementing EMF in clinical practice, future research with high-quality studies (e.g. adequate sample sizes, follow-up periods and no interfering co-interventions) is required to examine the (long-term) efficacy of EMF.


Introduction
Since drugs can only provide effect when actually taken, non-adherence to medication might worsen clinical outcomes, decrease quality of life, and will eventually lead to increased healthcare costs [1,2]. It is estimated that approximately 50% of all patients who are taking medication for chronic diseases do not adhere to their prescribed medication [3,4]. For five different diseases (hypertension, chronic obstructive pulmonary disease, chronic back pain, depression and rheumatoid arthritis), it has been estimated that productivity loss due to non-adherence to medication leads to annual costs of €28 to €50 billion in the United Kingdom, €38 to €75 billion in Germany and €9 to €13 billion in the Netherlands [2]. Therefore, interventions to enhance medication adherence in clinical practice are urgently needed.
Medication adherence is defined as the extent to which a patient's medication-intake behaviour corresponds with the medication regime prescribed by the healthcare professional [1,4,5]. Medication-taking behaviour should however not be understood as a single decisional process, but as a three-phase process: the initiation phase (patients must initiate treatment), the implementation phase (patients must implement the right dosing regimen) and the continuation phase (patients must persist with the treatment) [6]. In this review, we will focus on medication adherence. Medication adherence can be expressed in dose-and full adherence. Dose adherence is defined as the ratio between the number of doses taken and the number of doses prescribed, whereas full adherence is defined as dose adherence in the correct time schedule [7]. Several reasons for non-adherence to medication might exist in each of these three phases.
A way to identify non-adherent patients is to directly monitor a patient's adherence. Electronic devices, such as the Medication Event Monitoring System (MEMS), have been used for several years to measure medication adherence and are seen as a gold standard method to assess patient's medication-taking behaviour [8]. MEMS have the ability to register both doseand full adherence, whereas other methods, including questionnaires and refill adherence data, only provide data on dose adherence. The MEMS is an electronic device that consists of a MEMS cap with an electronic circuit that records opening of the MEMS bottle [3,7]. The electronically registered opening intervals offer healthcare professionals insight into the medication-taking behaviour of their patients.
Alongside this, the results from medication-intake behaviour recorded by electronic devices might also be used as an intervention to confront patients with detrimental medication-taking behaviour in particular. Data obtained from these devices provide the opportunity to give patients personalised feedback about their medication-taking behaviour and to intervene in cases of non-adherent behaviour. In cases of adherent behaviour, this technique also provides the opportunity to encourage patients to continue this behaviour [1,7,9]. Such personalised feedback is also known as Electronic Monitoring Feedback (EMF) [7]. In this study, EMF will be classified as instant EMF or non-instant EMF. The latter refers to EMF given by a healthcare professional or researcher after a certain period of use. This type of feedback can be used to address both intentional and unintentional non-adherent behaviour. A disadvantage of noninstant feedback is the evaluation of medication-intake behaviour over a longer time frame, which makes this type of feedback less appropriate for immediate intervention in cases of nonadherent behaviour.
In contrast to non-instant feedback, instant feedback allows to immediate intervention if patients are not taking their medication. MEMS devices with a MEMS-view-cap are considered to provide such instant feedback. A MEMS-view-cap shows the number of MEMS bottle openings over the past 24 hours and enables patients to visualize the number of doses taken. However, this type of feedback addresses only unintentional non-adherent behaviour, whereas intentional non-adherent behaviour is disregarded. As a comprehensive definition for EMF is lacking, we suggest the following: EMF is any personalised feedback (instant or non-instant) on previously electronically obtained adherence data to improve (intentional and/or unintentional non-adherent behaviour), or sustain medication adherence.
Although individual studies suggest a positive effect of EMF on medication adherence and clinical outcomes, results from these studies have not yet been systematically summarised. Therefore, this study aims to systematically summarise the evidence obtained in randomised controlled trials for the efficacy of EMF to enhance medication adherence and clinical outcomes in adult patients.

Methods
This systematic review, which assess the efficacy of EMF on medication adherence and clinical outcomes, follows the PRISMA-guidelines (S1 Checklist) [10,11].

Eligibility criteria
Eligible studies were: (1) randomised controlled trials; (2) aimed at enhancing medication adherence and, if assessed, clinical outcomes by EMF (3) in adult patients (aged 18 years or older) compared to usual care. Additional inclusion criteria for these RCTs were: 1. Only studies with EMF as the primary intervention component were included. Co-interventions were only permitted if these co-interventions incorporated a form of personalised feedback that was based on electronically obtained adherence data (for instance, a nonjudgmental discussion of the results, identifying barriers and facilitators for (non-)adherent behaviour, problem detection, problem solving techniques, clarifying treatment regime etc.). Instant or direct feedback from monitoring devices itself was also considered as personalised feedback. Studies in which EMF was used to assess the efficacy of an intervention on medication adherence were excluded. Examples of studies that were excluded based on this latter criterion were: Apter  2. Patients in the control group received entirely usual care (the use of electronic devices to measure adherence levels in the usual care group was permitted).
3. Medication adherence (i.e. dose-or full adherence) was the primary outcome measure and clinical outcomes were the secondary outcome measures.
Articles without an abstract and/or access to the full text manuscript were excluded. Studies that focused on institutional care were also excluded. There were no restrictions on language, sample size, diseases/conditions or duration of the follow-up period and articles were not excluded based on missing data regarding primary-and secondary outcomes. So, we focused on both chronic-and acute medication.

Search strategy
A literature search for this systematic review was performed up until August 8 th , 2017, in PubMed database, PsycINFO, EMBASE and Web of Science. A detailed search strategy can be found in S1 Appendix. Reference lists of included studies were checked to ensure literature saturation.

Study selection
Three reviewers (AH, BB and MH) independently selected eligible studies based on their titles and abstracts. Subsequently, full text evaluation of all the potentially eligible studies was independently carried out by the same reviewers (AH, BB and MH). Any disagreements about the final study selection were discussed to achieve consensus.

Data collection process
Relevant data available from eligible studies were collected in a data-sheet. The following data were extracted: • Study characteristics: author names, title, date of publication, journal, country in which the study was performed, study design, sample size, study duration and patient characteristics (gender, age, disease and prescribed medication).
• Intervention characteristics: type of intervention, type of (electronic) device, duration of the (pre)intervention period, duration and frequency of feedback, study medication, medication use and feedback provider.
• Outcome measures: medication adherence (i.e. dose-and full adherence) measured with an electronic device (including mean, standard deviation and 95% confidence interval) and data related to clinical outcomes.
Data extraction was conducted by CH; uncertainties were resolved by AH. All outcome data were verified by BB. P-values 0.05 were considered as statistically significant. In case of missing information regarding dose-and full adherence, authors were contacted to request the missing data for these primary outcomes.

Outcome measures and analysis
Medication adherence at the end of the intervention period registered with an electronic device was defined as the primary outcome measure, whereas clinical outcomes were defined as secondary outcomes in this systematic review. The effect of EMF on medication adherence per study was considered positive (+) if a significant effect was found on dose-or full adherence. In case no significant effect of EMF on medication adherence was found, the overall effect was considered negative (-). If studies did not report levels of significance or if results were contradictive, the overall effect per study was assessed as not available (NA) or neutral (+/-), respectively. The effect of EMF on clinical outcomes per study was considered positive (+) when a significant effect was found on at least 66.7% of the reported clinical outcomes in that individual study. For that purpose, we used results on clinical outcomes at the end of the intervention period or the first results after the intervention had been stopped. Otherwise, individual studies were scored as negative (-) or not available (NA) when clinical outcomes or levels of significance were not assessed. Finally, the overall effect of EMF on medication adherence and clinical outcomes was defined as positive if at least 66.7% of the total included studies reported positive results on medication adherence or clinical outcomes. Consensus of these percentage cut-offs was achieved in a consensus meeting (BB, EE and MH). Clinical heterogeneity was assessed by examining types of participants, interventions and outcomes in each study. Two reviewers (BB and MH) assessed clinical heterogeneity during a consensus meeting.

Quality of evidence
Apart from the data extraction a quality assessment was performed. The criteria for systematic reviews of the Cochrane Collaboration Back Review Group and the criteria described by Hayden et al. were used to assess the risk of bias of the included studies [16,17]. All included studies were independently evaluated by three reviewers (AH, BB and MH). These three individual quality assessments were discussed until consensus was reached. In order to obtain a final quality score, criteria were classified according to critical and less critical criteria during a consensus meeting (attendees: AH, BB and MH). This consensus meeting resulted in a quality assessment model that distinguishes between criteria that are critical and less critical to assess the risk of bias in studies focused on the efficacy of EMF on medication adherence and clinical outcomes. For instance, criteria focused on blinding of patients and blinding of care providers were less appropriate to assess as a risk of bias, due to the inherent awareness of being monitored with the EMF-devices by patients and care providers. The quality assessment model is provided in S1 Table. Studies that scored both ! 75% positive on the critical criteria and ! 50% positive on the less critical criteria were classified as high-quality studies. These percentage cut-offs were discussed and agreed during a consensus meeting.
Although the initial quality of evidence of randomised controlled trials is high, five explicit criteria of the GRADE approach might downgrade the total body of evidence for each patient outcome: risk of bias, inconsistency, indirectness, imprecision, and publication bias [18][19][20][21][22][23][24]. Factors that might increase the quality level of evidence are a large magnitude of the studied effect, a dose-response gradient, or the fact that all plausible confounding would reduce the demonstrated effect or suggest a spurious effect when results show no effect [19]. See S2 Table. for a detailed description of methods used to assess the quality of evidence based on these criteria. The GRADE approach was only used for the primary outcome measures. Eventually the body of evidence for each outcome was categorised in one of the specified categories: high, moderate, low, or very low [18][19][20][21][22][23][24].

Study selection
The literature search provided a total of 7,350 unique records that were screened for eligibility. After screening titles and abstracts 7,253 articles were excluded, which led to a total of 97 articles to screen as full-text. Of the 97 articles that were assessed for eligibility, ten articles had no abstract or full text available. An overview of the study selection process is shown in Fig 1. Ten RCTs met the inclusion criteria and were included in this systematic review. Reference lists of these included studies were checked, but resulted in no additional records that met the inclusion criteria.

Study characteristics
An overview of the sample and intervention characteristics of the included studies is presented in Table 1.

Sample characteristics
The main differences between studies were sample size and length of follow-up. Also the condition of the study population in which the intervention was studied varied considerably (asthma, smoking cessation, renal, heart, liver and lung transplantation, hypertension, HIV, secondary hyperparathyroidism and patients at risk from cardiovascular conditions) [25][26][27][28][29][30][31][32][33][34]. However, the efficacy of EMF in HIV positive patients and smoking cessation was reported by two studies for both conditions [26;28;30;31]. Most studies used MEMS as an electronic device to monitor medication adherence, except for Onyirimba et al., Brath et al. and Dobbels et al., in which metered-dose inhalers, electronic medication blisters and the Helping Hand tool with a traffic light system (green, orange and red light) were used, respectively [25,32,34]. De Geest et al. also used an electronic device, but this device was not further specified [27].    [32]. Instant feedback was also provided with the Helping Hand reminder system which consists of a beeping signal at time of scheduled medication intake [34]. Variability in costs of study medication and socioeconomic status of participants were also noted.
Six studies had dose adherence as the primary outcome, while only three studies reported both full-and dose adherence. Santschi et al. reported median adherence levels, but not mean adherence levels on dose-and full adherence [29]. The effect of EMF on clinical outcomes was reported by seven studies (three studies as primary outcome and four studies as secondary outcome). Brath et al. only reported the effect of EMF on clinical outcomes at the beginning and at the end of the crossover phase, thereby, representing inappropriate data for including these data in our overview [32]. An overview of the effect of EMF on medication adherence and/or clinical outcomes is presented in Table 2.

Feedback characteristics
The feedback varied between studies regarding the duration of a single feedback session, feedback provider, type of electronic device (metered dose inhalers, electronic medication blisters, Helping Hand, MEMS with or without 24h LCD readout), total intervention duration, the number of interventions in the specified study period and the components of the feedback. Most studies used a graphic form to confront patients with their medication-intake behaviour, and described face-to-face feedback [25][26][27][28][29][30][31][32][33][34]. However, De Geest et al. and Brath et al. used telephone calls to give feedback in the follow-up period [27,32]. Most studies did not describe the precise method of giving feedback to patients, as conversations did not follow a structured way. Only Forni Ogna et al. and Dobbels et al. used a semi-structured motivational interviewing method to discuss the results with patients in the intervention arm [33,34]. The monitoring phase prior to delivering feedback also varied between studies [25][26][27][28][29][30][31][32][33][34]. In the study of Schmitz et al. patients received graphic feedback after a week of monitoring, whereas in the study of de Bruin et al. the feedback was delivered after three months of monitoring [26,31]. The following issues were highlighted most during a feedback session: identification of potential barriers, strategies to remove barriers, exploration of strategies to improve adherence, verification of medication side-effects and evaluation of the previously elaborated adherence plan at the next visit.

Quality assessment
Of the ten studies included in this systematic review, four studies met the criteria for highquality studies, as described in the Methods section. Studies classified as low-quality studies were studies with shortcomings on statistics, treatment allocation, protocol adherence, and description of withdrawal or drop-out rate. An overview of the performed quality assessment, including scores for critical and less critical criteria, is represented in Table 3.

Summary of the results: Medication adherence
Ten randomised controlled trials studied the effect of EMF on medication adherence. Of these ten studies, Brath et al. used a crossover design without a 'wash-out period' which made these results difficult to interpret, since the intervention effect from the monitoring phase could still exist in the control phase [32]. Santschi et al. and Brath et al. reported median adherence levels instead of mean adherence levels compared with other included studies [29,32]. In the study of Santschi et al., only the intervention group was monitored with MEMS devices [29]. An  overview of the effect of EMF on mean dose adherence and full adherence levels is presented in Table 2.
The effect of EMF on mean dose adherence levels after the intervention period was studied in six randomised controlled trials. Of these six studies, four studies reported a significant positive effect of EMF on mean dose adherence levels. De Geest et al. reported a non-significant effect between both groups and Forni Ogna et al. did not describe the level of significance between the intervention and control arm [27,33]. Considering the number of included studies that described a significant positive effect (66.7%, four out of six included studies) of EMF on dose adherence, the overall effect of EMF on mean dose adherence levels was positive. A significant positive effect of EMF was also reported on mean full adherence levels. All of the five studies (100%) that described the effect of EMF on mean full adherence levels reported a significant positive effect.
Overall, the studies included showed a positive significant effect of EMF on dose (66.7%) and full adherence levels (100%), since at least 66.7% of the reported adherence outcomes at the moment of the last intervention or the first results after the intervention has been stopped were significant.
The high degree of heterogeneity in intervention characteristics and sample characteristics prevented pooling data from individual studies.
The strength of the body of evidence for the efficacy of EMF on medication adherence was very low (for a detailed description of the quality rating process, see S3 Table).

Summary of the results: Clinical outcomes
Seven randomised controlled trials studied the effect of EMF on clinical outcomes [25,[28][29][30][31][32][33]. Of these seven studies, four studies (HIV progression, secondary hyperparathyroidism and asthma) reported a significant effect of EMF on at least one clinical outcome [25,30,31,33]. An overview of the effect on clinical outcomes due to EMF is presented in Table 2. Four studies reported positive as well as negative findings on clinical outcome measures [25,30,31,33]. These findings also varied over time, as described by Forni Ogna et al., which reported a significant effect on intact parathyroid hormone level at month six, but no significant effect at month nine [33].
Although some studies reported a significant effect on at least one clinical outcome due to EMF, only one out of the six studies included had a positive overall score (at least 66.7% of the reported clinical outcomes at the moment of the last intervention or the first results after the intervention had been stopped were significant, as presented in Table 2, last column). The The effect of EMF on medication adherence and clinical outcomes high degree of clinical heterogeneity across studies prevented pooling of data from individual studies.

Summary of evidence
This systematic review demonstrates that EMF tends to have a positive effect on medication adherence (mean dose-and full adherence levels) and an inconclusive effect on clinical outcomes. The majority of the studies included were characterised by a low methodological quality and a high degree of clinical heterogeneity (i.e. patient, disease and study characteristics,).
In addition, the diversity in intervention characteristics (i.e. no or semi-structured interviews/ feedback, different feedback providers, different electronic devices, different frequencies of the intervention) might have contributed to the lack of evidence for the efficacy of this intervention. These study characteristics prevented pooling of data on adherence and clinical outcomes. However, the overall positive effect of EMF on medication adherence supports the hypothesis that EMF might be a promising strategy to enhance both dose-and full adherence.

Literature
To our knowledge, this is the first systematic review that summarises the single effect of EMF on medication adherence and clinical outcomes. Our results confirm the findings of Demonceau et al. (a systematic review including interfering co-interventions and, therefore, not examining the single effect of EMF) that EMF-feedback is a potentially effective approach to enhance patient's adherence to medication [7]. Although improving clinical outcomes and patient's quality of life are still the ultimate goals of adherence-improving interventions, clinical outcomes and/or quality of life are often not chosen as primary outcomes in adherence-improving studies [3]. We did not find evidence for the efficacy of EMF on clinical outcomes. This lack of evidence is most likely due to clinical heterogeneity across the studies included and small sample sizes.
There is also a time-to-event component in longitudinal adherence-improving studies. Although the effect of EMF on medication adherence is often expressed in dose-and full adherence, persistence and non-persistence may occur during this monitoring period. Measuring dose-and full adherence over a long period to provide feedback to patients might be inappropriate. However, these methodological insights in adherence-improving studies are slowly implemented in intervention studies. Therefore, we decided to study the effect of EMF on dose-and full adherence as primary outcomes in this review.
In contrast to the Cochrane review of Nieuwlaat et al., we did not exclude studies with 80% follow-up [3]. Our review included studies with a loss to follow-up range that varied from 6 to 56% (three studies would meet the inclusion criteria used in the Cochrane review). In this regard, our review provides a more realistic appraisal of the effect of EMF as a high degree of loss to follow-up often exists in adherence-improving studies and might also be the case in usual care.

Strengths and limitations
The most important limitation of this review is the lack of a worldwide definition regarding the content of electronic monitoring feedback, as no criteria for the minimum requirements of the intervention are described. This broad definition is considered to be the main reason for the high degree of heterogeneity in intervention characteristics of the included studies, making it either very complex to compare the efficacy of EMF on adherence in the different studies, and also reducing the possibility to assess the impact of the different intervention characteristics on the efficacy of EMF. We considered it not appropriate to perform additional analyses to evaluate the influence of heterogeneity in both intervention and study characteristics (including the quality of studies) due to the small number of studies included in this review, and therefore, a possible incorrect interpretation of the results.
In general, there is always a chance of publication bias, as positive studies are more likely to be published compared to negative studies. This might lead to an overestimation of the actual effect.
Also the use of electronic devices in the control arm (without the non-instant feedback component) in adherence-improving studies might increase patient's awareness of adequate medication-intake and consequently improve medication adherence. This awareness might decrease the contrast between the intervention and control arm, and thus effect sizes between groups. Furthermore, the type of electronic device might have influenced adherence levels, as MEMS with LCD displays (or the Helping Hand tool) provide instant feedback, whereas MEMS devices without LED displays provide no instant feedback. Feedback by a provider instead of a MEMS device has the advantage that both intentional and unintentional nonadherent behaviour can be targeted. Nevertheless, feedback by a provider after a (long) monitoring period might be less effective than instant feedback by a MEMS device in case of unintentional non-adherent behaviour. In addition, patients participating in a randomised clinical trial, irrespective of the allocated study arm and the use of an electronic device, might influence the results as usual care in a RCT is not usual care by definition.
At-home assistance is another potential concern of adherence-improving studies as adherence levels in both arms may be overestimated. Nearly all published adherence-improving studies do not evaluate or describe support from significant others (e.g. spouses and/or homecare). Therefore, there is often a lack of ability to control for support in the home context. However, support of significant others can also be used as an intervention component to improve medication adherence. For instance, in the study of Wu et al. the support of significant others was an essential part of the intervention [15]. Based on these general limitations of adherence-improving studies, individual studies are often characterized by a low methodological quality.
For the quality assessment of each study included in our review, consensus was achieved to differentiate the quality criteria in critical and less critical criteria. As all involved researchers agreed in a consensus meeting that an unequivocal majority of the critical quality criteria should be positive, a cut-off level of 75% was set for the critical criteria in order to assign a study as high-quality study. However, if a cut-off level of 50% for critical criteria had been agreed, six of the ten studies would have been assessed as high-quality studies. However, despite this, the strength of the body of evidence assessed with the GRADE approach did not change and remained very low.

Future research
This review has demonstrated that EMF might be a promising strategy to enhance dose-and full adherence, although high-quality, sufficiently powered studies are required in future research. As a significant effect of EMF on clinical outcomes cannot yet be excluded, future research with optimal study methodologies is required.
Future high-quality research should firstly focus on the short-and long-term efficacy of EMF on medication adherence. Alongside efficacy, other aspects, such as the effect of the intervention giver, the duration of the intervention, and the patient follow-up should be assessed. The latter is essential as non-adherence to medication is a dynamic process and patients might become non-adherent when the intervention is not continued. Finally, when a significant effect of EMF on medication adherence has been shown in multiple high-quality studies, further research should focus on the efficacy of EMF on clinical outcomes and the implementation of EMF in clinical practice.

Conclusions
This systematic review suggests that EMF might be a promising intervention in improving medication adherence. However, the efficacy of EMF on clinical outcomes is inconclusive. Considering the positive effect of EMF on medication adherence (mean dose-and full adherence levels) and the inconclusive results on clinical outcomes, evidence for the efficacy of EMF on both outcomes is limited. As our results are mainly based on low-quality studies, future research with high-quality studies is required to examine if EMF has significant effect on medication adherence. Only when a significant effect of EMF on medication adherence has been shown, studies should focus on the efficacy of EMF on clinical outcomes. Implementing EMF in clinical practice is not yet recommended.