Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Effectiveness of short message services and voice call interventions for antiretroviral therapy adherence and other outcomes: A systematic review and meta-analysis

  • Isaac Amankwaa,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Graduate School of Nursing, Midwifery & Health, Faculty of Health, Victoria University of Wellington, Wellington Regional Hospital, Wellington, New Zealand

  • Daniel Boateng ,

    Roles Data curation, Formal analysis, Investigation, Methodology, Software, Writing – original draft, Writing – review & editing

    Daniel.dboateng@gmail.com

    Affiliations Julius Global Health, Julius Center for Health Sciences and Primary Care, University Medical Centre, Utrecht University, Utrecht, The Netherlands, School of Public Health, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana

  • Dan Yedu Quansah,

    Roles Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing

    Affiliations Service of Endocrinology, Diabetes and Metabolism, Lausanne University Hospital, Lausanne, Switzerland, Department of Biomedical Sciences, University of Cape Coast, Cape Coast, Ghana

  • Cynthia Pomaa Akuoko,

    Roles Data curation, Investigation, Writing – review & editing

    Affiliations Christian Service University College, Kumasi, Ghana, School of Nursing, Queensland University of Technology, Brisbane, Australia

  • Catrin Evans

    Roles Conceptualization, Data curation, Methodology, Writing – review & editing

    Affiliation School of Health Sciences, The University of Nottingham, Queen's Medical Centre, Nottingham, United Kingdom

Abstract

Background

The potential of using mobile phone technologies to improve antiretroviral therapy (ART) adherence has provided a new facet to human immunodeficiency virus (HIV) research. The quality of evidence and the strength of recommendations of existing reviews, however, do not adequately support large-scale adoption of the intervention. This review adopted broad selection criteria to include all mobile phone-based interventions designed to improve patient’s adherence to ART.

Methods

We performed a systematic review and meta-analysis of randomized controlled trials and quasi-experimental studies. PUBMED, MEDLINE, EMBASE, PsychINFO, Cochrane Central Register of Controlled Trials (CENTRAL), CINAHL, AMED and Web of Science were searched. Online abstracts archives of relevant conference proceedings and trial registries were also searched. Thirty-Five (35) full-text articles were assessed for eligibility. Included studies were conducted in high, low and middle-income countries and reported ART adherence interventions delivered by mobile phones (standard or smartphones) in the form of voice calls, interactive voice response calls (IVR), and short message service (SMS).

Results

Thirteen (13) studies met the inclusion criteria, and 11 were used in the meta-analysis. Intervention characteristics of included studies ranged from mobile phone functionalities to provision of study phones to participants. SMS and voice call contents were tailored to participants’ specific adherence needs. Mobile SMS interventions improved adherence to ART compared with control conditions (OR, 95% CI = 1.59, 1.27–1.98). In subgroup analysis, only scheduled SMS was significant whereas triggered SMS had no effect on adherence to ART. Mobile voice calls did not significantly increase adherence to ART. The interventions were highly rated by > 90% of participants in the studies that reported on the experiences and satisfaction with the intervention.

Conclusion

Scheduled mobile phone text-messaging have demonstrated significant improvement in adherence to ART. Mobile SMS adherence interventions that allow for two-way communication may, however, be more acceptable than standalone SMS reminders, which are seen to be intrusive, producing habituation and response fatigue. Voice calls and triggered SMS functionalities do not have a significant effect on adherence to ART although there is a higher preference for voice functionality over SMS especially in limited-resource and low-literacy settings. Further exploration of the mobile voice functionality and its possible combination with scheduled SMS functionality is recommended. Evidence provided in this study will guide the implementation of mobile phone intervention to improve adherence to ART, by addressing practical challenges that could militate against scalability especially in resource limited settings.

Introduction

The introduction of Antiretroviral Therapy (ART) has led to a significant success in Human Immunodeficiency Virus (HIV) viral suppression, reduced the incidence of Acquired Immunodeficiency Virus (AIDS)-defining illnesses, and improved quality of life [1,2]. In recent times, ART has also been identified as a potent intervention to reduce the transmission of HIV among HIV serodiscordant couples [36]. Optimal adherence to ART has however emerged as a key determinant of ART’s success [7]. It is therefore desirable to provide adherence support for patients receiving ART to help preserve the effectiveness of first-line ART regimens.

Several approaches, ranging from the simplification of dosing schedules to the design of complex adherence interventions have been developed [8,9]. Among these interventions, mobile phones have emerged as a potentially more effective adherence support strategy [10] due to its increasing penetration rate, relatively low cost, convenience and the ability to offer varied communication approaches [1113]. Mobile phone coverage is now near ubiquitous, with an estimated 95% of the global population living in an area covered by a basic 2G mobile-cellular network [14]. With documented evidence of its use in the management of several chronic conditions [1517], mobile phones will be essential in facilitating the attainment of the 17 sustainable development goals (SDGs) [18].

Unfortunately, the quality of evidence and the strength of recommendations of existing studies and reviews do not support the widespread adoption of mobile phone interventions in ART adherence [19,20]. The recommendations from the two reviews (which included only RCTs and SMS-based interventions) may be limited to high literate populations who might already be familiar with sending and receiving text messages. The situation is summarised by two systematic reviews, which concluded that ‘mobile SMS is effective in improving adherence to ART [21,22]’. In the first systematic review, (carried out in 2012), Horvath and his colleagues [21] utilised the standard Cochrane review approach to identify studies utilising mobile phones for ART adherence. However, only two studies [23,24] met its strict inclusion criteria (only randomised controlled trials [RCT] utilising SMS messaging as an intervention). The authors found evidence of efficacy in using short weekly messages. The publication was followed by a recommendation by the World Health Organisation to consider text messaging ‘for promoting adherence to ART as part of adherence interventions [25,26].

Newer studies on SMS interventions have produced more contradictory results. The inconclusive findings, coupled with a need to determine the specific components of text-messaging trials that provided the observed effects, led to a second systematic review in 2014 by Finitsis et al., [22]. The eight RCTs that met the inclusion criteria found text-messaging interventions to be effective in supporting antiretroviral therapy adherence. Through sensitivity analyses, this review further reported on key design elements of the intervention that can potentially enhance the observed effects. These design elements include frequency of message delivery, directionality, tailoring of message content and matching the message to dosing timing. The literature so far has been inconclusive on these, and the authors recommended further exploration of these areas [22]

Despite the quality evidence generated by the two reviews, it remains unclear if the efficacies observed in these RCTs translates into effectiveness in broader and more realistic settings. While recommending RCTs to ascertain intervention’s effectiveness, attention could be paid to other forms of evidence (e.g. prospective cohort studies and pragmatic trials) that permit assessment of patients in their natural settings for longer periods.

Another real-world implementation dimension not addressed in the existing reviews is how SMS-based intervention may be implemented in low literacy populations. With emerging evidence suggesting that designs that encourage two-way communication may exhibit better outcomes [22], low literate populations may not benefit from the SMS-based intervention. A promising approach that may enhance engagement in this population is a phone call intervention. A preference for phone calls over text messages in populations with low-literacy and resource-constraint settings has been reported [27]. The limitations of existing SRs and the rate of incoming evidence calls for a more inclusive, up-to-date review that incorporates other forms of evidence. Our review adopts a broader scope to include mobile phone voice calls and non-RCTs. Findings of the current study will fill an essential gap in prior reviews that relied solely on SMS reminders and SMS-based interventions[23,28].

Methods

The review was guided by methods developed by Joanna Briggs Institute [29,30] and followed the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) format [31].

Search strategy

An initial limited search of major databases (PUBMED, CINAHL, MEDLINE and EMBASE) was undertaken to identify relevant keywords. This was followed by the development of a comprehensive and exhaustive search strategy, which was modified as appropriate across databases. The search strategy for PUBMED database is provided as a supplementary file (S1 Table).

The following electronic databases were subsequently searched in the period from January 1995 to June 2017: PUBMED, MEDLINE, EMBASE, Psych INFO, Cochrane Central Register of Controlled Trials (CENTRAL), CINAHL, AMED and Web of Science. Using different terms, trials that had been registered in the ‘ClinicalTrial.gov’ were also searched. Searches were limited to human studies but not by language or setting. The search strategy specified keywords, including ‘text messaging’, ‘SMS’, ‘voice calls’, and ‘phone calls’ combined with ‘adherence or compliance’. We also searched online abstract archives of relevant conference proceedings, including the conference on Retroviruses and Opportunistic Infections (CROI), International AIDS Conference (IAC), and the International AIDS Society (IAS) Conference on HIV Pathogenesis, Treatment and Prevention. Finally, a hand-search of the reference lists and bibliographies of potential studies was performed.

Study selection

Electronic search results were imported into Endnote X7 bibliographic software [32] and duplicates removed by one author (IA). This was followed by an independent screening of titles and abstracts against set inclusion and exclusion criteria by IA, DB and DYQ. Two authors (IA and DB) independently screened full texts of all potentially relevant studies. Disagreements were resolved by consensus or discussion by all authors.

Eligibility criteria

Papers eligible for inclusion were informed by inclusion and exclusion criteria applied to each of the following domains: study types, types of participants, types of interventions and types of outcomes. The inclusion and exclusion criteria within each of the domains are described below:

Study types: We included randomised controlled trials (RCTs) and quasi-experimental studies that examined the effectiveness of mobile phone-based interventions for ART adherence. Qualitative only designs, letters to editors and commentaries on published studies were excluded.

Types of participants (P).

This review included HIV positive individuals receiving ART, irrespective of age (to include carers in the case of vulnerable adults or children). Further, studies were included irrespective of gender or clinical stage of HIV infection in a primary, outpatient, community and hospital care settings.

Types of interventions (I).

Interventions of interest were voice calls, short message service (SMS) and Interactive Voice Response Calls (IVR) that were delivered by mobile phones (standard or smartphones) and targeted ART adherence. We defined IVR as an automated telephony system that interacts with callers, through mobile phones, gathers data and directs calls to the appropriate recipient [33]. Studies in which adherence interventions were augmented by smartphone applications (apps) or delivered to fixed telephone landlines were excluded.

Comparator (C).

Mobile phone-based interventions were compared with existing standard adherence interventions such as self-monitoring, reinforcement, counselling, and psychological therapy [34]. Studies were excluded if the comparator is also an adherence reminder such as an alarm system.

Types of outcomes (O).

The primary outcome was ART adherence. This was measured directly (e.g. pill count) or indirectly (e.g. self-reporting), and secondary outcomes were HIV viral load (quantity of virus in a given volume, copies/mL), quality of life and patient satisfaction with the mobile phone intervention.

Data extraction

A data extraction sheet based on Joanna Briggs Institute (JBI) data extraction forms [35] was developed, pilot-tested on four (4) randomly selected included studies and refined as appropriate. Details of the included studies were then extracted by one reviewer (IA), and checked by a second reviewer (DB). Disagreements were resolved with a third reviewer (DYQ). Data extracted from each study included study setting, the type of mobile technology employed, study size, gender, age of participants, ART treatment experience, mobile phone ownership and adherence status/viral load. We also included how message content was developed, duration of the study, delivery frequency, main outcome(s) measured, methods of outcome assessment and major findings.

Quality assessment and risk of bias across studies

The JBI critical appraisal checklist [36], (one each for included RCTs and experimental/cohort studies), comprising a list of questions with fixed responses, were used to assess the quality of included studies. Two authors independently graded all included articles, and a senior author (CE) reviewed all the grades. Each article was scored on certain intervention characteristics (e.g. randomisation, blinding) [29]. Any disagreements were resolved between the authors in a scheduled meeting. The quality assessment and scoring criteria are available in Tables A and B in S2 File.

Data synthesis and analysis

Intervention studies that provided sufficient data were used in separate meta-analyses using Cochrane Review Manager (RevMan)[37] software. Findings of the remaining studies were presented in a narrative format.

We focused on dichotomous outcomes and the odds ratios and 95% confidence interval (CI) for the effect of mobile phone interventions in adherence to ART. Using a formula by Chinn [38], the effects measures that were on a continuous scale were re-expressed as log odds ratios using standardized mean difference (SMD). The studies that reported odds ratios were also converted to log odds ratios. The log odds ratios and standard errors (SE) were combined in RevMan using the generic inverse-variance method [39]. Sub-group analyses were done for scheduled and triggered SMS. The overall effect of mobile phone intervention on immunological and virological outcomes was also estimated using the pooled odds ratio for viral load <400copies/ml [40]. We performed a fixed effect analysis, and heterogeneity was assessed using the Cochrane’s Q and degree of inconsistency (I2) [41]. A sensitivity analysis, however, showed little difference in estimates in the fixed or random effect model. Odds ratios were considered statistically significant at the 5% level (P<0.05) if the value 1 is not within the 95% CI.

Results

Characteristics of included studies

Thirteen (13) RCTs and 2 cohort and quasi-experimental studies met the inclusion criteria with 22 excluded with reasons, Fig 1.

thumbnail
Fig 1. Flow chart of inclusion and exclusion criteria of relevant articles.

https://doi.org/10.1371/journal.pone.0204091.g001

Table 1 summarizes the characteristics of the included studies. Studies were conducted in high- (USA, n = 2; Italy, n = 1), low- and middle-income countries (Kenya, n = 2; India, n = 2; Brazil, n = 1; China, n = 2; Nigeria, n = 1, Cameroon, n = 1, Uganda, n = 1), at Infectious Disease and ART clinics (n = 4), health care clinics (n = 2), community health centres (n = 2), hospitals (n = 4) and reported between 2010 and 2016. Sample sizes ranged from 21 to 631.

thumbnail
Table 1. Characteristics of included studies and participants.

https://doi.org/10.1371/journal.pone.0204091.t001

Majority of the studies recruited adult participants (> 18 years) of both sexes (54.63% male). Three studies focused their interventions on specific populations such as Latina women in Brazil [42] and the youth (14–29 years) [43]. Two studies [4345] recruited only participants with established suboptimal adherence. Studies considered participants who were receiving or initiating ART [46], had been on ART for at least one month [47], or for less than three months [23,24,48,49]. Shet et al., [50] recruited HIV positive individuals who were ART-naive and had adequate documentation on their HIV positive status. Belzer et al., [51] focused on documented HIV positive patients with a history of non-adherence to one or more components of ART. Huang et al., [52] on the other hand recruited both treatment-naive and treatment-experienced HIV positive patients. The most commonly cited inclusion criteria were age (9/13), followed by access to personal (or shared) phone (8/13). One study [48] removed participants with chronic disease necessitating daily medication while another [51] removed participants with cognitive impairment or substance abuse. Illiteracy was an exclusion criterion for one study [42].

Risk of bias

The RCTs employed adequate randomisation techniques, had groups that were similar at baseline and measured outcomes of the two groups in a similar and reliable way. Appropriate allocation concealment was adopted by all the trials except Pop-Eleches et al. [24], who failed to report on this. Understandably, blinding of participants to treatment allocation was a point missing in all studies since the intervention required overt participation. Risk of contamination was a concern in the study by Lester et al., [23] as several participants were reported to have forwarded their weekly text messages to non-intervention participants to share support. In the case of Kalichman et al., [45] bi-weekly calls to the control group was made by an assessment staff whereas the intervention group received calls from a counsellor who was a master’s level social worker with experience in AIDS services. The study further mailed supportive devices like an alarm clock and post-it reminders to participants in the intervention group who identified these items as adherent needs. The above constitute the provision of treatments other than the intervention of interest and might have caused a systematic difference; thereby risking skewed results, Table A S2 File.

The cohort/quasi-experimental studies instituted measures that ensured that samples were representative of the population, minimised selection bias, the reliability of measured outcome and conducted appropriate statistical analysis, Table B in S2 File. An element of selection bias might have been introduced in most of the included studies as the authors excluded participants who had no access to personal mobile phones.

Description of interventions

Studies varied in mobile phone functionality, content and delivery of the intervention (Table 2).

thumbnail
Table 2. Characteristics of interventions and outcome measurements.

https://doi.org/10.1371/journal.pone.0204091.t002

Mobile phone technologies.

Mobile phone functionalities employed by the included studies were SMS reminders (8/13), voice (4/13) and a combination of SMS and Interactive Voice Response calls (1/13). Sabin et al., [46] and Haberer et al., [49] adopted an electronic adherence-monitoring device, which trigger an SMS reminder message to be sent whenever the Wisepill system (a pillbox with a Global system for mobile [GSM] communication that makes use of mobile phone and internet technology to provide real-time medication management) failed to detect a device opening 30 minutes after scheduled dose time.

In other studies, participants who had no access to personal phones were provided with study phones [24,45,50,51], excluded (10/13) or made to share phones with close relations [23]. Haberer et al., [49] excluded 49 (37%) of potential study participants because they had no personal phones.

The cost of the entire intervention was not reported by any of the reviewed trials. An average amount of US$ 0.08 was however reported as the cost of sending an SMS by three studies [23,42,47]. One study reported a monthly $45.00 cell phone plan for participants who chose to use their phones or those who were given mobile phones [51]. Such mobile phone plans restricted participants from making toll calls, downloading applications and any other functions that would add additional costs. Incentives in the form of cash [24,43,46] and phone credit [24] were also reported. Before the intervention, 4 out of 13 studies trained participants on how to respond to incoming interventions but failed to report on the cost of this training. [23,24,45,53].

Message development, content and tailoring.

The processes involved in developing messages were described in 7 out of 13 studies. Three of these [23,24,42] involved a multidisciplinary team and study participants in message content development. Dowshen et al., [43] for example collaborated with patients to develop their messages; Sabin et al., [46] provided patients with the option of selecting from a list 10 options developed jointly by clinicians and subjects. Lewis et al., [54] on the other hand employed the services of three experts with extensive experience in HIV interventions for men who have sex with men (MSM). The content of messages was constructed to remind (8/13), support (6/13) or counsel (1/13) participants. In the case of Belzer et al., [51], adherence facilitators were recruited and together with participants, scheduled call times at the convenience of participants.

Evidence of tailoring content of messages to participant’s specific adherence needs was seen in only 7 studies [43,45,5054]. These studies either supported patients to design their messages [43] or depended on feedback/specific answers provided by patients to adherence questions. Huang et al., [52], Kalichman et al., [45] and Lewis et al. [54] for example delivered tailored messages that varied in content and frequency based on participants’ experience with ART regimen. In other studies, participants also chose the sex and language of a pre-recorded voice call to be delivered [50] and the language of the voice call to be made [53] In the case of Belzer and colleagues[51], participants who reported being treatment-naive/non-adherent received daily messages (1–3 times daily) whereas therapeutically adherent participants received weekly messages. Also, participants were given a chance to change their call times during the study period for their convenience in consultation with adherence facilitators [51].

Theoretical framework utilised in developing messages was made evident in two (2) studies. Kalichman et al. [45], relied on self-regulation models of medication adherence and offered corrective feedbacks that directly addressed the identified adherence needs of participants whereas Mbuagbaw et al., [47] utilised the health behaviour model of behaviour change. All included studies showed evidence of ensuring privacy and confidentiality of HIV status of the participant as depicted by messages being generally brief and neutral with no mention of ‘HIV’, ‘AIDS’, or ‘ART’. A participant withdrew from a study [47] due to privacy concerns. Six out of 286 participants from the Shet et al., [50] study expressed concerns about intrusiveness and loss of privacy at the end of the follow-up period of intervention.

Delivery strategies.

An automated message delivery function was employed by five (5) studies to send SMS messages to participants. Three (3) studies [4244,46] further designed a study specific, web-based interface to ensure efficient management of patient information and text messaging. One study [48], made use of an internet-based bulk SMS platform to send out pre-scripted messages to participants. There was also an automatic voice call delivery function employed by Shet et al., [50] which began by greeting participants with a hope message before inquiring if patients had taken their medications.

Frequency and timing of communication varied within and across studies. Messages were delivered on a daily [43,44,51], alternate days [42], weekly [23,50], twice a week [48], every two weeks [45,52] and a combination of daily and weekly [24]. Delivery of SMS messages/voice calls occurred close to or at the time of dosing [42,44], at fixed times in the day [23,24,47], or as specified by the participant [43,51]. Two-Way interactivity between study participants and investigators was evident in seven (7) studies [23,43,45,47,5052]. These studies made it possible for message recipients to message back and forth with clinic staff. These participants had the opportunity to respond to a specific cue, report a problem or ask questions regarding their care. A total of 15598 valid responses to SMS inquiries were received from 815 participants in 3 studies [23,43,47] that required some form of feedback. Reasons for responding were mainly to request medical, logistical, financial support and/or to express gratitude. In all the studies, calls and/or text messaging were initiated by the study investigator. The length and content of messages varied in most studies reviewed. Pop-eleches et al., [24] for example used a combination of long and short messages whereas Mbuagbaw et al., [47] designed a series of 11 messages that were changed every week. This variation in content and length of messages were to sustain interest, provide encouraging content and prevented habituation.

Effects of mobile phone-based interventions on ART

Primary outcome: ART adherence.

Characteristics of outcomes are summarised in Table 2. Adherence to ART was measured as a primary outcome in all included studies. Methods and time intervals employed by the various studies in measuring adherence varied across studies.

Eleven (11) RCTs were included in the meta-analysis. The other two quasi-experimental or prospective cohort and had different scales of measurement of the outcome and therefore could not be pulled together with the RCTs. Seven (7) of these measured the effect of SMS intervention [23,24,42,4649] whereas four (4) measured the effect of voice calls [45,5052]. Haberer et al., [49] had two SMS intervention arms, and these were used separately in the subgroup analysis.

Mobile SMS intervention had a significant effect on adherence to ART (OR, 95% CI;1.59;1.27–1.98), Fig 2.

thumbnail
Fig 2. Forest plot of pooled odds ratio and 95% CI for effect of SMS intervention on adherence to ART.

*Scheduled SMS arm, **triggered SMS.

https://doi.org/10.1371/journal.pone.0204091.g002

In the subgroups, however, only scheduled SMS significantly increased adherence to ART. Mobile voice interventions had no significant association with adherence to ART, Fig 3.

thumbnail
Fig 3. Forest plot of pooled odds ratio and 95% CI for effect of voice call intervention on adherence to ART.

*ART naïve group, **ART experienced group.

https://doi.org/10.1371/journal.pone.0204091.g003

Quasi-experimental studies.

The two (2) non-randomised trials also reported beneficial adherence outcomes. Dowshen et al. [43] demonstrated that personalised, daily text messages delivered to youth living with HIV/AIDS could significantly improve and sustain the mean baseline adherence from 74.7% to 93.1% within 24 weeks (p< 0.001).

Secondary outcomes.

Secondary outcomes reported in the included studies include immunologic and virologic outcomes, patient satisfaction with incoming messages, and quality of life.

Immunologic and virologic outcomes

Four (4) studies [23,45,50,51] involving 1194 participants (602 in intervention arm, 598 in control arm) were analysed. A positive effect of the mobile phone intervention on viral load was observed, but was not statistically significant (OR, 95%CI; 1.29, 0.99–1.68; p = 0.06), Fig 4.

thumbnail
Fig 4. Forest plot of pooled odds ratio and 95% CI for effect of mobile phone intervention on viral load reduction.

https://doi.org/10.1371/journal.pone.0204091.g004

Participant experience with the mobile phone intervention (SMS-based only)

Three (3) studies [46,48,52] provided data on how interventions were experienced by some participants. The majority rated the intervention as being useful in helping avoid missing doses, providing incentives for them to initiate their medications or maintain correct dosing schedule. Some participants further expressed a desire to continue receiving the SMS adherence messages even after the study [23,42,43] and were also willing to recommend the intervention to a friend [23,47]. Participants who did not find the intervention helpful cited timing between the arrival of SMS messages and actual medication intake being too far [42] or being at work or inconvenient place when messages arrived [43]. Adverse events reported include undelivered messages due to technical challenges [52] or lapse in cell phone service [42]; poor access to electricity to charge phones [24]; and stolen phones. A female participant in a study by Mbuagwa et al. [44] requested to withdraw from the study because she felt the intervention had compromised her undisclosed HIV positive status. Studies that employed mobile phone voice call functionality, however, did not report on how patients experienced the intervention.

Preference for voice or SMS as a means of message delivery was reported by Rodrigues et al. [53]. Of the 136 participants, 42 (34%), preferred only phone call, 15 (11%) preferred only SMS, 60 (44%) preferred both phone call and the SMS, while the rest preferred neither.

Quality of life

Quality of life was assessed with the WHOQOL-HIV BREF (World Health Organization Quality of Life in HIV-infected Persons instrument) [55] and the SF-12 QOL (Short From 12 Quality of Life) assessment form by Huang et al., [52] and Mbuagbaw et al., [47] respectively. Huang et al., [52] recorded a significant QOL (Quality of Life) improvements among treatment-naïve participants in the domains of physical health (p = 0.003), level of independence (p = 0.018), environment (p = 0.002), and spirituality/religion/personal beliefs (p = 0.021) whereas Mbuagbaw et al., [47] reported a mean quality of life score of 3.67 (SD-0.623) and 3.69 (SD-0.615) for participants in the SMS group and control groups respectively.

Discussion

Key findings

In this systematic review and meta-analysis, we investigated the effectiveness of mobile phone-based interventions for ART adherence and other outcomes among HIV positive individuals.

This review has four main findings. First, scheduled mobile phone SMS significantly improve adherence to ART among HIV positive patients. Triggered SMS did not have significant improvement in adherence to ART. Second, interventions delivered by voice call functionality did not improve adherence to ART. Third, current evidence from RCTs shows that mobile SMS interventions do not improve immunologic and virologic outcomes. However, A quasi-experimental study [44] also reports a statistically significant immunologic and virologic outcomes of SMS-based interventions. Finally, study participants expressed satisfaction with the mobile-phone-delivered ART adherence intervention.

How the finding relates to existing literature

The current review updates and expands on the evidence by Hovarth et al., [21] and Finitsis et al., [22], who focused on SMS-based interventions and RCTs and concluded that SMS-based interventions are effective in improving adherence to ART. The current review supports this finding, and further reports that sending scheduled text messages (as opposed to real-time, triggered reminders) to individuals with HIV can significantly improve adherence. Our review additionally reports that mobile phone adherence interventions that allow for two-way communication may be more acceptable than standalone, SMS reminders. This finding corroborates existing literature that seems to suggest that stand-alone, SMS reminders may have a deteriorating effect on quality [56,57], as the reminders are seen to be intrusive and therefore produces habituation and response fatigue [22,46]. The current review identified design elements (such as directionality and the optimal frequency of intervention delivery) that might have moderated adherence outcomes. However, the extent to which these differences influenced adherence outcomes remains speculative, as most studies did not report on the theoretical constructs that informed their intervention design. Developing atheoretical mobile phone interventions may lead to a situation where any information, content or delivery methods could be included or excluded for unspecified reasons [58]. We, therefore, recommend adequate reporting of theories that inform intervention design in future studies.

Individual studies employing voice call as an intervention in this review have shown the potential of being effective in improving adherence, even though the pooled effect in a meta-analysis was not significant. This review, however, points to some beneficial effects, albeit secondary, that if harnessed, can improve patient experience of the mobile phone intervention. For example, a study by Huang et al. [52] in China found voice calls to be effective in enhancing the quality of life of individuals receiving ART. This finding contradicts earlier finding that standalone mobile phone text messaging leads to habituation and decrease in quality of life [22,46]. Additionally, adherence interventions delivered by voice calls may be more acceptable compared to text messaging as demonstrated by Rodrigues et al. [53]. Individuals likely to show a high preference for voice calls are aged and the illiterate population.

Unanswered questions, future research and real-world implementation

More than half of the included studies (67%) used stand-alone, scheduled SMS reminders as an intervention to improve adherence. Beyond the research settings, little is known about how this intervention will be scaled up in low-literate populations and even among the aged [59] as SMS interventions will require these participants to read and respond to messages. Further exploration of the most suitable mobile phone functionality (SMS, Voice and IVR) is therefore warranted. It is also imperative that the World Health Organisation carefully review the current recommendation that SMS-based interventions, in general, are effective and instead focus on those interventions shown to be effective such as two-way text messages in their guidelines. Specific properties of mobile phone functionalities that can be tailored to specific populations such as the low literate group or the youth must be studied to provide specific recommendations to such groups.

Another finding of this review that can potentially impact scalability is the issue of phone ownership [59,60]. Access to a personal phone as reported in this review was the second most cited inclusion criteria. Lester et al., [23] for example excluded 49 (37%) of study participants for not having access to personal phones. Four (4) studies provided study phones [24,45,50,51]. It was however not clear how participants experienced the carrying of unfamiliar phones with different patterns of usage. We do not know in what situations cell phones may be provided and how this will affect cost and sustainability, or how we can determine who should be given a phone. These are practical issues that must be addressed if the intervention is to be scaled up in resource-constraint settings. Closely related to phone ownership is the issue of privacy. Even though our review points to practical steps adopted by individual studies to protect the undisclosed HIV status among study participants, privacy is still a potential challenge in using mobile phone to deliver care to HIV patients especially in developing countries. Mbuagbaw et al., [47] for example, reported that a patient withdrew from a study because of privacy concerns. This aspect of the intervention must be carefully considered on a scale up. Similarly, logistical concerns such as undelivered messages, broken phones and poor mobile network as reported in this review warrant attention. The review points to potential risks and unintended consequences associated with using mobile phones for adherence interventions and must be explored further.

Using a combination of weekly IVRs and neutral SMS reminders, Rodrigues and colleagues [53] have demonstrated significant improvement in adherence outcomes for patients in India, a setting characterised by illiteracy and multiple dialects. Given the lower use of SMSs in this setting, the study participants showed a preference for voice calls over SMS. A recent trial that looked at a combined mobile phone SMS and telephone call reminder (SMS reminder 3 days prior to scheduled clinic appointments and an average of 90sec lunch hour telephone call reminders a day prior to scheduled clinic appointment) and peer counselling found significant improvement in adherence and treatment outcomes among HIV positive patients on ART in Malaysia [61]. Acceptability of this intervention was high among the participants. Further exploration of combined functionality especially in resource-constraint settings is recommended.

Even though the focus of this review is not on SMART phones, it is important to state that new mobile messaging services such as Facebook Messenger, WhatsApp, SnapChat and Instagram, all have functions that are parallel and often surpass SMS and MMS functionality [62]. We expect that this intervention and other smartphone mobile phone interventions will become a more frequent feature for mobile phone adherence interventions for ART, especially in a high literate population and high-income countries. A study excluded from this review for example [63] found that augmented smartphone application that provided real-time information about the level of medication in the client’s blood and a pictorial depiction of immunoprotection significantly improved adherence to ART compared to the standard. We suggest further studies to look into how smartphone applications could be used to improve adherence to ART.

Strength and limitations of the review

This is a comprehensive review of both RCTs and quasi-experimental interventions delivered by mobile phones to improve adherence to ART and other outcomes. Our review has several strengths. The adoption of a broad selection criterion to include all mobile phone-based interventions may reduce inflation of overall effect sizes as a way of preventing publication bias. The methods employed are reproducible, screening of trials and data extraction were carried out by two authors, with disagreement settled with a third reviewer through discussion.

There are however some limitations to our review. This relates to the wider scope of the review and the inclusion of studies without a control group. This might have introduced some confounding and selection bias respectively. The variation in interventions, outcome measurements and follow-up times might also have influenced the pooled effect size of the intervention. However, in situations where there was more than one follow-up, we included data on the longest follow-ups from the various studies.

Conclusions

Scheduled mobile phone text-messaging interventions have demonstrated significant improvement in adherence to ART. Mobile SMS adherence interventions that allow for two-way communication may however be more acceptable than standalone SMS reminders, which are seen to be intrusive and therefore produces habituation and response fatigue. Mobile phone ownership is also a concern that needs to be addressed to enhance scalability in low and middle-income settings as this could affect the affordability and cost-effectiveness of the intervention. Voice calls and triggered SMS functionalities do not have a significant effect on adherence to ART although there is a higher preference for voice functionality over SMS in limited-resource and low-literacy settings. Further exploration of the mobile voice functionality and its possible combination with scheduled SMS is recommended. Evidence provided in this study will guide the implementation of the mobile phone intervention to improve adherence to ART among HIV positive patients, by addressing practical challenges that could militate against scalability and roll out especially in limited resource settings.

Supporting information

S2 File. Results of risk of bias assessment.

https://doi.org/10.1371/journal.pone.0204091.s002

(DOCX)

S1 Table. Search strategy for PUBMED.

This search strategy was modified across databases as appropriate.

https://doi.org/10.1371/journal.pone.0204091.s003

(DOCX)

References

  1. 1. Wand H, Falster K, Wilson D, Law M, Maher L. Disproportionate impact of combination antiretroviral therapy on AIDS incidence in Australia: Results from a modified back-projection model. AIDS Behav. 2012;16(2):360–7. pmid:21598032
  2. 2. De Geest S, Sabaté E. Adherence to long-term therapies: Evidence for action. Vol. 2, European Journal of Cardiovascular Nursing. 2003. p. 323. pmid:14667488
  3. 3. He N, Duan S, Ding Y, Rou K, McGoogan JM, Jia M, et al. Antiretroviral therapy reduces HIV transmission in discordant couples in rural Yunnan, China. PLoS One. 2013;8(11).
  4. 4. Lu W, Zeng G, Luo J, Duo S, Xing G, Guo-Wei D, et al. HIV transmission risk among serodiscordant couples: a retrospective study of former plasma donors in Henan, China. J Acquir Immune Defic Syndr [Internet]. 2010;55(2):232–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21423851%5Cnhttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3058178%5Chttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3058178 pmid:21423851
  5. 5. Wang L, Wang L, Smith MK, Li L, Ming S, Lu J, et al. Heterosexual transmission of HIV and related risk factors among serodiscordant couples in Henan province, China. Chin Med J (Engl) [Internet]. 2013;126(19):3694–700. Available from: http://www.cmj.org/ch/reader/create_pdf.aspx?file_no=2012-3311&flag=1&year_id=2013&quarter_id=19
  6. 6. Cohen MS, McCauley M, Gamble TR. HIV treatment as prevention and HPTN 052. Curr Opin HIV AIDS [Internet]. 2012;7(2):99–105. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3486734&tool=pmcentrez&rendertype=abstract pmid:22227585
  7. 7. Hardon A, Davey S, Gerrits T, Hodgkin C, Irunde H, Kgatlwane J, et al. From access to adherence: the challenges of antiretroviral treatment. Studies from Botswana, Tanzania and Uganda. From access to adherence challenges Antiretrovir Treat Stud from Botswana, Tanzania Uganda [Internet]. 2006;301:1–320. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=cagh&AN=20063192157%5Cnhttp://lshtmsfx.hosted.exlibrisgroup.com/lshtm?sid=OVID:caghdb&id=pmid:&id=doi:&issn=&isbn=9241563281&volume=&issue=&spage=&pages=&date=2006&title====&atitle=From+ac
  8. 8. Bain-Brickley D, Butler Lisa M, Kennedy Gail E, Rutherford George W. Interventions to improve adherence to antiretroviral therapy in children with HIV infection. Cochrane Database Syst Rev [Internet]. 2011;(12). Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009513/abstract
  9. 9. Kripalani S, Yao X, Haynes RB. Interventions to enhance medication adherence in chronic medical conditions: A systematic review. Arch Intern Med [Internet]. 2007;167(6):540–50. Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L46513223%5Cnhttp://archinte.ama-assn.org/cgi/reprint/167/6/540 pmid:17389285
  10. 10. Lemaire J. Scaling Up Mobile Health—Elements Necessary for the Successful Scale Up of mHealth in Developing Countries. 2011;(December):1–23. Available from: https://www.k4health.org/sites/default/files/ADA_mHealthWhitePaper.pdf
  11. 11. Kaplan WA. Can the ubiquitous power of mobile phones be used to improve health outcomes in developing countries? Global Health [Internet]. 2006;2(1):9. Available from: http://www.globalizationandhealth.com/content/2/1/9
  12. 12. Patrick K, Griswold WG, Raab F, Intille SS. Health and the Mobile Phone. Am J Prev Med. 2008;35(2):177–81. pmid:18550322
  13. 13. WHO UNICEF, UNAIDS. GLOBAL HIV/AIDS RESPONSE Epidemic update and health sector progress towards Universal Access. Vasa. 2011;1–233.
  14. 14. ITU. ICT Facts and figures 2016 [Internet]. 2016 [cited 2016 Dec 22]. Available from: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2016.pdf
  15. 15. Sarin Atun, Neil Gough the V group. The role of mobile of mobile phones in increasing accessibilty and efficiency in healthcare [Internet]. 2006. Available from: https://www.vodafone.com/content/dam/vodafone/about/public_policy/policy_papers/public_policy_series_4.pdf
  16. 16. Blake H. Innovation in practice: mobile phone technology in patient care. Br J Community Nurs. 2008;13(4):160, 162–5. pmid:18595303
  17. 17. Lewis T, Synowiec C, Lagomarsino G, Schweitzer J. E-health in low- and middle-income countries: findings from the Center for Health Market Innovations. Bull World Health Organ. 2012;90(5):332–40. pmid:22589566
  18. 18. Klapper Leora, Mayada El-Zoghbi and JH. Achieving the sustainable development goals. Curr Sci [Internet]. 2016;110(2):127–8. Available from: http://doi.wiley.com/10.1111/ner.12182
  19. 19. Anscombe J, Bacardit A, Hamid I. Improving the Evidence for Mobile Health. AT Kearney [Internet]. 2012;27. Available from: http://www.gsma.com/connectedliving/gsma-improving-the-evidence-for-mobile-health/
  20. 20. Mechael P, Searle S. Barriers and Gaps Affecting mHealth in Low and Middle Income Countries: Policy White Paper. Heal San Fr [Internet]. 2010;54(March):1–79. Available from: http://www.mobileactive.org/files/file_uploads/mHealth_Barriers_White_Paper.pdf
  21. 21. Horvath Azman, Kennedy G, Rutherford G. Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection. Cochrane Database Syst Rev [Internet]. 2012;(3):N.PA-N.PA. Available from: http://search.ebscohost.com/login.aspx?direct=true&amp%5Cndb=cin20&amp%5CnAN=2011498015&amp%5Cnlang=es&amp%5Cnsite=ehost-liv
  22. 22. Finitsis DJ, Pellowski JA, Johnson BT. Text message intervention designs to promote adherence to antiretroviral therapy (ART): A meta-analysis of randomized controlled trials. PLoS One. 2014;9(2).
  23. 23. Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): A randomised trial. Lancet. 2010;376(9755):1838–45. pmid:21071074
  24. 24. Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS [Internet]. 2011;25(6):825–34. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21252632%5Cnhttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3718389 pmid:21252632
  25. 25. Sharma P, Agarwal P. Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection [Internet]. The WHO Reproductive Health Library; Geneva: World Health Organization. 2012. Available from: http://apps.who.int/rhl/hiv_aids/cd009756_sharmap_com/en/
  26. 26. World Health Organisation. Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach. WHO Guidel [Internet]. 2013;(June):272. Available from: http://apps.who.int/iris/bitstream/10665/85321/1/9789241505727_eng.pdf
  27. 27. Momin Kazi A. The role of mobile phone-based interventions to improve routine childhood immunisation coverage. www.thelancet.com [Internet]. 2017 [cited 2018 Feb 4]; Available from: http://www.thelancet.com/pdfs/journals/langlo/PIIS2214-109X(17)30088-8.pdf
  28. 28. Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, Rowe AK, et al. The effect of mobile phone text-message reminders on Kenyan health workers’ adherence to malaria treatment guidelines: A cluster randomised trial. Lancet [Internet]. 2011;378(9793):795–803. Available from: pmid:21820166
  29. 29. The Joanna Briggs Institute. Joanna Briggs Institute Reviewers’ Manual: 2015 edition / Supplement. Joanna Briggs Inst. 2015;
  30. 30. Pearson A, Field J, Jordan Z. Evidence-Based Clinical Practice in Nursing and Health Care. 2006; Available from: http://doi.wiley.com/10.1002/9781444316544
  31. 31. Moher D, Liberati A, Tetzlaff J AD. PRISMA 2009 Flow Diagram. Vol. 6, The PRISMA statement. 2009. p. 1000097.
  32. 32. Oberhauser O. EndNote X7 –Bibliographies Made EasyTM [= Version 17]. Carlsbad, CA: Thomson Reuters, 2013. Mitteilungen der Vereinigung Österreichischer Bibl Bibl [Internet]. 2014;67(1):174–8. Available from: http://hdl.handle.net/10760/22986
  33. 33. TechTargete. What is Interactive Voice Response (IVR)? [Internet]. [cited 2018 Feb 4]. Available from: http://searchcrm.techtarget.com/definition/Interactive-Voice-Response
  34. 34. Rb H, Ackloo E, Sahota N, Hp M, Yao X, Haynes RB, et al. Interventions for enhancing medication adherence (Review) Interventions for enhancing medication adherence. Cochrane database Syst Rev. 2008;(2):2–4.
  35. 35. The Joanna Briggs Institute. The Joanna Briggs Institute Reviewers’ Manual 2015: Methodology for JBI scoping reviews. Joanne Briggs Inst. 2015;1–24.
  36. 36. The Joanna Briggs Institute. Critical Appraisal Tools | JBI [Internet]. 2016 [cited 2016 Oct 15]. Available from: http://joannabriggs.org/research/critical-appraisal-tools.html
  37. 37. Higgins JP, Green S. Guide to the Contents of a Cochrane Protocol and Review. In: Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. 2008. p. 51–79.
  38. 38. Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med. 2000;19(22):3127–31. pmid:11113947
  39. 39. Higgins JP, Altman DG. Assessing Risk of Bias in Included Studies. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. 2008. 187–241 p.
  40. 40. Mantel N, Haenszel W. Statistical aspects of the analysis odf data from retrospective studies od disease. J Natl Cancer Inst [Internet]. 1959;22(4):719–48. Available from: http://jnci.oxfordjournals.org/content/22/4/719.full.pdf pmid:13655060
  41. 41. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ Br Med J. 2003;327(7414):557–60.
  42. 42. Da Costa TM, Barbosa BJP, E Costa DAG, Sigulem D, De Fátima Marin H, Filho AC, et al. Results of a randomized controlled trial to assess the effects of a mobile SMS-based intervention on treatment adherence in HIV/AIDS-infected Brazilian women and impressions and satisfaction with respect to incoming messages. Int J Med Inform [Internet]. 2012;81(4):257–69. Available from: http://dx.doi.org/10.1016/j.ijmedinf.2011.10.002 pmid:22296762
  43. 43. Dowshen N, Kuhns LM, Johnson A, Holoyda BJ, Garofalo R. Improving adherence to antiretroviral therapy for youth living with HIV/AIDS: A pilot study using personalized, interactive, daily text message reminders. J Med Internet Res. 2012;14(2):168–75.
  44. 44. Ammassari A, Trotta MP, Shalev N, Tettoni MC, Maschi S, Di Sora F, et al. Timed short messaging service improves adherence and virological outcomes in HIV-1-infected patients with suboptimal adherence to antiretroviral therapy. J Acquir Immune Defic Syndr [Internet]. 2011;58(4):e113–5. Available from: http://ezproxy.derby.ac.uk/login?url = http://search.ebscohost.com/login.aspx?direct=true&db=cmedm&AN=22033235&site=ehost-live pmid:22033235
  45. 45. Kalichman SC, Kalichman MO, Cherry C, Swetzes C, Amaral CM, White D, et al. Brief behavioral self-regulation counseling for HIV treatment adherence delivered by cell phone: an initial test of concept trial. AIDS Patient Care STDS [Internet]. 2011;25(5):303–10. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3085946&tool=pmcentrez&rendertype=abstract pmid:21457056
  46. 46. Sabin Lora L., DeSilva Mary Bachman, Gill MS Christopher J., Li Zhong, Vian Taryn, Wubin Xie, Feng, MPH Cheng, Keyi Xu, Guanghua Lan, Haberer Jessica E., Bangsberg L David R., Yongzhen LH and ALG. Improving Adherence to Antiretroviral Therapy with Triggered Real Time Text Message Reminders: the China through Technology Study (CATS). J Acquir Immune Defic Syndr. 2015;33(4):395–401.
  47. 47. Mbuagbaw L, Thabane L, Ongolo-Zogo P, Lester RT, Mills EJ, Smieja M, et al. The Cameroon Mobile Phone SMS (CAMPS) Trial: A Randomized Trial of Text Messaging versus Usual Care for Adherence to Antiretroviral Therapy. PLoS One. 2012;7(12):6–12.
  48. 48. Maduka O, Obin-West CI. Adherence counseling and reminder text messages improve uptake of antiretroviral therapy in a tertiary hospital in Nigeria. Niger J Clin Pract. 2013;16(3):302–8. pmid:23771450
  49. 49. Haberer JE, Musiimenta A, Atukunda EC, Musinguzi N, Wyatt MA, Ware NC, et al. SMS reminders plus real-time adherence monitoring improve adherence to antiretroviral therapy in rural Uganda. Aids [Internet]. 2016;(November 2015):1295–9. Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L607779694%5Cnhttp://dx.doi.org/10.1097/QAD.0000000000001021%5Cnhttp://findit.library.jhu.edu/resolve?sid=EMBASE&issn=14735571&id=doi:10.1097/QAD.0000000000001021&atitle=SMS+reminders+pl
  50. 50. Shet A, De Costa A, Kumarasamy N, Rodrigues R, Rewari BB, Ashorn P, et al. Effect of mobile telephone reminders on treatment outcome in HIV: evidence from a randomised controlled trial in India. BMJ [Internet]. 2014;349(October):g5978. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=4459037&tool=pmcentrez&rendertype=abstract
  51. 51. Belzer ME, Naar-King S, Olson J, Sarr M, Thornton S, Kahana SY, et al. The use of cell phone support for non-adherent HIV-infected youth and young adults: An initial randomized and controlled intervention trial. AIDS Behav. 2014;18(4):686–96. pmid:24271347
  52. 52. Huang Dongsheng, Sangthong Rassamee, EdwardMcNeil Virasakdi Chongsuvivatwong, Zheng Weibin 1 andXuemei Yang. Effects of a Phone Call Intervention to Promote Adherence to Antiretroviral Therapy and Quality of Life of HIV / AIDS Patients in Baoshan, China: A Randomized Controlled Trial Internet-Facilitated, Voluntary Counseling and Testing (VCT) Clinic-Based. AIDS Res Treat. 2013;2013:1–18.
  53. 53. Rodrigues R, Shet A, Antony J, Sidney K, Arumugam K, Krishnamurthy S, et al. Supporting adherence to antiretroviral therapy with mobile phone reminders: Results from a cohort in South India. PLoS One. 2012;7(8):1–7.
  54. 54. Lewis M a., Uhrig JD, Bann CM, Harris JL, Furberg RD, Coomes C, et al. Tailored Text Messaging Intervention for HIV Adherence: A Proof-of-Concept Study. Heal Psychol. 2013;32(3):248–53.
  55. 55. World Health Organization. WHOQOL-HIV Instrument Users Manual. SubStance. 2002;1–13.
  56. 56. Mannheimer SB, Morse E, Matts JP, Andrews L, Child C, Schmetter B, et al. Sustained benefit from a long-term antiretroviral adherence intervention. Results of a large randomized clinical trial. J Acquir Immune Defic Syndr. 2006;43 Suppl 1(43):S41–7.
  57. 57. Wu AW, Snyder CF, Huang I-C, Skolasky R, McGruder HF, Celano S a, et al. A randomized trial of the impact of a programmable medication reminder device on quality of life in patients with AIDS. AIDS Patient Care STDS [Internet]. 2006;20(11):773–81. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17134351 pmid:17134351
  58. 58. Ryan P. Integrated Theory of Health Behavior Change. Clin Nurse Spec [Internet]. 2009 May [cited 2018 Aug 12];23(3):161–70. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19395894 pmid:19395894
  59. 59. DeSouza SI, Rashmi MR, Vasanthi AP, Joseph SM, Rodrigues R. Mobile phones: The next step towards healthcare delivery in rural India? PLoS One. 2014;9(8):1–9.
  60. 60. Cole-Lewis H, Kershaw T. Text Messaging as a Tool for Behavior Change in Disease Prevention and Management. Epidemiol Rev. 2010;32(1):56–69.
  61. 61. Abdulrahman SA, Rampal L, Ibrahim F, Radhakrishnan AP, Kadir Shahar H, Othman N. Mobile phone reminders and peer counseling improve adherence and treatment outcomes of patients on ART in Malaysia: A randomized clinical trial. Chang L editor. PLoS One [Internet]. 2017 May 16 [cited 2018 Feb 6];12(5):e0177698. Available from: pmid:28520768
  62. 62. Hall AK, Cole-Lewis H BJ. Mobile Text Messaging for Health: A Systematic Review of Reviews. Annu Rev Public Heal. 2015;36(20):393–415.
  63. 63. Perera AI, Thomas MG, Moore JO, Faasse K, Petrie KJ. Effect of a smartphone application incorporating personalized health-related imagery on adherence to antiretroviral therapy: a randomized clinical trial. AIDS Patient Care STDS. 2014;28(11):579–86. pmid:25290556