• Loading metrics

Scaling Up mHealth: Where Is the Evidence?

  • Mark Tomlinson ,

    Affiliation Centre for Public Mental Health, Department of Psychology, Stellenbosch University, Stellenbosch, South Africa

  • Mary Jane Rotheram-Borus,

    Affiliation Semel Institute for Neuroscience and Human Behaviour, University of California at Los Angeles, Los Angeles, California, United States of America

  • Leslie Swartz,

    Affiliation Centre for Public Mental Health, Department of Psychology, Stellenbosch University, Stellenbosch, South Africa

  • Alexander C. Tsai

    Affiliations Chester M. Pierce, MD Division of Global Psychiatry, Department of Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, United States of America, Center for Global Health, Massachusetts General Hospital, Boston, Massachusetts, United States of America

Scaling Up mHealth: Where Is the Evidence?

  • Mark Tomlinson, 
  • Mary Jane Rotheram-Borus, 
  • Leslie Swartz, 
  • Alexander C. Tsai

Summary Points

  • Despite hundreds of mHealth pilot studies, there has been insufficient programmatic evidence to inform implementation and scale-up of mHealth.
  • We discuss what constitutes appropriate research evidence to inform scale up.
  • Potential innovative research designs such as multi-factorial strategies, randomized controlled trials, and data farming may provide this evidence base.
  • We make a number of recommendations about evidence, interoperability, and the role of governments, private enterprise, and researchers in relation to the scale up of mHealth.

What Is the Problem?

There are over 6 billion mobile phone subscribers and 75% of the world has access to a mobile phone [1]. Service and care providers, researchers, and national governments are excited at the opportunities mobile health has to offer in terms of improving access to health care, engagement and delivery, and health outcomes [2]. Interventions categorized under the rubric “mobile health” or “mHealth”—broadly defined as medical and public health practice supported by mobile devices [2]—span a variety of applications ranging from the use of mobile phones to improve point of service data collection [3], care delivery [4], and patient communication [5] to the use of alternative wireless devices for real-time medication monitoring and adherence support [6].

A recent World Bank report tracked more than 500 mHealth studies, and many donor agencies are lining up to support the “scaling up” of mHealth interventions [7]. Yet, after completion of these 500 pilot studies, we know almost nothing about the likely uptake, best strategies for engagement, efficacy, or effectiveness of these initiatives. Currently, mHealth interventions lack a foundation of basic evidence [8], let alone a foundation that would permit evidence-based scale up. For example, in Uganda in 2008 and 2009 approximately 23 of 36 mHealth initiatives did not move beyond the pilot phase [9]. The current enthusiasm notwithstanding, the scatter-shot approach to piloting mHealth projects in the absence of a concomitant programmatic implementation and evaluation strategy may dampen opportunities to truly capitalize on the technology. This article discusses a number of points pertinent to developing a more robust evidence base for the scale up of mHealth interventions. The issues raised are primarily conceptual and methodological.

Industry's increasing role in pushing for mHealth scale up is also a cause for concern. At a recent mHealth conference in South Africa, there were repeated calls for scale up of mHealth initiatives across low- and middle-income countries (LAMICs). Many of these calls emanated from industry representatives rather than researchers, governments, or care providers [10]. It is likely that private enterprise has a quite different understanding of what scale up means, with growing market share, rather than improved health outcomes, at the core of their mission. The growing involvement by industry, predominantly mobile phone providers, warrants some caution in addition to perhaps a code of practice. Public–private partnerships will be of central importance in the evolution of the mHealth field (as we discuss later), but this cannot happen at the expense of good science and good public health.

In some ways, mobile technology has a magical appeal for those interested in global public health over and above the advantages that have been proven with good evidence [11]. Part of this magical promise is that mobile technologies may solve one of the most difficult problems facing global health efforts—that of structural barriers to access. Travel, especially to remote areas in LAMICs, is expensive, destructive to the environment, time-consuming, and exhausting and physically challenging to many. In the global health field, there are many practitioners whose personal and working lives are substantially disrupted by travel of this nature. Mobile technology may hold out the promise of a world within which these difficulties can be minimised or eliminated. There is an obvious appeal for people from higher-income contexts being able to remain at home and in their offices while interacting with and improving the health of people very far away and in straitened circumstances. Mobile technology may hold out the promise that the visceral challenges of travel and complex intercultural contact, so much a feature of the global health enterprise, may now be a thing of the past [12].

Current State of the Evidence

While enthusiasm for effective mHealth interventions in sub-Saharan Africa is high, little is known about their efficacy or effectiveness. Most randomized trials of mHealth interventions have employed text message reminder systems. Two systematic reviews have described a robust evidence base for the use of text message reminders to improve attendance at health care appointments [13],[14]. Yet, none of the studies included in these reviews was conducted in resource-limited settings. Similarly, few randomized trials evaluating the use of text message reminders to improve medication adherence for people with chronic illnesses have been conducted in LAMICs [15][18]. Three randomized trials studying HIV treatment adherence found benefits [19],[20] and one found no impact [21],[22]. Two recent systematic reviews [23],[24] found modest and suggestive evidence for the benefits of mHealth technology, and while both reviews recommended implementation, they argued that high quality (and adequately powered) clinical trials that measure clinical outcomes are essential.

The reviews of mHealth interventions would be more helpful if the results were organized according to 1) foundational functions (informing, training, monitoring, shaping, supporting, and linking to care); 2) content-specific targets (e.g., for Millennium Development Goal developmentally related tasks and challenges); and 3) local cultural adaptations (e.g., language) [25]. The inconsistency of results from mHealth studies demonstrates the importance of having an organizational framework.

What Constitutes Evidence?

The Institute of Medicine [26] and other communities of researchers [27],[28] have established standards for the phases of research that must be conducted in order to be considered efficacious, effective, and disseminated. Flay and colleagues [29] have adapted the evidentiary standards model published by the Society for Prevention Research [27]. These standards were developed in order to guide policy, research, and practice and provide a useful framework to determine what constitutes good and sufficient evidence. In this model (see Figure 1), scale up or country-wide implementation would be dependent on the completion (for each intervention) of (a) two high quality efficacy trials, (b) two high quality effectiveness trials, followed by (c) dissemination research that has established that the intervention can be delivered with fidelity to the model being tested, as well as (d) information about the intervention's costs. There are currently no mHealth interventions that meet these standards for scale up despite numerous calls to scale up mHealth projects.

Figure 1. Research stages and standards.

Adapted from Olds et al. [29] and Flay et al. [27].

Linked to the issue of standards for the phases of research is the question of theories of behaviour change. Aboud and Singla [30] have shown how programmes that simply provide health information (e.g., via SMS [short message service, or text messaging]) tend to be unsuccessful, while interventions providing skills through peer educators are more likely to be successful [30]. There are well validated theories of behaviour change common to many evidence-based interventions for prevention, diagnoses, and care, but none of the mhealth initiatives appear to be grounded in such theories [31]. We would argue that in the context of scarce resources, implementing untested mHealth interventions at scale without a theory of behaviour change is likely to result in many failed scale up projects and significant levels of wasted resources.

Finally, no major investments have been made to create a robust platform for mobile phones that could be used by designers of applications and electronic medical records that will allow cross-fertilization or integrated systems to be utilized [32]. Thousands of small applications have been propagated on closed-source platforms (e.g., iPhone applications and others) that each major mobile phone provider appears ready to replicate at high cost. Currently, a patient with two or more health conditions will have to make use of numerous applications for monitoring different health-related parameters such as medication adherence and health status, a disease-specific approach that he or she is unlikely to sustain [32]. Estrin and Sim make the case that there is a global communication network already in place to support an open mHealth architecture that could facilitate scalable and sustainable health information systems [32]. Interoperability will be critical to promote research initiatives. The largest investments to date in interoperable systems have been actively pursued by for-profit companies, given the staggering profits to be made in the proprietary applications market. What is needed is a concerted effort by governments, funders, and private enterprise to cooperate in order to set standards (e.g., number of bits) and to create a self-governing commercially viable ecosystem for innovation [32]. mHealth is in a period very similar to the early days of the Internet: not creating robust, interoperable platforms will ensure failure for mHealth initiatives to be scaled to improve health outcomes for at least the next decade.

What Needs to Happen Next: From Black Box to High Utility

The current wave of mHealth interventions are the equivalent of black boxes. Each small entrepreneur or researcher includes whatever bells and whistles that their funding allows in an attempt to demonstrate efficacy. For example, hundreds of small pilot studies are finding whether text messaging works. Text messaging is more likely to work under a set of parameters:

  • when there is follow-up;
  • when the message is personally tailored;
  • when the frequency, wording, and content are highly relevant.

Similar strategies are being experimented with for a range of topics, delivery strategies (web, phone, videos, social media sties), and populations. There are a set of principles that could potentially be established to identify the optimal strategies for delivering mHealth interventions. However, our current research is not aimed at identifying these principles and strategies. Each pilot study is examining whether their particular style of a black box application works better than not having any black box application. It is time to start funding randomized controlled trials of interventions that are based on researchers' best guesses about optimal implementation.

It is also time to consider the Multiphase Optimization Strategy (MOST) developed by Collins and colleagues [33]. The MOST strategy is grounded in an engineering approach and requires a two-stage process: 1) identifying the range of features that contribute to variation for a particular intervention; and 2) selecting a small set of factors and empirically testing them with a multi-factorial design. The initial set of factors to be screened might be determined on the basis of theory and/or experience and could be informed by research implementing evidence-based interventions with other delivery formats. The utility of such an approach has been demonstrated by Stretcher and colleagues [34] for web-based smoking cessation policies. Rather than having a single tested web-based, evidence-based intervention (EBI) that will then compete with other web-based EBI for smoking, there are a set of parameters that outline the optimal strategy for implementing a web-based programme. Similar strategies have long been adopted by health services researchers [35]. However, few of the existing studies utilizing mHealth delivery formats have adopted such an approach.

MOST is not the only approach that could potentially enhance the efficiency of existing mHealth studies. Duan [35] has advocated for the establishment of data farms. Nascent Internet companies such as Google, Yahoo, and Facebook provide informative case studies of data farms. Rather than use experimental research designs (such as randomized controlled trials), these companies can harvest data from billions of users of mobile, web, and social media, and computer-based interventions provide the evidence regarding the specific types of consumers who are attracted to specific types of delivery formats delivered with specific levels of doses at specific times. Data farms offer the opportunity to know the who, what, when, where, and how of reaching consumers [36]. Private enterprise has been outstanding at this function: mHealth needs to utilize their platforms and methods to optimize personal health.

Major donors could invest in creating a robust set of standards and a platform that can inform and support local adaptation of mHealth applications. The standardized features of the platform could then be available to all local technicians committed to improving the health of their local communities. At the very least, given that standards are expensive to establish, as well as often being complex and difficult to understand, one option is for an organization such as the World Health Organization to “certify” standards that meet particular criteria, or even to become a disseminator of standards. We also believe a global strategy for programmatic examination of the optimal features of the mobile platforms is needed, namely a platform that incorporates (for example) factorial designs to test the multiple features of interventions [37], the MOST strategy and even data farms. This could quickly identify and provide guidance to hundreds of thousands of programmers globally that could leverage donor investments to improve their communities' access to information, skills, telemedicine, or management of front line workers.

Box 1. Recommendations for Scale Up of mHealth

  1. Existing standards for research should be reconsidered in order to provide guidance as to when scale up is appropriate.
  2. mHealth interventions should be guided by a plausible theory of behaviour change and should use more than one technique depending on the targeted behaviour [38].
  3. We need to establish an open mHealth architecture based on a robust platform with standards for app development which would facilitate scalable and sustainable health information systems.
  4. Implementation strategies such as factorial designs that are able to test the multiple features of interventions must be explored, in order to provide the necessary evidence base.
  5. Scale-up of mHealth in LAMICs should be preceded by efficacy and effectiveness trials so that they are founded on an appropriate evidence base.
  6. Governments, funders, and industry must cooperate in order to set standards to create a self-governing commercially viable ecosystem for innovation.

Author Contributions

Wrote the first draft of the manuscript: MT MR-B. Contributed to the writing of the manuscript: MT AT LS MR-B. ICMJE criteria for authorship read and met: MT AT LS MR-B. Agree with manuscript results and conclusions: MT AT LS MR-B.


  1. 1. ICD (2012) Information and communications for development: maximizing mobile. Available: Accessed 6 January 2013.
  2. 2. van Heerden A, Tomlinson M, Swartz L (2012) Point of care in your pocket: a research agenda for the field of m-health. Bull World Health Organ 90: 393–394.
  3. 3. Tomlinson M, Solomon W, Singh Y, Doherty T, Chopra M, et al. (2009) The use of mobile phones as a data collection tool: a report from a household survey in South Africa. BMC Med Inform Decis Mak 9: 51.
  4. 4. Rotheram-Borus MJ, le Roux IM, Tomlinson M, Mbewu N, Comulada WS, et al. (2011) Philani Plus (+): a Mentor Mother community health worker home visiting program to improve maternal and infants' outcomes. Prev Sci 12: 372–388.
  5. 5. Siedner MJ, Haberer JE, Bwana MB, Ware NC, Bangsberg DR (2012) High acceptability for cell phone text messages to improve communication of laboratory results with HIV-infected patients in rural Uganda: a cross-sectional survey study. BMC Med Inform Decis Mak 12: 56.
  6. 6. Haberer JE, Robbins GK, Ybarra M, Monk A, Ragland K, et al. (2012) Real-time electronic adherence monitoring is feasible, comparable to unannounced pill counts, and acceptable. AIDS Behav 16: 375–382.
  7. 7. Qiang CZ, Yamamichi M, Hausman V, Altman D (2011) Mobile applications for the health sector. Washington: World Bank.
  8. 8. Collins F (2012) How to fulfill the true promise of “mHealth”: Mobile devices have the potential to become powerful medical tools. Sci Am 307: 16.
  9. 9. Lemaire J (2011) Scaling up mobile health: Elements necessary for the successful scale up of mHealth in developing countries. Geneva: Advanced Development for Africa.
  10. 10. Friedrichs C (2012) Market access & reimbursement strategies. Mobile Health Summit Cape Town. 29 May–1 June 2012.
  11. 11. Piette JD, Lun KC, Moura LA Jr, Fraser HS, Mechael PN, et al. (2012) Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here? Bull World Health Organ 90: 365–372.
  12. 12. Turkle S (2012) Alone together: why we expect more of technology and less of each other. New York: Basic Books.
  13. 13. Car J, Gurol-Urganci I, de Jongh T, Vodopivec-Jamsek V, Atun R (2012) Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database Syst Rev 7: CD007458.
  14. 14. Guy R, Hocking J, Wand H, Stott S, Ali H, et al. (2012) How effective are short message service reminders at increasing clinic attendance? A meta-analysis and systematic review. Health Serv Res 47: 614–632.
  15. 15. Vervloet M, Linn AJ, van Weert JC, de Bakker DH, Bouvy ML, et al. (2012) The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature. J Am Med Inform Assoc 19: 696–704.
  16. 16. Krishna S, Boren SA, Balas EA (2009) Healthcare via cell phones: a systematic review. Telemed J E Health 15: 231–240.
  17. 17. Horvath T, Azman H, Kennedy GE, Rutherford GW (2012) Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection. Cochrane Database Syst Rev 3: CD009756.
  18. 18. Cole-Lewis H, Kershaw T (2010) Text messaging as a tool for behavior change in disease prevention and management. Epidemiol Rev 32: 56–69.
  19. 19. Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, et al. (2010) Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 376: 1838–1845.
  20. 20. Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, et al. (2011) Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 25: 825–834.
  21. 21. da Costa TM, Barbosa BJ, Gomes e Costa DA, Sigulem D, de Fatima Marin H, et al. (2012) Results of a randomized controlled trial to assess the effects of a mobile SMS-based intervention on treatment adherence in HIV/AIDS-infected Brazilian women and impressions and satisfaction with respect to incoming messages. Int J Med Inform 81: 257–269.
  22. 22. Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer DH, et al. (2011) The effect of mobile phone text-message reminders on Kenyan health workers' adherence to malaria treatment guidelines: a cluster randomised trial. Lancet 378: 795–803.
  23. 23. Free C, Phillips G, Galli L, Watson L, Felix L, et al. (2013) The effectiveness of mobile-health technology based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 10: e1001362 doi:10.1371/journal. pmed.1001362.
  24. 24. Free C, Phillips G, Watson L, Galli L, Felix L, et al. (2013) The effectiveness of M-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med 10: e1001363 doi:10.1371/journal. pmed.1001363.
  25. 25. Rotheram-Borus MJ, Tomlinson M, Swendeman D, Lee A, Jones E (2012) Standardized functions for smart-phone applications: examples from maternal and child health. International Journal of Telemedicine and Applications 2012 doi:10.1155/2012/973237.
  26. 26. Mrazek PJ, Haggerty RJ (1994) Reducing risks for mental disorders: Frontiers for preventive intervention research. Washington, D.C.: Committee on Prevention of Mental Disorders: Institute of Medicine.
  27. 27. Flay BR, Biglan A, Boruch RF, Castro FG, Gottfredson D, et al. (2005) Standards of evidence: criteria for efficacy, effectiveness and dissemination. Prev Sci 6: 151–175.
  28. 28. Thornicroft G, Lempp H, Tansella M (2011) The place of implementation science in the translational medicine continuum. Psychol Med 41: 2015–2021.
  29. 29. Olds DL, Sadler L, Kitzman H (2007) Programs for parents of infants and toddlers: recent evidence from randomized trials. J Child Psychol Psychiatry 48: 355–391.
  30. 30. Aboud FE, Singla DR (2012) Challenges to changing health behaviours in developing countries: a critical overview. Soc Sci Med 75: 589–594.
  31. 31. Fishbein M, Bandura A, Triandis HC, Kanfer FH, Becker M, et al.. (2000) Factors influencing behavior and behavior change. In: Baum AS, Revenson T, Singer J, editors. Handbook of Health Psychology. New Jersey: Lawrence Earlbaum Associates.
  32. 32. Estrin D, Sim I (2010) Health care delivery. Open mHealth architecture: an engine for health care innovation. Science 330: 759–760.
  33. 33. Collins LM, Baker TB, Mermelstein RJ, Piper ME, Jorenby DE, et al. (2011) The multiphase optimization strategy for engineering effective tobacco use interventions. Ann Behav Med 41: 208–226.
  34. 34. Strecher VJ, McClure JB, Alexander GL, Chakraborty B, Nair VN, et al. (2008) Web-based smoking-cessation programs: results of a randomized trial. Am J Prev Med 34: 373–381.
  35. 35. Duan N, Alegria M, Canino G, McGuire TG, Takeuchi D (2007) Survey conditioning in self-reported mental health service use: randomized comparison of alternative instrument formats. Health Serv Res 42: 890–907.
  36. 36. Hay MC, Weisner TS, Subramanian S, Duan N, Niedzinski EJ, et al. (2008) Harnessing experience: exploring the gap between evidence-based medicine and clinical practice. J Eval Clin Pract 14: 707–713.
  37. 37. Collins LM, Chakraborty B, Murphy SA, Strecher V (2009) Comparison of a phased experimental approach and a single randomized clinical trial for developing multicomponent behavioral interventions. Clin Trials 6: 5–15.
  38. 38. Briscoe C, Aboud F (2012) Behaviour change communication targeting four health behaviours in developing countries: a review of change techniques. Soc Sci Med 75: 612–621.