Pneumonia is the leading infectious cause of death in children worldwide. Each year, pneumonia kills an estimated 935,000 children under five years of age, with most of these deaths occurring in developing countries. The current approach for pneumonia diagnosis in low-resource settings—using the World Health Organization Integrated Management of Childhood Illness (IMCI) paper-based protocols and relying on a health care provider’s ability to manually count respiratory rate—has proven inadequate. Furthermore, hypoxemia—a diagnostic indicator of the presence and severity of pneumonia often associated with an increased risk of death—is not assessed because pulse oximetry is frequently not available in low-resource settings. In an effort to address childhood pneumonia mortality and improve frontline health care providers’ ability to diagnose, classify, and manage pneumonia and other childhood illnesses, PATH collaborated with the University of Washington to develop “mPneumonia,” an innovative mobile health application using an Android tablet. mPneumonia integrates a digital version of the IMCI algorithm with a software-based breath counter and a pediatric pulse oximeter. We conducted a design-stage usability field test of mPneumonia in Ghana, with the goal of creating a user-friendly diagnostic and management tool for childhood pneumonia and other childhood illnesses that would improve diagnostic accuracy and facilitate adherence by health care providers to established guidelines in low-resource settings. The results of the field test provided valuable information for understanding the usability and acceptability of mPneumonia among health care providers, and identifying approaches to iterate and improve. This critical feedback helped ascertain the common failure modes related to the user interface design, navigation, and accessibility of mPneumonia and the modifications required to improve user experience and create a tool aimed at decreasing mortality from pneumonia and other childhood illnesses in low-resource settings.
Citation: Ginsburg AS, Delarosa J, Brunette W, Levari S, Sundt M, Larson C, et al. (2015) mPneumonia: Development of an Innovative mHealth Application for Diagnosing and Treating Childhood Pneumonia and Other Childhood Illnesses in Low-Resource Settings. PLoS ONE 10(10): e0139625. https://doi.org/10.1371/journal.pone.0139625
Editor: I. King Jordan, Georgia Institute of Technology, UNITED STATES
Received: June 30, 2015; Accepted: September 14, 2015; Published: October 16, 2015
Copyright: © 2015 Ginsburg et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files. Data are from the mPneumonia field testing in Ghana whose authors may be contacted at PATH.
Funding: The development of mPneumonia was funded by United States National Institutes of Health (ASG, JD, WB, RA) and National Science Foundation Research Grant No. IIS-111143 (WB, SL, MS, CL, GB, RA). The field testing in Ghana was funded by the United Nations Children's Fund (ASG, JD, CTA, SN). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Pneumonia is the leading infectious cause of death among children less than five years of age Estimates indicate that 935,000 children died from pneumonia in 2013—more than 2,500 children every day—accounting for 15 percent of child deaths globally. Childhood pneumonia can be difficult to diagnose, especially in low-resource settings (LRS)  Often pneumonia goes undiagnosed and untreated until the child is severely ill or is improperly managed by low-skilled health care providers (HCPs). Despite pneumonia-related mortality being preventable with simple interventions and appropriate treatment, diagnosing and managing childhood pneumonia in LRS remain challenging.
The current approach—using paper-based protocols and relying on an HCP’s ability to manually count breaths to calculate a respiratory rate (RR)—has proven inadequate in addressing this challenge [3–9] Recognizing difficult or fast breathing is central to the World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) and integrated Community Case Management algorithms, which are used to diagnose childhood pneumonia in LRS [10, 11] This is accomplished by visually observing the child’s breathing and measuring the RR, typically for one minute. However, assessing a child’s RR can be very challenging in a fast-breathing, moving child, and studies have shown that algorithms based on RR alone do not adequately identify children with pneumonia [2, 12–14] Furthermore, clinical assessments are not sufficient for determining the severity of the disease or the risk of complications. Pulse oximetry can noninvasively measure oxygen saturation and detect hypoxemia, a diagnostic indicator of the presence and severity of pneumonia often associated with an increased risk of death. Yet, hypoxemia is not routinely assessed in LRS because pulse oximetry is frequently not available  In addition, despite the WHO encouraging its use, pulse oximetry is not included in the current IMCI algorithm.
Recent evidence suggests that mobile health (mHealth)-based applications can improve HCP adherence to diagnostic and treatment guidelines [16–20] In an effort to address childhood pneumonia mortality and improve HCPs’ ability to diagnose, classify, and manage childhood pneumonia as well as other childhood illnesses, we developed “mPneumonia,” an innovative mHealth-based application using Android phone or tablet technology. mPneumonia integrates a digital version of the IMCI algorithm with a software-based breath counter to help HCPs more accurately count a child’s breaths and an off-the-shelf reusable pediatric pulse oximeter to detect hypoxemia; this provides a user-friendly diagnostic and management tool for childhood pneumonia and other childhood illnesses to improve diagnostic accuracy and facilitate adherence by HCPs to established diagnostic process. While pneumonia is a primary focus of this application given its large burden of disease among children, the mPneumonia application is not only intended for pneumonia diagnosis and treatment; it also facilitates the diagnosis and treatment of all childhood illnesses in accordance with IMCI.
In collaboration with the University of Washington Department of Computer Science and Engineering, PATH employed an iterative development methodology to create and design the integrated mPneumonia prototype. We used Open Data Kit (ODK) Survey, a free open-source platform, to implement the complex and adaptable workflows of IMCI, provide interactive nonlinear navigation capabilities, and enable run-time customizable navigation and question data types For example, ODK Survey was designed to accommodate questions with complex branching and user-directed nonlinear flow requirements. We used the ODK 2.0 Tools Suite to digitize and enhance the WHO IMCI diagnostic protocol, transforming the paper-based WHO IMCI algorithm into a step-by-step, user-friendly assessment questionnaire with embedded instructions, pictures, and videos on the Android operating system. The application did not require a network or wireless connection to function and could be used completely offline. It could store information locally until connected to a network, after which data files could be uploaded to a central server.
The software-based manual breath counter was developed to facilitate rapid identification of fast breathing, which was calculated and recorded by tapping the screen for each breath observed over a one-minute period. Visual, audio, and vibration indicators provided feedback to the user when a tap has occurred as well as when the measurement period has ended. The breath counter started as soon as the HCP began recording and stopped automatically when the measurement period was completed. The clinical algorithms were further adapted to interpret oxygen saturation data. The pulse oximeter data was recorded using an off-the-shelf Nonin 8000AP reusable pediatric finger clip pulse oximeter attached to a Nonin Xpod, enabling the connection between the pulse oximeter sensor and the mobile Android device. A custom device driver was written to integrate the Nonin pulse oximeter with the ODK Sensors framework. Nonin’s SmartPoint™ Algorithm was employed to indicate when the reading was high quality and the user should record the data (the numbers displayed on the Android device turned from pink to blue). This would prevent users from recording the pulse oximeter reading if the sensor was not worn correctly. While the pulse oximeter could accurately determine a child’s oxygen saturation, we chose not to use this feature in this study. Since we were only assessing the feasibility, usability, and acceptability of the mPneumonia application in this study, and not validating the application, breath counter, or pulse oximeter, we wanted to ensure the application was not used in the clinical care of the patient and that it did not influence the care of the patient in any way. Therefore, we set the oxygen saturation to read 99 percent at all times.
All components were integrated into a user-friendly decision support system, with clinical and physiological measurement results automatically guiding the protocol decision-making process and recommending the appropriate treatment or rapid referral to the closest facility with available treatments. The tablet language was in English, the preferred language among HCPs in Ghana. Requiring a username and password for access, mPneumonia was designed for secure use on Asus Nexus Google 7 tablets with a 7-inch portrait display running Android version 4.2.
mPneumonia is targeted for use by lesser-trained HCPs in LRS. In accordance with the consolidated criteria for reporting qualitative research, or COREQ, we undertook field usability testing in March 2014 among end-users in one district in the Brong-Ahafo region of Ghana  The objectives were to identify and resolve usability problems prior to a larger-scale introduction and pilot of mPneumonia within community health clinics in Ghana, and to increase the likelihood of successful integration into users’ daily surroundings and routines. The design and development process took into consideration local health policies, health system structures, and resource availability, which are especially important in LRS. After pilot testing the interview guides, purposive and snowball sampling techniques were used to recruit representative users from two peripheral health clinics. All HCPs that had knowledge about or provided direct treatment to children under five years of age were eligible. A tablet with the pulse oximeter hardware connected and software settings already preconfigured was provided to each HCP to use.
In a private setting at the clinic where the HCP worked, task analysis, concurrent think-aloud, and retrospective probing techniques were used to collect qualitative and quantitative usability metrics. Task analysis was used to understand in detail how the users interacted with mPneumonia. The researcher silently observed and focused on extracting the processes and tasks the users undertook when using the application. Concurrent with the task analysis exercise, the users were asked to “think aloud” and explain what they were doing, thinking, and feeling as they interacted with the application. The task analysis mapped the processes and steps the users performed, and this was supplemented by objective comments from the users as they verbalized their thought processes to yield information on the usability of mPneumonia. Retrospective probing was employed after the users completed their interactions with the application. The probing questions aimed to further examine user needs and perceptions, including perceived efficacy and usefulness of mPneumonia, and to get recommendations to improve the application. Thus, HCPs were asked to complete a case scenario without guidance or prior training in the use of mPneumonia, and to speak aloud while a researcher marked observations about user pathways and potential failure modes. A short questionnaire and system usability ratings were used to gather feedback around the perceived usability of mPneumonia.
Analysis of the data involved manually sorting and summarizing the data using a mixed methods semi-quantitative and qualitative approach guided by our evaluation criteria (Table 1). All data were cleaned, coded, and de-identified prior to analysis. The PATH research determination committee classified this as a design-stage evaluation activity. This human subject research was conducted according to the principles expressed in the Declaration of Helsinki, and written informed consent was obtained from all study participants.
Seven HCPs were enrolled in the design-stage usability evaluations in two peripheral government community health centers in the Nkoranza district of the Brong-Ahafo region in Ghana. Supervised by a midwife or community health officer (CHO) with the authority to prescribe medications, the health centers house a dispensary and basic laboratory. The IMCI-trained midwife or CHO oversees community health nurses (CHNs) who are also trained in IMCI to make referrals to the district hospital, which is staffed by physicians. Midwives, CHOs, and CHNs have completed at least two years of training. Nurses and field technicians with limited technical training provide additional health care support. All included HCP were literate and could read, write, and speak in English.
One midwife, one CHO, two CHNs, two enrolled nurses, and one field technician participated in the usability data collection and short interview. In order to maintain confidentiality, HCP responses are not reported by cadre here. The majority (five of seven) of these HCPs reported this experience to be their first time to use a touch-screen device and expressed the need to become familiar with the technology. Two HCPs owned standard mobile phones, two owned Android phones, and three did not own any mobile phones. Six of the seven HCPs were female. All seven HCPs were familiar with the IMCI assessment algorithm and medical terms presented within the application.
Initial reactions to the mPneumonia application and tablet device were positive (Table 2). Most HCPs first commented on the attractive size and shape of the tablet. HCPs reported that the portable, “modern,” and slim design of the tablet was comfortable to hold and “easy…to use without glasses.” A few concerns regarding the robustness and reliability of the tablet and accessories, the need for regular maintenance and replacement parts, and potential theft or damage were raised. HCPs suggested that theft could easily be mitigated by providing a hard, lockable case to enable storage within a drawer at the clinic or at home. Most HCPs commented that the software appeared to be fast but found it difficult and struggled to get the application started.
All seven HCPs successfully completed the application procedures, corresponding with one patient encounter, within a mean of 43.4 (range 24 to 100) minutes. The task analysis activities identified 17 critical (made without noticing) and nine noncritical (almost made, or made and corrected) errors that prohibited HCPs from completing tasks within the application effectively and efficiently (Tables 3 and 4). Most critical errors were related to user interface design and software navigation buttons. Other critical failure modes were related to the first-time use of the breath counter and pulse oximeter, and responses to the IMCI assessment questions. If the HCPs misunderstood the IMCI assessment questions and submitted incorrect responses, mPneumonia would produce an incorrect classification. Noncritical errors were related to the tablet and hardware itself. Most HCPs were unfamiliar with touch-screen technology and thus were initially hesitant to use the device. HCPs struggled to identify how to change their answer selections but were later able to self-correct with practice. HCPs took the most time using the application during system navigation (when starting the tablet, using the touch screen, and using the QWERTY keyboard), software navigation (when opening and starting the application, moving between pages, reading instructions and IMCI text, and submitting answers), breath counter use, and pulse oximeter use. The HCPs failed to read the directions carefully and frequently attempted to skip questions; however, in an effort to increase adherence to the IMCI protocol, mPneumonia prohibits unanswered questions. The inclusion of images, such as for palmar pallor, seemed to enable HCPs to answer questions quickly without having to read the text.
Ease of use and learnability
All seven HCPs reported that they imagined that most people would learn to use the application quickly. HCPs using a touch screen for the first time noted the need to become familiar with the technology and software. One user explained this commonly expressed sentiment by saying, “It wasn’t easy because I was not conversant with the keys. Frequent use of the device will help a lot.” HCPs who did own an Android phone demonstrated the ability to move through the application relatively quickly. Short interviews revealed that all HCPs expressed a desire and willingness to use the application and thought nominal training and technical support would be required for first-time users. HCPs reported that an average of 16.3 hours, or two days, of training would be required to become fully comfortable using the application with pediatric patients.
Content accuracy and appropriateness
HCPs provided their comments on the accuracy of the clinical information in the mPneumonia application and appropriateness at their level of the health system. HCPs expressed appreciation for having all of the assessment questions available in sequential order and predicted that use of mPneumonia would reduce the amount of paperwork compared to the paper-based protocol. Overall, all HCPs preferred mPneumonia to the paper-based protocol. HCPs commented that mPneumonia would provide a more accurate and automatic classification or referral decision. HCPs requested that additional detailed medical advice (e.g., predisposing factors for pneumonia, feeding assessments, and counseling information for mothers and children being referred) be included, as well as all diagnoses for use with all patient populations and conditions. Although researchers identified during the task analysis that HCPs struggled to use the breath counter and pulse oximeter the first time, the questionnaire revealed that HCPs found both of these tools to be easy to use and helpful for their work. One user summarized, “It would make our work easier and fast for treatment or referrals. It is easy to me because you will not have to manually count anything.”
Further analysis of the qualitative data revealed clinical and technical experience, resources, and context largely defined user expectations and behaviors. HCPs with minimal training, experience, or lack of adherence to IMCI guidelines often expressed surprise or confusion with the IMCI classification results. However, HCPs with greater levels of training and experience confirmed agreement with the IMCI protocol of the mPneumonia application. Study HCPs remarked that mPneumonia could add value to the management of childhood pneumonia by increasing patient demand and efficiency of clinic visits.
Initially hesitant, HCPs complained about the lack of instruction prior to use of mPneumonia and demonstrated difficulties during their first-time use. However, by the end of their user experiences, all seven HCPs expressed overall high levels of satisfaction and that they would use the application frequently if it were available (Table 5). HCPs also suggested that mPneumonia could be useful to those HCPs outside of the clinic who perform referrals and community outreach.
The growing success of mHealth applications in LRS demonstrates the potential for a childhood pneumonia application to transform health care service delivery and improve health outcomes. Recent evidence suggests that digital technology and mHealth-based algorithms can enhance the ability of peripheral and lower-level facility and community HCPs to follow diagnostic and treatment guidelines [9, 16–20, 23] Our approach is to expand the technology options for assessment, classification, and management of pneumonia and other childhood illnesses through the development of an innovative mHealth application that integrates IMCI and a diagnostic and clinical decision support algorithm with a breath counter and a low-cost portable pulse oximeter.
Current standards of care are critically dependent upon the correct application of the WHO IMCI and integrated Community Case Management protocols for identification and treatment of childhood illnesses. However, use of the WHO protocols remains limited due to a lack of trained HCPs, insufficient supervision, and inadequate financial resources [3, 5–7] Even with training, HCPs often do not adhere to the WHO guidelines in practice, leaving clinical assessments incomplete and children requiring urgent referrals missed [6, 24] mPneumonia is designed to minimize the need for training, improve adherence to the IMCI algorithm, and provide clear guidance for interpreting results. The step-by-step software increases adherence to the algorithm by not allowing HCPs to skip essential assessment questions; this enables lesser-trained HCPs to identify children with pneumonia and other childhood illnesses and, most importantly, those at highest risk of death and in need of immediate treatment.
Measuring a child’s breath rate through visual observation is challenging, requiring focused concentration. Inaccurate or imprecise measurements can stem from difficulty counting or remembering the count  The mPneumonia breath counter enables the HCP to focus exclusively on observing and recording each breath without also needing to keep track of the count or the time. mPneumonia includes a pulse oximeter. While pulse oximetry is a critical component in the management of severe pneumonia and its use encouraged by the WHO, there is no designated step for it in the current IMCI algorithm and it is frequently not available in LRS. By providing a quantitative indicator, mPneumonia allows for more effective and cost-effective use of oxygen, a lifesaving therapy  Additionally, pulse oximetry can be beneficial even in the absence of supplemental oxygen because it can be used to prioritize or urgently refer patients to facilities with more intensive care.
The results of the initial field test in Ghana provided valuable information for understanding the usability and acceptability of mPneumonia among HCPs. This critical feedback helped identify the common failure modes related to the user interface design, navigation, and accessibility of mPneumonia and the modifications required to improve the user experience (Tables 3 and 4). Larger-sized buttons and the use of colors have now been incorporated into the next iteration of the mPneumonia prototype to address the frequent misinterpretation of the software buttons as a set of instructions instead of as a key to press. Additional changes included: removing scrolling, enlarging font size of text, simplifying questions to “yes/no” answers, providing feedback to the HCP on answer selections, positioning answer selection buttons horizontally instead of vertically, and adding navigation buttons to the bottom of the screen (Fig 1). Appropriate utilization of different colors, shapes, sizes, shading, icons, and images may improve the user experience and have the potential to translate into significant improvements in the efficiency and accuracy of mPneumonia.
User expertise and experience were influential factors, determining preferences for customization of the navigation and content of mPneumonia. Decreasing the amount of text necessary for the HCP to read and inserting symbols and images in the application (e.g., how the pulse oximeter probe should be placed on the finger) can reduce confusion and facilitate easier and faster use (Fig 2). Possibly burdened by high patient workloads, HCPs frequently tried to skip assessment questions. Additional error messages have been inserted into the next iteration to prevent HCPs from accidentally jumping ahead in the IMCI protocol. Given that clinicians average 8.2 minutes per child for a consultation using the paper-based IMCI protocol, 43 minutes (average time to complete the task analysis) is likely not feasible for most HCPs  It will be important to monitor how long it takes HCPs to navigate the application with daily routine use and increased familiarity.
Overall, the initial field test revealed that HCPs viewed mPneumonia favorably. HCP were in agreement on the choice of the tablet rather than an Android phone. HCPs preferred the large size and shape of the tablet. The tablet also reduced the risk of HCPs inserting their SIM cards and using the device for personal use. The capability of mPneumonia to operate completely offline not only provided significant benefit for health clinics lacking connectivity, it also inadvertently promoted the restricted use of the tablet for patient care. Despite concerns expressed by HCPs regarding theft, damage, and lack of access to electricity, all HCPs commented on the perceived value of mPneumonia and expressed interest in using mPneumonia if it were available.
The mPneumonia application has limitations, including the language of IMCI questions and the tablet itself. To address these limitations for the planned pilot study, we did not change the text of the IMCI questions but instead addressed confusion related to double negative IMCI questions by altering the user interface design and inserting red color feedback to highlight answer selections correlating with an abnormal exam finding. We developed a pocketbook mPneumonia user manual, and will provide training and an opportunity to practice using the mPneumonia application during the pilot study. For example, we will try to address in training the basics of using a QWERTY keyboard and that the user must press “enter” on the keyboard when submitting a response to make the pop-up keyboard disappear. There was also limited ability to modify the structure of the ODK Survey platform to adapt it to context-specific needs of user populations. To accommodate varying levels of training and experience among HCPs and resource availability across clinics, we identified the need to improve the flexibility of the system and display format to permit the complex alignment of the content and sequence of mPneumonia with local Ghana IMCI protocols and clinical practice. Despite these limitations, ODK Survey provides a free, open-source, simple and usable platform for designing and implementing IMCI protocols. With additional feedback from HCPs on how to increase the usability of the platform, the functionality of the ODK Survey platform has the potential to expand and improve.
Recommendations for future iterations to expand the functionality of mPneumonia included adding and synchronizing patient and drug inventory registries as well as providing educational support. A patient registry could enable quick access to the child’s vaccination records and monitoring of the child’s growth over time. Since clinics varied in their availability of drug options, a drug inventory registry could allow the flexibility to select the appropriate treatments based on availability. A drug inventory registry allowing the user the ability to pre-select and maintain the drugs available in the clinic could also provide restock reminders when drugs are running low. Another suggestion was a tool tip pop-up window allowing the HCP to scroll onto an unfamiliar term for additional information (e.g., images or videos). Future pre- and post-evaluation studies are needed to assess HCPs’ knowledge and adherence to the IMCI guidelines before and after use of mPneumonia to validate whether mPneumonia is a useful tool to facilitate increased adherence to IMCI guidelines. Furthermore, the clinical effectiveness of mPneumonia still requires investigation, as does determining the sustainable uptake. Research regarding the cost-effectiveness of mPneumonia is also needed. A costing study was outside the scope of this field test, and most of the components of the mPneumonia application were donated. However, since ODK Survey and the breath counter application are freely available Android applications that run on commodity Android devices, the main driver of cost of mPneumonia is likely to be the Android-compatible pediatric pulse oximeter.
This study provides a model for assessing the usability and acceptability of an mHealth application among HCPs, and identifying approaches to iterate and improve. The areas for improvement identified during this design-stage field test were essential to ensure that the application was ready for the pilot study. The field testing allowed PATH to provide a set of recommendations to the technology developers that translated directly to the iterative design, improved interface, and enhanced functionality of an adapted mPneumonia. The implemented changes will be tested during a pilot study of the acceptability and feasibility of introducing mPneumonia among HCPs and caregivers of children in community health clinics in the Kintampo North and South districts of Ghana.
mPneumonia integrates an easy-to-use, step-by-step guide to improve adherence to diagnostic protocols, incorporates physiological measurements such as RR and pulse oximetry, and provides overall decision-making support. Accelerating the development of new childhood pneumonia innovations and spurring the adaptation of current mHealth devices could revolutionize pneumonia diagnosis and treatment. It will be important to prioritize the needs of HCPs and the constraints of using this technology in a LRS. New innovations must not only be evaluated for their effectiveness in diagnosing and treating children but also must be assessed for use in LRS. mPneumonia has the potential to enable rapid diagnosis and assessment, reduce the risk of death due to delayed treatment, and make an impact on the leading causes of childhood mortality.
The authors would like to acknowledge the support of Kintampo Health Research Centre and researchers Solomon Nyame and Charles Kyei in conducting the field test.
Conceived and designed the experiments: ASG JD WB CTA SN GB RA. Performed the experiments: ASG JD CTA SN. Analyzed the data: ASG JD CTA. Wrote the paper: ASG JD. Designed the concept and content of mPneumonia: ASG JD WB GB RA. Completed the software engineering: JD WB SL MS CL. Oversaw and conducted the field testing: ASG JD CTA SN.
- 1. Liu L, Oza S, Hogan D, Perin J, Rudan I, Lawn JE, et al. Global, regional, and national causes of child mortality in 2000–13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet. 2015;385(9966):430–40. pmid:25280870.
- 2. Rambaud-Althaus C, Althaus F, Genton B, D'Acremont V. Clinical features for diagnosis of pneumonia in children younger than 5 years: a systematic review and meta-analysis. Lancet Infect Dis. 2015;15(4):439–50. pmid:25769269.
- 3. Armstrong Schellenberg J, Bryce J, de Savigny D, Lambrechts T, Mbuya C, Mgalula L, et al. The effect of Integrated Management of Childhood Illness on observed quality of care of under-fives in rural Tanzania. Health Policy Plan. 2004;19(1):1–10. pmid:14679280.
- 4. Kallander K, Tomson G, Nsabagasani X, Sabiiti JN, Pariyo G, Peterson S. Can community health workers and caretakers recognise pneumonia in children? Experiences from western Uganda. Trans R Soc Trop Med Hyg. 2006;100(10):956–63. pmid:16455119.
- 5. Victora CG, Huicho L, Amaral JJ, Armstrong-Schellenberg J, Manzi F, Mason E, et al. Are health interventions implemented where they are most needed? District uptake of the integrated management of childhood illness strategy in Brazil, Peru and the United Republic of Tanzania. Bull World Health Organ. 2006;84(10):792–801. pmid:17128359; PubMed Central PMCID: PMC2627500.
- 6. Walter ND, Lyimo T, Skarbinski J, Metta E, Kahigwa E, Flannery B, et al. Why first-level health workers fail to follow guidelines for managing severe disease in children in the Coast Region, the United Republic of Tanzania. Bull World Health Organ. 2009;87(2):99–107. pmid:19274361; PubMed Central PMCID: PMC2636200.
- 7. Ahmed HM, Mitchell M, Hedt B. National implementation of Integrated Management of Childhood Illness (IMCI): policy constraints and strategies. Health Policy. 2010;96(2):128–33. pmid:20176407.
- 8. Mukanga D, Babirye R, Peterson S, Pariyo GW, Ojiambo G, Tibenderana JK, et al. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health. 2011;16(10):1234–42. pmid:21752163.
- 9. Shao AF, Rambaud-Althaus C, Swai N, Kahama-Maro J, Genton B, D'Acremont V, et al. Can smartphones and tablets improve the management of childhood illness in Tanzania? A qualitative study from a primary health care worker's perspective. BMC Health Serv Res. 2015;15:135. pmid:25890078; PubMed Central PMCID: PMC4396573.
- 10. Lehmann U, Sanders D. Community health workers: What do we know about them? The state of the evidence on programmes, activities, costs and impact on health outcomes of using community health workers. Geneva: World Health Organization, 2007.
- 11. WHO. Integrated Management of Childhood Illness Chart Booklet. Geneva: World Health Organization, 2014.
- 12. UNICEF. Synovate, ARI Timer Research Report. 2011.
- 13. UNICEF. Pneumonia diagnostics: current outlook and perspectives. 2013.
- 14. UNICEF. Strengthening pneumonia diagnostic tools for low-resource settings. 2013.
- 15. Ginsburg AS, Van Cleve WC, Thompson MI, English M. Oxygen and pulse oximetry in childhood pneumonia: a survey of healthcare providers in resource-limited settings. J Trop Pediatr. 2012;58(5):389–93. pmid:22170511.
- 16. DeRenzi B, Parikh T, Mitchell M, Chemba M, Schellenberg D, Lesh N, et al., editors. e-IMCI: Improving Pediatric Health Care in Low-Income Countries CHI; 2008 April 5–10; Florence, Italy.
- 17. Mitchell M, Lesh N, Cranmer H, Fraser H, Haivas I, Wolf K. Improving care–improving access: the use of electronic decision support with AIDS patients in South Africa. Int J Healthc Technol Manage. 2009;10(3):156–68.
- 18. Mitchell M, Hedt BL, Eshun-Wilson I, Fraser H, John MA, Menezes C, et al. Electronic decision protocols for ART patient triaging to expand access to HIV treatment in South Africa: a cross sectional study for development and validation. Int J Med Inform. 2012;81(3):166–72. pmid:22178295; PubMed Central PMCID: PMC3279573.
- 19. Mitchell M, Getchell M, Nkaka M, Msellemu D, Van Esch J, Hedt-Gauthier B. Perceived improvement in integrated management of childhood illness implementation through use of mobile technology: qualitative evidence from a pilot study in Tanzania. J Health Commun. 2012;17 Suppl 1:118–27. pmid:22548605.
- 20. Mitchell M, Hedt-Gauthier BL, Msellemu D, Nkaka M, Lesh N. Using electronic technology to improve clinical care—results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC Med Inform Decis Mak. 2013;13:95. pmid:23981292; PubMed Central PMCID: PMC3766002.
- 21. Open data kit 2.0: expanding and refining information services for developing regions [Internet]. ACM Digital Library. 2013.
- 22. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. pmid:17872937.
- 23. Black J, Gerdtz M, Nicholson P, Crellin D, Browning L, Simpson J, et al. Can simple mobile phone applications provide reliable counts of respiratory rates in sick infants and children? An initial evaluation of three new applications. Int J Nurs Stud. 2015;52(5):963–9. pmid:25712876.
- 24. Horwood C, Voce A, Vermaak K, Rollins N, Qazi S. Experiences of training and implementation of integrated management of childhood illness (IMCI) in South Africa: a qualitative evaluation of the IMCI case management training course. BMC Pediatr. 2009;9:62. pmid:19796377; PubMed Central PMCID: PMC2765431.
- 25. Duke T, Graham SM, Cherian MN, Ginsburg AS, English M, Howie S, et al. Oxygen is an essential medicine: a call for international action. Int J Tuberc Lung Dis. 2010;14(11):1362–8. pmid:20937173; PubMed Central PMCID: PMC2975100.
- 26. Duke T, Subhi R, Peel D, Frey B. Pulse oximetry: technology to reduce child mortality in developing countries. Ann Trop Paediatr. 2009;29(3):165–75. pmid:19689857.
- 27. Adam T, Manzi F, Schellenberg JA, Mgalula L, de Savigny D, Evans DB. Does the Integrated Management of Childhood Illness cost more than routine care? Results from the United Republic of Tanzania. Bull World Health Organ. 2005;83(5):369–77. /S0042-96862005000500013. pmid:15976878; PubMed Central PMCID: PMC2626239.