https://orcid.org/0000-0002-0214-1267
Figures
Abstract
Adult medical male circumcision (MC) is safe: global notifiable adverse event (AE) rates average below 2.0%. With Zimbabwe’s shortage of health care workers (HCWs) compounded by COVID-19 constraints, two-way text-based (2wT) MC follow-up may be advantageous over routinely scheduled in-person reviews. A 2019 randomized control trial (RCT) found 2wT to be safe and efficient for MC follow-up. As few digital health interventions successfully transition from RCT to scale, we detail the 2wT scale-up approach from RCT to routine MC practice comparing MC safety and efficiency outcomes. After the RCT, 2wT transitioned from a site-based (centralized) system to hub-and-spoke model for scale-up where one nurse triaged all 2wT patients, referring patients in need to their local clinic. No post-operative visits were required with 2wT. Routine patients were expected to attend at least one post-operative review. We compare 1) AEs and in-person visits between 2wT men from RCT and routine MC service delivery; and 2) 2wT-based and routine follow-up among adults during the 2wT scale-up period, January to October 2021. During scale-up period, 5084 of 17417 adult MC patients (29%) opted into 2wT. Of the 5084, 0.08% (95% CI: 0.03, 2.0) had an AE and 71.0% (95% CI: 69.7, 72.2) responded to ≥1 daily SMS, a significant decrease from the 1.9% AE rate (95% CI: 0.7, 3.6; p<0.001) and 92.5% response rate (95% CI: 89.0, 94.6; p<0.001) from 2wT RCT men. During scale-up, AE rates did not differ between routine (0.03%; 95% CI: 0.02, 0.08) and 2wT (p = 0.248) groups. Of 5084 2wT men, 630 (12.4%) received telehealth reassurance, wound care reminders, and hygiene advice via 2wT; 64 (19.7%) were referred for care of which 50% had visits. Similar to RCT outcomes, routine 2wT was safe and provided clear efficiency advantages over in-person follow-up. 2wT reduced unnecessary patient-provider contact for COVID-19 infection prevention. Rural network coverage, provider hesitancy, and the slow pace of MC guideline changes slowed 2wT expansion. However, immediate 2wT benefits for MC programs and potential benefits of 2wT-based telehealth for other health contexts outweigh limitations.
Author summary
Although digital health innovations hold promise to improve patient outcomes and reduce workload, there is very little evidence of success from routine settings in lower income countries. We successfully scaled a two-way texting (2wT) intervention from research to routine practice, providing men with SMS-based telehealth for male circumcision (MC) follow-up rather than requiring in-person reviews. 2wT operates without consistent connectivity, supports providers with decision-making tools, and requires patients to have only basic phones. Like in the research setting, 2wT reduced healthcare worker workload and maintained patient safety when implemented with routine MC teams. The COVID-19 pandemic led added urgency to 2wT expansion. We describe the successes and challenges of the 2wT scale-up, providing lessons learned that may enable others to adapt and adopt this open-source App in other health contexts that could benefit from a similar, low-cost, short-term, telehealth approach.
Citation: Marongwe P, Wasunna B, Gavera J, Murenje V, Gwenzi F, Hove J, et al. (2022) Transitioning a digital health innovation from research to routine practice: Two-way texting for male circumcision follow-up in Zimbabwe. PLOS Digit Health 1(6): e0000066. https://doi.org/10.1371/journal.pdig.0000066
Editor: Shelagh Mulvaney, Vanderbilt University, UNITED STATES
Received: January 23, 2022; Accepted: May 16, 2022; Published: June 15, 2022
Copyright: © 2022 Marongwe et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant aggregate data used in this analysis, including adverse events and follow-up visit data from ZAZIC routine reporting, is provided in the manuscript and supplemental file data tables. The individual-level routine data aggregated for the ZAZIC study is owned by the Ministry of Health of Child Care (MoHCC) and are not publicly available due to MoHCC data restrictions and confidentiality protections. However, data may be made available for researchers who sign a data sharing agreement and abide by strict confidentiality protocols. Interested researchers may contact Jane Edelson jedelson@uw.edu, Regulatory Specialist at University of Washington, for access requests.
Funding: The ZAZIC VMMC program and its 2wT component that provided data for these analyses was supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) (https://www.state.gov/pepfar/) through the US Centers for Disease Control and Prevention (CDC) (https://www.cdc.gov), under the terms of CDC cooperative NU2GGH002116 to the International Training and Education Center for Health and the University of Washington, Principal Investigator, SB. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Background
By 2020, voluntary medical male circumcision (MC) reached nearly 27 million men in Southern Africa [1,2] with an average moderate and severe adverse event (AE) rate of 0.8% (range: 0.4–8.0) [3–8]. This corresponds to 99% of men healing well. In routine MC programs, men are recommended to have multiple, in-person reviews by a nurse or doctor, ideally on post-operative days 2, 7 and 42, to assess healing progress [9]. Although in-person follow-up rates may be overreported [8,10], global adherence to these visits is reported as high [11]. Requiring multiple post-operative visits where most men heal without complication is highly inefficient, raises unnecessary COVID-19 exposure risks, and increases healthcare worker (HCW) burden.
A 2019 randomized control trial (RCT) led by the International Training and Education Center for Health (I-TECH) at the University of Washington (UW); technology partner, Medic, Nairobi, Kenya; and the ZAZIC consortium (a name comprising letters from Zimbabwe Community Health intervention Research project (ZICHIRE); Zimbabwe Association of Church-related Hospitals (ZACH); and Zimbabwe Technical Assistance Training and Education Centre for Health (Zim-TTECH)) determined that SMS interaction between patients and MC providers for 13 critical post-operative days reduced follow-up workload by 87%, doubled AE ascertainment, and lowered costs by over $2 per patient [12,13] as compared to routine in-person care. HCWs and patients found the system highly usable, and both felt reassured that men could heal independently or seek care when needed [12].
Like many other health services, COVID-19 infection prevention efforts forced a pause in ZAZIC’s MC service provision in Zimbabwe in March 2020. Although precautions were implemented to reduce COVID-19 infection spread during MC surgical procedures, returning for multiple reviews heightened provider and patient COVID-19 risks. To reduce risk of COVID-19 transmission, Zimbabwe’s Ministry of Health and Child Care (MoHCC) approved wide use of 2wT for MC follow-up in February 2021.
We detail transitioning 2wT-based follow-up from RCT to scale-up within ZAZIC’s routine MC services, comparing in-person visits (workload) and AEs between 1) 2wT men from RCT and routine contexts; and 2) routine 2wT and in-person follow-up from January 2021 to October 2021. We describe scale-up successes and challenges to inform MC policy and to increase the sustainability of future digital health innovations.
Methods
2wT program description
2wT Technology.
Since 2008, Medic (formerly, Medic Mobile), a leader in the global mHealth community, supported almost 40,000 health workers to provide over 60 million caring activities across 25 countries. Medic is the steward of Community Health Toolkit (CHT), an open-source project supporting dozens of mHealth implementations [14–17] and user-centred designed apps, including for 2wT in Zimbabwe [12,18] (Figs 1–3). CHT interventions and apps work with or without internet connectivity, in any language, on basic phones, smartphones, tablets, or computers. CHT apps are tailored to meet 80% of the key digital health characteristics recommended by World Health Organization (WHO) [19] and adhere to Fast Healthcare Interoperability Resources (FHIR) standards. 2wT is based on Medic’s CHT and includes automated and interactive messaging, task management features (incomplete task; message response needed), longitudinal patient records, data collection forms, routine syncing, and dashboards for routine monitoring (e.g., patient response rates, AEs).
This 2wT tab displays options for nurses to complete various 2wT Reports and Tasks for any enrolled patient on the PC or mobile (tablet or phone) app.
This 2wT tab provides an interface for interactive SMS between 2wT patients and providers on the PC or mobile (tablet or phone) app.
This 2wT tab documents all 2wT interactions, including patient daily responses and nurse follow-up, providing data for routine and 2wT-specific reporting.
2wT transition from RCT to routine care.
ZAZIC conducted a highly participatory design meeting with Medic and user stakeholders (MoHCC, other MC implementing partners) to guide HCW standard operating procedure (SOP) changes from RCT to routine settings (Table 1). SOPs from the meeting reflected human resource training needs and plans for overcoming electricity, cell service and network coverage challenges for scale up. For RCT, patient messaging, tracing, and in-person clinical reviews were completed by the 2wT nurse at the study sites using a centralized model (Fig 4). For scale-up, one 2wT nurse (Hub nurse) was stationed at MoHCC to provide system-wide oversight and quality assurance as part of national MC program while other 2wT-based activities occurred at site (facility) level. Additional RCT implementation details were previously published [12,18]. A 3-tier system was developed to take 2wT from RCT to scale (Fig 5).
Fig 4 flow displays hybrid 2wT, with both automated and interactive (manual) SMS, interaction between patients and providers in a centralized model where a single site’s clinical provider(s) manages 2wT patients enrolled at that specific site.
Fig 5 flow displays hybrid 2wT, with both automated and interactive (manual) SMS, interaction between patients and providers in a hub-and-spoke model where a Hub clinical provider(s) manages 2wT patients from multiple 2wT enrolment sites.
2wT expansion was initially delayed six months due to COVID-19. Thereafter, a virtual sensitization meeting was held by Zim-TTECH in October 2020, with MoHCC national and District Health Executive structures. With stakeholder buy-in, a phased scale up was planned to include training of site teams (nurses, data clerks, administration) from 3 sites to 32 over 12 months. 2wT was piloted in routine services starting in January 2021, rolling out in February 2021.
Tiered approach to 2wT at scale.
Tier 1: Patient level. For patients, any phone that receives and sends SMS messages is sufficient for 2wT. Patients receive 2wT sensitization at MC registration and request to opt-in. Site nurses enrol eligible patients into 2wT and confirm receipt of enrolment message on patient phone before discharge. 2wT patients receive short instruction on responding to 13 days of daily messaging. Normal SMS charges apply. Patients may send free SMS requesting call back. Patient is traced by phone or home visit if no SMS response by day 8.
Tier 2: Site level spokes (facility or outreach setting). Routine MC nurses at sites act as 2wT “spokes.” Spoke nurses are site-level MoHCC nurses who enrol 2wT patients and interact with patients enrolled at their site via 2wT. They receive a mobile phone, data bundles, and half-day training that includes: use of 2wT Android app, enroling patients, interacting with patients via messages, completing app reports, and syncing patient data. Spoke nurses receive 2wT system-driven “Tasks” from the central hub nurse to trace potential loss to follow-up (LTFU) patients and conduct reviews for those with potential AEs. Spokes receive a “nudge” (reminder) if they do not record subsequent patient follow-up. Site nurses complete all reporting in 2wT system and on routine MoHCC paper forms. 2wT app works offline for enrolment and documentation. 2wT must be synced daily using an Internet connection to ensure patients receive daily messages and to receive/send communication with the Hub.
Tier 3: Central level Hub. 2wT has a central Hub nurse to triage patients across different 2wT sites. Central Hub location employs an Android phone with a sim card to send and receive SMS (gateway) that is linked to a desktop PC with a stable internet connection to allow communication between 2wT system, Hub, and spokes. Hub nurse monitors up to 500 patients a day across all sites using the 2wT web interface, reviewing daily patient reports and providing patient reassurance or education via 2wT. To support Hub efficiency, a specific red icon (“!”) draws attention to all patients that report a potential AE. Hub triages those in need of referrals to care and pushes out a 2wT-based system “Task” to site spokes for follow-up. Hub also creates tracing tasks for patients without a response by day 8. Site nurses are required to close tasks by completing an outcome report on subsequent follow-up in-person reviews or tracing efforts. Hub nurse provides scheduled on-site supervision and routine mentoring of site nurses via a WhatsApp 2wT Support Group. Routine ZAZIC oversight support and system monitoring ensures quality care and builds team confidence.
Study participants.
Comparison between 2wT from RCT and routine scale-up 2wT draw upon previously published RCT data procedures and outcomes [12,18]. For the scale-up period, participants included all adult males over age 18 that received an MC in the ZAZIC routine VMMC program, either 2wT-based or in-person post-operative follow-up, between January 2021, and October 2021.
Routine study sites and recruitment.
2wT men were enrolled as part of routine MC service delivery across 32 VMMC sites. Specific demand creation efforts supported 2wT recruitment, including posters and branded 2wT face masks. Enrolments were done during pre- and/or post-operative 2wT education, depending on the preferences and patient flow of each MC setting. MC patients received no specific 2wT consent as part of this routine service delivery follow-up approach. 2wT participants agreed to respond to SMS follow-up for 13 days on their phones. No compensation was offered to offset patient texting costs.
Data collection.
Aggregate data on MC enrolments of adult male patients (ages ≥18), AEs, and in-person visits were gathered from routine MoHCC registers as part of routine reporting across 37 MC sites. Additional information on 2wT patient enrolment, SMS interactions, in-person follow-up, and referrals were ascertained from Medic’s 2wT database in 32 sites as part of this quality improvement activity.
Data analysis.
The primary outcomes of interest were cumulative notifiable AE rates calculated as: (# moderate + severe AEs)/ (total # MCs) per group (RCT, 2wT routine, and routine in-person follow-up). Secondary outcomes included: # with ≥1 daily SMS response; spontaneous SMS (patients who sent a free text message, not in response to the daily prompt); patients who reported a potential AE; and LTFU (no contact within 14 days). No SMS contact by day 8 is reported only for 2wT in routine settings. Cumulative rate of any moderate or severe AE, and in-person visits were compared using STATA 15 immediate commands for aggregate data (StataCorp, College Station, TX), reporting 2-sided p-values from Fisher’s exact test with Wilson 95% confidence intervals (CI).
Ethics.
The ethics committee of the University of Washington (UW) internal review board (IRB) instructs UW researchers to complete the Human Subjects Research Worksheet to self-determine if the proposed project meets the definition of research and therefore requires IRB review. This study using secondary aggregate data did not meet the definition of research, and therefore did not require UW IRB review.
Results
Characteristics of 2wT patient behaviours in the routine setting
2wT enrolment and expansion increased as COVID-19 restrictions eased (Fig 6). In the period from January 2021 to October 2021, 17417 men received in-person MC follow-up across all 37 sites while 5084 patients were followed-up by SMS (Table 2) from 32 sites. Of 5084, there were 5078 unique phone numbers, accounting for three sets of enrolled brothers who shared phones. Among the 5084 2wT participants, 3609 (71.0%) replied to at least 1 daily SMS and 1,475 (29%) were followed up (traced) for no SMS contact by day 8. Of those traced on day 8, patient phone or home tracing found 1389 (94.2%) who did not need additional follow-up, 20 (1%) needed care, and 66 (4.5%) were not reached. Of all 5084, 630 (12.4%) patients reported at least one potential AE, and 325/5084 (6.4%) received individualized 2wT-based telehealth from the Hub (reassurance, reminders on wound care, and hygiene advice). Of the 325, 261 (80.3%) were subsequently cleared of having an AE via SMS or phone interaction while 64 (19.7%) were referred to site-level spokes for in-person review. Thirty-two referred patients (50%) attended a visit. An additional 34 2wT patients attended a follow-up visit spontaneously without requesting help via SMS first. Of all 2wT patients, only 4 (0.08%) notifiable AEs (1 severe and 3 moderate) were identified and reported. Within 14 days, 99.2% men had at least one follow-up contact with a nurse, 93.2% via SMS, 4.8% via voice call and 1.2% via an in-person visit.
2wT-based outcomes: RCT and routine settings
As compared to the RCT setting (Table 2, columns a and b), in the routine context, 2wT ascertained fewer AEs, with AEs rates decreasing from 1.9% (95% CI: 0.7, 3.6) to 0.08% (95% CI: 0.03, 2.0), respectively (p<0.001). HCW workload also reduced from the RCT to routine setting: only 1.1% (95% CI: 1.0, 1.6) of routine 2wT men had a documented in-person visit versus 22.7% (95% CI: 18.6, 27.2) during the RCT (p<0.001). In the RCT, 92.5% (95% CI: 89.0, 94.6) responded at least once over 13 days, but the 2wT response rate fell to 71.0% (95% CI: 69.7, 72.2) at scale (p<0.001). Prevention of LTFU improved during 2wT at scale: only 0.8% (95% CI: 0.6, 1.1) of 2wT men had no contact by 14 days as compared to RCT at 6.9% (95% CI: 4.7, 10.0) (p<0.001).
Scale-up outcomes: Comparison of 2wT to routine in-person follow-up
During the January 2021-October 2021 period (Table 2, columns b and d), AE rates were similar across 2wT [0.08% (95% CI: 0.03, 2.0)] and routine follow-up [0.03% (95% CI: 0.02, 0.08)] (p = 0.0248). The four 2wT patients with AEs (three moderate and one severe) all exchanged texts with the Hub nurse and were referred to care before AE identification (Table 3). There were more severe AEs among routine care men (4 severe and 2 moderate).
HCW workload was dramatically lower with 2wT: only 1.1% (95% CI: 1.0, 1.6) of 2wT versus 67.0% (95% CI: 66.3, 67.7) of routine follow-up men had an in-person review. Among 2wT patients, 31.2% (95% CI: 29.9, 32.5) had no contact by day 8, but only 0.8% (95% CI: 0.6, 1.1) of 2wT men were ultimately reported as LTFU as compared to 33.0% of men with expected in-person follow-up (95% CI: 32.3, 33.7; p<0.001).
Discussion
Despite the challenges of MC service delivery during the COVID-19 pandemic, 2wT scaled successfully from the RCT to routine settings. In both the RCT and the routine setting, 2wT-based follow-up assured patient safety while dramatically reducing provider workload. Patient follow-up rates and AEs were similar in either follow-up method, suggesting equivalence in assuring healthy post-operative healing. Although 2wT enrolment was slower than expected, the advantages of 2wT for MC patients and providers is clear. In response to both COVID-19-created opportunities to advance digital health innovation [20–22] and recent shortfalls in global MC targets [23], MC policy and programs should support 2wT expansion. Several key aspects of this intervention’s success and potential challenges for scale-up merit discussion.
2wT successes
First, unlike simpler bi-directional texting [24–26], this 2wT system adds several value-added features. 2wT includes a daily care prompt to encourage men to observe their healing and engage directly with an MC nurse to ask questions or request a call back. The centralized Hub nurse provides personalized telehealth for each patient, triaging those that need follow-up by referring to MC spoke nurses in local healthcare centres. For men who do not respond to daily texts by day 8, a 2wT system-generated alert ensures that the Hub nurse follows-up potential LTFU with calls to next of kin or referrals to site-based tracing. 2wT generated alerts at site level close referral loops for in-person reviews or tracing, confirming that men receive in-person care when needed. Patient phone numbers are confirmed via visual receipt of the enrolment text on the patient phone. This authentication process also provides confirmation of MC patients, adding a layer of verification of MC productivity.
Second, in routine settings, 2wT maintains the quality of care while dramatically reducing in-person visits. Although underreporting of AEs is possible [8], similar AE rates between 2wT and routine follow-up groups suggests safety for either approach. All reported AEs from 2wT men followed an initial “potential AE” SMS or interaction with the 2wT nurse, suggesting that men were both comfortable and confident to seek care via 2wT if they were experiencing a challenge or potential complication during their healing process. Among 2wT patients who did not respond to a daily text by day 8, phone and in-person follow-up did not result in identification of additional AEs, suggesting that lack of SMS response was largely due to lack of patient concerns. Although in-person follow-up rates are often reported at nearly 100%, actual follow-up may be far lower [10]. In this period, approximately 70% of both 2wT and in-person patients had follow-up contact. However, with similar assurances of patient healing, the workload reduction of benefits of 2wT is a distinct advantage at scale.
Third, 2wT may provide an early warning system for issues in quality service delivery. 2wT patient and provider interaction identified a challenge among circumcised patients that was undetected or unreported among routine MC men: delayed dissolution of suture materials. A total of 23 men (5.3%) either texted or requested a call between post-operative days 9 to 40 with concerns about remaining intact sutures. Although review resulted in only mild, non-notifiable AEs, these cases triggered an investigation into the suture material and commodity review. The investigation identified suture materials in consumable packs nearing expiration or already expired, potentially signalling their reduced quality. MC teams were alerted, and the kits were disposed of according to MoHCC protocols for expired medicines and equipment.
2wT challenges
Although enrolment target was set for 60% of eligible men, only 28.7% of 2wT-eligible men were enrolled during this initial scale-up period. Several program-level factors likely contributed. First, urban centres experienced high COVID-19 cases, slowing MC services and shifting service delivery to outreach facilities with weaker network coverage. Second, MCs were also performed among groups without phones, such as prisoners. Lastly, most MCs were performed among boys ages 15–17 during this period, before 2wT was approved for guardians of minors in 2022. Moreover, at facility level, adoption of 2wT is highly dependent on providers’ willingness to enrol patients. 2wT patients still require completion of MoHCC paper forms, creating redundancy, added work, and potential data duplication. Providers may also be reticent to have additional oversight by Hub nurse and ZAZIC QA teams.
2wT is not unique in identifying digital health scale-up barriers [27,28]. To support wider scale-up and sustainability in Zimbabwe, several improvements are needed. First, 2wT practice-based training, on-site supervision, and consistent mentoring are needed. This will help ensure 2wT SOP adherence and consistency while expanding to more districts and increasing 2wT access. Second, enhancing 2wT system capacity and patient SMS costs could be assumed by ZAZIC or MoHCC, reducing patient participation costs. Third, to increase patient demand, ZAZIC is launching robust messaging of 2wT through pamphlets, posters, and training of demand creation cadres in recruiting and messaging. Fourth, although 2wT has proven safe for patients, international MC funders still require in-person visits for global reporting, creating fear that wide deployment of 2wT could create tension between donors and MoHCC policy. Supportive 2wT policy must be in place to support scale-up, including allowing enrolment for minors and guardians to aid uptake. Lastly, to avoid duplication and support workload reduction, reporting redundancies across MoHCC paper forms and 2wT must be swiftly addressed.
Limitations
Although the short 14-day follow-up period reduces concerns of phone loss or number changes, sixty-nine men responded to daily texts from numbers that were not entered into the 2wT system. These responses could not be linked to a specific patient, muddling the precision of response rates. AEs may have been misidentified or undocumented, potentially underreporting notifiable AEs in any group. 2wT should be considered as a critical tool to increase the quality of MC services and reduce workload but will not solve typical barriers to follow-up such as mobile populations, inaccessible roads, and overstretched VMMC teams.
Conclusions
There is strong evidence that 2wT scale-up in routine settings appears advantageous for both patients and nurses, reducing workload and maintaining patient safety. In the context of COVID-19, 2wT proved even more advantageous: 2wT reduced contact between service providers and patients in accordance with WHO guidelines on COVID-19 prevention. Despite its slow expansion, 2wT scaled more swiftly in 2022. By April 2022, 2wT enroled over 10,000 ZAZIC patients, up to 80% of eligible clients per week, a sign of growing 2wT momentum.
Globally, proven systems like 2wT still face substantial organizational, institutional, and global obstacles to potential impact, mirroring experiences from, and perspectives on, digital health interventions in low- to middle-income countries (LMIC) [29,30]. First, lack of coherence and cooperation between Ministries, donors, and implementing partners remains a major challenge for digital health to deliver maximum benefits to vulnerable communities. Ministries need support to foster digital health sustainability, calling for significant investments from capacity strengthening through supportive infrastructure. Second, policy and governance must keep pace. Digital health innovations are occurring swiftly, but efforts lag to standardize Ministry policy and align donor support for sustained progress and impact [31]. Third, emphasis on national-level systems or interoperability concerns should not prevent scale-up of evidence-based innovations. While national adoption of Health Information Systems (HIS), or electronic medical records (EMR) offer considerable promise, few have successfully scaled [27,32]. Immediate Provincial or District adoption of 2wT could serve millions of MC patients in comparison to years needed to develop, deploy, and optimize a national EMR system. Where possible, digital health innovations like 2wT could swiftly operationalize data exchange with HIS or EMR (a lower technical bar based on offline data transfers and syncing) without needing complex, and quickly out of date cross coding, or interoperable systems. WHO, donors, and Ministries should revise MC follow-up policies and support rapid expansion of 2wT across MC programs. Consideration of 2wT-based telehealth for other contexts could safely expand efficiency gains from MC to other outpatient or acute-care contexts.
Supporting information
S1 Data. ZAZIC aggregate data by month, January 2021-October 2021.
ZAZIC Total MCs, Adult MCs, and Adult 2wT MC Enrolments, January-October 2021
https://doi.org/10.1371/journal.pdig.0000066.s001
(XLSX)
References
- 1. Stegman PM, Yee R, Davis J, Tchuenche M, Linder R, Zembe L, et al. Estimating male circumcision coverage in 15 priority countries in sub-Saharan Africa. Journal of the International AIDS Society. 2021;24:e25789. pmid:34546643
- 2.
UNAIDS. Voluntary medical male circumcision Geneva: UNAIDS; 2019 [cited 2020 April 10]. Available from: https://hivpreventioncoalition.unaids.org/wp-content/uploads/2020/08/2019_vmmc-15-esa-countries_en.pdf.
- 3. Kigozi G, Gray RH, Wawer MJ, Serwadda D, Makumbi F, Watya S, et al. The safety of adult male circumcision in HIV-infected and uninfected men in Rakai, Uganda. PLoS medicine. 2008;5(6):e116. pmid:18532873
- 4. Herman-Roloff A, Bailey RC, Agot K. Factors associated with the safety of voluntary medical male circumcision in Nyanza province, Kenya. Bulletin of the World Health Organization. 2012;90(10):773–81. pmid:23109745
- 5. Reed J, Grund J, Liu Y, Mwandi Z, Howard AA, McNairy ML, et al. Evaluation of loss-to-follow-up and post-operative adverse events in a voluntary medical male circumcision program in Nyanza Province, Kenya. Journal of acquired immune deficiency syndromes (1999). 2015.
- 6. Bochner AF, Feldacker C, Makunike B, Holec M, Murenje V, Stepaniak A, et al. Adverse event profile of a mature voluntary medical male circumcision programme performing PrePex and surgical procedures in Zimbabwe. Journal of the International AIDS Society. 2017(20:21394). pmid:28362066
- 7. Hellar A, Plotkin M, Lija G, Mwanamsangu A, Mkungume S, Christensen A, et al. Adverse events in a large-scale VMMC programme in Tanzania: findings from a case series analysis. Journal of the International AIDS Society. 2019;22(7):e25369. pmid:31368235
- 8. Marongwe P, Gonouya P, Madoda T, Murenje V, Tshimanga M, Balachandra S, et al. Trust but verify: Is there a role for active surveillance in monitoring adverse events in Zimbabwe’s large-scale male circumcision program? PloS one. 2019;14(6):e0218137. pmid:31181096
- 9. President’s Emergency Plan for AIDS Relief. PEPFAR’s Best Practices for Voluntary Medical Male Circumcision Site Operations: A service guide for site operations. Managing, Monitoring, And Reporting VMMC Adverse Events [Internet]. 2017. Available from: https://www.malecircumcision.org/sites/default/files/document_library/2017.9.26_ch7_vmmc-site-ops.pdf.
- 10. Feldacker C, Murenje V, Makunike-Chikwinya B, Hove J, Munyaradzi T, Marongwe P, et al. Balancing competing priorities: Quantity versus quality within a routine, voluntary medical male circumcision program operating at scale in Zimbabwe. PloS one. 2020;15(10):e0240425. pmid:33048977
- 11. Davis SM, Hines JZ, Habel M, Grund JM, Ridzon R, Baack B, et al. Progress in voluntary medical male circumcision for HIV prevention supported by the US President’s Emergency Plan for AIDS Relief through 2017: longitudinal and recent cross-sectional programme data. BMJ open. 2018;8(8):e021835. pmid:30173159
- 12. Feldacker C, Holeman I, Murenje V, Xaba S, Korir M, Wambua B, et al. Usability and acceptability of a two-way texting intervention for post-operative follow-up for voluntary medical male circumcision in Zimbabwe. PloS one. 2020;15(6):e0233234. pmid:32544161
- 13. Babigumira JB, Barnhart S, Mendelsohn JM, Murenje V, Tshimanga M, Mauhy C, et al. Cost-effectiveness analysis of two-way texting for post-operative follow-up in Zimbabwe’s voluntary medical male circumcision program. PloS one. 2020;15(9):e0239915. pmid:32997710
- 14. Augustin C, Holeman I, Salomon E, Olsen H, Azar P, Ayyangar M. Pathways to Increasing Trust in Public Health Data: An Exploratory Analysis of Quality Issues and Potential Remediation for Data Collected Using the Community Health Toolkit. CHANCE. 2021;34(3):24–32.
- 15. Yang JE, Lassala D, Liu JX, Whidden C, Holeman I, Keita Y, et al. Effect of mobile application user interface improvements on minimum expected home visit coverage by community health workers in Mali: a randomised controlled trial. BMJ Global Health. 2021;6(11):e007205. pmid:34815242
- 16. Karlyn A, Odindo S, Onyango R, Mbindyo C, Mberi T, Too G, et al. Testing mHealth solutions at the last mile: insights from a study of technology-assisted community health referrals in rural Kenya. Mhealth. 2020;6.
- 17. Wasunna B, Zurovac D, Bruce J, Jones C, Webster J, Snow RW. Health worker performance in the management of paediatric fevers following in-service training and exposure to job aids in Kenya. Malaria Journal. 2010;9(1):1–7. pmid:20849650
- 18. Feldacker C, Murenje V, Holeman I, Xaba S, Makunike-Chikwinya B, Korir M, et al. Reducing Provider Workload While Preserving Patient Safety: A Randomized Control Trial Using 2-Way Texting for Postoperative Follow-up in Zimbabwe’s Voluntary Medical Male Circumcision Program. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2020;83(1):16–23. pmid:31809358
- 19. World Health Organization. Classification of digital health interventions v1. 0: a shared language to describe the uses of digital technology for health2018 October 1, 2019 [cited 2019 October 1, 2019]. Available from: https://apps.who.int/iris/bitstream/handle/10665/260480/WHO-RHR-18.06-eng.pdf.
- 20. Mbunge E, Batani J, Gaobotse G, Muchemwa B. Virtual healthcare services and digital health technologies deployed during coronavirus disease 2019 (COVID-19) pandemic in South Africa: a systematic review. Global Health Journal. 2022. pmid:35282399
- 21. Mitgang EA, Blaya JA, Chopra M. Digital Health in Response to COVID-19 in Low-and Middle-income Countries: Opportunities and Challenges. Global Policy. 2021;12:107–9. pmid:34230840
- 22. Mahmood S, Hasan K, Carras MC, Labrique A. Global preparedness against COVID-19: we must leverage the power of digital health. JMIR Public Health and Surveillance. 2020;6(2):e18980. pmid:32297868
- 23. UNAIDS. Update: Male circumcisions disrupted by COVID-19, fall short of the 2020 target 2021. Available from: https://www.unaids.org/en/resources/presscentre/featurestories/2021/november/20211129_male-circumcisions.
- 24. Gross R, Ritz J, Hughes MD, Salata R, Mugyenyi P, Hogg E, et al. Two-way mobile phone intervention compared with standard-of-care adherence support after second-line antiretroviral therapy failure: a multinational, randomised controlled trial. The Lancet Digital Health. 2019;1(1):e26–e34. pmid:31528850
- 25. Odeny TA, Bukusi EA, Cohen CR, Yuhas K, Camlin CS, McClelland RS. Texting improves testing: a randomized trial of two-way SMS to increase postpartum prevention of mother-to-child transmission retention and infant HIV testing. AIDS (London, England). 2014;28(15):2307. pmid:25313586
- 26. Wald DS, Butt S, Bestwick JP. One-way versus two-way text messaging on improving medication adherence: meta-analysis of randomized trials. The American journal of medicine. 2015;128(10):1139. e1–.e5. pmid:26087045
- 27. Huang F, Blaschke S, Lucas H. Beyond pilotitis: taking digital health interventions to the national level in China and Uganda. Globalization and health. 2017;13(1):1–11.
- 28. Fruchtman CS, Mbuyita S, Mwanyika-Sando M, Braun M, de Savigny D, Muñoz DC. The complexity of scaling up an mHealth intervention: the case of SMS for Life in Tanzania from a health systems integration perspective. BMC health services research. 2021;21(1):1–11.
- 29. Labrique AB, Wadhwani C, Williams KA, Lamptey P, Hesp C, Luk R, et al. Best practices in scaling digital health in low and middle income countries. Globalization and health. 2018;14(1):1–8.
- 30. McCool J, Dobson R, Muinga N, Paton C, Pagliari C, Agawal S, et al. Factors influencing the sustainability of digital health interventions in low-resource settings: Lessons from five countries. Journal of global health. 2020;10(2). pmid:33274059
- 31. Desveaux L, Soobiah C, Bhatia RS, Shaw J. Identifying and overcoming policy-level barriers to the implementation of digital health innovation: qualitative study. Journal of medical Internet research. 2019;21(12):e14994. pmid:31859679
- 32. Gimbel S, Kawakyu N, Dau H, Unger JA. A Missing Link: HIV-/AIDS-Related mHealth Interventions for Health Workers in Low-and Middle-Income Countries. Current HIV/AIDS Reports. 2018;15(6):414–22. pmid:30259258