Understanding mSOS: A qualitative study examining the implementation of a text-messaging outbreak alert system in rural Kenya

Outbreaks of epidemic diseases pose serious public health risks. To overcome the hurdles of sub-optimal disease surveillance reporting from the health facilities to relevant authorities, the Ministry of Health in Kenya piloted mSOS (mobile SMS-based disease outbreak alert system) in 2013–2014. In this paper, we report the results of the qualitative study, which examined factors that influence the performances of mSOS implementation. In-depth interviews were conducted with 11 disease surveillance coordinators and 32 in-charges of rural health facilities that took part in the mSOS intervention. Drawing from the framework analysis, dominant themes that emerged from the interviews are presented. All participants voiced their excitement in using mSOS. The results showed that the technology was well accepted, easy to use, and both health workers and managers unanimously recommended the scale-up of the system despite challenges encountered in the implementation processes. The most challenging components were the context in which mSOS was implemented, including the lack of strong existing structure for continuous support supervision, feedback and response action related to disease surveillance. The study revealed broader health systems issues that should be addressed prior to and during the intervention scale-up.

O utbreaks of epidemic diseases pose serious public health risks (1). Kenya, like other Africa countries, lacks the means to deliver adequate healthcare services. This weakness compromises the success of the World Health Organization's Integrated Disease Surveillance and Response (IDSR) and International Health Regulations (IHR) strategies and often results in incomplete, delayed, and poor-quality (i.e., not following standard case definitions in the IDSR guidelines) paper-based reporting from health facilities in remote areas. Furthermore, inadequate reporting limits health managers' ability to take appropriate and timely action in response to health events (2,3).
Widespread expansion of mobile phone coverage in Africa (4) offers opportunities to overcome weaknesses in health systems and to improve medical and public health practice through mobile health (mHealth) (5). Despite many mHealth projects undertaken in Africa, their effectiveness has rarely been rigorously evaluated, limiting evidence-based policy adoptions or project expansion in scope or geography (6)(7)(8)(9). In particular, evidence of effectiveness of mHealth interventions for enhancing disease surveillance is scarce (10). We undertook a clustered, randomized, controlled trial with 135 health facilities in Busia and Kajiado Counties in Kenya during November 2013-April 2014 to test the effectiveness of a mobile short-message-service (SMS)-based disease outbreak alert system (mSOS) for reporting immediately notifiable diseases.
The Study mSOS is a formatted text-messaging system that enables communications between healthcare facility workers and Ministry of Health managers and uses a Web-based portal to monitor disease notifications and response actions taken by health managers (Figure 1; online Technical Appendix, http://wwwnc.cdc.gov/EID/article/22/4/15-1459-Techapp1.pdf). In our trial, health workers used mSOS for 6 months to send information about suspected cases or health events that required notification within 24 hours. Twelve diseases and conditions were selected for the study (online Technical Appendix Table 1). Before mSOS was implemented, we conducted a 1-day refresher training course on IDSR for in-charges (i.e., medical officers in charge) of 135 participating health facilities; the training focused on case definitions of notifiable diseases and on paper-based reporting. During the training, facilities were randomized into intervention and control groups; the intervention group received an additional day of training on mSOS. Paper-based reporting continued throughout the study period for both groups, so the intervention group would report cases 2 ways.
Our primary outcome was determining how many of the cases that required immediate notification were reported within the time specified. Our secondary outcome was determining, from among the cases for which notifications were sent, the proportion for which response actions were taken. For evaluation purposes, data from health facilities were collected for 6-month periods before and after the intervention launch (i.e., IDSR and mSOS training and use of mSOS for 6 months). Cases detected, notifications submitted, and responses undertaken were extracted from facility records in both study groups. Notifications sent by SMS were retrieved from the mSOS system. Our primary analysis was intention-to-treat (i.e., analysis of cases from all health facilities as they were randomized, regardless of intervention exposure). Our secondary analysis was perprotocol (i.e., our trial protocol) and was restricted to cases reported by facilities whose in-charges had received training (i.e., IDSR training for control group; IDSR and mSOS training for intervention group; Figure 2).
Characteristics of health facilities and in-charges were similar; data from preintervention and postintervention

Effectiveness of a Mobile Short-Message-Service-Based Disease Outbreak Alert System in Kenya
surveys showed no significant differences between control and intervention groups (Table 1). Follow-up surveys conducted 6 months after the intervention showed that 34 (51.6%) of 66 intervention group in-charges received mSOS and IDSR training and 32 (49.2%) of 65 control group in-charges received IDSR training (Figure 2; online Technical Appendix).
A retrospective review of the baseline (preintervention) surveys showed that 36 cases (19 for intervention group, 17 for control group), all measles, required immediate notification. Of these 36 cases, only 1 immediately notifiable case was reported (from a control facility using paper forms). During the 6-month period after the intervention, 169 immediately notifiable cases (130 for the intervention group, 39 for the control group) were detected: 160 measles, 6 anthrax, 2 Q fever, and 1 guinea worm. Of the 39 cases detected in the control group, notification of only 1 case (2.6%), which was measles, was sent. Of the 130 immediately notifiable cases detected in the intervention group, 25 (19.2%) were reported to disease surveillance coordinators at the subcounty, county, and national levels. This proportion of cases reported was significantly higher than that reported by the control group (% difference 16.7, 95% CI 2.71-25.07; Table 2).
All 25 cases for which notifications were sent from the intervention group were measles cases reported through mSOS; 2 cases were also reported with paper forms. For these 25 mSOS notifications, the threshold for a measles outbreak response (5 suspected cases) was met once, and disease surveillance coordinators at the subcounty level responded to this event. Furthermore, 24 (96%) of the 25 suspected measles cases were reported within 24 hours.
In the per-protocol analysis, the percentage of cases for which notification was sent was greater in the intervention group than in the control group (27.3% vs. 4.8%), but the difference was of borderline statistical significance (% difference 22.5, 95% CI -0.32 to 34.13 by Wilson procedure with continuity correction [11]). Similar differences were found when the analysis was restricted to health

Conclusions
This study showed that SMS intervention significantly increased timely notifications; however, despite a relatively large improvement, response remained suboptimal, with timely notifications of only one fifth of detected cases. These findings mirror results of a study in Tanzania, which showed that SMS considerably increased vital registration coverage but fell far short of reporting actual birth and death events in the community (12). Our study has implications for health managers who implement interventions to improve disease surveillance in resource-limited settings. First, the number of detected cases requiring immediate notification increased postintervention. This effect was observed in both intervention and control groups but was higher in the group using SMS; this group had a 7-fold increase in detected cases compared with baseline findings. IDSR refresher training may have contributed to increased case detection, and the combined interventions, including the technology component, resulted in a greater detection effect. Second, expecting health workers to complete paper-based forms and deliver them without incentive within 24 hours is ineffective for ensuring notification of cases, with or without exposure to the refresher training. Third, we observed a large drop-out rate (47.4%) for health facility in-charges participating in the study. The study took place during a period of health management decentralization in Kenya, resulting in 47 new counties and in health worker transfers. Lack of on-the-job training for staff who did not attend the training and lack of support through posttraining follow-up and supportive supervision were weaknesses in the intervention. These systemic challenges, reported in other IDSR (13) and mHealth surveillance (14) projects, must be addressed to avoid compromising the sustainability of such interventions. Finally, attrition of health workers exposed to the intervention and lack of paper-based tools explain only part of our results. The short duration of the training deployed (15) and the possibly suboptimal quality of the training delivered (3) may have contributed to the unrealized full potential of the intervention.
Despite its limitations (online Technical Appendix), this study shows how technology in the form of mSOS can increase the rate of notifications of suspected disease outbreaks and enhance IHR compliance in resource-limited settings. Further investigation into ways to optimize the quality of delivery of mSOS interventions in countries with weak healthcare systems is justified.  County because values will be inverse of data for Kajiado County (i.e., N minus n). N = total facilities in both counties. The intervention group is the group of facility in-charges who were exposed to IDSR and mSOS training and to the mSOS intervention; the control group is the group of in-charges who were exposed to IDSR training only. FBO, faith-based organization; IDSR, Integrated Disease Surveillance and Response; in-charge, medical officer in charge of facility; IQR, interquartile range; NGO, nongovernment organization. †χ 2 test was used to compare the proportions between control and intervention groups. Wilcoxon Mann Whitney test was used to compare medians between control and intervention groups (i.e., age of in-charges). Analyses were conducted by using an α level of 0.05. The p value is shown for the postintervention period only. ‡Standardized IDSR paper-based reporting form for immediately notifiable diseases. §Data are median and Interquartile range rather than numbers and percentages. Denominator excludes 3 facilities with missing values in the preintervention control group and 1 facility with missing values for each of the remaining 3 study groups. *Intention-to-treat analysis indicates analysis of treatment groups as they were randomized, regardless of the intervention exposure; per-protocol analysis indicates restricted analysis of groups that completed the entire study according to the trial protocol.