Data analysis and narrative synthesis

As this review aimed to identify which interventions were most effective at increasing vaccination uptake in the specific context, the analysis aimed to evaluate the relative effectiveness of each included mHealth/DH intervention on vaccination coverage and timeliness. AB and CH conducted an assessment of the extracted data and made a decision on synthesis approach. Due to extremely high heterogeneity reported (Fig 3 and 4), a narrative synthesis guided by synthesis without meta-analysis (SWiM) reporting guidelines was conducted [51]. This approach was further supported by very high heterogeneity reported in similar meta-analysis studies [33,36]. For the narrative synthesis, studies were grouped together by mHealth/DH intervention-type. These intervention-type groups were: ‘SMS-Only’, ‘SMS-Plus’, ‘SMS and/or Voice Messages or Phone calls’, ‘Phone calls only’, and ‘Electronic Immunisation Alert Wristband’. A description of each group and the studies included in each is shown in S4 File. The synthesis method entailed a description of positive or negative associations, a consideration of point estimate precision, and each study’s risk of bias. Additionally, for narrative synthesis best practice suggests that syntheses are accompanied by discrete certainty of evidence assessment [49–51]. As mentioned, the certainty of the evidence for each intervention-outcome pairing using the GRADE framework was conducted. This considered five domains: risk of bias, inconsistency, indirectness, imprecision, and publication bias. AB and CMU independently conducted the assessments, resolving any discrepancies through discussion. Final certainty ratings were classified as high, moderate, low, or very low. Due to a lack of variation in results due to study design, it was deemed appropriate to analyse findings from both RCT and non-RCT study designs together.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flow chart outlining the study selection process for this systematic review.

Adapted from Page et al (2021) [39].

https://doi.org/10.1371/journal.pone.0324117.g001

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. The number of interventions investigated in this review by intervention-type subgroup and pictograms of each intervention.

Note that individual interventions in multiple-arm RCT and non-RCTs have been investigated separately. (Created on Microsoft Excel and PowerPoint, pictures taken from canva.com and Sampson et al (2023) [64]).

https://doi.org/10.1371/journal.pone.0324117.g002

Forest plots were created using Jamovi statistical software to provide visual representation of the review’s quantitative findings and to support the narrative synthesis. These only presented data on the third DTP/Pentavalent dose (DTP/Pentavalent3). Using the final dose provided a robust indicator of mHealth/DH intervention effectiveness on vaccine coverage and timeliness, and indicated whether the vaccination schedule was completed, thus allowing assessment of intervention effectiveness on schedule adherence. Each included intervention was analysed individually, therefore multi-arm studies provided multiple point estimates, allowing a visual comparison of all included interventions. It should be noted that each intervention from the multi-arm RCTs or non-RCTs was compared against the same control, however, this does not bias the results as pooled estimates were not calculated.

The standardised synthesis metric was logarithm of the odds ratio (logOR), which provided information on associations between mHealth/DH intervention and outcome (vaccination coverage or timeliness). All values were transformed to logOR, as this metric provided a more robust visualisation of strength and certainty of reported associations [52,53]. LogORs were calculated from the data extracted from included studies, the raw data used to calculate point estimates is shown in S6 File. 95% confidence intervals were provided for each point estimate, along with heterogeneity statistics (including I²).

Ethics approval

No primary data were collected for this study. Ethical approval was therefore sought and granted by the University of Edinburgh’s Usher Masters’ Research Ethics Group (UMREG).

Study characteristics

Table 1 summarises the characteristics of the 14 included studies [54–67]. Ten of the included studies were RCTs [54–60,62,65,67], with the remaining four studies comprising non-RCTs [61,63,64,66]. The majority of studies were conducted in West Africa (12 studies) [54–58,61–67], and only 4 of the 19 SSA countries targeted for malaria vaccine rollout (Nigeria [54,55,57,58,61–64,66,67], Burkina Faso [65], Cote D’Ivoire [56] and Kenya [59,60]) were represented in the study. All the interventions included related to appointment reminders to encourage and remind mothers/caregivers to bring their children to immunisation clinics to receive DTP/Pentavalent doses. As each intervention was analysed independently, the number of mHealth/DH interventions and the type of interventions investigated in this review is shown in Fig 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Study characteristics of the 14 included studies.

https://doi.org/10.1371/journal.pone.0324117.t001

Outcomes under investigation

The two key outcomes investigated in relation to vaccination uptake were vaccination coverage and timely receipt of doses. This review investigated mHealth/DH intervention effect on both these distinct but complementary outcomes to provide a more complete understanding of vaccination uptake. Including mHealth/DH intervention effect on coverage helped assess how well the target population was reached, while including intervention effect on timeliness provided insight into intervention effectiveness at ensuring vaccines were administered within the recommended time window, a critical factor for optimal immunity [41,42]. Seven studies included outcomes categorised as only relating to vaccination coverage [54–57,60,63,66]. Five studies measured both coverage and timeliness [58,59,61,65,67]. While only two studies only reported outcomes relating to timeliness [62,64].

Study findings

Table 2 shows the key findings from the included studies. The two key parameters (vaccination coverage and timeliness), align with Study Objectives 1 and 2. As mentioned, each intervention from the 14 studies is presented individually. In total, 17 interventions related to coverage, and 12 interventions related to timeliness were analysed.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. The key findings from the 14 included studies relating to the 3 study objectives.

https://doi.org/10.1371/journal.pone.0324117.t002

Risk of bias assessments

Fig 3 shows the results of the RoB2 and ROBINS-I risk of bias assessments. Three of the included RCTs were considered as ‘Low risk’ [56,57,60]. All three of these studies were categorised as reporting outcomes relating to vaccination coverage, suggesting the findings relating to this outcome may have greater internal validity. Four were scored as having ‘Some concerns’, reporting on a mix of coverage, timeliness or both outcomes [55,58,59,61]. Three were considered at ‘High risk’ [54,65,67]. Amongst the included non-RCTs, none were considered ‘Low risk’, two were scored as having ‘Moderate risk’ [61,66], and two were considered as having ‘Critical risk’ of bias [63,64]. This spread of risk of bias ratings highlights substantial variability in study quality, particularly among non-RCTs. The concentration of lower risk of bias studies reporting on vaccination coverage suggests more reliable conclusions can be made for this outcome. In contrast, caution is warranted when interpreting findings from studies assessed as having high or critical risk of bias (reported on both outcomes of interest). The presence of multiple studies with ‘some concerns’ or higher further emphasises the need for more rigorous, high-quality research to strengthen the evidence base.

Study Objective 1 Findings – mHealth/DH intervention effect on vaccination coverage

Fig 4 provides a visual representation of the mHealth/DH interventions effectiveness in improving vaccination coverage of DTP/Pentavalent3 by study design.

Fig 4A shows the outcomes from the included RCTs. Extremely high heterogeneity was reported (I² = 96.29%). Eleven interventions from the RCTs demonstrated a positive association between intervention and vaccination coverage. Only ‘Gibson 2017 (A)’, reported a negative outcome (−0.56 logOR, 95% CI −1.49 to 0.37) [59]. However, the confidence interval crosses the null (logOR = 0), indicating potential for a positive association. There are two outliers (which likely contribute to the elevated I² statistic) displaying notably high effect estimates: ‘Brown 2016 (A)’ (3.99 logOR, 95% CI 2.56 to 5.43) and ‘Brown 2016 (C)’ (3.28 logOR 95% CI 2.24 to 4.33), both are phone call reminder interventions [55]. The remaining point estimates show a more homogeneous range, from 0.22 logOR (95% CI −0.84 to 1.28) to 2.06 logOR (95% CI 1.66 to 2.47). Overall, these findings indicate a likely positive association between mHealth/DH interventions and vaccination coverage.

Fig 4B shows associations between mHealth/DH interventions and vaccination coverage amongst the included non-RCTs. All four interventions presented here show positive logOR point estimates, suggesting these interventions were likely associated with increased vaccination coverage. Oladepo et al (2020) reported 0.86 logOR (95% CI 0.65–1.06) but was not included in the forest plot as its large weighting (due to sample size) masks heterogeneity (I² = 0%), its overwhelming influence diminishes variability between studies [63]. With this excluded Fig 4B still displays moderate-high heterogeneity (I² = 54.71%), thus grounds for narrative synthesis still stands.

Study Objective 2 Findings – mHealth/DH intervention effect on vaccination timeliness

Fig 5 provides visual representation of the mHealth/DH intervention effect on timeliness of administration of DTP/Pentavalent3 by study design.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. The risk of bias assessments of the included studies.

(A) Risk of bias assessment of the RCTs using RoB2 [47], note that D7 ‘Other biases’ column refers to assessments of other important biases not assessed by RoB2, including assessing study population baseline imbalances, study design appropriateness, potential early stoppage of trials and potential conflicts of interest or funding biases (B) Risk of bias assessments of the non-RCTs using ROBINS-I [48].

https://doi.org/10.1371/journal.pone.0324117.g003

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Forest plots showing logOR for mHealth/DH intervention effectiveness for increasing vaccination coverage of DTP/Pentavalent3.

(A) logOR point estimates for the included RCTs (B) logOR point estimates for included non-RCTs*. % weighting of each study is shown which correlates to heterogeneity statistics along with 95% confidence intervals of the point estimates. As each intervention was analysed individually ‘Gibson 2017 (A)’ relates to Group A’s intervention from Gibson et al (2017) [59]. *Note Oladepo et al (2020) [63] has been excluded as an outlier as it had a weighting of 72.98% which resulted in masked heterogeneity (I² = 0%). This large influence in the plot diminishes variability between studies.

https://doi.org/10.1371/journal.pone.0324117.g004

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Forest plots showing logOR for mHealth/DH intervention effectiveness for improving timeliness of administration of DTP/Pentavalent3.

(A) logOR point estimates for the included RCTs (B) logOR point estimates for included non-RCTs. % weighting of each study is shown which correlates to heterogeneity statistics along with 95% confidence intervals of the point estimates.

https://doi.org/10.1371/journal.pone.0324117.g005

Fig 5A illustrates the interventions effect on timeliness from the included RCTs. Again, an extremely high I² is reported (96.52%). These interventions showed fairly homogenous results, ranging from 0.00 logOR (95% CI −0.32 to 0.33) to 0.58 logOR (95% CI 0.08 to 1.08), indicating a positive association between mHealth/DH intervention and DTP/Pentavalent3 timeliness. Yunusa et al (2024) was a sole outlier (SMS and Phone call reminder intervention), reporting enhanced point estimate of 2.60 logOR (95% CI 2.08 to 3.11) [67].

Fig 5B shows the interventions investigated amongst the non-RCTs effect on timeliness. A high I² is also reported (83.3%). Group B and C of Ibraheem et al (2021) and Sampson et al (2023), show fairly homogenous point estimates ranging from 0.08 logOR (95% CI −0.41 to 0.58) to 0.33 logOR (95% CI −0.18 to 0.83) [61,64]. The only outlier was Group A of Ibraheem et al (2021), showing 1.41 logOR (95% CI 0.79 to 2.03) [61].

Study Objective 3 Findings – Additional results relevant to mHealth/DH intervention implementation

Two studies provided useful additional information [56,66]. Dissieka et al (2019) reported that when given a choice between receiving SMS or voice message appointment reminders, 675 (84.6%) of mothers/caregivers chose to receive voice message reminders, whereas only 123 (15.4%) chose to receive SMS messages, and voice message reminders were preferred by 97.7% in rural areas [56]. Additionally, Yunusa et al (2022) recommended sending SMS messages in local languages, and to not assume English is understood, an important consideration for future intervention development [66].

GRADE certainty of evidence assessment

The GRADE assessment was conducted separately around each of the intervention-type subgroups and the two primary outcome groups (vaccination coverage and vaccination timeliness). Table 3 shows the GRADE summary of findings table. The full explanations for GRADE scoring are shown in S5 File. In the vaccination coverage outcome group, the findings from ‘SMS-only’ and ‘SMS reminders and/or Voice messages or Phone calls’ subgroups were scored as having ‘Moderate’ certainty. The ‘Phone call-only’ subgroup scored ‘Low’ certainty, whilst the ‘SMS-Plus’ subgroup’s findings were considered ‘Very low’ certainty. For the vaccination timeliness findings, the ‘SMS-only’ and ‘SMS-Plus’ intervention subgroups’ findings were considered as having ‘Moderate’ certainty, whereas the ‘SMS reminders and/or Voice messages or Phone calls’, ‘Phone call-only’ and ‘Electronic Immunisation Alert Wristband’ all scored ‘Very low’ certainty.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. GRADE Summary of findings table showing certainty of evidence for each outcome and intervention-type pairings.

https://doi.org/10.1371/journal.pone.0324117.t003

Narrative synthesis

For the narrative synthesis the studies have been grouped by mHealth/DH intervention-type. The description and which studies included in each subgroup is shown in S4 File.

Study Objective 1 Narrative Synthesis – mHealth/DH intervention effect on vaccination coverage

Four mHealth/DH intervention-type subgroups reported outcomes categorised as vaccination coverage (‘SMS-only’ appointment reminders, ‘SMS-Plus’, ‘SMS reminders and/or Voice messages or Phone calls’, and ‘Phone call-only’ reminders).

1. (i). ‘SMS-only’ appointment reminders (including automated and non-automated)

Five ‘SMS-only’ interventions showed a positive association with vaccination coverage [58,60,61,65,66]. Group A in Gibson et al (2017) reported a negative logOR (−0.56 (95% CI −1.49 to 0.37)) [59], however, as the confidence interval crosses the null, a positive association might be possible. Contrastingly, Group B of Ibraheem et al (2021) demonstrated a much greater point estimate of 3.01 logOR (95% CI 0.16 to 5.87) [61]. However, the broad confidence interval range suggests imprecision in the point estimate, which may limit the reliability of this finding. The remaining four studies all showed a positive association between ‘SMS-only’ reminders and vaccination coverage, ranging from 0.38 logOR (95% CI 0.11 to 0.66) to 1.71 logOR (95% CI 1.11 to 2.34) [58,60,65,66]. The intervention-outcome group pairing scored moderate certainty of evidence; therefore, we have moderate confidence in these findings and recognise that further research may lead to alternative conclusions.

Overall, the results suggest that implementing ‘SMS-only’ appointment reminders may have a modest positive effect on vaccination coverage in this context. For ‘SMS-only’ interventions to be effective, mothers/caregivers must receive the SMS reminder [68]. If a spouse or relative’s phone is used, then this opens the possibility of reminders being missed, however, it was unclear in included studies whether SMS reminders were received. As the SMS messages must be read and understood by recipients to be effective, the modest results exhibited could be explained because of lower literacy rates amongst recipients, particularly in rural or underserved communities where literacy rates might be lower [69,70]. These barriers (shared phone ownership and low literacy) may limit SMS-only intervention effectiveness, therefore must be considered when implementing SMS-based reminders to improve childhood vaccination uptake.

1. (ii). ‘SMS-Plus’ (SMS appointment reminders plus cash incentives or health education SMS messages)

The ‘SMS-Plus’ interventions show great variation in logOR, likely due to the diversity of intervention enhancements. Oladepo et al (2020) reported the most precise point estimate, 0.86 logOR (95% CI 0.65 to 1.06), however, due to potential limitations identified in risk of bias assessment, we acknowledge this finding’s reliability may be reduced [63]. Group B and C of Gibson et al (2017), which combined SMS reminders with cash incentives, reported 0.22 logOR (95% CI −0.84 to 1.28) and 0.46 logOR (95% CI −0.69 to 1.62), respectively [59]. Confidence intervals in both crossed the null, reflecting great uncertainty in point estimate precision. Group C of Ibraheem et al (2021), showed 2.25 logOR (95% CI 0.17 to 4.33), with a wide confidence interval range, again indicating extreme uncertainty in point estimate precision [61]. Overall, the certainty of evidence for this intervention-outcome pairing was very low, which limits the strength of conclusions regarding the effectiveness of ‘SMS-Plus’ interventions for increasing vaccination coverage.

The findings suggest that ‘SMS-Plus’ interventions may offer some benefit for increasing vaccination coverage, but current evidence is limited by imprecision and variability in intervention design. An interesting observation was that health education intervention reported by Oladepo et al (2020), showed greater effect in comparison to cash incentivisation interventions for vaccination coverage [61,63]. As ‘SMS-Plus’ interventions are also SMS-based, they also require SMS receipt and recipient understanding of SMS contents to be effective, this could explain the similar outcomes to ‘SMS-only’ interventions. As interventions were ‘enhanced’ through cash incentivisation or health educational messaging, stronger associations between interventions and outcomes were expected, however, were not found. Given the additional resource requirements, further research is needed to assess the contextual appropriateness and cost-effectiveness of these enhanced interventions in improving childhood immunisation uptake across SSA.

1. (iii). ‘SMS reminders and/or Voice messages or Phone calls’

All three studies in this subgroup showed positive associations between intervention and vaccination coverage with varying degrees of precision [56,57,67]. Yunusa et al (2024) reported the strongest association, logOR of 2.06 (95% CI 0.17 to 4.33), however, this finding’s validity may be influenced by the risk of bias assessment and should be interpreted accordingly [67]. Ekhaguere et al’s (2019) intervention (combining SMS and voice messages), reported a positive 0.32 logOR (95% CI −0.12 to 0.75), however, as the confidence interval range crosses the null there is uncertainty around the estimate’s precision [57]. Dissieka et al (2019) carried the most weight when considering risk of bias assessment, reporting 0.72 logOR (95% CI 0.51 to 0.93) [56]. The point estimate suggests that providing mothers/caregivers with a choice of SMS or voice messages is positively associated with increased vaccination coverage. For this intervention-outcome group, the certainty of evidence was again moderate, indicating sufficient confidence that these interventions will positively affect vaccination coverage.

These results suggest combining SMS reminders with voice-based communication (voice messages or phone calls), or allowing recipients to choose between the two, may be a more effective strategy for improving vaccination coverage. The positive associations observed may reflect the ability of voice-based reminders to overcome literacy-related barriers. Additionally, offering caregivers a choice could increase engagement by promoting a sense of autonomy and empowerment in relation to their child’s health [71]. Given the moderate certainty of evidence and promising findings, these interventions merit serious consideration in future immunisation programmes. However, attention should also be paid to feasibility and cost-effectiveness in different contexts given their more resource-intensive nature.

1. (iv). ‘Phone call-only’ reminders

All three interventions involving phone call reminders show great positive associations for vaccination coverage [54,55,61]. All point estimates were >1.00. Brown & Oluwatosin (2017) provided the most precise estimate in the subgroup, with logOR of 1.49 (95% CI, 1.13 to 1.85) [54]. Group A in Ibraheem et al (2021), reported 2.99 logOR (95% CI, 0.14 to 5.84), however the wide confidence interval range indicates substantial uncertainty in this point estimate’s precision, raising concerns around its reliability [61]. Group A of Brown et al (2016), reported highest logOR of 3.99 (95% CI, 2.56 to 5.43) [55]. Although the GRADE assessment rated this intervention-outcome group as having low certainty of evidence, the point estimates were consistently greater than those of other intervention-types, suggesting that phone call reminder interventions may offer the greatest potential for increasing vaccination coverage. While these findings support the hypothesis that interventions incorporating voice-based components may be more effective and appear well-suited to contexts where literacy or access to SMS technology is limited, the low certainty of evidence limits their immediate applicability for informing public health policy and practice. These findings reinforce the value of direct, human-to-human contact in promoting health behaviours. Further high-quality studies are needed to confirm effectiveness and explore implementation challenges, including workforce capacity and cost considerations.

Study Objective 2 Narrative Synthesis – mHealth/DH intervention effect on vaccination timeliness

Five mHealth/DH intervention-type subgroups reported outcomes categorised as vaccination timeliness (‘SMS-only’ appointment reminders, ‘SMS-Plus’, ‘SMS reminders and/or Voice messages or Phone calls’, ‘Phone call-only’ reminders, and ‘Electronic immunisation alert wristband’).

1. (i). ‘SMS-only’ appointment reminders (including automated and non-automated)

Three ‘SMS-only’ interventions showed positive associations between intervention and vaccination timeliness [58,61,62]. Group A of Gibson et al (2017), showed no effect, 0.00 logOR (95% CI −0.32 to 0.33), however, the wide confidence interval range, indicates uncertainty around true point estimate [59]. The remaining point estimates were fairly homogeneous showing slight positive associations, with Kawakatsu et al (2020) reporting 0.25 logOR (95% CI 0.16 to 0.34) [62], and Group B of Ibraheem et al (2021) reporting 0.33 logOR (95% CI −0.18 to 0.83) [61]. Although Ibraheem et al (2021) reported a positive association here, the wide confidence interval range crosses the null, indicates potential for no effect [61]. Eze and Adeleye (2015) also reported a slight positive association of 0.38 (95% CI 0.11 to 0.66) [58]. A moderate certainty of evidence was found for this intervention-outcome pairing. Similar to the vaccination coverage findings, ‘SMS-only’ interventions appear likely to be more effective than no intervention but may only lead to modest improvements in vaccination timeliness. The same contextual factors, such as literacy and phone access, may explain the limited effect sizes.

1. (ii). ‘SMS-Plus’ (SMS appointment plus cash incentives or health education SMS)

All three interventions showed positive associations between intervention and vaccination timeliness [59,61]. Group C of Ibraheem et al (2021), which received health education SMS, reported only a slight positive association of 0.08 logOR (95% CI −0.41 to 0.58) [61], with the wide confidence interval range suggesting the possibility of no effect. Groups B and C of Gibson et al (2017) combined SMS appointment reminders with cash incentives of 75KES and 200KES respectively [59]: Group B reported 0.29 logOR (95% CI −0.04 to 0.62), however, again the confidence interval crossed the null, whereas Group C demonstrated a stronger positive association of 0.53 logOR (95% CI 0.18 to 0.88). These findings were assessed as having moderate certainty, indicating satisfactory confidence. While combining SMS appointment reminders with larger cash incentives may enhance vaccination timeliness, more research, particularly cost-effectiveness analyses, is required to support decisions on enhancing interventions for increasing vaccination timeliness.

1. (iii). ‘SMS reminders and/or Voice messages or Phone calls’

Only two interventions in this subgroup reported outcomes categorised as timeliness [57,67]. Yunusa et al (2024) reported a strong association, logOR of 2.60 (95% CI 2.08 to 3.11) [67], while Ekhaguere et al’s (2019) intervention (combining SMS and voice messages), reported a smaller logOR of 0.46 (95% CI, 0.05 to 0.88) [57]. This intervention-outcome group was assessed as having very low certainty of evidence, indicating the true effect may be substantially different from the findings reported here.

Nonetheless, the results suggest that combining SMS reminders with a voice-based component may have a positive effect on vaccination timeliness, by potentially overcoming some of the limitations associated with ‘read-only’ SMS interventions. However, caution is advised given the limited number of studies and the low certainty.

1. (iv). ‘Phone call-only’ reminders

Group A of Ibraheem et al (2021) was the sole ‘Phone call-only’ reminder intervention reporting on timeliness with a logOR of 1.41 (95% CI, 0.79 to 2.03) [61]. It’s relatively narrow confidence intervals and satisfactory risk of bias, suggests a likely positive effect on vaccination timeliness. While this aligns with study findings, the very low certainty of evidence rating means that uncertainty remains, and further high-quality research is needed to strengthen the evidence base.

1. (v). ‘Electronic immunisation alert wristband’

Only one study implemented the wearable electronic immunisation alert wristband, and reported a slight positive association, 0.10 logOR (95% CI, −0.24 to 0.44) [64]. The confidence interval crosses the null, indicating potential for no effect. Again, this intervention received a very low certainty of evidence rating.

Overview

This systematic review and subsequent narrative synthesis investigated which mHealth/DH interventions are most effective at increasing vaccination uptake (of DTP/Pentavalent vaccine) in the 19 SSA countries that were due to roll out malaria vaccine in 2024. Our aim was to identify which mHealth/DH intervention types were most effective at improving childhood vaccination coverage and timeliness in contexts where new vaccine rollouts are planned, and to provide guidance on the development and use of mHealth/DH interventions in future SSA vaccination programmes. Most studies were conducted in West Africa (ten from Nigeria [54,55,57,58,61–64,66,67], one from Burkina Faso [65], and one from Cote D’Ivoire [56]), making the findings more generalisable to that setting. The remaining two studies were conducted in Kenya, East Africa [59,60]. The included mHealth/DH interventions were all related to communication technology and immunisation appointment reminders. The intervention-types investigated included: ‘SMS-only’ appointment reminders, ‘SMS-Plus’ appointment reminders, ‘SMS reminders and/or Voice messages or Phone calls’, ‘Phone call-only’ appointment reminders, and a wearable 'Electronic immunisation alert wristband'. Extremely high heterogeneity was observed across studies, which is common in mHealth/DH intervention research and has been noted in similar reviews [33,36]. Thus, a narrative synthesis was conducted by grouping mHealth/DH intervention-type to assess their effectiveness for improving vaccination coverage and timeliness.

A positive association between intervention and improvement in vaccination coverage was reported for 16 interventions (Fig 3), and 11 interventions showed a positive association between intervention and vaccination timeliness of DTP/Pentavalent3 administration (Fig 4). This consistency in findings across the included studies, enhances the evidence’s reliability. However, since almost all reported interventions showed a positive association, concerns around publication bias must be acknowledged. Conducting a more rigorous grey literature search and searching of trial registries may have reduced potential publication bias and increase this review’s external validity [72]; however, this was outside the scope of this review. Despite this limitation, the review provides robust evidence for informing policy and practice on mHealth/DH intervention implementation in SSA childhood immunisation programmes.

SMS-based interventions (‘SMS-only’ and ‘SMS-Plus’)

When considering intervention-type subgroups, the findings suggest that voice-based interventions (phone call or voice message reminders) are likely to be more effective than SMS-based interventions. Only slight positive associations were observed for both vaccination coverage and timeliness in ‘SMS-only’ appointment reminder interventions, with moderate certainty of evidence. This suggests that sending out ‘SMS-only’ appointment reminders is more favourable than no intervention, however, other types of mHealth/DH intervention may be more effective at improving vaccination coverage and timeliness. The stronger ‘SMS-only’ positive associations were reported in urban areas, indicating potentially greater generalisability to these settings.

‘SMS-Plus’ interventions also showed positive associations, however, stronger associations in comparison to ‘SMS-only’ interventions were not found despite ‘SMS-Plus’ interventions being enhanced. The certainty of evidence for the effect of ‘SMS-Plus’ interventions on vaccination coverage was very low, highlighting uncertainty in the findings. Whereas the intervention–outcome group for ‘SMS-Plus’ and vaccination timeliness was assessed as having moderate certainty of evidence, indicating much greater reliability in findings for that outcome.

Interestingly, although some ‘SMS-Plus’ interventions offered mothers/caregivers cash incentives for attending immunisation appointments, it was the SMS health educational interventions that demonstrated stronger associations with improved vaccination coverage. When considering SMS reminders combined with cash incentives, offering a larger incentive (200KES = ~1.2GBP), appeared to have greater effect on both vaccination coverage and timeliness compared to the lower amount (75KES = ~0.45GBP) [59]. However, according to the GRADE assessment, the evidence supporting these interventions for improving vaccination timeliness was more reliable. In the wider literature, providing mothers with 2USD when their child received timely pentavalent vaccine doses, was found to increase the proportion of individuals receiving timely pentavalent doses [73]. However, this study utilised a small study population, lowering its validity. In contrast, another systematic review, specifically investigating financial incentives for increasing coverage of child health interventions, found limited effect on immunisation coverage when financial incentives were offered, indicating this may not be a useful intervention [74]. Interestingly, an RCT in Ghana found that offering small financial incentives to health care workers improved vaccination coverage and timeliness of BCG and OPV [75], suggesting that incentivising those administering vaccinations might be more effective.

A major limitation of both ‘SMS-only’ and ‘SMS-Plus’ interventions is the possibility that target recipients do not receive the SMS reminders. The ability to log SMS delivery was beyond the scope of the included studies, and assumptions were made on target population receiving SMS messages. This presents an opportunity for future research investigating mHealth SMS intervention delivery success rate in SSA context. Additionally, for SMS-based interventions to be successful, recipients must read and understand the respective messages, highlighting another potential limitation. Low literacy amongst mothers/caregivers could result in them not understanding the contents of SMS reminders, severely limiting their effectiveness. ‘SMS-Plus’ interventions involving sending educational messages around vaccination may be particularly susceptible to this limitation. This could be particularly prominent in rural communities where literacy rates might be lower. The intervention showing least effect in the ‘SMS-only’ subgroup was conducted in a rural area which supports this hypothesis, therefore is an important consideration [59]. The potential limitation around literacy was also reported in a SMS-based data collection study amongst midwives in Liberia, it concluded SMS interventions must be targeted towards those with higher literacy [76]. Although, a certain literacy level is required for these interventions to be effective, new mothers/caregivers should be encouraged to seek help if illiterate, and strategies to complement SMS-based interventions developed to address this barrier. A study in rural Kenya highlighted the importance of keeping SMS messages simple and easy to understand [77]. Investigating the association between participant educational attainment and mother/caregiver literacy and SMS immunisation appointment reminder effectiveness, perhaps through an observational case-control study, would provide valuable insight into this potential limitation.

Despite these limitations, it could be argued that implementing basic SMS-based interventions increases accessibility and may have wider reach. Although SMS-based interventions have fewer capabilities (e.g., extremely limited multi-media content delivery and tracking capabilities), SMS-compatible devices are cheaper and are supported on basic ‘second generation’ 2G cellular connectivity networks which are much more widely available in rural areas [78]. Therefore, when considering connectivity and affordability barriers, implementing basic interventions may be more equitable.

The findings in this review, along with supporting evidence from the wider literature, suggest that ‘SMS-only’ interventions are likely to offer some benefit for improving vaccination outcomes, particularly where more advanced infrastructure or resources are lacking. Given their potential low cost and ease of implementation, such interventions may be a useful entry point in settings preparing for new vaccine rollouts [78]. The results also indicate that enhancing SMS interventions with educational content (‘SMS-Plus’) may improve vaccination coverage more effectively than adding cash incentives, while the inclusion of cash incentives may be more effective for improving timeliness, potentially by creating a sense of urgency around appointment attendance. However, larger cost-effectiveness analyses are needed to determine whether the additional resources required for enhanced interventions deliver sufficient value, particularly in resource-constrained settings.

Voice-based interventions (‘SMS reminders and/or Voice messages or Phone calls’ and ‘Phone call-only’)

In contrast to SMS-only interventions, those incorporating a voice-based component, such as phone calls or voice messages, demonstrated stronger associations with improvements in both vaccination coverage and timeliness.

The intervention showing the strongest association combined SMS and phone call reminders, suggesting that implementing combination would likely be effective for increasing vaccination coverage and timeliness [67]. The certainty of evidence for this intervention-type subgroup was assessed as moderate for vaccination coverage, and very low for vaccination timeliness, indicating that recommendations are more reliable for the vaccination coverage context. When considering risk of bias, Dissieka et al’s (2019) finding was considered the most reliable in the subgroup [56]. Their RCT was conducted in rural Cote D’Ivoire and offered participants the choice of receiving SMS or voice message reminders. Interestingly, most participants chose to receive voice messages, demonstrating that when given the choice, voice messages were the preferred intervention. This finding supports the hypothesis that effectiveness of SMS-based interventions is potentially limited in rural populations with lower literacy. Although Cote D’Ivoire does have relatively high literacy rates in comparison to other SSA nations, female literacy is lower [79]. As mothers/caregivers are the main target of these intervetions, this could explain the finding and further highlights the importance of considering target demographic literacy, particularly in rural settings, when developing mHealth/DH interventions. It is acknowledged that only this one study offered participants a choice of intervention, thus limiting its generalisability. However, it does highlight another research gap and need for further investigation into providing choices of intervention in the immunisation programme context.

Using voice messages has been successful in other contexts in SSA, for example a Nigerian RCT demonstrated that women receiving voice messages were more likely to attend antenatal care appointments [80]. Furthermore, they were implemented in rural Senegal to improve infant and child feeding practices [81]. This certainly highlights the potential for voice-message interventions; however, we recognise that combining SMS interventions with voice-based components may be more resource-intensive. Again, further cost-effectiveness study would help clarify its viability.

Overall, the findings demonstrate that these interventions are likely to be effective for increasing vaccination coverage in routine childhood immunisation programmes, and that offering participants a choice of intervention (SMS or voice message) should be considered during intervention development.

Generally, the positive associations reported in the ‘Phone call-only’ subgroup were stronger in comparison to the other subgroups. This further supports the hypothesis that interventions with voice-based components are more effective. This could be due to recipients responding better to more engaging and personal interventions [82]. Implementing phone calls also creates the opportunity for recipients to potentially seek clarification on queries relating to vaccination and may allow reinforcement messaging about importance of vaccination and adhering to outlined vaccination schedules by vaccinators. Furthermore, using phone calls would overcome the previously outlined limitations of SMS-based reminders, with verbal information more likely to be understood. A Cochrane review investigating various patient reminder interventions, mirrored this review’s findings reporting that phone call reminders were the most effective intervention [83]. However, they did raise the concern around phone call reminders being more costly than other methods. While interventions incorporating phone call reminders appear to be most effective in increasing vaccination coverage and timeliness and show promise, further study is required to increase the findings’ validity.

Despite the presented evidence showing this intervention is likely to be effective, context-specific cost-effectiveness comparisons between SMS and phone call-based reminders should be conducted to inform future vaccination programmes. Another systematic review reported that phone call reminders were effective, however, two of their included studies reported that costs per SMS reminder were considerably lower than phone calls, again highlighting this intervention’s potentially costly nature [84]. With expansion of AI and voice technology, the cost-barriers of these interventions may be reduced and should be strongly considered for future immunisation programmes following cost-effectiveness analysis [85].

Other interventions

The final intervention subgroup was the ‘Electronic immunisation alert wristband’. Although an interesting intervention, only a very slight positive association was shown. Its imprecision and very low certainty of evidence mean recommendations cannot be made supporting this intervention. A similar intervention in Pakistan also showed limited effect [86].

Implications for mHealth/DH intervention development

Based on the review’s findings, implementing mHealth/DH interventions appears to hold promise for improving childhood vaccination coverage and timeliness in SSA context. Interventions that combine SMS appointment reminders with voice-based components, such as phone calls or voice messages, emerged as potentially more effective than SMS-only approaches, particularly in settings with lower literacy. While these approaches may be more resource-intensive, they may offer a more equitable and engaging solution, especially in rural or lower-literacy populations. Therefore, future mHealth/DH immunisation reminder interventions should, where feasible, consider incorporating these elements.

Transferability of findings to the malaria vaccine context

Although this review synthesised evidence from interventions aimed at increasing uptake of the DTP/Pentavalent vaccine, the included countries were selected based on their involvement in the planned RTS,S/AS01 and R21/Matrix-M malaria vaccine rollouts in SSA. The introduction of the first malaria vaccines represents a momentous milestone in public and global health, with the potential to significantly reduce malaria-related morbidity and mortality among children under five [1,87,88]. As such, these findings may provide useful insights to inform the design of mHealth/DH interventions in that context and support improved uptake and timeliness of the new malaria vaccines. Despite the DTP/Pentavalent vaccines and the RTS,S/AS01 and R21/Matrix-M malaria vaccines both being multi-dose vaccines delivered via intramuscular injection, we must acknowledge fundamental differences in these vaccine types and immunisation age profiles. For example, as the DTP/Pentavalent vaccines are delivered at 6, 10, and 14 weeks of age, mHealth/DH interventions to increase uptake of these are targeting increased uptake of vaccine coverage and timeliness in a child’s early infancy [19,20]. Contrastingly, despite requiring multiple doses, the malaria vaccines are administered to children later, from around five months of age, through a three-dose primary schedule, followed by a booster dose after 12–18 months, meaning recipients are much older and care givers may have differing health seeking priorities depending on their baby’s age [87–89]. Thus, we recognise that the findings generated in this review may be more applicable to immunisation programmes for children in early infancy, such as the primary schedules of the EPI.

This review focused specifically on interventions targeting DTP/Pentavalent vaccines because these are the most widely implemented multi-dose vaccines across the EPI in SSA, providing a consistent and comparable evidence base across countries [21,90]. Furthermore, DTP/Pentavalent3 coverage is often used as a key metric to measure the performance of a country’s routine immunisation programme, with high DTP/Pentavalent3 coverage indicating that the health system is effectively reaching infants with the full multi-dose series [21,90]. We acknowledge, however, that the contextual factors influencing uptake of DTP/Pentavalent vaccines, typically administered in early infancy, may differ from those affecting multi-dose vaccines delivered later in childhood, such as the Measles-Containing Vaccine (MCV) or Measles, Mumps and Rubella (MMR) vaccine [21,91]. Interventions targeting these later-age vaccines may therefore offer insights that are more directly comparable to the malaria vaccines, which are administered at older ages. SMS text message reminder systems have been shown in the literature to be effective for MCV. Mekonnen et al. (2019) reported that SMS reminders significantly improved both coverage and timeliness for MCV, with timely coverage increasing from 79.3% to 91.5% (p < 0.001) in the intervention group compared with usual care [92]. Furthermore, an RCT in Kenya assessed the impact of SMS reminders, with and without small unconditional monetary incentives, on timely uptake of the first MCV dose. The study found that both SMS-only and SMS-plus-cash incentive interventions increased timely vaccination coverage compared with the control group, though only the SMS and cash incentive arm achieved a statistically significant effect [93]. This suggests that ‘SMS-Plus’ approaches, where SMS reminders are complemented by an additional component such as a monetary incentive, may be particularly effective for vaccines administered at older ages, such as the malaria vaccines. This represents an important route for future investigation.

While it is plausible that interventions effective for improving DTP/Pentavalent vaccine coverage and timeliness could have similar effects for malaria vaccines, we acknowledge that disease characteristics and differences in immunisation age schedules mean that further context-specific research is needed before drawing definitive conclusions. Nevertheless, the findings from this review provide useful, evidence-based insights to inform the design and implementation of mHealth/DH interventions in future multi-dose vaccination programmes across SSA.

Findings in comparison to other systematic reviews

As previously mentioned, several similar systematic reviews were identified during scoping searches. Gilano et al (2024) investigated the effect of mHealth on all essential childhood immunisations. Despite reporting extremely high heterogeneity, they conducted a meta-analysis and reported an OR of 2.21 for Penta3 dose [36]. Our narrative findings mirror this and further support use of mHealth/DH interventions to increase vaccination coverage. Eze et al (2021) solely focused on SMS appointment reminders in LMICs in general and like our review focused on DTP/Pentavalent vaccine [33]. Again, despite reporting high heterogeneity, a meta-analysis found that implementing SMS reminders significantly improved childhood immunisation coverage (RR = 1.16, 95% CI 1.10 to 1.21) and timeliness (RR = 1.21, 95% CI 1.12 to 1.30) [33]. Our systematic review did find similar conclusions that mHealth/DH interventions of all types including SMS appointment reminders, were likely to improve vaccination coverage and timeliness. The authors recognised phone call reminders would be advantageous for reaching populations with limited or no education, and future studies should explore a combined intervention for optimising immunisation outcomes. This complements the ideas generated by in review, further increasing the evidence-base.

Geographic generalisability of results in SSA

This review aimed to synthesise evidence on mHealth/DH interventions for increasing uptake of DTP/Pentavalent vaccines in the 19 SSA countries targeted to roll out malaria vaccines in 2024. However, studies from only four of these countries were included, meaning 15 were not represented, which limits the generalisability of the findings.

Most included studies were conducted in Nigeria, which likely reflects the country’s large population, and active mHealth/DH research landscape [94,95]. Therefore, the findings may be particularly applicable to the Nigerian context, where previous research has indicated that mHealth/DH interventions are generally acceptable [96–98]. However, challenges such as low mobile phone ownership and literacy rates, particularly in rural Nigerian communities show the need to consider these context-specific factors during intervention design to ensure equitable implementation [99].

Although 15 SSA countries were not represented in our review, there is evidence showing mHealth/DH interventions are being implemented in these countries. A comparative review on mHealth and eHealth use in SSA included studies from 17 SSA countries [100]. It reported Uganda, South Africa, Madagascar and Kenya as the SSA countries with the most mHealth/DH studies. Aboye et al (2023), also found Uganda to be most represented (along with Kenya and Nigeria) [5]. Whilst South Africa and Madagascar were out with this review’s scope, it is questionable why Uganda was not represented here. There is evidence of mHealth/DH use in Uganda in the context of vaccination, for example using DHIS2 [101]. Additionally, one of the excluded studies in this review was an RCT protocol on SMS to reduce vaccination dropouts in Uganda, showing research is happening there [102]. mHealth/DH interventions have also been used in more of the 15 SSA countries not represented here such as Ghana, Sierra Leone, and South Sudan [75,103,104]. This suggests that implementing a mHealth/DH intervention in childhood immunisation programmes in these countries would likely be effective, however, it is acknowledged that country-specific communication preferences should be explored.

Although this review focused on the stated 19 SSA countries, it is important to acknowledge that valuable mHealth/DH research is also being conducted in other countries across SSA. For example, countries such as Ethiopia, which were not within the review’s inclusion scope, are emerging as key contributors to this research area [92]. For example, an RCT in Northwestern Ethiopia, found that mHealth text message reminders improved both Penta3 coverage (95.8% vs 86.9%, p < 0.001) and timeliness of administration (88.7% vs 69.2%, p < 0.001) compared with no intervention, demonstrating this mHealth intervention’s potential effectiveness in Ethiopian context [92]. Future reviews may benefit from expanding geographic inclusion to capture insights from a broader range of SSA contexts.

Impact and important considerations

It is well established that essential childhood vaccinations contribute hugely to public health preventing millions of deaths annually, and finding strategies to increase their uptake and coverage, whilst ensuring their timely administration is essential to maximise their protective benefits. Although many of the mHealth/DH interventions in this review showed only a small effect, when scaled to population level they could have great positive impact. As full rollout of the malaria vaccines began in 2024, there is huge potential to alleviate some of the malaria-attributed childhood mortality and morbidity, and the presented evidence in this review suggests that mHealth/DH interventions could support health systems and immunisation programmes in SSA to increase vaccine reach. However, country-specific studies on intervention acceptability and feasibility in both urban and rural settings, must be conducted to assess their effectiveness in specific contexts. Additionally, the findings highlight the need for considerations around target population literacy and technology accessibility, particularly in relation to implementing SMS-based interventions. Implementing voice-based interventions may overcome some of these barriers, however, questions around costs may limit feasibility in more resource limited settings. It must be acknowledged that accessibility and affordability of mobile devices remain significant barriers in LMICs, with many people lacking mobile phone access or sufficient digital literacy [105–107]. High mobile data connectivity costs in LMICs are also potentially prohibitive, which would limit these interventions [32,107]. These challenges are arguably more prevalent in low-income rural populations, potentially excluding vulnerable groups from beneficial health interventions [105,107]. Despite the clear advantages of mHealth/DH interventions, care must be taken during intervention development not to exclude populations and avoid exacerbating inequity and growing the ‘digital divide’ [32].

Limitations

A key limitation of this review is the limited transferability of findings from interventions targeting DTP/Pentavalent vaccination to other immunisation programmes, particularly the malaria vaccine context. Although both are childhood vaccines, DTP/Pentavalent vaccines are administered in early infancy, whereas the malaria vaccines have a different disease profile and are delivered later in infancy (typically from around five months of age). Therefore, while the findings may offer useful insights, they should be interpreted with caution when applied to the malaria vaccine rollout. Overall, the review provides valuable evidence-based insights for the use of mHealth/DH interventions in SSA immunisation programmes, however, several other key limitations must be acknowledged. For example, only published literature was included, highlighting potential publication bias. A more thorough search of the unpublished material or clinical trial registries would have reduced this limitation but was not possible due to resource constraints. Additionally, it is recognised that in a narrative synthesis, although guided by quantitative findings, there is possibility for subjectivity in selection of findings which could have led to reporting bias, however, the included certainty of evidence assessment considers the potential impact of reporting and publication bias on the confidence in presented results. In terms of study findings, it is recognised that by limiting the study context to only 19 SSA countries, the search breadth and inclusion criteria was limited and excluded countries such as South Africa, Ethiopia and Tanzania where mHealth/DH research is occurring [100]. Another limitation is generalisability of findings, with Nigeria being disproportionately represented here. Although Kenya, Burkina Faso and Cote D’Ivoire were represented, the overall finding’s transferability to these countries is also limited due to study low numbers. Although the included studies were from a mix of urban and rural settings, it must be noted that different countries and settings likely have different preferences and cultural factors for communications. For example, it was shown that Kenyan farmers preferred phone calls over SMS [82].