https://orcid.org/0000-0001-7629-3389
Figures
Abstract
Around half of adolescent pregnancies in low- and middle-income countries are unintended, contributing to millions of unsafe abortions per year. Adolescents 360 (A360), a girl-centred initiative, aimed to increase voluntary uptake of modern contraceptives among adolescents in Nigeria, Ethiopia and Tanzania. We evaluated the effectiveness and cost-effectiveness of A360 in increasing modern contraceptive use in selected geographies. We used before-and-after cross-sectional studies of adolescent girls in four settings. Two Nigerian settings had purposefully selected comparison areas. Baseline and endline household surveys were conducted. The primary study outcome was modern contraceptive prevalence rate (mCPR). Secondary outcomes mapped onto the A360 Theory of Change. Interpretation was aided by a process evaluation along with secular mCPR trends and self-reported A360 exposure data. Incremental design and implementation costs were calculated from implementer systems, site visits, surveys, and interviews. mCPR change was modelled into maternal disability-adjusted life years (DALY) averted to calculate incremental cost-effectiveness ratios. In Oromia, Ethiopia, mCPR increased by 5% points (95% CI 1–10; n = 1,697). In Nigeria, there was no evidence of an effect of A360 on mCPR in Nasarawa (risk ratio: 0·96, 95% CI: 0·76–1·21; n = 5,414) or in Ogun (risk ratio: 1·08, 95% CI: 0·92–1·26; n = 3,230). In Mwanza, Tanzania, mCPR decreased by 9% points (-17 to -0.3; n = 1,973). Incremental cost per DALY averted were $30,855 in Oromia, $111,416 in Nasarawa, $30,114 in Ogun, and $25,579 in Mwanza. Costs per DALY averted were 14–53 times gross domestic product per capita. A360 did not lead to increased adolescent use of modern contraceptives at a population level, except in Oromia, and was not cost-effective. This novel adolescent-centred design approach showed some promise in addressing the reproductive health needs of adolescents, but must be accompanied by efforts to address the contextual drivers of low modern contraceptive use.
Citation: Krug C, Neuman M, Rosen JE, Weinberger M, Wallach S, Lagaay M, et al. (2023) Effect and cost-effectiveness of human-centred design-based approaches to increase adolescent uptake of modern contraceptives in Nigeria, Ethiopia and Tanzania: Population-based, quasi-experimental studies. PLOS Glob Public Health 3(10): e0002347. https://doi.org/10.1371/journal.pgph.0002347
Editor: Hannah Tappis, Jhpiego, UNITED STATES
Received: March 21, 2023; Accepted: September 26, 2023; Published: October 18, 2023
Copyright: © 2023 Krug et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data is available at the LSHTM Data Compass: https://doi.org/10.17037/DATA.00003599.
Funding: This work was supported by the Bill & Melinda Gates Foundation [OPP1134172] (SW, ML, AD, MN, MP) and the Children’s Investment Fund Foundation (SW, ML, AD, MN, MP). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Around half of adolescent pregnancies in low- and middle-income countries are unintended [1]. Ensuring that adolescent girls have access to sexual and reproductive health-care services is critical for achieving universal access (Sustainable Development Goal 3.7). Moreover, being able to control their fertility underpins educational and employment opportunities [2]. Although many programs have sought to increase contraceptive use among adolescents, their effectiveness has been limited [3, 4].
Adolescents 360 (A360) was a four-year (2016–2020) initiative to increase voluntary uptake of modern contraceptives among adolescent girls in four settings. A360 used human-centred design (HCD) to develop setting-specific interventions through an iterative process of research and prototyping. The program hypothesis was that meaningful engagement of adolescents would catalyse the development of novel, successful approaches. The expectation was that higher design costs would be offset by better-designed and more effective interventions to produce a cost-effective approach.
We describe outcome evaluation and cost-effectiveness studies of A360 in Northern Nigeria, Southern Nigeria, Ethiopia and Tanzania, and draw on process evaluation findings to interpret the results.
Methods
Study design
The study design is described in detail elsewhere [5]. We used repeat cross-sectional surveys to evaluate the impact of A360 on modern contraceptive use. In Nigeria, the study design included a comparison area purposively selected by the implementers, Society for Family Health, in collaboration with the state Ministry of Health and local government officials. Comparison-intervention pairs (two in Northern Nigeria and one in Southern Nigeria) were selected to be similar with respect to some or all of the following criteria: population density, estimated modern contraceptive prevalence rate (mCPR) among 15 to 49 year olds, number of health facilities and presence of World Bank support for Maternal and Child Health activities. In each setting, eligible girls were identified at the household level. The smallest available administrative unit was used as the primary sampling unit–enumeration area in Nigeria, kebele in Ethiopia and street in Tanzania. Study areas were selected in collaboration with the implementers. The same study design and the same primary sampling units were included at baseline and endline, where possible. Although the design means that it is possible that in each site the same households and individuals may be included in the baseline and endline surveys, no attempt was made to trace individuals or households from baseline to endline.
The outcome and cost-effectiveness studies were conducted in the same locations. In Northern Nigeria, ‘Matasa Matan Arewa’ (‘Adolescent Girls from the North’) was evaluated in four local government areas in Nasarawa state; in Southern Nigeria, ‘9ja Girls’ was evaluated in two local government areas in Ogun state. In Ethiopia, ‘Smart Start’ was evaluated in four woredas (districts) in Oromia state. In Tanzania, ‘Kuwa Mjanja’ (‘Be Smart’) was evaluated in Ilemela district in Mwanza region.
Participants
We included girls aged 15 to 19 years who would be eligible for the A360 intervention. In Nasarawa and Oromia, we included girls who were married or living as married; in Ogun, we included girls who were unmarried; and in Mwanza, we included girls who were married or unmarried.
In Nasarawa and Ogun, we collected baseline data between August and September 2017 and endline survey data between November and December 2020. In Oromia, we collected baseline data between September and October 2017 and endline data between November and December 2020. In Mwanza, we collected baseline data between September 2017 and January 2018 and endline data between May and October 2021.
Individual, written consent, was obtained from all participants before conducting the interviews. In Ogun, parental/guardian consent and adolescent girl assent were required for unmarried girls aged up to 17 years. In Mwanza, parental consent waiver was granted for this age group because of the sensitive nature of the survey.
Sample size
The target sample size of 23,481 (Nasarawa 4,555, Ogun 12,020, Oromia 1,926, Mwanza 4,980) was chosen to provide 90% power to detect an intervention effect at each of the study sites. Sample size calculations took into account the design effect (clustering), estimated non-response, and the fact that not all girls were sexually active. We assumed A360 would increase mCPR from: 3·0% to 5·1% in Nasarawa, 64·4% to 72·6% in Ogun, 44·0% to 50·8% in Oromia and 26·7% to 32·7% in Mwanza. The effect estimates and baseline mCPR rates were derived from a review of 25 studies as detailed in the protocol paper [5]. In Nasarawa, Ogun and Mwanza, baseline mCPR was higher than expected, which led to revised endline sample sizes (S1 Text).
Interventions
The final package of interventions was site-specific and is presented in Table 1, according to template for intervention description and replication (TIDieR) checklist and guide [6]. The A360 program tracked other sexual and reproductive health interventions in intervention (and comparison areas in Nigeria), which are also summarized in Table 1 [7].
Objectives
Our primary aim was to evaluate the effectiveness and cost-effectiveness of the A360 approach in increasing modern contraceptive use among sexually active girls aged 15–19 years. Our secondary aims align with the A360 Theory of Change components and are described in S1 Table. We also quantified the association between the respondents’ self-reported exposure to A360 and primary and secondary outcomes.
Outcomes
The primary study outcome was the proportion of fecund and sexually active girls who reported using modern contraception at the time of the surveys (mCPR). Modern contraception included male and female sterilisation, contraceptive implants, intrauterine contraceptive devices, injectables, contraceptive pill/oral contraceptives, emergency contraceptive pill, male condom, female condom, Standard Days Method, Lactational Amenorrhoea Method, diaphragm, spermicides, foams and jelly [5]. To better understand the pathways through which the A360 approach could affect mCPR, secondary outcomes were also measured (S1 Table). One of the secondary outcomes was proportion of current modern contraceptive users who were using a long-acting reversible contraceptive (LARC), which was measured due to the large global emphasis put on this type of methods to offer girls with access to the widest available contraceptive options.
Questionnaires were adapted from Demographic and Health Survey and Family Planning 2020 survey instruments and were pre-tested for comprehension, flow, appropriateness and feasibility of implementation. At baseline, all questionnaires were administered face-to-face, whereas at endline, in Nigeria and Ethiopia, the first part of the questionnaire was administered face-to-face while the second part of the questionnaire was administered by phone due to COVID-19 related restrictions on data collection (S1 Text).
Statistical methods
The impact of A360 interventions was assessed by quantifying change between baseline and endline, guided by a pre-specified analysis plan (S1 Text). In Nasarawa and Ogun, we used a difference in difference approach using Poisson regression (binary outcomes) or linear regression (continuous outcomes). The model included area (intervention versus comparison), time, and an interaction between area and time. The interaction term coefficient, reflecting the effect of A360 beyond the time trend, was used to assess impact. Models additionally included age, education level, number of living children, religion and wealth quintile. Clustering at the level of the primary sampling unit was accounted for by using cluster-robust standard errors to calculate p-values and confidence intervals. Differences between intervention and comparison areas at baseline were tested using t-test or Pearson χ2 test, as appropriate. The validity of the difference in difference approach depends on the assumption that, if the intervention had not occurred, the trend in mCPR would have been the same in the intervention and comparison areas [17]. To assess these trends, we used Health Management Information System service data.
In Oromia and Mwanza, linear regression models were fitted to data from the baseline and endline surveys aggregated at the level of the primary sampling unit. Matching was accounted for by including the primary sampling unit in the regression model as a categorical variable. The models also included time and the confounders variables listed above. Observed changes in mCPR could be due to secular trends [5]. Therefore, we examined trends over time in study settings, using secondary datasets. The details of this analysis are presented in S1 Text.
The association between self-reported exposure to A360 and mCPR was evaluated using data from intervention areas at endline. A girl was considered exposed if she reported hearing about A360 interventions available in the place where she lived (S1 Text). Poisson (Nasarawa and Ogun) or logistic regression (Oromia and Mwanza) were used for binary outcomes and linear regression was used for continuous outcomes. The models included a variable denoting exposure status, and the confounder variables listed above. Cluster-robust standard errors were used to account for the clustering at the primary sampling unit level.
Costing and cost-effectiveness
The cost-effectiveness analysis defined the comparator for A360 as the status quo for design and implementation of adolescent programming. The design comparator was Population Services International’s (PSI) DELTA design methodology [18], the standard used at the time A360 initiated. The implementation comparator was the existing contraceptive programming available to adolescents in the A360 study geographies. We estimated incremental cost-effectiveness ratios for each study setting. Incremental costs were A360 design and implementation costs minus the comparator cost. See S1 Text for additional details.
Role of the funding source
The final study designs were discussed and agreed upon with the funders. The funders had no role in the data collection, data analysis, or writing of the manuscript. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Ethics approvals
The study components were approved by the National Health Research Ethics Committee of Nigeria (NHREC/01/01/2007-25/05/2017; NHREC/01/01/2007-23/06/2020C; NHREC/01/01/2007-15 /03/2019C), the Oromia Health Bureau Research Ethical Review Committee in Ethiopia (BEFOIHBTFH/1-8/2844; BEF0/AHBTFH/1-16/3089), the Addis Ababa University, College of Health Sciences Institutional Review Board (074/16/SPH), the National Health Research Ethics Review sub-Committee of Tanzania (NIMR/HQ/R.8a/Vol. IX/2549; NIMR/HQ/R.8a/Vol. IX/3508; NIMR/HQ/R.8a/Vol.IX/2346) and the London School of Hygiene and Tropical Medicine Ethics Committee (Ref: 14145).
Results
A360 outcome evaluation
Survey response rates were between 69% and 100% (Fig 1). Selected sociodemographic characteristics of girls in the baseline sample are presented in Table 2. S2 Table illustrates differences in number of children, religion, wealth and mobile phone access in intervention and comparison areas at baseline.
HH, households, SA, fecund and sexually active girls during the 12 months before the interview.
Nasarawa, Nigeria
In Nasarawa, mCPR increased from 16% to 38% in the intervention areas and from 13% to 27% in comparison areas. After accounting for the change in mCPR in comparison areas via the difference in differences analysis, we found no evidence of an impact of A360 (risk ratio: 0·96; 95% CI: 0·76–1·21; p = 0·734; Table 3).
We observed intervention effects in two out of 17 secondary outcomes. For instance, girls in intervention areas reported a greater increase over time in positive attitudes to couples using modern contraceptives compared to those in comparison areas (S3 Table).
Self-reported exposure to ‘Matasa Matan Arewa’ was low in intervention areas, varying between 5% (95% CI: 4–6) and 7% (95% CI: 5–10). There was a positive association between self-reported exposure and mCPR (risk ratio 1·41; 95% CI: 1·13–1·76; p<0·001; Table 2), and between self-reported exposure and 10 out of 17 secondary outcomes (S4 Table).
Ogun, Nigeria
In Ogun, mCPR increased from 45% (95% CI: 41–48) to 49% (95% CI: 44–53) in intervention areas but remained constant at 51% in comparison areas (Table 2). In the difference and differences analysis, we found no evidence of an impact of A360 (risk ratio, 95% CI: 1·08, 0·92–1·26; p = 0·340; Table 3).
Out of 17 secondary outcomes, the only effect observed in the hypothesised direction was on the proportion of LARC users among all modern contraceptive users which increased from 0·3% to 2·2% at the intervention area and dropped from 1·4% to 1·0% at the comparison area (S3 Table).
Self-reported exposure to 9ja Girls was 8% (95% CI: 6–10) in the intervention area. There was no relationship between self-reported exposure and primary and secondary outcomes, except for a weak evidence of a positive relationship between being exposed and being aware of contraceptive products (S4 Table).
Oromia, Ethiopia
In Oromia, the kebele-average mCPR was 64% (95% CI: 57–71) at baseline and 68% (95% CI: 62–75) at endline. In the adjusted analysis, we estimated that mCPR increased by five percentage points between baseline and endline surveys (95% CI: 1–10; p = 0·025; Table 3).
This increase in mCPR was accompanied by an increase in four out of 17 secondary outcomes (S4 Table). For instance, there was a 10% absolute increase in the proportion of current modern contraceptive users using a LARC (95% CI: 3–17; p = 0·004), and an 11% increase (95% CI: 1–21; p = 0·025) in awareness of contraceptive products. There was also evidence of an in increase in the proportion of adolescent with positive attitudes towards the use of modern contraceptives (S3 Table).
Self-reported exposure to Smart Start was 24% (95% CI: 18–30). There was a positive association between self-reported exposure and mCPR (odds ratio: 2·09; 95% CI: 1·32–3·29; p = 0·002; Table 3), and between self-reported exposure and two out of 17 secondary outcomes (S4 Table).
Mwanza, Tanzania
The eligible population of girls was primarily unmarried sexually active girls (around 94%). mCPR was 50% (95% CI: 46–54) at baseline and 40% (95% CI: 37–44) at endline. Following adjustment, we estimated a 9% absolute decrease in mCPR over time (95% CI: -17 to -0·3; p = 0·043; Table 3). The decrease in mCPR was driven by a decline in self-reported male condom use from 34% (95% CI: 31–37) at baseline to 19% (95% CI: 16–22) at endline. There was an 8% decrease in mCPR over time (95% CI: -15 to -0·3; p = 0·042) among unmarried girls and a 4% increase in married girls (95% CI: -6 to +14; p = 0·407).
This decrease in mCPR was accompanied by declines in several secondary outcomes. For instance, there was a 5% decrease (95% CI: -9 to -3; p<0·001) in the proportion of adolescent girls agreeing that contraception can help adolescent woman/girl to complete their education, find a better job and have a better life. There was also a 14% decrease (95% CI: -24 to -4; p = 0·008) on the intention to use a modern method, and a 9% increase (95% CI: 2–16; p = 0.019) in the proportion of modern contraceptive users using a LARC over time (S3 Table).
Self-reported exposure to Kuwa Mjanja was 24% (95% CI: 21–26). There was a positive association between self-reported exposure and mCPR (OR: 1·63; 95% CI: 1·28–2·09; p<0·001), and between self-reported exposure and five out of 17 secondary outcomes (S4 Table).
Analysis of trend using other sources of data
In S1 Text, we present the results of the analysis of trends in modern contraceptive use for all three sites, using secondary data sources. In Nigeria, the data provided some evidence on the validity of the parallel trend assumption in Nasarawa and only weak evidence in Ogun. In Ethiopia, no clear trend was observed in mCPR, but in Tanzania, we observed an upward trend (S1 Text).
Cost effectiveness
A360 design costs were seven to nine times higher than the comparator DELTA approach. A360 implementation costs were also substantially higher than maintaining existing contraceptive programming available to adolescents in the A360 study geographies. Incremental cost for the study geographies was $484,900 for Nasarawa, $513,220 for Ogun, $970,667 for Oromia, and $120,479 for Mwanza. The annual per capita (total population) spending on A360 design and implementation was between $0.22 in Mwanza and $0.67 in Oromia. As a percent of total health spending per capita (using WHO estimates of per capita health spending, adjusted to 2020 USD) [19], spending on A360 represented between 0·3% in Ogun and 2·5% in Oromia. Spending per eligible girl (sexually active, fecund, married/unmarried as per geography) per year of implementation ranged between $13 in Mwanza to $102 in Nasarawa (Table 4).
The mCPR changes translated to 4·4 maternal disability-adjusted life years (DALY) averted in Nasarawa, 17·0 in Ogun, 31·5 averted in Oromia, and 4·7 in Mwanza. Despite the lack of change in Nasarawa and the declines in modern contraceptive prevalence in Mwanza seen in the outcome evaluation, DALY impacts were positive due to increases in the number of eligible adolescents that result in positive additional users over the life of the project.
Dividing incremental costs by incremental effectiveness, produced an incremental cost per DALY averted of $111,416 (53 times gross domestic product [GDP] per capita) in Nasarawa, $30,114 (14 times GDP per capita) in Ogun, $30,855 (33 times GDP per capita) in Oromia, and $25,579 (24 times GDP per capita) in Mwanza (Table 5).
Discussion
Our evaluation suggested evidence that A360 did not lead to increased adolescent use of modern contraceptives at a population level, except in Oromia, Ethiopia. Self-reported exposure to A360 was low, ranging from 5–24%, and in three settings was positively associated with mCPR. We observed a positive intervention impact on some secondary outcomes linked to the A360 Theory of Change. External data sources suggested an upward trend in mCPR among women aged 15–49 years. A360 was not cost-effective, with design costs higher than the comparator design approach. The absolute amount spent in relationship to population size and overall health spending was substantial.
The outcome evaluation had many strengths including accounting for clustering when calculating required sample sizes, the collection of comparable data before and after intervention implementation, use of multiple data sources to track population level mCPR and, in Nasarawa and Ogun, collection of data from populations both exposed and not exposed to the intervention. Participants were representative of A360 target populations increasing internal and external validity of the study. Data on self-reported exposure to the A360 programmes at endline allowed examination of the association between individual-level engagement with A360 and modern contraception use.
An important limitation was the lack of comparison areas in Mwanza and Oromia as a change in mCPR may reflect a time trend rather than an intervention effect [17, 20–22]. Our findings may not be generalizable to other areas of the countries where A360 was implemented. We relied on respondent self-reporting to measure modern contraceptive use, sexual activity and exposure to the programs; these behaviours may have been subject to reporting bias. The COVID-19 pandemic led to changes to endline survey procedures including the use of face masks and the administration of the second section of the questionnaire by phone in Nasarawa, Ogun and Oromia. In Nasarawa and Ogun, participants in intervention and comparison areas were not entirely comparable in terms of sociodemographic factors. While we adjusted for changes in sociodemographic factors there may be some residual confounding. Finally, the validity of the difference in difference approach used in Nasarawa and Ogun depends on the mCPR time trend being the same in both intervention and comparison areas [17]. The only data available to evaluate similarity in time trends were Health Management Information System data from female clients aged 15–49 years using modern contraceptives at health facilities between early 2016 and mid-2020. These data reflect a broader age range than was included in the survey population and provided only weak evidence on the validity of the parallel trend assumption.
The costing and cost effectiveness study used a consistent approach in all three countries, repeated measures, and drew much of its information from accounting systems. Limitations are detailed in S1 Text. One-way and multi-way cost sensitivity analysis addressed many of these limitations, producing plausible lower and upper ranges to total cost used in sensitivity analysis.
Human-centred design (HCD) was a key element of the A360 approach. As defined by Giacomin [23], HCD ‘is based on the use of techniques which communicate, interact, empathize and stimulate the people involved, obtaining an understanding of their needs’, and it incorporates several features of social, behavioural and community engagement interventions. Deploying HCD in combination with youth engagement, insights from different disciplines, and working adaptively, helped the interventions resonate with girls, communities and government stakeholders [7]. HCD helped A360 integrate aspirational content, which attracted girls to events, built government buy-in, and allowed the program to operate in the context of high levels of stigma. In Oromia, messaging built around couples’ counselling and financial planning, resonated strongly with married couples. Change was enabled by the integration of Smart Start into the Ethiopian government’s Health Extension Program with delivery by Health Extension Workers, who are known and trusted in communities [24]. Among married girls aged 15–19 years in Nasarawa and Oromia, A360 implementation was associated with a population-level change in girls’ belief in the benefits of modern contraception and in the proportion of girls with positive attitudes toward modern contraception.
Sociocultural factors remain a barrier to adolescent contraceptive use in Nasarawa, Ogun and Mwanza. For instance, in Nasarawa, one of the main reasons participants said they were not using contraception was the desire to bear children [25], reflecting established social norms [26]. In Ogun and Mwanza, A360 appeared less effective among unmarried girls perhaps because of insufficient impact on the stigma associated with premarital sex [27–29]. Unmarried girls need youth-friendly service delivery with ensured confidentiality and provider discretion. In both Ogun and Mwanza the majority of unmarried girls reported obtaining contraceptives at settings other than local health facilities. Process evaluation suggested A360 was not designed or resourced to address social norms [7]. A360 instead used light-touch approaches to engage communities, including: parents’ sessions in Mwanza and couples’ counselling in Oromia; working with community leaders, local government officials and trusted community structures; and, developing messaging that tapped into existing community concerns. Other studies suggest that community-level interventions need to be intensive and sustained to have long term impacts on knowledge, attitudes, practices and behaviors [4, 30].
In Mwanza, we observed a population level decrease in mCPR among adolescent girls. This might be explained by three main factors. First, during Kuwa Mjanja implementation in 2018, the Tanzanian former president made negative statements about contraception [31], and advertisements on contraception were later banned [32]. This at one point led to a halt to A360 outreach activities. Second, we saw an increase in the number of adolescent girls residing in study areas (which led to changes in sampling strategy at endline; see S1 Text), and a higher level of education at endline, both of which may have affected results.
The endline surveys of our outcome evaluation were conducted in late 2020, approximately one year after the start of COVID-19 pandemic. In Nigeria, data from Performance Monitoring for Action 2020 [33, 34] and from Krubiner and colleagues [35] indicate that family planning service and product availability for females aged 15–49 years are unlikely to have been impaired due to COVID-19, but it is difficult to know if this was also true among adolescent girls aged 15–19 years in our study geographies. In a State of Northern Nigeria (Kano), women aged 15–24 years who changed their contraceptive use status were more likely to adopt (4%) than to discontinue a method (<1%) [33]. On the other hand, in a State of Southern Nigeria (Lagos), women aged 15–24 years who changed their contraceptive use status were more likely to discontinue (11%) than to adopt a method (5%) [34], but only 2% of women aged 15–49 years stopped or interrupted their contraceptive method use due to COVID-19 restrictions [36]. In Ethiopia, even though some studies showed limited availability of family planning services and products due to COVID-19 [37–39], endline survey data, A360 monitoring data (collected by the implementers) and Health Management Information System service data, indicated that the effects of the pandemic were minimal [40]. Finally, in Tanzania, the COVID-19 pandemic stopped A360 service delivery between March and May 2020 [7], after which Kuwa Mjanja implementation was limited to door-to-door visits and short discussions, to avoid mass gatherings [7]. Nevertheless, the effects of COVID-19 on the supply of contraceptive commodities also seem limited in Tanzania, with the country following the World Health Organization guidance during the pandemic, by relaxing contraceptive prescription requirements and recommending that emergency contraception be available at pharmacies [35].
The low levels of reported exposure to A360 suggests that implementation intensity may have been lower than anticipated in the evaluation geographies and insufficient to achieve population-level change in mCPR. When the evaluation study was designed, the intervention implementation plans had not yet been finalised but the intention was for A360 to be implemented widely in the selected study geographies. As noted previously, HCD-based initiatives may require a phased evaluation approach with an outcome evaluation designed only when the programme and implementation strategy have been finalised [41]. Other alternative explanations for the low levels of reported exposure to A360 are: the tool was inappropriate to capture exposure to the program, although this is unlikely as the tool was validated with the implementers; high levels of migration, although this is also unlikely as in a secondary analysis we found very small proportion of girls having migrated for more than three months in the 12 months previous to the surveys.
The more intensive design effort in A360 was not cost effective in relation to the size of health outcomes achieved. Incremental cost-effectiveness ratios were far above the three times per capita GDP threshold for a cost-effective health intervention, per WHO-CHOICE standards [42]. They were also much higher than the $225 per DALY averted proposed as a cut-off for inclusion of interventions in Universal Health Care package, and far above the cost per DALY averted reported for other family planning interventions (between $235 and $587) [43]. Further analysis suggested that in Oromia, no level of mCPR increase would have led to cost-effective results, while in Nasarawa and Ogun large increases in mCPR would have been needed to reach a benchmark of three times GPD per capita. Efforts to reduce implementation costs will be needed to produce more cost-effective models. Ethiopia and Nigeria are shifting management and service delivery responsibilities for the A360 legacy interventions to governments, which may lower costs. In Mwanza, A360 costs were more in line with potential impact, and even just maintaining baseline mCPR could have led to cost-effective results.
Our results highlight the challenges associated with identifying cost-effective approaches to increase the voluntary use of modern contraceptives among adolescents. A360 is being scaled-up and modified in Ethiopia and Nigeria, and expanded to Kenya [44]. According to the A360 implementers (PSI), the decision to scale-up was driven largely by the governments of the implementing countries. We recommend increased efforts to address social norms that prevent girls from accessing or using contraceptives as well as additional evaluation of A360 among both married and unmarried adolescents. Accordingly, A360’s second phase aims to have a more consistent and rigorous approach to improving the enabling environment [44]. The modified A360 approaches are going to be evaluated through new cost-effectiveness studies.
Supporting information
S1 Text. Supporting information for the Adolescents 360 program evaluation.
S1 Text includes: a series of result figures; a detailed description of the methodology of the Adolescents 360 outcome evaluation and of the cost-effectiveness study; methods and results for the analysis of trends in modern contraceptive use prevalence; and the analysis plan of the Adolescents 360 outcome evaluation.
https://doi.org/10.1371/journal.pgph.0002347.s001
(DOCX)
S2 Text. Questionnaire on inclusivity in global research.
S2 Text includes a questionnaire on inclusivity in global research outlining ethical, cultural, and scientific considerations that were taken into account during Adolescents 360 evaluations.
https://doi.org/10.1371/journal.pgph.0002347.s002
(DOCX)
S3 Text. STROBE statement.
S3 Text includes a checklist of items that should be included in reports of observational studies.
https://doi.org/10.1371/journal.pgph.0002347.s003
(DOCX)
S1 Table. Description of secondary outcomes, by Adolescents 360 theory of change components.
1 Sexually active girls are those who report having sexual intercourse in the last 12 months. S1 Table presents a description of the secondary outcomes measured for the Adolescents 360 outcome evaluation, aligning with Adolescents 360 Theory of Change components.
https://doi.org/10.1371/journal.pgph.0002347.s004
(DOCX)
S2 Table. Demographic characteristics of adolescent girls pre- and post-intervention, by site.
Data are n (%) or mean (SE). 1 p values are for differences between intervention and comparison areas pre-intervention. S2 Table presents a description of demographic characteristics of adolescent girls included in the Adolescents 360 outcome evaluation, pre- and post-intervention, by site.
https://doi.org/10.1371/journal.pgph.0002347.s005
(DOCX)
S3 Table. Pre- versus post-intervention comparison of primary and secondary outcomes, by site.
mCPR, modern contraceptive prevalence rate, LARC, long-acting reversible contraceptive, Data are n (%) or mean (SE). 1 Girls who agreed with the sentence ‘Using modern contraception can allow an adolescent woman girl to complete her education, find a better job and have a better life’ 2 The impact of the Adolescents 360 approach is defined as the risk of mCPR pre- versus post-intervention. S3 Table presents a description of primary and secondary outcomes measured for the Adolescents 360 outcome evaluation, pre- and post-intervention, by site. This table also presents the impact of the Adolescents 360 approach in each site.
https://doi.org/10.1371/journal.pgph.0002347.s006
(DOCX)
S4 Table. Exposed versus non-exposed comparison of primary and secondary outcomes at endline, by site.
A360, Adolescents 360 approach, mCPR, modern contraceptive prevalence rate, LARC, long-acting reversible contraceptive, Data are n (%) or mean (SE). 1 Girls who agreed with the sentence ‘Using modern contraception can allow an adolescent woman girl to complete her education, find a better job and have a better life’ 2 Girls who agreed with the sentence ‘Using modern contraception can allow a girl to achieve her life goals’ 3 The impact of the A360 exposure is defined as the risk of mCPR in the exposed compared to girls not exposed to A360. S4 Table presents a description of primary and secondary outcomes measured for the Adolescents 360 outcome evaluation, by girls who reported being exposed to the intervention and girls who did not report being exposed to the intervention, by site.
https://doi.org/10.1371/journal.pgph.0002347.s007
(DOCX)
Acknowledgments
We would like to thank all the interviewees, particularly the adolescent girls who engaged with us for sharing their perspectives. We thank Itad as the lead organization responsible for the overall A360 evaluation. We thank PSI Headquarters, PSI Ethiopia, PSI Tanzania and Society for Family Health for their engagement in conversations around the A360 programme and its implementation, and the evaluation study design; and their contributions to the cost study, and their participation in discussions on interpretation of the findings. We thank all the government officials, health workers, and community volunteers interviewed for the cost study.
References
- 1.
World Health Organization [WHO]. World Health Organization 2020 [cited 2022 July 1]. Available from: https://www.who.int/en/news-room/fact-sheets/detail/adolescent-pregnancy.
- 2. Alano A, Hanson L. Women’s perception about contraceptive use benefits towards empowerment: A phenomenological study in Southern Ethiopia. PloS one 2018; 13 (suppl 9). pmid:30212500
- 3. Gottschalk LB, Ortayli N. Interventions to improve adolescents’ contraceptive behaviors in low-and middle-income countries: a review of the evidence base. Contracept 2014; 90 (suppl 3): 211–25. pmid:24916724
- 4. Chandra-Mouli V, Lane C, Wong S. What does not work in adolescent sexual and reproductive health: a review of evidence on interventions commonly accepted as best practices. Global Health: Sci Pract 2015; 3 (suppl 3): 333–40. pmid:26374795
- 5. Atchison CJ, Mulhern E, Kapiga S, Nsanya MK, Crawford EE, Mussa M, et al. Evaluating the impact of an intervention to increase uptake of modern contraceptives among adolescent girls (15–19 years) in Nigeria, Ethiopia and Tanzania: the Adolescents 360 quasi-experimental study protocol. BMJ open 2018; 8 (suppl 5). pmid:29858422
- 6. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014; 348. pmid:24609605
- 7.
Punton M, Wallach S. Adolescents 360 Process Evaluation Final Report. Itad; 2020 [cited 2022 July 1]. Available from: https://www.itad.com/knowledge-product/a360-process-evaluation-final-report/.
- 8.
Malakoff S, Cutherell M, Coppola A. Aligning Contraception with Family and Life Goals of Married Adolescent Girls in Northern Nigeria: The Case of Matasa Matan Arewa. Washington, DC: Population Services International; 2021. Available from: https://a360learninghub.org/wp-content/uploads/2021/02/A360_MMA_TechPub_FINAL_020321.pdf.
- 9.
Cole C, Cutherell M, Phillips M. Connecting Contraception to Girls’ Lives and Aspirations in Southern Nigeria: The Case of 9ja Girls. Washington, DC: Population Services International; 2020 [cited 2023 June 23]. https://a360learninghub.org/wp-content/uploads/2020/11/A360_9jaGirls_TechPub-11-30.pdf.
- 10.
Cutherell M, Cole C. Supporting Scalable, Youth-Powered Programming at the Community Level in Ethiopia: The Case of Smart Start. Washington, DC: Population Services International; 2019 [cited 2023 June 23]. http://a360learninghub.org/wp-content/uploads/2019/10/SS-Tech-Pub-WEB-10-17.pdf.
- 11.
Marie Stopes International Reproductive Choices [MSI]. Marie Stopes Nigeria. Available from: https://www.msichoices.org/where-we-work/nigeria/.
- 12.
Planned Parenthood Federation of Nigeria [PPFN]. Available from: https://www.ppfn.org/.
- 13.
The White Ribbon Alliance [WRA]. Available from: https://www.whiteribbonalliance.org/nigeria/.
- 14. Jackson E, McDougal L, Raj A. Community Experiences with a School-Based Intervention to Delay Marriage among Girls in Oromia Region. Ethiopia: 2017 [cited 2023 June 23]. Available from: https://gehweb.ucsd.edu/wp-content/uploads/2019/04/raj_mcdougapackard-pb-oda_ethiopia_9_12_17.pdf.
- 15. Kaba M, Adugna Z. School level menstrual hygiene management in Oromia region of Ethiopia. Ethiop J Repr Health 2020; 12(4): 7–7.
- 16. Kaba M, Geletu ZA. Mensural Hygiene Management in Schools of Oromia Region. Ethiopia; 2020.
- 17.
Shadish WR, Cook TD, Campbell DT. Quasi-experimental designs that either lack a control group or lack pretest observations on the outcome. In: Shadish WR, Cook TD, editors. Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin Company; 2002. pp. 103–34.
- 18.
Population Services International [PSI]. DELTA Companion Marketing Planning Made Easy Washington DC: PSI; 2009. Available from: https://www.psi.org/publication/delta-companion/.
- 19.
The World Bank Group. The World Bank. Current health expenditure per capital (current US$) 2022 [cited 2022 Aug 1]. Available from: https://data.worldbank.org/indicator/SH.XPD.CHEX.PC.CD?name_desc=true.
- 20. Marsden E, Torgerson CJ. Single group, pre-and post-test research designs: Some methodological concerns. Oxf Rev Educ 2012; 38 (suppl 5): 583–616.
- 21.
Robson L, Shannon H, Goldenhar L, Hale A. Before-and-after design: A simple evaluation design. Guide to Evaluating the Effectiveness of Strategies for Preventing Work Injuries: How to Show Whether a Safety Intervention Really Works. Druid Hills: Centers for Disease Control and Prevention; 2001.
- 22. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad pediatr 2013; 13 (suppl 6): S38–S44. pmid:24268083
- 23. Giacomin J. What is human centred design? The Design Journal 2014; 17 (4): 606–623.
- 24.
Population Services International [PSI]. A360 Monthly Snapshot. May 2020.
- 25.
Krug C, Bottomley C, Hargreaves J, Doyle A, Neuman M. Outcome Evaluation of Adolescents 360 in Nigeria. Itad; 2021. Available from: https://www.itad.com/knowledge-product/a360-outcome-evaluation-summary-report-on-the-endline-survey/.
- 26. Burchett HED, Griffin S, de Melo M, Picardo JJ, Kneale D, French RS. Structural Interventions to Enable Adolescent Contraceptive Use in LMICs: A Mid-Range Theory to Support Intervention Development and Evaluation. Int J Environ Res Public Health 2022; 19 (suppl 21). pmid:36361287
- 27. Williamson LM, Parkes A, Wight D, Petticrew M, Hart GJ. Limits to modern contraceptive use among young women in developing countries: a systematic review of qualitative research. Reprod health 2009; 6 (suppl 1): 3. pmid:19228420
- 28.
The International Planned Parenthood Federation [IPPF]. A multi-country study on legal barriers to young peoples access to sexual and reproductive health services. London: IPPF; 2014 [cited 2022 July 1]. Available from: https://www.childrenslegalcentre.com/userfiles/file/ippf_co-ram_el_salvador_report_eng_web.pdf.
- 29. Sidze E, Lardoux S, Speizer I, Faye C, Mutua M, Badji F. Young women’s access to and use of contraceptives: the role of providers’ restrictions in urban Senegal. Int Perspect Sex Reprod Health 2014; 40: 176–84. pmid:25565345
- 30. Villarruel AM, Zhou Y, Gallegos EC, Ronis DL. Examining long-term effects of Cuídate-a sexual risk reduction program in Mexican youth. Rev Panam Salud Publica 2010; 27: 345–51.
- 31.
Tanzania’s President Magufuli calls for end to birth control. BBC News. 2018 [cited 2022 July 1]. Available from: https://www.bbc.com/news/world-africa-45474408.
- 32.
Adebayo B, Odutayo D. Amnesty International condemns Tanzania’s ’attack’ on family planning. CNN. 2018 [cited 2022 July 1]. Available from: https://edition.cnn.com/2018/09/25/africa/tanzania-suspends-family-planning-advert-intl/index.html.
- 33.
PMA2020. 2020a. PMA Nigeria (Kano) Results from COVID-19 phone survey June 2020 [cited 2022 June 14]. Available from: https://www.pmadata.org/sites/default/files/data_product_results/Nigeria%20Kano%20Phase%201-FU_Results%20Brief_Final.pdf.
- 34.
PMA2020. 2020b. PMA Nigeria (Lagos) Results from COVID-19 phone survey June 2020 [cited 2022 June 14]. Available from: https://www.pmadata.org/sites/default/files/data_product_results/Nigeria%20Lagos%20Phase%201-FU_Results%20Brief_Final.pdf.
- 35.
Krubiner C, O’Donnell M, Kaufman J, Bourgault S. Addressing the COVID-19 Crisis’s Indirect Health Impacts for Women and Girls [Online]. Washington, DC: Center for Global Development. 2021 [cited 2021 June 14]. Available from: https://www.cgdev.org/sites/default/files/Addressing-Covid-19-indirect-health-impacts-women-and-girls_0.pdf.
- 36.
PMA2020. 2021. PMA Nigeria (Lagos) COVID-19 results from recent surveys December 2020—February 2021 [cited 2021 June 14]. Available from: https://www.pmadata.org/sites/default/files/data_product_results/Nigeria%20LAGOS_Phase%202_COVID_Results%20brief_Final.pdf.
- 37. Belay L, Hurisa T, Abas F, Daba M, Abebe B, Nigatu B, et al. Effect of Covid-19 Pandemic On Safe Abortion And Contraceptive Services And Mitigation Measures: A Case Study From A Tertiary Facility In Ethiopia. Ethiop J Health 2020; 12: 6–6.
- 38. Kassie A, Wale A, Yismaw W. Impact Of Coronavirus Diseases-2019 (Covid-19) On Utilization And Outcome Of Reproductive, Maternal, And Newborn Health Services At Governmental Health Facilities In South West Ethiopia, 2020: Comparative Cross-Sectional Study. Int J Womens Health 2021; 13: 479. pmid:34040456
- 39. Assefa N, Sié A, Wang D, Korte ML, Hemler EC, Abdullahi YY, et al. Reported Barriers To Healthcare Access And Service Disruptions Caused By Covid-19 In Burkina Faso, Ethiopia, And Nigeria: A Telephone Survey. Am J Trop Med Hyg 2021; 1. pmid:34161296
- 40.
Krug C, Bottomley C, Hargreaves J, Doyle A, Neuman M. Outcome Evaluation of Adolescents 360 in Ethiopia. Itad; 2021 [cited 2022 Dec 1]. Available from: https://www.itad.com/wp-content/uploads/2022/11/A360-OE-Ethiopia-Endline-Report-FINAL-FOR-WEBSITE-.pdf.
- 41. Doyle AM, Mulhern E, Rosen J, Appleford G, Atchison C, Bottomley C, et al. Challenges and opportunities in evaluating programmes incorporating human-centred design: lessons learnt from the evaluation of Adolescents 360. Gates Open Res 2019; 3. pmid:31363715
- 42.
World Health Organisation [WHO]. Macroeconomics and health: investing in health for economic development: executive summary/report of the Commission on Macroeconomics and Health. 2001 [cited 2022 Aug 1]. Available from: https://www.who.int/publications/i/item/924154550X.
- 43. Zakiyah N, van Asselt AD, Roijmans F, Postma MJ. Economic evaluation of family planning interventions in low and middle income countries; a systematic review. PloS one 2016; 11 (suppl 12).
- 44. Wilson M, Cutherell M, Musau A, Malakoff S, Coppola A, Ayenekulu M, et al. Implementing adaptive youth-centered adolescent sexual reproductive health programming: learning from the Adolescents 360 project in Tanzania, Ethiopia, and Nigeria (2016–2020). Gates Open Res 2022. pmid:35614963