Study design, setting and participants

The CHAMPION2/STRIPES2 (Community Health Promotion And Medical Provision and Impact On Neonates, and Support To Rural India’s Public Education System and impact on numeracy and literacy scores) trial was a cluster-randomized trial conducted in rural villages in the Satna and Maihar districts of Madhya Pradesh. At the time the trial was conducted the Maihar district was part of the Satna district. Full details of the STRIPES2 trial design and planned statistical analysis can be found in the CHAMPION2/STRIPES2 protocol paper [20], protocol update [21] and statistical analysis plan [22]. In brief, 196 villages were randomized, with 98 receiving a package of interventions to improve literacy and numeracy (STRIPES2) and 98 (CHAMPION2) receiving a package of maternal and newborn interventions. The villages receiving the CHAMPION2 intervention acted as controls for the STRIPES2 trial and vice-versa. The reason this design was adopted was (as with the original STRIPES and CHAMPION trials) to create a positive presence in all the villages in both trial arms, thereby encouraging STRIPES2 control children to give us data and attend the midline and endline tests even though they were not receiving any educational intervention. The CHAMPION2 intervention was primarily aimed at improving outcomes in pregnancy, whereas the STRIPES2 intervention aimed to improve educational standards in primary school-age children. We accordingly anticipated that the CHAMPION2 intervention would not materially impact on outcomes in STRIPES2 and vice-versa.

The trial was conducted in Satna district, Madhya Pradesh, India. Satna district is further divided into 10 tehsils (sub-districts). Three tehsils (Birsinghpur, Majhgawan and Raghurajnagar) were excluded due to difficult access (forest area), risk of violent robbery, and being an urban sub-district. The remaining seven tehsils comprised 1263 villages (68% of all villages in Satna), with a population of 1,441,930 [23]. From these we selected the 484 villages that (i) were considered ‘rural’ with a population less than 2500 inhabitants (ii) had at least 120 children under the age of 6 years and (iii) were accessible by road. An algorithm was written to select a maximally sized subset of these villages such that (iv) each village center was at least 5 km away from the nearest Community Health Centre (CHC) and (v) there was a minimum of 3 km between village centers to avoid contamination (buffer zones). The requirement that villages should have at least 120 children under the age of 6 was included because this gave an expectation of 20 children in each academic year and therefore a good chance of there being at least 15 children eligible for the intervention. In the event that (vi) fewer than 15 eligible children were identified at enumeration (or that villages were considered to be urban rather than rural) villages were not randomized.

In STRIPES2, the target group was children born between 16 June 2010 and 15 June 2013, whose caregivers were planning to enroll them in Grade 1 for the first time in the 2018–2019 school year, and resident in the trial villages. Because of delays in getting the trial started, enumeration took place twice (July 2017 and April 2019), so eligible children who were missed the first time had a second chance to participate in the trial.

All data collection and related research activities for the STRIPES2 trial were organized by independent teams recruited, trained and monitored by GH Training and Consultancy (GHTC), India.

This trial employed multiple tiers of consent: village and individual level consent from the caregiver on behalf of the child. Agreement to approach eligible villages was first obtained from the village heads (Sarpanchs). In the trial villages, consent was obtained from the village after the trial had been presented in a meeting with village elders representing all the castes and village residents. Consent was given orally by the elders during a village meeting, and the Sarpanchs of the villages signed (or gave their thumbprint) in a document that described the trial. During the enumeration survey, GHTC interviewers read a simple standard consent explanation of the trial in Hindi (local language) and caregivers of eligible children were offered the chance to participate, giving signed or thumbprint consent if they wished to. This trial’s protocol [20] and its amendment [21] were approved by the Ethics Committees of L V PRASAD Eye Institute, Hyderabad, India (LEC 02–16−008) and London School of Hygiene and Tropical Medicine (LSHTM Ethics Ref: 10482). We also obtained the approvals from the Indian Council of Medical Research, New Delhi and the Government of Madhya Pradesh to conduct this trial in Satna district. The trial complies with the Declaration of Helsinki, local laws, and the International Conference on Harmonization Good Clinical Practice (ICH-GCP).

Intervention design

The STRIPES2 intervention aimed at building foundational literacy and numeracy skills for Grade 1 and 2 children in a holistic manner, combining multiple interventions to promote learning outside and inside the child’s home. Caregivers (especially mothers) had a crucial role in engaging with their children’s learning. Also, its content extended beyond academic learning, as it included activities to develop children’s social and emotional skills. The approach involved a combination of individual, small group, and large group activities, incorporating play-based teaching methods that included: storytelling, reading, playing, singing, coloring, drawing, and role-playing. This approach aims to help make learning enjoyable and engage children in a manner that aligns with their natural instincts.

The STRIPES2 intervention design was based on Pratham’s experience across multiple states in India, and especially in Madhya Pradesh [11]. Pratham has been working in Madhya Pradesh with Grade 1 and 2 children since 2005, indirectly supporting projects that involved training and fostering community volunteers to interact with local schools’ teachers and mothers to engage in children’s academic, emotional and physical learning through the development of workbooks for Grades 1 and 2. However, this was the first time that Pratham was working directly in the trial area (Satna and Maihar districts).

Over the years, Pratham has worked closely with the state government to provide primary school teachers with innovative teaching methods and curricula, aiming to enhance children’s proficiency in core subjects. With a focus on Grades 1 and 2, programs in partnership with the government have addressed essential competencies in language and mathematics. Teacher training, utilizing Pratham’s “teaching-learning” and “teaching-at-the-right level” methods, have been central to these initiatives, overseen by both government and Pratham personnel. In 2013, Pratham partnered with the Madhya Pradesh government in selecting, training and sometimes hiring education coordinators to provide on-site support to teachers, enhance instructional materials and practices, ensure effective curriculum implementation, address challenges in the classroom (such as diversity in learning levels etc.) and improve overall quality in primary education. The design of the teaching component of the STRIPES2 intervention was based on Pratham’s innovative [18,24] activity-based content and curriculum package developed for Grades 1–2 that focuses on improving foundational literacy and numeracy (FLN) whilst fostering other developmental areas such as physical, social, and emotional growth. It follows a phased approach, starting with a warm-up phase to prepare children from diverse backgrounds (see Manual for Instructors/Teachers: Warm Up – Phase I Language & Math Class 1 & 2, in S1 Appendix), followed by instructional phases that strengthen literacy and numeracy skills [18,24].

The key components of Pratham’s method are:

• Simple Testing Tools: One-on-one oral tools and worksheets are used to track progress, with assessments conducted at baseline and end-line stages.
• Daily free play (10–15 minutes), group socio-emotional activities like storytelling and role-playing, and physical tasks such as running and jumping (see examples of games and activities in the Manmauji Ganit Booklet – S2 Appendix).
• Activity-Based Instruction: Activities adapted to individual learning levels and encourage gradual improvement in reading and arithmetic.
• Contextual and Low-Cost Materials: Locally relevant materials are used, including stories, flashcards, and print-rich environments to support language development. Context-specific materials and culturally adapted rhymes are included as optional elements. To ensure effective implementation, teachers are trained with hands-on practice, supported by activity booklets and a structured two-hour lesson plan framework.
• Focus Domains: Emphasis is placed on language (oracy, phonological awareness, vocabulary, fluency, comprehension, writing) and mathematics (pre-mathematics concepts, numbers and operations, measurement, geometry, data handling)

The para-educators, called Pratham instructors (PIs), were female residents in the intervention villages. All had completed 12^th Grade and passed a written exam during the selection process. By hiring female teachers, Pratham also intended to empower local women. PIs received a 6-day training by Trainers (Manual for Master Trainers – Grades 1 and 2 in S3 Appendix)

PIs were responsible for conducting classes either before or after school hours, maintaining contact with families, and engaging mothers in the learning process. They were overseen by a team of full-time cluster leaders (CLs), each CL being responsible for approximately 10–12 villages. CLs provided continuous monitoring, training, and mentoring support to PIs. Additionally, the CLs played a crucial role in facilitating the engagement of PIs with the local community, assisting them in talking to parents about the importance of these before/after school classes and finding adequate places in the village to hold the classes.

Before the launch of the program, Pratham teams piloted the activities and materials for these before/after school classes in rural villages of Bhopal district, Madhya Pradesh. In November 2019, Pratham started the 3-month preparatory phase for students’ readiness. The initial 2–3 weeks period was a warm-up phase and focused on nurturing attendance (see S1 Appendix). During that stage PIs worked on mobilizing families to send their children to the classes, motivating children to come on time to classes and getting them used to sitting for long activities. PIs carried out a series of activities such as playing, singing, coloring, and drawing, to make children feel at ease within the school and class so that they were able to interact comfortably and independently with the instructors and their peers (see Activity Book Manmauji Ganit – S2 Appendix).

After the warm-up phase, the STRIPES2 teaching-learning approach was implemented in three phases: readiness (phase I) and instructional (phases II and III). The readiness phase (for 4 weeks) included simple, interesting, and well-designed activities focusing on a preparatory stage, aiming to familiarize children with the classroom environment, develop emergent literacy and numeracy skills, and promote overall growth in various developmental areas. The emphasis on diverse activities and the teacher’s active role in understanding individual needs contribute to a comprehensive approach to early childhood education. Each of the instructional phases II and III were implemented for about 2–3 months and aimed to strengthen children’s literacy (Hindi) and numeracy learning.

In addition to teaching, the PIs were actively involved in maintaining frequent communication with caregivers through door-to-door interactions (home visits) and fortnightly meetings with caregivers (caregivers’ meetings). During these home visits and caregivers’ meetings, the PIs engaged with caregivers to discuss children’s learning progress, offer suggestions for fun activities that could support the learning process and investigate whether caregivers were conducting these activities (reading and mathematics), and check whether the activities were appropriate to the child’s learning level.

In March 2020, about 4 months after the intervention had started in the communities, the government of India declared a nationwide lockdown due to the COVID-19 pandemic. As a result, the intervention activities were first suspended and then resumed with adaptations. Between August and November 2020, the program had to be limited to calls and text messages by the PIs. The initial text messages were focused on health precautions and then shifted to focus on basic language and mathematics. These activities were adapted from Pratham’s national campaign the “Karona:thodi Masti, Thodi Padhai” (Do it: A little fun, a little study) [25]. It is important to highlight that these activities were not intended as formal teaching but rather as a means of promoting caregivers’ engagement and sparking children’s interest. In contrast, the instructor-led model that was adopted as soon as COVID-restrictions were lifted involved focused, structured learning for two hours daily [25]. Moreover, PIs also conducted daily follow-up calls to ensure the seamless delivery of the content. These calls played a crucial role in building a rapport with the parents and ensuring their active participation in the activities. Despite the changes made during the pandemic, the education program remained mainly instructor-led, not caregiver-led.

During the lockdown period, the relief schemes (like distribution of activity books and text messaging) provided by the government were designed and implemented uniformly, ensuring similar access and support in both trial arms.

As COVID-19 restrictions eased (November 2020), the subsequent phase involved setting up small group interactions with 2–5 children (small group classes) (Fig 1) to understand learning levels of children after a prolonged break and prepare them to resume classroom activities. In parallel, PIs restarted home visits to actively engage with children and their caregivers/family members (both individually and in small groups), during which they talked with caregivers/family members and children about queries or concerns. The frequency of these visits and group activities was related to the availability and willingness of parents and the village head, as well as the number of children that each PI was responsible for. In parallel with these activities, Pratham teams conducted six ASER-like assessments throughout the trial period to keep track of how children’s learning was improving. These are represented in Fig 1 as red stars (December 2019, November 2020, April and August 2021, February and June 2022).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Timeline of the STRIPES2 trial activities in Madhya Pradesh, India.

https://doi.org/10.1371/journal.pone.0330203.g001

Around July 2021, the intervention team, with the help of the local community, identified suitable class locations that allowed for social distancing, so that PIs could restart teaching the before/after school classes. Additionally, PIs were trained to discuss the risks associated with COVID-19, ways to reduce its transmission and sensitize caregivers to avoid sending their children to classes if anyone at home had symptoms.

From January 2022 onwards, a group of 163 young volunteers started conducting 20- to 40-minute-long fun activities with groups of 4–5 children on a regular basis (volunteer classes). The 163 volunteers engaged in this initiative were primarily young females from the community who had demonstrated a keen interest in working with children on a consistent basis. They had foundational skills in reading and mathematics, enabling them to facilitate learning activities with the children. To mobilize these potential volunteers, a month-long collaborative effort was undertaken by PIs and Community Leaders (CLs). This initiative involved various strategies, including door-to-door meetings and large group gatherings within villages, facilitated with the support of the local Sarpanch (village head) and other key community stakeholders.

A month later, the same volunteers were trained in performing (and creating materials for) storytelling (for instance: drawing, painting, and cutting cardboards to create characters and scenarios; and ways of dramatizing when telling a story). In March 2022, informal libraries were created in the local shops to provide a platform where children’s books could be easily borrowed, encouraging interest in books, and providing material for children to practice reading.

In April 2022, volunteers began using tablet games to reinforce and support learning through fun activities in sixty-five of the intervention arm villages. These sessions, conducted daily for 25–30 minutes with groups of 4–5 children, were offered to all enrolled children but targeted especially at children who were struggling to recognize and interpret the sounds associated with syllables (which is essential for reading and writing) and number recognition.

Table 1 presents a list of the components of Pratham’s intervention, indicating whether each was considered core or optional. Although each component could not be implemented simultaneously in all villages, Pratham teams followed a consistent process of implementation for each of them except the formal libraries.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Core and optional components of Pratham’s intervention.

https://doi.org/10.1371/journal.pone.0330203.t001

Outcomes

The primary outcome was the arithmetic mean of the child’s scores on EGRA (Early Grade Reading Assessment) and EGMA (Early Grade Mathematics Assessment) tests [26,27]. We chose to use EGRA and EGMA because of our positive experiences applying them in similar contexts (as they are sensitive in measuring small differences in ability among children who have very low levels of learning), and because they are widely used tests for assessing early grade literacy and numeracy.

Secondary outcomes included separate scores for literacy and numeracy; caregivers’ engagement in their child’s learning; enrolment in school; caregiver’s report of school attendance; and the cost effectiveness of the intervention. Owing to the interruptions caused by the COVID-19 pandemic, the STRIPES2 intervention finished later than had initially been planned, meaning that the intervention (and the final testing from July-September 2022) was carried out in older children than would otherwise have been the case.

Sample size and randomization

The sample size calculation was primarily driven by the CHAMPION2 (health) intervention (details published in the protocol and update) [20,21]. Originally, the intention was to randomize 300 villages, which would give over 90% statistical power to detect a difference of 0.25 standard deviations in mean standardized test scores for STRIPES2. In the first STRIPES trial (Telangana), [8] the estimated effect was a 0.75 standard deviation (SD) increase in mean score: however, an effect of smaller magnitude than this would still be important to detect.

After drawing the buffer zones, it turned out that only 204 villages could be selected. These 204 villages had a mean population of 1487 (minimum 558, maximum 2490) and a standard deviation of 505 (equating to a coefficient of variation of 0.34). Estimating the number of children in each school year from the number under the age of six years old, the mean number of children in each school year was 38.3 (minimum 20, maximum 71) with a standard deviation of 13.3 (a coefficient of variation of 0.35). Assuming that 25% of the children would not satisfy the eligibility criteria, the mean number of eligible children per village was estimated as 28.7 with a minimum of 15. Further assuming that i) 95% of the 204 villages would ultimately participate, ii) 60% of the eligible children in these villages would take the test at the end of the trial, iii) an intra-cluster correlation coefficient of 0.23 (as seen in the STRIPES trial [8]) and iv) a coefficient of variation in numbers taking the test by village of 0.35 gave the trial 88% power to detect a difference of 0.25 SD in mean standardized scores between intervention and control villages using a conventional 2-sided statistical significance level of 5%. In fact, 196 of the 204 villages were randomized (6 villages being removed because they were found to be too close to urban areas to be considered rural, and 2 being removed because there were insufficient eligible children).

All children were enumerated and enrolled before randomization. Randomization of clusters was performed by the trial statistician based in London in June 2019 using a random number generator, with stratification by village size (above or below median) and distance to the nearest Community Health Centre or Civil Hospital (above or below median).

Adherence

We measured adherence through children’s attendance at before/after school classes recorded by PIs, the participation of caregivers in fortnightly meetings and their compliance with reading and mathematics exercises (that were given by PIs for caregivers to practice with their children).

Before/after-school classes were not run in exactly the same way in all of the villages. Partly this was due to COVID-19 concerns and restrictions and the difficulty some children had to reach the place where these classes were run. As a result, there was variability in the number of planned classes per week, the length of these and the size of classes.

For simplicity, we decided to use counts of the numbers of classes i) offered to and ii) attended by each child. We assumed that, had the intervention run as planned, then each child would have been offered 360 classes (6 classes a week for 60 weeks, this corresponding approximately to a 16-month period with allowance for holidays etc.). We refer to this as the ideal number of classes. We calculated the total number of classes that were offered to each child and the total number of classes that each child attended. Where more than 360 classes were offered or attended, we took the number(s) to be 360. We defined adherence at child level as the proportion attended of ideal, proportion offered of ideal, and proportion attended of offered. We also did this at village level by calculating mean proportions within each village.

As well as recording attendance in classes, PIs recorded the attendance of caregivers at caregivers’ group meetings and asked whether they had carried out the language and mathematics activities/exercises with their children. These meetings occurred every two weeks so that caregivers could share experiences and talk about doubts related to their children’s learning. Attendance at these meetings was summarized in an analogous way to attendance in classes with ideal attendance being 24 classes. If the caregiver did not come to the meeting it was assumed that she had not carried out the activities with the child.

Baseline data collection

Prior to randomization, data were collected on children’s gender, age, whether or not parents were alive and who their primary female and male caregivers were (mother, father, grandmother, grandfather etc.). In addition, data were collected on the caregivers’ religion, caste, literacy and education levels.

Midline test and survey

GHTC teams conducted the midline tests with trial children about six months after PI classes restarted and a month after schools reopened. These midline ASER-like tests were developed by an expert in educational evaluations using the structure and rules of the main ASER reading and mathematics tools adapted to the context of rural Madhya Pradesh (S4 Appendix). They were conducted in both trial arms between December 2021 and January 2022 with the aim of understanding the learning levels of children soon after the pandemic-long break.

To ensure efficient and timely testing, this midline test and a midline survey were conducted in two separate periods. This minimized the risk of children gaining access to the tools before being tested.

The midline survey was conducted between February and April 2022 with the main caregivers. The aim of doing this survey was to monitor residence and school enrolment of participant children (both before and after the lockdown – academic years of 2019−20 and 2020−21) and gather some information about expenses related to schooling and educational challenges faced by and support given to children a few months after primary schools reopened in Madhya Pradesh.

Data were also collected at the midline survey (or at the endline survey if missed at midline) on the materials that houses were constructed from (whether the materials used for the floor, roof and walls were synthetic or natural) and on whether household members owned a television, radio, motorbike and/or a 4-wheeled vehicle. These variables were used to construct wealth indices.

Endline tests and survey

The EGRA and EGMA protocols were adapted to the local context by the National Foundation for Educational Research (NFER, [28]) and a board of local language and curriculum experts and primary teachers. Three rounds of workshops were organized to adapt and evaluate the subtasks and the individual items in the test in terms of relevance, accordance to curriculum, and suitability to the local context and language. The validation of the Hindi test instruments included expert review and qualitative trials with a small number of children. The validated EGRA and EGMA protocols were uploaded onto tablets (using the Tangerine platform designed for EGRA and EGMA- https://www.tangerinecentral.org/tangerine to facilitate tests’ application), which provided an opportunity for immediate online uploads of the test scores and monitoring of the data (the English versions of the full test papers are given in S5 Appendix). A quantitative pilot of the assessments was conducted with children from nearby non-trial villages to further trial the test administration process, investigate possible order effects, and inform the assessment design. NFER conducted psychometric analysis (Rasch Model) with the pilot data and the result informed the final revision of the EGRA/EGMA protocols. To minimize the risk of children sharing the test content after they had taken the test, we conducted the test in each trial village over the course of a single day.

GHTC recruited an independent team of test administrators for the Endline test. They were otherwise not involved in the trial and were unaware of the randomization. The tests were administered orally and sequentially by two groups of test administrators (one group each for EGRA and EGMA). Tests were administered in one-on-one sessions to each participant child present in the village on the day of the assessment. High attendance at the tests was ensured by the village level mobilization team (see S6 Appendix for a more detailed description of the process of data collection). Tests were conducted about one month after the intervention teams had stopped the main activities (between 24 July and 19 September 2022).

The NFER team coordinated the process of adaptation, translation, and initial monitoring of the assessment administration.

The endline survey was a brief survey conducted by GHTC between November 2022 and January 2023 with the main caregivers to assess the enrolment of participant children in school (2022–2023 academic year), their reported attendance in school during the two weeks prior to the interview, and caregivers’ engagement with children in reading and mathematics activities at home (estimated hours per week).

Statistical analysis

The main analysis was conducted according to the intention-to-treat principle. All enumerated children satisfying the eligibility criteria were included in the primary analysis. The primary outcome of the trial was the composite literacy and numeracy test score. Each test score was calculated as a simple arithmetic mean of the percentage of correct answers on each of the subtasks, evenly weighting each task and not accounting for time remaining (for subtasks composed of two parts (a and b), we used a mean of the two results): for a detailed description of the statistical analysis see the Statistical Analysis Plan [22]. Summary statistics were reported at both child level (in the main paper) and at cluster level (in S7 Appendix).

In the primary analysis the composite language and mathematics test scores at follow-up were compared using a linear regression model with the clustered sandwich estimator of variance (allowing for clustering at the level of the village). Covariates were randomization arm and the randomization stratification variables. The primary analyses were conducted using scores calculated on a percentage scale. We also present the comparison as a standardized effect to allow comparison with other studies. No external standard deviation (SD) was available, so the standardized effect was estimated by fitting a linear mixed model with the same covariates as above to the scores on the percentage scale. The linear mixed model included cluster-specific random effects, with the variance of these random effects and the residual variance allowed to differ between randomization arms. The standardized effect was estimated by dividing the estimated adjusted difference in scores by the estimated total SD in the control arm (this being the square root of the sum of the between- and within-cluster variances in the control arm). A nonparametric bootstrap confidence interval (bias corrected and accelerated, 2000 replications at cluster level, stratified by randomization arm) was computed for this standardized effect.

We used similar linear regression models to the main analysis to explore the effect of interactions between baseline factors and the intervention on the primary outcome. These factors were village population size, gender, two wealth indices, caste, female caregiver literacy and male caregiver education (the male caregiver was often not present to read the sentence, meaning many cases were missing). The data used to compute the two wealth indices were collected post-randomization, but it is implausible that the intervention will have impacted on these variables. In addition to these pre-specified analyses, we carried out a post-hoc interaction analysis investigating whether the effect on the primary analysis differed according to whether or not there was a pregnancy in the household as recorded in the CHAMPION2 trial, this being done to assess whether household involvement in the CHAMPION2 trial might have impacted on the STRIPES2 primary outcome. Were the CHAMPION2 intervention to have materially impacted on STRIPES2 outcomes it is likely that the most marked effects would be seen in children from households where there was a pregnancy.

For the primary outcome we also conducted a pre-specified per-protocol analysis comparing those with high adherence to all controls. High adherence was defined as attending more than 75% of the ideal number of before/after-school classes. We also utilized an instrumental variables approach to explore how the primary outcome varied with attendance as a percentage of ideal in a post-hoc analysis. We adopted the two-stage structural mean model (SMM(G)) approach described by Maracy and Dunn (2011) [29], assuming a linear relationship between attendance and outcome, stratifying by the randomization stratifiers and computing bootstrap 95% confidence intervals (bias corrected and accelerated, 2000 replications at cluster level, stratified by randomization arm) for the whole process.

We estimated the effects of the intervention on reading and mathematics test scores at midline, learning support (hours spent engaging child in reading or writing activities) and number of school days missed in the previous two weeks using analogous linear regression models to those used in the main analysis of the primary outcome. The intervention effect on whether the child was enrolled in school was expressed as an odds ratio with a 95% confidence interval obtained from a GEE model with a binary outcome, a logit link, and a ‘working’ assumption of independence, with robust standard errors to take account of clustering.

In a secondary analysis of the primary outcome, we addressed missing data using multiple imputation by chained equations. We included as auxiliary variables the randomization stratification factors, caste, gender, male and female primary caregiver literacy, the wealth indices, the adherence to intervention variables defined above, the midline test scores, school enrolment at endline, number of school days missed in the two weeks before endline interview, number of hours the caregiver spent engaging child in reading or writing activities post lockdown, caregiver’s report of school attendance, whether or not the child was enrolled in school pre and post the COVID-19 lockdown, school grade at endline, the child’s residence status and the variables quantifying the learning support (and spending) provided by family, school teachers, NGOs and/or private tutors during the time when schools were closed. We performed multiple imputation using the “jomo” package in R [30], which is able to impute clustered data. We carried out imputations separately by trial arm; we used 20 imputations.

In a further secondary analysis, we considered the composite score and the reading test score with one problematic item omitted. Soon after the tests started being administered, the NFER consultant (unaware of randomization status) noticed an issue with a reading comprehension task (first item in subtask 5b), as fewer children than expected were getting that item correct and the result was somewhat inconsistent with individual performances in the rest of the test. The question asked the child what the day/weather was like, after the child had read a related text about it being a windy day. This could have happened because in some parts of India, wind is associated with good weather (because it makes the usually hot days more pleasant), hence quite a number of children answered “good, nice” and sometimes “cool”, or because the question asking about the state of the weather was not accurately translated. Given that this issue was not flagged during the validation or the training, we decided to carry on conducting the tests as they were and conduct a sensitivity analysis omitting the score from EGRA subtask 5b question 1, which was judged to be potentially misleading.

We used a significance level of 0.05 and report 95% confidence intervals. For the interaction tests, claims of different effects in subgroups were only made if there was strong evidence (p < 0.01) of an interaction. We conducted all analyses apart from the multiple imputation in Stata 18 [31]. Full details of analyses are described in the statistical analysis plan [22].

Cost effectiveness

Cost effectiveness was calculated using actual budget expenditures. In 2020 and 2021, the before/after school classes had to be suspended for 10 and a half months because of COVID-19 restrictions. Although salaries continued to be paid normally for all the staff working for the program, we present the results of cost effectiveness per child both including and excluding the expenses during the period that these classes were not being conducted due to the COVID-19 pandemic.

For cost effectiveness, we assumed the total number of children impacted by the program to be the numbers taking the endline test.

Annual costs were converted to 2021 prices using the Indian GDP deflator, and then adjusted to USD using the average annual Indian Rupees (INR) to US dollars exchange rate.

Migration and mortality

The participant children’s residence status was verified at midline (March 2022) and endline (December 2022). A high percentage (89.6% in the intervention arm, 89.0% in the control arm) of the primary caregivers completed in the midline survey interviews, with the percentage being slightly higher at the endline survey (92.9% in the intervention arm, 91.6% in the control arm). Table 4 shows the percentage of children who were resident in the village across intervention and control groups in both surveys done during the trial. In both intervention and control arms, very few children were not resident in their village at each of the surveys. In total, 40 children died during follow-up (16 in the intervention arm and 24 in the control arm).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Children resident in the study villages at midline and endline.

https://doi.org/10.1371/journal.pone.0330203.t004

Adherence and attrition

Adherence measurement results can be found in Table 5. Adherence varied markedly between children. In some intervention arm villages, classes ran for up to 23 months (October 2019 to March 2020, January 2021 to April 2021 and July 2021 to July 2022 (inclusive)), longer than the initially planned ideal length of 17 months. However, there were significant disruptions due to the COVID-19 pandemic in many villages, impacting the continuity of classes even during these periods, leading to interruptions and potential gaps in learning for the children. On average 77% of the planned ideal number of classes were offered to children, with the average number of classes attended being 53% of that regarded as ideal. Only 37% of children attended more than 75% of the number of the classes that were regarded as ideal before the trial started. There were no substantial differences in patterns of attendance according to baseline characteristics (Table S4A in S7 Appendix).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Adherence: Before/after-school classes attended in the intervention arm.

https://doi.org/10.1371/journal.pone.0330203.t005

In the intervention arm 3054 out of 3405 children (89.7%) contributed the primary outcome as did 3275 out of 3667 (89.3%) children in the control arm.

Table 6 shows attendance at the caregiver’s meetings. 75% of caregivers attended at least one class, but only 32% attended more than 75% of the number regarded as ideal. Amongst those who attended, only a few reported that they had not done the language and mathematics activities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Attendance at the intervention arm caregivers’ group meetings.

https://doi.org/10.1371/journal.pone.0330203.t006

Primary outcome

The primary outcome from the endline test was obtained in 3054 out of 3405 children (89.7%) in the intervention arm and 3275 out of 3667 (89.3%) in the control arm. The primary outcome (composite score) and total scores for the reading (EGRA) and mathematics (EGMA) tests are displayed by randomization arm in Fig 3 with summary statistics shown in Table 7. On average, intervention children scored 14.17 percentage points higher than control children on the tests (95% CI 11.36, 16.97; p < 0.001), adjusting for the randomization stratification factors and including all children who did the test (even those who did not attend classes or attended very few of them). The standardized intention-to-treat estimate was 0.580 (95% CI 0.466 to 0.706). In the same table, we present the results for the total mathematics and reading scores separately. The differences between intervention and control children’s overall mathematics and reading test scores were similar to their difference in the primary outcome and both were highly statistically significant: 13.90 (95% CI 10.91 to 16.88) and 14.44 (95% CI 11.54 to 17.34) percentage points respectively, both p < 0.001. Results without the problematic item 5b were almost identical (Table S5 in S7 Appendix).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. EGRA and EGMA composite and separate total scores.

https://doi.org/10.1371/journal.pone.0330203.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Distribution of child endline test scores by intervention arm.

https://doi.org/10.1371/journal.pone.0330203.g003

Baseline characteristics in those children who did and did not take the test and contributed the primary outcome were similar (Table S4B in S7 Appendix) with minor differences being consistent in size and direction across the two trial arms (for example the percentages of children where the biological mother and father were not the primary caregivers were slightly higher when the test result was not taken in both trial arms). In the intervention arm the proportion of children with the primary outcome increased with increasing attendance at the before/after school classes (with 280/434 of those attending no classes contributing the primary outcome compared with 413/425 in those attending 100% of the classes, Tables 5 and 7). Our multiple imputation sensitivity analysis explored the potential for this and other differences between those with and without the primary outcome to introduce bias. With multiple imputation the estimated intervention effect was 14.04 (95% CI 9.17, 18.90; p < 0.001), adjusting for the randomization stratification variables, very similar to the result from the primary complete-case analysis.

Per protocol analysis and dose-response relationship with class attendance

The per-protocol population was defined as the children enumerated in the intervention villages who attended more than 75% of the ideal number of before/after-school classes. There were 1259 (37%) such children in the intervention arm. The mean composite score was 66.19 (SD 19.42) among intervention children who attended more than 75% of before/after-school classes. The per-protocol estimate was 23.77 (95% CI 20.86, 26.68; p < 0.001) percentage points. In a post-hoc analysis we explored how the effect of the intervention varied according to attendance at the classes. The mean composite score in the intervention arm increased with increasing attendance (Table 7) being similar to that in the control arm when no classes were attended and rising to 71.17 when attendance was ideal. Using the structural mean model (SMM(G)) described by Maracy and Dunn [29] it was estimated that each 10 percentage point increase in attendance increased the composite score by 2.63 (95% CI 2.13, 3.16).

EGRA and EGMA subtasks

Table 8 shows scores for each EGRA subtask, with the average percent correct for all subtasks, and the fluency scores for timed subtasks. Intervention children outperformed control children in reading in all subtasks, and the control-intervention difference was at least 10 percentage points (out of 100) for each. Children in the intervention arm demonstrated higher reading skill mastery across subtasks of all difficulty levels.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. EGRA subtasks and fluency scores.

https://doi.org/10.1371/journal.pone.0330203.t008

Analogous results from the EGMA subtask are shown in Table 9. The mean scores for children in the intervention arm are higher for all mathematics tasks, and the differences are broadly similar in magnitude to the ones observed in the reading tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. EGMA subtasks and fluency scores.

https://doi.org/10.1371/journal.pone.0330203.t009

Subgroup analyses

The results of subgroup analyses of the primary outcome by pre-specified variables of interest based on characteristics of the child and village are shown in Table 10. Numbers of observations, means and standard deviations are given by trial arm for each level of the moderators. As per our statistical analysis plan, we considered male caregiver education, rather than male caregiver literacy, as a potential moderator due to the large amount of missing data for the latter variable. The intervention effect was broadly consistent across subgroups, albeit with a suggestion of larger effects in poorer households. In particular, there was statistically significant evidence at the 1% level of a differential impact of the intervention by wealth and female caregiver literacy. Our results suggest that the positive gain in learning was higher among children from poorer households (as per the materials households are made of – wealth index 1). The children whose female caregivers were less literate also had higher gains compared to children of female caregivers who could read the entire sentence. A smaller difference is seen between the intervention effects in boys and girls, where the positive impact of the intervention seems higher in girls, albeit with the p-value being above the threshold (p = 0.01) we used for these interaction tests. There is also a suggestion of a larger difference in smaller villages: the 95% confidence interval here is wide because this is a completely between cluster comparison. In a post-hoc analysis there was no evidence that the intervention effect was different when there was a pregnancy in the household.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Composite test scores by subgroup, with interaction tests.

https://doi.org/10.1371/journal.pone.0330203.t010

School enrolment and attendance

School enrolment is presented in Table 11 School enrolment levels were high for both intervention and control children in the academic years of 2019−20, 2021−22, and 2022−23. Differences between the percentage of enrolment between control and intervention arms were small at all three time points, albeit statistically significant at endline (2022−23), the pre-specified hypothesis testing time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. Children’s enrolment in school.

https://doi.org/10.1371/journal.pone.0330203.t011

At the endline survey, caregivers were asked to report the number of days children had missed in school over the past two weeks (relative to the day of the interview). Attendance reported by the caregivers was similar for both arms (intervention arm mean 2.81 days missed: control arm mean 2.89 days missed: adjusted difference −0.10 days, 95% −0.40, 0.20; p = 0.492, full details in Table S10 in S7 Appendix).

Learning support

There was no evidence (Table 12) that caregivers in the intervention arm gave more time to support their children with learning-related activities, despite efforts from the intervention teams to engage caregivers (primarily mothers) in their children’s learning (through home visits, telephone messages and caregivers’ meetings).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 12. Learning support (endline).

https://doi.org/10.1371/journal.pone.0330203.t012

Midline test and survey results

Table 13 shows the results of the midline tests. In these ASER-like tools, the tasks increase or decrease in complexity depending on whether a child can comfortably solve a given task according to the rules of the test (S4 Appendix). The midline tests’ results show that the differences in basic literacy and numeracy were statistically significantly higher among children in the intervention arm. In mathematics, children in the intervention arm scored significantly higher than those in the control arm. In the most difficult task, subtraction with borrowing, there were 13 percentage points more children in the intervention arm who could solve two operations correctly. The results in the reading test are sizeable, with 39% of the intervention children reading a grade 2 level short story (i.e., fluently and with ease at a good pace and making 3 or fewer mistakes) compared to 23% of the control arm children.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 13. Midline test results (using ASER-like exam).

https://doi.org/10.1371/journal.pone.0330203.t013

Table 14 shows access to technology, learning materials and help at home during the time schools were closed (due to COVID-19) for children in the intervention and control arms separately, whilst Table 15 shows the expenses related to schooling and learning done by caregivers during the first academic year that schools were closed. There were no large differences in caregivers’ support at home for studying. Most caregivers in both arms were helping their children to study at home. There were not many families who bought new devices in either arm. Most children were studying through textbooks and worksheets (many of them provided by the school) and had very limited access to tablets. The percentage of children with access to a smartphone was slightly higher among intervention children (24%) than control children (17%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 14. Support during the COVID-19 schools’ closures (midline).

https://doi.org/10.1371/journal.pone.0330203.t014

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 15. Adult caregiver spending (INRs) on education between July 2020 and June 2021.

https://doi.org/10.1371/journal.pone.0330203.t015

Costs and cost-effectiveness

The primary expenditures of the intervention program along with their corresponding percentages are presented in Table 16. Forty-three percent of expenditures were on teaching staff (PIs) and 20% of spending covered cluster supervisors and manager salaries. Although PI salaries accounted for a large portion of the expenses, the net amount they received per month was 4,362 INR, in line with the government’s salary guidelines for half-time, unskilled labor. This amount is substantially lower than what local government teachers earn. For example, a third-grade teacher receives a minimum salary of 25,000 INR per month plus dearness allowance during their two-year probationary period, with the total nearly doubling after probation. There were minor capital costs (laptops and tablets), and these were fully depreciated over the three years of the trial. Since the program had been previously developed by the implementing partner, Pratham, only minor costs related to the development and piloting of the activities and materials for the before/after school classes were included.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 16. Costs of intervention program by item.

https://doi.org/10.1371/journal.pone.0330203.t016

The main components of Pratham’s education program were highly affected by the disruptions caused by COVID-19, thus, in our primary analysis we calculated the cost per child excluding the ten months and a half period in which the before/after school classes could not be conducted. Excluding the two periods of interruption (mid-March to October 2020 and April to June 2021) and converting all the prices to 2021 prices using the India GDP deflator, the cost per child was 13,631 INR (184 USD) and the cost per 0.1 SD improvement in learning was 2,476 INR (33.5 USD). Equivalently, the intervention yielded a gain of 0.299 per $100 spent.

Using the same price conversion, and accounting for the period PIs could not conduct before/after school classes due to COVID-19 restrictions, the cost per child in 2021 values would be 20,213 INR (279 USD), and the cost per 0.1 SD improvement would be 3744 INR or 50.66 USD (a gain of 0.197 SD per $100 spent).

Cost-effectiveness

In the STRIPES2 trial, cost per child (13,631 INR, 184 USD) and cost per 0.1 SD improvement (2,476 INR or 33.5 USD) were substantially higher than the original STRIPES trial [18], where the cost per child in 2021 values was 4143 INR (56 USD), and the cost per 0.1 SD improvement was 557 INR or 7.54 USD. Equivalently, STRIPES yielded 1.326 SD per $100 spent, while STRIPES2 yielded only 0.29 SD per $100 spent.

This relatively lower cost-effectiveness observed in the STRIPES2 trial on children’s outcomes can be attributed to several factors. The higher costs primarily reflected substantially higher labor costs, as well as the additional costs added due to an extra effort by the Pratham teams to compensate for the learning losses during the time schools were closed when the contact with children and parents was limited to mobile text messages. These additional costs reflect both a greater outlay of raw resources in the STRIPES2 intervention, and a greater expense of operating in India (holding resource levels constant) today as compared to then, given India’s rapid development and steady inflation over the 13 years that have elapsed since the end of the original study.

The STRIPES2 program design was labor intensive as we needed to hire many Pratham Instructors. This was partly needed because of the trial structure, given that houses in our trial area were spread over a large area and therefore it was challenging to visit all families and ensure that all trial children would come to the before/after-school classes. The overhead costs for management of the trial could be reduced if implemented within the existing educational structures. For that, teachers need to be fully supported by the existing structures, have clear instructions to implement the methods and devote a time slot of the daily class to apply it. As an example of this approach, a randomized evaluation of a teacher incentive program [43] found children in the treatment groups, where teachers received incentives calibrated to 3% of salaries, scored 0.17 and 0.27 SD higher on language and math scores compared with controls respectively. For a teacher earning a salary of 600,000 INR per year, and teaching 20 students, these incentives would cost 1800 INR per student over two years, markedly less expensive than the costs per student reported for STRIPES 2. Another option is to train volunteers to lead the activities in the schools. These two models have proven to be the most effective when scaling up the “Teaching at the Right Level” methods [11]

The STRIPES2 trial was also costlier than a similar project implemented in Gambia (pre-COVID-19). Although the average cost per child (adjusted to endline test takers) in the SCORE trial in Gambia [24] was high, the improvement in test scores was greater, and the cost per 0.1 SD improvement was 25.59 USD, (i.e., 0.389 SD per 100 USD). In rural Gambia, learning levels and learning trajectories are much lower and flatter, respectively, than in India, allowing more room for improvement there. Therefore, the cost-effectiveness will vary also depending on the existing learning levels.