Timeline.

In MP and UP, baseline surveys were administered between February-April 2006. The first round of intervention was carried out from September 2006 to January 2007, and the first follow-up surveys administered between February and April 2007. A year after the baseline survey, focus group discussions were held in select intervention GPs. A second round of intervention was from September 2008 to January 2009, with second follow-up surveys between February and April 2009. In Karnataka, baseline surveys were administered between July and August 2006. The first round of intervention was from February to May 2007, and the first follow-up surveys between July and August 2007. The second round of intervention was from February until May 2009, and second follow-up surveys administered between July and August 2009. Karnataka had a shifted timeline due to difference in school year timing.

Baseline survey.

In each GP, one school was randomly selected from all public schools that had grades 1 to 5. All grades 1–5 teachers were in the sample. In MP and UP, 15 students were randomly selected from each of the grades 2, 3 and 4, totaling 45 per school. In Karnataka, 30 students were selected per school, 15 each from grades 4 and 5. School enrollment registers were used for random selection of students. In the case that a given grade had less than the required number of students, all students were selected.

The surveys were conducted via in-person interviews by a team of trained research assistants with prior experience in administering rural household surveys in the region. Surveys were conducted in the local language, Hindi in MP and UP, and Kannada in Karnataka, and instruments pilot tested prior to use.

Outcomes measured at baseline.

Teacher attendance and activity: Four unannounced visits were made, one every 2–3 weeks, to record attendance and activity. Activity is a measure of whether a teacher was actively engaged in teaching when the team arrived. Attendance takes the value 1 if a teacher was present in school, 0 otherwise. Activity is 1 if the teacher was teaching, writing on the board, supervising written work, teaching by rote or another method, 0 if the teacher was absent, chatting, sitting idle or standing outside the classroom, keeping order in the classroom but not teaching, engaged in non-teaching work. Teacher attendance and activity variables are constructed as averages over the four visits for a teacher and interpreted as fraction of visits the teacher was present (or engaged in teaching). Both variables take values between 0 and 1.

Learning. Students were assessed on a competency based language and mathematics test that lasted approximately 20 minutes. The language test had reading and writing competencies. The mathematics test included addition, subtraction, multiplication and division. The competencies tested fell within or below those listed for the grade in the minimum level of learning (MLL) framework outlined by the Indian government. Learning outcomes are the shares of children who attain specific competencies: Percent of children who can read sentences and words, percent of children who can write sentences and words, percent of children who can do addition, subtraction and less (addition), multiplication and less (addition and subtraction), division and less (addition, subtraction and multiplication).

School council members were interviewed on their participation in school oversight activities and awareness of roles and responsibilities.

Intervention.

The intervention was an information campaign. The tools used in the campaign consisted of a short film of 6 minutes, poster, wall-painting, take-home calendar and learning assessment booklet. The tools were same across states, except the information contained was state-specific. There are state-specific differences in the nature and extent of devolution, that is, in the structures created for the community to exercise local governance of schools and the amount of oversight devolved (described in S1 Appendix). The information was acquired from the states’ education offices, and tools verified and approved for their content by each state government prior to the campaign.

The film, poster, wall-painting and calendar focused on the following information: details of roles and responsibilities of school councils; rules for selection of council members; rules for council meetings including number of mandatory meetings, minimum attendance required in meetings and record keeping of minutes; organization and funding of school accounts; right to information with respect to schools including right to obtain copies of any school record; where to complain about any problems; benefits that students in primary grades are entitled to such as cash stipend, textbooks, mid-day meal, school uniforms. The film, poster and wall-painting contained key information highlights (S2 Appendix has an example of the information highlights) while the calendar contained all of the information in detail. The learning assessment booklet outlined the minimum levels of language and math skills that children were expected to acquire by grade, based on the MLL framework recognized by the government. The booklet was an easy-to-use tool for parents to assess whether their child knew the minimum expected for his or her grade.

In addition to the information campaign treatment in the three states, there was a second treatment only in Karnataka. This was a slightly different treatment with an additional 2 minute capsule at the end of the film which showed average wages for different levels of schooling to increase awareness of the economic benefits of schooling and advocated the audience to become involved in monitoring school outcomes. All other tools were the same as in the first treatment.

Campaign teams were blind to baseline and follow up surveys. The first round consisted of three visits to a GP, each visit separated by a period of two-three weeks. A key feature of the campaign was community-wide big public meetings that parents, teachers and school committee were invited to attend. On each visit, the team organized 2–3 such meetings across different neighborhoods of the GP. The target audience comprised of parents, school council members, and teachers. Separate meetings were held in low- and high- caste neighborhoods. The invitations were broadly publicized 2–3 days in advance via an audio track, the local village announcer and enlisting the primary school teachers’ help.

A meeting lasted about 40 minutes during which the film was screened twice followed by opportunity to ask questions, and discussion among the audience. To ensure uniformity, research assistants followed a script, read out the introduction and were only allowed to reply to questions to which the answers were already written on the calendars. Any other questions or issues raised were not answered. The team was instructed to not participate in any discussion that took place among the community members following the film presentation. The audience was told that the information was collected from the government.

After a meeting, calendars and learning assessment booklets were distributed door-to-door. Posters and wall-paintings were displayed in the school and other prominent locations.

The second round was similar to the first—each GP visited thrice and visits separated by a period of three-four weeks. A focus group discussion in a small sample of treatment villages (see below) between the two intervention rounds informed the following changes based on suggestions from participants in the way meetings were organized without changing anything substantial. One change was that meetings were now mainly held at the school. If members from some caste groups or women were not willing to come to the common meeting due to village norms, then the team would hold another meeting in their neighborhood. A second change was that the film was excluded the last 2 times meetings were held. With the same film shown repeatedly, it was difficult to attract audience for the meetings. A third change was that in order to inform the illiterate audience, key information in the calendar was read aloud exactly as written to preserve uniformity. The campaign ended with 11 to 14 repeat meetings in every village cluster or GP over the two and half year period.

Follow-up survey.

Baseline survey participants were re-interviewed after 12 months (midline or first follow-up survey) and 36 months (second or final follow-up survey) by research assistants who had no knowledge of the intervention. To maintain this blinding, intervention group teachers, students and council members were not asked if they had attended any informational meetings.

Midline. In MP and UP, sample students in grades 2–4 at baseline were followed and were now in grades 3, 4, and 5 respectively. In Karnataka, baseline sample students, who were now in grades 5 and 6, were followed. Each cohort by grade was administered the same test form as in baseline. For example, grade 3 students were given the same test form administered to them in grade 2 at baseline.

Final follow-up. In MP and UP, sample students in grades 3 and 4 at baseline (or in grades 4 and 5 at midline) could not be followed since they were out of the primary school cycle at this time. A fresh sample of students in grades 3, 4 was randomly chosen, while those currently in grade 5 were followed from midline and baseline (at which time they were in grades 3 and 2 respectively). In case a current grade 5 student followed from midline no longer was in school, the research team traced him or her in the village to administer the test. In Karnataka, sample students at baseline were out of the sample schools by this time. So a fresh sample of students in grades 4, 5 was chosen.

Focus group discussions (FGDs) and qualitative interviews.

One year after the first round of intervention, we conducted FGDs in 10 randomly selected intervention GPs each in MP and UP. Two focus group meetings were held in every GP, one among residents from disadvantaged social classes or low caste (S3 Appendix has definition of low caste groups) and the other among non-low caste or high caste residents. Participants were asked if they remembered the campaign, post-campaign whether they had discussed the information with anyone in the village, whether they had raised school-related issues with teachers or school committees, why bigger changes were not seen and what could be done to improve the campaign.

Alongside FGDs, individual school committee members, 32 in UP and 50 in MP, were interviewed in the same GPs on questions posed to the focus groups, and a few additional questions on participation in school oversight. Note that the focus group was not conducted in Karnataka due to reasons of limited resources.

Teacher effort.

In UP, the intervention is associated with 12% (8 percentage points; of 68 percent at baseline) increase in teacher attendance between baseline and follow-up (Table 2). At midline the impact was similar in that it was on teacher attendance and similar in magnitude [32]. In MP, the intervention is associated with increases of 9% (6 percent points; of 69 percent at baseline) in teacher attendance and 25% (8 percent points; of 32 percent at baseline) in teacher activity (Table 2). At midline the impact was only on teacher activity in MP although similar in magnitude [32]. When we separate the sample by tenure type—into civil-service and contract teachers—the impact is primarily on civil-service teachers in both states. Civil-service teachers have permanent jobs which means greater job security and lower incentive to work compared to contract teachers. In UP, their attendance increased by 23% (14 percent points; of 61 percent at baseline). In MP, civil-service teachers’ attendance and activity increased by 16% (11 percent points; of 67 percent at baseline) and 43% (13 percent points; of 30 percent at baseline) respectively. In comparison, at midline the impacts on attendance in UP and activity in MP were similar regardless of contract type [32].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Difference-in-differences linear regression results in MP and UP where change in outcome (teacher effort) from baseline to follow-up is dependent variable.

https://doi.org/10.1371/journal.pone.0280803.t002

Disaggregating the analysis by characteristics of civil-service or permanent teachers, the impact is significant and greater in magnitude for high caste, male, and more educated teachers (S4 Table).

These findings can be due to increased monitoring by the communities as we find below. The extra scrutiny is likely to have affected those civil service teachers more whose motivation and effort levels were lower at baseline. Contract teachers, on the other hand, had higher effort rates, 16–22% higher attendance and 37–64% higher activity rates compared to civil service teachers at baseline [37]. They also had much lower salary levels, between one-fourth and one-fifth of civil service teachers’ salary [39]. This could explain why the initial impact on contract teachers observed at midline faded. It may not have been worthwhile for them to increase effort in the long(er) run, as not doing more would not invite social censure because in the community’s perception they may have been reasonably ‘maxed’ out.

Learning.

As explained in Methods section, the unit of analysis for learning outcomes is the grade because at follow-up the sample of grades 3 and 4 students is a fresh sample while grade 5 sample had participated in baseline. Therefore in learning outcome regressions we include an interaction term of a dummy variable representing grade 5 interacted with the treatment dummy variable. Results are reported accordingly, and coefficients on treatment variable are presented for grade 5 and grades 3/4 separately in the table below.

We find a consistent impact mainly in the share of children acquiring mathematics competencies (Table 3). In UP, increases are observed in the percent of children who can do addition (3 percentage points; 20% of baseline average of 15 percent), subtraction and less (2 percentage points; 25% of baseline average of 8 percent), multiplication and less (2 percentage points; 40% of baseline of 5 percent), and division and less (3 percentage points; 73% of baseline of 4 percent) in grades 3 and 4. In grade 5, increases are in the percent of children able to do addition (4 percentage points; 15% of baseline of 26 percent), and subtraction and less (3 percentage points; 20% of baseline of 15 percent).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Difference-in-differences linear regression results in MP and UP where change in learning outcome from baseline to follow-up is dependent variable.

https://doi.org/10.1371/journal.pone.0280803.t003

In MP, increases occur in the percent of children who can do addition (11 percent points; 27% of baseline of 41 percent), subtraction and less (6 percent points; 30% of baseline of 20 percent), multiplication and less (4 percent points; 44% of baseline of 9 percent), and division and less (2 percent points; 31% of baseline of 6 percent) in grade 5.

Increases occur in the share of children in grades 3 and 4 able to write (3 percent points, 55% of baseline of 5 percent) and read (5 percent points, 50% of baseline of 10 percent) in UP. It is important to note the size of percent impacts is generally large due to low levels of the absolute learning outcomes at baseline. At midline impacts on learning were fewer and only on reading in grade 3 in UP as well as MP [32].

School councils’ participation and awareness.

The difference in differences analysis was done using mean response at the GP level. Analysis was not possible at the individual member level in MP as school council elections were held anew during the course of intervention due to change in election rules that reduced the council term to one year. In UP 60% of original council members remained at follow-up, while 40% were new due to routine member turnover.

School councils became more functional after the campaign (Table 4). In UP, there is 12% (0.14 of an average of 1.20 meetings at baseline) increase in the number of school council meetings, 8% (4 percent points of 52 percent at baseline) increase in attendance in meetings, 18% (8 percent points of 44 percent at baseline) increase in member participation in school visits, and 14% (0.25 of an average of 1.77 members attending school visits at baseline) increase in total number of members attending school visits. At midline, the only impacts observed in UP were increases in the number of council meetings (25%) and member participation in school visits (25%) [32]. Although fewer impacts were observed at midline, their higher magnitudes compared to end line could perhaps be explained by two-fifths new council members in end line survey. In MP, an 11% (0.21 of an average of 1.92 visits at baseline) increase occurred in the number of school visits by council members and 14% (9 percent points of 63 percent at baseline) increase in member participation in school visits. At midline, no impact was observed on school council variables in MP [32].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Difference-in-differences linear regression results in MP and UP where change in school council outcome from baseline to follow-up is dependent variable.

https://doi.org/10.1371/journal.pone.0280803.t004

In both states, there is an increase in members’ awareness of school accounts and of oversight roles vis-à-vis teachers, although the increase is more in UP compared to MP (S5 Table).

Impact on participation and awareness is larger on councils with higher proportion of high caste members (S6 and S7 Tables, S3 Appendix). Analyzing by member type, the impact tends to be greater on chair and secretary members of councils than on other parent members. This suggests low caste and parent members experienced smaller gains in participation.

Teacher effort.

In Karnataka, the intervention is associated with no change in teacher attendance or activity between baseline and follow-up (Table 5). This is similar to the finding at midline when no impact on teacher effort was observed, and can be because teacher effort was high at baseline both in absolute terms and in comparison to the other two states; almost 90 percent of teachers were present and 80 percent of those present were teaching.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Difference-in-differences linear regression results in Karnataka where change in outcome from baseline to follow-up is dependent variable.

https://doi.org/10.1371/journal.pone.0280803.t005

The impacts for Karnataka are for the two treatments pooled together. When an additional treatment dummy variable is introduced to represent the differential effect of the second treatment, it is not significantly different from zero for the teacher effort and learning variables (S8 Table). This is similar to the findings of no additional effect of the second treatment at midline [32].

Learning.

We find a consistent impact mainly in the share of children acquiring mathematics competencies (Table 5), similar to results in MP and UP. Increases are observed in the percent of children who can do subtraction and less (8 percentage points; 15% of 52 percent at baseline), multiplication and less (8 percentage points; 24% of 33 percent at baseline), and division and less (8 percentage points; 29% of 28 percent at baseline). There is no change in the share of children able to write or read. In comparison, at midline increases were observed on one competency each in math (multiplication and less) and language (writing) [32].

School councils’ participation and awareness.

There is no overall change in school council members’ participation associated with the intervention and in fact a decrease in the reported attendance in meetings (Table 5). These findings could reflect new member elections that took place soon after the second round of intervention, not leaving enough time for member participation to change. These findings are similar to those at midline when no change in member participation was observed [32]. But there is an increase in members’ awareness of school accounts and oversight roles vis-à-vis other roles and responsibilities such as preparing schemes to improve school quality and undertaking school visits (S5 Table).

Focus group discussions.

Findings from MP and UP are aggregated in S9 Table. A focus group meeting had 12 participants on average. 63% (509/804) of focus group participants remembered the campaign. Of those who remembered, 72% had discussed the disseminated information with others in the village, and 59% said the discussion went on for weeks. The most frequently-mentioned topics of discussion were learning and teaching: all of the groups mentioned these among the main themes discussed, and 76% listed them ahead of other topics.

At least 33% reported they had taken up issues of teacher presence and learning with those responsible for service delivery in schools (S9 Table, column 2). More than 70% raised the issues with teachers, almost 30% with school council chairs and almost none with other council members. More than 75% went with others rather than alone to talk to this person. When asked about the reaction of the person with whom the issue had been raised, at least a third encountered an angry reaction and also about a third had difficulty discussing these concerns with the person they had raised them with.

More than 60% of focus group participants reported using the learning assessment booklet to assess learning levels and after assessing, about one-fourth raised it as an issue with a service provider. About 82% percent raised it with a teacher, 17% with the school committee chair and none with other council members. More than 75% went with others rather than alone to talk to this person.

When asked why bigger changes did not occur in learning or teacher effort, prominent themes mentioned in the responses were “teacher is dominating,” “it is difficult to talk to the teacher,” “teacher does not listen,” and “teacher does not care.” Among low caste groups, “people are afraid to talk to the teacher” was also mentioned. When asked how the information campaign can be made more effective, the dominant response (73%) was to have more frequent meetings to disseminate information.

Interviews of school council members.

Council members discussed the information widely, 87% had discussed the information with others and 72% said the discussion went on for weeks (S9 Table, column 3). When asked the three main important issues discussed, 83% cited learning, 64% cited teaching and 63% cited other issues. About 39% raised learning and 27% raised teacher attendance issues with teachers or council chairs. About 78% of members assessed learning levels via the tool. After assessing, 43% discussed it with the teacher (83%) and/or chair (21%). Unlike the focus groups, most members neither faced any anger nor found it difficult to discuss the issue with the person they went to.

Members perceived an improvement in committee meetings, which were now more inclusive and frequent after the campaign. When asked if the campaign made a difference to council meetings, 71% of members reported an improvement. When we asked what improved about meetings, main responses were “meetings are more frequent,” “learning is discussed,” “more members attend,” and “all are informed of when a meeting will take place.” They reported taking specific actions to monitor the schools such as checking school timings and teacher presence, and assessing grade appropriate learning levels. 76% of members reported attempting to assess learning levels in schools, 48% tried to verify teacher presence or school timings by visiting the school, and 25% directly talked to the teacher about his or her teaching.