Skip to main content
Advertisement
  • Loading metrics

Understanding why EmpaTeach did not reduce teachers’ use of violence in Nyarugusu Refugee Camp: A quantitative process evaluation of a school-based violence prevention intervention

  • Camilla Fabbri ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Software, Supervision, Validation, Writing – original draft, Writing – review & editing

    camilla.fabbri@lshtm.ac.uk

    Affiliation London School of Hygiene and Tropical Medicine, London, United Kingdom

  • Timothy Powell-Jackson,

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    Affiliation London School of Hygiene and Tropical Medicine, London, United Kingdom

  • Katherine Rodrigues,

    Roles Funding acquisition, Methodology, Writing – review & editing

    Affiliation International Rescue Committee, New York, New York, United States of America

  • Alexandra De Filippo,

    Roles Methodology, Writing – review & editing

    Affiliation Behavioral Insights Team, Brooklyn, New York, United States of America

  • Michael Kaemingk,

    Roles Methodology, Writing – review & editing

    Affiliation Behavioral Insights Team, Brooklyn, New York, United States of America

  • Gerard Torrats-Espinosa,

    Roles Methodology, Writing – review & editing

    Affiliation Columbia University, New York, New York, United States of America

  • Baptiste Leurent,

    Roles Formal analysis, Methodology, Writing – review & editing

    Affiliation University College London, London, United Kingdom

  • Elizabeth Shayo,

    Roles Funding acquisition, Writing – review & editing

    Affiliation National Institute for Medical Research, Dar es Salaam, Tanzania

  • Vivien Barongo,

    Roles Funding acquisition, Writing – review & editing

    Affiliation National Institute for Medical Research, Dar es Salaam, Tanzania

  • Karen M. Devries

    Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation London School of Hygiene and Tropical Medicine, London, United Kingdom

Abstract

EmpaTeach was the first intervention to address teacher violence to be tested in a humanitarian setting and the first to focus on reducing impulsive use of violence, but a cluster randomised trial found no evidence that the intervention was effective in reducing physical and emotional violence from teachers. We aimed to understand why. We conducted a quantitative process evaluation to describe the intervention implementation process (what was implemented and how); examine teachers’ adoption of positive teaching practices (was the content of the intervention taken up by participants), and test mechanisms of impact underlying the program theory (how the intervention was supposed to produce change). Despite participation in the intervention activities and adoption of intervention-recommended strategies (classroom management and positive disciplinary methods), we show that teachers who used more positive discipline did not appear to use less violence; and teachers in intervention schools did not experience gains in intermediate outcomes such as empathy, growth mindset, self-efficacy or social support. Our findings suggest that the intervention did not work due to the failure of some key hypothesised mechanisms, rather than because of implementation challenges.

Introduction

Background

The detrimental effects of violence on children’s physical and mental health as well as on a wide range of socio-economic outcomes are well known [13]. After decades of research into the causes and consequences of violence against children, academics, practitioners and donors have turned their focus to understanding how to effectively prevent violence in and through schools [4, 5]. School is meant to be a safe and protective environment where children can focus on building knowledge, skills and competencies that will allow them to achieve positive educational, economic, and social outcomes in adulthood. However, school is also a setting where children experience violence from both teachers and peers [6].

In resource-constrained settings teacher violence often takes the form of corporal punishment used to manage overcrowded classrooms and discipline misbehaving students. The use of violent discipline and corporal punishment by teachers may be particularly widespread in settings characterised by large class sizes, unqualified school staff, weakened social ties, and norms that condone it, even when violence is formally prohibited [79]. According to a recent systematic review of 53 studies, lifetime prevalence of school corporal punishment was above 70% in Africa and Central America, and past-week prevalence was above 40% in Africa and Southeast Asia. In primary schools these estimates were higher with lifetime prevalence above 80% and past-week prevalence around 50% in some settings [10]. Official statistics from humanitarian settings are scant, however evidence suggests that the risk of violence to displaced and refugee children may be even higher due to heightened stressors on education service provision and individual teachers [1113]. Humanitarian crises and displacement are likely to expose individuals and communities to economic strain, hunger, as well as to stress and traumatic events [14, 15]. Additionally, exposure to violence before, during or after displacement is likely to be compounded by traditional social norms that condone or promote use of violence leading to the normalisation of violence [16].

To date only a handful of interventions to prevent teacher violence have been rigorously tested globally [1721] and the evidence on what works and what doesn’t is still limited, especially in refugee and emergency settings [22]. Among 93 programmes aimed at preventing forms of violence against children in and through schools in the Global South, only 7% targeted teachers’ use of corporal punishment Of these programmes, 23% targeted multiple forms of violence, 17% targeted gender-based violence, 23% targeted bullying, 14% targeted sexual violence, 8% targeted aggression and 8% targeted other forms of violence against children and few studies reported the results of evaluations assessing the programme implementation and mechanisms [5]. Little is known about what the essential intervention components and key mechanisms of impact of successful school-based violence prevention interventions are. This lack of evidence precludes practitioners’ and policy makers’ ability to develop and/or adapt effective programmes to prevent and respond to violence in schools. When interventions fail to produce the expected results, it is even more important to conduct complementary process analyses to unpack where, how, and why breakdowns of the intervention theory may have happened to inform future efforts.

In this study, we conducted a quantitative process evaluation of the EmpaTeach intervention to investigate why the intervention was not successful in lowering levels of teacher violence in schools in Nyarugusu Refugee Camp, Tanzania. EmpaTeach was, to the best of our knowledge, the first school-based violence prevention intervention to be rigorously evaluated in a humanitarian setting. Between January and March 2019 and, the International Rescue Committee (IRC), implemented the 10-week teacher-led intervention, aimed at preventing violent discipline in primary and secondary schools in Nyarugusu Refugee Camp in Tanzania. The intervention was designed to address high levels of school violence known to exist in the camp, despite such practices being explicitly prohibited. A cluster randomised controlled trial of the intervention found no evidence of an effect of EmpaTeach on physical violence from teachers to students (primary outcome) and no measurable effect on students’ experiences of emotional violence, depression symptoms and school attendance (secondary outcomes) [23]. However, post-hoc analyses of the trial data suggested that the intervention improved some intermediate outcomes: teachers in intervention schools used more positive discipline strategies, showed attitudes that were less supportive of violent discipline and reported improved self-regulation compared to those in control schools [23].

The null effects on primary and secondary outcomes could be attributed to the magnitude of change in intermediate outcomes that may have been too small to lead to changes in teacher behaviour; to the possibility that teachers may have used new disciplinary and classroom management strategies, but they did so in addition, rather than in replacement, of traditional corporal punishment approaches; or to the absence of a sufficient causal link between these intermediate outcomes and teachers’ use of violence. The process evaluation that we present here relies on various evaluation frameworks [2426] and data sources to a) describe the intervention implementation process (what was implemented and how); b) examine teachers’ adoption of positive teaching practices (was the content of the intervention taken up by participants), and c) test mechanisms of impact underlying the programme theory (how the intervention was supposed to produce change).

Intervention package

EmpaTeach is a 10-week low-intensity, peer-led intervention targeted at teachers. Its aim is to reduce and prevent teachers’ use of violence in the classroom, as well as improve teacher wellbeing. The intervention, which was co-designed with refugee teachers by the International Rescue Committee in partnership with the Behavioral Insights Team, draws on behavioural insights and cognitive-behavioural therapy to upskill teachers with positive classroom management, alternative discipline strategies, and self-regulation and wellbeing techniques. The programme uses a booklet to guide teachers through 12 in-person sessions during which they engage in interactive exercises to identify personal triggers, explore how their thoughts and feelings lead to reactions, and learn and plan when to use new strategies to manage classroom dynamics and respond to student misbehaviours. Teachers in intervention schools are grouped with peers who serve as their support network and new social reference point; groups are led by one of the members, after they receive a short facilitation training.

The intervention instructs teachers on best practices in education (e.g., providing feedback, encouraging growth mindsets, checking for understanding, using questioning techniques, etc.) and classroom management to improve their ability to effectively deal with large overcrowded classrooms and maintain student focus. Strategies to manage the classroom include setting a lesson objective, creating classroom rules, using discipline in a consistent manner, using the classroom space, and using positive discipline. In the positive discipline session teachers are exposed to a number of encouragement (e.g., cheering, praising etc.), redirection (e.g. moving in the classroom, changing tone of voice, clapping etc.) and reflection techniques that they can use to provide emotional support to well-performing students, to de-escalate misbehaviours, and to punish those who engage in disruptive behaviours.

Programme theory of the EmpaTeach intervention

The EmpaTeach intervention rests on the premise that overcrowding and resource constraints in humanitarian settings make classroom management challenging and generate stress for teachers. In the absence of alternative discipline tools, it was hypothesized that teachers resort impulsively to the use of violent discipline to punish students’ behaviours that are perceived as problematic.

In-person weekly sessions were designed to provide teachers with an opportunity to reflect on their own experiences, learn and practice new classroom management and positive disciplinary methods with peers, while practising self-regulation and wellbeing techniques. Group dynamics and interactions during these meetings were considered important to prompt teachers to reflect on current practice, role-play new strategies and foster the creation of new social norms and attitudes towards corporal punishment. The intervention content (i) asked teachers to practice self-affirmation to reduce self-threat, encourage openness to change, and build self-efficacy [27]; (ii) prompted teachers to take different perspectives and to support each other and their students to build empathy [28]; and (iii) emphasised teachers’ potential to change and overcome challenges with effort [29]. The intervention theory was primarily based on the idea that teachers’ adoption of positive classroom management and alternative discipline strategies would lead to the replacement of violence in favour of newly acquired strategies. In other words, teachers would test and use the intervention-recommended strategies to find them so effective that they would resort less to the use of violent discipline. Additionally, improved teacher skills and competencies would contribute to improvements in attitudes towards violence and self-control which in turn would lead to a reduction in teachers’ use of violence.

Homework assigned to teachers on a weekly basis was designed to induce a process of behavioural activation through repeated practice of new discipline and classroom management strategies both in class and outside of school hours. Finally, the social support and feedback opportunities provided by the peer group and by a named supporter were supposed to encourage teachers throughout the change process and reinforce new norms and practices by providing a new reference group.

The intervention focused on building new skills for teachers and intentionally avoided discouraging the use of violence in the classroom because addressing attitudes on violence explicitly would have required expert facilitation that was not available in the camp and would have reduced the scalability of the intervention [23]. It was hypothesised that by practicing the new strategies teachers would be persuaded to abandon the use of corporal punishment in favour of the new, more effective positive disciplinary methods (replacement effect), and that empathy building exercises coupled with social reinforcement provided by the peer groups would promote a change in self-regulation and attitudes towards violent discipline.

Methods

Ethics statement

The Preventing Violence Against Children in Schools (PVACS) study was approved by the London School of Hygiene & Tropical Medicine Ethics Committee (ref. 16000) and the Tanzania National Institute for Medical Research (ref. NIMR/HQ/R.8a/Vol.IX/2920).

The following consent procedures were followed for the cross-sectional surveys [23]: headteachers provided consent for data collection and intervention implementation in schools, and consent to approach individual students; students provided informed assent, and individual teachers provided informed consent. In consultation with camp stakeholders, it was deemed appropriate in this setting to seek headteachers’ consent for children’s participation as they had full responsibility for students during school hours and seeking active parental consent would have precluded participation of many children, given high numbers of unaccompanied youth in the camp. Teachers and headteachers provided verbal consent to IRC’s classroom observations which were conducted as part of the monitoring of the intervention implementation. Teacher attendance and homework records were collected as part of teachers’ participation in the EmpaTeach intervention.

Study setting

The PVACS study included a cluster randomised controlled trial to evaluate the effectiveness of the EmpaTeach intervention to reduce violence from teachers to students in primary and secondary schools in Nyarugusu Refugee Camp in Tanzania. At the time of the study Nyarugusu was hosting around 150,000 refugees from the Democratic Republic of Congo and Burundi, some of whom had been there since the mid-1990s when the camp was established. The camp is operated by the United Nations High Commission for Refugees and the Tanzania Ministry of Home Affairs. The IRC provides education and gender-based violence response services and, at the time of the study, all child protection services. The camp is largely divided between Congolese and Burundian populations who reside in different zones; schools follow the curriculum of the country of origin. According to a needs assessment conducted in late 2017 more than 2 in 3 students reported feeling safe in school; however several child protection concerns were identified related to poor facilities, overcrowded classrooms, and the risk of violence in and on the way to and from school [30]. Formative research conducted prior to the PVACS trial also highlighted that resource constraints and overcrowding were important stressors for teachers in the camp. In this context, EmpaTeach was designed to improve teacher wellbeing and reduce stress-induced violence in schools [7].

Evaluation approach

We designed the process evaluation prospectively in September 2018 in collaboration with the International Rescue Committee and Innovations for Poverty Action. The original analytical plan was designed to focus on mediation analysis of teachers’ and students’ outcomes and less on intervention implementation. In light of our trial findings, in this study we extended our approach to follow each step in the hypothesised pathways that link intervention and outcomes, starting with reach and adoption and moving on to mechanisms of action of the intervention. Reach refers to the extent to which the intended audience comes into contact with the intervention [31] while adoption is defined as the intentional decision, in our case by teachers, to employ an innovation or practice; adoption is also referred to as “uptake” [32]. To measure reach, we used teachers’ attendance records to explore whether teachers actually participated in the programme. To measure adoption, we used a combination of teacher-classroom observations and survey data to examine whether teachers implemented the intervention-recommended strategies.

Additionally, we explored the main pathways through which the intervention was supposed to produce change (mechanisms): a) we tested for the presence of a replacement effect by looking for an association between teachers’ use of intervention-recommended strategies and their use of violence and b) estimated the intervention effect on teachers’ empathy, growth mindset, self-efficacy and social support which were all hypothesised to complement and sustain the use of the EmpaTeach strategies and to promote changes in self-regulation and attitudes towards violence.

Data

Our analyses drew on various sources of data (Table 1). We used (1) teacher attendance and homework completion records compiled by EmpaTeach group coordinators at the beginning of each intervention session and delivered to IRC staff on a weekly basis; (2) data from unannounced teacher classroom observations (TCOs) collected by the IRC’s programme team using standardised checklists as part of their monitoring activities; (3) and two rounds of cross-sectional surveys with teachers and students conducted within the school setting by the research team as part of the cluster randomised-controlled trial before (baseline) and 2 months after the end of the intervention (midline). All datasets used for analyses contained anonymised data and only participant IDs were used to link datasets.

Routine monitoring data such as teacher attendance and homework records were collected from IRC’s programme team and were not validated and were missing values for 3 teachers. The scales included in the teacher classroom observations checklist were pre-tested as part of prototyping and piloting before the implementation of the intervention by IRC staff and inter-rater reliability and validity of the indicators have been assessed elsewhere [33]. All measures included in the cross-sectional surveys were cognitively tested and adapted to the local context.

Analysis

First, we present descriptive statistics for indicators of teachers’ participation in intervention activities. Reach was measured using two indicators: teachers’ individual attendance to group sessions and teachers’ homework completion.

Then, we used an intent-to-treat approach to assess differences in teachers’ uptake of positive classroom management and alternative disciplinary strategies between intervention and control schools, drawing from teacher classroom observations data and PVACS survey data. Adoption was measured as the number and quality of classroom management and positive disciplinary practices that teachers were observed implementing and self-reported using in school. We used t-tests adjusted by clustering to assess the difference in means for continuous variables, such as the quality (score) of how teachers applied the different types of practices and the number of practices they were observed using in the classroom.

Finally, we used multilevel logistic regression models using teacher reported survey data at midline to test the mechanisms to impact of the intervention. To assess the association between teachers’ use of intervention-recommended strategies and their use of violence in the classroom we used pooled data from intervention and control schools to implement four separate models where teachers’ use of violence was included as the outcome and the number of practices they reported using in each of four classroom scenarios as the main exposure, controlling for teacher covariates. We checked the robustness of our findings by replicating the same analyses using student-reported experiences of violence from teachers as the outcome (in line with our main trial analyses) and a school-level mean of teachers’ use of the practices as the main exposure, controlling for student covariates. All models accounted for clustering at the school level and controlled for intervention assignment. Then we used an intent-to-treat approach to test whether the intervention led to changes in teachers’ empathy, growth mindset, self-efficacy, and social support which were hypothesised to be potential mediators (since a variable can only be a mediator of the intervention effect if there is a significant effect of the intervention on it) of the intervention effect on attitudes towards violence and self-control. All data were analysed in Stata/SE 17, and considered statistically significant at the two-sided 5% significance level.

Results

Reach: Teachers’ participation in intervention activities

The intervention was implemented in all 14 schools that were randomly assigned to receive the intervention. Overall, 600 teachers formed 77 peer-led EmpaTeach groups of varying sizes (each group included between 3 and 15 teachers (although the average group was intended to include 7–10 teachers). Attendance records were available for 597 teachers. Table 2 offers an overview of teachers’ participation in the intervention activities. Overall, around half of the teachers (52.4%) attended at least 10 out of 12 group sessions. Ten teachers did not attend any of the EmpaTeach sessions (four of which transferred to intervention schools during the intervention implementation period). Few teachers completed assigned homework: 24% of teachers did not complete any of the homework assignments and fewer than 1 in 3 teachers completed 80% or more of the homework practice. Homework completion was higher in secondary schools while attendance did not vary by level of schooling.

thumbnail
Table 2. Indicators of teachers’ participation in intervention activities.

https://doi.org/10.1371/journal.pgph.0001404.t002

As previously reported [23], the intervention implementation was affected by a major incident whereby 20% of teachers in the camp were laid off from their jobs during the intervention delivery period (January 2019). Although most teachers were subsequently re-hired in the same role within a period of 2 weeks, EmpaTeach intervention groups were disrupted over a period of approximately 4 weeks.

Adoption: Teachers’ use of intervention-recommended strategies

To explore teachers’ use of the strategies that they learnt during intervention sessions we relied on data from observation of teachers’ behaviours in the classroom in both intervention and control schools. Data offered insights into teacher’s use of teaching practices (TP) and classroom management strategies (CM), and their provision of emotional support to students (ES). There were generally no differences between teachers in intervention and control schools in the use of recommended teaching practices and classroom management strategies; however, teachers in intervention schools were significantly better at using positive disciplinary practices than those in control schools (mean score = CM: positive discipline was 1.488, SD = 1.010 in control schools and 2.182, SD = 1.382 in intervention schools, p = 0.002) and at providing emotional support to students by promoting self-compassion (mean = 1.494, SD = 0.904 in control schools and mean = 1.846, SD = 0.894 in intervention schools, p = 0.029) and recognising perseverance (mean = 1.434, SD = 0.853 and mean = 1.946, SD = 1.269 in control and intervention schools respectively, p = 0.006) (Fig 1). In Table 3 we examined more closely how frequently teachers used specific emotional support methods and positive discipline. We found that teachers in intervention schools used significantly more cheering than teachers in control schools, but generally no other statistically significant differences were observed in use of encouragement strategies. We note however that these differences were quite large (use of praise doubled in intervention schools) and our analyses may have been underpowered to detect them. On the contrary, teachers in intervention schools were observed using a statistically significant higher number of alternative disciplinary practices such as redirection techniques (such as moving closer to students or changing students’ seats) to manage student misbehaviours and disturbances in the classroom (Table 3). These results were broadly in line with findings from the intent to treat trial analyses (already presented in another publication and replicated here using mixed models) conducted using teacher self-reported use of these strategies to a series of scenario questions in the PVACS surveys (S1 Table).

thumbnail
Fig 1. Effect of intervention on teachers’ use of positive practices.

https://doi.org/10.1371/journal.pgph.0001404.g001

thumbnail
Table 3. Number of specific encouragement and positive discipline techniques used by teachers.

https://doi.org/10.1371/journal.pgph.0001404.t003

The figure shows the regression coefficients and 95% confidence interval for the effect of the intervention on the quality assessment of recommended practices (score 1 to 4, where higher values correspond to better use of each practise). TP = Teaching Practices, CM = Classroom Management practices, ES = Emotional Support.

Mechanisms of impact

Replacement of violence with positive strategies to manage and discipline students.

Using teacher self-reports, we found that the hypothesised replacement of violence with positive discipline strategies probably did not take place: Table 4 shows that none of the positive intervention practices were associated with significant decreases in use of physical violence from teachers. We replicated the same analyses using student self-reported experiences of violence and school-level indicators of teachers’ use of positive practices as exposure (students cannot be linked to individual teachers, since some students have multiple teachers) and results are reported in S2 Table. We found no evidence that teachers’ use of recommended EmpaTeach strategies was associated with lower use of violent discipline. If anything, teachers’ use of praise strategies was associated with higher odds of students reporting experiences of physical violence.

thumbnail
Table 4. Association between use of positive strategies and teachers ‘self-reported use of violence at midline.

https://doi.org/10.1371/journal.pgph.0001404.t004

Teachers’ socio-emotional skills: Empathy, self-efficacy and social support.

Table 5 shows that the intervention did not significantly impact teachers’ empathy levels, growth mindset, self-efficacy in the classroom or perceived social support; there was no statistically significant difference in these outcomes between teachers in control and intervention schools. These factors could not therefore mediate the effects on teachers’ attitudes and self-control as hypothesised in the programme theory.

thumbnail
Table 5. Intervention effect on teachers’ empathy, perspective taking and social support at midline.

https://doi.org/10.1371/journal.pgph.0001404.t005

Discussion

Summary of main findings

Our results showed that attendance and participation in the intervention were overall adequate considering the challenging context in which EmpaTeach was implemented and the fact that implementation was quite severely disrupted. We also demonstrated that teachers learnt and used new classroom management strategies and positive disciplinary methods confirming that the content of the intervention was accessible and relevant for participants. However, teachers’ adoption of the EmpaTeach strategies was not associated with their use of violence: teachers who used more positive discipline did not appear to use less violence; and teachers in intervention schools did not experience gains in empathy, growth mindset, self-efficacy or social support. This suggests that the hypothesized replacement of violence with nonviolent alternative discipline did not occur, and that intervention activities did not have an appreciable effect on intermediate outcomes thought to contribute to impulsive use of violence.

Understanding why EmpaTeach did not work

Although our results suggest that what did not work in EmpaTeach was related to the design of the intervention and not to the context in which it was implemented, we cannot exclude that the same intervention implemented in a non-refugee environment may have yielded different results.

Adoption of intervention-recommended alternative discipline techniques presented in the EmpaTeach booklet such as moving closer to or changing seats to misbehaving students or using a sudden silence to regain attention in the classroom, was significantly higher in intervention versus control schools. However, we found no significant difference between intervention and control schools in teachers’ use of encouragement strategies that were designed to reward positive behaviours of students, aside from cheering. It is possible that these encouragement techniques were the easiest to be adopted (and in terms of frequency, teachers in all schools reported using them more frequently than other strategies) and therefore participation to EmpaTeach sessions was not necessary to enable teachers’ uptake.

The intervention theory was based on the idea that teachers would test and practise these new strategies and that they would ultimately find them so useful and effective in managing student misbehaviour that teachers would substitute the use of violent discipline with these newly acquired methods. The intervention therefore did not explicitly discourage use of corporal punishment and relied on practice of new strategies, coupled with social support, to lead to changes in violent behaviour. Since in the main trial analyses we observed no measurable difference in the use of violence between intervention and control schools, we tested the validity of this theory and found that teachers’ adoption of the EmpaTeach strategies was not associated with their use of violence: teachers who used more positive discipline did not appear to use less violence. It is possible that teachers simply expanded the range of methods they used to manage student misbehaviours and that the replacement of violent behaviours could not take place without formal discouragement of corporal punishment. In contexts like Nyarugusu Refugee Camp where, despite formal bans, corporal punishment is condoned and perceived as a necessary and useful approach to discipline, it may be necessary to engage more directly with norms and attitudes around violence to achieve a shift away from violence. Alternatively, it is possible that the intervention period was simply not long enough to allow for the complete replacement of violent discipline in favour of these new methods and that a longer and more sustained engagement with teachers would have ultimately shifted behaviours away from corporal discipline.

The intervention activities and reflection exercises were supposed to improve teacher attitudes and self-control by strengthening their skills and competencies; however, our results showed that there were no differences in levels of empathy, growth mindset and self-efficacy between intervention and control teachers. It is therefore possible that the intervention content led to the small shifts in attitudes and self-regulation that were observed at midline and endline directly. Finally, improved social support was supposed to sustain and facilitate this change process but again we found no difference in the level of support reported by teachers in intervention and control schools; we conclude therefore that teachers could benefit from more formal and continued coaching and mentoring during the programme period.

Strengths and limitations

Our study had several strengths and some limitations. Strengths include high student and staff response rates and triangulation of a wide range of data including evaluation data collected as part of a randomised controlled trial, routine monitoring data collected by the International Rescue Committee, and attendance records filled out by teachers themselves during intervention sessions. Even though the process evaluation was designed prospectively and fully embedded into the overall study design our ability to make linkages between routine monitoring data and evaluation data was limited. Furthermore, the process evaluation analyses took place after findings from the impact evaluation were known which required changes to our analytical approach. Some of the analyses presented wide confidence intervals making it difficult to rule out moderate associations. As expected, some of the routinely collected attendance and homework completion data were incomplete, and process measures collected as part of programme implementation had not been tested to determine validity and reliability. Additionally, self-reported survey data and data from classroom observations may have been affected by social desirability bias.

What works (and what doesn’t) to prevent violence against children in schools?

We contribute to a limited, but growing, evidence base on strategies to address teacher violence. Of the handful of robustly evaluated interventions globally that achieved reductions in levels of teacher violence, very limited scrutiny of implementation strategies and mechanisms of action has been undertaken. This leaves open questions about how and which components of these complex interventions led to changes in behaviours.

Violence prevention is often addressed through complex multi component interventions characterised by long causal chains [17, 18, 20]. Understanding how these programmes work requires carefully designed evaluations that assess change along the pathways and enable identification of key mechanisms. When interventions fail to achieve their expected outcomes, it’s important to understand whether the programme theory was incorrectly formulated or whether implementation or contextual factors harmed a potentially successful intervention. A process evaluation of the Good School Toolkit showed that reductions in violence in Ugandan primary schools were achieved through encouragement and reward of positive student behaviours and reinforcement of teachers’ positive discipline methods suggesting that a key component of the intervention was the support provided to individuals to learn new ideas and behaviours that generated new norms supportive of positive student-teacher relationships [37]. Evidence showed that the Toolkit triggered change across several dimensions including students’ feeling of safety and connectedness, school governance structures, teacher-student and peer-to-peer relationships, in addition to teaching and disciplinary approaches [37, 38]. Limited evidence is available on the mechanisms that led Right to Play’s intervention in Pakistan, which used an intense life-skills and play-based curriculum for students complemented by training for parents and school staff, to reductions in teachers’ use of corporal punishment [18]. However, it was suggested that the intervention empowered children by improving their communication and conflict-resolution skills and through changes in social norms around violence and interpersonal relations [18, 39]. Shifts in social norms and attitudes, combined with improved communication and of conflict resolution skills were also the hypothesised kay pathways to lower levels of violence for a school based intervention implemented in Afghanistan [20]. The Irie Classroom Toolbox instead was hypothesised to have achieved reductions in violence by improving teachers’ wellbeing and self-efficacy and promoting their use of appropriate teaching techniques [19, 21]. Despite the fact that the intervention did not explicit discourage use of violence, it offered teachers tools to promote the creation of a safe and nurturing classroom environment [21]. Although it is less clear which specific aspects of the intervention were responsible for these changes, evidence suggests that its main mechanisms of action were indeed linked to improvements in teacher wellbeing and skills.

EmpaTeach’ s inability to achieve the anticipated reductions in physical and emotional violence from teachers generated an opportunity to reflect on what intervention components may be necessary or essential. EmpaTeach included several elements that were similar to those of successful interventions such as quality content on alternative classroom management and disciplinary strategies and the use of a group setting for intervention delivery that promoted social support. However, the uptake of such strategies alone was not sufficient to shift teachers’ behaviours away from violent discipline. The social dynamic provided by the peer-groups did not provide teachers with the required support to allow for these new practices to consolidate.

Results from our analyses and other studies seem to suggest that successful prevention of teacher violence is more likely in the context of whole-school interventions that involve a variety of stakeholders within schools and communities, foster changes in the overall school environment, focus on promoting alternative disciplinary methods for teachers, and offer continued mentoring and support throughout the complex process of change [5, 20, 37]. Despite differences in intervention content and delivery, most interventions achieved reductions in teacher violence through improvements in teacher-student relationships. In EmpaTeach, although empathy and perspective-taking abilities were used as key levers for change, teachers’ relationship with students was not the main focus of the intervention.

Evidence also shows that a prolonged engagement with school actors may be important to support complex changes in attitudes and behaviours around violence, but the delivery of long interventions is challenging in rapidly-changing fragile settings. Identifying essential intervention ingredients is a pressing need to prevent and reduce violence in humanitarian contexts in an efficient and effective manner.

Based on our results, we would suggest further exploration of three intervention components: 1) the current content on alternative discipline practices could be disseminated among other school stakeholders, beyond class teachers, to promote changes in the overall school climate and create more social support and championing; 2) ways to engage with norms around violence more directly could be explored, e.g. respected members of the community or individuals with high credibility among the teacher body could be trained to facilitate critical reflection on norms and explicitly discourage use of violence;3) coaching and mentoring for teachers and in-classroom feedback and support could be integrated to strengthen skills acquisition. Generally, further testing of light-touch programmes in a variety of settings is advisable to pinpoint which intervention components produce desired changes in which contexts.

Conclusion

The findings of these secondary analyses of the PVACS trial offer some important insights into why the EmpaTeach intervention did not lower levels of violence from teachers to students in Nyarugusu Refugee Camp and highlight the importance of conducting theory-informed process evaluations alongside randomised trials to generate evidence on the role of the different elements of complex school-based interventions and interrogate mechanisms of impact. We encourage academics and practitioners working on promising violence prevention interventions to embed, connect or merge adaptable process evaluations into their evaluation designs [40] and explore how programmes could be compressed to be adapted and tested in emergency contexts. With new humanitarian crises unfolding every day finding effective ways to provide safe education environments for displaced and refugee children is critical.

Supporting information

S1 Text. Classification of teacher responses to PVACS survey questions about classroom scenarios.

https://doi.org/10.1371/journal.pgph.0001404.s001

(DOCX)

S1 Table. Intervention effect on teacher self-reported use of strategies at midline.

https://doi.org/10.1371/journal.pgph.0001404.s002

(XLSX)

S2 Table. Association between school mean level of teachers’ use of positive strategies and student self-reported experiences of violence at midline.

https://doi.org/10.1371/journal.pgph.0001404.s003

(XLSX)

Acknowledgments

We wish to thank Mary Qiu, Martin Zuakulu, Dennis Nombo, Sarah Wragg and IRC’s Education and Protection teams for their support throughout the PVACS study. We are thankful to Autumn Brown who kindly helped us with cleaning and coding of IRC’s monitoring data. IPA’s field management team provided outstanding coordination of data collection efforts. We finally thank all the enumerators in Nyarugusu Camp and study participants for their time.

References

  1. 1. Hillis S, Mercy J, Amobi A, Kress H. Global Prevalence of Past-year Violence Against Children: A Systematic Review and Minimum Estimates. Pediatrics. 2016;137(3):e20154079. pmid:26810785
  2. 2. Stark L, Landis D. Violence against children in humanitarian settings: A literature review of population-based approaches. Social Science & Medicine. 2016;152:125–37. pmid:26854623
  3. 3. Osofsky JD. The Impact of Violence on Children. The Future of Children. 1999;9(3):33–49. pmid:10777999
  4. 4. Wodon Q., Fèvre C., Malé C., Nayihouba A., Nguyen H. Ending Violence in Schools: An Investment Case. 2021.
  5. 5. Mathews S, Achyut P, October L, Makola L. Evidence Review: Prevention of violence against children through schools in the Global South. 2021.
  6. 6. Devries K, Knight L, Petzold M, Merrill KG, Maxwell L, Williams A, et al. Who perpetrates violence against children? A systematic analysis of age-specific and sex-specific data. BMJ Paediatrics Open. 2018;2(1):e000180. pmid:29637183
  7. 7. Hecker T, Goessmann K, Nkuba M, Hermenau K. Teachers’ stress intensifies violent disciplining in Tanzanian secondary schools. Child Abuse & Neglect. 2018;76:173–83. pmid:29112855
  8. 8. Mendenhall M, Gomez S, Varni E. Paper commissioned for the 2019 Global Education Monitoring Report, Migration, displacement and education: Building bridges, not walls. 2018.
  9. 9. Human Rights Watch. “I Don’t Want My Child to Be Beaten" Corporal Punishment in Lebanon’s Schools. 2019.
  10. 10. Heekes S-L, Kruger CB, Lester SN, Ward CL. A Systematic Review of Corporal Punishment in Schools: Global Prevalence and Correlates. Trauma, Violence, & Abuse. 2022;23(1):52–72. pmid:32436472
  11. 11. Hecker T, Kyaruzi E, Borchardt J, Scharpf F. Factors Contributing to Violence Against Children: Insights From a Multi-informant Study Among Family-Triads From Three East-African Refugee Camps. Journal of Interpersonal Violence. 2021:088626052110139. pmid:33926287
  12. 12. Meyer SR, Yu G, Hermosilla S, Stark L. Latent class analysis of violence against adolescents and psychosocial outcomes in refugee settings in Uganda and Rwanda. Global Mental Health. 2017;4. pmid:29230315
  13. 13. Stark L, Asghar K, Yu G, Bora C, Baysa AA, Falb KL. Prevalence and associated risk factors of violence against conflict–affected female adolescents: a multi–country, cross–sectional study. Journal of Global Health. 2017;7(1). pmid:28607672
  14. 14. Blackmore R, Boyle JA, Fazel M, Ranasinha S, Gray KM, Fitzgerald G, et al. The prevalence of mental illness in refugees and asylum seekers: A systematic review and meta-analysis. PLoS Med. 2020;17(9):e1003337. pmid:32956381
  15. 15. Knipscheer JW, Sleijpen M, Mooren T, Ter Heide FJJ, van der Aa N. Trauma exposure and refugee status as predictors of mental health outcomes in treatment-seeking refugees. BJPsych bulletin. 2015;39(4):178–82. pmid:26755950
  16. 16. Fletcher EK, Gitter SR, Wilhelm S. Reporting Violence Against Children: Social Norms in Nyarugusu Refugees Camp. Migration Letters. 2017;15(3):377–88.
  17. 17. Devries KM, Knight L, Child JC, Mirembe A, Nakuti J, Jones R, et al. The Good School Toolkit for reducing physical violence from school staff to primary school students: a cluster-randomised controlled trial in Uganda. The Lancet Global Health. 2015;3(7):e378–e86. pmid:26087985
  18. 18. Karmaliani R, McFarlane J, Khuwaja HMA, Somani Y, Bhamani SS, Saeed Ali T, et al. Right To Play’s intervention to reduce peer violence among children in public schools in Pakistan: a cluster-randomized controlled trial. Global Health Action. 2020;13(1):1836604. pmid:33138740
  19. 19. Baker-Henningham H, Bowers M, Francis T, Vera-Hernández M, Walker SP. The Irie Classroom Toolbox, a universal violence-prevention teacher-training programme, in Jamaican preschools: a single-blind, cluster-randomised controlled trial. The Lancet Global Health. 2021;9(4):e456–e68. pmid:33631132
  20. 20. Corboz J, Siddiq W, Hemat O, Chirwa ED, Jewkes R. What works to prevent violence against children in Afghanistan? Findings of an interrupted time series evaluation of a school-based peace education and community social norms change intervention in Afghanistan. PLoS One. 2019;14(8):e0220614. pmid:31386683
  21. 21. Baker-Henningham H, Scott Y, Bowers M, Francis T. Evaluation of a Violence-Prevention Programme with Jamaican Primary School Teachers: A Cluster Randomised Trial. Int J Environ Res Public Health. 2019;16(15). pmid:31390743
  22. 22. Devries KM, Ward CH, Naker D, Parkes J, Bonell C, Bhatia A, et al. School violence: where are the interventions? The Lancet Child & Adolescent Health. 2022;6(1):5–7. pmid:34793723
  23. 23. Fabbri C, Rodrigues K, Leurent B, Allen E, Qiu M, Zuakulu M, et al. The EmpaTeach intervention for reducing physical violence from teachers to students in Nyarugusu Refugee Camp: A cluster-randomised controlled trial. PLOS Medicine. 2021;18(10):e1003808. pmid:34606500
  24. 24. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021:n2061. pmid:34593508
  25. 25. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15.
  26. 26. Cartwright N, Charlton L, Juden M, Munslow T, Williams RB. Making predictions of programme success more reliable. CEDIL Methods Working Paper. Oxford: Centre of Excellence for Development Impact and Learning (CEDIL). 2020.
  27. 27. Cohen GL, Sherman DK. The psychology of change: self-affirmation and social psychological intervention. Annu Rev Psychol. 2014;65:333–71. pmid:24405362
  28. 28. Broockman D, Kalla J. Durably reducing transphobia: A field experiment on door-to-door canvassing. Science. 2016;352(6282):220–4. pmid:27124458
  29. 29. Dweck CS. Mindset: The New Psychology of Success. How We Can Learn to Fulfill Our Potential. New York, NY: Ballantine Books; 2006.
  30. 30. Education Working Group. Joint Education Needs Assessment Report Tanzania. Report of an assessment of refugee education in three refugee camps in the Kigoma Region, Tanzania. 2018.
  31. 31. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350(mar19 6):h1258–h. pmid:25791983
  32. 32. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(2):65–76. pmid:20957426
  33. 33. Lee J, Brown A. Assessing Teacher Performance in Conflict and Crisis Settings: Qualities of the Teacher Classroom Observation (TCO) Tool in Nigeria and Tanzania [Technical Report]. 2021.
  34. 34. Zolotor AJ, Runyan DK, Dunne MP, Jain D, Péturs HR, Ramirez C, et al. ISPCAN Child Abuse Screening Tool Children’s Version (ICAST-C): Instrument development and multi-national pilot testing. Child Abuse & Neglect. 2009;33(11):833–41. pmid:19857897
  35. 35. Barrett-Lennard GT. The Relationship Inventory: A complete resource and guide. The Relationship Inventory: A complete resource and guide.: Wiley-Blackwell. ix, 186–ix, 186.; 2015.
  36. 36. Tschannen-Moran M, Hoy AW. Teacher efficacy: capturing an elusive construct. Teaching and Teacher Education. 2001;17(7):783–805.
  37. 37. Kyegombe N, Namakula S, Mulindwa J, Lwanyaaga J, Naker D, Namy S, et al. How did the Good School Toolkit reduce the risk of past week physical violence from teachers to students? Qualitative findings on pathways of change in schools in Luwero, Uganda. Social Science & Medicine. 2017;180:10–9. pmid:28314229
  38. 38. Naker D. Preventing violence against children at schools in resource-poor environments: Operational culture as an overarching entry point. Aggression and Violent Behavior. 2019;47:268–73.
  39. 39. McFarlane J, Karmaliani R, Maqbool Ahmed Khuwaja H, Gulzar S, Somani R, Saeed Ali T, et al. Preventing Peer Violence Against Children: Methods and Baseline Data of a Cluster Randomized Controlled Trial in Pakistan. Glob Health Sci Pract. 2017;5(1):115–37. pmid:28351880
  40. 40. Davey C, Hassan S, Bonell C, Cartwright N, Humphreys M, Prost A, et al. Gaps in Evaluation Methods for Addressing Challenging Contexts in Development. 2017.