Figures
Abstract
Background
Teachers are at heightened risk of poor mental health and well-being, which is likely to impact on the support they provide to students, and student outcomes. We conducted a cluster randomised controlled trial, to test whether an intervention to improve mental health support and training for high school teachers led to improved mental health and well-being for teachers and students, compared to usual practice. We also conducted a cost evaluation of the intervention.
Methods and findings
The intervention comprised (i) Mental Health First Aid training for teachers to support students; (ii) a mental health awareness session; and (iii) a confidential staff peer support service. In total 25 mainstream, non-fee-paying secondary schools stratified by geographical area and free school meal entitlement were randomly allocated to intervention (n = 12) or control group (n = 13) after collection of baseline measures. We analysed data using mixed-effects repeated measures models in the intention-to-treat population, adjusted for stratification variables, sex, and years of experience. The primary outcome was teacher well-being (Warwick-Edinburgh Mental Well-being Scale). Secondary outcomes were teacher depression, absence, and presenteeism, and student well-being, mental health difficulties, attendance, and attainment. Follow-up was at months 12 (T1) and 24 (T2). We collected process data to test the logic model underpinning the intervention, to aid interpretation of the findings. A total of 1,722 teachers were included in the primary analysis. Teacher well-being did not differ between groups at T2 (intervention mean well-being score 47.5, control group mean well-being score 48.4, adjusted mean difference −0.90, 95% CI –2.07 to 0.27, p = 0.130). The only effect on secondary outcomes was higher teacher-reported absence among the intervention group at T2 (intervention group median number of days absent 0, control group median number of days absent 0, ratio of geometric means 1.04, 95% CI 1.00 to 1.09, p = 0.042). Process measures indicated little change in perceived mental health support, quality of relationships, and work-related stress. The average cost of the intervention was £9,103 per school. The study’s main limitations were a lack of blinding of research participants and the self-report nature of the outcome measures.
Conclusions
In this study, we observed no improvements to teacher or student mental health following the intervention, possibly due to a lack of impact on key drivers of poor mental health within the school environment. Future research should focus on structural and cultural changes to the school environment, which may be more effective at improving teacher and student mental health and well-being.
Author summary
Why was this study done?
- Teachers have poorer mental health and well-being than other professions.
- Poor mental health among teachers has a negative impact on student mental health, possibly through poor quality relationships.
- Mental health interventions have been introduced and evaluated in schools, but none have focused on teachers’ own mental health.
What did the researchers do and find?
- We designed and tested an intervention in UK high schools that provided mental health training and a peer support service for teachers.
- We recruited 25 schools and randomly assigned them either to a group that received the intervention or to a comparison group who did not receive any additional mental health support.
- Two years after the intervention, we did not see any differences in mental health and well-being among teachers or students at the intervention schools compared to the comparison schools. Teachers in the intervention group had an average well-being score of 47.5 out of 70, and those in the comparison group had an average score of 48.4 out of 70.
What do these findings mean?
- The intervention that we introduced into the schools did not have an impact on teacher or student mental health and well-being.
- This may be because the intervention did not successfully increase how supported teachers and students felt, or improve the quality of relationships.
- It is important to keep trying to find interventions that improve teacher and student mental health. Possibly interventions that have a bigger impact on school culture—including the quality of relationships and level of perceived support—would be more effective.
Citation: Kidger J, Turner N, Hollingworth W, Evans R, Bell S, Brockman R, et al. (2021) An intervention to improve teacher well-being support and training to support students in UK high schools (the WISE study): A cluster randomised controlled trial. PLoS Med 18(11): e1003847. https://doi.org/10.1371/journal.pmed.1003847
Academic Editor: Mark Tomlinson, Stellenbosch University, SOUTH AFRICA
Received: May 3, 2021; Accepted: October 12, 2021; Published: November 11, 2021
Copyright: © 2021 Kidger et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available at the University of Bristol data repository, data.bris, at https://doi.org/10.5523/bris.2bfr2zmy03bio2wslt3673ey9n.
Funding: This study was funded by the National Institute for Health Research’s Public Health Research programme 13/164/06 (JK, WH, RE, RA, RC, TF, DG, SM) https://www.nihr.ac.uk/. The intervention costs were covered by Public Health England (JK, WH, RE, RA, RC, TF, DG, SM), Public Health Wales (JK, WH, RE, RA, RC, TF, DG, SM) and Bristol City Council (JK, WH, RE, RA, RC, TF, DG, SM). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: CACE, Complier Average Causal Effect; CAMHS, Child and Adolescent Mental Health Services; EAP, Employee Assistance Programme; FSM, free school meal; HSC, Healthy Schools Coordinator; ICC, intracluster coefficient; MAR, missing at random; MHFA, Mental Health First Aid; MNAR, missing not at random; PHQ, 8-item Patient Health Questionnaire; RCT, randomised controlled trial; SDQ, Strengths and Difficulties Questionnaire; WEMWBS, Warwick-Edinburgh Mental Well-being Scale; WISE, Well-being in Secondary Education; WPAI, Work Productivity and Activity Impairment
Introduction
Teachers are at risk of poor mental health and well-being [1,2]. Causes of work-related stress include excessive workload, challenging student behaviour, and pressure to meet an increasing number of externally determined targets [1,3]. Failure to support teachers may lead to longer-term mental health problems, presenteeism (going to work when one is mentally or physically ill), and sickness absence or quitting the profession [1,4]. Further, teachers’ mental health and well-being is associated with student mental health and well-being [5]. Supportive relationships with school staff can be protective of future depression for students in secondary schools [6], whereas poor teacher–student relationships among this age group predict mental health problems and future exclusion [7].
Most school mental health intervention studies have focused on classroom based psychological or educational approaches, with mixed evidence of effectiveness [8]. There is some evidence that whole-school approaches to improving student social, emotional, and/or behavioural adjustment, combining curriculum content, changes to the whole school ethos, and parental involvement, are successful [9]. Far fewer studies have focused on teachers’ skills in supporting vulnerable students, although evidence from 2 recent United Kingdom cluster randomised controlled trials (RCTs) has shown that enhancing teacher–student support can improve student outcomes [10,11]. To our knowledge, no school-based trials to date have targeted teacher mental health or well-being as the primary outcome. One systematic literature review examined 4 studies that aimed to support teachers’ mental health [12]. Results indicated that changes to the school environment including provision of mentoring could have a positive effect, but quality of the evidence was poor.
Given the evidence of the importance of better support for teachers, both for their own and their students’ mental health outcomes, we developed the WISE (Well-being in Secondary Education) intervention, which aimed to improve the support available to teachers for their own mental health and well-being and to increase their skills in supporting students [13]. The intervention makes use of Mental Health First Aid (MHFA) [14], and MHFA for Schools and Colleges, which is a version developed specifically in England for staff in schools. MHFA has been shown to improve social support, and increase lay people’s knowledge and confidence in helping others in crisis, and has high participant acceptability [14–17]. However, a recent review found a lack of evidence for impact of MHFA training on mental health outcomes [18]. Drawing on social support theory, we hypothesised that improving workplace social support would have a positive impact on teacher well-being and mental health [19,20]. We further hypothesised, based on previous findings that teachers’ own well-being may affect their abilities to support students both emotionally and academically [21,22], that improvements to teachers’ own mental health and well-being should lead to more positive teacher–student relationships, which is associated with improved student mental health [5].
We tested the acceptability and feasibility of the intervention in a pilot study of 6 schools, through observations of the training, interviews/focus groups with training attendees and the wider staff body, and surveys of staff and students [13]. This pilot study found that schools and staff within schools were willing and able to participate in the intervention, that the MHFA training was perceived to be appropriate for the British secondary school context, that it improved mental health knowledge and attitudes, and that the peer support service was feasible to establish and perceived to be helpful. Barriers to the success of the peer support service were identified, which we addressed in this main study. Specifically, we strengthened the guidance provided to peer support services relating to advertising and confidentiality procedures, we increased the number of peer supporters in each school, we extended the length of intervention time by 1 year, and we emphasised the importance of senior leadership support. We also added shorter training component delivered to the whole teaching team, which included information about the peer support services.
We undertook a cluster RCT of the WISE intervention [20] to evaluate its effectiveness at improving teacher and student well-being and mental health, reducing teacher absence, presenteeism, and quitting the profession, and improving student attendance and academic performance, compared to usual practice and the costs associated with this. The trial included an embedded process evaluation [23], which explored how well the intervention was implemented and used in schools (reported elsewhere) [24], and the extent to which the key mechanisms of change hypothesised in the logic model (Fig 1)—increased access to support for staff, improved staff–student relationships, improved mental health support for students, and reduced stress relating to supporting students—were achieved. This paper includes findings relating to this latter process question, reporting on the second and fourth process outcomes shown in the logic model. Forthcoming findings will report on the qualitative process data, which explored perceptions of the intervention in more depth and which, along with the implementation paper [24], relate to the first and third process outcomes.
CAMHS, Child and Adolescent Mental Health Services; MHFA, Mental Health First Aid; WISE, Well-being in Secondary Education.
Methods
Study design and participants
Twenty-five mainstream secondary schools (regular schools for students aged 11 to 18 where individuals with and without special educational needs are taught together) were recruited in 2 geographical areas—a city in South West England and surrounding area (comprising a sampling frame of 64 schools) and Central South Wales/South East Wales (comprising a sampling frame of 88 schools). These areas were selected due to proximity to the research team but provided a wide range of schools in terms of academic attainment, inspectorate rating, and socioeconomic catchment areas. Schools were excluded if they were fee paying, alternative provision (that is provision for students unable to attend mainstream school, for example, due to learning or physical disability or because they have been excluded from mainstream education), WISE pilot study schools, participating in similar research studies, already delivering MHFA or other mental health training, or lacking data on proportion of students eligible for free school meals (FSMs) as a proxy measure for poverty. We also excluded schools within the same academy trust (governing body) and local authority as one that had already been recruited, to avoid risk of contamination.
Our study population within each school was all teaching staff and all students in year 8 (aged 12 to 13 years) at baseline. We assumed that any effect on this year group would be representative of effects for all students, as the intervention was delivered at the school level. There were no inclusion or exclusion criteria for teachers or students in year 8. At the follow-up time points, a small number were ineligible to receive the questionnaire, due to having left the school, or being absent from school long term (for example, due to long-term sickness).
Signed research agreements consenting school participation in the study were obtained from all head teachers or their representatives. Teacher and student participants were given information sheets outlining how to opt out of participation—instructions stated that those who did not wish to complete the questionnaire should place the blank copy into the envelope provided and return to the study team. Consent was then assumed for those completing questionnaires. Parents/carers of students were given the option to withdraw their child from all research activities.
No serious adverse events were recorded during any of the data collections. Ethical approval was granted by the University of Bristol’s Faculty of Health Sciences Research Ethics Committee (FREC reference number: 28522).
Randomisation and masking
Schools were the unit of allocation as the intervention operated at the whole-school level. In Wales, the sample was divided into the 2 administrative regions and further stratified into 3 FSM eligibility levels (high, medium, and low proportion of students eligible for FSMs compared to the national average). Two schools were randomly selected from each stratum in each region and invited to participate. Schools that declined were replaced by a randomly selected school from the same stratum and region. In England, schools that were interested in participating following advertisement to all head teachers were stratified according to city/non-city location and high, medium, and low levels of FSM; 2 schools within each location and stratum were then randomly selected for participation. Following discussion about the possibility of school dropout, we randomly selected one additional school for participation from the English sampling frame. In total, there were 12 participating schools in Wales and 13 participating schools in England.
This sampling procedure resulted in 2 recruited schools within each location (England city/England non-city/South Central Wales/South East Wales) and FSM stratum (low/medium/high) combination. Within each of these pairings, one school was randomly allocated to each study arm. Block randomisation was undertaken by the study statistician, who was blinded to the schools’ identities, using the Stata ralloc command. This process resulted in 12 intervention schools (6 in Wales and 6 in England) and 13 comparison schools (6 in Wales and 7 in England). Random allocation to study arm took place after baseline data collection.
Schools and the study team including those who collected the questionnaire data were not blind to allocation at the follow-up data collections. In the case of school staff, this was because it was not possible to conceal the intervention delivery from them. In the case of the data collections, these staff had also been the ones to organise the intervention delivery due to resource limitations.
Procedures
All data collections were conducted in school, led by a study team member, accompanied by trained fieldworkers. Individual-level teacher outcomes were collected through self-report paper questionnaires during staff meetings in May to July 2016 (T0, preintervention), May to July 2017 (T1, 1-year follow-up), and May to July 2018 (T2, 2-year follow-up). Those who were absent were offered the option of completion via an online survey. Individual-level student outcomes were gathered via self-report paper questionnaires at T0 and T2 during tutor groups or lesson time. Where less than 5 students in a tutor group were absent, the paper questionnaires were left for them to complete when next in school, under the supervision of school staff. Where more than 5 were absent, a second data collection session administered by a study team member was arranged. At least one teacher was present in the room during student data collections to adhere to school child protection policies and assist with behaviour management. They were given a written information sheet explaining the data collection process and requesting that they did not look at the students’ questionnaires or involve themselves with the data collections.
Data on school-level teacher and student outcomes were collected from schools at T0 and T2.
Each intervention school received the following: (i) MHFA training for 8% staff who then provided a confidential peer support service for colleagues; (ii) MHFA for Schools and Colleges training for a further 8% teachers; and (iii) a 1-hour mental health awareness session for staff. In English schools, these sessions were delivered by an independent MHFA trainer. In Welsh schools, Healthy Schools Coordinators (HSCs) who are employed by Public Health Wales to support schools in health improvement work were trained to deliver the sessions. We planned for the HSCs and school link workers based within Child and Adolescent Mental Health Services (CAMHS) to provide ongoing support to the peer support services once they were set up. Further details of the intervention are shown in Fig 2 and are reported in the protocol [20].
ALGEE, Assess for risk, Listen nonjudgmentally, Give information, Encourage self-help, Encourage professional help if needed; MHFA, Mental Health First Aid; WISE, Well-being in Secondary Education.
Schools allocated to the comparison group continued with usual practice in terms of teacher support and training. In England and Wales, there is no standard training package for teachers regarding mental health support, although schools are free to choose to pay for any such training that is available. Similarly, there is no standard mental health support available to staff, although schools are free to buy such a service in, for example, there are several Employee Assistance Programmes (EAPs) available in the UK for workplaces.
Outcomes
Primary outcome.
The primary outcome was teacher well-being, measured using the 14-item Warwick-Edinburgh Mental Well-being Scale (WEMWBS) [25]. The WEMWBS incorporates elements of both subjective and psychological well-being, represented in 14 statements such as “I’ve been feeling cheerful” and “I’ve been feeling interested in other people.” For each statement, respondents select one out of five options: none of the time/rarely/some of the time/often/all of the time. Each response is scored from 1 (none of the time) to 5 (all of the time), and the total is summed to make up a well-being score of between 14 and 70, where higher scores represent better well-being. The scale has been validated in the general UK adult population, with a Cronbach’s alpha score of 0.91, and test–retest reliability at 1 week of 0.83 [21].
Secondary outcomes.
Teacher depression was measured using the 8-item Patient Health Questionnaire (PHQ-8) [26]. Items are summed to give a score between 0 and 24; a cut-point of 10 or more indicates moderate to severe depression.
Teacher absence was measured at the individual level by self-report (“number of days missed from school because of health problems in the past four working weeks”) and at the school level by data provided by schools (total number of days of absence over the previous year).
Teacher presenteeism was measured by self-report, using the relevant item from the Work Productivity and Activity Impairment (WPAI) questionnaire [27], adapted to fit teachers’ work schedule. Teachers were asked “during the last 4 working weeks, how much did health problems affect your productivity while you were working?” and selected a number between 0 (no effect) to 10 (completely prevented me from working).
Teacher retirements and leaving for other reasons were collected at the school level as total number for the previous year.
Student well-being was measured using the 14-item WEMWBS, which has been validated among teenagers from 13 years [28].
Student psychological distress was measured using the Strengths and Difficulties Questionnaire (SDQ), short form version without the impact assessment [29]. A total difficulties score was calculated, which can range from 0 to 40, with a higher score indicating greater difficulties.
Student attendance was collected at the school level from routine data sources (www.compare-school-performance.service.gov.uk for England and www.mylocalschool.wales.gov.uk for Wales).
Student attainment at end of year 11 was collected at the school level from the same routine data sources. Exams at the end of statutory school age (key stage 4) are graded and reported differently for the 2 countries. For English schools, we reported % achieving grade 5 or above in both English and Maths, and for Welsh schools, we reported % achieving grade C or above in English/Welsh, Maths, and Science.
Process outcomes
Process outcomes were collected in the same teacher and student surveys as the outcome measures. The teacher survey contained 5 questions, and the student survey contained 3 items, all of which were separate items as opposed to items on one scale.
Teacher perceived stress at work: We included the following question in the teacher survey, “In general, how stressful do you find your job?” [response options: not at all stressful, mildly stressful, moderately stressful, very stressful, extremely stressful].
Teacher perceived support and relationships: We included a question in the teacher survey, “Rate how much you agree with the following statements”: this school cares about staff well-being/this school cares about student well-being/staff generally have good relationships with each other in this school/teachers and students generally have good relationships in this school [response options: strongly agree, agree, disagree, strongly disagree].
Student perceived support and relationships: We included a question in the student survey, “Rate how much you agree with the following statements”: this school cares about student well-being/teachers and students generally have good relationships in this school/my teachers are there for me when I need them [response options: strongly agree, agree, disagree, strongly disagree].
Economic evaluation
The economic analysis took a public sector perspective, calculating the financial and opportunity costs to schools of participating in the WISE intervention. A proforma was completed after each training session documenting the resources used for trainer fees, HSCs’ time (based on salary), trainer and HSC travel expenses, MHFA handbook fees, and the time of the teachers attending training (based on salary).
Statistical analysis
Full details of the statistical analysis plan are published on the study website (https://www.bristol.ac.uk/population-health-sciences/projects/wise/).
The study was powered to detect a mean change of 3 points on the teacher WEMWBS score, chosen as the minimum meaningful change [30]. We assumed a mean of 50 teacher responses per school (with a coefficient of variation of sample size of 0.5)—which reflects the response rate of approximately 75% of teachers achieved in the pilot study [13]—and an SD for WEMWBS of 8.4 and an intracluster coefficient (ICC) of 0.01 (based on the pilot data [13]). A sample size of 24 schools (12 intervention and 12 control) would achieve 83% power for an ICC of 0.05. Sample sizes were calculated using the Stata clustersampsi command.
Analyses were carried out under the intention to treat principle. For the primary analysis and secondary teacher individual-level outcomes, mixed effects regression was used to compare outcome by arm over the course of follow-up (T0, T1, and T2), adjusted for stratification variables, sex, years of experience, and time. This included a random effect for individual participants, and another for school. Using a maximum likelihood estimator this analysis was robust to data missing at random (MAR) and allowed inclusion of every teacher who had at least one measure of the outcome. The primary analysis was repeated with a treatment by time interaction term added to the model to enable estimation of effect at T1 and T2 separately.
For all student individual-level outcomes and all school-level outcomes, the outcome at T2 was regressed on treatment arm and T0 of the outcome, accounting for clustering by school (using a random effect). All secondary outcome models were adjusted for stratification variables, and the individual student-level models were additionally adjusted for sex and ethnicity.
The impact of missingness and potential departure from MAR (i.e., missing not at random (MNAR)), on teacher WEMWBS and PHQ-8, and student WEMWBS and SDQ outcomes was examined by repeating analyses on multiply imputed data, which incorporated scaling parameters allowed to differ between arms [31].
Complier Average Causal Effect (CACE), using Instrumental Variable analysis [32], was used to investigate efficacy of the intervention (based on completion of MHFA training) for comparison with the primary analysis estimate of effectiveness. Adjustment was made for both school-level and individual-level covariates, as in the primary analysis.
To explore whether the hypothesised mechanisms of change—perceived support to staff and students and quality of staff–student relationships—differed by intervention arm at follow-up, all the process evaluation single items were recoded into binary variables (agree/disagree). Logistic regression models (one per item) were used to compare outcomes for these variables between arms at T2, adjusted for T0 scores, school-level FSM, and geographical area.
Analyses were conducted using Stata (version 14, College Station, TX: StataCorp LP). Findings are reported in line with CONSORT guidelines extension to cluster randomised trials.
We made the following changes to the published protocol [20]: Some levels within the FSM strata were merged during recruitment due to the limited number of schools available (specifically medium and high were merged for the English sampling frame, and medium and low were merged for the Welsh sampling frame); we recruited one extra school in case of drop out, and outcome measures at 3 schools were collected in teachers’ own time rather than during meeting time. All changes were approved by the Trial Steering Committee.
Results
All 25 schools remained in the trial until the end and participated in all 3 data collections. Participant flow through the trial is shown in the CONSORT chart (Fig 3). Some teachers were not eligible for data collection at each time point, for example, if they left the school, joined the school midway through the study, or were on maternity leave. Teacher response rates were above 75% at each time point but were slightly lower in intervention schools at T1 and T2 compared to controls. The total number of teachers included in the primary analysis—that is those who had outcome data at one or more time point—was 1,722.
Schools were well balanced across study arms for most teacher, student, and school-level characteristics at baseline (Table 1). Mean total difficulties score was slightly higher among control school students. Intervention schools had a considerably higher median number of teachers leaving for nonretirement reasons and were more likely to have national average or above in student attainment.
The ICC for the primary outcome teacher well-being at baseline was 0.021 (95% CI 0.007, 0.062).
Table 2 shows the results for all primary and secondary outcomes. There was no evidence of a difference between arms in mean teacher mental well-being over the course of follow-up: unadjusted difference in means −0.48, 95% CI −1.63, 0.67, p-value 0.412 and adjusted difference in means −0.90, 95% CI −2.07, 0.27, p-value 0.130. There was also no evidence that treatment effect varied over time (p-value for interaction term 0.654). Well-being appeared to improve in both arms over time, with a larger improvement in the control arm initially, as shown in Fig 4.
WEMWBS, Warwick-Edinburgh Mental Well-being Scale.
There was no evidence of a difference between intervention and control groups in teacher reported depression or presenteeism. There was weak evidence of a difference between groups in self-reported teacher absenteeism over the course of follow-up; in the adjusted model, the number of days of absence was 4% higher in those teachers in the intervention group compared to those in the control (unadjusted ratio of geometric means 1.03, 95% CI 0.98, 1.07, p-value 0.308 and adjusted ratio of geometric means 1.04, 95% CI 1.00, 1.09, p-value 0.042). There was no evidence of a difference in student mental health outcomes (WEMWBS or SDQ) between intervention and control groups at T2.
There was also no evidence of a difference between intervention and control groups in any of the school recorded outcomes (teacher absenteeism, teacher retirement, teachers leaving for other reasons, student attainment, and student attendance). There were more teachers retiring in the intervention schools, although this may be explained by participants being older (56% of teachers in the intervention arm had worked in a school for more than 10 years compared to 52% in the control arm at T0).
Under an MAR assumption, there was no difference in conclusion drawn from the analysis of the imputed data compared to the primary analysis (Tables A-D in S1 Text).
There was no evidence of a treatment effect for the primary outcome WEMWBS in those who attended MHFA training, although there was a weak tendency in the direction favouring participants in the control schools in the adjusted analysis (unadjusted mean difference −2.51, 95% CI −7.51, 2.49, p = 0.326, adjusted mean difference −2.26, 95% CI −4.93, 0.40, p = 0.096).
Table 3 shows the results for all mechanisms of change outcomes. There was no evidence of a difference between intervention and control groups at T2 with respect to teacher self-reported stress at work, or teachers’ perceptions of school attitude towards student well-being, and quality of relationships between staff or between teachers and students. In the unadjusted model, teachers in the control group were more likely to agree that school cares about staff well-being (OR = 0.48, 95% CI 0.28, 0.82, p = 0.01), but this difference was attenuated once adjustment had been made for T0 scores and stratifying variables (adjusted OR = 0.58, 95% CI 0.29, 1.19, p = 0.14). There was also no difference by study arm at T2 in student perceptions of whether their school cares about student well-being or teachers being there for them when needed. There was a difference in favour of the control arm in students who agreed that their school had good teacher-student relationships in both the unadjusted and adjusted models (unadjusted OR = 0.82, 95% CI 0.70, 0.95, p = 0.01 and adjusted OR = 0.80, 95% CI 0.66, 0.96, p = 0.02).
The average cost of the intervention was £9,103 per intervention school. The cost was slightly lower in English schools (£8,263) than in Welsh schools (£9,943), primarily due to the upfront cost of training the HSCs in Wales. Staff salaries and costs for supply teachers (£5,566) accounted for the majority (61%) of the total cost of the intervention. There was no evidence that this additional cost was justified by improvement in teacher well-being, depressive symptom scores, presenteeism or absenteeism, or by any of the student reported outcomes.
Discussion
We found no evidence that the WISE intervention had an effect on the primary outcome teacher well-being; in both arms, there was an increase in well-being over time, with this increase being larger in control schools in the first year of the intervention. We also found no intervention effects for teacher depression or presenteeism, or student well-being, psychological difficulty, attendance, or attainment. There was one small effect in favour of the control arm for teacher absence in the fully adjusted models. However, this was not a large effect and may have been a chance finding due to the number of analyses undertaken. Although the total mean cost of the intervention per school was relatively small, we found insufficient evidence that it was justified by any improvements in staff or student outcomes at the individual or school level.
Our process evaluation findings showed no difference between groups at the study end regarding teacher stress at work, how far teachers felt schools cared about well-being, and teachers’ perceptions of the quality of relationships. Among students, there was a general pattern across both groups of a decline in positive perceptions regarding how far school cares about well-being and whether teachers are there for them when needed. Intervention students were also less likely to agree that teachers and students generally have positive relationships by time two, whereas control students’ views remained constant across the time points. In other words, the intervention did not appear to have had an impact on the mechanisms of change outlined in our logic model, which is likely to at least partially explain the lack of effectiveness of the intervention.
Strengths of the study include the high response rates from both teachers and students, and retention of all recruited schools, which ensured sufficient statistical power to identify important differences in mental health outcomes between study arms. High response rates and the use of multiple imputation in our analysis helped limit the risk of reporting bias due to missing outcome data. The collection of baseline data before randomisation further helped to mitigate risk of bias. The stratified approach to recruitment helped ensure balance across study arms at baseline, with the exception of student attainment and teachers leaving for reasons other than retirement. This stratification process also ensured that the study sample spanned the full range of socioeconomic catchment areas of schools in England and Wales (as measured by FSM eligibility), and results were similar in both the English and the Welsh schools, suggesting that findings are likely to be generalisable across all mainstream schools in these 2 countries.
The lack of blinding of participants was a limitation, as we cannot be sure that knowing they were in an intervention school did not affect teachers’ responses to the self-report surveys. For example, if participating in the intervention led to raised expectations regarding changes in school, and if these expectations were not perceived to be met, then teachers in intervention schools may have given more negative responses about their school environment or their own mental health and well-being at follow-up. A second problem with using self-report measures of well-being and mental health is the risk that gaining knowledge about mental health through the intervention itself may affect how individuals then report their own mental health [33]. A third limitation for school-level outcomes was that we did not always have data from all schools, leading to a lack of precision in some of the secondary outcomes, and potential for bias if these were not MAR. Finally, although we planned to work with school link workers based within CAMHS in England and with the HSCs in Wales to provide ongoing support to the peer supporters, this did not happen in practice. In England, this was because the school link worker role was not filled for many of our study schools, and in Wales, this was because the HSCs who had the capacity to take on the trainer roles were not those who were supporting the study schools. This limitation is unlikely to explain the study’s main finding that overall teacher well-being did not improve, but it may have contributed to the peer support services losing some sense of momentum and being fully embedded in school life, as discussed below.
There are a number of possible explanations for the null finding reported here. First, the intervention may not have been sufficiently differentiated from usual practice, particularly given the context of the UK government’s focus on schools as a setting for improving mental health support [34]. We did see improvements in teacher well-being in both arms, as shown in Fig 4, which may be due to an effect of national policy, although the lack of prescriptive guidance on how schools should support teacher mental health training and a lack of focus on teacher well-being perhaps makes this unlikely. A second explanation is that a MHFA training and peer support intervention is not an effective way in which to address the mental health training and support needs of teachers. Findings from our pilot study indicated that the MHFA training was effective at improving knowledge, skills, and confidence in supporting mental health and that the peer support service increased the opportunity for staff to gain support from colleagues [13]. However, evidence from the process findings reported here indicate that this increase in knowledge, skills, confidence, and available support did not translate into change at the population and whole-school level regarding improved experience of support for mental health, improved quality of relationships, or reduced stress in the workplace. One reason for this may have been a problem with the implementation of the intervention. Previously published evidence from this part of our process evaluation reported implementation to be good overall, but it also revealed some difficulties both in delivering the full length, uninterrupted training to enough staff in all schools, and in embedding the peer support service into school life [24]. This led to limitations in dose and reach which may have meant that the intervention as it was delivered in practice was not sufficient to create whole-school, sustained changes, although it may have had positive effects for certain individuals.
Reflecting further on these implementation findings, it is possible that the context in which the intervention was delivered contained barriers both to its successful implementation and to the intervention having a large enough impact. It is increasingly acknowledged that context must be taken into account when evaluating real world public health interventions, due to its key role in determining the success of an intervention being implemented and having an impact [35]. In the case of schools, various aspects of the context have been found to be associated with both implementation of mental health interventions [36] and with student and teacher mental health outcomes [2,37]. We attempted to address such contextual factors in the present intervention through strengthening supportive relationships and giving mental health a higher profile, but our process outcome findings indicate that it was not successful in this regard. We reported previously that the challenges encountered in implementation were due at least in part to support from senior leadership waning over time [24]. Visible and meaningful support from senior leaders is likely to be an important factor in the creation of a school culture that enables mental health interventions to thrive and have an impact. Our sample of teachers had poor well-being compared to the general UK working population [5], and, elsewhere, teachers have noted the challenging environment in which they work [3,38,39], describing a culture of not feeling able to ask for help or disclose problems [1,3]. If those in the intervention schools had the promise of improved support, only to find that not enough changed in terms of the challenging culture, it is perhaps not surprising that there was no sizeable improvement in well-being.
In conclusion, poor teacher mental well-being and the knock-on effects for student learning and mental health remain important public health issues, particularly in the aftermath of the COVID-19 pandemic and related disruptions to school life, which have created additional stressors for teachers [40] and concerns for children and young people’s mental health and well-being [41]. Existing evidence indicates that peer support is important for well-being in the workplace [42–44], including for teachers [12, 45], but rigorous evaluation of workplace peer support interventions is lacking. Further, a need for improving teachers’ skills in supporting student mental health has been identified [3,34], but previous studies have focused on this without also explicitly including support for teacher mental health [22]. To our knowledge, we conducted the first ever cluster RCT of an intervention to improve teacher and student mental health, comprising a staff peer support service and MHFA training. The intervention was not effective compared to usual practice, possibly due to the challenges of introducing an intervention that has a large and sustainable impact at the whole-school level. More research is needed to identify effective approaches to improve teacher mental health. This should include interventions that focus on addressing the challenging context and culture of schools, which can often be detrimental to mental health and well-being. Policy changes to reduce the structural determinants of poor teacher mental health—including high workload and performance management based on academic outcomes—and assessment of mental health support as part of government inspections of schools to ensure that it is prioritised would help create a wider context in which mental health interventions could prosper.
Supporting information
S1 Text. Supporting information: results from imputed texts.
Table A. Results from imputed models for teacher WEMWBS outcome. Table B. Results from imputed models for teacher PHQ-8 outcome. Table C. Results from imputed models for student WEMWBS outcome. Table D. Results from imputed models for student SDQ outcome.
https://doi.org/10.1371/journal.pmed.1003847.s001
(DOCX)
Acknowledgments
The work was undertaken with the support of The Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), a UK Clinical Research Collaboration (UKCRC) PHR Centre of Excellence. This study was designed and delivered in collaboration with the Bristol Randomised Trials Collaboration (BRTC), a UKCRC Registered Clinical Trials Unit in receipt of NIHR Clinical Trials Unit support funding, and we gratefully acknowledge support from Jodi Taylor in providing ongoing trials advice.
We also extend our thanks to the Schools Health Research Network in Wales and local public health partners in England for their support in recruiting schools. We gratefully acknowledge the invaluable contribution of Hannah Baber (Trial Manager from 2018). We thank Aideen Ahern for contributing to the design and conduct of the economic analysis, Dr Mari-Rose Kennedy for support with data analysis, and Professor Kate Tilling for early advice regarding study design and statistical analysis plan. Finally, our thanks go to all the staff and student participants of the study, and the MHFA trainers and Healthy Schools Coordinators for their contributions to intervention delivery.
References
- 1.
Education Support Partnership. Teacher Wellbeing Index 2019. London: Education Support; 2019.
- 2. Kidger J, Brockman R, Tilling K, Campbell R, Ford T, Araya R, et al. Teachers’ wellbeing and depressive symptoms, and associated risk factors: A large cross sectional study in English secondary schools. J Affect Disord. 2016;192:76–82. pmid:26707351
- 3. Kidger J, Gunnell D, Biddle L, Campbell R, Donovan J. Part and parcel of teaching? Secondary school staff’s views on supporting student emotional health and well-being. Br Educ Res J. 2009;36(6):919–35.
- 4. Henderson M, Harvey SB, Øverland S, Mykletun A, Hotopf M. Work and common psychiatric disorders. J R Soc Med. 2011;104(5):198–207. pmid:21558098
- 5. Harding S, Morris R, Gunnell D, Ford T, Hollingworth W, Tilling K, et al. Is teachers’ mental health and wellbeing associated with students’ mental health and wellbeing? J Affect Disord. 2019;242:180–7. pmid:30189355
- 6. Kidger J, Araya R, Donovan J, Gunnell D. The effect of the school environment on the emotional health of adolescents: a systematic review. Pediatrics. 2012;129(5):925–49. pmid:22473374
- 7. Lang I, Marlow R, Goodman R, Meltzer H, Ford T. Influence of problematic child-teacher relationships on future psychiatric disorder: population survey with 3-year follow-up. Br J Psychiatry. 2013;202(5):336–41. pmid:23637109
- 8. Caldwell DM, Davies SR, Hetrick SE, Palmer JC, Caro P, López-López JA, et al. School-based interventions to prevent anxiety and depression in children and young people: a systematic review and network meta-analysis. Lancet Psychiatry. 2019;6(12):1011–20. pmid:31734106
- 9. Goldberg JM, Sklad M, Elfrink TR, Schreurs KMG, Bohlmeijer ET, Clarke AM. Effectiveness of interventions adopting a whole school approach to enhancing social and emotional development: a meta-analysis. Eur J Psychol Educ. 2019;34:755–82.
- 10. Bonell C, Allen E, Warren E, McGowan J, Bevilacqua L, Jamal F, et al. Effects of the Learning Together intervention on bullying and aggression in English secondary schools (INCLUSIVE): a cluster randomised controlled trial. Lancet. 2018;392(10163):2452–64. pmid:30473366
- 11. Ford T, Hayes R, Byford S, Edwards V, Fletcher M, Logan S, et al. The effectiveness and cost-effectiveness of the Incredible Years Teacher Classroom Management programme in primary school children: results of the STARS cluster randomised controlled trial. Psychol Med. 2019;49(5):828–42. pmid:30017006
- 12. Naghieh A, Montgomery P, Bonell CP, Thompson M, Aber JL. Organisational interventions for improving wellbeing and reducing work-related stress in teachers. Cochrane Database Syst Rev. 2015. pmid:25851427
- 13. Kidger J, Stone T, Tilling K, Brockman R, Campbell R, Ford T, et al. A pilot cluster randomised controlled trial of a support and training intervention to improve the mental health of secondary school teachers and students–the WISE (Wellbeing in Secondary Education) study. BMC Public Health. 2016;16(1):1060. pmid:27716226
- 14. Kitchener BA, Jorm AF. Mental health first aid training in a workplace setting: a randomized controlled trial [ISRCTN13249129]. BMC Psychiatry. 2004;4(1):23.
- 15. Bovopoulos N, LaMontagne A, Martin A, Jorm A. Delivering mental health first aid training in Australian workplaces: exploring instructors’ experiences. Int J Ment Health Promot. 2016;18(2):65–82.
- 16. Jorm AF, Kitchener BA, Sawyer MG, Scales H, Cvetkovski S. Mental health first aid training for high school teachers: a cluster randomized trial. BMC Psychiatry. 2010;10(1):51. pmid:20576158
- 17. Kelly CM, Mithen JM, Fischer JA, Kitchener BA, Jorm AF, Lowe A, et al. Youth mental health first aid: a description of the program and an initial evaluation. Int J Ment Health Syst. 2011;5(1):4. pmid:21272345
- 18. Hadlaczky G, Hökby S, Mkrtchian A, Carli V, Wasserman D. Mental Health First Aid is an effective public health intervention for improving knowledge, attitudes, and behaviour: A meta-analysis. Int Rev Psychiatry. 2014;26(4):467–75. pmid:25137113
- 19. Thoits PA. Mechanisms linking social ties and support to physical and mental health. J Health Soc Behav. 2011;52(2):145–61. pmid:21673143
- 20. Kidger J, Evans R, Tilling K, Hollingworth W, Campbell R, Ford T, et al. Protocol for a cluster randomised controlled trial of an intervention to improve the mental health support and training available to secondary school teachers–the WISE (Wellbeing in Secondary Education) study. BMC Public Health. 2016;16(1):1089. pmid:27756268
- 21. Hoglund SLG, Klingle KE, Hosan NE. Classroom risks and resources: Teacher burnout, classroom quality and children’s adjustment in high needs elementary schools. J Sch Psychol. 2015;53:337–57. pmid:26407833
- 22. Sisask M, Värnik P, Värnik A, Apter A, Balazs J, Balint M, et al. Teacher satisfaction with school and psychological well-being affects their readiness to help children with mental health problems. Health Educ J. 2014;73(4):382–93.
- 23. Evans R, Brockman R, Grey J, Bell S, Harding S, Gunnell D, et al. A cluster randomised controlled trial of the Wellbeing in Secondary Education (WISE) Project–an intervention to improve the mental health support and training available to secondary school teachers: protocol for an integrated process evaluation. Trials. 2018;19:270. pmid:29728149
- 24. Fisher H, Harding S, Bell S, Copeland L, Evans R, Powell J, et al. Delivery of a Mental Health First Aid training package and staff peer support service in secondary schools: A process evaluation of uptake and fidelity of the WISE intervention. Trials. 2020;21:745. pmid:32847622
- 25. Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5(1):63. pmid:18042300
- 26. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009;114(1–3):163–73. pmid:18752852
- 27. Reilly MC, Zbrozek AS, Dukes EM. The validity and reproducibility of a work productivity and activity impairment instrument. PharmacoEconomics. 1993;4(5):353–65. pmid:10146874
- 28. Clarke A, Friede T, Putz R, Ashdown J, Martin S, Blake A, et al. Warwick-Edinburgh Mental Well-being Scale (WEMWBS): Validated for teenage school students in England and Scotland. A mixed methods assessment. BMC Public Health. 2011;11(1):487.
- 29. Goodman R. Psychometric Properties of the Strengths and Difficulties Questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337–45. pmid:11699809
- 30.
Putz R, O’Hara K, Taggart F, Stewart-Brown S. Using WEMWBS to measure the impact of your work on mental wellbeing: A practice-based user guide. Health Scotland; 2012.
- 31. White IR, Horton NJ, Carpenter J, Pocock SJ. Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ. 2011;342:d40. pmid:21300711
- 32. Shrier I, Steele RJ, Verhagen E, Herbert R, Riddell CA, Kaufman JS. Beyond intention to treat: what is the right question? Clin Trials. 2014;11(1):28–37. pmid:24096636
- 33. Goodman MS, Madni LA, Semple RJ. Measuring mindfulness in youth: Review of current assessments, challenges, and future directions. Mind. 2017;8(6):1409–20.
- 34.
Department of Health and Department for Education. Transforming children and young people’s mental health provision: a green paper [Internet]. 2017 [cited 2021 Jul 30]. Available from: https://www.gov.uk/government/consultations/transforming-children-and-young-peoples-mental-health-provision-a-green-paper
- 35.
Craig P, Di Ruggiero E, Frohlich KL, Mykhalovsky E, White M, on behalf of the Canadian Institutes of Health Research (CIHR)-National Institute for Health Research (NIHR) Context Guidance Authors Group. Taking account of context in population health intervention research: guidance for producers, users and funders of research. Southampton: NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.
- 36. Flannery KB, Kato MM. Implementation of SWPBIS in high school: why is it different? Prev Sch Fail. 2016;61(1):69–79.
- 37. Wong MD, Dosanjh KK, Jackson NJ, Runger D, Dodvitz RN. The longitudinal relationship of school climate with adolescent social and emotional health. BMC Public Health. 2021;21(207). pmid:33485308
- 38. Barmby P. Improving teacher recruitment and retention: the importance of workload and pupil behaviour. Educ Res. 2006;48(3):247–65.
- 39. Chaplain RP. Stress and psychological distress among trainee secondary teachers in England. Educ Psychol. 2008;28(2):195–209.
- 40. Kim L E, Leary R, Asbury K.‘We need clear directions, if we’re going to move forward. It’s as simple as that’: Teachers’ narratives during partial school reopenings in the COVID-19 pandemic. PsyArZiv [Internet]. 2020 [cited 2021 Aug 05]. Available from: https://psyarxiv.com/m8scj/
- 41.
NHS Digital. Mental Health of Children and Young People in England, 2020: Wave 1 follow up to the 2017 survey 2020 [Internet]. [cited 2020 Oct 23]. Available from: https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-of-children-and-young-people-in-england/2020-wave-1-follow-up
- 42. Laine H, Saastamoinen P, Lahti J, Rahkonen O, Lahelma E. The associations between psychosocial working conditions and changes in common mental disorders: a follow-up study. BMC Public Health. 2014;14(1):588. pmid:24916716
- 43. Linnan L, Fisher EB, Hood S. The power and potential of peer support in workplace interventions. Am J Health Promot. 2013;28(1):TAHP2–10. pmid:24137807
- 44. Van F. Peer support: healthcare professionals supporting each other after adverse medical events. BMJ Qual Saf. 2008;17(4):249–52.
- 45.
Bricheno P, Brown S, Lubansky R. Teacher wellbeing: a review of the evidence. Teacher Support Network; 2009.