Effectiveness of home fire safety interventions. A systematic review and meta-analysis

Purpose To assess the effectiveness of Home Fire Safety (HFS) interventions versus other interventions/no interventions/controls on HFS knowledge and behaviour at short-, intermediate- and long-term follow ups. Design Systematic review and meta-analysis of randomized controlled trials. Data sources MEDLINE, EMBASE and PubMed databases were searched from January 1998 to July 2018, and studies retrieved. Participants Toddlers, children (primary or secondary school), teenagers or adults. Interventions/Comparison HFS interventions compared to other interventions / no interventions / controls. Outcomes HFS knowledge and behaviour. Results 10 studies were identified (8 RCTs and 2 prospective cohort). Two studies assessed the effects of HFS interventions vs no interventions on HFS knowledge at up to 4 months follow up in school children and demonstrated significant difference between groups (very low quality, 2 RCTs, 535 participants, SMD 0.38, 95% CI: 0.21 to 0.55, p < 0.001). One study examined the effects of different modes of HFS interventions (computer-based vs instructor-led) on HFS knowledge and behaviour immediately post-intervention in adults and displayed no significant difference between groups (HFS knowledge; very low quality, 1 RCT, 68 participants, SMD -0.02, 95% CI: -0.50 to 0.45, p = 0.92) and (HFS behaviour; very low quality, 1 RCT, 68 participants, SMD 0.06, 95% CI: -0.41 to 0.54, p = 0.79) respectively. Conclusion The limited evidence supports the use of HFS interventions to improve HFS knowledge and behaviour in children, families with children and adults.


Introduction
Every day, 7 people die from home fires in the United States (US) [1]. Residential fires remain a major public health burden [2][3][4]. Between 2011 and 2015, U.S. fire departments responded to an average of 358,500 home structure fires per year, which resulted in an average of 2,510 fatalities annually [1]. In 2016, the rate at which U.S. home structure fires were reported was 1.1 per thousand population [1]. In United Kingdom, it was estimated that at least 500 deaths and 15,000 injuries were due to residential fires in 1998 [5][6].
Fire prevention requires multiple strategies. One strategy is to identify and target risk factors. Several systematic reviews have identified factors associated with higher rates of fires high number of residents, male homeowner, children under age of 5 years, smoking, low-income, buildings in poor conditions, frailty/disability, young and old age tenants, as the distinguishing risk factors associated with such incidents [7][8][9].
Another approach to fire safety is early detection of fire initiation in the homes, to prevent progression. To date, two meta-analyses examined smoke alarm coverage interventions by comparing the intervention to no interventions or to usual care [10][11]. A later network meta-analysis evaluated the effectiveness of smoke alarm interventions, but instead, compared several types of interventions, and found that the most effective method was the most intensive (includes education, low cost equipment fitting and in-home safety inspection) [12]. Home Fire Safety (HFS) knowledge and behaviour outcomes were also examined in a 1999 review [13]. The results concluded that there is a need for program evaluation especially among school-based education programs.
While the review by Warda et al. (1999) provides valuable insights, it has important limitations. For example, the review is outdated, included no critical appraisal or meta-analyses. Furthermore, since 1999, numerous studies have emerged [14][15][16][17][18][19][20][21][22][23][24][25][26][27], which warrants the need for a systematic review and meta-analysis. Therefore, the aims of this review were: 1. to quantify the effects of Home Fire Safety (HFS) interventions versus other interventions/ no interventions/controls on HFS knowledge and behaviour at short-, intermediate-and long-term follow ups, 2. to rate the quality of the body of literature that compares the effectiveness of HFS interventions versus other interventions/control according to GRADE guidelines across each outcome.

Methods
We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Cochrane collaboration guidelines [28][29] (S1 Checklist) PROSPERO registration number: CRD42018106866.

Eligibility criteria
Studies were included in this systematic review if the below criteria were met [30][31][32][33]: • Design: randomized controlled trials (RCTs) or non-randomized studies published in a peer reviewed journal, • Participants: toddlers, children (primary or secondary school), teenagers or adults-no age limit, • Intervention/Comparison: studies that compared HFS interventions to other interventions/ no interventions/controls • Outcomes: HFS knowledge and behaviour, Reports, conference abstract and posters were excluded from this systematic review [30][31][32][33].

Information sources
We conducted systematic electronic searches to identify relevant studies in MEDLINE, EMBASE and PubMed from January 1998 to July 2018. Several different combinations of keywords were used, such as: "home fire safety", "home fire safety knowledge", "home fire safety behavior", "effectiveness of home", fire safety, "residential fires", "fire prevention programs", "fire prevention programs adults", "fire prevention programs children", "fire prevention". (S1 File). In addition, we carried out a manual search of the reference lists of the identified studies.

Study selection
Two independent reviewers (MS and GN) performed the systematic electronic searches in each database. We then identified and removed the duplicate studies. In the next stage, we independently screened the titles and abstracts and retrieved in full text any article marked include or uncertain by either reviewer. Lastly, we carried out an independent full text review to assess final eligibility. In case of disagreement, a third reviewer, (JM), provided a consensus through discussion.

Data collection process
Two independent researchers (MS and GN) extracted the data from the eligible studies. In case of disagreement, a third reviewer (JM), provided a consensus through discussion. Data extraction included the author, year, study setting, study population, sample size, age, intervention/comparison groups, follow up periods and the primary and secondary outcomes. When insufficient data were presented, GN contacted the authors by email and requested further data.

Assessment of risk of bias in individual studies
Two independent review authors (JM and GN) assessed the RCTs and non-randomized studies for risk of bias. The risk of bias assessment in the included RCTs was performed using the Cochrane Risk of Bias tool [29]. The Cochrane Risk of Bias tool is based on 7 items, random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting and other bias [29]. We defined the other bias category as trials that did not include statements on sources of funding/ potential sources of conflicts of interest. The adequacy of each of the seven risk of bias domains was rated as "low", "unclear" or "high" risk according to criteria provided in the Cochrane Handbook for Systematic Reviews of Interventions [29].

Assessing the quality of evidence
We used the GRADE approach for systematic reviews, to determine the quality of evidence related to each outcome to summarize the extent of our confidence in the estimates of the effect [31][32][33][34][35][36]. The GRADE approach considers the risk of bias, publication bias, consistency of findings, precision, and the applicability of the overall body of literature to provide a rating of quality of evidence (high, moderate, low, or very low) per outcome [34][35][36][37][38][39].

Summary measures
To quantify and interpret our data, a Minimal Clinically Important Difference (MCID) of 0.5 standard deviation points for HFS knowledge and behaviour was used [40]. Timing of outcome assessment were categorised as short-term (3-4 months), intermediate-term (6 months) and long-term (12 months).

Subgroup analysis and exploring heterogeneity
In the presence of heterogeneity, we planned to perform the following subgroup analyses (a priori): trials at low risk of bias (low risk of bias in allocation concealment and blinding of outcome assessor), type of HFS intervention used. An I 2 estimate of at least 50% and a statistically significant Chi 2 statistic (P = 0.10) were used to indicate evidence of a substantial problem with heterogeneity [41].

Synthesis of results
We performed 7 meta-analyses of studies comparing HFS interventions to other interventions/no interventions/controls, using the outcomes HFS knowledge or behaviour, at short-, intermediate-and long-term follow ups. We used the Review Manager 5.3 (RevMan 5.3) software to conduct our review and used the standardized mean difference (SMD) with a randomeffects model to pool outcomes.

Study characteristics
The 8 eligible RCTs were conducted between 2003-2017 and included 1962 participants. Study size ranged from 76 to 499 participants. Trials were conducted in Canada, USA and UK [18][19][20][22][23][24][25][26]. Only two out of the eight trials were registered in a clinical trials register [20,22]. In addition, three trials did not include statements on sources of funding or potential sources of conflicts of interest [24][25][26]. A summary description of all the included RCTs is displayed in Table 1. The 2 eligible prospective cohort studies were conducted in 2003 and 2017, and included 1491 participants (study sizes were 671 and 820). Studies were conducted in UK and Australia. A summary description of all the included studies is displayed in Table 2.

GRADE Evidence Profile (EP) and Summary of Findings (SoF)
The EP (Table 3) displays a detailed quality assessment and includes a judgment of each factor that determined the quality of evidence for each outcome. The SoF tables (Tables 4-6) include an assessment of the quality of evidence for each outcome.

Effects of intervention vs no intervention in primary school children (RCTs)
Home fire safety knowledge. Two studies were pooled to examine the effects of interventions (Risk Watch and Great Escape) vs no interventions on home fire safety knowledge at short-term (up to 4 months) follow up. The pooled results, demonstrated significant difference between groups (very low quality, 2 RCTs, 535 participants, standardized mean difference (SMD) 0.38, 95% CI: 0.21 to 0.55, p < 0.001, Fig 3; Analysis 1.1.1). Heterogeneity was absent. Given that an MCID is approximately 0.5 SD, the pooled results were not clinically important. However, more data are required to make a definitive conclusion.  Abbreviations: HFS; home fire safety, SMD; standardized mean difference, CI; confidence interval. 1 We downgraded by one level due to high risk of bias.
Home fire safety behaviour. Two studies were pooled to assess the effects of interventions (Risk Watch and Great Escape) vs no interventions on home fire safety behaviour at shortterm (up to 4 months) follow up. The pooled results, displayed no significant difference between groups (very low quality, 2 RCTs, 609 participants, SMD 0.34, 95% CI: -0.21 to 0.89, p = 0.23, Fig 3; Analysis 1.1.2). Heterogeneity was high, and we were not powered to conduct sub-group analysis. Given the MCID of 0.5 SD, the pooled results were not clinically important. However, more data are required to make a definitive conclusion. Abbreviations: HFS; home fire safety, SMD; standardized mean difference, CI; confidence interval. 1 We downgraded by one level due to high risk of bias. 2 We downgraded by one level due to a relatively small sample size. 3 We downgraded by one level due to publication bias.

Effects of Intervention vs no intervention in primary school children (prospective cohort)
Home fire safety knowledge. One study examined the effects of intervention (Life-skill protocol) vs no interventions on home fire safety knowledge immediately post-intervention. The results, displayed significant difference between groups (1 study, 671 participants, SMD 1.64, 95% CI: 1.44 to 1.84, p < 0.001, Fig 4; Analysis 1.1.1). We found similar results at short-term (3 months) follow up (1 study, 671 participants, SMD 0.86, 95% CI: 0.68 to 1.04, p < 0.001, Fig 4; Analysis 1.1.2). Given that an MCID is approximately 0.5 SD, the results were clinically important. However, this was a prospective cohort study, and we were unable to make a definitive conclusion on the effectiveness of the Life-skill protocol in improving home fire safety knowledge.

Effects of intervention vs control in families with children (RCT)
Home fire safety knowledge. One study assessed the effects of home fire safety intervention vs control (minimal intervention) on home fire safety knowledge at short-term (2 months) follow up. The results, displayed significant difference between groups (very low quality, 1 RCT, 96 participants, SMD 0.66, 95% CI: 0.25 to 1.07, p = 0.002, Fig 5; Analysis 2.1.1). Given that an MCID is approximately 0.5 SD, the results were clinically important. However, more data are required to make a definitive conclusion.
Home fire safety behaviour. One study examined the effects of home fire safety intervention vs control (minimal intervention) on home fire safety behaviour at intermediate-term (6 months) follow up. The results demonstrated significant difference between groups (very low quality, 1 RCT, 277 participants, SMD 0.35, 95% CI: 0.09 to 0.60, p = 0.007, Fig 5; Analysis 2.1.2). We found similar results at long-term (12 months) follow up, (very low quality, 1 RCTs, 277 participants, SMD 0.36, 95% CI: 0.11 to 0.61, p = 0.005, Fig 5; Analysis 2.1.3). Given the MCID of 0.5 SD, the results were not clinically important. However, more data are required to make a definitive conclusion.

Effects of different modes of intervention in adults (RCT)
Home fire safety knowledge. One study assessed the effects of different modes of home fire safety interventions (computer-based vs instructor-led) on home fire safety knowledge immediately post-intervention. The results, displayed no significant difference between groups (very low quality, 1 RCT, 68 participants, SMD -0.02, 95% CI: -0.50 to 0.45, p = 0.92, Fig 6; Analysis 3.1.1). Given that an MCID is approximately 0.5 SD, the results were not clinically important. However, more data are required to make a definitive conclusion.
Home fire safety behaviour. One study assessed the effects of different modes of home fire safety interventions (computer-based vs instructor-led) on home fire safety behaviour immediately post-intervention. The results, displayed no significant difference between groups (very low quality, 1 RCT, 68 participants, SMD 0.06, 95% CI: -0.41 to 0.54, p = 0.79, Fig 6; Analysis 3.1.2). Given that an MCID is approximately 0.5 SD, the results were not clinically important. However, more data are required to make a definitive conclusion.

Discussions
This review identified and synthesized the most rigorously designed intervention studies, finding that there is a small number of studies examining diverse HFS interventions on knowledge and behaviour. In fire prevention research a major challenge is how researchers can ascertain whether a fire was prevented. Hence, they rely on test of knowledge of fire prevention strategies. The limitation, which is substantial, is that this may not insure these strategies are implemented. However, promising results were found in the small pool of studies in that statistically and clinically important improvements in HFS knowledge were found when different interventions were compared to the control or no intervention groups, in primary school children and families with children at up to 4 months follow up. We also found that there was no immediate difference in HFS knowledge and behavioural improvements between two ways of delivering HFS programs (instructor-led vs. computer-based). Warda et al. (1999) review concluded that there is a need for intensive program evaluation, especially among school children demographic. In our review, we identified 3 RCTs and 1 prospective cohort study that examined the effectiveness of HFS interventions in this population. However, the magnitudes of intervention effects were different between the two study designs. In the Lamb et al. (2006) prospective cohort study (interventions vs no intervention groups), SMDs of 1.64 (95% CI: 1.44-1.84) and 0.86 (95% CI: 0.68-1.04) were reported for improvements in HFS knowledge and behaviour, respectively. These values were much higher than those reported in the 3 included RCTs (Kendrick et al. 2007; Morrongiello et al. 2012;Hwang 2006). It is likely that the magnitude of intervention effects was over-estimated by Lamb et al. (2006).
All eight trials identified in this review were rated at high risk of bias. The rating of very low-quality evidence per outcome across trials was based on the judgement of serious limitations (risk of bias), very serious imprecision and likely publication bias in all the outcomes across trials. This can be challenging area to conduct RCTs, and it likely that cluster-randomized trials may be needed to evaluate group interventions on a larger scale. Given that multiple approaches are likely to reach and benefit different target audiences, it will require a much larger pool of studies to define the optimal approaches. Despite the limitations in current research, it is reassuring that the methods evaluated have had positive effects on knowledge, and this suggests that the methods that are currently being used at least have a positive effect on this precursor to behaviour change. Effectiveness of home fire safety interventions

Limitations
We focused on RCTs and prospective cohort studies and did not include conference papers, posters or abstracts. Therefore, there might be a source of publication bias within our search strategy.

Conclusions
The limited evidence supports the use of HFS interventions to improve HFS knowledge and behaviour in children, families with children and adults. Large-scale well-designed randomized controlled trials that consider the unique nature of prevention research and look at behavioural or fire rates as outcomes in larger scale implementation are needed to further assess the effectiveness of HFS interventions.

What is already known on this subject
• Residential fires remain a major public health burden • There is a need to evaluate the effectiveness of Home Fire safety (HFS) programs among adults and especially among school-based education programs.

What this study adds
• The state of the research is currently poor quality. Even randomized controlled studies are found to be poor quality. Future studies should be designed to remedy these flaws.
• Statistically and clinically important improvements in HFS knowledge were found between different interventions vs control or no intervention groups, in primary school children and families with children at up to 4 months follow up.
• No immediate differences were found in HFS knowledge and behavioral improvements between instructor-led vs. computer-based programs.