A. Identification of FGC services in the UK.

We will conduct a survey of local authorities and their equivalents across the UK to identify the current use of FGCs, and differences in practice and contextual factors between authorities. The aim of the survey is to describe the FGC landscape in the UK. It will not provide enough depth to inform the programme theory (B below), but the information gathered about FGC practice and contextual differences will be used in combination with the programme theory to inform the development of the evaluation measures (C below). Results from the survey will also inform local authority selection for WP3.

Sampling. The sample frame will comprise all local authorities in England, Scotland and Wales, and Health and Social Care Trusts in Northern Ireland (n = 212). Given that most authorities are likely to have implemented a related service at some point, all will be included in the sample frame. It will also be helpful to know about local authorities who have either never used FGCs and those who have disinvested in them.

Setting/context. Local authorities have statutory responsibility for child welfare in England, Wales, and Scotland. In Northern Ireland the equivalent bodies are Health and Social Care Trusts, but for brevity, we use the term ‘local authorities’ here in reference to all UK nations. They both provide their own services and commission services from the private and third sectors.

Data collection. An online survey will be developed in collaboration with the Family Rights Group and in consultation with experienced FGC practitioners. There are two routes to reaching FGC services. Firstly, we will send the online survey link to all heads of children’s services in the UK and ask them to cascade the survey to relevant staff, such as FGC service managers. Secondly, local independent FGC services will also be identified directly via a systematic web search (‘family group conference’ and each local authority name). Identified FGC services will also be emailed the survey link and asked to take part. Consent will be sought via a tick-box on the first page of the questionnaire. The anticipated respondents are FGC service managers but there will be no criteria imposed as to who can complete the questionnaire.

The online questionnaire will be fairly brief and focused, including questions about:

• Whether or not FGCs are currently used in the local authority
• Whether FGCs were offered in 2017–19 (the baseline period for WP3)
• The stage of child welfare concern when FGCs are offered
• The family circumstances for which FGCs are used
• Which kind of organisation delivers the FGCs
• Whether there is any theoretical model that informs the approach used
• How services have been functioning in light of Covid-19
• Number of conferences run each year
• What data are currently recorded for evaluation purposes and how valuable this is
• Staff involved including contact details (for WP2)

There will be a mixture of fixed-response quantifiable questions and open questions with free text boxes.

Comparable surveys have achieved response rates ranging from 39% to 86% [25, 26]. Even a low response rate would mean a viable study, however, the main aim is to generate an understanding of different models and different types of implementation, rather than to achieve a highly representative sample. More important is diversity in context–data from different UK nations and different size local authorities–to capture variation. Additional aims are to make initial contact with practitioners who can form a community of practice for help in delivering WP2 and to identify local authorities where FGCs are not offered for WP3.

Data analysis. Descriptive statistics on survey responses will be produced using Stata software. Free text responses will be thematically synthesised.

B. Modelling of FGC programme theory.

This element of the study will involve working with key stakeholders–families and practitioners–to co-design the FGC programme theory and appropriate outcomes. This will help inform the evaluation design.

Programme theory is the main output of a realist study and is defined as a ‘causal model linking programme inputs and activities to a chain of intended or observed outcomes, and then using this model to guide the evaluation’ [27, p.30]. Mechanisms, contexts, and outcomes are key concepts in realism and will be essential to identify to produce, test and refine a programme theory/theories. Mechanisms are underlying and often unobservable causal forces (e.g., how someone thinks and feels) that explain the observable outcomes [28] of FGCs. They are comprised of two parts–resources offered by a programme (like private family time in FGCs) to people, and how people respond or react to those resources [29]. Context is any pre-existing factor outside of the formal FGC intervention, that enables or inhibits the activation of programme mechanisms [27]. Context refers to any level of influence in a complex system, family problems, the FGC service, the local authority, or national policy that affects FGC provision.

We will conduct a series of consultations with stakeholders to generate an understanding of the overarching programme theory of how FGCs work in the UK, to construct a complex systems logic model and determine appropriate evaluation measures. This piece of work is framed primarily as stakeholder consultation on research design rather than data collection. The consultations will be preceded by a literature review of the international qualitative evidence on FGCs, with an emphasis on UK studies. Key databases will be searched including Medline, PsycInfo, Embase, Scopus, and CINAHL. These will be supplemented with grey literature searches. The identified literature will be explored for evidence of contexts, mechanisms and outcomes, and causal relationships between these, that will be brought together to produce a candidate programme theory and logic model, that will be presented to families and practitioners, and revised following their input.

Sampling and research participants. We have identified two FGC sites with contrasting contexts in which to conduct consultation workshops in London and North Wales. Both areas have FGC services with strong track records of service delivery and ongoing family involvement. These are the Camden Family Group Conference Service and Y Bont, based near Caernarfon. The two areas are contrasting in several respects. Camden is an inner city, multi-ethnic London borough. North Wales–Y Bont serves Gwynedd, Conwy, and Denbighshire local authorities–is a mixture of smaller towns and rural areas, mostly White and containing a high proportion of Welsh language speakers, especially in Gwynedd. Camden and North Wales both have pockets of extreme deprivation. From each of these areas we have recruited a peer researcher with experience of having taken part in FGCs as a family member. These staff were recruited with the help of FGC services in the two sites. They are full members of the research team and will take part in all aspects of WP1, but with particular responsibility for setting up and running the consultation with family members. They are supported by the CASCADE research centre’s framework for employing people with lived experience of social care.

Within each of these FGC services we will purposively sample a group of families who have received an FGC (n = 8–12 per site); and a group of practitioners, including local FGC co-ordinators, local social workers, and other FGC co-ordinators from across the UK (n = 16–24). The individuals from the two FGC sites will be identified and recruited with the help of the local FGC service and the peer researchers. Any family members, and any number of individuals from any one FGC will be eligible to take part, with no restriction in terms of recency of the FGC. However, families in the middle of legal proceedings about a child, will not be able to take part due to ethical reasons. The sample should ideally include a mixture of participants in terms of satisfaction with the FGC process. If more family members come forward than we can accommodate, we will explore their satisfaction with their FGC via response to a single question and purposively sample to achieve a range of satisfaction levels. There will be no restriction for professionals participating in terms of their level of experience or seniority. Emails and text messages will be sent by the two FGC services to families and coordinators who have taken part in FGCs, with information about the workshops. In recruiting participants, careful attention will be given to equality and diversity. Especially relevant diversity criteria are likely to be ethnicity and language use. We will ensure the consultation event is pitched appropriately for people with few educational qualifications and for ethnic minority participants who speak English as a second language. The workshops in North Wales will include use of both Welsh and English. The other FGC co-ordinators from across the UK will be identified through the survey of FGC services (see A. Identification of FGC services in the UK). Respondents will be asked if they would like to take part in other activities for this research project, and those who indicate interest will be emailed and invited to participate in the consultation workshops.

Data collection. In each site, we will host two in-person workshops for adult family members. We will also hold two online (video conference) workshops for children and young people across the two sites. Both adult and young person workshops will be three months apart, allowing for the research team to act on participants’ views following the first workshops and for sense-checking of actions the researchers have taken. There will also be two online workshops for practitioners, also three months apart.

Online workshops will be used to allow practitioners to attend from each of the UK nations, in addition to the two local sites. They will be used with young people for two reasons. One is the pragmatic consideration that young people may be more difficult to recruit, given that not all FGCs have children or young people attending, therefore merged workshops with young people from both sites together may be more realistic. The other consideration is that feedback from the pandemic service delivery experience was that many young people were positive about online communication, and this could make the meetings more likely to happen than in-person meetings [30].

The first round of consultations will focus on testing and refining the candidate programme theory/logic model, and the second on specific evaluation measures. The family events will be jointly run by the peer researchers and other members of the research team. In advance of the first consultation, the research team, including the peer researchers, will draw together data from the existing research evidence (reviewed qualitative studies and a recent realist review [31]) to develop initial candidate logic model(s) that depict the overarching programme theory for FGCs in the UK.

The presentation of the programme theory and logic model(s) will be discussed and agreed upon between the peer researchers and other members of the research team in advance of the first consultation events, with accessibility of language and concepts in mind. Through the consultations we will explore: 1) Does the candidate programme theory fit with the actual experience of being in an FGC? 2) How do FGCs impact on how people think and feel (mechanisms)? 3) What other things (contexts) affect how people experience FGCs (including the effect of Covid-19) and how successful they are? 4) What do good and poorly delivered FGCs look like? 5) What kinds of things do families have happen, or want to happen because of an FGC, that we should be measuring in an evaluation (outcomes)? For young people participating, recognising that this type of consultation may be less familiar to them, the agenda of the online workshops will include some lighter content as well as core FGC discussion points.

If recruited individuals do not want to participate in the group consultation, we will offer the opportunity to participate in an interview. Data will be recorded (audio for in-person meetings and video for online meetings) and transcribed verbatim. Written consent will be sought from participants 16 and over. For any children under 16, written consent will be sought by adults or carers with parental responsibility.

Data analysis. Ideas and notes generated through the consultations will be analysed for further evidence of contexts, mechanisms and outcomes (and causal relationships between these concepts), which will be used to confirm, refute or refine the initial candidate programme theory and logic model(s) to produce an overarching understanding of FGC practice in the UK, while recognising that not all data will be complementary, and a single unified understanding may be unlikely. Such discrepancies will be noted and explored more fully in the later stages of evaluation (e.g., semi-structured interviews in WP2). Consultation data will be analysed thematically [32] using a combination of a priori (e.g., theory, context, implementation, outcomes) and in vivo codes. The logic models will be revised so that they are more comprehensive and encompass context and implementation issues. These models will be accompanied by a narrative overview of the programme, which will provide a more detailed account.

There is a pragmatic compromise between maximum complexity and what can realistically be conveyed in a single-page summary model (and realistically measured in questionnaires–see below). It should be acknowledged that there is a risk that outcome indicators might become too homogenised, given the complexity of context and mechanisms. This is an inevitable limitation of a logic model that is concise enough to be useful to busy practitioners.

Acknowledging that the two case study sites could be considered enthusiasts for FGCs, we will use the WP1 survey of FGC services, which will cover a wider range of FGC practice across the UK, to contextualise our theorising of FGC process in context.

C. Identification and co-design of evaluation measures.

We will build on elements A and B above to identify evaluation measures to operationalise our findings for the later stages of the evaluation. Evaluation tools (questionnaires) will be developed and piloted in WP1 to reflect the programme theory, including implementation, contexts and outcomes. Following a literature review of existing relevant measures and further consultation with families and practitioners, suitable scale measures will be selected for use in WP2. This will include a questionnaire for FGC participants at three time points–with existing measures adapted if necessary–and a questionnaire for FGC co-ordinators (i.e., staff who arrange and facilitate the conferences). The initial selection of potential measures will include peer researchers and options will be discussed at the second round of consultation events with family members, young people and practitioners. Some outcomes from the programme theory will be best measured longer-term using administrative data as part of WP3.

The problem of how to deal with multiple perspectives will be explored and resolved with family members and practitioners. Options include separately self-reported measures and a consensus score for the family unit.

The WP1 survey will identify FGC services across the UK and those opting for further involvement with the study will be invited to form a community of practice, to implement the use of the standardised evaluation questionnaires in routine practice in WP2. The questionnaire for FGC participants will be refined in consultation with this community of practice. The questionnaire will then be piloted with WP1 workshop participants and other family members recruited from the two WP1 local sites, using cognitive interviews to assess validity and likely feasibility. Cognitive interviews are a method for testing respondents’ understanding of survey response items. Participants are presented with the draft questionnaire and probes are used to check what they understand from the questionnaire wording. These cognitive interviews will include adult family members, young people, and some younger children (in connection with the accessible version of the families’ questionnaire). There will be approximately 20 interviews in total across all these groups.

The questionnaire for FGC co-ordinators will characterise the family situation, decisions made and the co-ordinator’s own rating of how effective the FGC was. The content of the co-ordinator questionnaire will link to the programme theory and questionnaire wording will be discussed in the second round of practitioner consultation workshops. It will also capture resource use and will be used as a basis for estimating the cost of delivering FGCs. The design of the resource use aspect will be informed by our previous systematic review of economic analyses of FGC 11 and it will be piloted to ascertain its acceptability, comprehension, and reliability.

A Families’ Research Advisory Group will also be set up, to meet biannually for the whole duration of the project (four years). This will be made up of adult family members with lived experience of participating in FGCs or of other aspects of involvement with children’s social care services. The acceptability of routine data follow-up for WP3 will be explored with members of this group, as well as the question of which outcomes discernible from these data sets should be assessed.

Outcomes to be measured.

Outcomes will be determined in collaboration with stakeholders in WP1. Candidate outcomes to be measured in this exploratory work package include the empowerment of FGC attendees two weeks and six months after the conference. In addition, the following outcomes will be explored: family communication, adult family member well-being, and perceived child safety and well-being (also reported by the child/young person if they have attended the FGC meeting and agree to take part in the study). None of these is of primary importance, but for the purposes of sample size calculation (i.e. the number of families needed to complete WP2 questionnaires), parental empowerment has been assumed to be the primary outcome.

Sampling, settings and research participants.

WP2 involves working with a community of practice of FGC practitioners across the UK, identified in the WP1 survey of FGC services, to adopt the families’ questionnaire designed in WP1 into routine use. The research team will keep in regular contact with this community. Any child welfare FGC service that wants to take part in WP2 will be able to do so; there will be no exclusion criteria. Within and between services that volunteer to take part there should be variation in FGC use that will allow the impact of context to be considered. Within each service, any child welfare FGC would be eligible for inclusion. The only exclusion criterion would be FGCs convened because of adult social care needs, which have no child welfare dimension.

Participating FGC services will comprise a prospective single-arm study of the impact of complex systems on family outcomes six months after an FGC, using family self-report via a questionnaire and semi-structured interviews with a sub-sample. All family members who have participated in a conference will be invited to take part, with no lower age limit imposed. Those who give written consent to take part will have contact details recorded at the point of recruitment (baseline) and these will be used for subsequent follow-ups. Participants are likely to include extended family members as well as biological and social parents and children/young people. The co-ordinator will be asked to complete a questionnaire, but no other professional who has attended an FGC, such as a social worker or advocate.

As noted above, the evaluation tools will be co-designed with families during WP1, and outcomes and sample sizes will be decided during this co-design phase. To obtain an estimate of the scale of the study we assume a primary outcome of empowerment and a single composite score across all FGC participants, whilst acknowledging this may not prove to be a preferred measure. We further hypothesise there could be a difference in empowerment according to a characteristic of the FGC–e.g., higher or lower levels of participation or an FGC taking place at a late stage of concern or earlier.

Having consulted results from comparative studies of family group decision-making that have measured empowerment at later time points [33–35], it is possible to estimate a sample size for comparing different kinds of FGC (e.g., early help vs. edge of care–contexts where the level of empowerment could be quite different). To detect a 0.25 difference between two groups of equal size on a five-point scale, with a standard deviation of 0.50, 95% confidence and 80% power, a sample size of 128 families will be required (two-sample t-test). For a pre-post comparison to estimate the mean improvement from baseline to six months in all FGC participants, N = 128 will allow us to detect a minimum effect size of 0.125 with 80% power, or a minimum of 0.144 with 90% power (one-sample t-test). Assuming a 36% attrition rate between the first questionnaire and six-month follow-up, which is conservative in light of the same rate having been found by Dijkstra et al. over a 12-month follow-up period [34], an initial sample size of 200 FGCs will be needed. The current English trial of FGCs at pre-care-proceedings stage 6 shows an average of 73 conferences a year planned in each of the 24 participating local authorities, so for a minimum of 128 families to participate in the full survey across several different services seems feasible.

Data collection.

We envisage survey data will be collected from families at three time points, although this timing will be discussed and agreed with families and practitioners in WP1 workshops:

• At baseline–i.e., before the FGC takes place–data collection will be challenging. Who will attend is often unpredictable in advance. Once participants have arrived for the FGC but before it starts, there are often difficult family dynamics, meaning that completion of a full questionnaire would probably not be feasible. We will assess in WP1 what is possible. Several administration methods will be considered, such as paper, phone, or online questionnaire, distributed via email, WhatsApp or text messaging.
• Follow-up a few weeks after the FGC will capture short-term impacts, in particular the initial family appraisal of the FGC experience and of decisions made. Mechanisms and context will be important to assess at this point from questionnaire responses.
• Medium-term impacts will be measured at six months, to capture enduring consequences of the FGC process for families. Some measures will be repeated from baseline and short-term follow-up data collection.

Questionnaire wording, determined through co-design in WP1, will be as simple as possible, bearing in mind a range of reading abilities. There will also be a shorter, accessible version for use with young children (e.g., under 11 years) and people with learning difficulties or low levels of literacy, which will also be co-designed with families and piloted, including checking with a learning disability charity. Administration methods will be determined during WP1, but at this point, we can envisage a choice for participants–e.g., online self-completion as the dominant method with researcher-supported data collection over the telephone where requested. We will provide specific training for researchers doing telephone questionnaires, with a script-based approach to maximise reliability.

Soon after each conference, the FGC co-ordinator will fill in a separate short questionnaire (also developed in WP1) to characterise the family situation, decisions made, rating of FGC effectiveness and resource use.

We will draw on the substantial evidence base for maximising response rates and retention [36, 37] and consult with families in WP1 about effective approaches. We will offer a financial incentive of £30 per respondent (£10 per questionnaire) to be paid shortly after completion of each questionnaire. Parkinson et al.’s review of incentives in clinical trials [38] found it was better to split payment between tasks and to make payment as soon as possible. Our recruitment and retention protocol will include a variety of methods to chase up non-response, including email and text message. We will ask participants for full contact details and permission to contact them via their preferred method when first recruited, and we will consider requesting secondary contact details (e.g. for a partner), if participants are happy to share these with us. A short practitioner-facing protocol will be developed to provide clear instructions to recruiting practitioners. Site-specific feedback will be an important incentive for services to support healthy recruitment. Follow-up at short- and medium-term time points will be done by the research team.

Semi-structured telephone interviews will be conducted with family participants, for further exploration of mechanisms and context. The interview schedule will not be drawn up until baseline questionnaire data are in, in order to pick up on emerging themes, but broadly the areas covered will be the same as in the questionnaire, though obviously in more depth and expressed in the participants’ own terms. We will use a purposive sample of five FGC participants from each of four services that participate in WP2 questionnaire data collection. We estimate this sample size to be sufficient to capture variation of context–e.g., stage of child welfare concern when FGCs take place and provision of services (i.e., by local authorities or by independent organisations). Participants willing to be interviewed will signal via the questionnaires and a purposive sample will be selected using variation of context as the main criterion. The interviews will be audio recorded and transcribed.

The intention is for the evaluation tools to become embedded into routine practice, enabling ongoing self-evaluation by services, an essential part of fostering a culture of service improvement. Ongoing support will be provided from research project staff to support the community of practice, with a view to it becoming self-sustaining after the project has finished.

Some important considerations for embedding the research tool into routine practice include co-designed content and data collection method (WP1); liaison with umbrella organisations such as the Family Rights Group, FGC networks for each of the four nations and the British Association of Social Workers FGC practitioners group; collecting no more data than families and services can tolerate; a real-time system for monitoring completion; ensuring feedback to people using services and FGC staff to show how data are being successfully collected; preparedness to be flexible, e.g., in terms of data collection mode preferences; and recruiting local champions for sites to support completion and contribute to problem solving.

We have explored the feasibility of routine use with FGC practitioners and one practical reality to consider is that funders often require some specific evaluation data. In the case of Y Bont, this is different in the three local authorities who commission their FGC service. Any questionnaires for families or practitioners developed in WP1 will need to be appropriately short and focused, as they may well have to be used alongside another type of data collection for local purposes. The logistics of this will be explored with services participating in WP2.

Data analysis.

Descriptive summaries of outcomes, by time point, as well as simple pre-post comparisons (mean difference with two-sided 95% confidence interval) will be produced using Stata or R software. Contexts will be assessed per FGC, from a combination of family and practitioner questionnaire data. Additional analyses will involve fitting two-level regression models with random family effects to study the interplay of multiple contextual and implementation factors per follow-up time-point and longitudinally, as well as outcome variability between family members attending the same FGC. Fixed effects will include time-point, contextual/implementation factors, and possibly time-by-contextual factor interaction terms. Examples of possible contextual and implementation factors would be family size, stage of the child welfare process (e.g., early help, child in need, edge of care) and local authority culture, as assessed by family and practitioner (possibly a scale measure). Estimates of the effects of contextual factors will be presented with two-sided 95% confidence intervals and p-values.

Data from the WP2 family survey data a few weeks after FGC will be used to assess the reliability and validity of outcome measures by repeating or extending some of the original validation work on key measures, particularly where they have been adapted for use in this study, using e.g., Cronbach’s alpha, criterion validation and examination of factor structure and ceiling and floor effects. A statistical analysis plan will be developed to cover both WP2 and WP3.

Semi-structured interviews will be coded in NVivo software, using thematic analysis31. The coding frame will be developed through a combination of the programme theory from WP1 and inductive initial analysis of interview transcripts, with context, mechanisms and outcomes featuring as analytic codes.

Sampling, settings and research participants.

Families who have attended FGCs will be identified by local authorities–we will work with several large authorities in England, and access anonymised data from the Department for Education, NHS Digital and the Office for National Statistics (ONS). This part of the study is not linked in any way to the WP2 families. In these participating local authorities, we will also access routine data on families who have been referred to children’s services but have not had an FGC, in order to delineate the circumstances in which families do and do not take part in FGCs in these particular authorities.

A comparison group of local authorities which do not offer FGCs will be identified in the WP1 survey. Families from these authorities will be identified from the Children in Need Census. This part of the study uses data from only one UK country–England–because the datasets are different for each nation and given the complexity of data linkage it would not be feasible to attempt this study in more than one nation. We have chosen England as having the largest population.

Eligible families will be those with at least one child under 18 years of age referred to departments of Children’s Social Care Services from 1.1.17 to 31.12.19 at participating local authority sites in England (there is no intention to connect these local authorities to those taking part in earlier work packages). No additional exclusion criteria will be applied. At this stage we are not imposing a lower age limit. However, although we are confident that linkage to health records for school-age children is viable, there is some uncertainty about this linkage for pre-school children. We will therefore have to test the feasibility of extending the age profile to include all children from birth, in the early stages of WP3.

To make the two groups of families from different local authorities comparable for analysis, we will use statistical techniques to improve balance across the two groups for several known confounders. We will use propensity score weighting [40, 41] to estimate the average ‘causal’ effect of having an FGC on the outcomes.

A realistically achievable sample size of 300 families who have had an FGC and 1000 ‘control’ families in local authorities that do not offer FGCs would allow for the detection of a difference of 10.5 percentage points between groups for a binary outcome in a ‘worst-case scenario’ (i.e. proportions around 0.5) with 90% power and 95% confidence (two-sample test for proportions), but since propensity score weighting tends to shrink the ‘effective’ sample size the minimum detectable difference will be somewhat larger.

Data collection and linkage.

Children referred to children’s services in applicable English local authorities (FGC providers and non-providers) during the intervention period (2017–19) will be identified from the Child in Need dataset within the National Pupil Database (NPD, Department for Education) which can be used to identify all school-aged children. This period has been selected on the basis of this being the most recent available where two-year follow-up in routine data is possible. The unique pupil ID can be used to link to other relevant NPD datasets, including Child Looked After, school attendance and special educational needs. Matching to Hospital Episode Statistics (HES) records will be undertaken using standard linkage strategies for HES and NPD data (e.g., using date of birth, gender, postcode) and quality of linkage assessed by estimating rates and distribution of linkage errors. Linkage will be achieved via the E-CHILD database [42]. Identifiers for FGC families will be supplied by local authorities. Health care records for birth mothers will also be matched in E-CHILD. The exact model for data linkage will be established in WP1. Anonymised, linked data will be accessed via the ONS Secure Research Service (ONS-SRS) and analysed by Cardiff and Oxford University staff via verified remote access.

We will follow up children in these datasets for two years following the date of referral. We have allowed for this work package to run throughout the four years of the project in recognition that permissions for data linkage and the delivery of linked data are very time consuming and often subject to delays. We also need to allow for follow-up. The latest follow-up data will be recorded in December 2021 and this data set will be first available to researchers in the Spring of 2023, two years into the project.

The Covid-19 lockdowns will influence the patterns of service provision from the end of March 2020, but this will be the same for families who have and have not taken part in an FGC between 2017 and 2019 and changes in service patterns will be interesting in their own right. Any difference between FGC and non-FGC families in post-lockdown trends can be noted. How Covid-19 and lockdowns affect the data will be considered when we write a definitive analysis plan.

Data analysis.

We will use Stata or R software to estimate a family’s propensity for having an FGC with a logistic regression on variables including stage of the child welfare process (referral, child in need, child protection assessment, child protection plan), reason for referral and family demographics. The analysis model for the primary outcome variable (whether the level of social care intervention at two years after the FGC–or equivalent baseline date for the control group–has increased or not) will be two-level logistic regression, with random effects for local authorities and weighting of families according to their propensity for having an FGC. Fixed effects will include intervention (whether a family has had an FGC or not) as well as other factors–to be determined in WP1 –to account for the different ways FGCs are set up in different local authorities. Effect estimates will be presented with two-sided 95% confidence intervals and p-values. Secondary outcomes will be analysed using the same propensity score weighting approach and appropriate two-level regression models depending on the type of outcome variable (e.g., logistic for binary outcomes, linear for continuous outcomes).

We will undertake multiple imputation to explore the impact of potential missing data. We will also consider alternative covariate balancing or matching methods, such as entropy balancing [43], Mahalanobis distance matching [44] and coarsened exact matching [45] if propensity score weighting does not achieve good enough covariate balance and/or as sensitivity analyses. Detailed statistical analysis plans for WP2 and WP3 will be finalised prior to any analysis being performed, and these will be published on the study website.

Economic evaluation.

The following health economic analyses will be conducted:

• Cost-consequences analysis with costs encompassing health care, social care and education, presented in a disaggregated manner.
• Cost-effectiveness expressed as incremental cost per additional case of child protection or out-of-home care avoided.

the economic evaluation of FGCs will be conducted from a public sector perspective. Primary micro-costing will be undertaken in WP2 via the practitioner questionnaire to estimate the cost of delivering FGCs across service model delivery types and geographical settings, including the costs of the identification and referral processes, training of accredited providers, delivery of sessions in alternative formats, participant monitoring activities and any follow-up/management. Where applicable the unit costs of resource inputs will be estimated in accordance with the principles of opportunity costs.

Broader health care, social care and education resource use over the follow-up period will be extracted from HES and NPD records and costed using national reference cost schedules. Resource use will be measured by the number and where possible, the duration of admissions, appointments and attendances during the 2-year follow-up period. Mean differences in children’s health and social care use and special educational needs provision and their associated costs (between the intervention and control arms) will be estimated using t-tests and bootstrap 95% confidence intervals that will be computed based on 1,000 (or more) replications.

The primary objective of the health economic evaluation is to estimate the incremental cost per additional case of child protection or out-of-home care avoided. Bivariate regression of costs and consequences, with multiple imputation of missing data (e.g., as a result of loss to follow-up), will be conducted to generate within-study estimates of incremental cost-effectiveness associated with FGCs.

A cost-consequences analysis will additionally be conducted, and we will present disaggregated costs and disaggregated consequences (outcomes 1 to 3 above), together with the estimates of the mean costs of the comparator interventions with appropriate measures of dispersion. Cost-consequences analyses are recommended for complex interventions that may have multiple implications [46], and for public health interventions which may have an array of benefits that are difficult to synthesise in a common unit [47].

Our analytical strategy will be informed by recent guidance on accounting for selection biases within economic evaluations using individual patient level observational data [48]. A key methodological challenge will involve generating expressions of cost-effectiveness amenable to broader cost-effectiveness comparisons by decision makers. A range of economic values for avoiding adverse child welfare outcomes will be informed by a review of the revealed and stated preference literature related to children’s social care, as well as by WP1. A series of sensitivity analyses will be undertaken to explore the implications of uncertainty on the incremental cost-effectiveness ratios.

Fig 1 illustrates the relationship between the three work packages. They connect insofar as WP2 and WP3 each assess different elements of the programme theory developed in WP1. Family-reported mechanisms, context and outcomes will be assessed in WP2. Outcomes related to service use (e.g., reduced state intervention to prevent child maltreatment) and some aspects of context (e.g., local authority) will be assessed in WP3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flow chart of study work packages.

https://doi.org/10.1371/journal.pone.0300834.g001