Study aims

This protocol outlines a study to assess the feasibility and utility of applying NLP to DHRs in collaboration with Imkaan, a UK-based umbrella organisation committed to addressing violence against Black and minoritised women and girls (VAWG). The study has three key aims:

1. To assess whether NLP can be leveraged to:
   1. a. Scale analyses of DHRs: previous studies using DHRs have been limited by region or focusing on a specific issue, with large-scale analyses only being conducted at irregular intervals. This study will be the first to utilise the entirety of the DHR Library (publicly and freely available via: https://homicide-review.homeoffice.gov.uk) which, to date, is populated with a total of 611 completed and published DHRs. Conducting such an analysis manually is naturally limited by resource capacity.
   2. b. Highlight patterns and relationships that have not previously been identified in research: by utilising a larger sample of DHRs for this study, we will assess the feasibility of whether NLP models can be used to identify and interpret complex problems and relationships. Leveraging NLP to identify these relationships before conducting in-depth (human-driven) qualitative analyses to interpret ensures that NLP is used to assist research capacity, rather than replace it.
2. To ensure voluntary and community sector organisation (VCSO) involvement in developing NLP applications and recognising model limitations: different types of data on violence and abuse can hold bias. This can include both how DHRs report demographic data, as well as the nature of the recommendations that they produce. Therefore, it is important that we recognise how NLP might recycle these biases and what processes can be put in place to prevent bias.
3. To identify outputs of high impact/utility to VCSOs impacted by reviews of DA-related deaths: VCSOs are regularly involved in the DHR process and impacted by the recommendations that DHRs produce. Therefore, it is essential that VCSOs are involved in research that makes use of DHRs as well as research that can improve DHRs. Practitioners hold substantive expertise regarding the nature, risks, and dynamics of domestic violence and abuse that can shape the direction of research.

Identifying research questions

This study is part of a large UK Prevention Research Partnership consortium, VISION (Violence, Health and Society). The consortium is a collaboration of specialised services, criminologists, sociologists, data scientists, economists, epidemiologists, drawing upon a wider network of national and local government departments, police, health trusts, and think tanks.

The first stage of this project consisted of an initial call for collaboration to interested partners. This call was sent to partners who had either previously been involved in a DHR (as a panel member, chair or report author) and/or had offered services in relation to issues typically covered or addressed in a DHR. These requests were sent via email in October 2024, with a further follow-up in January 2025. Interest was received from five individuals/organisations with which follow-up discussions were conducted. This included representatives from the Domestic Abuse Commissioner’s Office and three VCSOs (Imkaan, Women’s Aid, and SafeLives), as well as a member of the National Centre for Violence Against Women and Girls and Public Protection (NCVPP). An initial round of consultations and discussions were held in December 2024 and January 2025 over Microsoft Teams. In these discussions, EC and DC explained the context of the research, the potential role of NLP, and to what extent this technology would be of any utility in applying to DHRs. We asked partners whether there were specific questions or priority areas that it would be useful to explore using NLP in relation to their organisation’s work. We discussed that these applications would be limited by if, and how, these questions firstly, could be answered within the scope of a DHR, secondly, were within the scope of a DHR and reported to sufficient quality, and thirdly, NLP could feasibly be applied to these ends. We chose to engage a limited number of organisations at this stage to reflect the depth of expertise that each service holds in their field, including knowledge representing minoritised and under-represented voices in the DA sector. Therefore, we did not intend for this consultation to be representative, but a purposeful selection of expertise. We also supplemented by reflecting on our ongoing qualitative analysis of DHRs as well as evidence briefings completed commissioned by the Domestic Abuse Commissioner’s Office [21].

From these discussions, a draft list of potential research questions was generated which were then ranked based on two factors: impact and feasibility. Impact was assessed based on the usefulness to the partner based on previous discussions and whether existing data could be identified elsewhere that could be employed to answer the question. Feasibility was assessed based on both the predicted capability of NLP, complexity of the research question, and the availability and quality of free-text available within a DHR. The results of this exercise are summarised Table 1 below.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Longlisted research questions identified through first phase of collaboration.

https://doi.org/10.1371/journal.pone.0348948.t001

The initial phase of the collaboration generated a draft list of 28 viable research questions addressing both the nature and extent of different types of domestic abuse (for example, types of abuse experienced by different demographic groups), as well as the procedural aspects of how DHRs conducted (for example, the involvement of different panel members or the use of victim-blaming language). These are represented visually in Fig 1 below.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Proportion of research questions by feasibility and impact here.

https://doi.org/10.1371/journal.pone.0348948.g001

Accounting for impact and feasibility, we created a finalised list of research questions that was thematically synthesised into five priority areas. These priorities are summarised in Table 2 below.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Shortlisted research questions synthesised by theme.

https://doi.org/10.1371/journal.pone.0348948.t002

This phase of the collaboration generated five priority areas that will guide this feasibility assessment of applying NLP to DHRs. Firstly, we will apply NLP to identify the availability of information that is reported on victim and perpetrator demographics within DHRs, as well as assess the quality of how different demographics are reported. Secondly, we will identify the extent to which different services were involved prior to a death, such as police, VCSOs (including specialist ‘by and for’ organisations which are “run by and for the users and communities they aim to serve” [22, p.2]), schools, health care, social care, and employers etc. We will also assess whether service involvement differs by victim or perpetrator group. Thirdly, we assess whether NLP can be applied to identify and interpret victim-blaming language, including whether this language changes according to different victims and perpetrators. Fourth, we will compare characteristics of domestic homicides (specifically, intimate partner homicides) to other DA-related deaths to understand how they differ in terms of victim and perpetrator demographics, nature and extent of abuse, and type of service involvement. Recent qualitative analyses of DA-related suicides have demonstrated that this is an emerging but under-recognised area of research and, as yet, have not been compared to domestic homicides [17,23]. Fifth, we will assess the feasibility of using NLP to identify clusters of recommendations made to different services. For example, whether types of recommendation (e.g., additional training, education) are associated with certain services or sectors (e.g., adult or child social care, police, health). This collaborative approach to generating research questions therefore ensures that outputs address both substantive and procedural issues to improve the conduct of DHRs as well as the knowledge that they produce.

Study setting and design

There has been a statutory requirement in England and Wales to conduct a DHR into any DA-related death since 2011, with a central repository collecting these reports available since November 2023. The study setting for this project will therefore be restricted to England and Wales, acknowledging that models of DA-related death reviews are either recently established or currently under development in other jurisdictions.

Data source

DHRs are statutory multi-agency reviews that aim to identify lessons that can improve responses to domestic abuse and prevent future deaths. Each review involves analysing information from a range of agencies and practitioners. A DHR Chair will also seek information from family members and the wider community. A core purpose of a DHR is to build a picture of the circumstances leading up to the victim’s death.

While the format and quality of DHRs vary due to local commissioning practices, all include detailed free-text narratives, chronologies, and agency analyses. Reviews vary in length, often spanning 30–150 pages These documents contain rich qualitative data on victim-perpetrator dynamics, service contact, missed opportunities for intervention, and policy or system-level recommendations for change.

Data acquisition

Reviews will be accessed and retrieved from the Home Office DHR Library using an automated web scraper developed in Python by one of the authors (DC). As part of responsible data collection, the library website has been checked for a robots.txt file, a standard web protocol used to indicate whether automated tools such as web scrapers are permitted to access site content. At the time of writing, no such file is present on the DHR Library website, suggesting there are no stated restrictions on automated access. Once a review URL is identified by the scraper, the corresponding PDF will be downloaded and processed by a secondary function that extracts the full document content and saves it locally for further processing. All publicly available DHRs in the Library will be included at the outset, with no additional inclusion or exclusion criteria applied at this stage.

Data cleaning and preparation

The text of each PDF will be parsed using a dedicated Python script. As PDFs are not inherently machine-readable, several automated preprocessing steps will be applied to improve the quality and consistency of the extracted text. These include removing boilerplate content such as headers, footers, footnotes, page numbers, title pages, URLs, and tables of contents. Where possible, headings and subheadings will be inferred based on font size and formatting features. Any tables present in the documents will be either standardised into plain-text format or removed, depending on their structure and extractability. The output of each preprocessing step will be subject to spot-checking and revision to ensure a high level of data extraction.

Natural language processing

The first step in NLP will be to transform the text into a format a computer can understand. This involves preprocessing steps such as tokenization, part-of-speech tagging, stemming, and dependency parsing [24]. Unlike humans, machines require text to be represented numerically rather than linguistically. Common machine-readable ways of representing text include word frequencies (the count of a word in a document) or embeddings — high-dimensional vectors that capture the semantic meaning of a word based on its context within a larger corpus [25]. Projecting text into a high-dimensional vector space enables us to perform common NLP tasks including classification, clustering, and information extraction. The precise techniques we will use will differ between each of the priority areas and research questions outlined in Table 2.

1. Quality and completeness of reporting of victim and perpetrator demographics — This analysis will involve collating and tabulating key demographic descriptors such as age, ethnicity, and gender. Recording of this information is not standardised within DHRs, based on a preliminary review, and is instead inconsistently referenced under headings such as ‘Background’, ‘Family Background’, or ‘Chronology’. We will develop a combination of keywords and syntax rules to determine if a specific text segment contains demographic information. If necessary, we will progress to more sophisticated semantic information retrieval methods to capture more nuanced or implicit references to demographic information. Descriptive statistics will then assess the frequency and completeness of demographic reporting.
2. Extent of statutory and non-statutory service involvement prior to a death — The vast majority of DHRs should contain a chronology of events, outlining the activities that took place prior to the victim’s death. These chronologies present information in temporal order, enabling a systematic extraction of the relevant events. We will identify and extract references to both statutory and non-statutory service involvement using a combination of rule-based matching, named entity recognition (NER), and event detection techniques. These methods enable us to identify points of service contact even when phrased inconsistently or indirectly. Extracted data will then be used to analyse the frequency, nature, and timing of service involvement prior to death.
3. Victim-blaming language — We will develop and validate a victim-blaming lexicon based on the surrounding literature on DA discourse. This lexicon will include words, phrases, and rhetorical constructions associated with the attribution of blame or responsibility to victims. We will then apply this lexicon to each DHR to assess the prevalence of victim-blaming, using frequency-based and context-aware techniques. We can also examine variation in the use of victim-blaming language across demographic groups by linking this analysis to the demographic detection process described in priority area #1.
4. Comparing domestic homicides to other DA-related deaths — Using linguistic markers and content cues, we will classify DHRs into subgroups of interest, including IPV-related, suicides following abuse, and other DA-related deaths. Classification will be carried out using a combination of rule-based keyword matching, and document-level text classification approaches. For the latter, we will explore few-shot learning techniques (such as those described by Brown et al. [26]) to support accurate classification using minimal annotated training data. Once categorised, we will use information extraction techniques to compare incident characteristics, patterns of abuse, and risk factors across these subgroups.
5. Clusters of DHR recommendations – Each DHR typically concludes with a series of recommendations, which can include general recommendations as well as agency-specific guidance. We will use a combination of unsupervised machine learning techniques including Latent Dirichlet Allocation (LDA) [27] to identify recurring themes and patterns in recommendations. To facilitate a more fine-grained sectoral analysis, we will use the embedded agency subheadings to group recommendations by originating organisation (e.g., police, health, social care). This will enable us to examine both global thematic trends as well as sector-specific patterns in post-incident learning.

All code produced for this project will be collated, stored, and publicly accessible via GitHub under an MIT licence (allowing others to use, modify, and redistribute the code without seeking formal permission). The repository will include detailed documentation in the form of a README markdown file, as well as demonstrations via annotated notebooks and example workflows to encourage reuse and reproducibility.

Evaluation and feasibility criteria

Evaluating the components in an NLP information extraction system requires a combination of quantitative and qualitative performance metrics. We define a set of feasibility criteria to avoid post-hoc rationalisation and ensure a high-level of transparency, based on coverage, extractability, model performance, and relevance to stakeholders. Coverage metrics assess completeness, and whether the proposed methodology captures the relevant concepts in full. Extractability covers how accurately relevant information is identified via precision and recall metrics, while model performance is used to evaluate how well a technique accomplishes its stated task using global measures such as accuracy and F1 scores. Lastly, stakeholder interpretability assesses the relevance, utility, and informativeness of the output in a practical context. Collectively, these measures allow us to rigorously evaluate whether NLP can be reliably, responsibly, and meaningfully applied to DHRs. These criteria are summarised in Table 3 below.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Evaluation plan split by thematic area.

https://doi.org/10.1371/journal.pone.0348948.t003

Collaboration

We will take an iterative approach to the analysis and interpretation of the DHRs. Building on the formative consultation work, we will continue to update all stakeholders by sharing findings. In particular, we will work closely with Imkaan to iteratively develop analysis, gather feedback, and ensure that interpretations remain grounded in the lived realities of domestic abuse and service provision. Imkaan have two decades of experience working with other ‘by and for’ organisations to address VAWG, including domestic and honour-related abuse. This collaboration also helps to ensure that inputs to and directions of analyses are supervised by expert stakeholders and feedback loops are built in to avoid amplifying bias that may be found within DHRs (see ‘Discussion’).

Imkaan will be invited to continue to shape the framing of the research questions, reflect on emerging themes, and identify areas requiring further exploration or clarification. This process will help to contextualise the findings and support the development of outputs that are meaningful to both practice and policy. Engagement with practitioners in this sense is to increase the relevance and utility of the research questions and contribute to the wider sector, as endorsed by the Women’s Aid’s Research Integrity Framework on Domestic Violence and Abuse [28]. This also means being transparent about who and how you have engaged with partners and acknowledging how the research team shapes research priorities.

Engagement will be adapted to stakeholders’ capacity and preferences, using flexible formats such as virtual meetings, written summaries, or interactive demonstration sessions. Updates will be provided monthly. Outputs and authorship will be agreed upon with stakeholders. Any draft written materials for publication will be provided to relevant stakeholders for review at least three weeks in advance of target submission date.

Ethical considerations

Although the data used in this study is publicly available, several ethical considerations require careful attention due to the sensitive and potentially distressing nature of the content. DHRs often contain detailed accounts of violence, abuse, and system-level failures, and may refer to individuals, events, or circumstances that are identifiable to those familiar with the case.

To mitigate risks of harm or unintended disclosure, all data processing will be conducted with a strong emphasis on confidentiality and data minimisation. While full names and identifiers are typically redacted in published DHRs, the potential for re-identification, particularly in small or specific communities, will be acknowledged and managed during analysis and dissemination. Extracted data will be stored securely on encrypted, access-controlled university servers in accordance with institutional data protection policies and the UK General Data Protection Regulation (GDPR). Data will be reported at an aggregate level. Any human labelling in support of machine learning classification will use neutral, non-harmful codes, and will utilise a consensus-coding approach (i.e., multiple coders with reported interrater agreement scores). Any outputs produced by an unsupervised machine learning algorithm will be qualitatively reviewed to ensure identified topics or clusters are not stigmatising or harmful. Human oversight at key decision points will be maintained to contextualise ML findings and avoid presenting false narratives. Any qualitative excerpts that are reported as examples will be anonymised and not be identifiable at case level.

This study has received ethical approval from City St George, University of London’s Research Ethics Committee (ETH2526−0304).

Data governance

All data collection, storage, and processing activities in this study are governed by principles of secure, ethical, and legally compliant data handling. While detailed procedures are outlined in the relevant sections above, data governance is informed by City St George’s policies on research data management and aligns with UK GDPR and the ethical conditions approved under the VISION consortium. One of the authors (DC) has completed the Safe Researcher Training programme offered by the UK Data Service, which provides guidance on best practices for working with sensitive and potentially disclosive information. Data access will be restricted to authorised members of the research team, and all files will be stored on encrypted, access-controlled servers with multi-factor authentication (MFA). No attempts will be made to re-identify individuals, and data will be retained for the period specified in institutional policy (ten years).

Anticipated outputs and dissemination

This project requires iterative supervision and feedback. Therefore, outputs will be developed in discussion with Imkaan and take into consideration preferred timelines, formats, co-authorship and framing. We anticipate that we will produce manuscripts to be submitted to peer-reviewed journals (prioritising journals that have open-access agreements) and policy briefings available on the project website (vision.ac.uk). Additional outputs will be agreed according to stakeholder interest and capacity. We will also present findings to relevant national and international conferences and meetings. Outputs will also be delivered to relevant research and practice networks (for example, the International Community of Practice on Domestic Homicide) as well as policy units (for example, Home Office Interpersonal Abuse Unit).

Outputs and authorship will be agreed upon with stakeholders in advance. Any draft written materials for publication will be provided to relevant stakeholders for review at least three weeks in advance of the target submission date. The anticipated timeline for this study is outlined in Table 4 below. The first stage (stakeholder engagement) has been completed. The first stage (stakeholder engagement) has been completed. Participant recruitment and data collection are not applicable to this study.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Study timeline.

https://doi.org/10.1371/journal.pone.0348948.t004

Feasibility and anticipated challenges in using DHRs

This study will assess the feasibility of applying NLP techniques to the entire corpus of DHRs available via the Home Office repository. While advances in NLP have made large-scale text analysis more accessible, we acknowledge several anticipated challenges.

First, DHRs are variable in their structure, format, and writing style, as they are locally commissioned and authored by different review Chairs and involve different panel membership. Typically, each review consists of an executive summary, individual management reviews (IMRs) for each contributing service, an overview report, and an action plan. The suggested format of each review is recommended within Home Office statutory guidance [29], although variations still remain by report (for example, the inclusion of different subheadings, sections and information). In addition, DHRs address different types of DA-related deaths and victim/perpetrator relationships. Research shows that the gendered nature of DA shifts depending on the type of DA. For example, Hoeger et al. [1, p.23] identified that differences in victim sex are more pronounced for deaths in the context of intimate partner relationships (83% female victims), than for familial relationships (55% female victims). These gendered contexts can be captured within a DHR report and are important to distinguish between. These heterogeneities and inconsistencies introduce complexity in designing preprocessing pipelines and in applying models that assume uniform input.

Second, while the text volume is large in aggregate, the total number of reports presents limitations for some advanced machine learning methods (i.e., those based on deep learning architectures), particularly those reliant on very large training datasets. In line with the principle of methodological parsimony, we will prioritise simpler interpretable models (e.g., rule-based, traditional NLP pipelines) over more sophisticated deep learning approaches (e.g., neural networks), unless warranted by the complexity of the analysis or the structure of the data.

Third, ensuring that the outputs of NLP models are interpretable and relevant to expert stakeholders requires careful design of both the modelling approach and the reporting format. We conducted an initial consultation with five organisations and will ensure that they are provided with information at regular intervals. In addition, we will work closely with Imkaan so that decisions are staged sufficiently, and flexibility is built into the timeline and communication formats for outputs.

Lastly, we must take steps to ensure that an absence of a variable of interest within a report is not treated or interpreted as an absence of the characteristic itself. In other words, our approach is intended to evaluate the recording practices (e.g., inclusion, omission and patterning) of DHRs rather than the underlying prevalence of demographic characteristics.

Bias in machine learning

Bias in machine learning models is a well-documented issue, particularly when applied to sensitive social data and in the public sector [30]. As part of a partnership between the Sexual Violence Research Initiative (SVRI) and The MERL Tech Initiative (MTI), Amazon-Brown et al. [19] developed a guidance resource for researchers engaging with AI in relation to VAWG. Critically, they point out that “human expertise is irreplaceable. AI can improve efficiency, but final analysis and interpretation should always lie with researchers” [20, p.2]. Integrating AI into VAWG research in a controlled and transparent way can is essential to not only minimising inaccuracies but building capacity to use AI responsibly.

In the context of DHRs, model outputs may reflect or amplify biases present in the source documents such as under-reporting of certain identities, cultural and racial stereotypes, or omissions of certain cases altogether [17]. For example, Chantler et al.’s [9, p.487] analysis of 141 DHRs showed that information on the ethnicity was available for just over half of victims and perpetrators, while information on age was missing for just under a quarter of victims and a third of perpetrators. In addition, as no information is available from CSPs on decisions whether to commission cases as DHRs, it is difficult to identify whether certain deaths are being excluded from this dataset [31]. There is also a risk that NLP methods may flatten nuance or obscure contextual detail, particularly if models are trained or interpreted without sufficient stakeholder involvement. To mitigate against this, our study will adopt an iterative and collaborative approach to analysis, ensuring that expert stakeholders are involved in reviewing intermediate outputs and shaping final interpretations.

A related challenge concerns the interpretability of NLP outputs to non-technical audiences. VCSOs, policymakers, and practitioners may not be familiar with the technical language or statistical reasoning behind NLP models, and there is a risk of either over-trusting or underestimating model results. To address this, outputs will be developed in plain language, and visual summaries and contextual explanations will be provided where appropriate. We will also hold demonstration sessions where appropriate and required. Emphasis will be placed on transparency and explainability, including the limitations of the models, the confidence of findings, and any underlying assumptions or preprocessing decisions that affect interpretation.

Contribution and potential impact

This study is uniquely situated at the intersection of data science and the DA field, with empirical, methodological and practical implications for both.

The study will assess the feasibility and utility of applying NLP to free-text data contained within DHRs. As many studies are often based on a sub-sample of DHRs, and large-scale analyses are limited by resource and capacity, this study will assess whether NLP can meaningfully identify and extract summary data from DHRs as well as understanding its capabilities for interpreting more complex relationships and causal pathways. To date, no study has been conducted utilising the entire corpus of DHRs available within the Home Office library. Therefore, this study has the potential to impact and develop the existing evidence base on DA, in particular, unpicking emerging trends and patterns through large-scale comparisons and interpreting associations between more latent or hidden issues.

This study will also provide a statement on the quality of DHRs as a mode of data collection for the prevention of DA. For example, we will apply NLP to identify the percentage of DHRs that report adequate data on victim, perpetrator, and case demographics (such as age, ethnicity or disability). We will use these findings to make recommendations to improve the conduct, quality and reporting of DHRs. For example, if NLP can assess levels of missing data or interpret differences between DHRs that are conducted in different ways (e.g., where there is family engagement), then guidance could be issued which evidences how DHRs are conducted and reported affects the quality of evidence. Improving the quality of DHRs has the potential to impact organisational learning, as well as identifying the conditions necessary (e.g., technical, structural, resource or otherwise) for developing DHRs as a source of knowledge for practitioners to utilise.

We will demonstrate the importance of ensuring that applications of NLP, including setting the scope, iteratively developing the models, and supervising and checking its results, are generated in collaboration with expert stakeholders. In this study, we will work directly with Imkaan who are involved in DHRs as panel members, are impacted by the recommendations that DHRs generate, and have extensive knowledge about DA and how services operate in response. This has potential for sector-wide impact in two ways: firstly, in improving the quality of reporting in DHRs and therefore the recommendations that they produce and, secondly, in supporting the identification and access to relevant evidence for organisations to conduct their own in-depth research upon.

Collectively, the five priority areas identified through the consultation demonstrate how NLP can support both substantive and procedural insights from DHRs. Analyses of demographic reporting and service involvement provide evidence on the completeness and distribution of key case information, enabling assessment of data quality and gaps in institutional knowledge. Examination of victim-blaming language offers a systematic means of identifying how victims are framed within DHR narratives, including whether such framings vary across demographic groups, with implications for guidance, training, and the interpretation of findings. Comparative analysis of different types of DA-related deaths supports a more nuanced understanding of pathways to fatal outcomes, while clustering of recommendations enables the identification of recurring patterns in organisational learning across sectors. These outputs move beyond descriptive analysis to provide a structured evidence base that can inform improvements in how DHRs are conducted, interpreted, and used to support prevention and policy development.

Conclusion

As the use of AI accelerates across a range of sectors, a distinct set of challenges emerge for the DA field [19]. Collaborating with expert stakeholders to undertake this feasibility study of applying NLP to DHRs means that we can better understand the benefits and limitations of these types of technologies. We hope this provides not only a technical assessment of whether NLP can be applied to DHRs but contributes to a collective discussion about how to make AI work meaningfully for the public sector. In doing so, this study provides a reflection on the methodological and ethical challenges of using AI in the context of DA and identify best practices for applying NLP in the future