The context in England and Wales

In England and Wales, SPS treat people with severe mental health problems who are deemed to be a serious risk to others. This includes those who are subject to criminal proceedings and those who cannot be safely cared for in general, non-secure mental health settings. SPS are commissioned at three security levels: low, medium and high, with the level of security determined by a service user’s assessed level of risk [4]. Referrals to SPS originate from community settings (including police custody], secondary mental health services, other SPS or the criminal justice system (subject to Part 3 of the Mental Health Act. revised 2007) [5]. Upon receipt of a referral, service users receive an ‘access assessment’ to determine if the criteria for admission to secure care are met and to consider any alternatives to admission to such a setting. Where admission is indicated, the appropriate level of security should be identified according to the principle of ‘least restrictive practice’ (as outlined in the Mental Health Act. revised 2007) [5].

Establishing who is eligible for admission to SPS is complex. Admission criteria are inconsistently applied and are dependent on a range of regional (e.g., variations in local population characteristics and individual SPS provision) [6] and individual factors (e.g., the nature of an individual’s disorder, the notoriety of their offence, lack of alternative arrangements) [7]. This disparity means that generalisation from studies of single SPS is not reliable. There are few national studies of SPS, and there is just one study from 2004 that has focused on factors associated with admission and acceptance for care [8]. This study captured data from 418 admission assessments across 34 medium SPS in England and Wales and examined the ways in which such services prioritised admissions [8]. Of those assessed as requiring medium security, 82% were accepted or placed on the waiting list [8]. In total, 77% of those accepted or placed on the waiting list for medium security were those assessed as requiring medium security while 23% were initially assessed as needing a lower security level [8]. This indicates that service users may be cared for in services with higher security restrictions than required, or possibly that security level requirements can change after an initial assessment [8]. The authors concluded that: (i) overall, more people require admission to medium SPS than can be accommodated; and (ii) the insufficient range of SPS provision available leads to the inappropriate use of medium SPS beds, where those who are assessed as needing lower levels of security are admitted over those who are considered to require long-term medium security [8]. This pattern has been observed across levels of security since the early 1990s [8–14]. There is a lack of research conducted more recently and the length of time since these studies were conducted limits current applicability and relevance, especially given continual developments in secure service provision.

Little is known about clinical decision-making in the context of SPS admission assessments. However, early qualitative research with clinicians working within SPS suggested that a range of contextual factors and individual clinician values and assumptions guide and influence who is accepted for admission to SPS [15]. Clinicians described decisions to admit as ‘discretionary professional judgements’, which are in part influenced by extraneous factors, such as relationships with external colleagues, policy context, an excess of demand over availability of beds, and the need to ensure turnover [15]. Whilst important for service management, these factors may not be directly relevant to the immediate treatment needs of an individual service user and might contribute to the inequity of access to SPS. This raises the question whether approaches to clinical decision-making when assessing for admission should account for extraneous factors or should be standardised with reduced influence of subjective judgement.

NHS England and NHS Improvement now delegate specified commissioning responsibilities for adult low and medium SPS to regional Provider Collaboratives (PCs) [16]. PCs involve partnerships between two or more trusts with one aim being reduction of inequality in outcomes and access to services [16]. Embedded within the PC model is the Access Assessment Service (AAS). This a single point of access referral pathway which streamlines the referral and assessment process, with the aim of reducing multiple assessments and delays. The standards for carrying out access assessments are outlined in the service specifications for low and medium SPSs [17, 18]. These standards describe how the AASs are responsible for delivering against specific timescales and that access assessments should be conducted within a structured framework which includes validated assessment guidelines [17, 18].

Assessment approaches

Assessment approaches in clinical practice, specifically for the assessment and management of violence risk in those with mental health problems, have developed extensively in the last 30 years. In England and Wales, the SPJ approach is proposed as a practical and standardised framework for clinical risk assessment and management [19, 20]. This approach focuses on harm prevention through an understanding of individual risk and what is required to address the risk and protective factors most relevant to mitigating adverse outcomes [21]. The SPJ approach is embodied in evidence-based guidelines or frameworks (they may also be referred to as ‘tools’ or ‘instruments’) that promote systematisation and consistency but are centred around discretionary rather than formulaic decision-making and are flexible enough to account for case-specific influences and the contexts in which assessments are conducted [22, 23]. The SPJ approach was developed because of the limitations of alternative approaches, specifically unstructured clinical judgement and non-discretionary formulaic approaches [23]. Unstructured clinical judgement involves clinician discretion and instinct in the absence of formal guidance [23]. As the unstructured approach lacked reliability and predictive accuracy, the non-discretionary approach was developed to increase reliability and validity, with set rules on what to consider when evaluating risk [23]. Using this approach, algorithmic instruments known as actuarial risk assessment instruments (ARAIs) are used to score the likelihood of a specific risk outcome and categorise individuals according to risk [23]. While reliability and predictive accuracy improved as a result of this development, this approach lacked flexibility and could not account for individual characteristics and context, such as different settings and change over time [23]. In light of these limitations, the SPJ approach was evolved to combine discretion with evidence-based guidelines [23]. The approach is particularly suited to the risk assessment and management task because the structure offered ‘helps to mitigate the biases and errors that plague all decisions made under conditions of uncertainty’ [24, p. 320]. There is an extensive development process for the development of a set of SPJ guidance, with recent examples including support for key assessment tasks such as risk formulation and scenario-planning, as well as comprehensive risk management planning.

While SPJ has a specific meaning in relation to risk assessment and management, the core principles of the SPJ approach are being reflected in other areas of practice involving complex decision-making, leading to subtle variations in how the term ‘SPJ’ is used. For example, the combination of explicit evidence to guide assessment, structure underpinning the decision-making process, and systematisation (which helps to increase transparency, reliability, accountability, and consistency), alongside maintaining the importance of discretionary clinical judgement, have inspired developments in admission decision-making [23, 25]. These developments have followed a similar trajectory to that for risk assessment and management. There have been attempts to move away from the established practice of using discretionary unstructured clinical judgement in admission decision-making to develop a standardised and reliable approach to the SPS admission assessment process. Thus, formulaic instruments focusing on the security and placement needs of those referred to SPS and based on clear and objective criteria for each security level [7, 26–28] were followed by guidance based on the SPJ approach [29, 30]. One such example is the Dangerousness, Understanding, Recovery and Urgency Manual (DUNDRUM), consisting of four sets of guidelines designed to support decisions related to admission and step-up/step-down security level movement [25]. The DUNDRUM toolkit has broadly adopted the principles of the SPJ approach [25]; while items in the DUNDRUM guidelines can be scored by clinicians–scoring is no longer encouraged in SPJ risk guidance–this is to support clinical judgement rather than produce a decision using an algorithm or ‘cut-off’ score [25].

There is an extensive international literature that provides evidence for how the SPJ approach, embodied in guidance, can be used effectively for risk assessment and management [31–33]. This body of research has to date focused on understanding risk in the individual case and using that understanding to guide interventions and therefore limit if not prevent further harmful behaviour. The research has improved general understanding of risk and protective factors alongside interventions that impact on harm prevention. This is to be expected as this is the context in which the SPJ approach was developed. However, SPJ risk guidance is being used in admission decision-making, and it remains to be seen how applicable this is in this context. In this review, the term SPJ approach is not limited to the SPJ approach used in the risk assessment and management literature. Instead, its use in this paper intends to convey a broader meaning where core SPJ principles, specifically a combination of discretion and systemisation, have been applied to admission decision-making to ensure consistent, accountable and evidence-based practice.

This paper involves a systematic search and narrative synthesis of international research concerned with the applications of the SPJ approach in SPS admission assessments and decisions concerning changes in level of security need. It is intended that findings from this review will provide a knowledge base that will help inform the future implementation of the SPJ approach in admission assessment practice for SPS, in England and Wales and also internationally.

Study inclusion/exclusion criteria

Literature search

Electronic digital databases.

The systematic search was conducted across the databases PubMed, PsycInfo, Medline, and Cochrane Library. The search focused on research published from January 2007 to August 2022, when the initial search was conducted. As the application of the SPJ approach to admission assessments has developed more recently, with the DUNDRUM Manual published in 2013, a 15-year search period was judged to be sufficient. The two discrete search strategies were initially piloted in PubMed and were adapted slightly for other databases to match subject headings specific to each platform. The quantitative search strategy included terms related to a population of psychiatric patients, structured professional judgement, admission, and secure services (see Table 1 of search terms). The qualitative search strategy included terms related to a population of clinicians, structured professional judgement, clinicians’ experiences, admission, and secure psychiatric services (see Table 2 for search terms). Searches were limited to English language studies but not a specific country. Searches were repeated in January 2024 to ensure the most recent research was included.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Quantitative search strategy.

https://doi.org/10.1371/journal.pone.0308598.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Qualitative search strategy.

https://doi.org/10.1371/journal.pone.0308598.t002

Grey literature

Grey literature was identified using Google and Google Scholar (first 100 outputs), alongside reading articles the research team were already aware of and searching reference lists of included articles. These alternative search techniques are recommended to ensure inclusion of all relevant articles in the review [35]. This is particularly important due to the possibility of limited evidence in this area.

Study selection

Articles returned from each database search were exported to Endnote (2013) to allow cross-referencing for duplicates. A list of titles was then extracted into Microsoft Excel. Titles and abstracts were screened independently by two members of the research team based on the inclusion and exclusion criteria. Reviewers first independently screened 5% of the abstracts and met to assess screening consistency before independently screening the remaining abstracts. Once screening was complete, reviewers cross-referenced articles identified as fitting the inclusion/exclusion criteria. In instances of disagreement around inclusion or exclusion, a full-text review was conducted. If an agreement was not reached following full-text review, a senior member of the research team decided whether to include the article. This title and abstract screening was followed by full-text review of the included articles to confirm eligibility for data extraction. This was completed by one member of the research team with support from another researcher where input was required. The screening and selection process is further illustrated in the PRISMA flow diagram [36] in Fig 1. As no qualitative articles were included following the full-text screen, the planned methodology for extracting and synthesising qualitative research was not utilised. Please see the published PROSPERO protocol (CRD42022351425) for the anticipated methodology [34].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flow diagram, adapted based on PRISMA guidance [36].

https://doi.org/10.1371/journal.pone.0308598.g001

Data extraction

One member of the research team extracted data from included articles. Extracted data provides insight into the use of the SPJ approach for admission assessments. This included the accuracy of the SPJ approach to inform admission decision-making. Endnote and Microsoft Excel were utilised to aid data extraction. The extracted data included: author, country, year, study title, research aim, study design, methodology, clinical setting, level of security, client group, sample size, sample characteristics, SPJ guidance utilised, outcome measure(s) and findings, alongside the accuracy of the SPJ approach to guide admission.

Quality assessment

The Mixed Methods Appraisal Tool (MMAT) was selected to conduct quality appraisal due to utility across a range of study designs including qualitative, quantitative (descriptive, randomised, and non-randomised) and mixed-methods studies [37]. Due to the inclusion criteria not limited by study design, this tool was judged to be appropriate for the purposes of the review. While only quantitative studies were identified from the search, this tool remained suitable by accounting for a range of quantitative designs, allowing all studies to be appraised using comparable criteria. The MMAT criteria includes the clarity of the research question, the suitability of the selected design to address the research question, participant selection, analysis, and interpretation [37]. The clarity of the research question was considered for all study designs, whilst the remaining criteria to assess study quality were specific based on research design. The tool does not employ a numerical scoring system, rather a more detailed judgement regarding quality is reached based on whether the answer was yes/no/not clear to the criteria. A judgement identifying limitations and areas for future research to improve was considered more helpful than a numerical score for this review, particularly given the possibility of limited research in the area. Two reviewers conducted the quality assessment, with a third resolving any disagreements. The quality was variable across studies, although certain limitations were more prevalent such as a lack of accounting for confounding variables, lack of clarity regarding outcome measures and lack of information regarding the selected sample. All studies were included, regardless of perceived quality, as recommended when using the MMAT [37] and to present the current evidence base completely. However, the limitations are highlighted and discussed. The complete quality appraisal is presented as supporting information [S1 and S2 Tables].

Analysis

This paper was a systematic review that involved no analysis of data.

Data synthesis

The minimum number of studies required for data synthesis in this review was one. Findings were first grouped by SPJ guidance and then further grouped by outcome. Outcomes from the included studies are summarised to provide an overview of the current use of SPJ-informed guidance for secure service admission, which is focused on the accuracy of the SPJ approach to inform admission decision-making. The synthesis was guided by the Synthesis Without Meta-analysis (SWiM) [38]. The SWiM guidance “is a nine-item checklist to promote transparent reporting for reviews of interventions that use alternative synthesis methods of quantitative studies” [38, p. 1].

Description of full-text articles retrieved

The systematic search identified 1,247 articles, published between January 2007 and August 2022. Following removal of duplicates and screening, seven quantitative articles were identified as relevant to the review [39–45]. Five further quantitative articles were identified as relevant for the review through the grey literature search [46–50] resulting in a total of 12 articles included for quality appraisal and synthesis. Fig 1 shows the search process and how many articles remained following each stage [36]. No further articles met inclusion criteria when searches were repeated prior to completion of the review in January 2024. Extracted information for each study is presented in Table 3, grouped by study design. SWiM guidance recommends grouping studies to allow comparison and meaningful synthesis of findings across articles with a variety of research designs [38]. The extracted information was synthesised using narrative synthesis with findings grouped by SPJ guidance [51].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Information extracted from included articles, grouped by study design.

https://doi.org/10.1371/journal.pone.0308598.t003

Study characteristics

SPJ guidelines

The most frequently studied SPJ guidance was the DUNDRUM-1 triage security measure [n = 9; 40–46, 48, 49]. The DUNDRUM-1 intends to guide decision-making regarding the most appropriate security level for someone requiring psychiatric hospital admission [25]. The studies in this review explored the ability of the DUNDRUM-1 to indicate need for admission and/or differentiate between the security level required. The DUNDRUM-1 was initially developed with eleven items and all eleven items were used across six studies [40, 41, 43, 44, 46, 48]. Three studies [42, 45, 49] used the adapted nine-item scale (exclusion of item 2: Seriousness of Self-harm and item 4: Immediacy of self-harm) recommended due to two items found in earlier studies to correspond less with service user placement than the remaining nine [25]. The authors of the DUNDRUM-1 recommend calculating a mean score for each service user to indicate the most appropriate security level (mean >3 = high security, >2 = medium security, >1 = low security, 0–1 = open hospital or community) [25]. While calculating mean scores for each service user (before calculating an overall sample mean) is recommended, only two studies used the mean score for each service user across nine items [42, 45] with seven using service user total scores of eleven [40, 41, 43, 44, 46, 48] or nine items [49]. All findings should be read within this context and throughout the review findings DUNDRUM-1 score may refer to the mean or total score, depending on study.

The DUNDRUM-2 was included in five studies [41, 43, 45, 48, 49]. The DUNDRUM-2 intends to guide decision making regarding priority for admission to secure services from a waiting list [25]. The studies in this review explored the ability of the DUNDRUM-2 to indicate priority for admission alongside need for admission.

Five studies also included the HCR-20 [39, 40, 46, 47, 50]. While the HCR-20 [Currently on Version 3] was initially developed to support risk assessment and risk management decisions [52], the studies explored the ability of the HCR-20 to indicate a need for admission or distinguish between service users moved across security levels. Three studies used the HCR-20 (Version 2) [39, 46, 47] while one used the HCR-20 (Version 3) [50].

The DUNDRUM-3 and DUNDRUM-4 were the focus of three studies [46, 47, 50]. The DUNDRUM-3 and DUNDRUM-4 were developed to be used together and support decisions on service users’ readiness to move to a lower security level [25]. The DUNDRUM-3 is specifically intended to indicate whether treatment programmes have been completed to reduce risk while the DUNDRUM-4 focuses on recovery [25]. The studies in this review explored how ratings on the DUNDRUM-3 and DUNDRUM-4 may distinguish between service users either moved to higher or lower security levels or not moved between security levels. The remaining synthesis is presented according to SPJ guideline to allow for direct comparison of findings. Item level findings are presented as supplementary information (S3 Table and S1 Text).

DUNDRUM-1

Need for admission. Five studies examined how the need for admission related to DUMDRUM-1 outcome and whether the DUNDRUM-1 can distinguish between those requiring or not requiring admission, as assessed by clinicians [40, 41, 43, 45, 48]. It was observed that those admitted to high secure hospital (F = 4.21, p = 0.042 [45]; p<0.001 [41]), medium secure services (p<0.001 [41]) and ‘general security’ hospitals (t = 2.10, p = 0.04 [43]) had significantly higher DUNDRUM-1 scores than those not admitted. Similarly, those recommended for admission by assessing professionals had significantly higher DUNDRUM-1 scores (Forensic psychiatrist: mean = 3.58, SD = 0.31; Forensic social worker: mean = 3.55, SD = 0.30) compared to those not recommended (Forensic psychiatrist: mean = 3.34, SD = 0.38; ANOVA F = 22.83, df. = 1, p < 0.001; Forensic social worker: mean = 3.33, SD = 0.42; ANOVA F = 16.21, df. = 1, p < 0.001) [45]. Likewise, those admitted to secure services were observed to have significantly higher DUNDRUM-1 scores when initially placed on a waiting list (F = 4.4, df = 1. P = 0.04) and when leaving the waiting list (F = 7.6, df = 1, p = 0.008), compared to those not requiring admission [48]. Prisoners requiring a full psychiatric assessment that were subsequently admitted to any available psychiatric hospital (including but not limited to secure care; mean = 15.77, SD = 5.33) had significantly higher DUNDRUM-1 scores compared to those not admitted and receiving psychiatric follow-up (mean = 4.14, SD = 4.57) or discharged to the prison GP (mean = 0.21, SD = 0.79; Bonferroni statistics not provided) [40]. Likelihood of admission was also observed to increase as DUNDRUM-1 scores increased (Total: OR = 1.13, 95%CI 1.05–1.21, p<0.001) [43].

Required security level. Six studies examined how DUNDRUM-1 scores may be able to differentiate between those requiring varying levels of security [40–44, 49]. The first DUNDRUM-1 study observed that, among those diverted from prison to psychiatric hospital, there were significantly higher DUNDRUM-1 scores for those admitted to higher levels of security, however post-hoc Bonferroni statistics were not reported: Medium and high secure mean = 22.87 (SD = 4.56); Psychiatric intensive care unit [PICU] mean = 15.81 (SD = 4.39); Open wards mean = 10.74 (SD = 3.26) [40]. Five studies further examined the ability of the DUNDRUM-1 to differentiate between required security level in settings other than where the DUNDRUM-1 was initially developed [41–44, 49]. Initial analyses indicated that there were significant differences in DUNDRUM-1 scores across security levels (H = 0.737, df = 2, p<0.000 [44]; p<0.05 [42]; p<0.05 [49]), however did not indicate between which security level these differences were present. Further post-hoc analyses indicated that those placed in low secure settings (mean = 23.86) had significantly higher DUNDRUM-1 scores compared to those placed in open settings (mean = 20.00; p<0.050 [44]) and significantly lower DUNDRUM-1 scores compared to those placed in medium security settings (p<0.001 [41]; mean = 29.64 p<0.005 [44]) and high security settings (p<0.001 [41]). Service users placed within medium security settings had significantly higher DUNDRUM-1 scores compared to those needing open settings (p<0.005 [44]; p<0.005 [49]) and PICU (p<0.001 [41]) and significantly lower DUNDRUM-1 scores than those needing high security (p<0.01 [42]). Scores for service users needing high security were also significantly higher than those needing open settings (p<0.01 [42] p<0.005 [49]). Those requiring high intensity hospital in Canada also had significantly higher DUNDRUM-1 scores than those admitted to general security hospital (t = 3.27, p = 0.003) [43]. In the context of a Belgian forensic psychiatric setting, court recommendations for higher security levels were significantly, although weakly associated with higher DUNDRUM-1 mean score (r_s = 0.25, p<0.05) [42].

Service user movement between security levels. One study examined how DUNDRUM-1 scores may be able to differentiate between those moved, or not, between security levels [46]. Higher DUNDRUM-1 scores were associated with decreased likelihood of movement to lower security (OR = 0.60 95% CI 0.39–0.90, p = 0.014) and increased likelihood of movement to higher security (OR = 1.22, 95%CI 1.01–1.48, p = 0.038) [46]. Scores were significantly higher for those moved to higher security levels (F = 3.8, df = 1, p = 0.053) and significantly lower for those moved to lower security levels (F = 5.2, df = 1, p = 0.026) [46].

Relationship between clinical or court decision and DUNDRUM-1 scores. Two studies examined the percentage of court decisions and DUNDRUM-1 scores in agreement regarding the required security level [42, 49]. One study observed that a greater percentage of court decisions recommended high security compared to the DUNDRUM-1, with 50% of cases in which the DUNDRUM-1 recommended medium security corresponding to a high-security court decision [49]. While 83% of DUNDRUM-1 scores indicating high security corresponded to high-security court decision, only 33% of scores indicating medium security corresponded to medium-security court decision [49]. This was supported by another study observing that overall agreement between court recommendations and DUNDRUM-1 score (40%) was poor (k = 0.06, p = 0.15), with court recommendations indicating higher security more frequently than the DUNDRUM-1 score (45.5%) [42].

One study compared (non-SPJ) clinical decision-making with security levels indicated by the DUNDRUM-1 [41]. Comparison of a clinical recommendation by a forensic psychiatrist with a panel-scored DUNDRUM-1 indicated agreement in 66% of cases (n = 107), although it should be noted that the DUNDRUM-1 was scored by the panel based on information provided by the clinician [41]. Clinicians recommended admission significantly more frequently than the panel using the DUNDRUM-1 (n = 51; 31.5% of cases), with only four cases (2.5%) where the panel recommended admission when the clinician had not, (χ2(1) = 27.3, p < .001;) [41].

Predictive validity of the DUNDRUM-1. Eight studies assessed the predictive validity of the DUNDRUM-1 [40–44, 46, 48, 49]; six observed admissions to secure services as the outcome, one observed movement between security levels [46] while another study examined the predictive validity of the DUNDRUM-1 for admission from the waiting list [48]. The predictive validity of the DUNDRUM-1 was measured using receiver operating characteristics (ROC), specifically the Area under the Curve (AUC). Four studies found that the DUNDRUM-1 was predictive for indicating admission to secure services, with the AUC ranging from 0.75 (95%CI 0.63–0.88) to 0.98 (95% CI 0.97–0.98) [40, 41, 43, 44]. The highest predictive accuracy for admission when compared to no admission was identified in the first DUNDRUM-1 validation study among a sample identified for full psychiatric assessment (AUC = 0.98, 95%CI 0.97–0.98) [40].

The findings were variable regarding whether the DUNDRUM-1 was predictive for decisions to admit to low, medium, or high secure services. It was found to be predictive of medium or high secure admission when compared to PICU (AUC = 0.87, 95%CI 0.78–0.95 [40]), alongside distinguishing between medium secure and high secure admission (AUC = 0.87, 95%CI 0.72–1.00 [41]) and medium secure and PICU/low secure admission (AUC = 0.91, 95%CI 0.84–0.98 [41]). Two studies found the DUNDRUM-1 to be predictive of high secure admission (AUC = 0.63, p<0.01- AUC 0.70, p<0.05 [42, 49]) and able to distinguish between high intensity and general hospital admission (AUC = 0.83, 95%CI 0.67–0.98 [43]).

The DUNDRUM-1 was also predictive of admission from the waiting list, distinguishing between those admitted or not admitted, both based on scores when services were placed on the waiting list (AUC = 0.79, 95%CI 0.67–0.90) and when they were removed from the waiting list (AUC = 0.91, 95%CI 0.84–0.99) [48]. Location upon referral did not influence this predictive accuracy as the DUNDRUM-1 was predictive of admission for those referred while on remand (AUC = 0.94, 95%CI 0.87–1.00), sentenced (AUC = 0.89, 95%CI 0.73–1.00) or in less secure hospitals (AUC = 0.88, 95%CI 0.59–1.00) [48]. With regards to movement between security levels, the DUNDRUM-1 was only predictive of ‘negative moves’ to higher security levels (AUC = 0.70, 95%CI 0.54–0.86) [46]. Comparisons for non-secure environments are presented in Table 3.

Inter-rater reliability. Two studies measured the inter-rater reliability of the DUNDRUM-1 [40, 49]. One study reported that 18 cases were independently rated by two clinicians [40]. Cohen’s Kappa was calculated for 7/11 items and was greater than 0.85 (p<0.001) for these items, indicating strong agreement between the two clinicians [40, 53]. There was also a significant correlation between DUNDRUM-1 scores of both clinicians for all 11 items (correlation greater than rs = 0.75, p<0.001 for each item) and the total scores (for 11 items) of both clinicians were significantly correlated (rs = 0.96, p<0.001) [40]. The second study reported excellent inter-rater reliability for the total DUNDRUM-1 score, although inter-rater reliability was better for those that had received DUNDRUM training (Intraclass correlation coefficient [ICC] = 0.90) compared to the rater that had not (ICC = 0.50) [49].

DUNDRUM-2

Priority for admission. Four studies examined whether the DUNDRUM-2 could prioritise those most urgently in need of admission [41, 45, 48, 49]. The initial validation study observed that the DUNDRUM-2 was able to prioritise those admitted from the waiting list over those not admitted [48]. Those admitted scored as higher priority on the DUNDRUM-2, with significantly higher scores compared to those not admitted, both when placed on the waiting list (F = 11.00, df = 1, p = 0.002) and when leaving the waiting list (F = 33.2, df = 1, p<0.001) [48].

Three studies examined priority for admission by investigating the relationship between how urgently a service user was admitted to secure services and DUNDRUM-2 score [41, 45, 49]. Scores were not significantly associated with time to admission [41] or the speed of diversion from prison [49]. In a high secure setting, those admitted to two different treatment pathways (mental illness or personality disorder) did not differ in urgency of need for admission, based on DUNDRUM-2 score [45].

Need for admission. Research in a forensic psychiatric setting in Canada investigated what they termed “need for admission” based on the DUNDRUM-2 score and observed a significant relationship, with higher scores related to increased likelihood of admission (OR = 1.36, 95%CI 1.18–1.57, p<0.001) [43].

Predictive validity of the DUNDRUM-2. Two studies measured the predictive validity of the DUNDRUM-2 for indicating admission [43, 48]. The DUNDRUM-2 was able to differentiate between those that were or were not admitted from the waiting list, including at two time points of service users being placed on the waiting list for admission (AUC = 0.76, 95%CI 0.63–0.88) and when leaving the waiting list (AUC = 0.79, 95%CI 0.67–0.91) [48]. When comparing groups on the waiting list, scores were predictive of admission for those on remand (AUC = 0.91, 95%CI 0.81–1.00) but not those sentenced or in a less secure hospital setting [48]. When the DUNDRUM-1 was added as a covariate, the model remained predictive both when service users were accepted onto the waiting list (Combined AUC = 0.79, 95%CI 0.68–0.90 compared to DUNDRUM-2 AUC = 0.76 95%CI 0.63–0.88) and when leaving the waiting list (Combined AUC = 0.87, 95%CI 0.78–0.96 compared to DUNDRUM-2 AUC = 0.79 95%CI 0.67–0.91) [48]. With regards to predictive validity for admission to a specific security level, the DUNDRUM-2 was predictive of need for admission compared to no admission required (AUC = 0.84, 95% CI 0.74–0.94) and Canadian general security hospital admission compared to no admission required (AUC = 0.81, 95%CI 0.67–0.94) however was not predictive of high intensity admission [43].

Inter-rater reliability. Inter-rater reliability of the DUNDRUM-2 was examined across two studies, with one study reporting no significant difference in mean scores between raters and exact agreement was significant (p<0.05) across all items (Agreement ranging from 38%, p = 0.036 for Item 3: Suicide Prevention– 100%, p<0.001 for Item 5: Systemic) [48]. The second study reported overall average reliability (Three items with good reliability included Item 1: Current location, Item 3: Suicide prevention and Item 4: Humanitarian and three items scoring low reliability included Item 2: Mental health, Item 5: Systemic and Item 6: Legal urgency) [49].

HCR-20, versions 2 and 3

Need for admission. One study evaluated the extent to which differences in HCR-20 (Version 2) scores could distinguish between those admitted or not admitted [39]. Those admitted to a medium secure personality disorder service did not significantly differ in HCR-20 total scores compared to those not admitted [39].

Service user movement between security levels. Two studies examined the extent to which the HCR-20 could differentiate between service users moved to higher security levels, lower security levels or remaining at the same security level [46, 47]. Service users moved to higher security levels had significantly higher total HCR-20 scores (F = 4.0, df = 1, p = 0.048) and this difference remained significant when location at baseline was included as a covariate (F = 7.1, df = 1, p = 0.009) [46].

When comparing scores on individual subscales, higher scores on the HCR-20 Dynamic scale (Combined Clinical and Risk scales) were observed to be associated with a reduced likelihood of movement to lower security (OR ranging from 0.41 95%CI 0.204–0.821, p = 0.012 [46] to 0.77 [95%CI 0.66–0.90, p = 0.001 [47]) and increased likelihood of movement to higher security (OR = 1.27 [95%CI 1.05–1.54, p = 0.014 [46]). Those moved from high security to medium security had significantly lower scores on the HCR-20 Dynamic scale (Combined Clinical and Risk scales) than those not moved (F = 33.199, p<0.001) [50]. With location at baseline included as a covariate, the HCR-20 Dynamic scores were significantly higher for those moved to a higher security level compared to those not moved or moved to a lower security level (F = 6.8, df = 1, p = 0.011) [46]. When the components of the HCR-20 Dynamic scale, the Clinical (HCR-20-C) and Risk (HCR-20-R) scales, were evaluated separately, those moved from high to medium security scored significantly lower on the HCR-20-C scale compared to those remaining in high security (F = 30.993, p<0.001) [50]. Similarly, the HCR-20-C score was significantly lower for step-down movements to lower security compared to no move or step-up movement to higher security (F = 11.2, p = 0.001) and significantly higher for step-up movements to higher security compared to no move or step-down movement to lower security (F = 4.8, df = 1, p = 0.031), although these differences were only present when location at baseline was included as a covariate [46].

The HCR-20-R score was significantly higher for those moved to higher security levels compared to no move or step-down movement to lower security (F = 4.7, df = 1, p = 0.03) [46]. This difference remained significant with location at baseline as a covariate (F = 6.9, df = 1, p = 0.01) [46]. Those moved from high to medium security scored significantly lower on the HCR-20 Future scale score than those remaining in high security (F = 22.88, p<0.001) [50]. One study observed that the HCR-20 Historical (HCR-20-H) score was significantly higher for those moved to lower security levels compared to those not moved or moved to higher security (F = 4.7, df = 1, p = 0.034), although this difference was no longer significant when location at baseline was included as a covariate [46].

The predictive validity of the HCR-20 for access assessments. The two studies examining the predictive validity of the HCR-20 used service user movements to higher or lower security levels as the outcome measure [46, 47]. The HCR-20 Dynamic scale (Combined clinical and risk scale) was found to be predictive of positive moves to lower security levels (AUC = 0.791, 95%CI 0.68–0.91, p<0.001 [47]) and negative moves to higher security levels (AUC between 0.70, 95%CI 0.54–0.87, p = 0.048 and 0.71, 95% CI 0.55–0.86, p = 0.063 [46, 47]). The HCR-20 total score and remaining sub-scales (the historical, clinical and risk management subscales separately) were not significantly predictive of service user movements.

DUNDRUM-3 and DUNDRUM-4

Service user movement between security levels. Three studies examined how DUNDRUM-3 and DUNDRUM-4 scores may differentiate between those moved or not between security levels [46, 47, 50]. While all three studies focused on service user movement as the outcome, one study specifically focused on movement from high security to medium security, with those moved to medium-security having significantly lower DUNDRUM-3 (F = 9.24, p<0.001) and DUNDRUM-4 scores compared to those remaining in high security (F = 6.86, p = 0.001) [50].

Two studies compared movements to both higher (step-up) and lower (step-down) security levels [46, 47]. The more recent study observed that DUNDRUM-3 scores were significantly higher for those moved to higher security (F = 5.4, p = 0.023) and significantly lower for those moved to lower security (F = 9.4, p = 0.003) compared to those not moved [47]. Higher DUNDRUM-3 scores were associated with reduced progress and decreased likelihood of movement to a lower security level (OR = 0.88, 95%CI = 0.81–0.97) [47]. Meanwhile, the earlier study observed that DUNDRUM-3 scores were only significantly higher for those moved to higher security compared to no move or move to lower security (F = 6.2, df = 1, p = 0.015) and significantly lower for those moved to lower security compared to no move or move to higher security (F = 5.4, df = 1, p = 0.022) when location at baseline was included as a covariate [46].

Both studies that measured step-up and step-down movements observed that those moved to higher security levels had significantly higher scores on the DUNDRUM-4 compared to those that were not moved to higher security (F = 6.9, p = 0.011 [47]) and those not moved or moved to lower security (F = 4.2, df = 1, p = 0.043 [46]). Service users moved to lower security levels also had significantly lower DUNDRUM-4 scores compared to those that were not moved to lower security (F = 6.7, p = 0.012) [47]. However, in one study lower scores for those moved to lower security levels compared to those not moved or moved to higher security levels were only observed when location at baseline was included as a covariate (F = 5.7, p = 0.02) [46].

Predictive validity of the DUNDRUM-3 and DUNDRUM-4. Two studies examined whether the DUNDRUM-3 and DUNDRUM-4 were predictive of service user movements across security levels [46, 47]. The DUNDRUM-3 and DUNDRUM-4 were found to be predictive of moves to higher security levels in both studies [46, 47]. DUNDRUM-3 AUC ranged from 0.64 (95%CI 0.51–0.87) to 0.76 (95%CI 0.59–0.93) while DUNDRUM-4 AUC ranged from 0.72 (95%CI 0.57–0.87) to 0.78 (95%CI 0.64–0.93) [46, 47]. Both measures were also found to be predictive of moves to lower security levels in one of the two studies (DUNDRUM-3, AUC = 0.72, 95%CI 0.59–0.85 and DUNDRUM-4, AUC = 0.70, 95%CI 0.56–0.83) [47].

Use of the SPJ approach for admission decision-making

There is an emerging evidence base about the use of the SPJ approach to support admission decision-making. However, the current evidence does not indicate the replacement of other decision-making processes with the use of SPJ guidance only. Across the evidence presented in this review, the admission recommendation based on the scoring of the studied SPJ guidance was compared to service user placement based on the usual decision-making process, such as court decision or clinical assessment. This demonstrates that SPJ guidelines can, at times, fit with existing decision-making, indicating potential value as a complementary approach. However, it does not provide evidence for the use of the SPJ approach in preference to existing procedures. While, theoretically, SPJ-informed guidance might be able to increase consistency and systematise decision-making, further investigations across a wider range of security levels, locations, and service types would be needed to confirm this. Establishing inter-rater and intra-rater reliability of SPJ guidance in the context of admission decision-making is also important. While inter-rater reliability was briefly examined for the DUNDRUM-1 [40, 49] and DUNDRUM-2 [48, 49] it was not widely explored across all SPJ guidelines or studies. This should include considerations of whether, as observed in one of the included studies, training in using a particular set of guidelines can improve reliability [49]. Developments in the consideration of protective factors, such as the Structured Assessment of Protective Factors for Violence Risk (SAPROF), could also be explored in the context of admission decision-making [54].

Contextual and service factors have a role in admission decision-making [15]. One such factor is location at referral, as highlighted within the included articles. Location at referral influenced whether scores on the chosen SPJ guidance (DUNDRUM-2, DUNDRUM-3, DUNDRUM-4 and HCR-20) could differentiate between service users, both for decisions regarding priority for admission and movement between security levels [46, 48]. Among high secure hospital referrals, those referred from prison were observed to have an increased need for admission, reflected through higher DUNDRUM-1 scores, than those referred from medium secure hospitals [45]. This raises the question of whether SPJ guidelines should take such factors into account, in line with the original purpose of the SPJ approach to account for context, or to improve equality regarding who is admitted, contextual factors should have a reduced role in admission decision-making. The interaction between such factors and SPJ guidelines during admission decision-making is an area requiring further exploration.

The evidence presented should be interpreted while considering the origins and subtle variations in what is meant by the SPJ approach. The integral element of the SPJ approach, as it was developed, is the use of clinical discretion while enabling consistency and systematisation [55]. These core principles form the basis of the HCR-20 and were used to develop the DUNDRUM toolkit. However, admission assessment research has focused solely on statistical associations with and predictive accuracy of service user placement or movement, as opposed to the integration of these guidelines into clinical practice. Throughout the research in this review, the chosen SPJ guidance was scored, with scores being compared and used to calculate predictive accuracy. While from a research perspective, this can provide a robust evidence base, it does not represent an SPJ-informed approach as initially intended, and in fact, more closely reflects an actuarial approach. This is a valuable first step in investigating the validity of these guidelines for admission assessments. However, the current evidence base does not reflect the context of intended use. To truly adhere to the principles of SPJ, future studies should examine application in clinical practice without focusing on scores and gain clinicians’ perspectives regarding the use and perceived acceptability of these guidelines.

Limitations and future directions

Regarding limitations in the review methodology, a narrative synthesis addressed the aims of the review. However, if more research was available, across different settings and samples, a future systematic review using meta-analysis would be pertinent. A meta-analysis would enable the investigation of wider aims beyond the scope of individual studies, including how consistently SPJ guidelines are predictive of admission decision-making across a range of services and service user populations [56]. The robustness of the review is also, to some extent, affected by the limitations of the included papers. Not all study designs incorporated blinding to outcomes between those rating SPJ guidelines and making decisions regarding admission, this limits predictive validity as it is unclear whether lack of independent decision-making can explain agreement between the SPJ guidelines and alternative clinical assessment. Due to reporting errors and missing values required to interpret findings, some findings are omitted from the synthesis, to avoid false interpretation, but are highlighted in Table 3. Across the research, there were also differences in the scoring of the DUNDRUM-1, with mean score for each participant and total score for each participant both being used to calculate an overall average score. The authors of the DUNDRUM-1 recommend calculating a mean score for each service user to correspond with the most appropriate security level [25]. Seven studies did not score in this manner and deviation from intended scoring impacts the integrity of study findings. Inconsistencies in the total number of DUNDRUM-1 items used were also observed, with six studies using all 11 items whilst three used nine items, omitting two items relating to self-harm. By applying different parameters while using the same SPJ guidance, comparisons between findings are less robust as admission decisions are not based on the same DUNDRUM-1 criteria.

While both national and international research was identified, there was a limited number of articles. The identified research spanned five countries in total. There was one study in Canada, the remaining studies were conducted across Europe, this offers a Westernised, primarily Eurocentric perspective, limiting the international picture. With five studies in England and Wales, the national evidence is also based on a limited amount of research. Evaluating the use of SPJ-informed guidance for admission internationally offers information on application in different contexts, however, it is difficult to compare findings with a limited number of studies spread across different countries. The variability in service provision and decision-making further limits the extent of comparability. For example, in Belgium, only high secure services are obliged to accept all service users sent by the court, resulting in an increased likelihood of court recommendations for high security. As this policy does not apply to all countries included in the review, the decision-making processes may be context-dependent and influenced by different factors across studies. Multiple studies in a particular country could indicate how the application of SPJ guidelines may work nationally, and comparison of findings and national patterns across countries could then enhance the international evidence base. Specific researchers, in some instances those that developed the guidance, were consistently a part of much of the research in this review. Having experts within the research team, while to some extent helpful, limits understanding of how SPJ guidelines are used by researchers and, ultimately, clinicians that are less familiar with the guidance. The next step would be for more research teams to expand the investigation to new settings.

While the secure service population is predominantly male, reflected in the predominantly male sample included in much of the research, admission decisions for samples outside of the majority are equally important. One study included an all-female sample and small female samples were included in three studies. However, there is a clear gap in the research regarding the use of SPJ-informed guidance for admission among women. Secure service types include those specifically for women, with admission criteria differing from services for men, and it is unclear how applicable current SPJ guidelines would be for admission to women’s secure services. Women requiring secure service admission do not present in the same way as men, with an increased prevalence of borderline personality disorder, complex trauma, and disordered attachments, alongside the prevalence of serious self-injury or attempted suicide [17, 18]. While DUNDRUM-1 items related to self-harm were omitted from three studies due to previous findings of reduced association with placement, further research should explore whether these items may be more beneficial for women’s secure service admission assessments. There are a range of SPS for specific populations, including acquired brain injury, learning disability, autism, and personality disorder but there remains a gap regarding how SPJ guidelines could be used across these more specialist services. Insight from service users who have undergone admission assessments regarding their experiences would provide an important perspective and could guide developments in SPJ guidelines tailored to specific populations and their needs.