Monitoring risk assessment on an acute psychiatric ward: Effects on aggression, seclusion and nurse behaviour

Evidence of risk assessment procedures is scarce and inconclusive. The aim of this study is to evaluate the effects of risk assessment on aggression and the use of coercive interventions in an acute psychiatric admission setting. In addition, we evaluated nurse behaviour before and after the use of risk assessment. To take the fluctuations with regard to aggression and coercive interventions into account, we allowed 26 weeks for baseline measurements, followed by a 26 weeks steady-state period after the implementation of the risk assessment instrument. Contrary to expectations, no positive effects of risk assessment were found on aggression or on coercive interventions. Time spent in seclusion increased significantly with more than 10 hours on average after implementation. Furthermore, there were only negative effects on nurse behaviour and experiences. Among other things, they felt more stressed, spent more time on administration tasks and spent less time with patients after the implementation. In conclusion, there is insufficient evidence to use structured short-term risk assessment to reduce aggression or coercive interventions.


Introduction
Aggression and violence in psychiatric wards remain a major problem [1,2]. The use of risk assessment tools in inpatient psychiatric settings may help to manage this problem [3,4]. Risk assessment tools are developed to monitor the possibility of aggressive behaviour in individual patients and allow the staff to anticipate problems and make decisions to adjust management strategies and prevent further escalation.
Risk assessment is no panacea. One of the main criticisms is that it is useless when no reasonable intervention exists or is available to reduce future harm [5]. It is questionable if the staff will actually adjust their routines to prevent further escalations on basis of the risk assessment measurements. Also, there is little evidence for the effectiveness of risk assessment in reducing aggression. Accordingly, the ethical dilemma has been raised why to identify highrisk patients and treat them differently when risks are not actually prevented [5][6][7][8][9]. There are four published controlled studies that assess the use of risk assessment tools to reduce harm and aggressive incidents in different settings. Abderhalden et al. [10] and Van de Sande et al. [11] found a reduction in violence and aggression incidents when risks are monitored systematically on wards for acute inpatients. They also found a reduction in (the duration of) coercive interventions. In contrast, Kling et al. [12] found no beneficial effects in an acute hospital setting. Also, Troquette et al. [13] found no effects of risk assessment on violent or criminal behaviour in an out-patient forensic setting. None of these studies focussed on changes in staff behaviour towards patients. In short, the evidence for the use of risk assessment tools is inconclusive. Viljoen et al. [14] state in their systematic review that there is need for additional research.
This pilot study is an extension of the study by Van de Sande et al. [11]. It uses the Crisis Monitor [15] to assess violence risks as well as changes of staff behaviour and experiences on an acute psychiatric ward. In study 1 we assess the impact of the implementation of the Crisis Monitor on aggression and coercive interventions. We hypothesized that risk assessment reduces (the severity of) aggression. We also predicted a reduction of the number and duration of seclusions. Since involuntary medication is considered less intrusive compared to seclusions, we assumed that its use will not be affected. The frequency of applying a specific oneon-one de-escalation protocol, was expected to rise as an alternative for seclusion or other coercive interventions. In study 2 we assessed behaviour and experiences of staff. We hypothesized that the nurses will spend more time on the ward floor with patients, reduce avoidance behaviour and be more present when there is tension and threat. These changes in routine practice by nurses are known to have a positive effect on the therapeutic milieu and in keeping the unit safe [16][17][18].
decisions were made to manage risks of individual patients, by introducing de-escalating interventions. This was especially the case when the total score of the BVC was above cut-off (>2). Also, the team reviewed the daily and weekly ratings for each patient in more detail in the weekly care coordination meetings. This double strategy allows for short and long-term decision making. As in the study by Van de Sande and colleagues (2011), administration of the daily instruments of the Crisis Monitor took approximately 5 minutes, and the weekly version took about 15 minutes per patient (average investment with 20 patients: 1700 minutes/week). The team practised de-escalating strategies such as the use of the intensive care unit and offered one-on-one treatment.
Measures. The same instruments assessed the baseline and the steady state post-implementation period (see Table 1 for the assessment schedule).
Crisis monitor. The Crisis Monitor, developed by Van de Sande et al. [15,19], is a risk assessment method which combines five standard observation scales (Kennedy short and long, BVC, BPRS, SDAS) to objectify the risk of violent behaviour in inpatient psychiatric settings.
Kennedy-Axis V: The Kennedy [20] has seven scales; i) psychological impairment, ii) social skills, iii) violence, iv) Activities of Daily Living (ADL)-Occupational skills, v) substance misuse, vi) medical impairment, vii) ancillary impairment. It generates a global functioning assessment and is an alternative to the DSM-IV-TR Axis 5 GAF Scale. In the Dutch version, an 8th scale ("motivation for treatment") is added [21]. The interrater reliability for the Dutch version assessed by nurses was 0.79 [22]. The short version of the Kennedy-Axis V consists of the first four daily-assessed domains. Rating of the extended version occurs weekly.
BVC (Brøset Violence Checklist): The BVC [23] was also assessed twice per day (after each shift except the night shift) and aims to predict violence within the next 24-hour period. Each of the six items (i) confusion, ii) irritability, iii) boisterousness, iv) physical treat, v) verbal treat, vi) attack on objects) is scored as present or absent. It has a sensitivity of 63% and a specificity of 92% with the suggested cut-off score of 2. The Kappa value for the total BVC score was 0.44.   [24,25], consisted of 16 items. The scale focusses primarily on inpatient psychopathology and is particularly suitable for psychotic patients. The BPRS was modified and expanded. The Crisis Monitor uses the BPRS version of Bigelow et al. [26]. It is a 26 item scale rated from 1 to 7 and was measured on a weekly basis. Factor analysis yields 5 clusters; "positive symptoms/thought disorder", "depression", "negative symptoms/anergia", "mania", "disorientation" [27].

SDAS (Social Dysfunction and Aggression Scale):
The primary outcome measure is the type and severity of aggression, recorded with the SDAS [28]. This instrument was completed every week (or at the day of discharge) for each patient admitted longer than three days. The SDAS consists of 9 items covering outward aggression and two questions (9 and 11) that assess inward aggression: i) non-directed verbal aggression, ii) directed verbal aggression, iii) irritability, iv) negativism, v) dysphoric mood, vi) socially disturbing behaviour, vii) physical violence to personnel, viii) physical violence to others, ix) self-mutilation, x) physical violence to things, xi) suicidal thought and impulses). Assessors rated the SDAS on 5-point scales. Interobserver reliability is 0.97. Cronbach's alpha coefficient of SDAS-9 (outward aggression) was 0.79, and the Loevinger coefficient of homogeneity was 0.40.
Argus and systematic digital registration of coercive interventions. Episodes of seclusion were recorded using the Argus scale which provides information about the incidence and duration [29,30]. A seclusion incident is a sequence of periods of seclusion separated by no more than 24 hours. Besides this, the systematic digital registration of separation and seclusion on the psychiatric wards was used to evaluate the administration of involuntary medication and the number of specific one-to-one treatment protocols on the intensive care unit (ICU), in the proximity of the psychiatric ward.
Statistical analysis. Although Van de Sande et al. [11] did report data regarding the use of the Crisismonitor, a classic power analysis was not useful because the N is mainly influenced by unpredictable fluctuations of seclusions and aggression incidents over time. That is why we maximised the duration of the baseline and post-implementation steady state period to feasible limits (both 26 weeks). Differences in patient and nurse characteristics between the baseline and new steady state period were tested by chi-squared and t-tests. Because some subjects were admitted both in the pre-and post-phase, and therefore data were nested within subjects, multi-level regression analyses were used to assess the effect of the intervention on aggression and coercive interventions.
All analyses were performed with Stata (version 14; Stata corp., College Station, TX, USA) function xtreg (for ordinal) or xlogit (for nominal variables). The significance level for all statistical tests was set at p < .05, two-tailed.
Study ethics. This study was in accordance with the legislation and ethical standards on human experimentation in the Netherlands and in accordance with the Declaration of Helsinki (amended version 2013). The study was approved by the Commission for Scientific Research of the Mondriaan Mental health Trust in Heerlen/Maastricht, the Netherlands.

Results
Inclusions amounted to 293 admissions (224 individual patients) during baseline and 252 admissions (208 individuals) during the post-implementation steady-state period. Thirty-five individual patients were admitted in both periods (see Table 2 for an overview of patient characteristics). We found no significant differences for gender, ethnic minority, involuntary admissions or diagnosis between patients in both periods. Patients during the post-implementation steady-state period were younger than the patients admitted at baseline (z = -11.39, p < .01).
Coercive interventions. The mean hours spent in seclusion increased significantly with more than 10 hours per episode after the introduction of the Crisis Monitor (β = 27.05, p = .03). The number of seclusion incidents did not differ between the two periods (β = -.05,

Methods
Participants. The second study assessed the experiences and behaviour of the nursing staff on the acute psychiatric wards of Mondriaan before and after the introduction of the Crisis Monitor. During baseline there were 38 nurses working on the ward, 19 men and 19 women, during post-implementation steady state there were 34 nurses working on the ward, 14 men and 10 women. All nurses on the acute psychiatric ward were asked whether they were willing to participate in the study. They all agreed to participate. Because only 2 nurses per shift were needed in the specific weeks of measurement, not everyone was included. A total number of 19 nurses (5 men, 14 women) were included during baseline and 22 nurses (5 men, 17 women) during post-implementation steady state period. All participants were fully informed and consent was collected.
Measures. PsyMate ™ . The Experience Sampling Method (ESM; [31]) was adapted to sample the staff behaviour and experiences in different situations. This information was registered by nurses using the PsyMate™ App. The PsyMate™ App is designed to monitor experiences and behaviour in daily life. It runs on smartphones or iPods, has a user-friendly interface and allows easy and flexible programming. At random moments during the day, nurses receive a signal on their device and are asked to respond to several questions in reference to the moment directly before the beep. Questions include: What are you doing (i.e. administration, in contact with patient, having a meeting), where are you (i.e. outside the ward, in room of patient, in shared communal areas) and with whom are you (i.e. alone, patient, colleagues). There were also questions about the experiences and feelings at that moment, scored on a 7-point Likert scale: "I feel relaxed", "I feel safe", "I feel stressed", "I feel in control", "I feel anxious", "I feel tired", "I feel cheerful", "I feel lonely", "I feel work pressure", "this beep disturbed me". During every 8-hour shift (3/day), 2 nurses registered information with the Psymate™. Each nurse received 6 random assessment moments per shift, leading to a maximum of 36 registrations each day. Each registration took less than 90 seconds. Registration days were clustered in blocks of 1 week (7days), once every 2 months, resulting in 21 assessment days during baseline and 21 in the post intervention steady state period (average investment: 48 minutes/week).
Statistical analysis. The data are clustered within individuals and therefore not independent. Consequently, multilevel linear regression techniques were used to analyse the ESM data using the xtreg or xtlogit packages from Stata (version 14; Stata corp., College Station, TX, USA) with subject id as random intercept factor. The significance level for all statistical tests was set at p < .05, two-tailed.

Results
A total number of 19 nurses were included during baseline and 22 nurses during post-implementation steady state period. 15 of these nurses were included in both periods. In both groups 5 male nurses participated in the study. Mean age of the nurses was 35 years at baseline and 32.6 years during the post-implementation steady-state period. Working experience as nurse was 9.9 years in baseline and 6.9 years in post-implementation period. There were no significant differences in age (χ 2 (15) = 5.21, p = .99), gender (χ 2 (1) = .07, p = 0.79), school level (χ 2 (1) = .03, p = .87) or working experience (χ 2 (16) = 6.43, p = .98) for both periods. 519 moments were documented with ESM during baseline period and 471 during post-implementation steady state period.
At random moments within the day nurses were asked what they did, where they were and with whom, and how they felt (see PsyMate™ summary data in Table 4). Overall, time spent on administration tasks increased with 26.5% (32.9/26), while time spent in contact with patients alone decreased more than 19% after the implementation of the Crisis Monitor. Also, nurses were 19% more often in the nursing post, an open counter on the ward and 27% less present at the communal spaces at the ward, such as the living room, kitchen, etcetera. In regression analyses, the overall tests of time budgets (what the nurses were doing at the moment, with whom they were and where they were) were not significant (χ 2 (5) = 2.06, p = .84; χ 2 (6) = 4.68, p = .59; χ 2 (7) = 10.97, p = .14 respectively). The PsyMate™ also assessed how nurses felt while at work. During the post-implementation steady-state period they were less relaxed and cheerful (β = -.56, p < .01; β = -.73, p < .01), they felt less safe and in control (β = -.27, p < .01; β = -.36, p < .01) and they felt more stressed, anxious and tired (β = .41, p < .01; β = .22, p < .01; β = .63, p < .01). The experience of work pressure increased (β = .40, p < .01) and they were more disturbed by the beeps of the Psy-Mate™ in new steady state period (β = 1.19, p < .01).
To evaluate the objective level of stress during the two periods of the study a post-hoc regression was analysed in which for every day the objective stress level is calculated using minutes spent in seclusion, number of administered involuntary medication and use of coercive interventions outside office hours. The overall objective stress level was higher in the postimplementation steady-state period compared to baseline (β = .43, p < .01). None of the subjective feelings measured with the PsyMate™ were related to the objective level of stress of that day (relaxed β = -.01, p = .81; safe β = -.01, p = .88; stressed β = .01, p = .78; in control β = .03, p = .44; anxious β = .01, p = .57; tired β = -.003, p = .95; cheerful β = -.02, p = .73; lonely β = -.01, p = .74).

Discussion
Our study did not confirm the anticipated beneficial effect of the use of a structured shortterm risk assessment tool in an acute psychiatric admission ward. Results showed no reduction of aggression nor in the use of coercive interventions. Time spent in seclusion increased significantly with more than 10 hours on average after the introduction of the Crisis Monitor in post-implementation steady state period when procedures were routine. These results contrast with the study of Van de Sande et al. [11]. They found beneficial effects on the reduction of aggression and time spent in seclusion using the same risk assessment tools as in the present study.
There are several possible explanations for these unexpected results. The patients differed between the control and experimental conditions. We tried to control for case-mix variation in admitted patients by using a 26-week window in the baseline and post-intervention steadystate period. We hoped this would have reduced potential differences between the groups. But it remains still possible that sampling fluctuations interfered with the results. It could be that during the post-implementation steady-state period a more difficult patient group was admitted than during baseline. This was the reason for a prolonged assessment period of 26 weeks. The strategy seems to have worked since there were no significant differences in patient characteristics in both periods, except for age. Moreover, the proportion of involuntarily admitted patients was lower in the post-implementation steady-state period. It is, therefore, improbable that the absence of positive effects is due to a more difficult patient group.
The teams in the studies by Van de Sande et al. [11] and Abderhalden et al. [10] were enrolled in a program to reduce seclusion and restraint and not blind for the condition. Expectancy and enthusiasm for non-coercive practices may have created non-intervention related differences between the wards. The staff of the present study was part of a nation-wide program to reduce coercive methods for some years and had already reduced these practices by 50% at baseline. The same team was involved in the pre and post phases, and changes were related to historical factors as well as a higher threshold for seclusion. With an already low baseline, a reduction is more challenging to reach. Staff in the present study were used to focus on the precursors of aggression in their routine clinical observations and respond accordingly without the help of a structured risk assessment procedure. Differences in staff or ward culture, unrelated to the introduction of the Crisis Monitor, were more pronounced in the control conditions in the Van de Sande et al. study. Therefore, the present study potentially could not demonstrate a positive effect of the Crisis Monitor.
The same team was in charge at baseline and follow-up. After the implementation of the Crisis Monitor nurses spent more time on administration tasks, spent less time with patients and felt more stressed at follow-up. Filling in a risk assessment tool is an administrative task, so it is not surprising that the nurses spent more time on this after implementation. However, an increase of more than 25% is unexpected, and this raises the question whether it is acceptable that nurses in an acute psychiatric ward spent more time on administrative tasks while they should be in contact with patients. In fact, in the post-implementation steady-state period the nurses spent 19% less time with patients. The administrative burden due to study-related additional data collection (questionnaires and PsyMate™) was equal in the pre-and post-periods and could not account for the difference. Also, PsyMate™ ratings refer to the moment before the beep and is not part of the administration time. On itself the administrative burden of the risk assessment (5 minutes by patient for 2 shifts each day and 15 minutes for each patient per week) results in an extra administrative burden of 25 hours each week on a ward with 22 patients and an average occupancy rate of 80%.
To further explore the reasons for the present results, we interviewed the nurse practitioner and the head nurse. Both were intensively involved in the process of implementing the Crisis Monitor. They reported that feelings of anxiety, stress and work pressure often are related to team composition. The presence of inexperienced nurses or non-permanent team members with a flexible contract is considered a stressor with potentially detrimental effects on the mood, feelings and behaviour of the other nurses. The head of nurses stated that there were 9 nurses in the baseline period with less than one-year experience at the specific ward compared to 6 nurses in post-implementation steady state period. Unfortunately, there are no specific records of the exact team composition (i.a. substitution forces) at the time the study took place, so there was no opportunity to examine the possible differences in team composition between baseline and post-implementation steady-state period.
According to the nurse practitioner and head nurse, another stressor is the anticipation that coercive interventions outside of office hours, with lower staffing, would be needed. The administrative data contained records of the specific time of the use of coercive interventions. A posthoc analysis was performed to examine whether nurses had to use more coercive interventions outside office hours during post-intervention steady-state period. It appeared not to be the case for both seclusion and the use of involuntary medication (β = -.12, p = .33; β = .08, p = .65 respectively). Overall, on weekend days in which generally fewer nurses are present at the ward no more coercive interventions occurred than on other days (seclusion β = 143.67, p = .38; involuntary medication β = -.08, p = .75).
A final factor reported by the nurse practitioner and the head nurse was the presence of a specifically challenging patient during four months of the post-implementation steady-state period. This person behaved particularly violent, and on many occasions, police had to be called in to make sure the nurses and doctors were safe when face to face contact took place or during coercive interventions. The multi-level regression analysis accounts for high-frequency subjects and posthoc analyses with outliers removed, ruled out that this particular patient caused the unexpected finding. But it is possible that his behaviour caused a more stressful ward climate.
Kling et al. [12] and Troquette et al. [13] showed no beneficial effects of the use of structured risk assessment. They studied different settings (acute hospital setting and out-patient forensic setting respectively) and found the same results as the present study. Van de Sande et al. [11] and Abderhalden et al. [10] found a positive effect of risk assessment, but their studies have limitations. In conclusion, there is insufficient evidence to use structured short-term risk assessment to reduce aggression or coercive interventions. Potentially, the use of such a tool can negatively influence the experiences and behaviour of the nurse staff.