Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Long Term Effect on Professionals’ Knowledge, Practice and Attitudes towards User Involvement Four Years after Implementing an Organisational Development Plan: A Controlled Study

  • Marit By Rise ,

    Affiliation Department of Public Health and General Practice, Norwegian University of Science and Technology, Trondheim, Norway

  • Aslak Steinsbekk

    Affiliation Department of Public Health and General Practice, Norwegian University of Science and Technology, Trondheim, Norway

Long Term Effect on Professionals’ Knowledge, Practice and Attitudes towards User Involvement Four Years after Implementing an Organisational Development Plan: A Controlled Study

  • Marit By Rise, 
  • Aslak Steinsbekk



Health service organisations are increasingly implementing user involvement initiatives according to requirements from governments, such as user representation in administrational boards, better information to users, and more involvement of the users during treatment. Professionals are vital in all initiatives to enhance user involvement, and initiatives to increase involvement should influence the professionals’ practice and attitudes. The implementation of a development plan intending to enhance user involvement in a mental health hospital in Central Norway had no effect on the professionals after 16 months. The objective was therefore to investigate the long term effect on the professionals’ knowledge, practice and attitudes towards user involvement after four years.


This was a non-randomized controlled study including professionals from three mental health hospitals in Central Norway. A development plan intended to enhance user participation was implemented in one of the hospitals, including establishing a patient education centre and a user office, purchasing of user expertise, appointing contact professionals for next of kin, and improving of the centre’s information and the professional culture. The professionals at two other hospitals constituted the control group. All professionals were invited to answer the Consumer Participation Questionnaire (CPQ) and additional questions, at a four year interval.


A total of 399 professionals participated (43% response rate). Comparing the changes in the intervention group with the changes in the control group, the results showed that the plan had improved some aspects of the professionals’ knowledge about the user involvement taking place in the hospital. In addition, some parts of the professionals’ practice of providing information to the service users was improved, and the development plan might have raised their awareness about insufficient involvement of next of kin.


This is the first controlled study on the long term effect on professionals from implementing a development plan to enhance user participation in a mental health hospital. Since there was more effect after four years than after 16 months, this study indicates that it takes time before the effect of complex interventions to enhance patient participation in organisations can be detected among the professionals. More long-term studies are thus warranted.


For several decades user participation in health care has been highly recognized and advocated as a way to improve and ensure the quality of health services [14]. There exist different definitions and models for user participation [59]. Service users can be involved in decision-making regarding their own treatment or be involved in development of the services; in evaluating the services through patient surveys, participating in decision-making in organisational settings such as committees or boards, or being involved in teaching, training or research processes [5].

To ensure user participation in the health services some Western countries such as Norway have legislated the users’ rights to participate, as well as the services’ duty to involve users [10,11]. This means that health service professionals are obligated to involve service users, both in individual treatment and in provision and development of services. Norwegian governmental regulations and important policy documents show high expectations towards user participation in quality improvement and decision-making; to improve treatment outcome, coordination of services, transparency and patients’ access to information [12]. To comply with the requirements, health service organisations are implementing various initiatives to involve service users in the services [1315]. However, studies have shown that user involvement is not a matter of course in the health services. Shared decision-making in health care consultations are not widely practiced [13] and although involvement of service users takes place to a certain degree in organizations [16], user participation in health service development is still not the rule [17].

The increased demand on health service organisations to incorporate patient participation in their everyday work [12] might pose challenges, since implementation of change in large organisations is difficult and often fails [1821]. Furthermore, the implementation of user participation in organisations represents very complex interventions. Complex interventions are described as interventions which include several interacting components, several organisational levels, several expected outcomes, and several behaviours which are required from those who take part [22]. A complex intervention attempts to introduce new patterns of collective action, or to modify existing patterns [23]. There are several factors which determine whether a complex intervention, such as the implementation of user participation, becomes a normalized part of the day-to-day practice [24]. Research investigating the implementation of new and complex initiatives in health service organisations has shown that important factors might be how the initiatives are “enacted” by the professionals and service users [25], how the initiatives are understood by those involved, and who has the responsibility for different parts of the work entailed [25,26]. Focusing only on staff given new tasks without paying attention to their organisational working context may also hinder successful implementation [26].

There exist many facilitators and barriers towards user participation. Involvement of users is highly dependent on the professionals’ knowledge about and attitudes towards involving service users in their daily work. Although service users must be willing and motivated to participate [9,27], professionals are equally vital for bringing about involvement of service users. User participation prerequisites collaboration and a good relationship between service users and professionals [2729], as well as the processes of dialogue and shared decision-making [9]. Collaboration between service users and professionals can be made difficult if they understand and practice user participation differently [28,30]. In addition, health service professionals are not always equipped with the necessary tools or guidance to be able to involve users [12,31]. Some have also described professionals’ resistance towards user involvement as an important barrier [29,32]. Studies have for instance shown that although professionals agree that service users should be involved in decisions regarding their own treatment, not all agreed that user participation should be extended to service development in organisational settings [33]. Similarly, professionals in some settings still prefer that service users have a consultative role in organisational settings instead of participating in genuine decision-making [32].

However, attitudes towards involvement of users can change over time. Nathan and colleagues investigated professionals’ attitudes before and after the appointing of community representatives in several decision-making committees in the health services [34]. Although professionals felt a lack of knowledge on how to work with representatives before representatives were appointed, one year afterwards, professionals felt that the representatives’ role in the collaboration was clearer, and they had more knowledge on how to work together with the representatives. And more importantly, more professionals felt that the involvement was beneficial and that the health services were ready for such involvement initiatives.

An argument for user involvement, which is used frequently, is that it improves the health services. However, the evidence of effects of user participation is so far mixed [12,14,28,35,36]. Research on initiatives to involve users in organisational settings have shown increased empathy and communication skills in trainees, improvement of information material and interview data, as well as more service user engagement [17]. In addition, involving service users in processes on healthcare improvement processes can change priorities [37]. Several have studied whether implementation of initiatives to involve users can improve the health services by resulting in more systematic integration of patient experiences, increased shared decision-making, higher patient satisfaction, more focus on user involvement, increased collaboration between patient and professional, and improvement of information material and communication, but showed a lack of effect [12,14,28,35,36]. Other expected outcomes in studies have been patient reported knowledge, quality of life, symptom strength, patient adherence [38], anxiety level before surgery, patient understanding and consumer engagement [39]. Overall, the evidence-base for an effect of user participation initiatives in individual treatment and in organizational settings for service development is weak [13, 17, 3840]. In summary, it is so far an unanswered question whether involving services users on different areas of the health services fills the expectations of improving the services.

Few studies have investigated the long-term effect from user participation initiatives. The reason might be that complex interventions and changes in large organisations often demand a liberal time frame [18,22]. None of the aforementioned studies lasted more than two years [12,14,28,35,36], and it is an open question if this is too little time for measurable changes to settle in the organisations.

In 2009 a comprehensive development plan intended to strengthen user involvement was implemented in a mental health hospital in Central Norway. Implementing multiple initiatives intended to enhance user participation ought to influence the professionals in the organisation. We therefore investigated the effect on professionals’ knowledge, practice, and attitudes towards user involvement 16 months after implementation. The study showed no effects after 16 months [40], and it was discussed whether 16 months might have been too short to see any effects from the development plan. This was strengthened by qualitative interviews exploring the professionals’ experiences with the implementation process where the employees said that the implementation process had taken a long time and that the changes probably were not fully manifested after 16 months [41]. Considering these factors and the lack of studies on long term effects, we wanted to investigate the effect of the development plan after four years.

The aim of this study was therefore to investigate whether implementing an organisational development plan intending to enhance user participation in a mental health hospital had any effect on the professionals’ knowledge, practice, or attitudes towards user participation four years afterwards. Our main interest was whether there was any change in the intervention hospital professionals’ knowledge, practice, or attitudes towards user participation, compared to the professionals in the control hospitals.


Study design and ethics

The present study was a non-randomized controlled study involving professionals from three mental health hospitals in Central Norway. One of the hospitals implemented in 2009 an organisational development plan to enhance user participation (the intervention hospital). The hospital was therefore chosen as the setting for a study to evaluate several aspects of the implementation of the plan. The other two hospitals participated as control group. All professionals working in these hospitals were invited to participate before the implementation and four years afterwards. The study took place from November 2008 to December 2012. The study was approved by the regional committee for medical and health research ethics in Central Norway, the Norwegian Data Inspectorate, and the hospitals’ management. Participants consented to take part in the study by filling out and returning the questionnaire. The ethics committee approved of this consent procedure.


The three mental health hospitals (district psychiatric centres) are part of the same university hospital trust. The intervention hospital covers a catchment area of 96.000 persons, with urban and semi-rural areas including parts of a large Norwegian city. The two control hospitals cover catchment areas of 74.000 and 47.000 persons respectively, with urban, semi-rural, and rural areas including parts of the above-mentioned city. The three hospitals provide the same types of service; in-patient treatment (5.4 beds per 10.000 inhabitants), out-patient treatment, and ambulatory services. The intervention hospital was relocated and reorganised in January 2009. Units were merged and co-localised and an ambulatory acute treatment team was established. The reorganisation was based on an over-arching organisational plan focusing on professional development and improvement of collaboration and patient flow. The reorganizational plan also included a development plan for user participation, which was the intervention in this study.


As part of the comprehensive structural reorganization and relocation, the intervention hospital formulated and approved an organisational development plan which was intended to increase user participation. The development plan was formulated by a project group. The group included administrators, health professionals, and user representatives (recruited from mental health user organisations). The group worked from fall 2007 to June 2008. The final plan was approved by the hospital trust in June 2008.

Table 1 gives an overview of the initiatives in the plan. The main aim was to enhance user participation on the individual and system level. The initiatives in the plan were chosen by the project group based on their clinical experience, knowledge about user participation, and the hospital administration’s aim for the health services they provide. We assessed the implementation status in April 2010—fifteen months after the implementation started—and found that most of the initiatives were completed (Table 1). This information was collected through interviews with key professionals and documents produced by the hospital. The implementation status in April 2010 was confirmed by the hospital’s management.

The development plan was implemented from January 2009. The hospital manager was in charge of the implementation process, and the everyday executive responsibility was ensured by one of three unit managers at the hospital. An implementation group consisting of the unit manager, several administrators, health professionals, and user representatives recruited from mental health user organisations was established in August 2009 to supervise and follow up the implementation. The group had six meetings until January 2010 when the group was dissolved.


All employees including psychiatrists, nurses, psychologists, other health and social workers, and administrative professionals in the three mental health hospitals were invited to participate in this study. They were identified from lists including all paid employees registered by the hospitals’ administrative offices the current month. To maintain anonymity and to invite all employees at both measurement points, we did not match the employees participating at the different time points. Although several employees probably participated at both time points, the group of employees answering the questionnaire were not exactly the same at the first and second time of measurement. They were thus treated and analysed as four independent samples. The differences in results from baseline to follow-up in sample 1 and sample 2 was compared with the differences in results from baseline to follow-up in sample 3 and 4.

Data collection

All employees at the intervention hospital were invited to fill out a paper questionnaire in December 2008 (before the implementation of the development plan) and in December 2012 (four years afterwards). All employees at the control hospitals were also invited to fill out the same questionnaire twice with a four year interval during the same period. All invitations to fill out questionnaires were sent by post to all employees’ private address with pre-paid return envelopes which were returned directly to the researchers. One reminder was sent in December 2008. No reminders were sent in December 2012.

Outcome measures

To measure knowledge, practice, and attitudes towards user participation among the professionals the Consumer Participation Questionnaire (CPQ) was used [42]. The questionnaire was translated to Norwegian for this study [40]. The original CPQ questionnaire includes 20 questions. To include question about user organisations, question no. 8 in the original questionnaire (Does your agency solicit user input for planning of mental health service?) was supplemented with a new question no. 8a (Does your agency solicit input from user organisations for planning of mental health service?). Question no. 14 in the original (Should users be involved in the evaluation and diagnosis of their presenting problems?) was split in two questions to ensure interpretable results (no. 14 and 14a). Answers to question no. 2 were dependent on the answer to no. 1 and the question was omitted in the analyses for this paper.

We also added 8 questions to ensure that aspects of the professionals’ views and practices regarding user participation not asked about in CPQ were included. These questions were formulated, discussed, and refined during meetings in the research group. The questionnaire used in this study thus included a total of 29 questions.

The questions were organised into three thematic areas for this study: Eleven of the 29 questions measured professionals’ knowledge on user participation, seven measured practice, and 11 measured attitudes.

Statistical analysis

The results from the two control hospitals were combined. Pearson’s chi square tests were used to identify any differences in proportions within the two groups at the two points of measurement. For each questionnaire item, differences in proportions between baseline sample and follow-up sample were calculated using binary logistic regression. We conducted analyses for the intervention hospital and the control hospitals, respectively. Demographic variables with trends for difference (p<0.1) within each group (intervention group; current position, control group; gender and current position, Table 2) were added to the time of measurement variable in the regression model. To compare the odds ratio (OR) in the intervention group to the OR in the control group we used a test of proportion [43] to calculate ratio odds ratios (ROR). A ratio odds ratio tells us whether there are any differences between the changes in two different groups. A test of proportions is used to compare two estimated quantities, such as means or proportions (in this case odds ratios) of the same quantity in two independent samples. All statistical analysis was done with SPSS 21.0 for Windows (IBM Corp., Armunk, NY) with a significance level of 5% (p<0.05).

Table 2. Demographic variables—comparison of proportions at baseline and four years afterwards.

Numbers are percentages of total N for each sample unless otherwise stated.


Eighty-nine of 184 (48%) members of professionals responded at the intervention hospital at baseline (sample 1), and 84 of 226 (37%) responded 4 years later (sample 2). At the control hospitals, 133 of 221 (60%) professionals responded at baseline (sample 3), while 93 of 297 (31%) responded after 4 years (sample 4).

The total sample (N = 399) included 78.3% females, 28.3% nurses, 28.3% health/social workers, 22.2% psychologists, 12.6% administrative/others, and 8.6% medical doctors, while 8.6% had leadership responsibility. Of the participants 38.5% worked with in-patients, 44.3% with out-patients, 10.8% worked with both groups, and 6.4% did not work therapeutically with patients. The participants at baseline had worked between three months and 29 years in the unit with a mean of 6.7 years (SD 6.2) and a median of 5 years. The participants at follow-up had worked between two months and 31 years with a mean of 6.3 years (SD 6.1) and a median of 5.0.

Differences in demographics between baseline and follow-up within each of the two groups are described in Table 2. In the intervention group there were significantly more psychologists in the sample at follow-up compared to baseline (p = 0.040). In the control group there was a trend for more psychologists (p = 0.098) and more females (p = 0.064) in the follow-up sample, compared to baseline. These variables were added to the binary logistic regression models for the groups respectively and were thus controlled for when comparing the changes within and between the groups.

The proportions of answers in the intervention group and control group were not systematically different at baseline. (Proportions of answers at baseline given in Table 3, analysis of baseline comparison is not shown).

Table 3. Personnel’s knowledge, practice, and attitudes at baseline and four years afterwards—intervention and control group.

Numbers are percentages of total N for each sample unless otherwise stated.

Changes within the study groups

Table 3 shows the distribution of answers and tests of difference in proportions between baseline to follow-up after 4 years for the intervention group and control group. In question no. 10 “Are service users involved in the hiring decisions of the unit’s staff?” the proportion of respondents answering “yes” (2 participants) was too small to make any meaningful comparisons. The question was therefore omitted from the analyses. Table 4 (first two columns) shows the comparison of changes within the study groups for variables with p≤0.2.

Table 4. Comparison of changes within and between the groups.

The table only show variables with p≤0.2. ROR>1.0 favours intervention. AdjOR >1.0 favours increase since baseline.

Knowledge about user participation.

At the intervention hospital there was significant change at a 0.05 level in five out of eleven questions; No 1 “Does your unit have a complaints procedure for clients?” (AdjOR = 2.97, CI = 1.57–5.63, p = 0.001), no. 11 “Are users invited to participate in staff training meetings at your unit?” (AdjOR = 11.27, CI = 2.50–50.88, p = 0.002), no. 12 “Has your unit ever asked users to act as teachers at staff training events?” (AdjOR = 2.74, CI = 1.00–7.52, p = 0.05), no. 13 “Does your unit sponsor events/forums that educate consumers about their rights and entitlements?” (AdjOR = 3.7, CI = 1.97–6.96, p<0.001) and no. 21 “Does the unit have a users’ committee?” (AdjOR = 1.98, CI = 1.05–3.73, p = 0.036).

At the control hospitals there was significant change at a 0.05 level in three out of eleven questions; No 8a “Does your unit solicit input from users for the planning of mental health services?” (AdjOR = 0.54, CI = 0.30–0.96, p = 0.036), no 12 “Has your unit ever asked users to act as teachers at staff training events?” (AdjOR 1.95, CI = 1.00–3.78, p = 0.049) and no 13. “Does your unit sponsor events/forums that educate consumers about their rights and entitlements?” (AdjOR = 0.57, CI = 0.33–0.98, p = 0.043). There were no trends for change (p<0.1) in either of the groups.

Practice of user participation.

There were significant changes on the 0.05 level on two out of seven questions in the intervention group; No. 4 “Are clients informed about the facts about confidentiality and privacy regarding information contained in their records?” (AdjOR = 3.42, CI = 1.28–9.13, p = 0.014) and no. 25 “Do you have enough time to ensure users’ participation?” (AdjOR = 2.36, CI = 1.19–4.70, p = 0.014). There was a trend for change on question no 3 “Are clients told they have a right to see and/or correct their records? (AdjOR = 1.73, CI = 0.91–3.27, p = 0.091).

In the control group there were significant changes in three out of five questions; No 25 (AdjOR = 2.45, CI = 1.21–4.93, p = 0.012), no. 26 “In your opinion, are next of kin in general sufficiently involved?” (AdjOR = 3.27, CI = 1.80–5.96, p<0.001) and no. 27 “How would you describe the collaboration with next of kin in general?” (AdjOR = 2.07, CI = 1.17–3.68, p = 0.013). There were no questions with a trend for change in the control group.

Attitudes towards user participation.

In the intervention hospitals there were significant changes from baseline to follow-up regarding the professionals’ attitudes towards user participation at one of the 9 questions; No 23 “How would you describe the unit’s general attitude towards user participation?” (AdjOR = 2.44, CI = 1.30–4.58, p = 0.006). There was a trend for change within the intervention group in one question; No 24 “In your opinion, do users understand the information you give about their illnesses and treatment opportunities?” (AdjOR = 0.13, CI = 0.02–1.16, p = 0.067).

There were no significant changes or any trends for change in the control group.

Comparison between the groups

Comparing the changes in the intervention group with the changes in the control group (Table 4, third column with Ratio Odds Ratio) revealed statistically significant differences at a 0.05 level in four of the questions.

Knowledge about user participation.

For knowledge about user participation questions there was a significant difference on question no. 13 “Does your unit sponsor events/forums that educate consumers about their rights and entitlements?” (ROR = 6.5, CI = 2.8–14.9, p<0.001). In the intervention hospital significantly more professionals answered “yes” after four years.

There was a trend for difference in two questions; No. 11 “Are users invited to participate in staff training meetings at your unit?” (ROR = 5.2, CI = 0.9–30.6, p = 0.066) and no. 21 “Does the unit have a users’ committee?” (ROR = 2.3, CI = 1.0–5.2, p = 0.057). In both questions more professionals in the intervention hospitals answered “yes” after four years.

Practice of user participation.

For the practice with user participation questions we found significant differences in two questions; No. 4 “Are clients informed about the facts about confidentiality and privacy regarding information contained in their records?” (ROR = 4.8, CI = 1.4–15.9, p = 0.012) and no. 26 “In your opinion, are next of kin in general sufficiently involved?” (ROR = 0.3, CI = 0.1–0.8, p = 0.016). For the question no. 4 significantly more of the professionals at the intervention hospital after four years answered that clients usually or always was informed. For question no. 26 significantly more professionals at the control hospitals answered yes after four years.

Attitudes towards user participation.

For the questions regarding attitudes towards user participation we found a significant difference on one question; no. 20 “In your opinion, how would the service develop if users were involved in planning and/or carrying out the mental health service?” for the answer “Downgrading of the services and delivery” (ROR = 24.2, CI = 1.1–546.4, p = 0.045). Significantly more professionals at the intervention hospital answered yes after four years.

There was a trend for difference on question no. 18 “In your opinion, what are the most important reasons when users of mental health care don’t want to be involved?” for the answer “Lacking in ability or knowledge” (ROR = 2.8, CI = 0.8–9.2, p = 0.093). More professionals at the intervention hospital gave this as a reason after four years.


Professionals’ at the intervention hospital reported significantly more that the hospital sponsored events that educate consumers and that patients were informed about confidentiality, while at the control hospital a higher proportion of the professional reported that next of kin in were sufficiently involved. A higher proportion of professionals in the intervention hospital reported that involving users in planning and/or carrying out the mental health service would downgrade of the services and delivery. Overall there were more changes after four years (this study) compared to after 16 months [40].

Strengths and limitations

This study is one of very few investigating the long term effect on professionals from user involvement. It is also one of very few controlled studies investigating the effect on professionals from an extensive and comprehensive development plan intended to enhance users’ participation in a health care organisation. These are major strengths of the study.

One very important limitation was that the samples were not matched as we wanted to ensure the participants’ anonymity. This means that the four samples could be very different and this might have influenced the findings. The results is important for these hospitals but cannot be generalized to professionals in other hospital trust or other parts of the country.

The development plan was intended to enhance user involvement across the whole organisation. We therefore included all employees in this study, regardless of whether they worked directly with patients or not. This might constitute an important limitation, since we might expect different experiences with user involvement when working in close contact with patients than when working in administrational settings. The total sample was large and the sample was representative for the gender distribution among those invited to participate. In addition, the distribution of occupational groups was similar to professionals in mental health hospitals in Central Norway [44].

As discussed in the previous paper from this study [40], the questionnaire used in this study has not been adequately tested or validated. The questionnaire has previously been used in cross-sectional studies [42,4548] and the sensitivity of the questionnaire to measure changes over time is not known. Neither have we tested or validated the grouping of the single questions into the three themes; knowledge, practice, and attitudes. To ease the presentation of the results and the comparison of the results from this study with the previously published study after 16 months, we chose to keep the grouping. The low response rate (43%) is a limitation, but was similar to other studies on professionals in mental health services [16,36,42,45,46,48,49]. A response rate of 37 and 31% four years after the intervention is disappointing, but not surprising since the time span is long. The lower response rate at follow-up can also be attributed to not sending a reminder. It is interesting that the response rate at baseline was higher in the control hospitals (60%) than in the intervention hospital (48%). It would be reasonable to assume that the hospital, where a strategy for user development was implemented, also had professionals who were interested in answering questionnaires related to this theme.

Time might be important to detect change

Compared to the results 16 months after the implementation [40], the results from the present study showed more effect from the development plan. Sixteen months after implementation there were no significant differences between the intervention hospital and the control hospitals and thus no significant effects from the development plan.

The results in the present study thus support the argument that organisational changes take time and confirm our assumption in the previous paper; that 16 months might not be enough time for the changes to result in any measurable effects. Although some authors have argued that organisational changes not necessarily need a lot of time to influence daily practice [50], others have argued that changing health service organisations might take considerable amounts of time before any changes have settled [18]. Several authors have stated that outcomes from user participation initiatives are dependent on how the implementation is conducted [51] and who initiated the initiative [28]. In a study exploring the implementation of user participation in different health care settings, Armstrong and colleagues outlined the following tips for successful involvement; a clear rationale for patient involvement, using the right model to achieve the desired outcomes, clarity regarding roles and responsibilities for patients, and ensuring that involvement is meaningful [52]. In our qualitative study exploring the professionals’ and user representatives’ experiences with the implementation of the development in the intervention hospital [41] we found that the professionals experiences of the implementation process depended on their position in the organisation during the process. Some described the implementation as a success and that the organization had embraced the concept of user participation. Most of those who described this had been central in the implementation processes. Those who had been less central in the implementation described on the other hand that the implementation had limited impact on the organisation’s work. This means that changes are dependent on various factors, such as responsibility and decision-making, and that the criteria for success regarding user participation often are unclear.

Multiple tests increase the risk of bias. Since the outcome measurements in this study include many questions and the analyses thus encompass many tests, the results should be interpreted with caution. We have focused on the overall direction of change in the discussion of the results.

Interpretation of findings

Regarding knowledge on user participation the results showed that more professionals knew that users were involved in organisational settings (in educational forums and in users’ committees) after four years. The within-group changes in the intervention group confirmed an improvement on several areas of knowledge four years after implementation. These findings indicate that the increased focus on user participation due to the development plan had influenced the knowledge level among the professionals at the intervention hospital. There was no similar improvement on knowledge in the control group.

Compared to the control hospitals, there was also significantly more increase in the intervention hospital of professionals reporting that they informed clients that the information in their treatment records were confidential (question no. 4). This difference was confirmed by a significant increase within the intervention group and a non-significant decrease within the control group. This indicates that the development plan had increased the professionals’ attention on informing the service users about confidentiality. Difference was however not found in a similar question; “Are clients told they have a right to see and/or correct their records?” (No. 3) An increased attention on informing patients about their rights should be reflected also in this question. It might be that informing the patients about confidentiality and thereby building trust and alliance is easier than informing the patients that they have the right to read and correct treatment records made by the professionals.

A significant difference between the two study groups was also found on the question “In your opinion, are next of kin in general sufficiently involved?” (no. 26). A significant increase in the proportion of professionals answering “yes” was found in the control group, resulting in significantly more increase in the control group than in the intervention group. It might be that the increase in the control group reflected that the control hospitals have improved in involving the patients’ next of kin. Looking at Table 3 we see that the proportion of professionals in the control hospital who reported adequate involvement had doubled during four years, from 25 to 53%.

Although there were no differences between the groups, on the question “Do you have enough time to ensure users’ participation?” (no. 25) there were a significant increase in the proportion of professionals who answered that they had enough time to ensure user involvement in both hospitals. This implies a stronger overall focus on user involvement in the mental health services and a stronger feeling of having enough time for this work. This is in line with the results from the qualitative study exploring the implementation process, where the professionals reported that the hospital in general had increased the focus on user involvement [41].

Although a very large proportion of the professionals reported that clients should be involved in the planning of their own treatment and in the evaluation of their presenting problems, it was a surprising finding that professionals in the intervention hospital more frequently reported that it would lead to a downgrading in the services if users took part in planning and conduct of the services. This is also contrary to previous research that has indicated that attitudes towards involvement of users can be improved after having experienced user involvement in practice [15,53] and that those positive towards user involvement have previous experience from user participation [28].


This non-randomised controlled study is the first to investigate the long term effect on mental health professionals four years after implementing an organisational development plan to enhance user participation. The plan led to some significant improvements in professionals’ knowledge and practice of user participation. As there was more effect after four years than after 16 months, this study indicates that it takes long time before the effect of complex interventions to enhance patient participation in organisations can be detected among the professionals.

Clinical and research implications

More long-term studies are needed to build further knowledge on the effect on professionals from implementing complex initiatives to enhance user participation in health care organisations. Robust instruments to measure the development of professionals’ knowledge practice and attitudes towards user involvement are needed. In clinical work it is important to allow for enough time and patience for complex interventions to make their mark on the professionals and their practice.

Supporting Information

S1 Dataset. Minimal dataset underlying the findings.



Author Contributions

Conceived and designed the experiments: MBR AS. Performed the experiments: MBR AS. Analyzed the data: MBR AS. Wrote the paper: MBR AS.


  1. 1. World Health Organisation (1978) Declaration of Alma-Ata.
  2. 2. European Commission (2001) White paper: European Governance.
  3. 3. World Health Organisation (2002) The World Health Report 2002. Reducing Risks, Promoting Healthy Life.
  4. 4. Norwegian directorate for health and social affairs (2005) [National strategy for quality improvement in welfare and health services]. IS-1162.
  5. 5. Tritter JQ (2009) Revolution or evolution: the challenges of conceptualizing patient and public involvement in a consumerist world. Health Expect 12: 275–287. doi: 10.1111/j.1369-7625.2009.00564.x. pmid:19754691
  6. 6. Coulter A (2011) Engaging patients in health care. Maidenhead, England: Open University Press.
  7. 7. Arnstein S (1969) A ladder of citizen participation. J Am Plann Assoc 35: 216–224.
  8. 8. Charles C, Demaio S (1993) Lay participation in health care decision making: a conceptual framework. J Health Polit Policy Law 18: 881–904. pmid:8120350
  9. 9. Thompson A (2007) The meaning of patient involvement and participation in health care consultations: A taxonomy. Soc Sci Med 64: 1297–1310. pmid:17174016
  10. 10. Patients' Rights Act (2012) 1999-07-02 nr. 63.
  11. 11. Health Authorities and Health Trusts Act (2001) 2001-06-15 nr. 93.
  12. 12. Wiig S, Storm M, Aase K, Gjestsen MT, Solheim M, Harthug S et al. (2013) Investigating the use of patient involvement and patient experience in quality improvement in Norway: rhetoric or reality? BMC Health Serv Res 13: 206. 1472-6963-13-206 [pii]; doi: 10.1186/1472-6963-13-206. pmid:23742265
  13. 13. Coulter A, Ellins J (2006) Patient-focused interventions. A review of the evidence.
  14. 14. Storm M, Knudsen K, Davidson L, Hausken K, Johannessen J (2011) Service user involvement in practice: The evaluation of an intervention program for service providers and inpatients in Norwegian Community Mental Health Centers. Psychosis 3: 29–40.
  15. 15. Nathan S, Johnston L, Braithwaite J (2011) The role of community representatives on health service committees: staff expectations vs. reality. Health Expect 14: 272–284. doi: 10.1111/j.1369-7625.2010.00628.x. pmid:21029280
  16. 16. Omeni E, Barnes M, MacDonald D, Crawford M, Rose D (2014) Service user involvement: impact and participation: a survey of service user and staff perspectives. BMC Health Serv Res 14: 491. s12913-014-0491-7 [pii]; doi: 10.1186/s12913-014-0491-7. pmid:25344210
  17. 17. Tempfer CB, Nowak P (2011) Consumer participation and organizational development in health care: a systematic review. Wiener Klinische Wochenschrift 123: 408–414. doi: 10.1007/s00508-011-0008-x. pmid:21739200
  18. 18. Arnold J, Silvester J, Patterson F, Robertson I, Cooper C, and Burnes B (2005) Work psychology. Understanding human behaviour in the workplace. Harlow, England: Pearson Education Limited.
  19. 19. Walker HJ, Armenakis AA, Bernerth JB (2007) Factors influencing organizational change efforts—An integrative investigation of change content, context, process and individual differences. Journal of Organizational Change Management 20: 761–773.
  20. 20. Grol R, Wensing M (2004) What drives change? Barriers to and incentives for achieving evidence-based practice. Medical Journal of Australia 180: S57–S60. pmid:15012583
  21. 21. Dixon-Woods M, McNicol S, Martin G (2012) Ten challenges in improving quality in healthcare: lessons from the Health Foundation's programme evaluations and relevant literature. BMJ Qual Saf 21: 876–884. bmjqs-2011-000760 [pii]; doi: 10.1136/bmjqs-2011-000760. pmid:22543475
  22. 22. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M (2008) Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337: a1655. doi: 10.1136/bmj.a1655. pmid:18824488
  23. 23. May CR, Mair FS, Dowrick CF, Finch TL (2007) Process evaluation for complex interventions in primary care: understanding trials using the normalization process model. BMC Fam Pract 8: 42. 1471-2296-8-42 [pii]; doi: 10.1186/1471-2296-8-42. pmid:17650326
  24. 24. May C, Finch T (2009) Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory. Sociology-the Journal of the British Sociological Association 43: 535–554.
  25. 25. Forster DA, Newton M, McLachlan HL, Willis K (2011) Exploring implementation and sustainability of models of care: can theory help? BMC Public Health 11 Suppl 5: S8. 1471-2458-11-S5-S8 [pii]; doi: 10.1186/1471-2458-11-S5-S8. pmid:22168585
  26. 26. Bamford C, Poole M, Brittain K, Chew-Graham C, Fox C, Iliffe S et al. (2014) Understanding the challenges to implementing case management for people with dementia in primary care in England: a qualitative study using Normalization Process Theory. BMC Health Serv Res 14: 549. s12913-014-0549-6 [pii]; doi: 10.1186/s12913-014-0549-6. pmid:25409598
  27. 27. Phillips RL, Short A, Kenning A, Dugdale P, Nugus P, McGowan R et al.(2014) Achieving patient-centred care: the potential and challenge of the patient-as-professional role. Health Expect. doi: 10.1111/hex.12234.
  28. 28. Fudge N, Wolfe CD, McKevitt C (2008) Assessing the promise of user involvement in health service development: ethnographic study. British Medical Journal 336: 313–317. doi: 10.1136/bmj.39456.552257.BE. pmid:18230646
  29. 29. Crawford M, Aldridge T, Bhui K, Rutter D, Manley C, Weaver T et al. (2003) User involvement in the planning and delivery of mental health services: a cross-sectional survey of service users and providers. Acta Psychiatr Scand 107: 410–414. pmid:12752016
  30. 30. Rise MB, Solbjor M, Lara MC, Westerlund H, Grimstad H, Steinsbekk A (2011) Same description, different values. How service users and providers define patient and public involvement in health care. Health Expectations 2011 Aug 12. doi: 10.1111/j.1369-7625.2011.00713.x.
  31. 31. Storm M, Davidson L (2010) Inpatients' and providers' experiences with user involvement in inpatient care. Psychiatr Q 81: 111–125. doi: 10.1007/s11126-009-9122-6. pmid:20069363
  32. 32. Gagliardi AR, Lemieux-Charles L, Brown AD, Sullivan T, Goel V (2008) Barriers to patient involvement in health service planning and evaluation: an exploratory study. Patient Educ Couns 70: 234–241. pmid:18023129
  33. 33. Daykin N, Sanidas M, Barley V, Evans S, McNeill J, Palmer N et al. (2002) Developing consensus and interprofessional working in cancer services: the case of user involvement. Journal of Interprofessional Care 16: 405–406. pmid:12487848
  34. 34. Nathan S, Harris E, Kemp L, Harris-Roxas B (2006) Health service staff attitudes to community representatives on committees. J Health Organ Manag 20: 551–559. doi: 10.1108/14777260610702299. pmid:17168106
  35. 35. Rise MB, Steinsbekk A (2013) Does implementing a development plan for user participation in a mental hospital change patients' experience? A non-randomized controlled study. Health Expect. doi: 10.1111/hex.12105.
  36. 36. Rose D (2003) Partnership, co-ordination of care and the place of user involvement. Journal of Mental Health 12: 59–70.
  37. 37. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R (2014) Involving patients in setting priorities for healthcare improvement: a cluster randomized trial. Implement Sci 9: 24. 1748-5908-9-24 [pii]; doi: 10.1186/1748-5908-9-24. pmid:24555508
  38. 38. Joosten EA, DeFuentes-Merillas L, de Weert GH, Sensky T, van der Staak CP, De Jong CA (2008) Systematic review of the effects of shared decision-making on patient satisfaction, treatment adherence and health status. Psychotherapy and Psychosomatics 77: 219–226. 000126073 [pii]; doi: 10.1159/000126073. pmid:18418028
  39. 39. Nilsen E, Myrhaug H, Johansen M, Oliver S, Oxman A (2010) Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material (Review). Cochrane Database Syst Rev.
  40. 40. Rise MB, Grimstad H, Solbjor M, Steinsbekk A (2011) Effect of an institutional development plan for user participation on professionals' knowledge, practice, and attitudes. A controlled study. BMC Health Services Research 11: 296. 1472-6963-11-296 [pii]; doi: 10.1186/1472-6963-11-296. pmid:22047466
  41. 41. Rise MB, Solbjor M, Steinsbekk A (2013) Experiences from the implementation of a comprehensive development plan for user involvement in a mental health hospital: A qualitative case study. Int J Soc Psychiatry 60: 387–395. 0020764013491739 [pii]; doi: 10.1177/0020764013491739. pmid:23812411
  42. 42. Kent H, Read J (1998) Measuring Consumer Participation in Mental Health Services: Are Attitudes Related to Professional Orientation? Int J Soc Psychiatry 44: 295–310. pmid:10459513
  43. 43. Altman D, Bland J (2003) Interaction revisited: the difference between two estimates. BMJ 326: 219. pmid:12543843
  44. 44. Muusmann research and consulting (2008) [Mapping of community mental health hospitals i Norway 2008].
  45. 45. Grant J (2007) The Participation of Mental Health Service Users in Ontario, Canada: A Canadian Application of the Consumer Participation Questionnaire. International Journal of Social Psyciatry 53: 148–158.
  46. 46. Soffe J, Read J, Frude N (2004) A survey of clinical psychologists' views regarding service user involvement in mental health services. Journal of Mental Health 13: 583–592.
  47. 47. Richter J, Halliday S, Gromer LI, Dybdahl R (2009) User and Carer Involvement in Child and Adolescent Mental Health Services: A Norwegian Staff Perspective. Administration and Policy in Mental Health and Mental Health Services Research 36: 265–277. doi: 10.1007/s10488-009-0219-x. pmid:19350380
  48. 48. Sibitz I, Swoboda H, Schrank B, Priebe S, Amering M (2008) [Mental health service user involvement in therapeutic and service delivery decisions: professional service staff appear optimistic.]. Psychiatr Prax 35: 128–134. pmid:17987541
  49. 49. McCann TV, Clark E, Baird J, Lu S (2008) Mental health clinicians' attitudes about consumer and consumer consultant participation in Australia: A cross-sectional survey design. Nurs Health Sci 10: 78–84. doi: 10.1111/j.1442-2018.2008.00388.x. pmid:18466379
  50. 50. Murray E, Richardson P (2003) Fast forward: A new framework for rapid organizational change. Ivey Business Journal March/April.
  51. 51. Contandriopoulos D (2004) A sociological perspective on public participation in health care. Social Science & Medicine 58: 321–330.
  52. 52. Armstrong N, Herbert G, Aveling EL, Dixon-Woods M, Martin G (2013) Optimizing patient involvement in quality improvement. Health Expect 16: e36–e47. doi: 10.1111/hex.12039. pmid:23374430
  53. 53. Crawford M, Rutter D, Manley C, Weaver T, Bhui K, Fulop N et al. (2002) Systematic review of involving patients in the planning and development of health care. BMJ 325: 1263. pmid:12458240