Interviews–step 4 –individual and focus groups including observing thinking aloud.

Two semi-structured interview guides were defined, one for the individual interviews, and one for the group interviews. The individual interviews with healthcare professionals were inspired by Hak et al.’s depiction of a think-aloud approach [31]. The specific interview process looked like this (S1–S3 Files):

1. Informants read the guide to SDM:KOMPASS, (S1 Fig) and afterwards the SDM:KOMPASS (Fig 1) and thought aloud as he/she did so.
2. Informants were provided a short questionnaire to fill in.
3. Researcher asked informants about thoughts and reactions while the informant was reviewing the SDM:KOMPASS.
4. Researcher asked additional questions regarding face validity and content validity mentioned above: 1) The tool’s readability and clarity, 2) the construct’s content, and 3) the tool’s perceived usability in practice.

In addition to listening to the thinking aloud, the observation focused on how the tool and guide were handled, read, noted on and how much time was used to go through. For the majority, the individual interview took place at the informants’ office, and the individual interviews with patients took place at virtual meetings. For the focus group interview, the informants had prepared by reviewing the tool and guide and responding to the survey beforehand. One group of informants consisted of implementation consultants with the task of implementing SDM in the clinical departments at the specific regional hospital, where they were employed. The other group of implementation consultants consisted of employees from the Center for Shared Decision Making, with the task of assisting and supporting the regional implementation consultants. The focus group interviews were conducted by two persons from the development group. One of them acted as the interviewer while the other simultaneously took real-time notes to complement the audio recording. By inviting both SDM experts, healthcare professionals, healthcare leaders and patients we sought to reach saturation. During the interview process, we reached saturation, meaning that different perspectives were presented and nothing new came through. The interviews were audio recorded and lasted for 45 to 60 minutes each. Furthermore, two individuals who took part in step 2 answered the survey only, and were not interviewed. The development group decided to include patient representatives in the field testing. However, the patient representatives were only interviewed concerning the sub-categories including patients’ or relatives’ activities or outcomes and were not included in the survey, as it focused on the hospital workflow and implementation. A structured interview guide was formulated specifically for the patient interviews, and structured notes were taken during the interview (S3 File).

Survey–step 4.

In line with Hak et al’s description of the think-aloud approach [31] all informants completed a survey containing ten items. These items encompassed aspects such as the perceived assistance of the tool in terms of planning, goal-setting, and evaluation of the process, the significance of the four themes, overall perceived usability, elements for potential removal, and open-ended sections for informants to provide input in their own words.

Data processing and analysis–step 4.

The audio files were carefully listened through multiple times by the interviewer. Notes were taken and for each interview, selected citations were transcribed verbatim (author DGH). Additionally, the individual interviews were transcribed to secure that no details were missed during the first round of listening. Subsequently, they were subjected to secondary analysis (author LL). Based on the interview guides (S1–S3 Files), we conducted a content analysis (authors DGH, LL) focusing on informants’ experiences and meaning regarding: 1) The tool’s readability and clarity, 2) the construct’s domains and content, 3) the tool’s perceived usability in practice. Statistical analysis of the survey data was restricted to descriptive statistical methods.

Interviews–individual and focus group.

When presenting the results derived from analysis of the interviews, including observations during the interviews and survey data (step 4, Fig 2), the three focus areas outlined in our content analysis framework (Fig 3) are utilized to affirm the positive testing outcomes of the tool concerning face validity and content validity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Content analysis framework–step 4.

https://doi.org/10.1371/journal.pone.0312990.g003

The tool’s readability and clarity.

During the development and field testing (step 2, Fig 2) and the prototype revisions (steps 3 and 5) we spent much effort in tailoring readability to the constructs in SDM vocabulary and making these operational into the clinical vocabulary used in their everyday practice. When assessing the readability and clarity of the tool, we focused on the layout and design and its visual appearance. Readability is understood as the perceived ease with which a reader can understand a written text. The readability of a text depends on its content (complexity of syntax and vocabulary) and its presentation (e.g. font size, line height, character spacing, and line length and colors), which are the typographic aspects that affect readability. The practical suggestions regarding improvement of the tool expressed during the think-aloud interviews, have been adjusted in the final version of the tool of the guide SDM:KOMPASS (S1 Fig) and the tool SDM:KOMPASS (Fig 1). Even though the informants had comments of adjustments they all found the basic elements in the layout of the tool understandable. Below we have chosen some of the most illustrative testimonials from health care professionals, managers, Chief Executive Officers (CEOs), Chief Medical Officers (CMOs) and SDM implementation consultants which show how well they experienced the tool’s readability and clarity.

A chief nurse expressed that it is “Good to have the colors that separate the four themes”(#14). And a nurse specialist stated: “You can feel that it’s a well-thought-out material. It’s nice to hold in your hands" (#13). A quality coordinator stressed the importance of recognition from other improvement models and approaches. She stated:

“Recognizable format, similar to the Virginia Mason concept that we use in the quality department, in improvement management. It is easy to use, I can quickly read up on how to jump into it. The tool is well set up and it is nice that it is in line with the setup that we know with the nice colors and I am not overwhelmed by the form, it’s nice to have something that takes you by the hand” (#16)

Informants highlights that the layout, choice of colors and design were easily accessible. As implementation consultants stated, the tool is “…both highly visual and operational. And it clearly illustrates the many aspects that are important for successful implementation, it provides an overview” (#1–5). A chief physician stresses that the tool is “…delicious and well thought out. Makes it easy to say yes. Energy has been spent on ’selling it’, and the colors help it to be perceived and read as steps” (#15). The tool’s readability and clarity is summed up in this statement from a nurse specialist where she underlined her experience of usability due to the readability.

“It makes it visible, so instead of it being in my head what we are going to do, it will be more visible here [in the form–the schematic chart]. It will also be a chart I could hang up. That I could look at and maybe keep checking off. I like that way, to check off. Then we are this far. Then I think it’s nice to have those headings. We are still operationalizing. The form also clarifies our thoughts on how to structure how to get new clinicians on board”(#13).

The tool’s domains and content.

As previously noted, the development group closely adhered to the Southern Danish Improvement model (39), and a crucial aspect of this tool is the operationalizability of the content related to the development of an SDM approach. Concerning content validity, it’s imperative that the tool’s terminology remains relevant to both end users and SDM experts. The development group adjusted the wording after step 4 in the development process based on comments from the informants. The quotations below illustrate the overall perceived satisfaction with the domains and content.

A CMO from a hospital in the Region of Central Jutland described the tool’s content (the CMO called it “concrete suggestions”) having great value while the suggestions are specific:”Extremely concrete and that’s good for me. The tool tackles relatively complex issues with some very concrete suggestions. That’s taking the implementation work really seriously” (#19).

An implementation consultant from the Region of Southern Denmark expressed that they find the right content choices in vocabulary use in the tool: “It seems like the right things have been thrown into the schedule. Even though it’s a big mouthful, it’s still manageable, albeit perhaps especially as a manager"(#7–9). Likewise the implementation consultants from the Center for Shared Decision Making applauded the content and wording used as they underline the importance of a recognizable structure.

“The tool is in line with ’competency models’ already known in the clinic. Recognizable structure, step-by-step progression from novice to expert. It is okay to be at level 1. [The terms and content] supports working with one area at a time. Really good tool, readable level by level, makes me wise about the process, but time consuming. The tool includes all the many things we have learned are necessary [during an implementation process]. Grouping into themes is visual and operational, and makes it clear. But it is more a tool for managers than for the clinician ’on the floor’, and managers may feel it is useful because it appeals to ’winging it’. And the [terms and content] will be read differently by people at different levels”(#1–5).

Likewise a chief nurse also reflects on the aspects of levels of maturity and how the tool’s terms can help develop a common vocabulary among professionals.

“It could be a bit of fun for the team members to assess individually and then have a team discussion. Make a joint decision about where we are and how we want to work with it. Create a common language, it will make good sense and for us to get to the next level, we can then agree on what is needed and also on our intermediate goals”(#14).

As we explored the informants experience of the tool’s terms and content it became clear that they especially related this to the role of the manager, and leaders of the implementation process. A nurse specialist expresses how the tool can support her work with implementing SDM while her leaders are novices into SDM.

“The difficult thing is that the two managers [in the department] don’t know much about it [SDM]. And I can go back and use this schedule [SDM:KOMPASS], it will help me go back and say this is what we need [refer to the content of the themes]. It also makes it clear to the clinicians what their role is. I would definitely recommend it to others and use it myself, and also take it to clinical conferences, preferably with dates on it”(#13)

The informants described how the division of the four themes is meaningful in their daily life at the clinic to ensure they get started with a meaningful implementation. In the beneath quotation, a chief nurse contributed to content validity:

“It’s really operational. It [SDM:KOMPASS], can work as a process plan I can have in my whiteboard meetings. It’s great that it’s divided into the four groups because then it becomes clear at the meetings. We often work for a long time in management before the clinicians are involved. This will help us get the clinicians involved sooner. It is very clear. A really nice process plan. I will use it. It will make sense to take to my colleagues, this is what we work with, this is the way we think about shared decision-making, and there is actually also a culture and some management and competence development”(#13).

In line with the above statement, a quality coordinator talked about the content value of the tool as it makes the process visible to managers: “The tool makes it visible to the management how many facets are important in implementing SDM”(#16). Also a CMO appreciated the four themes chosen to operationalize the implementation of SDM in hospitals and especially he underlined the value of the tool for managers: "The four themes are very natural and right steps" (#19). Especially in relation to the leadership theme, he says:

“It’s very coherent. The management team has backed it very strongly. They have asked about the process. It fits in very well with the leadership work our departmental managers have requested. Who has it [who is responsible for what?], and what framework we have to work under? Will there be something regional, or is it something we have to invent locally? [the managers in his hospital ask]“(#19).

The informants acknowledged the tool’s terminology, choice of phrasing, and levels of maturity, which they found consistent and familiar in relation to the structure they encounter in their daily utilization of competency models already known in the clinical setting. As observed above, the informants perceived the levels of maturity as relevant.

The tool’s perceived usability in practice.

The informants expressed their perception of SDM:KOMPASS as a valuable tool for the purpose of implementing SDM in their day-to-day clinic operations. Specifically, they elaborated on how the tool can assist them, as managers, in orchestrating the deployment and active integration of SDM. Furthermore, they noted its potential usability in keeping staff informed about the necessary steps to foster the advancement of SDM realization. As articulated by a chief nurse, the levels of maturity serve as guiding markers for the process of SDM implementation. As this chief nurse stated the levels of maturity guide the process:

“It’s a tool that allows you to work it into the department. It makes good sense. It is a good picture of how I can see where we are and where we need to focus. And I can use it as a manager. We are a small group and we can use it to keep an overview, to see where we are. I can just pull it out. Refer to what we said last time and then take it from there. It would be good to do [a process where we use SDM:KOMPASS] for the department and for the team, and take them with us [by looking at the SDM:KOMPASS] so we can see we are moving forward. Maybe also for the outpatient clinic where they have been working with it for several years”(#14)

Regarding the tool’s usability, it was significant for the informants to highlight that the tool can bolster the entire management team. They stressed that the different leaders should be seen as a unit that works together operating in accordance with uniform metrics and shared objectives. As pointed out by the following chief nurse:

“It’s really good for ensuring that you get the same thing done, it’s good to have it so clear. It’s good for me to understand the process, so I’m more confident and can give employees direction. SDM:KOMPASS will help me get shared decision-making out to my functional managers so we can see how far we’ve come. Such an overview [from the tool] will be really good to have. In the long term, it could also be something to have in the management team as a supportive tool. Perhaps you could typically ask on a weekly basis how much are documented in the patient charts? And how many have been trained and then also include that in the whiteboard meeting”(#12)

A quality coordinator expressed how she sees it as a good process to start using the tool in the implementation team and then later on it should become part of the entire department’s way of working with it. The tool will support both objectives and ensure that it becomes visible what you are working with.

“I would start in the management/implementation team. But it has to be a staff tool—where are we—where are we going? And then used in the same way as the improvement boards. You also need to look at the low-hanging fruit so it doesn’t all feel like really hard work. It could work as a motivational tool in the process showing us that we have reached/have managed, etc. It must be experienced as an opportunity to be realistic about one’s own department. It will be good to have at hand when we need to see how far we are. And I will be happy to have it myself. Both for shared decision-making and also in many other contexts. We can use it to talk about where we want to go as a goal. And it provides an opportunity to make visible the many things that we as leaders must need to see “the bigger picture” and be held accountable“(#16)

The above two quotes underline how the tool clarifies the department’s progress in the process aka the maturity aspect of the tool in the formative evaluative process supporting the leadership team in succeeding in their responsibility of the process. A chief physician also stated how the leaders can use this tool as a joint venture and setting the scene for the next steps to come:

“I want to use it together with my new manager when we have to introduce shared decision- making in the department. If we are going to do this [Shared Decision-Making], we need resources and that the development process will take half or a whole year and this is well signaled by the chart. It also provides an opportunity for reflection. I reflect on where we are in our clinic—both the team and the department as a whole. It gives us the opportunity to motivate clinicians, we can identify focus areas, e.g. supervision and patient preparation, where we are ’behind’. It is good for highlighting areas of action. My immediate assessment is that it is a great tool. I would like to use that and take it with me to my new position. Really good and thorough. Clear and concise. Makes it easy to say yes and to get started and allocate resources”(#15)

The Implementation consultants from the Region of Southern Denmark agreed that the tool has practical benefits during a focus group interview. They mention the following points as listed below:

SDM:KOMPASS is a concrete working tool and can be used as a management tool. It can be used for internal feedback and for clarifying different perspectives in e.g. a department. And it can be used to identify, formulate and set implementation goals at different levels and stages of development. The tool can be used to guide action, including in the sustainability process. Furthermore it can be used during specific discussions with or in departments to find out where we are now and where we would like to go at the moment. It is experienced especially as a management tool and as an auditing tool, especially because of the goal hierarchy (#6–9).

The implementation consultants further debated about the possibilities of uneven assessment in the same department. They underline the importance of the aspect of different assessments gives the opportunity to different voices and experiences when processing a formative evaluation. In line with the chief nurse (#14) above stating that the tool can foster a common vocabulary:

“It is only good to have different assessments of where you are in the process, because it can help to clarify the different perspectives and assessments. Differences in assessment are therefore not a problem, but can lead to some good discussions. It is therefore also important that not only one person is evaluating the implementation effort with this tool, but several people, i.e. the team itself/the people who are in the process”(#6–9).

This is in line with the implementation consultants from the Center for Shared Decision Making, who underline the tool’s role as a mediating the dialogue, as they explained:

“The tool is a good starting point for planning; agreeing on the starting point; disagreement or uneven assessments will be a good starting point for dialog. In particular, [the themes] ‘clinical practice’ and ‘culture’ will be read differently depending on position”(#1–5).

The implementation consultants from the Region of Southern Denmark additionally indicated that the tool holds considerable usability for managers, aligning with the previous observations within the construct’s content. However, they anticipated that for clinicians, it might initially appear somewhat overwhelming, as articulated by one of the informants:

"…"Uh, that’s quite extensive. But once you’ve spent a bit of time on it, it looks really nice, and the structure is well-designed and covers the necessary aspects‥(…)… The document [SDM:KOMPASS] may seem overwhelming in its entirety, so it is important to emphasize that it can be used in “chunks” and in relation to the current focus area"(#6–9).

Summarizing the qualitative data from step 4, it is evident that SDM:KOMPASS is regarded as a valuable tool, particularly for managers and those being in front of the implementation process.

Surveys.

As described in the methodology section, the informants who were interviewed completed a ten item survey (Table 2) prior to their qualitative interviews. Before responding to the survey, they were presented with SDM:KOMPASS and the guide. No discussions about the tool took place prior to survey completion. Using a Likert scale of 1 to 5 (where 1 is: ‘not at all’ and 5 is: ‘to a very high degree’), a majority of all respondents have indicated agreement to a ’high degree’ or ’very high degree’ for the initial nine questions related to the face validity of the SDM:KOMPASS tool. This underlines the strength of face validity and content validity, as evidenced in the survey results.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Survey results.

https://doi.org/10.1371/journal.pone.0312990.t002

Questionnaire item ten encompassed an open-text section, allowing informants to provide further insights into their assessments. These inputs were considered by the development group. Presented below are two quotes extracted from the open-ended responses, which encapsulate the informants’ reflections on the tool’s usability.

“At first glance, the compass [SDM:KOMPASS] seems a bit large and overwhelming. However, when you sit down with it, it works fine. But it might be concerning that some clinicians give up on it, due to its size. On the other hand, I don’t think any elements should be removed. For some departments it can definitely be a useful tool if it is consistently used if the department sees value in it, and if it’s prioritized. I might be worried that in a busy everyday setting in some departments, it could be challenging to manage. Perhaps it could be integrated as a preparation for meetings with the departments, where discussions can be based on areas that aren’t progressing and need to be worked towards. It is difficult to say how useful the tool is and whether it can be used in its current form before you have sat with it in a department”.

Another informant elaborates on this in the open-ended descriptions, emphasizing how the tool can help evaluate the maturity level of the department:

“The tool clarifies the importance of leadership responsibility. It creates an overview and offers both department and management insight into what needs to be done, how far we are and where we would like to end up. I think it gives a good overview of what needs to be addressed. It would be good to show the department where we are in the process”.

These responses, coupled with the overall positive survey feedback (Table 2), signal that our preparatory work was primed for real-world testing across hospital departments and units. In summary, the 20 questionnaire responses affirmed that SDM:KOMPASS effectively targets relevant themes and proved immediately beneficial for teams or departments implementing SDM. In essence, the face validity and content validity aligned with end users’ perspectives, considering these assessments are inherently subjective. The data analysis from step 4 informed minor adjustments for step 5.

Step 5 –Revision, second round.

During this step, an additional sub-category labeled "documentation” was introduced and a few wording adjustments were made. For instance, the sub-category "relatives" was incorporated into an existing sub-category. The arrangement of sub-categories within the "Management" theme was modified based on insights from interviews, with "Managerial responsibility" placed first to emphasize its significance. Following these refinements, SDM:KOMPASS underwent linguistic review, resulting in minor refinements. The final version of the guide (S1 Fig) and the tool SDM:KOMPASS (Fig 1) was now ready to be introduced to end users.

Step 6 –Final version test–user experience.

The commencement of the bottom-up approach for the introduction of the SDM:KOMPASS tool to the SDM implementation consultants culminated in a workshop were 15 consultants actively participated. During this workshop, each informant was tasked with delivering a presentation elucidating the extent and manner of their incorporation of the SDM:KOMPASS tool within the context of the SDM implementation process in their hospital. Additionally, the informants were required to expound upon the methodologies employed in this integration. Predominantly, it was observed that the tool had been assimilated into the educational activities directed towards leaders and health care professionals, as well as incorporated into various planning activities conducted throughout the implementation process. It is noteworthy that the modalities employed for this integration exhibited some diversity; however, the prevailing method consisted of presenting SDM:KOMPASS tool to pertinent stakeholders within the clinical departments, subsequently followed by a brief workshop for their practical engagement. Asked if anything had not worked well, one of the consultants claimed; “….it takes time to familiarize yourself with the tool. But that’s more a reflection of the complexity of the implementation than the format of the tool”. Another consultant indicated; “… It does not work just to hand out the tool at meetings or educational classes”. There were also given examples of what worked well. One consultant said:

”….It worked well to visualize in joint discussions; it raises awareness of challenges, and when several teams/departments look at them together, they also realize that they may be the same challenges–making it a mutual sparring tool”.

Another consultant told about a statement from fellow manager working with patient involvement. The manager said; “….this is the most concrete and useful management tool for the work with patient involvement I have ever seen”. All 15 implementation consultants indicated they would recommend using SDM:KOMPASS for future implementation work.