https://orcid.org/0000-0002-9002-6145
Figures
Abstract
Background
Family involvement in care can be seen as a prerequisite for high-quality family-centered care. It has been identified to improve both patient safety and the quality of care by reducing patient complications and hospital length of stay.
Objective
To develop and evaluate the content validity of a questionnaire measuring family involvement in inpatient care.
Methods
The study followed a systematic approach in building a rigorous questionnaire: identification of domain, item generation, and assessment of content validity. The content validity index was calculated based on ratings of item relevance by an expert group consisting of seven senior nurses. Subsequently, 19 online cognitive interviews using the Think-aloud method were conducted with family members of former patients who had undergone open-heart surgery.
Results
Five aspects of family involvement were identified, and the initial pool of items were selected from two preexisting questionnaires. The experts’ ratings resulted in item content validity of 0.71–1.00, and the scale content validity/averaging was 0.90, leading to rewording, exclusion, and addition of items. The pretesting of items through two rounds of cognitive interviews with family members resulted in the identification of three main problem areas: defining family involvement, misinterpretation of different terms, and underuse of the not relevant response option. The problems were adjusted in the final version of the questionnaire, which consists of 16 items with a four-point Likert scale and two open-ended items.
Citation: Drakenberg A, Prignitz Sluys K, Ericsson E, Sundqvist A-S (2023) The Family Involvement in Care Questionnaire—An instrument measuring family involvement in inpatient care. PLoS ONE 18(8): e0285562. https://doi.org/10.1371/journal.pone.0285562
Editor: Elisa Ambrosi, University of Verona, ITALY
Received: November 25, 2022; Accepted: April 26, 2023; Published: August 15, 2023
Copyright: © 2023 Drakenberg et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This study was approved by the Swedish Ethical Review Authority (Reg. no. 2020-05276, 2021-01982). Region Örebro County (regional board) is responsible for all processing of personal data used for research in health care in Örebro Region, Sweden. The data processing used in this study follows The Swedish Authority for Privacy Protection (‘IMY’), The EU General Data Protection Regulation (Regulation (EU) 2016/679) (‘GDPR’) and The Swedish Patient Data Act (SFS 2008:355). The raw data in the study contains interviews using audio recordings (only available in Swedish) and is by GDPR considered identifying personal data, and can therefore not be publicly shared. All data used in the manuscript has been de-identified. Contact information for data request e-mail address: dso@regionorebrolan.se, or postal address: Data Protection Officer, Region Örebro County, Box 1613, SE-701 16 Örebro, Sweden.
Funding: ASS was supported by grants from Region Örebro county https://www.regionorebrolan.se (no Grants number provided), and AD was financed by grants from the Swedish state under the agreement between the Swedish government and the county councils, the ALF-agreement https://www.regionorebrolan.se/globalassets/media/dokument/platina/riktlinjer/forordnandeprogram-for-alf-finansierad-forskning-vid--region-orebro-lan-i-samverkan-med-orebro-universitet.463036.pdf (Grants number OLL-960017).
Competing interests: The authors have declared that no competing interests exist.
Introduction
When a person needs hospital care, the persons family often has an important role in supporting the person’s emotional and physical well-being [1]. Involving family in care has several advantages, including improved patient safety [2] and quality of care [3, 4], and family members’ satisfaction with care [5]. Family involvement has the potential to reduce patient complications [6], hospital length of stay [7], and is a prerequisite for high-quality family-centered care [8]. Furthermore, it constitutes an important part of the framework for patient and family engagement in health and health care [9]. There are a variety of definitions of family involvement [1, 5, 10–12]. In previous research, patient involvement and family involvement have been closely linked [12] and, at times, used interchangeably [9, 13]. This project is grounded in the family-centered paradigm with a systemic approach to families [8]. In the family-centered paradigm, family involvement is not only a means of improving patient outcomes but also of supporting the whole family’s health [8, 14, 15]. Family involvement in care can be burdensome for the family, whereas support for the family is an important aspect of this concept [16]. “Family” is defined by Whall [17] as follows:
The family is a self-identified group of two or more individuals whose association is characterized by special terms, who may or may not be related by bloodlines or law but who function in such a way that they consider themselves to be a family (17, p. 241).
“Family involvement” is defined as taking part in the care of a family member and having one’s own needs of care met, whereas “taking part” refers to active involvement in care, decision-making and communication [12]. “Care” is specified in this study as the inpatient care provided during the patient’s hospitalization.
Elective open-heart surgery involves preoperative information and preparations [18] and postoperative intensive care and/or care in a step-down unit [19]. After the critical postoperative care period, the patient is referred to the ward prior to discharge, where rehabilitation at home begins [18]. The intensive care unit (ICU) stay may be prolonged if complications occur [20]. The need for surgery and postoperative intensive care puts a great deal of strain on both the patient and the family [21, 22].
When evaluating interventions promoting family involvement, previous research has focused on family involvement’s effects on patient outcomes [23]. In addition to evaluating patient outcomes, it is important to adequately measure if the intervention in fact has promoted family involvement from the family’s perspective. Therefore, valid measurements assessing family involvement in care should be used in research and clinical practice [23]. There are existing questionnaires measuring various aspects of the family experience of psychiatric care [24, 25], geriatric care [26–29], and intensive care [30]. In these context, the patient’s possibility to participate in their own care is reduced due to compulsory care, cognitive decline or critical illness and medical sedation. The family may therefore serve as a proxy for the patient. We believe that the family perspective on family involvement in these context differ from inpatient care settings where the patient is able to make decisions about their own care, treatment and can be active partners in their own care. To the best of our knowledge, there are no generic questionnaires measuring family involvement in the inpatient care setting. As described above, the standardized care pathway of open-heart surgical care results in that patients and their families are exposed to various care settings within the hospital. This care context enables, in our opinion, a suitable setting for testing the generic properties of a questionnaire applied in inpatient care. The purpose of the study was therefore to develop and evaluate the content validity of a generic questionnaire measuring family involvement in inpatient care.
Materials and methods
Design.
The design of this study was inspired by the guidelines for best practices for developing and validating measurement scales outlined by Boateng et al. [31], and reported according to the Consolidated criteria for reporting qualitative research (COREQ) checklist [32]. To create a rigorous scale, Boateng and colleagues [31] describe three phases: item development, scale development, and scale evaluation. These phases are further broken down into nine steps. The first three steps, concerning item and scale development, were used in this study:
- identification of domain and item generation,
- evaluation of content validity by an expert group rating relevance of items,
- pretesting the questions in cognitive Think-aloud interviews with the target population.
The scale development and testing process are displayed in Fig 1.
Step I. Identification of domain and item generation
Identification of domain.
The domain to be identified in this study was “family involvement” and the context “inpatient care”. The context was further specified as care related to open-heart surgery. The domain was identified in previous research using a broad scoping search strategy in the databases CINAHL and PubMed with the search terms family, involvement, participation, and engagement during the fall of 2019. The literature was deemed relevant if it concerned inpatient care and family involvement.
Item generation.
Item development started by reviewing relevant literature regarding questionnaires and consulting leading researchers in the field of family nursing in Sweden who have developed questionnaires that evaluate patient and/or family involvement in care. The inclusion criteria regarding the questionnaires were that they had been developed in an inpatient care context relevant to surgical care and validated in Swedish.
Step II. Content validity
Setting.
Nurses in the expert group in step II were recruited from a cardiothoracic and vascular surgery department situated in a university hospital in Sweden. Approximately 500 open-heart surgeries are performed annually at the department, which consists of a surgical site, a thoracic ICU, and a ward. Patients come from the university hospital’s own catchment area and other parts of Sweden.
Evaluation by experts.
The first version of the questionnaire was evaluated by seven senior nurses with extensive experience caring for patients and their families after open-heart surgery. This was done in December 2019 to determine item relevance in relation to family involvement in care. The nurses in the expert group were selected based on their years of experience as surgical nurses in the ward (i.e., a purposive sampling), and they were contacted by their work place e-mail. All nurses had the experience of being a family member to a person who had been admitted to inpatient care. The nurses holding a bachelor’s or master’s degree, as presented in Table 1, also had postgraduate education in surgical, vascular respectively operating room care.
The content validity index (CVI) was used to summarize the degree to which the expert group agreed about item content validity. The assessment considered both the item (I-CVI) and scale (S-CVI/Averaging) levels [33]. The experts individually rated the relevance of each item in writing on a four-point Likert scale: (1) “not relevant,” (2) “somewhat relevant,” (3) “relevant” and (4) “highly relevant.” They were also able to add comments explaining their responses. The I-CVI score was calculated by counting the number of experts giving a rating of “3” or “4” on the four-point relevance scale and thereafter dividing that number by the total number of experts. The S-CVI/Averaging score was calculated by adding the sum of all I-CVI scores and thereafter dividing by the total number of items. A value of 0.78 for the I-CVI and 0.90 for the S-CVI/Averaging scores are considered excellent content validity [33].
Step III. Pretesting questions
Cognitive interviews.
To assess the appropriateness of the questions and response options for the target population, the questionnaire was evaluated in two rounds of cognitive interviews with family members. The cognitive interview method Think-aloud was used to analyze respondents’ cognitive processes while responding to the questionnaire [34]. In this method, well-designed questionnaires are formed through observation, reasoning, and cognitive interviews in which problems with the items are noted, and thereafter, refinements of the items are made. The sample size in cognitive interview studies should be small when conducted in several rounds, with no more than ten participants recommended in each round [34]. Data for step III were collected from January to June 2021. Due to the COVID-19 pandemic, cognitive interviews were conducted online using the videoconferencing platform Zoom Enterprise Video Communications (www.zoom.us, San Diego, CA).
Setting.
The family members of former patients were recruited from the same setting as the nurses in the expert group, as described in Step II.
Recruitment and participants.
The inclusion criteria for participants in the cognitive interviews were that they must be aged 18 years or older, a family member of someone who had undergone open-heart surgery, and be able to read, understand, and communicate in Swedish. To avoid the limitation of the participants’ range of experiences, no requirements regarding level of participation or time of presence at the hospital were made.
Recruitment started with a random selection of patients from the 2019 register of open-heart surgery at the university hospital. These patients received a letter with basic information about the study and were asked if a person in their family would be interested in participating. Former patients responded by providing contact information for presumptive participants. These family members of former patients who had undergone open-heart surgery were contacted with a letter providing full information about the study, including the objectives of the study and requesting their informed consent. A week later, they received a phone call to ask whether they had any questions and whether they were interested in participating in the study. Letters were sent out in several rounds to ensure that the number of twelve participants was not exceeded in any round. In total, thirty-six former patients were asked about presumptive participants, twenty-six of whom provided information about their family member. Out of these twenty-six family members of former patients, nineteen were willing to participate, four declined, and three could not be reached. No reason was requested for not participating. However, one of the four declining family members spontaneously stated that the reason they were not able to participate was that they had no e-mail address that could be used to send a link to a digital meeting and that they were not interested in having one. Ten of the nineteen participants were interviewed in the first round, and nine were interviewed in the second round (see Table 1).
Data collection.
Think-aloud interviews were carried out online individually with the participants in their own homes or private offices. Each participant was interviewed at one occasion. Standardized instruction [35] and probes [34] were used. On two occasions, the participant had company at home, but this did not disrupt the interview. The purpose of the study was discussed and clarified before starting each interview. The interviews were conducted by the first author. Supervision and continuous feedback were provided by the second author, who had prior experience in using the Think-aloud method.
The instructions for the interview process were pilot tested during the first interview, and no problems were identified. Problems with the probing strategy were, however, identified after the analysis of the first round. Therefore, the instructions were modified prior to the second round. In the first round of cognitive interviews, no concurrent probes besides the prompt “think out loud” were intended to be used. However, concurrent probes were used on twenty-six occasions, as pauses might indicate technical problems. This led to a change in strategy for the second round. In the second round, it was decided to include five concurrent and four additional retrospective probes in the instructions as a strategy to standardize the concurrent probes. The instructions are presented in Fig 2.
All interviews were audio-recorded. The participants were able to choose whether they wanted their video on or not during the interview, and thirteen out of the nineteen chose to have their video on. The participants could always see the interviewer. The questions were displayed using screen sharing, one at a time. The participants read the question aloud, verbalized their thoughts about the question/answer alternatives, chose an answer, and then indicated that they were ready for the next question. After the Think-aloud interview, the participant was asked to comment on any problematic wording or missing items and to give feedback on the layout and when they would have wished to receive the questionnaire.
Data analysis.
The “inspect-and-repair” model for the analysis of cognitive interviews was used. This model focuses on improving survey questions and on reducing response error in four steps [34]. The four steps of analysis are exemplified in Fig 3.
The first interview was transcribed by an experienced transcriber, and thereafter, the whole research team listened to the first interview while simultaneously reading the transcript. It was agreed that transcribing full interviews did not contribute to the analysis process in regard to identifying problematic areas. Listening carefully and repeatedly was regarded as an adequate analysis method, as tone of voice, pauses, and emphasis all contributed to problem identification. The interviews were analyzed deductively using the Think-aloud protocol with codes inspired by French et al. [35]: (A) no significant problems were identified, (B) rereading an item, (C) questioning an item’s content, and (D) misunderstanding an item. Notes were made after every interview regarding audio quality, main issues, and important comments made by the participant.
All four researchers independently coded and analyzed all ten interviews in the first round. Each researcher transcribed problematic sequences verbatim and took thorough notes that were compared and discussed in the research group until consensus was reached concerning coding and significant problems.
In the second round, the first and second authors analyzed each of the nine interviews independently and thereafter compared their analyses. The other two authors analyzed two randomly selected interviews from each of the two rounds. All four researchers independently analyzed the same four interviews at this last stage of the analysis.
After each round of interviews, intercoder agreement was used to improve the quality of the coding of Think-aloud interviews [36]. In other words, two or more researchers were able to resolve through discussion any coding discrepancies there may have been for the same item and interview.
Ethical considerations
The study was conducted in accordance with the ethical guidelines laid out in the Helsinki Declaration [37]. According to Swedish law (SFS 2003:460) [38], ethical approval was not required for steps I and II since this part of the study did not involve patients and no sensitive data were elicited. Step III was approved by the Swedish Ethical Review Authority (Reg. no. 2020–05276, and 2021–01982). All participants gave their written informed consent.
Results
Step I. Identification of domain and item generation
Identification of domain. The definition of “family involvement” used in this study originates from a scoping review of patient and family involvement in adult critical and intensive care settings [12]. The review identified five aspects of family involvement on a continuum from passive to active: (1) family presence, including visits and presence at bedside, (2) family needs being met/family being supported, (3) communication and informational support between family and health care professionals, (4) family participation in decision-making regarding treatment plans and life support, and (5) family contribution to care, such as bathing, massaging, and giving moral support to the patient.
Item generation.
Two existing questionnaires relevant for the context of inpatient surgical care of the adult person and validated in Swedish were identified during the item generation phase in 2019 and then assessed for content and structure: the Patient Involvement Questionnaire (PIQ) [39] and the Swedish Family Satisfaction Intensive Care Questionnaire (SFS-ICQ) [40]. Permission to reuse items from these two questionnaires were obtained from the originators of the instruments (i.e., from Dr. Judy Arnetz for PIQ, and Mr. Johan Thermaenius and colleagues for SFS-ICQ). Items from these questionnaires were selected based on their correspondence to the domain definition of family involvement outlined above. The item selection phase and the authors’ judgment of item correspondence to the domains are illustrated in Table 2.
Questions selected from the Patient Involvement Questionnaire (PIQ) and Swedish Family Satisfaction Intensive Care Questionnaire (SFS-ICQ) and their correspondence to the domains used in the Family Involvement in Care Questionnaire (FICQ).
The PIQ measures patient involvement in patient care following myocardial infarction. The questionnaire consists of 54 items, including demographic questions, divided into six subscales [39]. The scales in the PIQ deemed relevant for this study were “Patient involvement”, “Information”, “Patient needs”, and “Treatment planning”. The scales “Illness experience” and “Activity” were considered to be too patient specific, whereas questions in regard to “Patient needs”, for example, could sometimes be changed to “Family needs”.
The SFS-ICQ is a measurement of family satisfaction with care and is used as a proxy for patient satisfaction with care in the intensive care setting [40]. It has been validated in general and cardiothoracic ICU care in Sweden. It consists of 24 scaled items divided into six areas. Since the validation study of the SFS-ICQ, an additional item regarding asking the family for the patients presumed wishes has been added and is used in clinical practice. The four areas in the SFS-ICQ relevant to this study were “Treatment/Care of the patient”, “Participation in care”, “Reception of family members”, and “Information”. The area “Environment” was deemed to be too ICU specific, and the area “Overall satisfaction” was related only to satisfaction with care and therefore did not correspond well with the definition of family involvement used in this study. Some items in the PIQ and SFS-ICQ were similar, mainly regarding information and how the person was received by health care providers. After obtaining the original authors’ consent, an initial pool of items was selected from the PIQ (16 items) and SFS-ICQ (8 items) and changed in form and wording to fit the context and target population.
One additional item was generated through discussion within the research group, asking for the rating of the importance of participation in nursing care. The added item followed the structure of the items in the PIQ and was intended to cover the domain “contribution to care”. The first version of the questionnaire consisted of 25 items. Each item was phrased, as in the SFS-ICQ, as a statement with response options on a four-point Likert scale: (1) “I fully agree”, (2) “I largely agree”, (3) “I somewhat agree”, and (4) “I fully disagree”. There was also a “Not relevant” option. One open-ended question was included, giving the respondent an opportunity to share additional views on what they considered important regarding family involvement. In addition, demographic data were collected regarding age, gender, and the family member’s relationship to the patient. All versions of the questionnaire tested in steps II and III are shown in Table 3.
After the evaluation by experts in step II, the authors decided that the aspect of “having needs met and being supported” was not represented among the initial items where the participant was asked to rate the importance of the domains. An item rating the importance of having one’s own needs met was therefore added. An additional open-ended question was also added at this stage, giving respondents an opportunity to elaborate on any specific answers they had given to the prior items. The second version of the questionnaire, tested in the cognitive interviews in step III, consisted of 23 scaled items and two open-ended questions. The questionnaire instructions indicated, “Please note that the questions only refer to the time when your family member was taken care of in the cardiothoracic surgical department”. Instructions for the first seven items indicated, “these questions concern aspects important for your sense of being involved in your family members’ care”. Instructions for the following 16 items indicated, “these questions concern how you were involved in your family members’ care”.
Step II. Content validity
Evaluation by experts.
The I-CVI varied between 0.71–1.00, and the value for S-CVI/Averaging was 0.90. The I-CVI of each individual item is shown in Table 3. One item was excluded due to its low I-CVI (I-CVI = 0.71). The free-text comments on relevance were discussed by the research group, resulting in the rewording of twelve items and the exclusion of two. Since comprehensibility should be evaluated by the target group, the experts’ comments regarding item understandability were discussed within the research group but did not lead to the exclusion of items at this point.
Step III. Pretesting questions
Cognitive interviews with Think-aloud: Round one.
When conducting the analysis of the interviews, three main problem areas with the questionnaire were identified as misinterpretation of the purpose of the questions related to important aspects of family involvement, misinterpretation of the term “nursing care,” and underuse of the “Not relevant” answer option. A problem regarding the probing strategy was also identified. Rereading an item was not identified as a problem. Therefore, the participants could potentially have had two problems with one item, i.e., misunderstanding an item and/or questioning its content, making the maximum number of problems in round one 460. No participant had more than one problem per item, giving a total number of 38 problems in round one.
Defining family involvement.
Regarding Items 1–7, the instructions asked the participants about what was important for them to be involved in their family members’ care in general terms. However, as these seven items was frequently misinterpreted or questioned, they were considered to be the main problem area after the first round. Problems were mainly misinterpretations. Participants described the circumstances during the specific care period instead of rating how important these aspects were for the participant’s sense of being involved.
Well … involved in care… Received clear information… Um, I hardly got that, any information. I’d probably say that I somewhat agree. (Participant 1, round one, answering Item 1)
Questioning of item content in round one concerned the relevance of participating in decision-making and the items being closely linked to one another.
I actually think those questions overlap a bit. Umm, but, umm well… Umm, I suppose it’s good. Of course, you feel involved when you are part of the discussion. Umm. I largely agree with that. (Participant 2, round one, answering Item 4)
The extent of the problems with Items 1–7, interpreted as being related to the questionnaire instructions in this first round, indicated that alterations were needed. This was confirmed by the answers to the final open-ended questions and in the retrospective probe of problematic items. These items were reworded, at the end of the version used in round two, and instructions for these items were clarified with the statement “the questions on this page concern what you as a family member regard as important for you to feel involved in the care of your family member”.
Misinterpretation of the term “nursing care”.
The item asking whether the family member had been given the opportunity to participate in nursing care to the extent he or she wished was questioned by one participant: “Didn’t you ask this before?” (Participant 3, round one, answering Item 23). The participant probably referred to the defining item (Item 6 in Table 3) “I am involved in the care of my family member when I get to participate in the nursing care of my family member”. Item 23 was also misinterpreted by two other participants, where one of them disregarded the aspect of “to the extent I wished”. The other case of the term participation in nursing care not being understood in the way it was intended to be was when the participant referred to participation in nursing care at home and not in the hospital.
Well, I must say, nursing care, that was when she got home so, I fully agree, because it was necessary for me to participate in nursing care because complications meant that she needed my help. I fully agree to that. (Participant 4, round one, answering Item 23)
Underuse of the “Not relevant” option.
On occasions, participants said that an item was irrelevant to their situation but chose the answer “I somewhat agree” or “I fully disagree” instead of choosing the option “Not relevant,” with the result that a possibly neutral experience appeared negative.
What should I answer to this one [this question]? That was truly hard. Umm … No, I fully disagree, it must be that. That one was slightly funny because it might not represent how it was at all. However, it could be others who, other situations, ohh. No, I can’t answer that, it’s not right. (Participant 5, round one, answering Item 15)
The response options for all items in the questionnaire were shortened to (1) “Fully agree”, (2) “Largely agree”, (3) “Somewhat agree”, and (4) “Fully disagree”, possibly eliminating the risk of misinterpreting answer options. The option “Not relevant” was not altered.
Regarding involvement in care, the participants identified a lack of questions related to their receiving information prior to elective surgery and information on whom to turn to after the care period at the hospital. The research group thought that these aspects were covered by items inquiring whether they had received sufficient information leading to no alterations at this point.
All items with problems in round one were discussed by the research group. Given the extent of the changes to the first seven items, changes to items related to nursing care were set aside to see whether these problems persisted in the second round.
Cognitive interviews with Think-aloud: Round two
In the second round, the problems identified in round one were confirmed. Rereading an item was not identified as a problem in this round, making the potential maximum number of problems in round two 414. The most prominent problem concerned Items 1–7, where the participant was asked to define important aspects of involvement in general terms and not to relate their answers to the specific care period. The total number of problems identified in round two was 42, whereas 32 concerned Items 1–7.
Defining family involvement.
Problems with misinterpreting the purpose of Items 1–7 remained after the second round of cognitive interviews, and they were therefore excluded from the questionnaire.
Yes, but there you are, I think we were just grateful for the care given, so I don’t think, I don’t think we questioned or participated in discussions in that way. Because, you know, now this is one of those questions you can interpret a little from my point of view, “participate in discussions”, that is … If I’d wanted to, I would probably have gotten the opportunity. (Participant 6, round two, answering Item 4)
Misinterpretation of terms.
Consistent with the results in round one, the term “nursing care” was considered problematic, a problem that was also pointed out by the expert group in step II.
No, that’s not true at all, I … That’s taken care of by the professionals at the hospital, so I don’t need to get involved with that. Fully disagree. (Participant 7, round two, answering Item 23)
In the final version of the questionnaire, the term “nursing care” was replaced with examples of basic care activities. The term “support” was also questioned, as it was ambiguous.
Well, thinking of this in terms of a questionnaire construct … It’s actually an open question, a very open question. What does it mean? What kind of support could one get? The question is kind of unclear then, in my opinion. “The support I needed” … A hug?–No, I did not get that. *Laughs* You understand? It’s sort of unclear, what do you need? Umm, but in this case I suppose I got what I needed from the staff, kind of, so I suppose I would have answered largely agree or maybe … yes, I would have. (Participant 8, round two, answering Item 21)
In response to this observation, it was specified that the “support” referred to was emotional support.
Underuse of the “Not relevant” option.
As in round one, when faced with items that the participants regarded as not relevant for them, they chose the option “Fully disagree,” making it a negative experience instead of a neutral one.
No, well he could speak for himself. *Rereads the question. * No that * rereads the question again*, well I don’t know what to answer, but he could speak for himself. Fully disagree then. Must be right, right? Yes, that one is it. (Participant 9, round two, answering Item 15)
As is clear from the response above, the item regarding whether the family member had been given an opportunity to express the patient’s presumed will was problematic. Therefore, an example of when patients might be unable to express their will (e.g., while sedated in the ICU) was added, and the “Not relevant” option was explained in the instruction to the questionnaire and changed to bold font.
Retrospective probes- possibly identifying unverbalized problems.
Answers to the retrospective probes in round two revealed that three of the nine participants had difficulty limiting their reported experiences to the specific care period in the cardiothoracic surgical department. “Discussions” and “decision-making” were reported as somewhat confusing concepts in retrospect. One person reported missing items of information regarding possible outcomes and persisting issues for the patient at home.
What I think is that I should have been given the opportunity to express my point of view on my experiences in the time afterward. Because I don’t know, I usually say to my wife, you are not the same as you were before you went away. That is, her temper changed and umm, these things. There were no questions about that. (Participant 10, round two, answering probe of problematic/missing items)
Potential missing items proposed by participants were discussed by the research group but deemed to be covered by items relating to information. The final version of the questionnaire consisted of 18 items: 16 items with a four-point Likert scale and two open-ended items. An overview of the problems and alterations made in the three tested versions of the Family Involvement in Care Questionnaire (FICQ) is shown in Table 3. The final version of the FICQ is displayed in the Supporting Information file S1 Appendix.
Discussion
FICQ was developed as a measure of family involvement in inpatient care. The questionnaire consists of 18 items of which 16 has been reformulated and adapted from two existing instruments, the PIQ [39] and the SFS-ICQ) [40]. FICQ is designed for family members of patients who have received inpatient care, and its content validity has been evaluated in the context of open-heart surgical care. The expert group’s rating of relevance indicated excellent content validity prior to the target group interviews. If the investigation had stopped at that point, the problems for the target population would not have been identified [41]. This indicates that evaluation using the target population is important, and theory- and expert-based instruments are not sufficient.
In this study, having items with two separate purposes was problematic. In the initial questionnaire, important aspects of family involvement were rated in a general manner (Items 1–7 in Table 3). These items were followed by items evaluating the participant’s rating of being involved during a specific care period. This made some participants uncertain about the procedure. It was therefore decided not to mix questions with different purposes, and Items 1–7 were excluded. The original version of these items has not been reported as problematic [39].
Active family involvement in hospital care can contribute to a reduction in surgical complications, early detection of delirium, and facilitation of early mobilization [7] and should therefore be encouraged. Personal hygiene [10], oral care, and mobilization [7, 10] are examples of basic care activities the family can be offered to contribute to in a hospital setting. However, the term “nursing care” was problematic for some participants in this study, as was also found by Thermeanius et al. [40]. Nursing care was seen by some family members as a professional domain in the hospital [40], or as only becoming the responsibility of the family when the patient was discharged [42]. Perhaps the term “basic care”, as described in previous litterature [7, 10], is better suited for this purpose. The family members can be given concrete examples of caring tasks that they can carry out in the hospital environment. In this study, the term “nursing care” was ultimately changed to examples of caring activities, e.g., helping with meals (see Item 23 in Table 3).
Another problematic term in our questionnaire was the “support” given by the staff to the family. Patients have described recovery after surgery as possibly harder on the family than on the patients themselves [18]. Previous research has explored the importance of supporting the whole family when one family member has undergone surgery [42]. In our study, some participants questioned the relevance of evaluating the support they themselves had received or asking what kind of support they could have had. It has previously been shown that family members can find it difficult to identify their own needs during a loved one’s hospitalization, except for the need for information [1]. The types of support nurses provide to family members have been categorized as either cognitive or emotional [43]. Cognitive support entails education and information to support coping, while emotional support has been described as helping the family member handle the range of emotions experienced when a family member is ill [43]. Informational support has been described as the most important aspect of supporting the family [44] and is adequately covered in the FICQ. Therefore, the item regarding support (Item 21 in Table 3) was modified to specify “emotional support”.
Alterations were made after the second round of cognitive interviews (see Items 15, 21 and 23 in Table 3), and Items 1–7 were excluded. The remaining items had a total of 10 problems, with one item having three problems. One could therefore argue that a third round of cognitive interviews would have been required to evaluate these changes [45]. However, as Willis [34] states, when conducting cognitive interviews, investigators must at some point decide that enough is enough. It was thus decided that these items could be tested further in a future quantitative psychometric study with a focus on scale evaluation [31].
Strengths and limitations
The initial three steps in best practices guidelines for questionnaire development were followed [31]. Attempts were also made to achieve the goals of trustworthiness and credibility, transferability, dependability, and confirmability as described by Shenton [45]. Credibility was strengthened by using well-established research methods. All authors are nurses with experience in intra- and/or postoperative care settings. The first and last authors are very familiar with the specific context where the study took place as they were employees at the clinic, but they had no ongoing or previous interactions with neither the family members being interviewed nor the expert nurses that evaluated the questionnaire. Sampling for the cognitive interviews was random, which is a strength in some instances [45]. Triangulated qualitative methods were used to test validity through the convergence of information from health care providers and family members. One person conducted the interviews, thus contributing to the standardization of the interviews. All four authors analyzed interviews from both rounds using a structured protocol [35], strengthening credibility both in terms of frequent debriefing and the trustworthiness of the analysis. The six aspects enabling the assessment of transferability have been accounted for [45]. Regarding dependability and confirmability, our aim has been to preserve an audit trail that will be helpful as a framework for future investigations relating to cognitive interviews on videoconferencing platforms. We found the method of using cognitive interviews in the validation process to be very useful in informing further questionnaire development.
Our study does have some limitations. One was that the researcher who conducted the interviews was a doctoral student with limited experience from interviewing prior to this study. She was provided continuous support regarding the interview technique from the second author which have prior experience in using the Think-aloud method, as well as supervision from the other two authors. One was the limited ability to observe and assess nonverbal communication when using online videoconferencing platforms rather than face-to-face interviews [46]. Not all participants chose to have their video on. The practice of allowing long pauses was compromised, as pauses could indicate technical problems. In the first round, the intention was to conduct Think-aloud interviews using no verbal probing besides the prompt “think out loud”. Following this strategy consistently was difficult in the first round, as silence could be an indication of a cognitive problem of the participant or technical problems such as losing internet connection. With the broader probing strategy in the second round of cognitive interviews, a few new problems were identified, such as the fact that three out of the nine participants in the second round had difficulties limiting their responses to the one care period asked about. Several previously identified problems were confirmed, such as defining family involvement and misinterpreting the term nursing care. This suggests that the method is suitable and that a structured concurrent and retrospective probing strategy could be beneficial when using videoconferencing platforms such as Zoom. There were, in fact, some instances of technical problems with interrupted connections and audio failure. However, previous qualitative research on the use of Zoom indicates that technical problems can contribute to building rapport between the investigator and the study participant [47]. Such problems can generate collaboration, which contributes to building trust, a stable ground when collecting qualitative data [47].
The technical demands on participants led to one person declining to participate, indicating an unequal aspect of technical requirements on participants. The aspect of availability to presumptive study participants is addressed by Carter et al. [48], that states availability does not decrease but rather changes when interviews are conducted online. The solution to the technical demands on participants could be to provide all necessary equipment and travel to the participants’ homes to set up the technology [49]. This does nevertheless have some drawbacks that affect the advantages of cost, time, and efficiency [47].
One limitation of this study is that a potential generic questionnaire measuring family involvement in inpatient care was tested in only one inpatient care setting. Therefore, the questionnaire needs further psychometric testing in various settings before any conclusions regarding validity and reliability as a generic inpatient measure can be drawn. However, as previously stated, the inpatient cardiothoracic surgical care pathway is diverse, including different care settings in one, and was therefore chosen in this first step of development and validation. The items in FICQ could be perceived as only relevant to critical care, such as after myocardial infarction or while in intensive care. This was, however, not found to be problematic in our study by either experts or target group participants. Questions developed for patient participation in a cardiology context [39] adapted for family involvement were regarded as valid by both experts and the targeted population, with the exception of items 1–7 which therefore were excluded.
Conclusions
The usage of this cognitive validation model resulted in a refined questionnaire, ready to be tested quantitatively regarding reliability and further aspects of validity. Direct target population feedback was effective to receive prior to administration to a large sample of family members. Before using FICQ in a clinical setting, further research needs to be conducted to determine whether the questionnaire is psychometrically sound. After being validated and reliability tested, the FICQ can be used to provide clinicians and policymakers with information on how families are supported, informed, and treated in connection with inpatient care. The extent of their involvement could be used as an indicator of the quality of care and used when evaluating interventions that aim to increase family involvement in care.
Supporting information
S1 Appendix. Final version of the Family Involvement in Care Questionnaire (FICQ).
https://doi.org/10.1371/journal.pone.0285562.s001
(PDF)
Acknowledgments
The authors wish to thank Dr. Judy Arnetz and Mr. Johan Thermaenius and colleagues for supplying their original questionnaires. We would also like to acknowledge the nurses who contributed their expertise and the family members for helping us understand how the questionnaire was understood. Without their participation, our research would not have been possible.
References
- 1. Bélanger L, Desmartis M, Coulombe M. Barriers and facilitators to family participation in the care of their hospitalized loved ones. Patient Exp J. 2018;5(1): 56–65.
- 2. Park M, Giap TT. Patient and family engagement as a potential approach for improving patient safety: a systematic review. J Adv Nurs. 2020;76(1):62–80. pmid:31588602
- 3. Leino-Kilpi H, Gröndahl W, Katajisto J, Nurminen M, Suhonen R. Participation of family members and quality of patient care—the perspective of adult surgical patients. J Clin Nurs. 2016;25(15–16):2242–50. pmid:27334560
- 4. Mackie BR, Mitchell M, Marshall AP. Patient and family members’ perceptions of family participation in care on acute care wards. Scand J Caring Sci. 2019;33(2):359–70. pmid:30507038
- 5. Huffines M, Johnson KL, Smitz Naranjo LL, Lissauer ME, Fishel MA, D’Angelo Howes SM, et al. Improving family satisfaction and participation in decision making in an intensive care unit. Crit Care Nurse. 2013;33(5):56–69. pmid:24085828
- 6. Schreuder AM, Eskes AM, van Langen RGM, van Dieren S, Nieveen van Dijkum EJM. Active involvement of family members in postoperative care after esophageal or pancreatic resection: a feasibility study. Surgery. 2019;166(5):769–77. pmid:31285045
- 7. Eskes AM, Schreuder AM, Vermeulen H, Nieveen van Dijkum EJM, Chaboyer W. Developing an evidence-based and theory informed intervention to involve families in patients care after surgery: a quality improvement project. Int J Nurs Sci. 2019;6(4):352–61. pmid:31728386
- 8.
Shajani Z, Snell D. Wright & Leahey’s nurses and families: a guide to family assessment and intervention. Edition 7. Philadelphia: F.A. Davis Company; 2019.
- 9. Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). 2013;32(2):223–31. pmid:23381514
- 10. Goldfarb M, Bibas L, Burns K. Patient and family engagement in care in the cardiac intensive care unit. Can J Cardiol. 2020;36(7):1032–40. pmid:32533931
- 11. Heydari A, Sharifi M, Moghaddam AB. Family participation in the care of older adult patients admitted to the intensive care unit: A scoping review. Geriatr Nurs. 2020;41(4):474–84. pmid:32059826
- 12. Olding M, McMillan SE, Reeves S, Schmitt MH, Puntillo K, Kitto S. Patient and family involvement in adult critical and intensive care settings: a scoping review. Health Expect. 2016;19(6):1183–1202. pmid:27878937
- 13. Andersen IC, Thomsen TG, Bruun P, Bødtger U, Hounsgaard L. Patients’ and their family members’ experiences of participation in care following an acute exacerbation in chronic obstructive pulmonary disease: a phenomenological-hermeneutic study. J Clin Nurs. 2017;26(23–24):4877–89. pmid:28722760
- 14. Bell JM. Family Systems Nursing: re-examined. J Fam Nurs. 2009;15(2):123–9. pmid:19423766
- 15. Saveman BI. Family nursing research for practice: the Swedish perspective. J Fam Nurs. 2010;16(1):26–44. pmid:20145285
- 16. Gwaza E, Msiska G. Family involvement in caring for inpatients in acute care hospital settings: a systematic review of literature. SAGE Open Nurs. 2022;8:23779608221089541. pmid:35434303
- 17. Whall AL. The family as the unit of care in nursing: a historical review. Public Health Nurs. 1986;3(4):240–9. pmid:3547384
- 18. Kemp KA, Naqvi F, Quan H, Paolucci EO, Knudtson ML, Santana MJ. Eliciting patient experiences about their care after cardiac surgery. CJC Open. 2020;3(4):427–33. pmid:34027345
- 19. Gimpel D, Shanbhag S, Srivastava T, MacLeod M, Conaglen P, Kejriwal N, et al. Early discharge from intensive care after cardiac surgery is feasible with an adequate fast track, stepdown unit: Waikato experience. Heart Lung Circ. 2019;28(12):1888–95. pmid:30528814
- 20. Onwochei DN, Fabes J, Walker D, Kumar G, Moonesinghe SR. Critical care after major surgery: a systematic review of risk factors for unplanned admission. Anaesthesia. 2020;75 Suppl 1:e62–e74. pmid:31903563
- 21. Joseph HK, Whitcomb J, Taylor W. Effect of anxiety on individuals and caregivers after coronary artery bypass grafting surgery: a review of the literature. Dimens Crit Care Nurs. 2015;34(5):285–8. pmid:26244244
- 22. Saeid Y, Salaree MM, Ebadi A, Moradian ST. Family intensive care unit syndrome: an integrative review. Iran J Nurs Midwifery Res. 2020;25(5):361–8. pmid:33344205
- 23. Mackie B, Mitchell M, Marshall P. The impact of interventions that promote family involvement in care on adult acute-care wards: an integrative review. Collegian. 2018;25(1):131–40.
- 24. Ewertzon M, Lützén K, Svensson E, Andershed B. Developing the concept of family involvement and the alienation questionnaire in the context of psychiatric care. Int J Ment Health Nurs. 2008;17(6):439–49. pmid:19128291
- 25. Harvey K, Catty J, Langman A, Winfield H, Clement S, Burns E, et al. A review of instruments developed to measure outcomes for carers of people with mental health problems. Acta Psychiat Scand. 2008;117(3):164–76. pmid:18241311
- 26. Reid RC, Chappell NL, Gish JA. Measuring family perceived involvement in individualized long-term care. Dementia. 2007;6(1):89–104.
- 27. Verho H, Arnetz JE. Validation and application of an instrument for measuring patient relatives’ perception of quality of geriatric care. Int J Qual Health Care. 2003;15(3):197–206. pmid:12803347
- 28. Westergren A, Behm L, Lindhardt T, Persson M, Ahlström G. Measuring next of kin’s experience of participation in the care of older people in nursing homes. PLoS One. 2020;15(1):e0228379. pmid:32004352
- 29. Schick-Makaroff K, Karimi-Dehkordi M, Cuthbertson L, Dixon D, Cohen SR, Hilliard N, et al. Using patient- and family-reported outcome and experience measures across transitions of care for frail older adults living at home: a meta-narrative synthesis. Gerontologist. 2021;61(3):e23–e38. pmid:31942997
- 30. van den Broek JM, Brunsveld-Reinders AH, Zedlitz AM, Girbes AR, de Jonge E, et al. Questionnaires on family satisfaction in the adult ICU: a systematic review including psychometric properties. Crit Care Med. 2015;43(8):1731–44. pmid:25821917
- 31. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. 2018;6:149. pmid:29942800
- 32. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. pmid:17872937
- 33. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459–67. pmid:17654487
- 34.
Willis GB. Analysis of the cognitive interview in questionnaire design. Oxford: Oxford University Press; 2015.
- 35. French DP, Cooke R, Mclean N, Williams M, Sutton S. What do people think about when they answer theory of planned behaviour questionnaires?: a `think loud’ study. J Health Psychol. 2007;12(4):672–87. pmid:17584818
- 36. Campbell JL, Quincy C, Osserman J, Pedersen OK. Coding In-depth semistructured interviews: problems of unitization and intercoder reliability and agreement. Sociol Methods Res. 2013;42(3):294–320.
- 37.
World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. [cited 16 November 2022]. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/
- 38.
Utbildningsdepartementet. Lag om etikprövning av forskning som avser människor [Act on ethical review of research involving humans] (SFS 2003:460). [cited 16 November 2022]. Available from: https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460
- 39. Arnetz JE, Höglund AT, Arnetz BB, Winblad U. Development and evaluation of a questionnaire for measuring patient views of involvement in myocardial infarction care. Eur J Cardiovasc Nurs. 2008;7(3):229–38. pmid:18077219
- 40. Thermaenius J, Schandl A, Sluys KP. Development and initial validation of the Swedish family satisfaction intensive care questionnaire (SFS-ICQ). Intensive Crit Care Nurs. 2019;50:118–24. pmid:29935980
- 41. Ryan K, Gannon-Slater N, Culbertson MJ. Improving survey methods with cognitive interviews in small- and medium-scale evaluations. Am J Eval. 2012;33(3):414–30.
- 42. Bjørnnes AK, Moons P, Parry M, Halvorsen S, Tønnessen T, Lie I. Experiences of informal caregivers after cardiac surgery: a systematic integrated review of qualitative and quantitative studies. BMJ Open. 2019;9(11):e032751. pmid:31719093
- 43. Bruce E, Dorell Å, Lindh V, Erlingsson C, Lindkvist M, Sundin K. Translation and testing of the Swedish version of Iceland-family perceived support questionnaire with parents of children with congenital heart defects. J Fam Nurs. 2016;22(3):298–320. pmid:27402026
- 44. Kynoch K, Ramis MA, Crowe L, Cabilan CJ, McArdle A. Information needs and information seeking behaviors of patients and families in acute healthcare settings: a scoping review. JBI Database System Rev Implement Rep. 2019;17(6):1130–53. pmid:31192898
- 45. Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Educ Inf. 2004;22(2):63–75.
- 46. De Villiers C, Farooq MB, Molinari M. Qualitative research interviews using online video technology—challenges and opportunities. Meditari Account Res. 2021. Epub 2021 Aug 27.
- 47. Archibald MM, Ambagtsheer RC, Casey MG, Lawless M. Using Zoom videoconferencing for qualitative data collection: perceptions and experiences of researchers and participants. Int J Qual Methods. 2019;18:1–8.
- 48. Carter SM, Shih P, Williams J, Degeling C, Mooney-Somers J. Conducting qualitative research online: challenges and solutions. Patient. 2021;14(6):711–18. pmid:34114170
- 49. Shepperd JA, Pogge G, Hunleth JM, Ruiz S, Waters EA. Guidelines for conducting virtual cognitive interviews during a pandemic. J Med Internet Res. 2021;23(3):e25173. pmid:33577464