https://orcid.org/0000-0002-9488-9458
Figures
Abstract
Background
Cluster randomized trials (cRTs) on the effectiveness of training programs face complex challenges when conducted in real-world settings. Process evaluations embedded within cRTs can help explain their results by exploring possible causal mechanisms impacting training effectiveness.
Objective
To conduct a process evaluation embedded within a cRT by comparing the impact of team-based vs. individual clinician-focused SICP training on primary healthcare professionals’ (PHCPs) intention to have serious illness conversations with patients.
Methods
The cRT involved 45 primary care practices randomized into a team-based (intervention) or individual clinician-focused (comparator) training program and measured primary outcomes at the patient level: days at home and goals of care. To perform this theory-informed mixed-methods process evaluation embedded within the cRT, a different outcome was measured at the level of the PHCPs, namely, PHCPs’ intention to have serious illness conversations with patients as measured with CPD-Reaction. Barriers and facilitators to implementing the conversations were identified through open-ended questions and analyzed using the Theoretical Domains Framework. The COM-B framework was used to triangulate data. Results were reported using the CONSORT and GRAMMS reporting guidelines.
Results
Of 535 PHCPs from 45 practices, 373 (69.7%) fully completed CPD-Reaction (30.8% between 25-34 years old; 78.0% women; 54.2% had a doctoral degree; 50.1% were primary care physicians). Mean intention scores for the team-based (n = 223) and individual clinician-focused arms (n = 150) were 5.97 (standard error (SE): 0.11) and 6.42 (SE: 0.13), respectively. Mean difference between arms was 0.0 (95% CI -0.29; 0.30; p = 0.99) after adjusting for age, education and profession. The team-based arm reported barriers with communication, workflow, and more discomfort in having serious illness conversations with patients.
Conclusions
Team-based training did not outperform individual clinician-focused in influencing PHCPs’ intention to have serious illness conversations. This process evaluation suggests that team-based training could improve intervention effectiveness by focusing on interprofessional communication, better organized workflows, and better support and training for non-clinician team members.
Registration: ClinicalTrials.gov (ID: NCT03577002).
Citation: Gomes Souza L, Archambault PM, Asmaou Bouba D, Dofara SG, Guay-Bélanger S, Cortez Ghio S, et al. (2025) Impact of a team-based versus individual clinician-focused training approach on primary healthcare professionals’ intention to have serious illness conversations with patients: A theory-informed process evaluation embedded within a cluster randomized trial. PLoS ONE 20(3): e0298994. https://doi.org/10.1371/journal.pone.0298994
Editor: Mohammed Abutaleb, Jazan Health Cluster, SAUDI ARABIA
Received: March 26, 2024; Accepted: January 14, 2025; Published: March 26, 2025
Copyright: © 2025 Gomes Souza et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting information files.
Funding: Research reported in this paper was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (PLC-1609-36277) The results presented in this paper are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee. FL holds the Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Mobilization (Funder Grant: N/A). LGS received the scolarship Fonds stratégique de développement de la recherche from VITAM-centre de recherche en santé durable (Funder Grant: N/A). PA received a Fonds de recherche du Québec – Santé Clinical Scholar Award.The funders did not play any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Cluster Randomized Trials (cRTs) using a pragmatic approach aim to evaluate the effectiveness of interventions in real-world clinical settings, providing advantages such as enhanced generalizability and relevance to routine care [1]. However, pragmatic cRTs present unique challenges. Implementing and maintaining interventions in cRTs within the complex environment of routine clinical care is difficult due to variations in clinical practice, healthcare systems, and participant populations [2]. Process evaluations embedded within cRTs are therefore increasingly used to shed light on the ‘black box’ of complex interventions and to provide information that helps interpret outcome results and move them into practice [3,4].
In 2017, the Meta-Network Learning and Research Center Advanced Care Planning (Meta-LARC ACP) cRT team designed a comparative effectiveness cRT to assess two training approaches for the Serious Illness Care Program (SICP) developed by Ariadne Labs [5,6]. The cRT compared a team-based training approach to conducting the SICP (intervention) with the traditional clinician-based training approach (comparator) [7]. The rationale for this cRT stemmed from the initial assumption in the design of the SICP, that only individual clinicians such as physicians, nurse practitioners, or physician assistants, are responsible for having serious illness conversations with patients. However, insights from studies on team approaches to chronic and complex care suggest that adopting a team-based approach could facilitate the implementation of serious illness conversations within clinical practice [8,9].
In recognition of the pivotal role played by primary care teams in patient care, the cRT explored a training approach that integrated task-sharing strategies. The shared tasks included patient identification, conversation preparation, discussion initiation, and follow-up [8,10]. The team approach aligned with systematic reviews which identified time constraints as a significant barrier to implementing these discussions in primary care settings. The expectation was that involvement of multiple primary healthcare professionals (PHCPs) could facilitate implementation by reducing the time commitment required from each individual [11,12]. However, evidence supporting the efficacy of the team-based approach in primary care was limited at the time the cRT was conceived [13]. This underscored the need to evaluate the approach in a real-world primary care setting.
Process evaluations embedded within cRTs aim to provide insights into how an intervention was delivered, how participants received it, and whether it was implemented as intended [4]. In the parent Meta-LARC ACP cRT, the impact of a team-based training approach compared to a clinician-based training approach was measured using patient centered outcomes [7]. In this process evaluation, measuring the impact of the intervention on PHCPs themselves instead of patients and, specifically, on PHCPs’ intention to have serious illness conversations with patients was necessary to provide insights into how the team-based training approach was delivered, received and implemented in the parent cRT.
A process evaluation focusing on PHCPs’behavioral intention, and its psychosocial determinants would thus provide information about key factors affecting effective implementation of the intervention and could inform recommendations for its improvement. Key factors would include the modifiable psychosocial factors that influence PHCPs’ intention to have serious illness conversations, and barriers and facilitators that they perceive to implementing the conversations. Thus, to influence behavior through behavioral intention, one would need to modify psychosocial factors using known behavior change techniques [14]. No studies have specifically addressed the factors that drive PHCPs’ intentions to have serious illness conversations with patients. Understanding these components is crucial for designing training interventions that effectively promote the adoption of this important behavior in clinical practice.
According to several socio-cognitive theories, behavioral intention is the central factor influencing the adoption of a given behavior and is therefore an acceptable surrogate for behavior change [15–19]. It provides a more immediate and cost-effective measure of actual clinical behavior change and is widely used to evaluate the efficacy of professional development training interventions [20]. Measuring intention would thus allow for a targeted assessment of the intervention’s influence on PHCPs, the primary implementers of the intervention in the parent cRT. It could also provide information on intervention strengths and weaknesses to inform the interpretation and adaptation of future SICP interventions [21]. Ultimately, it could contribute to a the cumulative implementation science knowledge base.
In order to achieve these aims, three established socio-cognitive theories of behavior change were incorporated into this process evaluation [14,16,22,23]. These theoretical frameworks played key roles in identifying and addressing specific implementation challenges, enhancing our understanding of the intervention, and suggesting modifications that could increase the impact of future such interventions [21].
We hypothesized, based on our previous theory-informed studies on interprofessional interventions [24,25], that a team-based training would outperform an individual clinician-focused training in influencing PHCPs’intention to have serious illness conversations with their patients. These studies demonstrate that an interprofessional team-based model could pottentiallypromote positive intentions to have serious illness conversations with patients. Furthermore, these studies on interprofessional interventions elucidated the modifiable psychosocial factors that positively correlate with intention within an interprofessional context [24,25].
Therefore, we performed a theory-based process evaluation embedded within the parent cRT to assess the effect of the two SICP training approaches on PHCPs’ intention to have serious illness conversations with patients and to identify potential facilitators and barriers to the adoption of this behavior.
Materials and methods
Study design and setting
This study is a mixed-methods concurrent embedded process evaluation embedded within a cRT [26–28]. Secondary post-intervention qualitative and quantitative data from the parent cRT were used in the analysis. Process evaluations often employ mixed methods designs which provide a more comprehensive and multifaceted understanding of intervention implementation [21,29]. We used the integrative model of behavior prediction in healthcare professionals by Godin et al. for our quantitative analysis [15,16,30], the Theoretical Domains Framework (TDF) for our qualitative analysis [22,31], and the Capability, Opportunity, Motivation and Behavior (COM-B) model to triangulate findings [32]. We reported this study following the extension of Consolidated Standards of Reporting Trials (CONSORT) for cRTs (S1 Checklist) and followed the Good Reporting of a Mixed Methods Study (GRAMMS) checklist (S2 Checklist) [33,34]. The parent cRT (subject of the process evaluation) is registered at ClinicalTrials.gov (NCT03577002). The protocol is published elsewhere [7] and the results for the cRT primary outcomes at the patient level (days spent at home and goal concordant care) are under review.
The parent Meta-LARC ACP cRT was conducted in community-based primary care practices in five US states (Colorado, Iowa, North Carolina, Oregon, Wisconsin) and two Canadian provinces (Quebec and Ontario) recruited through Meta-LARC, a consortium of practice-based research networks (PBRNs) [7]. A cRT design was selected as the knowledge learned in the SICP trainings was to be implemented at a practice level.
The Medical Research Council framework [29] was used to guide the process evaluation. This framework suggests that the context and mechanisms of impact of an intervention are the key functions of a process evaluation. To explore the main mechanisms of impact, this study measured PHCPs’ intention (and its modifiable psychosocial factors) to adopt the target behavior, defined as having serious illness conversations with patients. A previous study focused on the development process of the intervention [35]. Another study explored the contextual factors that influenced the sustainability of its impacts, particularly in the context of the COVID-19 pandemic [36]. This combined approach allows to address all the different aspects of a process evaluation [29].
Ethical approval
The parent Meta-LARC ACP cRT was approved by the Trial Innovation Network Single Institutional Review Board at Vanderbilt University Medical Center (#181084) for the U.S. sites; by the Research Ethics Board of the Centre Intégré Universitaire de Santé et de services sociaux (CIUSSS) de la Capitale-Nationale in Quebec City, Canada (#MP-13-2019-1526) for the sites in Quebec, and by the Health Sciences Research Ethics Board of the University of Toronto (#36631) for the sites in Ontario. Data and outcomes for this process evaluation were included in the approval. All subjects willingly agreed to take part in the study, and their consent (verbal or written) was obtained and registered by PBRNs in accordance with the regulations of the Institutional Review Board or Research Ethics Board in effect.
Participants and eligibility
To participate in the parent cRT a practice had to a) be willing and able to be randomized to either the team-based or individual-clinician-based SICP approach, b) have sufficient staff to participate in the team-based arm, and c) not be engaged in another standardized advanced care planning (ACP) program. Detailed criteria for practice eligibility are published elsewhere [7]. In this process evaluation, we analyzed a secondary outcome, the impact of the intervention on PHCPs’ behavioral intention, and therefore participants for this process evaluation were the PHCPs (primary care clinicians and other primary care team members) working in the primary care practices enrolled in the parent cRT. PHCPs from participating eligible practices were invited to participate in the training and to answer the after-training questionnaires, but not required to participate.
Randomization
The units of randomization were the primary care practices stratified by PBRN. To assure allocation concealment, involvement in randomization was limited to statisticians not involved in other aspects of the project. Staff at the PBRNs, practices, and the Meta-LARC coordinating center were not involved in the randomization. Statisticians completing the analysis were blinded to allocation until the parent cRT primary outcomes analysis was completed. Investigators, PBRN leadership, practices, and research staff were not blinded to the assignment. Practices and participating PHCPs could not be blinded to which approach they were assigned, as they needed to be actively trained and implement the intervention or comparator. More details regarding randomization are published elsewhere [7].
Intervention and comparator arms
The SICP developed by Ariadne Labs was adapted to be used by interprofessional teams of PHCPs [5]. Training lasted three hours per arm: a 1.5-hour online module (Part A), and a 1.5-hour in-person role-play session (Part B) (Fig 1). Training materials are available at https://primarycareacp.org, including the Serious Illness Conversations Guide (SICG), a tool designed by the original developers of the SICP to facilitate communication with patients with serious illnesses [5].
Recruitment and training of PHCPs took place between 26-10-2018 and 30-11-2019. A train-the-trainer model was used. The intervention proceeded as described in the study protocol [7].
Intervention arm
The intervention arm received SICP training session adapted to a team-based approach, whereby primary care clinicians (i.e., primary care physicians, physician assistants, nurse practitioners, and medical residents) and other team members such as medical assistants, nurses, dietitians, and social workers were invited to complete the training. The adapted team-based approach was based on an interprofessional shared decision-making model and workshop [37,38] that has been used in a variety of contexts and showed a positive impact on behavioral intention [39,25]. Training involved guidance on establishing a common understanding of the processes and goals of serious illness conversations, dividing and sharing tasks, recognizing each member’s contributions, promoting continuous communication in the team, and recognizing the organizational or functional constraints of each profession [35].
Comparator arm
The comparator arm received the individual clinician-focused approach as designed by Ariadne Labs [5]. It focuses only on primary care clinicians (i.e., primary care physicians, physician assistants, nurse practitioners, and medical residents). Other primary care team members, such as medical assistants, nurses, dietitians, and social workers did not participate in the training. This approach assumes that primary care clinicians alone are responsible for selecting patients, conducting, and documenting serious illness conversations.
Outcomes and measurements
The outcome of interest for this process evaluation was the impact of our training intervention on PHCPs’ intention to have serious illness conversations with patients. This focus on intention was based on its established role as a proxy for behavior and its demonstrated predictive capacity for behavior change up to six months following training interventions [15–18].
We used the CPD-Reaction tool to measure PHCPs’ behavioral intention to have serious illness conversations with patients. CPD-Reaction is a self-administered, theory-informed, validated tool that assesses five psychosocial constructs (intention, beliefs about capabilities, beliefs about consequences, moral norm, and social influences) using a 12-item self-administered questionnaire [17,40]. Most items are measured using a Likert scale of 1–7, where 1 represents “strongly disagree” and 7 represents “strongly agree.” One item, the social influences construct, ranges from 1 to 5 [30].
However, as proposed by Lou, Atkins and West [22], behaviors do not occur in isolation. Depth of analysis is increased by expanding the range of relevant behaviors measured while maintaining appropriate selectivity [14]. Therefore, as additional behavior-related outcomes for this process evaluation, participants were also asked about their likelihood to engage in ACP and to use the SICG. ACP was included because it is a more established, broader concept in the field that encompasses end-of-life discussions, care objectives, and their legal documentation [41]. Questions about PHCPs’ likelihood to engage in ACP were presented before and after Part A, using a retrospective pretest-posttest evaluation design [42]. Questions about the likelihood to use the SICG in their practices were presented after Part B. Face and content validity of the likelihood measures were established based on studies on similar training programs with PHCPs [43,44]. These questions also used a Likert scale of 1 to 10 where 1 represented “extremely unlikely”, 5 represented “moderate,” and 10 “extremely likely.”
Finally, the facilitators and barriers to PHCPs incorporating the knowledge acquired in their practices and to have serious illness conversations with patients were evaluated using two open-ended questions. These facilitators and barriers were mapped onto the TDF. The TDF was developed through a consensus of experts who consolidated 33 psychosocial theories of behavior change to generate 14 domains [45]. Quantitative and qualitative findings were then triangulated using the the COM-B model of behavior aiming to formulate recommendations for future training programs or SICP adaptations [14]. Fig 1. details all the process evaluation measurements and outcomes sought throughout the different steps of the intervention.
Data collection
Data collection took place between October 2018 and November 2019. Before randomization, sociodemographic data were collected as well as additional data directly from practices (e.g., country and practice environment) (Fig 1). This analysis uses the data collected through questionnaires administered immediately after training was completed in each practice, as we were interested in PHCPs’ immediate reactions to each training approach. All questionnaires in the study were self-administered. Open-ended questions were used instead of semi-structured interviews or focus groups to explore barriers and facilitators. These decisions streamlined data collection, enhancing both efficiency and participant convenience.
Sample size
The sample size calculation for the parent study is available in the published protocol [7]. A sample size estimate for PHCPs involved in this process evaluation study was not calculated as their responses were not the primary outcome of the parent cRT. A total of 535 PHCPs attended the trainings, 326 in the team-based and 209 in the individual clinician-focused arms. Data from 373 PHCPs (223 team-based and 150 individual clinician-focused) who attended the training and completed the intention construct in the CPD-Reaction tool were analyzed. For these 373 PHCPs, a post-hoc power of 0.75 based on an unadjusted mixed linear regression model was obtained.
Analysis
Quantitative analysis.
Descriptive statistics were used to report our variables including practice and participant characteristics. Categorical variables were described with absolute (n) and relative (%) frequencies; and continuous variables with their central tendency measures (mean) and dispersion (standard deviation).
Then, a mixed linear model was used to compare mean scores between study arms for PHCPs’ intention to have serious illness conversations with patients. To account for clustering the practice identifier was used as a random factor when fitting the models.
Next, a bivariate mixed linear model for each sociodemographic variable of interest to examine its effect on PHCPs’ intention to have serious illness conversations with patients was applied. Subsequently, a multivariable mixed linear model including all variables for which we detected a potential effect (p < 0.20) on PHCPs’ intention in the bivariate analyses was used. Then manual backward stepwise selection based on variable significance for a final adjusted model was performed. The study allocation arm variable was kept in the model regardless of significance as the objective was to compare the impact of training approaches.
Finally, a mixed linear model was used to evaluate participants’ perceived likelihood of engaging patients in ACP (i.e., before (reported retrospectively) and then after training), and of using the SICG in their practice after both parts of the training. The mean differences between the likelihood to engage in ACP measured before and after Part A of the training between each arm was compared, as were mean differences between the likelihood to use the SICG, also measured between each arm. The practice identifier was used as a random factor in all models. All analyses were performed with SAS (Statistical Analysis Software) 9.4.
Qualitative analysis.
To further explore intention, using the TDF as a guide, a descriptive analysis of the answers to the two open-ended questions was performed [22,46]. Codes for thematic analysis were developed using a deductive approach by two researchers (SGD, LGS) [47]. Then, two researchers (LGS, DAB) conducted the analysis and resolved any disagreements by consensus. If the disagreement remained, a third senior researcher was consulted (FL). Data were then deductively classified into TDF domains. The frequency of each barrier and facilitator was calculated by recording the number of times it was mentioned.
Triangulating qualitative and quantitative data.
This process evaluation aimed to propose practical theory-driven recommendations to improve the intervention. A triangulation of qualitative and quantitative data was therefore performed for a broader understanding of the psychosocial factors influencing the targeted behavior [48]. A comparison between the five psychosocial constructs assessed in the CPD-Reaction questionnaire and the TDF domains identified was conducted. Points of convergence and divergence between quantitative and qualitative data were identified and instances where both types of data provided insights into the same constructs were explored. Recommendations were formulated utilizing the COM-B model of behavior, which posits three criteria—capacity, opportunity, motivation—for the occurrence of a behavior [14,49]. These criteria and their subcategories were linked to the TDF domains, along with their associated barriers or facilitators (S1 Fig) [50]. The COM-B model further proposes nine intervention functions, aligned with TDF domains, that promote behavior change, namely education, persuasion, incentivization, coercion, training, restriction, environmental restructuring, modeling, and enablement [14,32,51,52]. By discerning which intervention functions corresponded to the results, behavior change techniques associated with the relevant functions were identified and selected to derive recommendations [14]. The proposed recommendations were reviewed by all authors.
Results
Practice and PHCP characteristics and flow diagram
Table 1 describes characteristics of randomized practices and demonstrates that the randomization of clinics in the parent cRT was adequate.
Fig 2 illustrates the flow of practices and PHCPs in this process evaluation study embedded within the Meta-LARC ACP cRT. Thirty-eight practices participated (19 in each arm), representing a participation rate of 84.4% (38/45). A total of 373 (69.7%) PHCPs fully completed the intention construct in CPD-Reaction. The sociodemographic characteristics of the participating PHCPs are detailed in Table 2.
Quantitative results
Behavioral intention.
Table 3 shows the scores for behavioral intention and its psychosocial determinants. On a scale of 1–7, the mean intention score was 5.97 (standard error (SE): 0.11) for the team-based arm (intervention) and 6.42 (SE: 0.13) for the individual clinician-focused arm (comparator). The difference in mean intention scores and 95% confidence interval (95% CI) between arms was -0.45 (-0.79 to -0.11), with a p-value of 0.01 (Table 4). There was no statistically significant difference in the other CPD-Reaction constructs between arms.
However, after adjusting for age, education, and profession, the difference in intention between the two groups was no longer statistically significant: 0.00 (95% CI: -0.29 to 0.30), with a p-value of 0.99 (Table 4).
Likelihood to engage in ACP and to use the SICG.
The mean difference in scores between the two arms for “How likely were you to engage patients in ACP before training?” was -1.09; p = 0.01, and for “How likely are you to engage patients in ACP after training?” was -0.75 (p = 0.01) (S1 Table). Mean difference between pre- and post-training likelihood to engage in ACP was 2.95 (p < 0.001) in the team-based arm and 2.60 (p < 0.001) in the individual clinician-focused arm (Table 5), the difference between arms was not statistically significant (p = 0.31). Likewise, mean difference between PHCPs’ self-reported likelihood to use the SICG in their practice after Part A and after Part B was 0.46 (p = 0.01) in the team-based arm and 0.50 (p = 0.02) in the individual clinician-focused arm, the difference between arms was also not statistically significant (p = 0.88) (Table 5).
Qualitative results
Fourteen facilitators were identified across seven of the 14 TDF domains, while 15 barriers were mapped onto nine of the TDF domains. The predominant barriers and facilitators were consistently associated with the environmental contexts and resources domain, with five barriers and six facilitators falling under this category. The second most prevalent facilitator was linked to the skills domain, while the second most common barrier was attributed to social influences domain. The most common theme among the barriers and facilitators in both arms was having enough time to have serious illness conversations. Having an organized workflow was an important facilitator in the team-based arm, but not in the individual clinician-focused arm. Similarly, communication problems with patients and PCHPs’ own discomfort with serious illness conversations were more frequent barriers in the team-based arm than in the clinician-based arm. in addition, the team-based approach seems to have added barriers related to interprofessional coordination and interprofessional communication that were not an issue in the individual clinician-focused arm (Table 6).
Triangulation
The psychosocial factors associated with CPD-Reaction matched the TDF domains of beliefs about capabilities, beliefs about consequences and social influences. Within the TDF, we identified eight additional psychosocial variables: knowledge, skills, reinforcements, behavioral regulation, environmental context and resources, emotion, social/professional role and identity, and goals. The most frequent barrier and facilitator in both arms (i.e., environmental context and resources) was classified in the “opportunity” component of COM-B. Concurrently, comparative analysis of the arms using the COM-B revealed a large number of the barriers for the team-based arm derived from the domains of opportunity (social influences) and motivation (emotion and social/professional role and identity). Triangulating the results led to comprehensive recommendations (S2 Table). These were grounded in behavior change techniques linked to functions such as education, modelling, enablement, environmental restructuring, training, and persuasion. Notably, environmental restructuring (n = 7) and education (n = 6) emerged as the most frequently recommended functions (S2 Table).
Discussion
This study is a mixed-methods theory-driven process evaluation embedded within a cRT that compared the impact on patients of a team-based SICP training approach for PHCPs with the impact of an individual clinician-focused SICP training approach, with post-intervention measures only. This process evaluation explored the causal mechanisms in operation relating to the PHCPs taking the training. Mechanisms that may have influenced the success or not of the intervention, and specifically PHCPs’ intention to have serious illness conversations with patients. Contrary to the main hypothesis, intention was lower in the team-based (intervention) arm than in the individual clinician-focused arm, but after adjusting for age, profession and education this difference was not statistically significant. Concurrently, the self-reported likelihood to engage in ACP before training (assessed retrospectively) was lower in the team-based arm than in the individual clinician-focused arm. Both arms assessed likelihood to engage patients in ACP and to use the SICG as higher after training than before, but the mean difference between the likelihood before and after the training was no higher in the team-based arm than in the individual clinician-focused arm. Environmental context and resources were the most common TDF domain that emerged from the barrier and facilitators questions. Lack of time was a major barrier in both arms, while organized workflow was a key potential facilitator in the team-based arm. Additionally, the team-based arm reported more emotional discomfort and concerns about patients’ reactions to serious illness conversations, as well as challenges coordinating multiple personnel and interprofessional communication that mapped onto the social influence TDF domain. The latter correlated with the lower mean on the team-based arm in the social influences CPD-Reaction construct. Triangulation of results using the COM-B framework showed most barriers and potential facilitators for both arms related to the opportunity domain. The COM-B model informed theory-based recommendations for improving future interprofessional SICP interventions. The recommendations primarily focused on environmental restructuring. These results lead us to the make the following observations:
First, contrary to the main hypothesis, a lower level of intention to have serious illness conversations with patients was found in the team-based arm than in the individual clinician-focused arm. However, the difference was no longer statistically significant when adjusted for age, profession and education. To the best of our knowledge, this is the first study to measure PHCPs’ intention to have serious illness conversations with patients using the CPD-Reaction, so results cannot be compared with those of other studies. Yet, related evidence on ACP programs tends to favor team-based approaches [13,53]. The differences in the professions found in each arm of the parent cRT may explain these findings. This process evaluation also suggested that certain participants in the team-based arm may not, in their practices or healthcare systems, have had the deontological responsibility or professional expectations to have this type of discussion. Qualitative data too suggested that more PHCPs in the team-based arm perceived serious illness conversations as outside their usual scope of practice. Practices vary between countries, states, and provinces regarding integrating palliative care practices into primary care, and especially regarding who should initiate serious illness conversations [11,54,55]. For some of the other primary care team members this may have been the first time they considered engaging in such conversations [56]. In a survey of interprofessional healthcare professionals in Colorado, most agreed that it was exclusively the physician’s role to discuss ACP with patients and family [57]. Future interprofessional trainings should take a more comprehensive approach, fostering support from managers or team leaders and using the expertise of more experienced PHCPs to encourage a sense of responsibility and confidence for holding serious illness conversations among those less experienced [54,55].
Second, PHCPs in both arms assessed their likelihood to engage in ACP before training as low, and the team-based arm rated it even lower than the individual clinician-focused arm. This could indicate that at study outset, baseline intention was low in the team-based arm, supporting the assumption that the lack of experience of certain team members in the team-based arm may have affected results. However, in both arms the training appeared to increase the likelihood of engaging in ACP and using the SICG, suggesting an overall positive impact of the SICP training sessions using either approach. A 2019 literature review on ACP in multiple settings also showed that training improved healthcare professionals’ comfort with discussing end-of-life decisions [58]. Similarly, equipping PHCPs with the tools to have end-of-life discussions, such as the SICG, proved to be positive, as shown by the likelihood to use the SICG. However, the qualitative data showed there is interest in other tools too, such as cheatsheets and EHR-embedded tools. These findings are corroborated by research in other settings, where tools supplementing the SICG (e.g., SIC reminders, behavioral nudges) emerged as crucial elements in facilitating implementation [59–62]. Future adaptations of the SICP to primary care should prioritize equipping PHCPs with a wider range of tools which can be easily embedded in their clinical practice routines.
Third, the most frequent barriers and facilitators raised by PHCPs were related to the environmental context. For example, time emerged as both a barrier and a facilitator in both arms. We expected that a team-based approach would reduce the time PHCPs need for discussions by promoting the sharing of tasks. It is possible that team-based SICP still lacks better methods of sharing tasks to be more effective. A study that adapted the SICP to other contexts indicated that a well-organized division of tasks and workflow is an essential element for implementation of team-based approaches [13,61]. In this study participants indicated that team-based training needed to help teams organize their workflow and clarify and recognize the role of each professional. The lack of effect observed in the team-based approach could indeed be attributed to the additional time and burden required for coordination and communication among team members, as seen in qualitative data. Adaptations of the team-based SICP should prioritize designating tasks across roles and building profession-specific skills. In addition, participants in the team-based arm expressed more discomfort about serious illness conversations and had more doubts about patients’ readiness for such discussions than participants in the individual clinician-focused arm. This finding supports our observation that despite the team-based training, there was a gap in certain participants’ understanding and preparedness for engaging in serious illness conversations. Further efforts are needed to address their discomfort, fears of negative reactions and concerns about patients’ preparedness.
Fourth, the CPD-reaction construct “social influences” exhibited the lowest mean score among the psychosocial factors influencing intention. This finding aligned with qualitative results from the team-based arm, where barriers such as interprofessional team coordination and communication difficulties were frequently mentioned and pertain to the domain of social influences within the TDF. “Social influence” is defined as the perception of approval or disapproval from significant peers or social referents regarding the adoption of a specific behavior [17]. One might expect PHCPs in the team-based training to report higher levels of social influence than those in the individual training group, but this was not the case. However, this finding aligns with other studies on the relationship between social influence and behavioral intention. A systematic review of 52 studies revealed that social influence had the lowest average score range among the four psychosocial factors influencing intention [63]. Nevertheless, Michie et al. established the social influence domain as a crucial target for health intervention planning [45], further supported by a systematic review analyzing 277 articles about behavior change techniques [64]. Improving social support measures in the intervention therefore remains essential, including providing more information about peer approval and facilitating understanding and trust through social comparison [14,15].
Lastly, data triangulation revealed that the majority of barriers and facilitators to behavior adoption, especially in the team-based arm, stemmed from the opportunity domain of the COM-B framework, i.e., external physical or social factors in the environment that affect adoption of the behavior. The framework itself posits that opportunity influences capacity (encompassing intention), which in turn influences behavior (S1 Fig) [14,49]. Therefore, there remain upstream barriers and potential facilitators in the Environmental context and resources domain, that limit behavior change. Consequently, in line with the COM-B framework, modifications targeting environmental changes could improve the success of the intervention [32]. Training alone cannot address these issues. Structural adjustments within the clinical setting are crucial for facilitating such conversations. Incorporating these changes in team-based SICP training could help. However, this study encompassed primary care settings in two countries whose healthcare systems vary greatly, making it very difficult, for example, to integrate a “one-size-fits-all” workflow into the training or incorporate an EHR reminder into all practices. This finding highlights a critical challenge in large-scale interventions: balancing the need to standardize content across diverse clinical contexts with the need to adapt to each practice’s specific needs [65–67]. Our findings, therefore, support the significance of environmental context modifications as effective tools for behavior change in future SICP implementations, wheter in research settings or healthcare policies.
Limitations
A true pre-intervention measure of baseline intention to engage in serious illness conversations with patients would have been preferable, but pragmatic considerations precluded this. It is difficult to show differences in comparative effectiveness studies in real-world clinical practice, particularly when interventions are similar [68]. Second, this study is a process evaluation embedded within a cRT in which randomization units were the clinical practices and not the PHCPs. Although characteristics of the clinical practices were balanced between arms, PHCPs’ characteristics were not balanced, as involving different professions in each group was part of the design. Therefore, meaningful comparison between arms for the main outcome, intention, could be limited. However, when controlled for these variables in the multivariable analysis the finding that intention increased more in the individual clinician-focused arm was no longer significant. This study analyzes team and clinician outcomes of a cRT for which the sample size and statistical power were calculated for patient outcomes, as these were the primary outcomes. Participation of PHCPs was encouraged, but the main focus was on patient recruitment and the number of practices, not the number of PHCPs. Thus, our results should be interpreted with caution, as their lack of significance may be secondary to a lack of statistical power. Future studies evaluating a team-based approach should therefore increase the number of team participants. Finally, the absence of semi-structured interviews or focus groups may have restricted the depth and complexity of qualitative data. This restriction stems from the time constraints imposed by real-world implementation projects. Despite this limitation, the open-ended question findings correlated with quantitative data and supported the study’s conclusions.
Conclusion
This theory-based process evaluation embedded within a cRT found that a team-based SICP training approach had no more impact than an individual clinician-focused training approach on increasing PHCPs’ intention to have serious illness conversations with patients. However, both training programs were useful for increasing PHCPs’ likelihood of engaging in ACP discussions and using the SICG guide. A team-based approach requires further focus on training all team members to feel empowered to have serious illness conversations and should address their fears of negative reactions and their concerns about patients’ preparedness. Our findings suggest that adaptations of the team-based SICP training approach should prioritize building more relevant skills and designating tasks across roles. Finally, our data triangulation suggested that in addition to training, structural environmental adjustments within clinical settings are crucial for facilitating serious illness conversations in teams in primary care. These findings will contribute to future interventions that improve the effectiveness of SICP training and support PHCPs in having more meaningful, high-quality and timely serious illness conversations with their patients.
Supporting Information
S1 Fig. The COM-B model and its correlation with the TDF domains.
https://doi.org/10.1371/journal.pone.0298994.s001
(DOCX)
S1 Table. ACP engagement scores before and after training.
https://doi.org/10.1371/journal.pone.0298994.s002
(DOCX)
S2 Table. Recommendations for improving the training based on barriers and facilitators, using the COM-B model, the Theoretical Domains Framework and the CPD-Reaction questionnaire.
https://doi.org/10.1371/journal.pone.0298994.s003
(DOCX)
Acknowledgments
We acknowledge the precious work of Louisa Blair for her editorial help with the manuscript and Stéphane Turcotte for his vital insights in the statistical analysis. We also thank the members of the Meta-LARC ACP cRT team for their involvement in this project.
COLLABORATORS META-LARC ACP TRIAL TEAM:
Angela K. Combe, Oregon Health & Science University
Annette M. Totten, Oregon Health & Science University
Barcey T. Levy, University of Iowa
Cat Halliwell, University of Colorado
David A. Dorr, Oregon Health & Science University
David Nowels, University of Colorado
Deb Constien, Patient partner
Deborah Dokken, Patient partner
Donald E. Nease, Jr., University of Colorado
Dr. B. Angeloe Burch Sr., Patient partner
Elizabeth Fernley, Oregon Health & Science University
France Légaré, Université Laval and VITAM - Centre de recherche en santé durable
Gail Drey, Patient partner
Gurnoor Kaur Brar, University of Toronto
Jacqueline D. Alikhaani, Patient partner
James Pantelas, Patient partner
Jean-Sébastien Paquette, Université Laval and VITAM - Centre de recherche en santé durable
Jeanette M. Daly, University of Iowa
Jessica E. Ma, Duke University
Jodi Lapidus, Oregon Health & Science University - Portland State University School of Public Health
Judy Katz, Patient partner
Kate Hanrahan, University of Iowa
Kathy Kastner, Patient partner
Katrina Ramsey, Oregon Health & Science University
Keith Provin, Patient partner
Kirsten Wentlandt, University Health Network
Kylie Lanman, Oregon Health & Science University
LeAnn C Michaels, Oregon Health & Science University
Lyle J. Fagnan, Oregon Health & Science University
Mary F. Henningfield, PhD, University of Wisconsin-Madison
Mary M. Minniti, Patient partner
Matthew Howard, Oregon Health & Science University
Megan Schmidt, University of Iowa
Meredith K. Warman, University of Colorado
Michelle Greiver, University of Toronto
Olga Petrova, Patient partner
Patrick M. Archambault, Université Laval, VITAM - Centre de recherche en santé durable, and Centre de recherche intégrée pour un système apprenant en santé et services sociaux de Chaudière-Appalaches
Peter Kim, University of Iowa
Rowena J. Dolor, Duke University
Sabrina Guay-Bélanger, VITAM - Centre de recherche en santé durable
Sarah Bumatay, Oregon Health & Science University
Sarina Schrager, University of Wisconsin-Madison
Sean Rice, Oregon Health & Science University-Portland State University School of Public Health
Sharon E. Straus, University of Toronto
Shelbey Hagen
Shigeko (Seiko) Izumi, Oregon Health & Science University
Souleymane Gadio, VITAM - Centre de recherche en santé durable
Suélène Georgina Dofara, VITAM - Centre de recherche en santé durable
Susan Lowe, Oregon Community Health Information Network - Columbia Gorge Health Council
Taryn Bogdewiecz, University of Colorado
References
- 1. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011;13(2):217–24. pmid:21842619
- 2. Wallis KA, Elley CR. Trial and error: challenges conducting pragmatic trials in general practice. Br J Gen Pract. 2022;72(715):54–5. pmid:35091399
- 3. Saarijärvi M, Wallin L, Bratt E-L. Process evaluation of complex cardiovascular interventions: How to interpret the results of my trial?. Eur J Cardiovasc Nurs. 2020;19(3):269–74. pmid:32054300
- 4. French C, Pinnock H, Forbes G, Skene I, Taylor SJC. Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?-a systematic review. Trials. 2020;21(1):916. pmid:33168067
- 5. Bernacki R, Hutchings M, Vick J, Smith G, Paladino J, Lipsitz S, et al. Development of the Serious Illness Care Program: a randomised controlled trial of a palliative care communication intervention. BMJ Open. 2015;5(10):e009032. pmid:26443662
- 6. Bernacki R, Paladino J, Neville BA, Hutchings M, Kavanagh J, Geerse OP, et al. Effect of the Serious Illness Care Program in Outpatient Oncology: A Cluster Randomized Clinical Trial. JAMA Intern Med. 2019;179(6):751–9. pmid:30870563
- 7. Totten AM, Fagnan LJ, Dorr D, Michaels LC, Izumi SS, Combe A, et al. Protocol for a Cluster Randomized Trial Comparing Team-Based to Clinician-Focused Implementation of Advance Care Planning in Primary Care. J Palliat Med. 2019;22(S1):82–9. pmid:31486729
- 8. Hui D, Hannon BL, Zimmermann C, Bruera E. Improving patient and caregiver outcomes in oncology: Team-based, timely, and targeted palliative care. CA Cancer J Clin. 2018;68(5):356–76. pmid:30277572
- 9. Fortin G, Dumont S. Goals of Care Conversations at the End-of-Life: Perceived Impact of an Interprofessional Training Session on Professional Practices. J Soc Work End Life Palliat Care. 2021;17(4):296–316. pmid:33722168
- 10. Brighton LJ, Selman LE, Gough N, Nadicksbernd JJ, Bristowe K, Millington-Sanders C, et al. “Difficult Conversations”: evaluation of multiprofessional training. BMJ Support Palliat Care. 2018;8(1):45–8. pmid:29118100
- 11. De Vleminck A, Houttekier D, Pardon K, Deschepper R, Van Audenhove C, Vander Stichele R, et al. Barriers and facilitators for general practitioners to engage in advance care planning: a systematic review. Scand J Prim Health Care. 2013;31(4):215–26. pmid:24299046
- 12. Rosa WE, Izumi S, Sullivan DR, Lakin J, Rosenberg AR, Creutzfeldt CJ, et al. Advance Care Planning in Serious Illness: A Narrative Review. J Pain Symptom Manage. 2023;65(1):e63–78. pmid:36028176
- 13. Lakin JR, Benotti E, Paladino J, Henrich N, Sanders J. Interprofessional Work in Serious Illness Communication in Primary Care: A Qualitative Study. J Palliat Med. 2019;22(7):751–63. pmid:30688549
- 14.
Michie S, Atkins L, West R. The behaviour change wheel. A guide to designing interventions. 2014.
- 15.
Godin G. Les comportements dans le domaine de la santé. 2013.
- 16. Godin G, Bélanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals’ intentions and behaviours: a systematic review of studies based on social cognitive theories. Implement Sci. 2008;336. pmid:18631386
- 17. Légaré F, Borduas F, Freitas A, Jacques A, Godin G, Luconi F, et al. Development of a simple 12-item theory-based instrument to assess the impact of continuing professional development on clinical behavioral intentions. PLoS One. 2014;9(3):e91013. pmid:24643173
- 18. Bakwa Kanyinga F, Gogovor A, Dofara SG, Gadio S, Tremblay M, Daniel SJ, et al. Evaluating the impact of continuing professional development courses on physician behavioral intention: a pre-post study with follow-up at six months. BMC Med Educ. 2023;23(1):629. pmid:37661265
- 19. Hrisos S, Eccles MP, Francis JJ, Dickinson HO, Kaner EFS, Beyer F, et al. Are there valid proxy measures of clinical behaviour? A systematic review. Implement Sci. 2009;437. pmid:19575790
- 20. Goulet F, Hudon E, Gagnon R, Gauvin E, Lemire F, Arsenault I. Effects of continuing professional development on clinical performance: results of a study involving family practitioners in Quebec. Can Fam Physician. 2013;59(5):518–25. pmid:23673591
- 21. Grimshaw JM, Presseau J, Tetroe J, Eccles MP, Francis JJ, Godin G, et al. Looking inside the black box: results of a theory-based process evaluation exploring the results of a randomized controlled trial of printed educational messages to increase primary care physicians’ diabetic retinopathy referrals [Trial registration number ISRCTN72772651]. Implement Sci. 2014;986. pmid:25098442
- 22. Atkins L, Francis J, Islam R, O’Connor D, Patey A, Ivers N, et al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12(1):77. pmid:28637486
- 23. Davis R, Campbell R, Hildon Z, Hobbs L, Michie S. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev. 2015;9(3):323–44. pmid:25104107
- 24. Adekpedjou R, Haesebaert J, Stacey D, Brière N, Freitas A, Rivest L-P, et al. Variations in factors associated with healthcare providers’ intention to engage in interprofessional shared decision making in home care: results of two cross-sectional surveys. BMC Health Serv Res. 2020;20(1):203. pmid:32164669
- 25. Légaré F, Stacey D, Brière N, Fraser K, Desroches S, Dumont S, et al. Healthcare providers’ intentions to engage in an interprofessional approach to shared decision-making in home care programs: a mixed methods study. J Interprof Care. 2013;27(3):214–22. pmid:23394265
- 26. Lockwood I, Walker RM, Latimer S, Chaboyer W, Cooke M, Gillespie BM. Process evaluations undertaken alongside randomised controlled trials in the hospital setting: A scoping review. Contemp Clin Trials Commun. 2022;26:100894. pmid:36684693
- 27. Liu H, Mohammed A, Shanthosh J, News M, Laba T-L, Hackett ML, et al. Process evaluations of primary care interventions addressing chronic disease: a systematic review. BMJ Open. 2019;9(8):e025127. pmid:31391188
- 28. Creswell JW, Clark VP. Mixed methods research. SAGE Publications.; 2011.
- 29. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350h1258. pmid:25791983
- 30. Légaré F, Borduas F, Freitas A, Turcotte S. User manual—The Continuing Professional Development (CPD) Reaction Questionnaire. [Online] Laval: Canada[Cited on december 20, 2023]. 2015. Available from: https://wwwdecisionchairefmedulavalca/outil-en/601ac886dc64b246f77888c7,
- 31. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;737. pmid:22530986
- 32. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;642. pmid:21513547
- 33. Campbell MK, Piaggio G, Elbourne DR, Altman DG, CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345e5661. pmid:22951546
- 34. O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8. pmid:18416914
- 35. Izumi SS, Caron D, Guay-Bélanger S, Archambault P, Michaels L, Heinlein J, et al. Development and Evaluation of Serious Illness Conversation Training for Interprofessional Primary Care Teams. J Palliat Med. 2023;26(9):1198–206. pmid:37040304
- 36. Asmaou Bouba D, Gomes Souza L, Dofara SG, Guay-Bélanger S, Gadio S, Mochcovitch D, et al. Long-Term Effects of Individual-Focused and Team-Based Training on Health Professionals’ Intention to Have Serious Illness Conversations: A Cluster Randomised Trial. J CME. 2024;13(1):2420475. pmid:39502858
- 37. Adekpedjou R, Stacey D, Brière N, Freitas A, Garvelink MM, Dogba MJ, et al. Engaging Caregivers in Health-Related Housing Decisions for Older Adults With Cognitive Impairment: A Cluster Randomized Trial. Gerontologist. 2020;60(5):947–57. pmid:31095318
- 38. Stacey D, Légaré F, Pouliot S, Kryworuchko J, Dunn S. Shared decision making models to inform an interprofessional perspective on decision making: a theory analysis. Patient Educ Couns. 2010;80(2):164–72. pmid:19945813
- 39. Dogba MJ, Menear M, Stacey D, Brière N, Légaré F. The Evolution of an Interprofessional Shared Decision-Making Research Program: Reflective Case Study of an Emerging Paradigm. Int J Integr Care. 2016;16(3):4. pmid:28435417
- 40. Légaré F, Freitas A, Turcotte S, Borduas F, Jacques A, Luconi F, et al. Responsiveness of a simple tool for assessing change in behavioral intention after continuing professional development activities. PLoS One. 2017;12(5):e0176678. pmid:28459836
- 41. Sudore RL, Hickman SE, Walling AM. Controversies About Advance Care Planning. JAMA. 2022;327(7):685. pmid:35166807
- 42. Little TD, Chang R, Gorrall BK, Waggenspack L, Fukuda E, Allen PJ, et al. The retrospective pretest–posttest design redux: On its validity as an alternative to traditional pretest–posttest measurement. International Journal of Behavioral Development. 2019;44(2):175–83.
- 43. Adisso EL, Borde V, Saint-Hilaire M-È, Robitaille H, Archambault P, Blais J, et al. Can patients be trained to expect shared decision making in clinical consultations? Feasibility study of a public library program to raise patient awareness. PLoS One. 2018;13(12):e0208449. pmid:30540833
- 44. Stacey D, Brière N, Robitaille H, Fraser K, Desroches S, Légaré F. A systematic process for creating and appraising clinical vignettes to illustrate interprofessional shared decision making. J Interprof Care. 2014;28(5):453–9. pmid:24766619
- 45. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33. pmid:15692000
- 46. Kim H, Sefcik JS, Bradway C. Characteristics of Qualitative Descriptive Studies: A Systematic Review. Res Nurs Health. 2017;40(1):23–42. pmid:27686751
- 47. Nowell L, Norris J, White D, Moules N. Thematic analysis: Striving to meet the trustworthiness criteria. International journal of qualitative methods. 2017;16(1):1609406917733847.
- 48. O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341c4587. pmid:20851841
- 49. West R, Michie S. A brief introduction to the COM-B Model of behaviour and the PRIME Theory of motivation. 2020.
- 50. McDonagh LK, Saunders JM, Cassell J, Curtis T, Bastaki H, Hartney T, et al. Application of the COM-B model to barriers and facilitators to chlamydia testing in general practice for young people and primary care practitioners: a systematic review. Implement Sci. 2018;13(1):130. pmid:30348165
- 51. Deom Tardif A, Gogovor A, Guay-Bélanger S, Audet D, Parent N, Gaudreau A, et al. Integration of sex and gender in a continuing professional development course on diabetes and depression: a mixed methods feasibility study. BMJ Open. 2022;12(4):e050890. pmid:35459660
- 52. Agbadjé TT, Menear M, Dugas M, Gagnon M-P, Rahimi SA, Robitaille H, et al. Pregnant women’s views on how to promote the use of a decision aid for Down syndrome prenatal screening: a theory-informed qualitative study. BMC Health Serv Res. 2018;18(1):434. pmid:29884169
- 53. Beck BM, Coda C, Gerges J, Allen J, Agarwal A, Mutchie HL, et al. Advance Care Planning: An Interprofessional Approach to Resident Education. J Am Geriatr Soc. 2020;68(11):E66–8. pmid:33020932
- 54. Dixon J, Knapp M. Whose job? The staffing of advance care planning support in twelve international healthcare organizations: a qualitative interview study. BMC Palliat Care. 2018;17(1):78. pmid:29793469
- 55. Buchman S, Evans JM, Mackinnon M, Gradin S, Wright FC. Bridging silos: Delivering integrated care to patients with cancer in Ontario, Canada. Psychooncology. 2018;27(12):2673–6. pmid:30187984
- 56. Patel MI, Khateeb S, Coker T. Lay Health Workers’ Perspectives on Delivery of Advance Care Planning and Symptom Screening Among Adults With Cancer: A Qualitative Study. American Journal of Hospice & Palliative Medicine. 2021;38(10):1202-11. PubMed pmid:152006792. Language: English. Entry Date: 20210824. Revision Date: 20210824. Publication Type: Article.
- 57. Arnett K, Sudore RL, Nowels D, Feng CX, Levy CR, Lum HD. Advance Care Planning: Understanding Clinical Routines and Experiences of Interprofessional Team Members in Diverse Health Care Settings. Am J Hosp Palliat Care. 2017;34(10):946–53. pmid:27599724
- 58. Chan CWH, Ng NHY, Chan HYL, Wong MMH, Chow KM. A systematic review of the effects of advance care planning facilitators training programs. BMC Health Serv Res. 2019;19(1):362. pmid:31174530
- 59. Manz CR, Zhang Y, Chen K, Long Q, Small DS, Evans CN, et al. Long-term Effect of Machine Learning-Triggered Behavioral Nudges on Serious Illness Conversations and End-of-Life Outcomes Among Patients With Cancer: A Randomized Clinical Trial. JAMA Oncol. 2023;9(3):414–8. pmid:36633868
- 60. Karim S, Levine O, Simon J. The Serious Illness Care Program in Oncology: Evidence, Real-World Implementation and Ongoing Barriers. Curr Oncol. 2022;29(3):1527–36. pmid:35323328
- 61. Paladino J, Sanders J, Kilpatrick L, Prabhakar R, Kumar P, O’Connor N. Serious illness care programme—contextual factors and implementation strategies: a qualitative study. BMJ Supportive & Palliative Care. 2022.
- 62. Kumar P, Paladino J, Gabriel P, Ferrell W, Jones J, Schuchter L. The Serious Illness Care Program: Implementing a Key Element of High-Quality Oncology Care. NEJM Catalyst Innovations in Care Delivery. 2023;4(2):CAT.22.0309.
- 63. Ayivi-Vinz G, Bakwa Kanyinga F, Bergeron L, Décary S, Adisso ÉL, Zomahoun HTV, et al. Use of the CPD-REACTION Questionnaire to Evaluate Continuing Professional Development Activities for Health Professionals: Systematic Review. JMIR Med Educ. 2022;8(2):e36948. pmid:35318188
- 64. Carey RN, Connell LE, Johnston M, Rothman AJ, de Bruin M, Kelly MP, et al. Behavior Change Techniques and Their Mechanisms of Action: A Synthesis of Links Described in Published Intervention Literature. Ann Behav Med. 2019;53(8):693–707. pmid:30304386
- 65. Aarons GA, Sklar M, Mustanski B, Benbow N, Brown CH. “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implement Sci. 2017;12(1):111. pmid:28877746
- 66. Power J, Gilmore B, Vallières F, Toomey E, Mannan H, McAuliffe E. Adapting health interventions for local fit when scaling-up: a realist review protocol. BMJ Open. 2019;9(1):e022084. pmid:30679286
- 67. Plaisance A, Witteman HO, LeBlanc A, Kryworuchko J, Heyland DK, Ebell MH, et al. Development of a decision aid for cardiopulmonary resuscitation and invasive mechanical ventilation in the intensive care unit employing user-centered design and a wiki platform for rapid prototyping. PLoS One. 2018;13(2):e0191844. pmid:29447297
- 68. Dang A, Kaur K. Comparative effectiveness research and its utility in In-clinic practice. Perspect Clin Res. 2016;7(1):9–14. pmid:26955571