Introduction

Most research on uses of artificial intelligence (AI) in medicine has asked “what can AI be used for” rather than “what should AI be used for?” [1] Use-cases for AI should be based on current clinical challenges for which physicians value AI support [2–4]. A thorough evaluation of such clinical challenges and their hypothetical solutions, prior to the development of solutions, is a key pillar of design thinking (e.g., a problem solving approach that focuses on solutions) and human-centric design (e.g., problem solving technique that puts people – users, observes, and those impacted – at the center) and can facilitate successful implementation of AI tools in healthcare settings [5,6].

Prediction of survival duration (e.g., survival estimation) for patients with serious illness (e.g., incurable and progressive medical conditions) through the automated synthesis of individual patient health information is one AI use-case [7]. AI-derived survival estimates could address prognostic uncertainty, a known challenge for family physicians aiming to adequately prepare their patients for progressive illnesses [8,9]. However, prior evaluation of family physicians’ perspectives on the applications of AI in their practice has largely assessed opinions of AI in general [10]. Little research exits to inform whether survival estimation tools would be welcome in family medicine.

Engaging clinical end users prior to the deployment of AI tools can improve their clinical relevance, acceptability, and safety [11–13]. Newly licenced physicians (e.g., medical residents) are the ultimate end users of the novel AI tools currently transforming the landscape of healthcare. Evaluation of their perspectives can also identify, and potentially address, concerns for automation bias [14–16]. Thus, our objective was to elicit the perceived utility and concerns of family medicine residents regarding a hypothetical AI tool for survival estimation among patients with serious illness. Such approach identifies contextualized opportunities, barriers, and risks of AI for illness prognostication in the primary care setting.

Methods

Results

Overall, participants (N = 18) were receptive to integrating AI in the management of serious illness, though identified some limitations of using AI for estimating survival in serious illness. Summarizing these perspectives, we identified four themes: (1) improving patient care with AI, (2) AI with a grain of salt, (3) patient-driven use of AI, and (4) augmenting, not replacing family physicians. Across the four themes, we constructed 17 parent codes and 55 child codes (S3 File).

Discussion

Using principles of design thinking, [29,30] we evaluated the perceived clinical utility of a hypothetical solution (AI-derived survival estimates) to support the care of patients with serious illness prior to the development and implementation of such solutions. Family medicine residents believed AI-derived survival estimates could support patients with serious illness, though they identified pertinent risks and challenges. Together, these thoughts are summarized by four themes around AI survival estimation: (1) improving patient care, (2) grain of salt, (3) patient-driven use, and (4) augmenting, not replacing family physicians. Our findings highlight how and when AI survival estimation can augment prognosis conversations in primary care, and can thus, guide development of AI survival estimation tools.

Family physicians are known to be largely positive about AI, with attitudes hinging on the context of adoption [31]. Perceptions of AI are more favorable for automating administrative tasks than for making clinical decisions [32]. We extend on prior general knowledge by exploring in-depth perspectives on a specific use-case and clinical context for AI in primary care. With our approach, we identify the following considerations for developing and deploying AI models for survival estimation of serious illness in family medicine:

AI derived survival estimation would be valuable in certain contexts and for certain patients. Participants believed having accurate survival estimation is clinically important as it provides an objective starting point for prognosis conversations, potentially improving patients’ illness understanding at diagnosis and the alignment of care with patient goals. Indeed, participants noted how survival estimation may be most important in initial serious illness discussions (e.g., diagnosis), throughout management (e.g., when new complications arise), and in the final weeks of life. The implication for model development is that such models should not run automatically but be triggered by clinicians when appropriate in a patients care. Such recommendation also aligns with environmental consideration for AI and with participants’ views that AI derived survival estimation is most valuable for patients where prognostication support is required (e.g., difficult to anticipate clinical trajectories). Likewise, such a model would likely be of use if its design facility access to both specialists and family physicians, enabling their collaborative management of patients with serious illness.

Survival estimation has ethical implications and use of such tool must be patient- and family-oriented. Patients may not want to know the time they have left. Survival estimation would also need to be highly accurate as it may have serious impacts on care choices; in such clinical scenarios, the use of AI as augmented intelligence (e.g., integrating AI with clinical insights) is likely the most appropriate route. Considerations of liability in the context of augmented prognostication need to be considered thoughtfully, particularly in instances of shared liability with developers or hospitals. However, participants of our study proposed that survival estimation could theoretically alter the outcome, and thus, were concerned that accuracy, performance bias, and dataset shift over time of such a model could never truly be accurately assessed. To address these concerns, predicting ranges of time alive (vs. exact number of weeks) and prompting physicians to inquire whether patients desire the information before communicating it with patients may ensure the model is more clinically meaningful.

Prognostications of serious illness involves more than the number of weeks left to live. Care for patients with serious illness involves ensuring illness understanding, aligning care plans with patient goals, addressing patients’ current symptoms, and liaising needed psychosocial supports. Indeed, having a numeric survival estimate in serious illness communication is much less important than family physicians believe [33]. Palliative care research suggests patients value time alive and at home more than time alive [34]. Ensuring relevant and clinically meaningful outcomes will positively influence the value of an AI tool and its perception as complementing (vs. replacing) clinical judgement [35]. To address these concerns, AI prediction tools could instead integrate multiple patient-centered outcomes (e.g., predict response to therapy, functional outcomes, or complications) on one model interface, or alternatively be used to improve patient’s illness understanding (i.e., as a chatbot or to generate tailored descriptions of expected trajectories).

Successful AI deployment requires education and role modelling By elucidating perspectives on an AI tool with a specific use-case (i.e., survival duration prediction), we identified AI educational gaps needed prior to implementation. Lack of formalized education and familiarity has been reported as a barrier to AI acceptance in family medicine [35]. Family medicine residents were not confident in their understanding of AI and confused generative and non-generative AI. Understanding these concepts will be critical for safely leveraging AI in clinical care as they allow physicians to correctly identify relevant limitations such as algorithmic biases and hallucinations. Further, participants desired role modelling of integrating novel AI technologies into clinical workflow. These findings support the Diffusion Innovation Theory, which outlines the trajectory of achieving widespread acceptance of a novel idea within an established population or social system. Our findings specifically highlight trialability and observability for successful AI deployment [36]. Overcoming well-described deployment barriers [37] should also involve strategies to highlight perspectives of early AI adopters within primary care.

To date, no general AI mortality or prognosis estimation models have been approved or deployed in primary care in Canada. Prior AI survival estimation models have shown strong performance but have predominantly been designed for oncology, [38] and are disease specific – such as for lung cancer, [39] skin cancer, [40] and breast cancer [41]. These models have primarily focused on prediction of survival at fixed time points (i.e., 2-year mortality), with few models estimating survival time directly. We were interested in exploring the use of a general prognostic model by generalist physicians, analogous to the use of the Palliative Performance Scale [42] or other general prognostic tools. This is particularly compelling to study because the information gained from a prognostic model might be more informative to a generalist clinician who has less expert knowledge of the anticipated trajectory of a given illness. Further, taking a hypothetical non-disease-specific model, allowed us to explore general concepts about the use of such models that are applicable to disease-specific and non-disease-specific models alike, whereas anchoring our study to a specific prognostic model would have narrowed the applicability of our findings to the specific strengths and weaknesses of that model.

Another consideration – and source of diversity across prognostication models – is how patients’ medical information is extracted and inputted into the model. Many require physicians to extract the medically relevant information. For example, an oncology survival estimation model leveraged the unstructured text on a doctor’s note to predict survival [43]. Indeed, AI model performance is closely linked to data (quality, comprehensiveness, format) provided [44]. Such reliance on data structure and quality is one reason why the performance of an AI model will often drops when deployed in a new clinical setting [45]. As noted by our participants, models that do not automatically extract the clinically relevant information from electronic health records have limited value add, as challenges in prognosticating are identifying, documenting, and synthesizing all medically relevant information.

Integrating principles of human-centric (or solutions-oriented) design and eliciting stakeholder perspectives prior to prototyping can facilitate successful implementation and impact in healthcare settings. A salient clinical example of such an approach is PODS (Patient Oriented Discharge Summaries) – the development of which involved stakeholder feedback prior to its design, implementation, and evaluation [5,6]. Indeed, this type of approach is largely missing from the medical AI landscape, with most work to date focusing on what AI can do, versus what AI should do. For such reason, we identified a clinical challenge and evaluated the clinical utility of hypothetical AI solutions, identifying considerations for its clinical acceptability and utility.

Strengths and limitations

Strengths of our study include the use of a flexible semi-structured interview approach and two interviewing rounds, which enabled participants to guide the discussion and the research team to follow up on ideas not anticipated in the first round. Regarding its limitations, our exploratory research included only a subset of family medicine residents in Canada with a sample size of 18. We did not collect sociodemographic data of these respondents. However, the purpose of the study, built from a constructivist paradigm, was to describe meaningful stories of conceptual interactions between AI and mortality that might resonate with researcher and policy-makers to begin a new scholarly conversation [46]. Although our findings might be conceptually transferable (i.e., they apply to the experiences of individuals outside of our sample), they are not intended to be statistically generalizable [47]. Our findings may not reflect all physicians currently established in their practice owing to generational differences in physician training and comfort with AI prognostication. To ensure we provided meaningful and useful results that might be transferable to similar contexts, we used Nowell’s criteria of trustworthiness (credibility, transferability, dependability, and confirmability) [20,21,48,49]. Additionally, our findings provide the perspectives of family medicine residents prior to the deployment of AI survival estimation, and thus, may not be reflective of those post-deployment. Future research should evaluate the perspectives of all individuals potentially affected by such tool (e.g., patients, bioethicists, and hospital decision makers) prior to deployment as well as when such tools have been clinically deployed.

Conclusion

Our qualitative study identified that family medicine residents were receptive to AI-derived survival estimates for patients with serious illness, particularly given the need to deliver expert advice over a broad range of disease states in primary care. However, AI tools were viewed as supporting tools to achieve improved accuracy with survival estimation, rather than as providing definite survival estimation calculations, particularly given the need to integrate elements of the clinical gestalt, visual and sensory information (e.g., patients’ work of breathing), and patients’ goals of care. Additionally, caring for serious illnesses involves more than survival estimation. Pertinent clinical management challenges that may additionally be addressed with AI include building illness understanding and anticipating functional trajectories and quality of life outcomes. Future prognostication models for serious illness should consider use of patient-centered outcomes and varying model outcomes based on prediction timepoints and patient needs. To successfully deploy these technologies, clinical acceptance should be fostered through role modelling from early AI adopters.

Supporting information

Acknowledgments

The authors would like to acknowledge the Temerty Centre for Artificial Intelligence Research and Education in Medicine (T-CAIREM). The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding sources; no endorsement is intended or should be inferred.