https://orcid.org/0000-0002-5037-0523
Figures
Abstract
Background
We aimed to investigate the performance of clinician prediction of survival (CPS) and the association between CPS and the prognostic confidence of clinicians in ambulatory medical oncology outpatients.
Methods
Eight medical oncologists estimated the expected survival of their patients in a prospective cohort study. They were asked to provide a temporal type of CPS in weeks, together with their level of confidence for each prediction (0–100%). We analyzed the accuracy of CPS, the association between CPS and the prognostic confidence, and the characteristics of patients showing inaccurate CPS.
Results
A total of 200 advanced cancer patients were enrolled and the median overall survival was 7.6 months. CPS was accurate in 67 (33.5%) patients, underestimated in 87 (43.5%), and overestimated in 46 (23.0%). The overall accuracy of CPS for 12-week, 24-week, 36-week, and 48-week survival was 83.0%, 72.0%, 75.5%, and 74.0%, respectively. The specificity was highest for 12-week survival (90.2%) and the sensitivity was highest for 48-week survival (96.1%). The sensitivity of 12-week CPS was 51.4% and the area under the receiver operating characteristic (AUROC) curve was 0.79 (95% confidence interval [CI] 0.71–0.87). The prognostic confidence of clinicians was not significantly associated with the accuracy of prediction (P = 0.359). Patients with overestimated survival had significantly poorer global health status and physical/role/emotional functioning in the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30). Additionally, they showed significantly higher levels of fatigue, nausea/vomiting, pain, dyspnea, and loss of appetite.
Conclusion
The overall accuracy of CPS in predicting 12-week to 48-week survival was high in medical oncology outpatients. However the sensitivity of 12-week CPS was low and prognostic confidence was not associated with the accuracy of CPS. Patients with overestimated CPS showed poorer quality of life and higher symptom burden.
Citation: Kim YJ, Yoon SJ, Suh S-Y, Hiratsuka Y, Kang B, Lee SW, et al. (2022) Performance of clinician prediction of survival in oncology outpatients with advanced cancer. PLoS ONE 17(4): e0267467. https://doi.org/10.1371/journal.pone.0267467
Editor: Julian C. Hong, University of California San Francisco, UNITED STATES
Received: June 26, 2021; Accepted: April 8, 2022; Published: April 21, 2022
Copyright: © 2022 Kim et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (Ministry of Science, Informatics, Communication and Technology; No. 2015R1C1A2A01053357). Dr SYS received this fund.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Accurate prediction of survival in advanced cancer patients is crucial for various reasons. Better prognostication may allow patients to make better decisions regarding their medical care near the end of life [1, 2]. Thus physicians consistently try to estimate the remaining survival of their patients using clinician prediction of survival (CPS) or validated prognostic scores [1, 3].
CPS is the most common method to predict survival in advanced cancer patients [4, 5]. Although there is a great deal of controversy regarding the accuracy of CPS, it is considered an independent prognostic factor, along with other known prognostic factors. CPS has a significant correlation with actual survival in terminally ill cancer patients, with a correlation coefficient of 0.2 to 0.65 [1]. However, it is well known that physicians tend to overestimate survival [6, 7]. Physicians can be overly optimistic because accurate prognostication itself is difficult and they want to preserve the hopes of patients. One study reported that experienced physicians and physicians who do not have long-term relationships with the patient may be better at predicting prognosis [8].
In addition, CPS is known to be more accurate for short-term predictions than for long-term predictions [1]. The variability of actual survival increases with longer CPS [6]. Indeed, the vast majority of studies on CPS were performed in terminally ill cancer patients with an expected survival of less than 2 months [5]. There have been scarce studies on the accuracy of CPS in ambulatory advanced cancer patients with prognoses of months to years [9, 10]. One large prospective cohort study from Japan investigated the accuracy of CPS in 2,036 patients in various groups, including hospital palliative care teams, palliative care units, home palliative care services, and active chemotherapy [10]. Although this study reported a high correlation coefficient of 0.70 between CPS and actual survival in patients receiving active chemotherapy, the median survival was only 34 days (95% confidence interval [CI] 29–39) in these patients.
In this study, we aimed to investigate the performance of CPS and the association between prognostic confidence and accuracy of CPS in ambulatory medical oncology outpatients. We hypothesized that CPS would not be accurate in predicting survival times longer than 12 weeks in advanced cancer patients. Prognostic confidence was expected to increase the accuracy of CPS.
Materials and methods
Patients
This study was part of a prospective cohort study aimed at developing a prognostic model in medical oncology outpatients [11]. The prognostic model predicts 3-month survival to facilitate palliative care referral at least 3 months before death. We enrolled advanced cancer patients treated at a comprehensive cancer center of a university hospital from March 2016 to January 2019. Patients were eligible if they had a diagnosis of advanced cancer, when their oncologists estimated their survival to be less than a year and were 18 years or older. We defined advanced cancer as a metastatic or recurrent disease or progressive locally advanced disease not amenable to curative treatment. Patients were excluded if they had hematologic malignancies, were expected to survive less than a month, and were not able to communicate. Written informed consent was obtained from each patient before enrollment. The protocol was approved by the Institutional Review Board (IRB) of Seoul National University Bundang Hospital (IRB number: B-1601/332-302).
Data collection
Medical oncologists were asked to estimate CPS with a temporal question, “How many weeks will this patient have?” They were also asked to provide their level of confidence for each prediction in percentage (0% [not at all] to 100% [full confidence]). The age, sex, years of clinical experience, and palliative care training experience of each oncologist were collected. The patients were interviewed face to face by a clinical research nurse after enrollment. Patient performance status was assessed by the Eastern Cooperative Oncology Group (ECOG) performance status and Karnofsky Performance Scale (KPS). All patients were asked to complete the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30) version 3.0 [12]. The EORTC QLQ-C30 consists of multi-item scales including functional scales (physical, role, emotional, cognitive, and social), symptom scales (fatigue, pain, nausea/vomiting, pain, dyspnea, insomnia, appetite, constipation, diarrhea, and financial difficulties), and a global health status/quality of life (QoL) scale. All of the scales and single-item measures range in score from 0 to 100. High scores for the functional scale and the global health status/QoL represent a high/healthy level of functioning and high QoL. On the contrary, high scores in the symptom scale mean high symptom burden. We obtained demographic data, clinical information, and laboratory test results from the electronic medical records of patients.
Statistical analysis
Descriptive analyses were performed to summarize the baseline characteristics of the patients. The CPS was considered accurate if it fell within ±33.3% of the actual survival as previously described in other studies [8, 13, 14]. The median survival and 95% CI were calculated using the Kaplan-Meier method. Survival time was observed from enrollment into this study until death. Live cases at the end of the study were dealt with as censored data. The comparison of survival outcomes was performed using log-rank tests. To assess the discriminatory ability of the CPS, we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), overall accuracy, and area under the receiver operating characteristic (AUROC) curve. The comparison of EORTC QLQ-C30 scores among different CPS groups (accurate, underestimated, and overestimated) was performed using one-way analysis of variance (ANOVA). All tests were two-sided, and P-values <0.05 were considered significant. All analyses were performed using IBM SPSS Statistics for Windows, version 26.0 (IBM Corp., Armonk, NY, USA).
Results
A total of 200 advanced cancer patients were enrolled in this study. With a median follow-up period of 7.7 months (range, 0.5–36.8), the median overall survival was 7.6 months (95% CI 1.3–33.3), and 159 patients had died at the time of analysis. The predictions of 8 oncologists were distributed as S1 Fig. CPS was accurate in 67 (33.5%) patients, underestimated in 87 (43.5%), and overestimated in 46 (23.0%). A total of 64 patients (32%) were alive after a year. Baseline patient characteristics according to prognostic accuracy are summarized in Table 1.
Eight medical oncologists participated in this study. The characteristics of the clinicians are shown in Table 2. Half of the clinicians were female, and most of them had more than 10 years of clinical experience. Three clinicians (37.5%) had palliative care training and dedicated palliative care experience.
The overall accuracy of CPS for predicting 12-week, 24-week, 36-week, and 48-week survival was 83.0%, 72.0%, 75.5%, and 74.0%, respectively (Table 3). The specificity was highest for 12-week survival (90.2%) and the sensitivity was highest for 48-week survival (96.1%). The sensitivity of 12-week CPS was 51.4% and the AUROC curve was 0.79 (95% CI 0.71–0.87). The AUROC curves of CPS for 24-week, 36-week, and 48-week survival were 0.81 (95% CI 0.75–0.87), 0.83 (95% CI 0.77–0.88), and 0.82 (95% CI 0.76–0.88), respectively.
The prognostic confidence of clinicians was not significantly associated with the accuracy of CPS (P = 0.359) (Fig 1). CPS was accurate in 35.1% of clinicians, with a 40–60% confidence level and 31.4% of clinicians with a 70–100% confidence level. The correlation coefficients between CPS and actual survival time were 0.60 (P<0.001) and 0.62 (P<0.001) in clinicians with a 40–60% confidence level and a 70–100% confidence level, respectively.
aP-value was 0.359 by chi-square test. bAccurate estimation was defined as clinician prediction of survival being within ±33.3% of actual survival. cClinicians’ prognostic confidence was measured by a percent scale from 0% (not at all) to 100% (full confidence).
Patients with accurate CPS, underestimated CPS, and overestimated CPS had median survival times of 6.6 months (95% CI 5.1–8.2), 19.7 months (95% CI 13.6–25.8), and 2.1 months (95% CI 1.5–2.7), respectively (Fig 2). The differences in survival in the three groups were all significant (accurate CPS vs. underestimated CPS: P<0.001; accurate CPS vs. overestimated CPS: P<0.001) by the log-rank test.
Patients with accurate clinician prediction of survival (CPS), underestimated CPS, and overestimated CPS had median survival times of 6.6 months (95% CI 5.1–8.2), 19.7 months (95% CI 13.6–25.8), and 2.1 months (95% CI 1.5–2.7), respectively. The differences in survival in the three groups were all significant (accurate CPS vs. underestimated CPS: P<0.001; accurate CPS vs. overestimated CPS: P<0.001) by the log-rank test.
Table 4 depicts the association between the accuracy of the CPS and EORTC QLQ-C30 scores. Patients with overestimated survival had significantly poorer global health status/QoL scores (P = 0.004) and poorer physical (P = 0.001), role (P = 0.003), and emotional (P = 0.001) functioning scores. In addition, they showed significantly higher levels of fatigue (P = 0.014), nausea/vomiting (P = 0.001), pain (P<0.001), dyspnea (P<0.001), and loss of appetite (P<0.001) compared with the other two groups of patients.
Discussion
We found that the overall accuracy of CPS in predicting 12-week to 48-week survival in ambulatory medical oncology outpatients was higher than expected in this preplanned analysis of a prospective cohort study. The overall accuracy of 12-week CPS was 83.0%, and the specificity was 90.2%. However, the sensitivity of 12-week CPS was limited to 51.4%. This low sensitivity may show the limitation of using CPS as a sole prognostic factor to predict 12-week survival. To improve the accuracy of CPS, clinicians may need to combine other prognostic factors or scores [1]. We are currently developing a prognostic model to predict 3-month survival in oncology outpatients.
In terms of the accuracy of temporal CPS (defined as ±33.3% of the actual survival), 67 patients (33.5%) were accurately predicted in our study. The percentage of accurate estimates is reported to be 23–78% in various studies [15]. However, direct comparison of the results is not possible because of heterogeneous categories and definitions used in different studies. The result of our study is similar to previous studies that used the same definition as ours. In one study, the accuracy of temporal CPS was 32% for physicians and 18% for nurses [14]. In another study, 35% of survival estimations were classified as accurate for palliative care physicians in 2,036 patients [10]. Another study that included only CPS by medical oncologists as our study reported that CPS was accurate in 29% of patients [9].
Unexpectedly, underestimated CPS was more frequent than overestimated CPS in our study. It is well known that CPS is overly optimistic in terminally ill patients [6, 8]. However, the nature of CPS in ambulatory medical oncology outpatients has not been clearly clarified. A recent systematic review reported that clinicians in 5 studies underestimated patient survival out of 15 studies [7]. In a study on CPS by medical oncologists, the median estimated survival time was 11 months, while the median actual survival time was 9 months [9]. The study excluded patients with an expected survival of less than 3 months, and 61% of patients lived longer than the estimates of oncologists. According to a study that compared the CPS of medical professionals in advanced cancer patients with brain metastases, neurosurgeons and radiation oncologists were generally more optimistic [16]. Interestingly, only medical oncologists underestimated survival. The reasons for the higher frequency of underestimation in medical oncologists are unknown. However, because they play a major role in the care planning of metastatic cancer patients with limited survival, the negative consequences of overestimation would be particularly distressing to medical oncologists. Additionally, a consistent rise in patients responding to newer chemotherapeutic agents may have contributed to longer survival, which lies behind the underestimation. In our study, 32% of patients lived longer than a year, despite the inclusion criteria of expected survival less than a year. Interestingly, one study reported that CPS tended to be optimistic in patients with short-lived survival (≤6 months) and pessimistic in patients with longer survival (≥9 months) [17]. There were no significant differences in the direction of survival prediction between physicians, nurses, and radiation therapists in that study.
We hypothesized that the prognostic confidence of clinicians would have a positive association with the accuracy of CPS. Unlike our assumption, there was no significant association between the prognostic confidence and accuracy of CPS in this study. Clinicians with a lower level of prognostic confidence (40–60%) showed accuracy similar to clinicians with a higher level of prognostic confidence (70–100%). The correlation coefficient between CPS and actual survival time in the lower confidence group was 0.60 (P<0.001) and that of the higher confidence group was 0.62 (P<0.001). Thus, the correlation coefficients were similarly high in the two groups. Previous studies have reported that the CPS of experienced physicians was more accurate than that of less experienced physicians, but this may not mean that experienced physicians have more prognostic confidence [8, 18]. In our study, all medical oncologists were highly experienced, and their prognostic confidence might have been influenced by modest personalities. Further studies are needed to investigate the association between prognostic confidence and the accuracy of CPS.
In our study, patients with overestimated CPS showed an absolutely and significantly short median survival of 2.1 months compared with the other two groups of patients. Furthermore, these patients had significantly poorer global health/QoL, poorer functioning sores, and higher symptom scores according to the EORTC QLQ-C30. It is not surprising that patients near the end-of-life suffer from poor QoL and high symptom burdens. However, physicians should not forget that poor QoL and high symptom burden are indicators of poor prognosis. As described in previous studies, physical symptoms such as nausea/vomiting, dyspnea, and fatigue may reflect the consequences of cancer cachexia and terminal diseases [19]. It is not easy for physicians to recognize wide range of symptoms instantly in busy clinical setting. Thus there is a possibility that patients’ high symptom burden might have been overlooked at the time of prediction in our study. In this regard, we suggest routine symptom screening using symptom assessment tools in everyday practice. We believe proactive symptom screening in medical oncology outpatients can improve the accuracy of CPS.
The strengths of our study may include the prospective design, investigation of CPS in medical oncology outpatients with a relatively longer survival compared with terminally ill patients receiving hospice care, and demonstration of high overall accuracy of CPS in this population. However, our study was conducted in a single tertiary cancer center in Korea, which may limit the general applicability of the study results. Another limitation may include the small number of participating medical oncologists (n = 8). Finally, the main eligibility criterion was a life expectancy of less than a year. This might have biased the CPS towards underestimation.
Conclusions
In conclusion, the survival estimates of medical oncologists were not highly accurate (33.5%) in this study, but the overall accuracy of CPS in predicting 12-week to 48-week survival was high (72–83%). Prognostic confidence was not associated with the accuracy of CPS. Finally, patients with overestimated CPS showed poorer QoL, higher symptom burden, and significantly poorer survival.
Supporting information
S1 Fig. Distribution of clinician prediction of survival (CPS, weeks).
aFrequency is expressed as percentage (%).
https://doi.org/10.1371/journal.pone.0267467.s001
(TIF)
Acknowledgments
We thank the participating patients and clinical research nurses (Jin Suk Kim and Esther Jeon) for supporting this study.
References
- 1. Maltoni M, Caraceni A, Brunelli C, Broeckaert B, Christakis N, Eychmueller S, et al. Prognostic factors in advanced cancer patients: evidence-based clinical recommendations–A study by the Steering Committee of the European Association for Palliative Care. J Clin Oncol. 2005;23(25):6240–6248. pmid:16135490
- 2. Butow PN, Clayton JM, Epstein RM. Prognostic awareness in adult oncology and palliative care. J Clin Oncol. 2020;38(9):877–884. pmid:32023158
- 3. Stone PC, Lund S. Predicting prognosis in patients with advanced cancer. Ann Oncol. 2006;18(6):971–976. pmid:17043092
- 4. Glare PA, Sinclair CT. Palliative medicine review: prognostication. J Palliat Med. 2008;11(1):84–103. pmid:18370898
- 5. Hui D, Paiva CE, Del Fabbro EG, Steer C, Naberhuis J, van de Wetering M, et al. Prognostication in advanced cancer: update and directions for future research. Support Care Cancer 2019;27:1973–1984. pmid:30863893
- 6. Glare P, Virik K, Jones M, Hudson M, Eychmuller S, Simes J, et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ. 2003;327(7408):195–198. pmid:12881260
- 7. Cheon S, Agarwal A, Popovic M, Milakovic M, Lam M, Fu W, et al. The accuracy of clinicians’ predictions of survival in advanced cancer: a review. Ann Palliat Med. 2016;5(1):22–29. pmid:26841812
- 8. Christakis NA, Lamont EB. Extent and determinants of error in doctors’ prognoses in terminally ill patients: prospective cohort study. BMJ. 2000;320(7233):469–472. pmid:10678857
- 9. Kiely BE, Martin AJ, Tattersall MH, Nowak AK, Goldstein D, Wilcken NR, et al. The median informs the message: Accuracy of individualized scenarios for survival time based on oncologists’ estimates. J Clin Oncol. 2013;31:3565–3571. pmid:24002504
- 10. Amano K, Maeda I, Shimoyama S, Shinjo T, Shirayama H, Yamada T, et al. The accuracy of physicians’ clinical predictions of survival in patients with advanced cancer. J Pain Symptom Manage. 2015;50(2):139–146. pmid:25847848
- 11. Kim YJ, Hiratsuka Y, Suh SY, Kang B, Lee SW, Ahn HY, et al. A Prognostic Model to Facilitate Palliative Care Referral in Oncology Outpatients. Cancer Res Treat 2021 Jul 13. Online ahead of print. pmid:34265891
- 12. Yun YH, Park YS, Lee ES, Bang SM, Heo DS, Park SY, et al. Validation of the Korean version of the EORTC QLQ-C30. Qual Life Res. 2004;13(4):863–868. pmid:15129896
- 13. Llobera J, Esteva M, Rifa J, Benito E, Terrasa J, Rojas C, et al. Terminal cancer. Duration and prediction of survival time. Eur J Cancer. 2000;36(16):2036–2043. pmid:11044639
- 14. Hui D, Kilgore K, Nguyen L, Hall S, Fajardo J, Cox-Miller TP, et al. The accuracy of probabilistic versus temporal clinician prediction of survival for patients with advanced cancer: A preliminary report. Oncologist. 2011;16:1642–1648. pmid:21976316
- 15. White N, Reid F, Harris A, Harries P, Stone P. A systematic review of predictions of survival in palliative care: How accurate are clinicians and who are experts? PLoS One. 2016;11(8):e0161407. pmid:27560380
- 16. Kondziolka D, Parry PV, Lunsford LD, Kano H, Flickinger JC, Rakfal S, et al. The accuracy of predicting survival in individual patients with cancer. J Neurosurg. 2014;120(1):24–30. pmid:24160479
- 17. Chow E, Hruby G, Harris K, Enright K, Sinclair E, Chan G. Determining the accuracy of health care professionals in predicting the survival of patients with advanced metastatic cancer. J Pain Manag. 2010;3:73–79.
- 18. Tavares T, Oliveira M, Gonçalves J, Trocado V, Perpetuo J, Azevedo A, et al. Predicting prognosis in patients with advanced cancer: A prospective study. Palliat Med. 2018;32(2):413–416. pmid:28488922
- 19. Vigano A, Donaldson N, Higginson IJ, Bruera E, Mahmud S, Suarez-Almazor M. Quality of life and survival prediction in terminal cancer patients. Cancer. 2004;101:1090–1098. pmid:15329920