Figures
Abstract
Introduction
Anxiety is one of the most common psychological symptoms in patients in a palliative care situation. This study aims to develop a predictive model for anxiety using data from the standard documentation routine.
Methods
Data sets of palliative care patients collected by the German quality management benchmarking system called Hospice and Palliative Care Evaluation (HOPE) from 2007 to 2011 were randomly divided into a training set containing two-thirds of the data and a test set with the remaining one-third. We dichotomized anxiety levels, proxy rated by medical staff using the validated HOPE Symptom and Problem Checklist, into two groups with no or mild anxiety versus moderate or severe anxiety. Using the training set, a multivariable logistic regression model was developed by backward stepwise selection. Predictive accuracy was evaluated by the area under the receiver operating characteristic curve (AUC) based on the test set.
Results
An analysis of 9924 data sets suggests a predictive model for anxiety in patients receiving palliative care which contains gender, age, ECOG, living situation, pain, nausea, dyspnea, loss of appetite, tiredness, need for assistance with activities of daily living, problems with organization of care, medication with sedatives/anxiolytics, antidepressants, antihypertensive drugs, laxatives, and antibiotics. It results in a fair predictive value (AUC = 0.72).
Conclusions
Routinely collected data providing individual-, disease- and therapy-related information contain valuable information that is useful for the prediction of anxiety risks in patients receiving palliative care. These findings could thus be advantageous for providing appropriate support for patients in palliative care settings and should receive special attention in future research.
Citation: Hofmann S, Hess S, Klein C, Lindena G, Radbruch L, Ostgathe C (2017) Patients in palliative care—Development of a predictive model for anxiety using routine data. PLoS ONE 12(8): e0179415. https://doi.org/10.1371/journal.pone.0179415
Editor: Tifei Yuan, Nanjing Normal University, CHINA
Received: August 5, 2016; Accepted: May 29, 2017; Published: August 3, 2017
Copyright: © 2017 Hofmann et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are in the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Psychological symptoms in patients with terminal diseases are very frequent [1–5]. Anxiety is one of the most common psychological symptoms in patients in palliative care settings [2]. In a meta-analysis of 24 studies on mood disorders in patients with terminal diseases, a pooled prevalence for anxiety disorders of 9.8% (Range: 6.8%– 13.2%) was reported [2]. This high prevalence and the evidence that psychological symptoms can lower quality of life should lead to increased sensitivity toward psychological burdens among clinicians who care for such patients [6–8].
In addition to the description of the prevalence of symptom burden in patients suffering from a terminal disease, many studies focus on interrelations of symptoms [6]. For anxiety, several associations with other symptoms in patients with advanced cancer or in palliative care settings can be reported from previous research: Delgado-Guay et al. [7] and Oechsle et al. [8] describe associations of anxiety with physical symptom burden. In particular, there are findings of interrelations with pain [7, 9, 10], fatigue [7], sleep disturbance [6], nausea [7, 9], dyspnea [7, 9, 11], and cardiac arrhythmias [9]. Although there are findings indicating no relationship between anxiety and socio-demographic or personal data [12], later analyses showed associations between anxiety and younger age [6] as well as being female [6, 13, 14]. Further findings, such as associations of anxiety with use of medications [13], acceptance [15], and belief in an afterlife [13], are reported elsewhere [11, 16, 17].
Due to the small sample sizes and the heterogeneity of many studies, it is still challenging to find reliable predictors for anxiety in terminally ill patients [6].
Objectives of the study
The objectives of the study are (i) to describe sample characteristics of routine data collected in palliative care settings in Germany, (ii) to explore differences between data sets of patients with high anxiety scores and those with low anxiety scores, (iii) to discover possible predictors of anxiety, and (iv) to find and test a predictive model based on routine data.
Materials and methods
Study material and data sets
For this study we used anonymized data of adult patients from palliative care services in Germany. Data were collected using the Hospice and Palliative Care Evaluation (HOPE), a long-term, web-based quality management benchmarking system for palliative care, which is well-established in Germany. HOPE was developed as a core data set by experts from a range of professions from the German Association for Palliative Medicine, the German Hospice and Palliative Care Association, and the German Association for Cancer [18–20]. It is composed of different modules and can be used by inpatient or outpatient hospice and palliative care services. To collect information about personal, disease-related and therapy-related data, HOPE includes a core data set (see Supporting Information S1, S2 and S3 Figs), which is assessed by medical staff (e.g. physicians, nurses). To assess the individual symptom burden of patients with terminal diseases, it contains a validated symptom and problem checklist (HOPE-SP-CL) [21]. Using a four-point grading scale (none, mild, moderate, severe), HOPE-SP-CL assesses individual physical (pain, nausea, vomiting, dyspnea, constipation, weakness, loss of appetite, tiredness), psychological (feeling depressed, anxiety, tension, disorientation/confusion), social (organization of care, overburdening of family) and nursing (wound care, assistance with activities of daily living [ADLs]) burden at least once on admission to the service and once on the day of discharge or death of the patient. HOPE is not used to diagnose mental disorders. Rather, it is used to assess the current suffering of patients. In particular, mental symptoms such as anxiety and feeling depressed can be very painful if they occur in the course of a serious, fatal illness, even if the criteria of a mental disorder are not met. Data sets are extracted from the standardized documentation systems of the participating services each year for a period of three months (March 15 –June 14) to maintain a continuous benchmarking process. Data sets are centrally merged and processed and are then fed back to the services in benchmark format through a predefined and programmed interface. For this study we used data sets of patients in different inpatient and outpatient institutions (palliative care units, hospital support teams, oncology wards, hospices, palliative home care services) in Germany from 2007 to 2011 [20].
Ethics and data protection
No additional data collection was necessary and we did not make any changes to the care or treatment of patients to perform this study. The data sets used were completely depersonalized. The institutional ethics review board of Friedrich-Alexander-Universität Erlangen-Nürnberg was formally approached and did not have any ethical or legal concerns about this study.
Data analysis and model development
IBM SPSS Statistics 21.0 for Windows (IBM Corporation, Armonk, NY) was used for statistical evaluation.
In order to characterize the sample, descriptive analyses were performed. Symptom scores of anxiety were dichotomized into none or mild intensity versus moderate or severe intensity [22, 23].
The data set was divided into a training set consisting of two thirds of the data and a test set consisting of one third of the data. On the basis of the training set, a broad range of variables (e.g. demographic, medical and psychosocial items) of the routine data were used to search for bivariate interrelations with the command variable. Afterwards, variables showing a significant association (using a significance level of p < .25) with the dichotomized ratings of anxiety were considered for the development of a multivariable logistic regression model, which was generated using a backward stepwise selection. Due to the high intercorrelations with the target variable, other items related to psychological burden such as feeling depressed, tension and disorientation/confusion were not included in the analysis.
On the basis of the test set, which comprises the remaining one third of data, predictive accuracy of the final model was evaluated by the area under the receiver operating characteristic curve (AUC) corresponding with the c statistic. An AUC of .80 or higher indicates good sensitivity and specificity, whereas fair sensitivity and specificity is described between AUCs of .70 and .80. Positive and negative predictive values were also calculated [24].
Results
Sample characteristics
Overall, 12124 data sets of patients transferred to HOPE by the participating inpatient or outpatient palliative care services during the period under review in the years 2007 to 2011 were used in this study (2007 n = 3184, 2008 n = 2148, 2009 n = 2293, 2010 n = 2444, 2011 n = 2055). The following data sets were excluded from further analyses: 368 data sets due to missing date of admission to the palliative care service, 866 data sets because date of admission to the palliative care service was outside the annual three month period of data collection, 956 data sets without any assessment for anxiety and 10 data sets due to a documented age under 18. Fig 1 shows this selection of the data sets. Comparisons between participants (n = 9924) and non-participants (n = 2200) yielded no significant differences in age, gender, performance status (ECOG) or palliative care services at time of admission.
Table 1 shows the characteristics of the study sample of 9924 patients who were assessed with HOPE at time of admission to a palliative care service (see Supporting Information S1 File). Just over half were female (51.9%) and the average age was 68.8 (Range: 19.3–109.4; SD = 12.6). The majority of patients showed a low performance status (33.7% ECOG 3, 37.5% ECOG 4) at the time of admission. For 36.2% of the population under review, no level of nursing care support had been approved by the health insurance providers. (Note: According to the German long-term care legislation in the period under review, patients may receive reimbursement for nursing care. In order for this to occur, they have to apply for a level of nursing care support. Reimbursement is granted for one of four levels of nursing care support (level 1: necessary support for a minimum of two activities in fields of personal hygiene, eating and mobility (at least once a day), housekeeping (at least several times a week) with an overall duration of care at least 90 min a day; level 2: necessary support at least three times a day with an overall (average) duration of 3 h a day; level 3: ongoing need for support (including at night), average duration of at least 5 h a day; level 3+: special cases of hardship with an excessive high need for support, and need for support for artificial home ventilation). Compared to other variables, level of nursing care support showed by far the highest rate of missing values (14.7%) and was therefore excluded from further calculations. Most of the patients had a cancer diagnosis (89.2%) and lived with family members (66.4%) or alone (21.7%). Advance directives were formulated by approximately one third (32.2%) of the study population. Two thirds were patients from a specialized inpatient palliative care unit (60.4%); 24.1% were supported by outpatient services. 38.3% of patients included in this study were assessed with moderate or severe anxiety levels.
Development and evaluation of the predictive model
For 35 variables assessed by HOPE, bivariate interrelations with anxiety were calculated on the basis of the training set. The results of these analyses are shown in Table 2. The following variables showed no significant association with anxiety and were therefore not included in further calculations: type of service, advance directives, type of disease, brain metastases, medication with opioids (WHO step 2), and medication with diuretics. Twenty-nine variables showing a significant association with anxiety were put into a backward stepwise logistic regression model as candidate predictor variables. Table 3 shows the final model comprising those variables, whose influence on the target variable persists when controlling for this wide range of variables. The following 15 variables remained in the final logistic regression model: gender, age, ECOG, living situation, pain, nausea, dyspnea, loss of appetite, tiredness, assistance with ADL, organization of care, medication with sedatives/anxiolytics, medication with antidepressants, cardiac medication, and medication with antibiotics.
The model was evaluated by the test set, which consists of one third of the whole data set and showed a fair predictive value (AUC = 0.72). Fig 2 shows the ROC curve. The optimal cut-off score of our model, which maximizes sensitivity and specificity, was found to be ≥ 0.3714649. Sensitivity was 0.68 and specificity 0.66. Under consideration of the prevalence of anxiety in the test set, the negative predictive value was 0.70 and the positive predictive value was 0.61.
AUC (95% CI) 0.723 (0.704, 0.742). Optimal cut-off score > 0.3714649.
We also considered the impact of missing data. In most variables there is only a small proportion of missing values of 5% or less. Due to a conspicuously high level of missing data (14.4%), the item level of nursing care support was excluded from further analyses. Nevertheless, the final model shows 10.7% missing values. Since we could not find any significant difference between data sets excluded from further analyses due to missing values and those included in the final model, we assumed missing data were most likely caused by random rather than by systematic bias.
Discussion
To the best of our knowledge the present study is the first in which such a broad range of different variables from routine data was used to find a predictive model for anxiety in patients with terminal diseases on the basis of a data set of this size which was collected nationwide.
Regarding predictors of anxiety in patients in palliative care settings, the present study yielded several results. Like previous findings, it confirms interrelations between physical symptom burden and anxiety [7–10]. For all aspects of physical burden assessed by HOPE, calculations showed highly significant bivariate associations with anxiety. Therefore, higher physical symptom burden is more likely to be accompanied by higher anxiety levels. However, despite these significant interrelations, only symptom burden from pain, nausea, dyspnea, loss of appetite, and tiredness remained in the final multivariable model after backward stepwise selection. This indicates that the influences of those variables showing a significant bivariate connection with anxiety but having been removed by backward stepwise selection were exceeded by the stronger influences of remaining variables.
In particular, our findings concerning the predictive value of dyspnea are also consistent with previous research [11]. In recent years, the linkage of anxiety and dyspnea in terminally ill patients has drawn more attention to the treatment of dyspnea with anxiolytics and with psychological interventions such as relaxation/imagery with breathing retraining in addition to treatment with opioids [25–27]. Furthermore, this connection emphasizes the importance of awareness while supporting and treating patients who are suffering from physical symptoms.
Although there are different indications concerning a linkage between age and anxiety [6, 13], our findings affirm that younger age is more likely to be connected to higher anxiety levels [6]. Additionally, our data also showed a correlation of gender of patients in palliative care settings with anxiety and thus confirmed previous findings of higher anxiety levels in women with advanced cancer or terminal diseases than in men [6, 13, 14].
Furthermore, our data underpins some findings of a study by Kolva et al. [13]. A difference in anxiety levels between inpatients and outpatients was reported in this study but this variable showed no significant interrelation in a multiple regression analysis. This is also evident in comparison with the present data analysis, where no significant influence of inpatient and outpatient palliative care services on patients’ anxiety levels could be found. However, Kolva et al. linked the influence that seems to be exerted by this variable to remaining survival time [13], which was not included in our analyses. In contrast to the findings of a previous study [28], we did not identify higher ratings of anxiety levels in specialized palliative care services than in other wards. Not surprisingly, medication with anxiolytics and antidepressants are predictors of anxiety and can also be found in the final multiple regression model described by Kolva [13].
Concerning medication, our calculations also indicate effects of antibiotic and cardiac usage. In particular, lower anxiety levels were reported in patients receiving cardiacs, whereas medication with antibiotics is likely to be accompanied by higher anxiety levels. This seems to be a suitable finding which proves a positive association between cardiac arrhythmias and high anxiety levels [9]. Those indications could be linked to the psychophysiological model of panic attacks as a vicious circle postulated by Pauli et al. [29]. In this study, differences in cardiac perceptions, anxiety levels related to those perceptions and heart period after those perceptions were investigated in patients with and without panic attacks. Although patients with panic attacks reported somewhat more cardiac perception than patients without any anxiety disorder, this difference was not significant. However, a highly significant difference could be found between these two groups regarding anxiety related to cardiac perceptions. Whereas in healthy controls cardiac perceptions did not elicit anxiety, subjects with panic attacks reported anxiety related to cardiac perceptions. Additionally, the study showed a clear positive association between levels of anxiety elicited by cardiac perception and faster heartbeat after those perceptions [29]. Since patients with a history of anxiety disorders may be more affected by cardiac perceptions, this suggests that those patients may benefit from medication with cardiacs in particular. For this reason, the assessment of psychiatric history could have a limited advantage. However, especially in patients with a shortened life span, the potential benefits must be balanced against the additional burden.
HOPE is already used as a quality management benchmarking system by many palliative care services in Germany [18–20]. Based on this assessment, we found a multivariable regression model to predict the risk of moderate or severe anxiety levels in terminally ill patients with a fair value. We therefore recommend supplementing clinical evaluation. The model can be integrated into existing digital documentation routines without much additional programming effort. It can facilitate early prediction of anxiety, which could be particularly important, for example, in the increasing efforts toward early integration of palliative care in the treatment of serious illnesses [30–32]. Above all, no additional effort is required from patients. This is consistent with requirements that diagnostic instruments, particularly for patients with terminal diseases, should be not very stressful and time-consuming [33, 34]. Further evaluations of the screening performance, possibilities to complement the model to enhance its predictive accuracy, and the benefits in clinical practice are advisable.
Study limitations
Our results are based on a comfortably large sample size, but some limitations have to be taken into account when interpreting the results. The prevalence of anxiety in our population is very high compared to other findings [2]. This could be promoted for example by the given items in the tick boxes or perhaps it could be enhanced by an overestimation of anxiety levels by staff compared to self-assessments. However, on the other hand there are findings of similar or even higher prevalence of anxiety [35–37]. Therefore, it is doubtful whether there is an overrepresentation of anxiety in the present study. More importantly, due to the study design no statements about causalities are permitted. Even though we assumed different variables as possible predictors of anxiety, they were assessed simultaneously at the time of admission to palliative care service, and there could be a common underlying cause (such as disease progression) rather than a causal relationship between the variables. The vicious circle of anxiety and cardiac perception postulated by Pauli et al. [29] indicates a complex interrelation between physical symptoms and anxiety. Further research is needed here to explore causal and mutual or reciprocal influences.
HOPE, with its symptom and problem checklist, is a validated routine proxy assessment tool used by physicians, nurses and other healthcare personnel. When interpreting results it must be considered that patient reported outcome measures (PROM) would be the gold standard [38] and differences in clinicians’ and patients’ judgments of symptoms are evident [39–41]. However, due to weakness and/or cognitive impairment, for many patients in a palliative care situation it is impossible to use PROMs and thus a proxy assessment can safeguard data documented from all patients [42, 43]. There is evidence of satisfactory convergent validity for HOPE-SP-CL in investigations on patients’ self-assessment tools such as the MIDOS_2 or POS [21, 44]. In our service it is routine to use both.
Results are calculated only on the basis of data sets for which all variables have been completed. The proportion of missing values should be considered in the interpretation, particularly in the multivariable logistic model. No systematic errors were detected here. The reasons for missing data are unknown, and one hypothesis would be that missing data results because different occupational groups and team members bear responsibility for assessing the data. Additionally, no immediate beneficial effect results for healthcare personnel or patients, which could lead to a lack of motivation to complete the questionnaire conscientiously.
HOPE contains details on further psychological aspects such as feeling depressed, tension, and disorientation/confusion. Those psychological aspects were excluded from further analyses for two reasons. Firstly, the psychological items showed high intercorrelations. Secondly, those aspects as they are used in HOPE are not meant to be interpreted as psychiatric diagnoses. In particular, the variable feeling depressed, is not to be compared with the diagnosis of the mood disorder depression.
In addition, although we controlled for a wide range of variables there is no information about the influence of variables which are not included and by nature underrepresented in our analyses, such as a history of psychiatric disorders, acceptance of prognosis or religious beliefs [13, 15].
The comparability with other studies may be affected by the diversity of possible predictor variables determined by using HOPE. Furthermore, it must be considered that the present model has only a fair predictive value. Further research is needed to expand the range of and adapt possible predictor variables to develop models with higher predictive values and to further study interrelations with anxiety in patients in palliative care settings.
Conclusions
The present study describes the development of a predictive model for anxiety in palliative care patients using data from the standard documentation routine in Germany. We conclude that this approach can improve the assessment of anxiety in patients in a palliative care situation.
This model could enhance the screening process but will not replace further clinical evaluations. It will be indispensable to perform individual interviews to provide appropriate support for anxious patients in palliative care settings, but this model could sensitize care providers for a plurality of interacting predictors and thus also improve early detection of possible risk factors of anxiety and anxiety itself.
Supporting information
S1 Fig. HOPE 2011 Basisbogen page 1.
HOPE Basisbogen 2011 page 1.
https://doi.org/10.1371/journal.pone.0179415.s001
(TIF)
S2 Fig. HOPE 2011 Basisbogen page 2.
HOPE Basisbogen 2011 page 2.
https://doi.org/10.1371/journal.pone.0179415.s002
(TIF)
S3 Fig. HOPE 2011 core data set.
HOPE core data set questionnaire.
https://doi.org/10.1371/journal.pone.0179415.s003
(TIF)
S1 File. HOPE core data set 2007–2011.sav.
Core data set of the study participants assessed with HOPE.
https://doi.org/10.1371/journal.pone.0179415.s004
(SAV)
Acknowledgments
HOPE is supported by the German Palliative Care Association, the German Hospice Association, and the German Cancer Association. While the original development of HOPE was supported by an unrestricted grant from Mundipharma GmbH, Limburg, the current costs are borne by the participating institutions and the German Palliative Care Association. The authors received no specific funding for this work.
The present study was performed in fulfillment of the requirements for the first author to obtain the doctoral degree “Dr. rer. biol. hum.”.
Author Contributions
- Conceptualization: S. Hofmann CO.
- Data curation: S. Hofmann.
- Formal analysis: S. Hofmann.
- Investigation: S. Hofmann.
- Methodology: S. Hofmann.
- Project administration: S. Hofmann.
- Resources: GL.
- Supervision: CO LR.
- Validation: S. Hofmann.
- Visualization: S. Hofmann CK.
- Writing – original draft: S. Hofmann.
- Writing – review & editing: S. Hofmann S. Hess CK GL LR CO.
References
- 1. Caplette-Gingras A, Savard J. Depression in women with metastatic breast cancer: a review of the literature. Palliat Support Care. 2008;6(4):377–87. pmid:19006593.
- 2. Mitchell AJ, Chan M, Bhatti H, Halton M, Grassi L, Johansen C, et al. Prevalence of depression, anxiety, and adjustment disorder in oncological, haematological, and palliative-care settings: a meta-analysis of 94 interview-based studies. Lancet Oncol. 2011;12(2):160–74. pmid:21251875.
- 3. Walker J, Holm Hansen C, Martin P, Sawhney A, Thekkumpurath P, Beale C, et al. Prevalence of depression in adults with cancer: a systematic review. Ann Oncol. 2013;24(4):895–900. pmid:23175625.
- 4. Hotopf M, Chidgey J, Addington-Hall J, Ly KL. Depression in advanced disease: a systematic review Part 1. Prevalence and case finding. Palliat Med. 2002;16(2):81–97. pmid:11969152.
- 5. Janberidze E, Hjermstad MJ, Haugen DF, Sigurdardottir KR, Lohre ET, Lie HC, et al. How are patient populations characterized in studies investigating depression in advanced cancer? Results from a systematic literature review. J Pain Symptom Manage. 2014;48(4):678–98. pmid:24681108.
- 6. Gilbertson-White S, Aouizerat BE, Jahan T, Miaskowski C. A review of the literature on multiple symptoms, their predictors, and associated outcomes in patients with advanced cancer. Palliat Support Care. 2011;9(1):81–102. pmid:21352621.
- 7. Delgado-Guay M, Parsons HA, Li Z, Palmer JL, Bruera E. Symptom distress in advanced cancer patients with anxiety and depression in the palliative care setting. Support Care Cancer. 2009;17(5):573–9. pmid:19005686.
- 8. Oechsle K, Wais MC, Vehling S, Bokemeyer C, Mehnert A. Relationship between symptom burden, distress, and sense of dignity in terminally ill cancer patients. J Pain Symptom Manage. 2014;48(3):313–21. pmid:24766742.
- 9. Stoklosa J, Patterson K, Rosielle D, Arnold RM. Anxiety in palliative care: causes and diagnosis #186. J Palliat Med. 2011;14(10):1173–4. pmid:22004149.
- 10. Wilson KG, Chochinov HM, Allard P, Chary S, Gagnon PR, Macmillan K, et al. Prevalence and correlates of pain in the Canadian National Palliative Care Survey. Pain Res Manag. 2009;14(5):365–70. pmid:19862371;
- 11. Mystakidou K, Tsilika E, Parpa E, Katsouda E, Galanos A, Vlahos L. Assessment of anxiety and depression in advanced cancer patients and their relationship with quality of life. Qual Life Res. 2005;14(8):1825–33. pmid:16155770.
- 12. Pinder KL, Ramirez AJ, Black ME, Richards MA, Gregory WM, Rubens RD. Psychiatric disorder in patients with advanced breast cancer: prevalence and associated factors. Eur J Cancer. 1993;29A(4):524–7. pmid:8435204.
- 13. Kolva E, Rosenfeld B, Pessin H, Breitbart W, Brescia R. Anxiety in terminally ill cancer patients. J Pain Symptom Manage. 2011;42(5):691–701. pmid:21565460;
- 14. Radbruch L, Nauck F, Ostgathe C, Elsner F, Bausewein C, Fuchs M, et al. What are the problems in palliative care? Results from a representative survey. Support Care Cancer. 2003;11(7):442–51. pmid:12774219.
- 15. Thompson GN, Chochinov HM, Wilson KG, McPherson CJ, Chary S, O'Shea FM, et al. Prognostic acceptance and the well-being of patients receiving palliative care for cancer. J Clin Oncol. 2009;27(34):5757–62. pmid:19826116.
- 16. Mystakidou K, Tsilika E, Parpa E, Gogou P, Theodorakis P, Vlahos L. Self-efficacy beliefs and levels of anxiety in advanced cancer patients. Eur J Cancer Care. 2010;19(2):205–11. pmid:19659666.
- 17. Wilson KG, Chochinov HM, Skirko MG, Allard P, Chary S, Gagnon PR, et al. Depression and anxiety disorders in palliative cancer care. J Pain Symptom Manage. 2007;33(2):118–29. pmid:17280918.
- 18. Lindena G, Nauck F, Bausewein C, Neuwohner K, Heine O, Schulenberg D, et al. Quality assurance in palliative medicine—results of the core documentation of 1999–2002. Z Arztliche Fortbild Qualitatssich. 2005;99(9):555–65. pmid:16398197.
- 19. Radbruch L, Ostgathe C, Elsner F, Nauck F, Bausewein C, Fuchs M, et al. What is the profile of palliative care in Germany. Results of a representative survey. Schmerz. 2004;18(3):179–88. pmid:15221422.
- 20.
HOPE. Hospiz- und Palliativ-Erhebung. EXPOSE. [updated 2014 Feb 27; cited 2014 Sep 27]. https://www.hope-clara.de/download/HOPE2014Expose.pdf. German
- 21. Stiel S, Pollok A, Elsner F, Lindena G, Ostgathe C, Nauck F, et al. Validation of the Symptom and Problem Checklist of the German Hospice and Palliative Care Evaluation (HOPE). J Pain Symptom Manage. 2012;43(3):593–605. pmid:22071164.
- 22. Barbera L, Seow H, Howell D, Sutradhar R, Earle C, Liu Y, et al. Symptom burden and performance status in a population-based cohort of ambulatory cancer patients. Cancer. 2010;116(24):5767–76. pmid:21136579.
- 23. Ostgathe C, Alt-Epping B, Golla H, Gaertner J, Lindena G, Radbruch L, et al. Non-cancer patients in specialized palliative care in Germany: what are the problems? Palliat Med. 2011;25(2):148–52. pmid:20937612.
- 24. Henderson AR. Assessing test accuracy and its clinical consequences: a primer for receiver operating characteristic curve analysis. Ann Clin Biochem. 1993;30 (Pt 6):521–39. pmid:8304720.
- 25. Clemens KE, Klaschik E. Dyspnoea associated with anxiety—symptomatic therapy with opioids in combination with lorazepam and its effect on ventilation in palliative care patients. Support Care Cancer. 2011;19(12):2027–33. pmid:21153667.
- 26. Allcroft P, Margitanovic V, Greene A, Agar MR, Clark K, Abernethy AP, et al. The role of benzodiazepines in breathlessness: a single site, open label pilot of sustained release morphine together with clonazepam. J Palliat Med. 2013;16(7):741–4. pmid:23597092.
- 27. Pan CX, Morrison RS, Ness J, Fugh-Berman A, Leipzig RM. Complementary and alternative medicine in the management of pain, dyspnea, and nausea and vomiting near the end of life. A systematic review. J Pain Symptom Manage. 2000;20(5):374–87. pmid:11068159.
- 28. Neuwohner K, Lindena G, Team H, Palliative Care E. Assessment of distress with physical and psychological symptoms of patients in german palliative care services. Onkologie. 2011;34(3):94–8. pmid:21358213.
- 29. Pauli P, Marquardt C, Hartl L, Nutzinger DO, Holzl R, Strian F. Anxiety induced by cardiac perceptions in patients with panic attacks: a field study. Behav Res Ther. 1991;29(2):137–45. pmid:2021376.
- 30. Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med. 2010;363(8):733–42. pmid:20818875.
- 31. Parikh RB, Kirch RA, Smith TJ, Temel JS. Early specialty palliative care—translating data in oncology into practice. N Engl J Med. 2013;369(24):2347–51. pmid:24328469;
- 32. Greer JA, Jackson VA, Meier DE, Temel JS. Early integration of palliative care services with standard oncology care for patients with advanced cancer. CA Cancer J Clin. 2013;63(5):349–63. pmid:23856954.
- 33. Teunissen SC, Wesker W, Kruitwagen C, de Haes HC, Voest EE, de Graeff A. Symptom prevalence in patients with incurable cancer: a systematic review. J Pain Symptom Manage. 2007;34(1):94–104. pmid:17509812.
- 34. Janssen DJ, Spruit MA, Wouters EF, Schols JM. Daily symptom burden in end-stage chronic organ failure: a systematic review. Palliat Med. 2008;22(8):938–48. pmid:18801874.
- 35. Barata PC, Cardoso A, Custodio MP, Alves M, Papoila AL, Antonio B, et al. Symptom clusters and survival in Portuguese patients with advanced cancer. Cancer Med. 2016;5(10):2731–9. pmid:27624994;
- 36. Veldhuisen H, Zweers D, de Graaf E, Teunissen S. Assessment of anxiety in advanced cancer patients: a mixed methods study. Int J Palliat Nurs. 2016;22(7):341–50. pmid:27444163.
- 37. Tai SY, Lee CY, Wu CY, Hsieh HY, Huang JJ, Huang CT, et al. Symptom severity of patients with advanced cancer in palliative care unit: longitudinal assessments of symptoms improvement. BMC Palliat Care. 2016;15:32. pmid:26968159;
- 38. Richards MA, Ramirez AJ. Quality of life: the main outcome measure of palliative care. Palliat Med. 1997;11(2):89–92. pmid:9156103.
- 39. Stromgren AS, Groenvold M, Pedersen L, Olsen AK, Spile M, Sjogren P. Does the medical record cover the symptoms experienced by cancer patients receiving palliative care? A comparison of the record and patient self-rating. J Pain Symptom Manage. 2001;21(3):189–96. pmid:11239737.
- 40. Hodgkins M, Albert D, Daltroy L. Comparing patients' and their physicians' assessments of pain. Pain. 1985;23(3):273–7. pmid:4069724.
- 41. Grossman SA, Sheidler VR, Swedeen K, Mucenski J, Piantadosi S. Correlation of patient and caregiver ratings of cancer pain. J Pain Symptom Manage. 1991;6(2):53–7. pmid:2007792.
- 42. Urch CE, Chamberlain J, Field G. The drawback of the Hospital Anxiety and Depression Scale in the assessment of depression in hospice inpatients. Palliat Med. 1998;12(5):395–6. pmid:9924602.
- 43. Lees N, Lloyd-Williams M. Assessing depression in palliative care patients using the visual analogue scale: a pilot study. Eur J Cancer Care. 1999;8(4):220–3. pmid:10889619.
- 44. Stiel S, Matthes ME, Bertram L, Ostgathe C, Elsner F, Radbruch L. Validation of the new version of the minimal documentation system (MIDOS) for patients in palliative care: the German version of the edmonton symptom assessment scale (ESAS). Schmerz. 2010;24(6):596–604. pmid:20882300.