Figures
Abstract
Objective
Depression research has resulted in knowledge about neurobiology, pharmacological strategies and short-term cost-effective treatments. However, more than two-thirds of all depressed patients experience insufficient improvement. Therefore, a better understanding of what patients, carers and professionals perceive as most helpful in the treatment of depression is needed.
Methods
Concept mapping, a mixed-method design, was used to identify the patients (n = 33), carers (n = 22) and professionals (n = 50) perspectives. In six brainstorm sessions, the patients, carers and professionals generated 795 ideas, which were condensed into 55 unique statements. Subsequently, 100 participants prioritized and sorted these statements, which were analysed by multidimensional scaling and cluster analysis and visualized as a two-dimensional map.
Results
The 55 statements were clustered in 10 factors and further grouped into four main-factors (meta-clusters): ‘Professional therapist’, ‘Treatment content’, ‘Structured treatment process’ and ‘Treatment organisation’. Patients and carers prioritized ‘Treatment organisation’ higher than professionals, but overall there was considerable agreement about the factors of treatment the participants perceived as most helpful including factors related to the therapeutic relationship and the ‘creation of hope’.
Conclusions
Our study identified factors of depression treatment perceived as helpful according to patients, carers and professionals. Findings suggest that in a scientific era with emphasis on biological psychiatry, not only patients and carers, but also professionals consider aspecific factors the most helpful. Further studies might show that factors we found to be helpful in the treatment for depression, can be generalized to mental health treatment in general.
Citation: van Grieken RA, Verburg HF, Koeter MWJ, Stricker J, Nabitz UW, Schene AH (2016) Helpful Factors in the Treatment of Depression from the Patient’s, Carer’s and Professional’s Perspective: A Concept Map Study. PLoS ONE 11(12): e0167719. https://doi.org/10.1371/journal.pone.0167719
Editor: Pietro Cipresso, IRCCS Istituto Auxologico Italiano, ITALY
Received: June 16, 2016; Accepted: November 18, 2016; Published: December 16, 2016
Copyright: © 2016 van Grieken et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data contain private patient information. Anonymized data are available on request from the Corresponding Author.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the current era with an emphasis on biological psychiatry and time-limited protocollized treatments, depression research has resulted in knowledge about neurobiological mechanisms, pharmacological strategies and cost-effectiveness of short-term treatments [1, 2]. However, it is well known that more than two-thirds of all depressed patients experience insufficient clinical improvement, despite the wide range of therapeutic approaches available today [1]. Insufficient therapeutic success may be due to the fact that too often interventions fail to take into account that patients prefer treatments in accordance with their personal ideas, goals and preferences [3]. More importantly, the patient’s perspective on the treatment of depression influences treatment outcomes [4, 5].
Knowing that major depressive disorder (MDD) is one of the most common psychiatric disorders with devastating effects both for those who suffer from it and their carers [6], it is important to have a better understanding of what patients perceive as most helpful in the treatments for depression. This knowledge is especially important because depressed patient’s perspectives about treatment often differ from those among professionals [7]. For example, findings from a systematic review suggest that patients prefer psychotherapy rather than the occasional professional advice for pharmacological treatment strategies [8]. Moreover, the role of the carer in depression research is relatively invisible, but invaluable, since they are the daily witnesses in perceiving how treatment affects the depressed patient. Although professional depression guidelines [9, 10] require the carer’s engagement into treatment, carers often feel frustrated about their exclusion from treatment [11].
In today’s real-world practice, professionals may want to focus on more than just the application of evidence-based protocollized approaches, since multiple factors contribute to the success of treatments [12]. As said, perspectives of professionals may differ from those of patients and carers, and so we studied in an integrative way what the three groups—patients, carers and professionals—perceive as helpful factors in the treatment of depression [13]. We identified the following research questions: 1) What are helpful factors in the treatment of depression from the patient’s, carer’s and professional’s perspective? 2) What factors do patients, carers and professionals perceive as most helpful? 3) What kind of differences exist in the perspectives of patients, carers and professionals regarding these factors? With this knowledge, professionals will have the potential to adjust the specific needs of depression treatment for each individual patient to develop personalized treatment, ameliorate shared goals, and consequently accomplish better treatment compliance and effectiveness.
Methods
Design
In order to answer these questions and based on earlier experiences in our group [14, 15], we found ‘concept mapping’ to be the most promising and adequate method. Trochim [16] developed this structured mixed method in order to organize diverse ideas into an understandable and coherent framework. Concept mapping involves advanced statistical analyses of qualitative data, for the conceptualisation of a specific, mostly complex, subject. This method is successfully applied in the field of health, social and management sciences [17, 18].
Moreover, the combination of both qualitative and quantitative analyses makes this method more data-driven than other qualitative research methods like in-depth interviews. Through the usage of group processes, joint discussion and exploration, this method allows the encouragement of participants to bring up more ideas than would appear in individual approaches like interviews. Moreover, concept mapping generates the conceptual framework by a statistical algorithm, which can be replicated by others. The internal validity of the approach is high and the findings allow identification of similarities and differences between various perspectives [17].
Participants
Although there is no ‘correct’ number of participants when using concept mapping, and some argue that 20 participants in total is sufficient, we aimed for approximately 100 participants, a much larger group, to reach the maximum in terms of the exploration of ideas [18]. The study was presented to the Medical Research Ethics Committee of the Academic Medical Centre Amsterdam. In line with the Dutch legislation, this Committee decided that the study did not require extensive ethical review as participants were recruited on a volunteer basis and were not requested to undergo any incriminating intervention. Written informed consent was obtained from the participants.
Patients.
Patients were eligible for the study if they met the following criteria: (a) at least one major depressive episode (MDE) according to DSM-IV-TR criteria at some point within three years preceding participation; (b) professional treatment for the most recent MDE (e.g. from a psychiatrist or a psychologist) and (c) currently in remission, as indicated by a score of less than seven on the Hamilton Depression Rating Scale (HDRS) [19], in order to minimize bias because of ‘negative MDE related cognitions’. Exclusion criteria were: age younger than 18 years; insufficient understanding of the Dutch language; a terminal disease; mental retardation; suicidality and bipolar-, psychotic-, anxiety- or cognitive disorder. Patients were recruited through a request for study participation by e-mail to members of the Dutch ‘Depression Association’.
Carers.
Carers were included if they were partner, family member or friend of a patient who received depression treatment. We recruited carers by asking participating patients to contact their relations who were involved with their depression treatment.
Professionals.
Professionals were included if they had substantial experience with treatment of depression for at least one year. We purposively sampled professionals with diverse professional backgrounds (e.g. psychiatrists, psychologists, occupational therapists and psychiatric nurses) by approaching several Dutch academic and non-academic mental health care organisations.
Brainstorm sessions to generate statements
Concept mapping starts by defining the focus. In the current study, the participants were encouraged to brainstorm about their perspectives concerning the focus: ‘Which factors of depression treatment do you perceive as helpful for recovery, from your experience?’ We carried out six different brainstorm sessions: patients and carers were mixed and equally divided over four sessions. The two sessions with professionals were held separately to ensure patients and carers could talk freely and without reservation. The number of participants of these six brainstorm sessions varied between 6 and 13 and each session took approximately 1.5 hours. All sessions were guided by trained researchers (RAvG, HFV and JS; respectively experienced as a psychiatric trainee, research fellow and occupational therapist).
We approached 90 patients, of whom 21 (23.3%) could be included and participated. Patients invited 22 carers. We invited 39 professionals, of whom 20 (51.3%) participated. Participants in the brainstorming sessions were not required to be the same participants who carried out the prioritizing and sorting assignment [18]. See Table 1 for all participants.
During the brainstorm sessions, all ideas generated around the focus were visualized for the participants via a digital projector. The collection of ideas was continued until no new ideas emerged (saturation).We used audiotape and full transcription of the sessions to catch the ideas in an accurate manner. Based on these transcriptions, the researchers (RAvG, HFV and JS) combined 795 ideas and removed ideas that overlapped or that were not related to our focus. A total of 256 unique ideas remained. Next, ideas were further condensed by combining similar ideas, e.g. ‘the clinician should involve carers’ and ‘to look for a solution in treatment together with carers’, to formulate the quality demand statements with an abstraction level that was neither too specific nor too general. Consensus meetings between members of the research group led to a final set of 55 unique statements.
Prioritizing and sorting the statements
These 55 final statements were each printed onto cards and sent to the participants as a homework assignment. For this stage, we approached 76 patients, of whom 31 (34.4%) participated. Of the 22 carers invited, 19 participated. We invited 53 professionals, of whom 50 (94.3%) participated (see Table 1). We invited more participants than only the ones in the first stage. In the prioritizing assignment, each individual participant had to prioritize the 55 statements by dividing them into 5 groups of equal size. Group 1 was defined as ‘least helpful’ in depression treatment and group 5 as ‘most helpful’.
In the sorting assignment, each participant was asked to sort the 55 statements based on their own meaning of whether they belonged to the same category. Participants were asked to use between 2 and 12 and each category had 2 to 20 statements.
Statistical analyses
The prioritizing and sorting data from all participants were analysed using the computer programme ‘Ariadne’, designed to support concept mapping [20–22]. First, the multidimensional scaling analyses positioned the statements as points in a two-dimensional map: ‘point map’. The point map showed the typical circumplex distribution of the statements, in which the distance between the statements represents how often the participants sorted them in groups.
Second, the positioned statements on the point map were clustered, the ‘cluster map’, based on a hierarchical cluster analysis [18]. Subsequently, the relative importance of the statements and differences between the three participant groups were calculated based on the participant’s prioritizing score. Univariate analysis of variance (ANOVA) and multiple comparison tests of the mean priority of the clusters and statements were carried out. The mean cluster score was defined as the sum of all statements of the cluster. These data were analysed using SPSS (version 21) for Windows.
Results
Table 2 displays the demographics of each participant group and shows that participating carers were partners (80%), family members (10%) and friends (10%). Participating professionals were psychiatrists (27%), psychologists (28%), psychiatric nurses (30%) and creative or psychomotor therapists (5%). Seventy percent of these professionals had more than 5 years of experience with depression treatments. Of all professionals, 52% worked for different regional mental health care institutions and 48% were academic professionals. Patient characteristics are shown in Table 3.
Cluster map
The hierarchical cluster analyses resulted in a ten-cluster (1–10) solution and one single statement (ST 11): see Fig 1 for the cluster map. The proximity of the 55 statements in the cluster map is based on which statements were more likely to have been sorted in the same category based on their meaning according to the participants. The ten-cluster solution was considered the best interpretable one by consensus of all researchers according to the content of the statements belonging to these clusters. The circular distribution of the statements with its empty centre shows that there is little ambiguity about the positioning among participants: they agree about the reciprocal relation of the statements [18].
The map displays the 55 statements presented as dots, the 10 clusters (1–10) and 4 meta-clusters (I-IV).>
Additional meaning about their interrelation appeared when we combined these ten clusters into four meta-clusters. We numbered the (meta-)clusters in order of their perceived importance according to all participants. For each cluster we formulated a descriptive name based on the content and importance of the statements comprising the cluster [18]. An overview of the clusters with their corresponding statements, sorted by participant’s priority score, is presented in Table 4. We had the translation of the Dutch statements and (meta-) cluster names performed by a British native speaker.
Meta-cluster I.
‘Professional therapist’, contains 13 statements grouped into two clusters: 1. Therapeutic relationship: statements in this cluster all emphasize aspects that make the relationship between the participants meaningful; and 4. Professional therapist: statements imply that patients and carers feel taken seriously when the therapist’s attitude is professional.
Meta-cluster II.
‘Treatment content’, contains 25 statements grouped into four clusters: 2. Discovering and tackling underlying emotions and thoughts: comprising statements focus on technical factors of treatment; 3. Involving the social environment: statements emphasize the involvement of carers within treatment; 5. Encouraging self-reliance with specific activities: statements all comprise factors that increase self-reliance; and 6. Self-management: this cluster contains three statements that emphasize patients to work on their own recovery.
Meta-cluster III.
‘Structured treatment process’, contains 12 statements grouped into three clusters: 7. Focussing information and treatment on the patient’s needs: statements indicate that a careful exploration of the patient’s need for information about the various aspects of depression treatment is helpful; 9. Structure in the treatment process: statements imply that a clear structure within treatment is perceived as helpful; and 10. Clarity concerning the course of treatment: statements refer to the wish to be well informed about what the course of treatment looks like.
Meta-cluster IV.
‘Treatment organisation’, contains three statements grouped into one single cluster: 8. Treatment organisation: statements focus on organisations factors.
Finally, because the statistical analysis placed the single statement ‘Creating hope during treatment’ (ST 11) in the center of the cluster map, this statement has to be regarded as unique and helpful; it was not, or only minimally related to any broader category according to the participants.
Differences between patients, carers and professionals
As Table 4 shows, the three participant groups generally agreed on which clusters they perceived as most helpful, as mean priority scores are aligned with each other. However, patients and carers perceived the cluster ‘Treatment organisation’ (CL 8) significantly more helpful compared to professionals (p<0.05). In addition, we explored differences between the three groups considering single statements. We took the top 10 statements perceived as most helpful as these were prioritized by the patients (see Table 5). Patients’ and carers’ top 10 appeared to be similar except for one statement (ST 7). The statements ‘a social environment which takes the depression into account’ (ST 2), ‘the possibility to have contact with the therapist at short notice’ (ST 7) and ‘a therapy which quickly starts after the intake’ (ST 49) were considered significantly more helpful by patients compared to professionals.
Discussion
In this study we were able to identify and describe 55 different, helpful factors in the professional treatment of depression from the combined patients’, carers’ and professionals’ perspectives. These 55 factors can be summarized into four meta-clusters: ‘Professional therapist’, ‘Treatment content’, ‘Structured treatment process’ and ‘Treatment organisation’. Interestingly, the clusters we found to be helpful were considered as almost of equal importance among patients, carers and professionals. The three groups only differed on one cluster (‘Treatment organization’) with patients and carers rating this higher than professionals. Findings suggest that, in a treatment era with emphasis on biological psychiatry and short term protocolled treatments, patients, carers and professionals still perceive what has become known as aspecific factors as the most helpful in the current psychiatric treatment of depression.
Our results about aspecific factors perceived to be helpful are not only in line with our previous study, in which we found similar inhibiting factors of depression treatment that are often not addressed in clinical practice [23], but it also reminds us of Lambert’s common factor theory [24, 25]. Common factors (e.g. therapeutic alliance and hope) account for a major part of the effectiveness of psychotherapy treatment, and that only 15% of treatment outcome is related to technical factors [24, 26, 27]. He discovered four primary factors that influence positive treatment outcome and calculated the relative importance of each factor, which are very similar to our findings: 40% of therapeutic change comes from extra-therapeutic factors (in our results represented by the meta-clusters ‘structured treatment process’ and ‘treatment organization’), 30% from therapeutic relationship (‘professional therapist’), 15% from hope and expectancy (‘creating hope during treatment’) and 15% from therapeutic technique (‘treatment content’).
Lambert’s findings result from an extensive literature review of over 40 years of psychotherapy outcome studies in another era, whilst our study uses a structured empirical method and includes the perspectives of patients, carers and professionals of depression treatment. This finding suggests that, despite the numerous studies and randomized controlled trials that have been conducted in the last decade on depression and treatment, patients, carers and professionals still perceive the therapeutic relationship, hope and other aspecific factors, that are not easy to ‘catch’ in research with quantitative bounders, to be the most important.
However, our results also show some specific factors to be important. For example, the status apart statement ‘creating hope during treatment’ (ST 11) was perceived as unique and very helpful in the treatment of depression, especially according to patients and professionals. Patient’s hope and expectations have long been considered a key ingredient of successful psychotherapy [24] and a more recent meta-analysis by Constantino [28] demonstrated positive effects of patients’ expectations on their treatment outcomes. Moreover, the loss of hope has been described as a cause of treatment resistance in depression [29]. Although many psychotherapies include elements that address various expectations, such strategies are rarely emphasized in scientific literature or clinical practice [30]. Furthermore, treatment guidelines for depression remain relatively silent about this complex aspecific factor [9, 10]. This result indicates that the factor hope has to be explored more profoundly, to discover the meaning and usage of hope in treatment.
It might be simplistic to relate decisions for treatment to perspectives only, because treatment for depression is a complex process where a number of variables must be taken in account, such as severity, comorbidity and patient’s history [31]. However, preferences are strongly associated with particular outcomes such as entry into treatment and development of the therapeutic alliance [32]. Furthermore, supporting preferences as part of depression treatment result in more patients receiving the treatment that is most suitable to them [8]. Further initial evidence [33] supports the idea that patients who are able to gain control over their treatment decisions may experience improved outcomes. Even though the participant’s perspectives about helpful treatment factors might not result in objective remission, according to our results it is obvious that clinical judgment should stop concentrating on drugs as the only ‘cure’ of the disease [12] and should enlighten perspectives to improve e.g. patient treatment commitment and support from carers.
Strengths and limitations of the study
The main strengths of our study are the empirical exploration of three different participant perspectives, and the use of both qualitative and quantitative methods which offered the opportunity to address the experiences regarding all factors of the treatment of depression, from aspecific to specific. Our study also has some limitations. First, the concept map is a result of the three participant groups combined. Therefore, certain caution is necessary in comparisons of statement priorities and ranking of the different clusters between the patients, carers and professionals, due to the differences in the amount of participants within each group. Second, perspectives of participants are inherent to subjectivity. However, we aimed to explore the different perspectives, which is typical of qualitative research. Third, there may be different, helpful treatment approaches during different stages of a depressive episode, from mild to severe, and to specific (cultural) subgroups. Also, the participants were ethnic Dutch adults. Therefore, the perspectives are generalizable to adults from countries comparable to the Netherlands in terms of population and (mental) health care system. Future studies might include participants belonging to specific subgroups to specify helpful treatment factors. Fourth, we translated the Dutch statements and (meta-)cluster names to English and this might have resulted in slightly different meanings, however, we purposely selected a native British physician to minimize the possibility of bias in the meaning of the words.
The knowledge about the patient’s, carer’s and professional’s perspectives is a first step; further work needs to be done. For example, the development of a questionnaire to identify patient’s treatment perspectives and guidance for their carers, all in cooperation with the professional. This may help professionals to personalize treatment [34] and may contribute to higher levels of patient satisfaction and thereby, could make treatments more effective. Next, more research is needed to find out about if effectiveness really occurs if the patient’s, carer’s and professional’s perspectives are optimally used in treatment. Moreover, the factor ‘hope’ needs to be explored to discover the meaning and usage of this appearing new factor in treatment.
We believe that our findings broaden the view of clinical practice and clarify the importance of ‘common factors’ in psychotherapy research and real-world practice. We also believe that the implementation of our results in clinical practice might considerably improve a patient’s and carer’s engagement to treatment and better treatment outcome. Because of some overlap of our results with other research [25], further research might show that the factors we found to be perceived most helpful in the treatment for depression treatment can be replicated and generalized to mental health treatment in general.
Acknowledgments
We would like to thank the former patients, their carers and the mental health professionals who participated in this study. Special thanks to GGz Noord Holland Noord, GGz Arkin, and the program for mood disorders at the department of psychiatry of the Academic Medical Center. Our beloved colleague dr. M.W.J. Koeter passed away before the submission of the final version of this manuscript. R.A. van Grieken accepts responsibility for the integrity and validity of the data collected and analyzed.
Author Contributions
- Conceptualization: RAvG MWJK AHS.
- Data curation: RAvG HFV JS.
- Formal analysis: RAvG HFV UWN.
- Investigation: RAvG HFV JS.
- Methodology: RAvG HFV JS UWN AHS.
- Project administration: RAvG.
- Resources: RAvG UWN.
- Software: UWN.
- Supervision: AHS.
- Validation: RAvG UWN.
- Visualization: RAvG.
- Writing – original draft: RAvG HFV.
- Writing – review & editing: RAvG HFV JS UWN MWJK AHS.
References
- 1. Rush AJ, Trivedi MH, Wisniewski SR, Nierenberg AA, Stewart JW, Warden D, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry 2006;163(11):1905–1917. pmid:17074942
- 2. Driessen E, Van HL, Don FJ, Peen J, Kool S, Westra D, et al. The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatement of major depression: a randimized clinical trial. Am J Psychiatry 2013;170(9):1041–1050. pmid:24030613
- 3. Bultman DC, Svarstad BL. Effects of physician communication style on client medication beliefs and adherence with antidepressant treatment. Patient Educ Couns 2000;40(2):173–185. pmid:10771371
- 4. Deen TL, Fortney JC, Pyne JM. Relationship between satisfaction, patient-centered care, adherence and outcomes among patients in a collaborative care trial for depression. Adm Policy Ment Health 2011;38(5):345–355. pmid:20978932
- 5. Winter SE, Barber JP. Should treatment for depression be based more on patient preference? Patient Prefer Adherence 2013;7:1047–1057. pmid:24143080
- 6. Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med 2006;3(11):e442. pmid:17132052
- 7. Hansson M, Chotai J, Bodlund O. What made me feel better? Patients' own explanations for the improvement of their depression. Nord J Psychiatry 2012;66(4):290–296. pmid:22211274
- 8. van Schaik DJ, Klijn AF, van Hout HP, van Marwijk HW, Beekman AT, de Haan M, et al. Patients' preferences in the treatment of depressive disorder in primary care. Gen Hosp Psychiatry 2004;26(3):184–189. pmid:15121346
- 9.
American Psychiatric Association (APA). Practice Guideline for the Treatment of Patients with Major Depressive Disorder, 3rd ed. Arlington, VA, American Psychiatric Association (APA), 2010.
- 10.
Spijker J, Bockting CLH, Meeuwissen JAC, Vliet IM van, Emmelkamp PMG, Hermens MLM, et al. Multidisciplinaire Richtlijn Depressie (3rd rev). Richtlijn voor de diagnostiek, behandeling en begeleiding van volwassen patiënten met een depressieve stoornis. Utrecht, Trimbos-instituut, 2013.
- 11. Badger TA. Family members’experiences living with members with depression. West J Nurs Res 1996;18(2):149–171. pmid:8638421
- 12. Fava GA, Tomba E, Grandi S. The road to recovery from depression—don't drive today with yesterday's map. Psychother Psychosom 2007;76(5):260–265. pmid:17700045
- 13. Prins MA, Verhaak PF, Bensing JM, van der Meer K. Health beliefs and perceived need for mental health care of anxiety and depression—the patients' perspective explored. Clin Psychol Rev 2008;28(6):1038–1058. pmid:18420323
- 14. van Grieken RA, Kirkenier AC, Koeter MW, Schene AH. Helpful self-management strategies to cope with enduring depression from the patients’point of view: a concept map study. BMC Psychiatry 2014;14:331. pmid:25495848
- 15. van Grieken RA, Kirkenier ACE, Koeter MWJ, Nabitz UW, Schene AH. Patients’ perspective on self-management in the recovery from depression. Health Expect 2015;18(5):1339–1348. pmid:23910797
- 16. Trochim WMK. An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning 1989;12:1–16.
- 17. Rosas SR, Kane M. Quality and rigor of the concept mapping methodology: a pooled study analysis. Eval Program Plann 2012;35(2):236–245. pmid:22221889
- 18.
Kane M, Trochim MK. Concept mapping for planning and evaluation. Vol 50, London, Sage publications Inc, 2007.
- 19. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry 1960;23:56–62. pmid:14399272
- 20.
Severens P. Handbook concept mapping. National Center for Mental Health. Linde, Talcott, 1995.
- 21. De Vries G. Hees HL, Koeter MWJ, Lagerveld SE, Schene AH. Perceived impeding factors for return-to-work after long-term sickness absence due to major depressive disorder: a concept-mapping approach. PLOS One 2014;9(1):e85038. pmid:24454786
- 22. Nabitz U, van Randeraad-van der Zee C, Kok I, von Non-Martens M, Severens P. An overview of concept mapping in Dutch mental health care, Eval Program Plan 2016. [Epub ahead of print]
- 23. van Grieken RA, Beune EJAJ, Kirkenier ACE, Koeter MWJ, Zwieten MCB, Schene AH, et al. Patients’ perspectives on how treatment can impede their recovery from depression. J Affect Disord 2014;167:153–159. pmid:24972363
- 24.
Lambert MJ. Psychotherapy outcome research: Implications for integrative and eclectic therapists; in Norcross JC, Goldfried MR (eds): Handbook of Psychotherapy Integration. New York, Basic Books, 1992, pp 94–129.
- 25.
Lambert MJ. The Efficacy and Effectiveness of Psychotherapy. In Lambert M.J. (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 6th ed. New York, Wiley, 2013.
- 26. Thomas ML. The contributing factors of change in a therapeutic process. Contemp Fam Ther 2006;28:201–210.
- 27. Grawe K. Grundriss einer allgemeinen psychotherapie. Psychotherapeut 1995;40:130–145.
- 28. Constantino MJ, Arnkoff DB, Glass CR, Ametrano RM,Smith JZ. Expectations. J Clin Psychol 2011;67(2):184–192. pmid:21128304
- 29. Lewis L, Hoofnagle L. Treatment-resistant depression: the patient perspective. Biol Psych, 2003;53(8):635–639.
- 30. Greenberg RP, Constantino MJ, Bruce N. Are expectations still relevant for psychotherapy process and outcome? Clin Psychol Rev 2006;26(6):657–678. pmid:15908088
- 31. Tomba E, Fava GA. Treatment selection in depression: the role of clinical judgment. Psychiatr Clin North Am 2012;35(1):87–98. pmid:22370492
- 32. Gelhorn HL, Sexton CC, Classi PM. Patient preferences for treatment of major depressive disorder and the impact on health outcomes: a systematic review. Prim Care Companion CNS Disord 2011;13(5).
- 33. Geers AL, Rose JP, Fowler SL, Rasinski HM, Brown JA, Helfer SG. Why does choice enhance treatment effectiveness? Using placebo treatments to demonstrate the role of personal control. J Pers Soc Psychol 2013;105(4):549–566. pmid:23915042
- 34.
Yeung A, Feldman G, Fava M. Self-Management of Depression. A Manual for Mental Health and Primary Care Professionals. Cambridge: Cambridge University Press, 2010.