https://orcid.org/0000-0002-8598-5106
Figures
Abstract
Objective
Numerous randomized controlled trials (RCTs) demonstrate the efficacy of cognitive behavioral therapy (CBT), metacognitive therapy (MCT), and methods to reduce intolerance of uncertainty (IU-CBT) in the treatment of generalized anxiety disorder (GAD). However, few studies have investigated these treatments under conditions of routine clinical care. The main objective of this study was to investigate the effectiveness of psychotherapy for GAD in an outpatient setting and to identify factors influencing treatment outcome.
Methods
Fifty-nine GAD patients received naturalistic CBT (including MCT and IU-CBT) in an outpatient clinic and postgraduate training center for psychotherapy. Patients completed self-report questionnaires at the beginning and end of therapy regarding the main outcome worry as well as metacognitions, intolerance of uncertainty, depression, and general psychopathology.
Results
Worry, negative metacognitions, intolerance of uncertainty, depression, and general psychopathology decreased significantly (p’s < .001) with large effect sizes for all symptoms (d = 0.83–1.49). A reliable change in the main outcome worry was observed in 80% of patients, and recovery occurred in 23%. Higher worry scores at posttreatment were predicted by higher pretreatment scores, female sex, and less change in negative metacognitive beliefs during treatment.
Conclusions
Naturalistic CBT for GAD appears to be effective in routine clinical care for worry as well as depressive symptoms, with particular benefits associated with altering negative metacognitions. However, a recovery rate of only 23% is lower than the rates reported in RCTs. Treatment needs to be improved, especially for patients with more severe GAD and for women.
Citation: Krzikalla C, Morina N, Andor T, Nohr L, Buhlmann U (2023) Psychological interventions for generalized anxiety disorder: Effects and predictors in a naturalistic outpatient setting. PLoS ONE 18(3): e0282902. https://doi.org/10.1371/journal.pone.0282902
Editor: Stephan Doering, Medical University of Vienna, AUSTRIA
Received: September 7, 2022; Accepted: February 25, 2023; Published: March 10, 2023
Copyright: © 2023 Krzikalla et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data cannot be shared publicly, because the informed consent and ethical approval for this study did not include that the anonymized data may be made freely available in public repositories. The data that support the findings of this study are available upon reasonable request from the psychotherapy outpatient unit of the University of Münster (contact via pta@uni-muenster.de).
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Generalized anxiety disorder (GAD), which affects about 4% of the population at least once in their lifetime [1], is characterized by excessive anxiety and uncontrollable worry for at least six months, accompanied by physical or cognitive symptoms such as impaired concentration, irritability, or sleep disturbances [2]. GAD is associated with high rates of comorbidity [3, 4], impairment [5–7], long-term disability [8], and low quality of life [9]. Recent meta-analyses have reported medium to large pooled effect sizes for psychological interventions for GAD and medium effect sizes for comorbid depressive symptoms [10–12]. These meta-analyses further suggest that cognitive-behavioral therapy (CBT) is effective in treating GAD. CBT for GAD compromises applied relaxation, cognitive therapy, and exposure therapy, all of which have proven effective in randomized controlled trials [RCTs; e.g., 13–15; for an overview of CBT methods for GAD, see 16]. However, it is important to note that 40%–50% of patients undergoing treatment fail to achieve the necessary level of improvement on self-report questionnaires of worry to be considered recovered [17, 18].
Metacognitive therapy (MCT) was developed as a means of improving treatment efficacy [19]. This intervention is based on the metacognitive model of GAD, which highlights the importance of metacognitive beliefs about worry in the maintenance of the disorder. Positive metacognitions are beliefs about the need to engage in worrying, whereas negative metacognitions are beliefs about the uncontrollability and dangerousness of worrying [20]. Accordingly, MCT aims at modifying maladaptive metacognitive beliefs rather than the content of worry itself [21]. Several clinical trials indicate that MCT is an effective treatment over a range of disorders, including GAD [22].
Another rather new CBT approach is intolerance of uncertainty therapy (IU-CBT). Intolerance of uncertainty (IU) describes the tendency to react negatively to uncertain situations and events [23; for an overview of the construct, see 24]. IU is elevated in GAD patients and is suspected to lead to information processing biases [23]. IU-CBT includes worry awareness training, uncertainty recognition, behavioral and imaginal exposure, reevaluation of the usefulness of worry, and relapse prevention [25]. Several randomized controlled trials support the efficacy of IU-CBT for GAD [26–29].
Despite the evidence from RCTs, it remains unclear how effective psychological interventions for GAD are in routine clinical care. Specifically, patients with comorbid disorders are excluded in some clinical trials, which may limit the generalization of RCTs to routine clinical care [30, 31]. In addition, therapists in naturalistic settings might not use treatment protocols as developed in academic research [32] and, hence, produce different treatment effects. To address this, several studies have investigated treatment effects in naturalistic settings. In their meta-analysis, Hans and Hiller [33] reported large treatment effects for CBT in outpatients with anxiety and related disorders (but not GAD in particular) in routine clinical care. Stewart and Chambless [34] included in their meta-analysis 11 studies on generalized anxiety (in patients with GAD and other anxiety-related disorders) in naturalistic settings and found large pre- to posttreatment effects compared to benchmarking studies. Altogether, CBT seems to be effective in treating GAD in naturalistic settings. However, little information exists on the administered methods and levels of psychopathology of study participants [35, 36], and strict treatment protocols have often been used [37–39], which limits generalizability.
Another relevant aspect concerns the factors influencing treatment efficacy. Meta-analyses have given inconclusive findings as to whether age and gender affect treatment outcomes [12, 40, 41]. Further, it remains unclear whether GAD patients with comorbid disorders benefit less from psychological interventions [41–43]. Only a few studies thus far have examined predictors of change in GAD directly in context with a therapeutic intervention [13, 44, 45] or in a naturalistic setting [44, 46–48]. Studies that examined predictors in routine care have either not focused exclusively on psychological interventions [46, 47] or have reported results across different anxiety disorders [44, 48] so predictors specific to GAD are not clear. With regard to metacognitions and IU, to our knowledge, there is no publication demonstrating how these factors are influenced by psychological interventions in routine care and whether this potential influence predicts treatment outcome.
Against this background, the current study aimed to elucidate the effectiveness of CBT (including methods from MCT and IU-CBT) for GAD in a naturalistic outpatient setting. We expected these treatments to lead to a significant reduction of worry as well as metacognitive beliefs, IU, depression, and general psychopathology from pre- to post-assessment. We further examined in an explorative way the role of age, sex, comorbidity, metacognitions, and IU on treatment outcome.
Methods
Participants
All participants were patients who presented on their own seeking treatment at the university’s psychotherapy outpatient clinic between 2012 and 2020. The study included 59 patients (Mage = 32.8 years, SDage = 11.5, range: 19–58 years, 66.1% women) with a primary diagnosis of GAD, as assessed with the Structured Clinical Interview for DSM-IV Axis I Disorders [SCID; 49]. Note that although the original DSM-5 was published in 2013 [2], the German version of the SCID based on DSM-5 was published only in 2019 [50]. For our study, we decided to interview all patients using DSM-IV-criteria. In total, 52.5% had at least one current comorbid disorder: The most common comorbid disorders were depressive disorders (28.8%) and other anxiety disorders (20.3%). For further information, see Table 1.
Eligibility criteria for inclusion into this study were (a) GAD as the primary diagnosis, (b) an indication for CBT, and (c) at least five sessions of psychotherapy. There were no specific exclusion criteria concerning any comorbidity or current medication status.
Setting and procedure
The study took place at the outpatient clinic and postgraduate training center for psychotherapy at the University of Münster between 2012 and 2020. When contacting the clinic, patients received an initial appointment with licensed psychotherapists to check indication for outpatient therapy. Treatment was conducted in a naturalistic setting by psychologists in training to become psychological psychotherapists under close supervision by licensed psychotherapists. The postgraduate training included a specific workshop for treating GAD with CBT (including modern enhancements such as MCT and IU-CBT). Before therapy was requested via the health insurance provider, four to six diagnostic sessions occurred during which the pre-assessment took place. The post-assessment usually took place three to four sessions before end of therapy. Both the pre- and post-assessment consisted of self-report questionnaires filled out by the patient and basic documentation filled out by the therapist. Patients provided written informed consent to scientific use of the data at the beginning of the therapy either via the therapy contract or via a separate informed consent form. The procedure was approved by the Institutional Review Board of the Department of Psychology and Sport Science at the University of Münster.
Patients received an average of 51.4 sessions (SD = 18.6, range: 15–107). Treatment duration had a mean of 19.9 months (SD = 6.4, range: 6–45). Applied methods, number of sessions, and duration of treatment was not limited and varied according to patient needs. Therapists had the discretion, in consultation with their supervisors, to employ a combination of techniques from various approaches in the administration of treatment, and were not restricted to adhering to a specific treatment manual. A total of 82 patients consented to data analysis. Following treatment, the final treatment reports were rated by two independent evaluators (C.K. and a postgraduate psychologist in training) to ensure that GAD was the main focus of therapy and to determine the applied therapeutic interventions and comorbid disorders. We used a coding scheme to cluster interventions into the categories exposure therapy, cognitive restructuring, MCT, and IU-CBT. We excluded 23 patients because either GAD was not the main focus of therapy (n = 14) or patients took part in another manualized intervention study prior to therapy (n = 9). Interrater reliability was assessed with Cohen’s kappa [52], κ = .78. Discrepancies between the two independent evaluators (n = 4) were resolved in joint discussion with a third clinically experienced person (T.A.). Following an intent-to-treat approach, the analyses were conducted with all patients who met the inclusion criteria. This included five patients who terminated therapy prematurely (8.5%). Most treatments combined different kinds of interventions: 79.7% of all patients received cognitive restructuring, 78.0% MCT, 76.3% exposure therapy, and 52.5% IU-CBT. In three cases (5.1%), only one therapy method was used. Additional methods included relaxation techniques, methods from acceptance and commitment therapy, emotion-focused therapy, and dialectic behavioral therapy.
Measures
Outcome measures.
To assess excessive worry as our primary treatment outcome, we applied the Penn State Worry Questionnaire [PSWQ; 53; German version: 54]. This self-report consists of 16 items rated on a 5-point Likert scale ranging from “not at all typical of me” to “very typical of me” (example item: “I worry all the time”). The PSWQ has good psychometric properties [55, 56], and the internal consistency in the present study was Cronbach’s α = .83 for the pre-assessment.
The short form of the Metacognitions Questionnaire [MCQ; 57; German version: 58] includes 30 items measuring metacognitive beliefs about worry on a 4-point Likert scale ranging from “do not agree” to “agree very much”. In this study, we used two of its five subscales: positive beliefs about worry (MCQ-POS, example item: “Worrying helps me to avoid problems in the future”) and negative beliefs about the uncontrollability and danger of worrisome thoughts (MCQ-NEG, example item: “My worrying is dangerous for me”). The MCQ possesses good psychometric properties [57–59]; in the current sample, the internal consistencies were α = .88 for MCQ-POS and α = .72 for MCQ-NEG.
To assess IU, we used the Intolerance of Uncertainty Scale [IUS; 60]. The shortened German version [61] consists of 18 items rated on a 5-point Likert scale ranging from “not at all characteristic of me” to “entirely characteristic of me” (example item: “Unforeseen events upset me greatly”). It has been shown to have good psychometric properties [61]; the internal consistency in the current sample was α = .93.
The Beck Depression Inventory-II [BDI-II; 62; German version: 63] is a self-report instrument consisting of 21 items, each with four statements with increasing severity, from which participants are asked to select the statement that best describes their condition over the past two weeks. A total sum of ≥9 indicates at least minor depressive symptoms. The BDI has high internal consistency, validity, and test-retest reliability [64, 65]; the internal consistency in the current sample was α = .88.
The Symptom Check-List [SCL-90-R; 66; German version: 67] measures general psychopathology as a self-report questionnaire with 90 items rated on a 5-point Likert scale. Participants are asked to rate how much they had been bothered by, e.g., headaches, crying easily, or overeating during the past week. The SCL-90-R has nine subscales and three summary scores; we used the summary score Global Severity Index (GSI, total sum for all items) as an indicator for general psychopathology for further analysis. The SCL-90-R has good psychometric properties [68], and the internal consistency in the current sample was α = .94.
Data analyses
Statistical analyses were conducted with the statistical processing language R [69] mainly using the packages mice version 3.14.0 [70] and miceadds version 3.12–26 [71]. Prior to analyses, missing data were multiply imputed; 63.4% of all variables showed missing values. The amount of missing data ranged from 1.7% (PSWQ item 4 and several items on the SCL-90-R at pretreatment) to 13.6% (MCQ item 10 at posttreatment). According to Little’s MCAR Test [72], data were missing completely at random, χ2 < .001, df = 4855, p = > .999. We applied item-level and parcel summary multiple imputation using predictive mean matching [73, 74] as implemented in the R package mice [70] generating 50 sets of imputation. Auxiliary variables were identified using correlation analyses between the variables and missing variables as well as univariate t-test comparisons to improve the estimations and statistical power [75, 76]. We included all variables within the data set correlating ≥ ±.40 with one of the variables with missing values or with the missing variables (which indicate whether a data point is missing) as auxiliary variables into the imputation model. Further, we included variables which showed a significant univariate t-test and thus indicating statistically significant differences between participants with vs. without missing values. The potential scale reduction factor and graphical diagnostics were used for convergence diagnostics [75, 76]. Statistical analyses were performed separately on each completed dataset. Subsequently, all estimated parameters were pooled into a single set of results using Rubin’s rules [77]. Pooling estimates from multivariate analysis of variance (MANOVA) is still the subject of research [e.g., 78]. Since there are currently no established pooling rules for repeated-measures MANOVA, we decided against pooling in this specific case and instead report the range of the 50 MANOVAs performed [79].
The assumptions for the statistical procedures [80] were examined in the original dataset. There were three univariate outliers in three different subscales based on p < .001 under the z-distribution [81]. After inspection, it became clear that these were cases where either the initial symptom severity was low (pretreatment PSWQ) or the posttreatment values were higher than in the rest of the data (BDI and GSI). Removing or trimming these values might magnify effects, as this would only affect the patients showing a small pre-post difference. Therefore, we kept the values in the data. The violation of the multivariate normal distribution was responded to with an adequate statistical procedure (semi-parametric MANOVA). All other assumptions were met.
To analyze the outcome of the naturalistic treatment, we used a repeated-measures MANOVA, with time of measurement as the within-subjects factor and all outcome measures as dependent variables. Due to multivariate non-normality, we conducted a semi-parametric MANOVA using the multRM-function of the MANOVA.RM package version 0.5.3 [82]. This function outputs a Wald-type statistic (WTS), modified ANOVA-type statistic (MATS), and p values based on parametric bootstrapping (we applied 10,000 bootstrap resamples), which is recommended in a multivariate setting [83]. Post-hoc paired t-tests with Bonferroni-adjusted alpha levels (αadjusted = .05/6 = .008) and effect sizes according to Cohen’s formula, d = (M1-M2)/SDpooled [84], were calculated to determine effectiveness for each outcome variable. Due to the naturalistic study design, there is no control group; therefore, effect sizes might be inflated. To investigate clinical significance in addition to statistical significance, for the main outcome worry (PSWQ), we calculated reliable change and rate of recovery [85] based on criteria from Fisher [17]. In his review, he determined that a 7-point change on the PSWQ is needed for reliable change, and a cut-off for recovery of 46 points or below.
To inspect predictors of treatment outcome, we conducted a hierarchical multiple regression analysis with PSWQ posttreatment scores as the criterion. PSWQ pretreatment scores were entered in the first step to control for initial symptom severity. Age, sex, and comorbidity status were entered in a second step. In the final step, difference scores of MCQ-NEG, MCQ-POS, and IUS were entered in the regression model.
Results
A table with the pooled correlations of all measures can be found in the supplementary material (S1 Table).
Effectiveness of naturalistic psychotherapy
The repeated-measures MANOVA yielded significance in all 50 datasets (p values after parametric bootstrapping < .001 for WTS and MATS), indicating that the psychotherapy had an effect on the combined dependent variables. The WTS statistic (df = 6) ranged between 173.30 and 216.29, and MATS ranged between 269.25 and 302.26. Table 2 presents an overview of means, standard deviations, correlations, and effect sizes for the paired t-tests. All paired t-tests were significant apart from the pre- to posttreatment difference for MCQ-POS, which failed significance with the Bonferroni-adjusted alpha level of αadjusted = .008. Effect sizes were highest for PSWQ (d = 1.49) and MCQ-NEG (d = 1.48).
Reliable change was achieved by n = 47 patients (79.66%, range: 46–48 in the 50 imputed datasets) with a reduction in PSWQ of at least 7 points. One patient deteriorated over the course of the treatment (1.7%). Considering recovery, only patients who were over the cut-off before the treatment were evaluated (N = 56). Of these, 13 patients (23.21%, range: 12–15) reached the cut-off criteria of 46 points or below, indicating recovery.
Predictors of primary treatment outcome
The pooled results of the stepwise multiple regression for the PSWQ posttreatment score as the criterion are shown in Table 3.
Initial worry severity predicted higher posttreatment worry in all three regression equations. In the final model, PSWQ pretreatment scores explained 27% of the variance in posttreatment values. In the second step, only sex was shown to be an additional predictor and remained significant in the final model, explaining 7% of the variance. Based on the coding, it can be concluded that women have higher worry scores at posttreatment. Age and comorbidity status did not influence treatment outcome. Concerning MCQ-NEG, MCQ-POS, and IUS, only the difference score for MCQ-NEG was a significant predictor in the final model, explaining 15% of the variance in posttreatment PSWQ scores. Higher difference scores in MCQ-NEG predicted lower posttreatment PSWQ scores. The final model explained 48% of the variance in worry posttreatment scores. A post-hoc power-analysis was performed using the R-package “pwr” [86]. The analysis indicated a power of .78 to detect the given effect, based on the adjusted R2 = .26 (the lower bound of the 95% confidence interval for the R2 for the final model) and the use of all seven predictor variables. A sample size of n = 85 would have been required to achieve a power of .95.
Discussion
This study provides further insights into the treatment for GAD in routine clinical care by determining the effectiveness of CBT (including methods from MCT and IU-CBT) and identifying important predictors of treatment outcome in a naturalistic outpatient setting. Patients showed significant reductions in worry, negative metacognitions, IU, depression, and general psychopathology with large effect sizes from pre- to posttreatment. Eighty percent of the patients showed reliable improvements in worry, and 23% recovered. Only one patient deteriorated. Higher worry at posttreatment was predicted by higher symptom severity at pretreatment, female sex, and less change in negative metacognitions.
Naturalistic psychotherapy for GAD seems to be effective in not only reducing worry but also reducing general psychopathology and depressive symptoms. Effect sizes were large and consistent with previous literature [10, 33]. To our knowledge, this is the first publication to also investigate the influence of naturalistic psychotherapy on metacognitions and IU, which are known to play an important role in GAD [e.g., 24, 87]. Negative metacognitions were significantly reduced from pre- to posttreatment with a large effect size; yet, there was no significant effect on positive metacognitions after Bonferroni correction. This aligns with the metacognitive model, which specifically emphasizes the influence of negative metacognitions in the maintenance of GAD [87]. The large effect of naturalistic psychotherapy on IU is comparable to other studies using CBT approaches [88, 89]; this is particularly noteworthy, since strategies to reduce IU were only explicitly addressed in about 50% of therapies under study. However, it seems that IU might also be altered when not explicitly addressed [27].
Alongside statistical significance, we also investigated clinical significance. Eighty percent of the patients showed reliable change in terms of the main outcome worry. This is in line with or even exceeds previous studies under routine conditions [35–37]. However, only 23% of the patients recovered by the end of treatment. Although this is lower than the recovery rates of approximately 50% that have been found in RCTs [17, 18], our rate of 23% is within the range of other naturalistic studies, in which recovery rates ranged from 13% to 50% [36, 37, 43]. In RCTs and, as it seems, even more in naturalistic studies, there is still room for improvement in psychotherapy for GAD, as a big proportion of patients still fall into the clinical range at the end of treatment. One way to improve treatment is to better understand what factors influence the outcome of psychotherapy.
Therefore, the current study also examined which factors predicted worry at posttreatment. We found that pretreatment symptom severity predicted posttreatment symptom severity, such that patients with higher initial symptom severity remained more burdened with worry after treatment; this adds to a previously inconclusive body of studies [41]. In addition, after controlling for pretreatment symptom severity and other predictors, female sex predicted higher worry at posttreatment. Most previous studies have not found a sex-specific effect [12, 41], but those that have found this effect noted that it was in the same direction that we found in the current study [47, 90]. Since the current study included more female than male participants, this could also have influenced the rather poor recovery rate in the whole sample. Overall, these results highlight the need to improve psychotherapy for GAD. Treatment in routine clinical care should especially be optimized for patients with more severe manifestations of GAD and women, as they continue to have worse scores at posttreatment. Age and comorbidity appear to require less consideration, as these factors failed to predict treatment outcome. Although some dissenting findings exist, our findings largely match previous evidence [12, 41, 46].
Concerning treatment factors, only change in negative metacognitions predicted worry at posttreatment. This fits with the theoretical framework of MCT and previous literature on the efficacy of MCT [22]. While IU decreased significantly over the course of treatment, this change did not predict treatment outcome in worry. This contradicts previous findings that have shown an association between changes in IU and changes in worry [24]; however, the studies cited by Carleton [24] did not control for metacognitions. The current study suggests that negative metacognitions play a more important role in reducing worry than IU. Notably, in the current study, IU was explicitly addressed in only about 50% of the treatments. More research is needed to further investigate the role of IU and metacognitions on treatment outcomes for GAD. To date, therapists in routine clinical care should focus on changing negative metacognitive beliefs to reduce worry.
Strengths and limitations
The current study is representative of the target population and provides relevant insight into patient and treatment characteristics. Our sample consisted of patients with pretreatment worry scores comparable to such scores in RCTs [40] and in naturalistic settings [91]. It included patients with comorbidities and medication, as is often seen in routine clinical care. In addition, dropout was low (8.5%), and data was analyzed by applying an intent-to-treat-approach with an adequate method to deal with missing data, which is often lacking in other studies [10]. Treatment outcome was analyzed using multiple methods to gain insights into statistical and clinical significance, which was particularly important because discrepancies occurred between different methods. In this study, no treatment protocols restricted therapists to use and/or omit certain methods. In 95% of treatments in the current study, therapists mixed interventions from different treatment approaches (CBT, MCT, IU-CBT); such an eclectic approach is typical for therapists in routine clinical care [32, 92]. Nevertheless, this prevented us from making any statements about the efficacy of individual treatment methods.
In addition to the strengths, some important limitations must also be considered when interpreting the current study’s results. Although the naturalistic design has advantages regarding external validity, internal validity might be compromised. First, because there was no control group, it is not possible to establish a causal relationship between the change in outcome measures and the psychotherapy administered. Further, the treatment duration averaged 20 months, leaving time for maturation, external factors, or other threats to internal validity. Due to the naturalistic design of effectiveness studies, they usually lack control groups. Nevertheless, it is important to study treatment effects in this setting, as the effects may differ from those in RCTs [30, 31]. Another limitation of this study is the reliance on self-report questionnaires as a method of data collection, which can be affected by bias [93]. However, a meta-analysis of GAD patients found that the effect sizes based on self-report questionnaires were smaller than those based on clinically-rated instruments [94], suggesting that the results based on self-report questionnaires may not overestimate the effectiveness of therapy. While the sample size provided sufficient power in this study, future replication efforts could benefit from a larger sample size to enhance the robustness of the findings and provide stronger evidence for the conclusions drawn. Overall, results of the current study are comparable to other studies under routine conditions [e.g., 35, 37, 91].
Conclusions
The current study demonstrated the effectiveness of CBT (including methods of MCT and IU-CBT) in reducing worry, negative metacognitions, IU, depressive symptoms, and general psychopathology in routine clinical care. Effect sizes were large and comparable to those of previous studies. It seems especially beneficial to focus on changing negative metacognitions, as this predicted a better outcome. Yet, in terms of clinical significance, only 23% of patients achieved recovery status. In particular, women and patients with more severe GAD at baseline had poorer treatment outcomes. Additional research is necessary to further increase the effectiveness of CBT with respect to the recovery rate of GAD in routine clinical care.
Supporting information
S1 Table. Pooled correlations between all measures.
https://doi.org/10.1371/journal.pone.0282902.s001
(DOCX)
Acknowledgments
The authors would like to thank Katharina Schwieters for her assistance in the study.
References
- 1. Grant BF, Hasin DS, Stinson FS, Dawson DA, June Ruan W, Goldstein RB, et al. Prevalence, correlates, co-morbidity, and comparative disability of DSM-IV generalized anxiety disorder in the USA: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Psychol Med. 2005; 35:1747–59. pmid:16202187.
- 2.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Arlington; 2013.
- 3. Vesga-López O, Schneier F, Wang S, Heimberg RG, Liu S-M, Hasin DS, et al. Gender differences in generalized anxiety disorder: Results from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). Journal of Clinical Psychiatry. 2008; 69:1606–16. Available from: https://pubmed.ncbi.nlm.nih.gov/19192444/
- 4. Kessler RC, Chiu WT, Demler O, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005; 62:617–27. pmid:15939839
- 5.
Holaway RM, Rodebaugh TL, Heimberg RG. The epidemiology of worry and generalized anxiety disorder. In: Davey GCL, Wells A, editors. Worry and its psychological disorders. Theory, assessment, and treatment. West Sussex: Wiley & Sons; 2006. pp. 3–19.
- 6. Kessler RC, Wittchen H-U. Patterns and correlates of generalized anxiety disorder in community samples. J Clin Psychiatry. 2002; 63 Suppl 8:4–10. Available from: http://hdl.handle.net/11858/00-001M-0000-000E-A295-8. pmid:12044107
- 7. Burstein M, Beesdo-Baum K, He J-P, Merikangas KR. Threshold and subthreshold generalized anxiety disorder among US adolescents: Prevalence, sociodemographic, and clinical characteristics. Psychol Med. 2014; 44:2351–62. pmid:24384401.
- 8. Hendriks SM, Spijker J, Licht CMM, Hardeveld F, de Graaf R, Batelaan NM, et al. Long-term disability in anxiety disorders. BMC Psychiatry. 2016; 16. pmid:27431392.
- 9. Lieb R, Becker ES, Altamura C. The epidemiology of generalized anxiety disorder in Europe. Eur Neuropsychopharmacol. 2005; 15:445–52. pmid:15951160.
- 10. Carpenter JK, Andrews LA, Witcraft SM, Powers MB, Smits JAJ, Hofmann SG. Cognitive behavioral therapy for anxiety and related disorders: A meta-analysis of randomized placebo-controlled trials. Depress Anxiety. 2018; 35:502–14. pmid:29451967.
- 11. Cuijpers P, Cristea IA, Karyotaki E, Reijnders M, Huibers MJH. How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry. 2016; 15:245–58. pmid:27717254.
- 12. Carl E, Witcraft SM, Kauffman BY, Gillespie EM, Becker ES, Cuijpers P, et al. Psychological and pharmacological treatments for generalized anxiety disorder (GAD): A meta-analysis of randomized controlled trials. Cogn Behav Ther. 2020; 49:1–21. pmid:30760112.
- 13. Hoyer J, Beesdo K, Gloster AT, Runge J, Höfler M, Becker ES. Worry exposure versus applied relaxation in the treatment of generalized anxiety disorder. Psychother Psychosom. 2009; 78:106–15. pmid:19218829.
- 14. Arntz A. Cognitive therapy versus applied relaxation as treatment of generalized anxiety disorder. Behav Res Ther. 2003; 41:633–46. pmid:12732372
- 15. Kaczkurkin AN, Foa EB. Cognitive-behavioral therapy for anxiety disorders: An update on the empirical evidence. Dialogues in Clinical Neuroscience. 2015; 17:337–46. pmid:26487814
- 16.
Newman MG, Zainal NH, Hoyer J. Cognitive-behavioral therapy (CBT) for generalized anxiety disorder (GAD). In: Gerlach AL, Gloster AT, editors. Generalized anxiety disorder and worrying: A comprehensive handbook for clinicians and researcher. Hoboken: Wiley-Blackwell; 2020. pp. 203–30.
- 17.
Fisher PL. The efficacy of psychological treatments for generalised anxiety disorder. In: Davey GCL, Wells A, editors. Worry and its psychological disorders. Theory, assessment, and treatment. West Sussex: Wiley & Sons; 2006. pp. 359–77.
- 18. Hanrahan F, Field AP, Jones FW, Davey GCL. A meta-analysis of cognitive therapy for worry in generalized anxiety disorder. Clin Psychol Rev. 2013; 33:120–32. pmid:23168445.
- 19. Wells A. Meta-cognition and worry: A cognitive model of generalized anxiety disorder. Behav Cogn Psychother. 1995; 23:301–20.
- 20.
Wells A. Metacognitive therapy for anxiety and depression. New York: Guilford Press; 2009.
- 21. Wells A. Cognition about cognition: Metacognitive therapy and change in generalized anxiety disorder and social phobia. Cogn Behav Pract. 2007; 14:18–25.
- 22. Normann N, Morina N. The efficacy of metacognitive therapy: A systematic review and meta-analysis. Front Psychol. 2018; 9:2211. pmid:30487770.
- 23. Dugas MJ, Koerner N. Cognitive-behavioral treatment for generalized anxiety disorder: Current status and future directions. J Cogn Psychother. 2005; 19:61–81.
- 24. Carleton RN. The intolerance of uncertainty construct in the context of anxiety disorders: theoretical and practical perspectives. Expert Rev Neurother. 2012; 12:937–47. pmid:23002938.
- 25.
Dugas MJ, Robichaud M. Cognitive-behavioral treatment for generalized anxiety disorder: From science to practice. New York: Routledge; 2012.
- 26. Dugas MJ, Brillon P, Savard P, Turcotte J, Gaudet A, Ladouceur R, et al. A randomized clinical trial of cognitive-behavioral therapy and applied relaxation for adults with generalized anxiety disorder. Behav Ther. 2010; 41:46–58. pmid:20171327.
- 27. Van der Heiden C, Muris P, Van der Molen HT. Randomized controlled trial on the effectiveness of metacognitive therapy and intolerance-of-uncertainty therapy for generalized anxiety disorder. Behav Res Ther. 2012; 50:100–9. pmid:22222208.
- 28. Dugas MJ, Ladouceur R. Treatment of GAD: Targeting intolerance of uncertainty in two types of worry. Behav Modif. 2000; 24:635–57. pmid:11036732
- 29. Ladouceur R, Dugas MJ, Freeston MH, Léger E, Gagnon F, Thibodeau N. Efficacy of a cognitive–behavioral treatment for generalized anxiety disorder: Evaluation in a controlled clinical trial. J Consult Clin Psychol. 2000; 68:957–64. pmid:11142548
- 30. Desmet M. Experimental versus naturalistic psychotherapy research: Consequences for researchers, clinicians, policy makers and patients. Psychoanalytische Perspectieven. 2013; 31:59–78. Available from: http://hdl.handle.net/1854/LU-5645063.
- 31. Hoertel N, Le Strat Y, Blanco C, Lavaud P, Dubertret C. Generalizability of clinical trial results for generalized anxiety disorder to community samples. Depress Anxiety. 2012; 29:614–20. pmid:22495990.
- 32. Asnaani A, Gallagher T, Foa EB. Evidence-based protocols: Merits, drawbacks, and potential solutions. Clin Psychol.: Sci Pract. 2018; 25:e12266.
- 33. Hans E, Hiller W. A meta-analysis of nonrandomized effectiveness studies on outpatient cognitive behavioral therapy for adult anxiety disorders. Clin Psychol Rev. 2013; 33:954–64. pmid:23988455.
- 34. Stewart RE, Chambless DL. Cognitive-behavioral therapy for adult anxiety disorders in clinical practice: a meta-analysis of effectiveness studies. J Consult Clin Psychol. 2009; 77:595–606. pmid:19634954.
- 35. McAleavey AA, Youn SJ, Xiao H, Castonguay LG, Hayes JA, Locke BD. Effectiveness of routine psychotherapy: Method matters. Psychother Res. 2019; 29:139–56. pmid:29096584.
- 36. Westbrook D, Kirk J. The clinical effectiveness of cognitive behaviour therapy: Outcome for a large sample of adults treated in routine practice. Behav Res Ther. 2005; 43:1243–61. pmid:16086979.
- 37. Hirsch CR, Beale S, Grey N, Liness S. Approaching cognitive behavior therapy for generalized anxiety disorder from a cognitive process perspective. Front Psychiatry. 2019; 10. pmid:31780964.
- 38. Jiang MYW, Kandris E, El-Masry Y, Drobny J, St. Clare T, Chen JTH. Effectiveness of transdiagnostic group cognitive-behavioural therapy for anxiety disorders in a naturalistic clinical setting. Clin Psychol. 2021; 25:69–77.
- 39. Kehle SM. The effectiveness of cognitive behavioral therapy for generalized anxiety disorder in a frontline service setting. Cogn Behav Ther. 2008; 37:192–8. pmid:18608310.
- 40. Covin R, Ouimet AJ, Seeds PM, Dozois DJA. A meta-analysis of CBT for pathological worry among clients with GAD. J Anxiety Disord. 2008; 22:108–16. pmid:17321717.
- 41. Schneider RL, Arch JJ, Wolitzky-Taylor KB. The state of personalized treatment for anxiety disorders: A systematic review of treatment moderators. Clin Psychol Rev. 2015; 38:39–54. pmid:25795293.
- 42. Newman MG, Przeworski A, Fisher AJ, Borkovec TD. Diagnostic comorbidity in adults with generalized anxiety disorder: Impact of comorbidity on psychotherapy outcome and impact of psychotherapy on comorbid diagnoses. Behav Ther. 2010; 41:59–72. pmid:20171328.
- 43. Destoop M, Van den Eede F, Ansseau M, Albert A, Vanbelle S, Mignon A, et al. Prevalence and clinical characteristics of remission during treatment in generalized anxiety. Int J Psychiatry Clin Pract. 2013; 17:90–7. pmid:23485125.
- 44. Davis L, Barlow DH, Smith L. Comorbidity and the treatment of principal anxiety disorders in a naturalistic sample. Behav Ther. 2010; 41:296–305. pmid:20569779.
- 45. Goldman N, Dugas MJ, Sexton KA, Gervais NJ. The impact of written exposure on worry: a preliminary investigation. Behav Modif. 2007; 31:512–38. pmid:17548543.
- 46. Schat A, Van Noorden MS, Noom MJ, Giltay EJ, Van der Wee NJA, Vermeiren RRJM, et al. Predictors of outcome in outpatients with anxiety disorders: The Leiden routine outcome monitoring study. J Psychiatr Res. 2013; 47:1876–85. pmid:24074517.
- 47. Rodriguez BF, Weisberg RB, Pagano ME, Bruce SE, Spencer MA, Culpepper L, et al. Characteristics and predictors of full and partial recovery from generalized anxiety disorder in primary care patients. J Nerv Ment Dis. 2006; 194:91–7. pmid:16477186.
- 48. DiMauro J, Domingues J, Fernandez G, Tolin DF. Long-term effectiveness of CBT for anxiety disorders in an adult outpatient clinic sample: A follow-up study. Behav Res Ther. 2013; 51:82–6. pmid:23262115.
- 49.
Wittchen H-U, Zaudig M, Fydrich T. Strukturiertes Klinisches Interview für DSM-IV [Structured clinical interview for DSM-IV]. Göttingen: Hogrefe; 1997.
- 50.
Beesdo-Baum K, Zaudig M, Wittchen H-U. Strukturiertes Klinisches Interview für DSM-5-Störungen—Klinische Version. Deutsche Bearbeitung des Structured Clinical Interview for DSM-5 Disorders—Clinician Version [German Version of the Structured Clinical Interview for DSM-5 Disorders—Clinician Version]. Göttingen: Hogrefe; 2019.
- 51.
World Health Organization. International statistical classification of diseases and related health problems (10th ed.). 2016. https://icd.who.int/browse10/2016/en.
- 52. Cohen J. A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas. 1960; 20:37–46.
- 53. Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behav Res Ther. 1990; 28:487–95. pmid:2076086
- 54. Stöber J. Besorgnis: Ein Vergleich dreier Inventare zur Erfassung allgemeiner Sorgen [Worrying: A comparison of three questionnaires concerning everyday worries]. Zeitschrift für Differentielle und Diagnostische Psychologie. 1995; 16:50–63. Available from: https://psycnet.apa.org/record/1995-87603-001.
- 55. Fresco DM, Heimberg RG, Mennin DS, Turk CL. Confirmatory factor analysis of the Penn State Worry Questionnaire. Behav Res Ther. 2002; 40:313–23. pmid:11863241
- 56.
Glöckner-Rist A, Rist F. Deutsche Version des Penn State Worry Questionnaire (PSWQ-d) [German version of the Penn State Worry Questionnaire]. Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS); 2014.
- 57. Wells A, Cartwright-Hatton S. A short form of the Metacognitions Questionnaire: Properties of the MCQ-30. Behav Res Ther. 2004; 42:385–96. pmid:14998733
- 58. Arndt A, Patzelt J, Andor T, Hoyer J, Gerlach AL. Psychometrische Gütekriterien des Metakognitionsfragebogens (Kurzversion, MKF-30) [Psychometric properties of the short German version of the Metacognitions Questionnaire (MKF-30)]. Zeitschrift für Klinische Psychologie und Psychotherapie. 2011; 40:107–14.
- 59.
Möbius J, Hoyer J. Metakognitionsfragebogen (MKF) [Metacognitions Questionnaire]. In: Hoyer J, Margraf J, editors. Angstdiagnostik: Grundlagen und Testverfahren. Berlin: Springer; 2003. pp. 220–3.
- 60. Freeston MH, Rhéaume J, Letarte H, Dugas MJ, Ladouceur R. Why do people worry. Pers Individ Dif. 1994; 17:791–802.
- 61. Gerlach AL, Andor T, Patzelt J. Die Bedeutung von Unsicherheitsintoleranz für die Generalisierte Angststörung: Modellüberlegungen und Entwicklung einer deutschen Version der Unsicherheitsintoleranz-Skala [The significance of intolerance of uncertainty in generalized anxiety disorder: Possible models and development of a German version of the intolerance of uncertainty scale]. Zeitschrift für Klinische Psychologie und Psychotherapie. 2008; 37:190–9.
- 62.
Beck AT, Steer RA, Brown GK. Beck Depression Inventory-II (BDI-II). San Antonio: Psychological Corporation; 1996.
- 63.
Hautzinger M, Keller F, Kühner C. Das Beck Depressionsinventar II. Deutsche Bearbeitung und Handbuch zum BDI II. Frankfurt am Main: Harcourt Test Services GmbH; 2006.
- 64. Beck AT, Steer RA, Carbin MG. Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clin Psychol Rev. 1988; 8:77–100.
- 65. Wang Y-P, Gorenstein C. Psychometric properties of the Beck Depression Inventory-II: a comprehensive review. Braz J Psychiatry. 2013; 35:416–31. pmid:24402217.
- 66.
Derogatis LR. SCL-90-R: Administration, scoring and procedures manual. Baltimore: Clinical Psychometric Research; 1977.
- 67.
Franke G. SCL-90-R. Symptom-Checkliste von L. R. Derogatis: Deusche Version. Göttingen: Beltz; 2002.
- 68. Schmitz N, Hartkamp N, Kiuse J, Franke G, Reister G, Trees W. The Symptom Check-List-90-R (SCL-90-R): A German validation study. Qual Life Res. 2000; 9:185–93. pmid:10983482
- 69.
R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2021.
- 70. Van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. J Stat Softw. 2011; 45.
- 71.
Robitzsch A, Grund S. miceadds: Some Additional Multiple Imputation Functions, Especially for ’mice’. R package version 3.12–26. 2022. https://CRAN.R-project.org/package=miceadds.
- 72. Little RJA. A Test of Missing Completely at Random for Multivariate Data with Missing Values. J Am Stat Assoc. 1988; 83:1198–202.
- 73.
Heymans MW, Eekhout I. Applied Missing Data Analysis With SPSS and (R)Studio. Amsterdam: 2019. https://bookdown.org/mwheymans/bookmi/.
- 74.
Van Buuren S. Flexible imputation of missing data. Boca Raton: CRC press; 2018.
- 75.
Enders CK. Applied missing data analysis. New York: Guilford Press; 2010.
- 76. Enders CK. Multiple Imputation as a Flexible Tool for Missing Data Handling in Clinical Research. Behav Res Ther. 2017; 98:4–18. pmid:27890222
- 77.
Rubin DB. Introduction and summary of repeated-imputation inferences. In: Rubin DB, editor. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, Inc; 1987. pp. 1–26.
- 78. Vallejo G, Fernández MP, Rosário P. Combination rules for homoscedastic and heteroscedastic MANOVA models from multiply imputed datasets. Behav Res Methods. 2021; 53:669–85. pmid:32804343.
- 79. Van Ginkel JR, Linting M, Rippe RCA, Van der Voort A. Rebutting Existing Misconceptions About Multiple Imputation as a Method for Handling Missing Data. J Pers Assess. 2020; 102:297–308. pmid:30657714.
- 80.
Field AP, Miles J, Field Z. Discovering statistics using R. London: SAGE Publications; 2012.
- 81.
Tabachnick BG, Fidell LS, Ullman JB. Using multivariate statistics. Boston: Pearson; 2019.
- 82. Friedrich S, Konietschke F, Pauly M. Resampling-Based Analysis of Multivariate Data and Repeated Measures Designs with the R Package MANOVA.RM. The R Journal. 2019; 11:380.
- 83. Friedrich S, Pauly M. MATS: Inference for potentially Singular and Heteroscedastic MANOVA. J Multivar Anal. 2018:166–79.
- 84.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Routledge; 1988.
- 85. Jacobson NS, Truax P. Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991; 19:12–9. pmid:2002127
- 86.
Champely S, Ekstrom C, Dalgaard P, Gill J, Weibelzahl, Anandkumar A, et al. ’pwr’. R package version 1.3–0. https://cran.r-project.org/web/packages/pwr/.
- 87. Wells A. Metacognitive theory and therapy for worry and generalized anxiety disorder: Review and status. J Exp Psychopathol. 2010; 1:133–145.
- 88. Dugas MJ, Ladouceur R, Léger E, Freeston MH, Langlois F, Provencher MD, et al. Group cognitive-behavioral therapy for generalized anxiety disorder: Treatment outcome and long-term follow-up. J Consult Clin Psychol. 2003; 71:821–5. pmid:12924687.
- 89. Gosselin P, Ladouceur R, Morin CM, Dugas MJ, Baillargeon L. Benzodiazepine discontinuation among adults with GAD: A randomized trial of cognitive-behavioral therapy. J Consult Clin Psychol. 2006; 74:908–19. pmid:17032095.
- 90. Rubio G, López-Ibor JJ. Generalized anxiety disorder: A 40-year follow-up study. Acta Psychiatr Scand. 2007; 115:372–9. pmid:17430415.
- 91. Asnaani A, Benhamou K, Kaczkurkin AN, Turk-Karan E, Foa EB. Beyond the Constraints of an RCT: Naturalistic treatment outcomes for anxiety-related disorders. Behav Ther. 2020; 51:434–46. pmid:32402259.
- 92. Szkodny LE, Newman MG, Goldfried MR. Clinical experiences in conducting empirically supported treatments for generalized anxiety disorder. Behav Ther. 2014; 45:7–20. pmid:24411110.
- 93. McLeod J. An administratively created reality: Some problems with the use of self-report questionnaire measures of adjustment in counselling/psychotherapy outcome research. Couns Psychother Res. 2001; 1:215–26.
- 94. Cuijpers P, Sijbrandij M, Koole S, Huibers M, Berking M, Andersson G. Psychological treatment of generalized anxiety disorder: A meta-analysis. Clin Psychol Rev. 2014; 34:130–40. Epub 2014/01/10. pmid:24487344.