Guided internet-delivered cognitive behavior therapy (ICBT) has been tested in several trials on social anxiety disorder (SAD) with moderate to large effects. The aims of this study were threefold. First, to compare the effects of ICBT including online discussion forum with a moderated online discussion forum only. Second, to investigate if knowledge about SAD increased following treatment and third to compare the effects of inexperienced versus experienced therapists on patient outcomes.
A total of 204 participants with a primary diagnosis of SAD were included and randomized to either guided ICBT or the control condition. ICBT consisted of a 9-week treatment program which was guided by either psychology students at MSc level (n = 6) or by licensed psychologists with previous experience of ICBT (n = 7). A knowledge test dealing with social anxiety was administered before and after treatment. Measures of social anxiety and secondary outcomes dealing with general anxiety, depression, and quality of life were administered before and after treatment. In addition, a 1-year follow-up was conducted on the treated individuals.
Immediately following treatment, the ICBT group showed superior outcome on the Liebowitz Social Anxiety Scale self-report version with a between group posttreatment Hedges g effect size of g = 0.75. In addition, significant differences on all the secondary outcomes were observed. Gains were well maintained one year later. Knowledge, as assessed by the knowledge test, increased following treatment with little gain in the control group. Therapist experience did not result in different outcomes, but experienced therapists logged in less frequently compared to the inexperienced therapists, suggesting that they needed less time to support patients.
We conclude that guided ICBT reduce symptoms of SAD, increase knowledge about SAD and that therapist experience does not make a difference apart from the finding that experienced therapist may require less time to guide patients.
Citation: Andersson G, Carlbring P, Furmark T, on behalf of the S. O. F. I. E. Research Group (2012) Therapist Experience and Knowledge Acquisition in Internet-Delivered CBT for Social Anxiety Disorder: A Randomized Controlled Trial. PLoS ONE 7(5): e37411. https://doi.org/10.1371/journal.pone.0037411
Editor: Leonardo Fontenelle, Institute of Psychiatry at the Federal University of Rio de Janeiro, Brazil
Received: January 27, 2012; Accepted: April 18, 2012; Published: May 23, 2012
Copyright: © 2012 Andersson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was sponsored in part by a grant from the Swedish Research Council to Dr Furmark, and a grant from Linköping University to Dr Andersson. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: Co-author Gerhard Andersson is a PLoS ONE Editorial Board member. This does not alter the authors’ adherence to all the PLoS ONE policies on sharing data and materials.
Internet-based cognitive-behavior therapy (ICBT) was developed in the late 1990’s , , and has been investigated in a large number of randomized controlled trials as attested by systematic reviews and meta-analyses on anxiety disorders , mood disorders , and somatic health conditions . Social anxiety disorder (SAD), often referred to as social phobia , is a condition for which strong empirical support exists regarding standard CBT provided in groups or individually , , . There is also some evidence that SAD can be treated with bibliotherapy , , , but more studies have been conducted on ICBT. Indeed, following the first ICBT trial , and further later studies conducted by our group , , , , , the findings have been replicated by three separate research groups with one in Australia , , , , , , and two additional groups in Europe , , , totaling at least 15 controlled trials. Interestingly, stable long-term effects have also been found up to five years after treatment completion , . In addition, there is evidence to suggest that the treatment is effective under clinically representative conditions , , so called effectiveness studies , where patients are treated in regular clinical settings with therapists who work there not only as researchers. One topic that has been investigated in several trials is the need for human support in ICBT for SAD. While the overall message from the literature suggests that support in the form of human guidance is needed to generate good outcomes in ICBT , , this is not necessarily the case in the treatment of SAD if a proper diagnosis has been established , , . Another question concerns who should provide the support. The variance explained by the therapist factor is small to non-existent in ICBT , , and Titov et al. have found that support can be provided mainly from a practical and technical point of view , , . However, a potential problem with the above mentioned trials is that few therapist have been involved and hence it is not known if inexperienced and experienced ICBT-therapists differ in terms of effects and time needed to give the support in ICBT.
Even if CBT always include psychoeduction there are very few trials in which knowledge acquisition has been directly addressed. Indeed, the treatment sessions in ICBT are often referred to as lessons , or as modules , making it motivated to ask about knowledge acquisition. One exception is a study by Scogin et al. , in which a test of knowledge regarding depression was administered before and after bibliotherapy for depression. While knowledge increased in that study, the knowledge gained was not related to improvement in depressive symptoms. On the other hand, a more recent study found that knowledge about affective disorders and treatment was predictive of outcome two years later in a study on depression .
We had three aims of this study. First, we wanted to compare guided ICBT for SAD with a moderated online discussion group instead of a pure waitlist control group. There is conflicting evidence regarding the general effectiveness of online support groups , . However, using the Internet has been perceived as helpful for some persons with SAD , and there are persons who benefit from getting support from others in the same situation and even advice regarding treatment, including therapeutic advice on how to handle anxiety. Moreover, participation in an online discussion group is distinctly different from pure waiting. In line with the previous results we expected guided ICBT to be superior to participation in a moderated discussion group. Second, we investigated if knowledge about SAD and its treatment components was influenced by our treatment. We hypothesized that knowledge would increase but we did not have a clear hypothesis regarding the correlation between knowledge and clinical improvement. Third, we wanted to randomly assign participants to either inexperienced or experienced therapists from whom support was provided during the treatment. Again we did not have a clear hypothesis, as previous studies have not found any major differences between categories of therapist/support persons in guided ICBT. However, we also investigated the number of log ins (to a secure messaging system) needed to give the support, as we expected that experienced therapists would spend less time with their patients.
The study protocol and supporting CONSORT checklist for this trial are available as supporting information; see Checklist S1 and Protocol S1. This was a superiority trial within the context of a parallel group study with blocked randomization in 1∶1 ratio. Outcome assessors were blind to treatment status.
The trial was approved by the regional ethics committee at Uppsala University, Sweden. The trial is registered at University Hospital Medical Information Network (http://www.umin.ac.jp/) UMIN000001383.
Recruitment and Selection
The recruitment procedure was largely similar to our previous studies on ICBT for SAD , , . Participants were recruited via a research web page (www.studie.nu), which had been advertised by mass media in Sweden. A web portal including online questionnaires and a secure email service was used in the trial. The system handles security issues with two factor authentication in order to decrease the probability that the requestor is presenting false evidence of its identity. On the study web page, information about the trial was presented. Potential participants were advised to complete an application form and an online screening battery consisting of the Social Phobia Screening Questionnaire (SPSQ) , the self-rated version of the Montgomery Åsberg Depression Rating Scale (MADRS-S) , and additional questions regarding current and past treatments including medication. Persons who passed this initial step were contacted for a telephone interview, and if not suitable given advice and a reason why they were not included. To be included, they had to meet the following criteria: (a) a DSM-IV  diagnosis of SAD according to the SPSQ; (b) scoring <31 on the MADRS-S depression scale and <4 on the suicide item of this scale to prevent the inclusion of individuals in strong need of specialist consultation; (c) not undergoing any other psychological treatment during the study period; (d) if on prescribed medication for anxiety/depression, dosage had to be constant for 3 months before the treatment onset and kept constant throughout the study; (e) being at least 18 years old; (f) living in Sweden; (g) having access to a computer with Internet connection; (h) not admitting another serious or dominant disorder (e.g. psychosis, substance misuse) that could be expected to influence the outcome of the study; and (i) a primary diagnosis of SAD according to the Structured Clinical Interview for DSM-IV, SCID-I . The last criterion was evaluated by a telephone-interview in which the diagnostic questions from the SAD section of the SCID-I were posed. When a person failed to meet the inclusion criteria an individual electronic message was sent with advice on where to seek more appropriate help.
Of the 365 individuals who applied to participate, 204 individuals meeting all inclusion criteria were eventually randomized to either treatment (n = 102) or a control condition (n = 102). Randomization was performed by an independent third-party using an online true random-number service (www.random.org). The control group received delayed treatment after 9 weeks and the outcome of their treatment will not be reported here. Eight participants in the treatment group and 2 in the control group did not complete posttreatment data yielding a 5% dropout. In accordance with the intention-to-treat principle, all participants were asked to complete posttreatment and follow-up assessments, regardless of how many treatment modules they had completed and all were included in the analyses. One-year follow-up data were not provided by 10 subjects in the treatment group (10.6%). Written informed consent was obtained. An overview of the procedure is given in Figure 1. Patient characteristics are presented in Table 1.
Four social anxiety questionnaires were used as outcome measures: the Liebowitz Social Anxiety Scale self-report version (LSAS-SR) , , the Social Phobia Scale (SPS) , the Social Interaction Anxiety Scale (SIAS) , and the SPSQ . We regarded the LSAS-SR as the primary outcome measure . In addition to the questionnaires administered at pre-, post-, and follow-up assessments, participants in the treatment group completed the LSAS-SR online every week (Sundays). In case of missing data, a brief and neutral reminder was sent 24 hours later via e-mail, and if necessary, followed by another reminder sent as an auto-generated short-text-message (SMS) to the person’s mobile phone. The response rate on these weekly assessments ranged between 80–100% across modules and the last previous LSAS-SR score was used to replace missing data.
General anxiety, depression and quality of life.
We developed a knowledge test dealing with the condition of SAD and its treatment. The test construction involved selection of test items, consulting experts in the field of knowledge tests and social anxiety, and pilot testing with psychology students (n = 24) and a second pilot testing (n = 18 students) after revisions based on feedback from students and experts. The final test included 11 items with multiple choice response options (three options). In addition, each response was rated in terms of how certain the participant was about the response with three response options (Guessing, Pretty certain, Confident or convinced). A higher score indicate more knowledge. Items included were: 1. What is the core problem in SAD; 2. One important rule in treatment of phobias is…; 3. What is a safety behavior?; 4. Which one of these alternatives is a typical automatic thought?; 5. Which one of these alternatives characterizes a thought trap?; 6. What is the most important reason for defining treatment goals in CBT?; 7. What is a core belief according to CBT?; 8. What is exposure?; 9. One component in CBT for SAD is called “shifting focus”, what does that stand for?; 10. Which technique is recommended in CBT if you want to express your dissatisfaction to someone?; 11. Which one of these alternatives is the main reason why it can be difficult to get rid of SAD? We scored the knowledge test in two ways. First we calculated a total score based on total number of correct answers. Second we calculated a weighted total score in which certainty of answers were factored in. Basically this meant a higher score if you were correct and certain, a lower score if you were right, yet uncertain, and finally a negative score if you were wrong but certain. Reliability analyses showed a low Cronbach´s alpha of α = .40 for the raw scores, a high Cronbach´s alpha of α = .86 for the certainty ratings, and an alpha of α = .56 for the weighted scores.
Administration of Self-report Measures
All self-report instruments were administrated via the Internet at pretreatment (baseline), posttreatment and one-year follow up. Adequate psychometric properties have previously been demonstrated for Internet-administered questionnaires relating to SAD , with Cronbach´s alpha values ranging between α = .89 to α = .94 for the SAD measures, and α = .81 to α = .89 for the secondary outcome measures used in this study.
Global Functioning and Improvement
A telephone interview was conducted using the Clinical Global Impression Improvement Scale (CGI-I)  to measure global improvement after the treatment period. Outcome assessors were not aware of treatment status before the interview.
All participants had to have access to a computer with an Internet connection, a web browser and the ability to print out files in PDF format. Participants were recommended to get a free online e-mail service that automatically encrypts messages in 2048 bits instead of using their personal e-mail. However, this was just used for reminders to log in to the secure messaging system.
The main treatment component was our previously evaluated self-help manual for SAD , which consists of 186 pages divided into nine chapters (modules) adapted for use via the Word Wide Web. In brief, the manual starts with an introductory module describing SAD and facts about CBT. Modules 2–4 describe a cognitive model for SAD and introduce cognitive restructuring. Modules 5–7 introduce exposure and attention shifting exercises. Modules 8–9 mainly concern social skills and relapse prevention. Participants were asked to complete one module every week, i.e. a 9-week treatment period was recommended. Each module consisted of information, exercises (home-work assignments) and ended with a short quiz to check adherence. Feedback was given each week by the therapists. Text messages were used to remind participants to log in or to complete weekly reports on the LSAS-SR.
Online Discussion Forum
Participants in both groups had access to separate moderated online discussion forums. For each week, participants were asked to post at least one message in the discussion group about a new but predetermined topic. Discussions were monitored but the study personnel did not take active part in them unless it was needed. The study team posted the topics for discussion (e.g., “What are your experiences of seeking help for SAD”), and were ready to comment if questions were asked directly to the study team or if discussions would be seen as inappropriate (e.g., negative comments about other participants, expression of suicidal intent, etc). The latter did not occur. The discussion group was open during the whole study period. Directly following the waiting period for the online discussion group, the participants commenced the same Internet-delivered self-help treatment as for the ICBT group.
The treatment group had access to an Internet therapist during the 9 week treatment period. E-mail correspondence occurred weekly (Sundays) and generally concerned the results of homework assignments as described in the self-help manual. The rationale behind the homework assignments was to promote learning and enable the Internet therapists to decide whether the participants had assimilated the information and completed their exercises. In general, therapist feedback on the homework assignment was provided within 24 hours after the participant had submitted a massage. We collected data on how often the therapist logged in, which is a proxy of how much time they devoted to the participant. Participants could ask questions all week and receive a response, but feedback on progress was mainly given after the Sunday deadline. We did not register how long the therapists were in the system, but experienced therapists were more likely to handle all participants once they were in the system, whereas inexperienced therapists (who were students) had more time to check the system frequently and spend more time on each participant. Moreover, therapists were instructed to only be in the system when responding and giving feedback, and not stay in the system when doing other things. When the homework was completed, the next module was made accessible. Alternatively, an instruction on what needed to be completed to proceed to the next module was sent to the participant. On average, Internet therapists spent 15 min per week for each participant reading messages and providing feedback via the online contact handling system.
There were 13 Internet-therapists in the study, 7 of which were licensed clinical psychologists with an average or 3 years of clinical experience (range 2–6 years), and 6 clinical psychology students in their last semester of the five years master’s degree program. All therapists had completed a basic training in CBT including supervised face-to-face therapies. The students (i.e., inexperienced therapists) had clinical supervision during the trial. Participants were randomly allocated to their therapist, with the restriction that each therapist only could be allocated a limited number of participants. The experienced therapists had worked with previous Internet trials on anxiety and mood disorders. There were four men and three women among the experienced therapists and one man and five women among the student therapists. The average age of the experienced therapists was M = 28.9 years (SD = 2.4). The corresponding age among the student therapists was M = 29.6years (SD = 9.0; mainly caused by one of the student therapist being older). The student therapists treated 60 participants (10 each) and the experienced therapists 42 participants (6 each).
Sample Size and Power
The trial was originally powered to investigate genetic effects (see Protocol S1), and we aimed to include 250 participants, with 125 in each arm. However, we were not able to recruit the full sample which required consent to genetic testing. For the present report and in light of the previous studies on ICBT for SAD (see introduction), a moderate between group effect size of g = 0.50 would require at least 32 participants in each group, with an alpha at 05 and a power of 80%. The study was thus sufficiently powered for detecting treatment effects. The power for the difference between experienced and inexperienced therapist could not be based on previous studies, and if we assume a small difference (g = 0.20), a much larger sample would have been required (at least 160 in each group).
Statistical Analyses and Clinical Significance
We used a mixed models approach to analyze the data with full information maximum likelihood estimation . Linear mixed effect models are able to accommodate missing data and integrate time-varying factors. It has been recommended to use mixed models analyses as a way to handle intention to treat data . However, one assumption is that the lost data is missing at random and not non-ignorable. We made this assumption as no obvious pattern was observed for the missing data and the actual loss of data was relatively small. We used an unstructured covariance structure for the analyses. An unstructured covariance structure has the property which means that the correlations between measurements at different time points are allowed to vary. Hence, the correlation between the Pre vs. Post measurement and post vs. 1-year follow-up did not necessarily have to be constant.
Clinically significant improvement was determined for the LSAS-SR in accordance with Jacobson and Truax criteria  by using the Reliable Change Index for each individual and a post-test score within two standard deviations (SDs) of the mean of the normal population . Chi-square was used to test distribution differences with regard to clinically significant improvement and demographic/descriptive characteristics.
Randomization Check, Attrition and Treatment Completion
There were no statistically significant differences between the treatment and the control groups with regards to demographics (Table 1) or pretreatment self-report scores (Tables 2 and 3). The average number of completed modules in the treatment group was 6.8 (SD = 3.07) out of a total of nine. In total there were 46/102 (45%) participants who failed to complete all nine modules during the nine week treatment period. The activity in the online discussion forums varied in line with previous investigations , which means that a majority made few comments and postings in addition to the postings linked to the “topic of the week”, some were more active with discussions, and some mainly read were passive.
Treatment Effects – Primary Outcome
On the main outcome measure LSAS-SR, a large interaction effect was identified (Table 2), with a between group posttreatment Hedges g effect size of g = 0.75 (95% CI = 0.46–1.03). Results for the weekly measurement points are presented in Figure 2 for the treatment group. A linear trend was seen as a decrease in symptoms over the weeks in treatment. One year follow-up data were collected for the treatment group. On the LSAS-SR scores remained improved compared to pretreatment (M = 40.39, SD = 23.6). The change from posttreatment to one year follow-up was in the direction of further improvement, albeit not significant (t 89 = 1.92, p = .058).
The number of participants meeting the criteria for clinically significant improvement was calculated for the LSAS-SR. In the treatment group, 45.1% (n = 46/102; CI 95% 35.3%–54.9%) reached this criterion versus 11.8% (n = 12/102; CI 95% 5.4%–18.1%) in the online discussion forum control group. The difference was statistically significant χ2(1) = 27.8, p<.001.
Treatment Effects – Secondary Outcomes
As evident from Table 2, statistically significant interactions were found for the three additional measures of social anxiety symptoms; the SPS (p<.001), the SIAS (p<.001), and the SPSQ (p<.001). Improvements, as evidenced by significant interaction effects, were also identified on measures of general anxiety (BAI, p<.001), depression (MADRS-S, p<.001), and quality of life (QOLI, p<.01).
Between group effect sizes at posttreatment ranged between a low of g = 0.25 for the QOLI and a high of g = 0.72 for SIAS. One year follow-up data for these measures are presented in Table 4 (separated for the two types of therapist experience). On all secondary outcome measures the changes obtained were retained or slightly improved at follow-up.
Treatment Effects – Knowledge Test
Results for the knowledge test are presented in Table 3. Both the raw scores and the weighted scores improved following ICBT, but not in the control group (both interactions highly significant). As can be inferred from Table 3, the treatment group became more certain of their answers to the questions. Improvement on the weighted knowledge test was not associated with improvement on the primary outcome measure LSAS-SR, but did correlate with improvement on the SPSQ (Pearson’s r = .26, p = .01) and on the SPS (Pearson’s r = .23, p = .03).
At post-treatment, 36 participants (35.3%) in the treatment group were classified as very much improved or much improved according to the independent CGI-I (95% CI, 25.9%–44.7%). In the control group, the corresponding number of participants was 6 (14.3%) as assessed by the CGI-I (95% CI, 0.6%–9.2%). The difference was statistically significant χ2(1) = 26.9, p<.001. Adverse events, defined as a CGI-I score of 5 (minimally worse) was found in one case in the treatment group and six cases in the control group. The deteriorations observed could not be linked to the treatment and were rather a worsening of their SAD and related problems.
Effects of Therapist Experience
The third aim of the study was to investigate if previous therapist experience would make a difference. Results are presented in Table 4. There were no statistically significant interactions suggesting that the two categories of therapist experience produced equal outcomes. On the primary outcome measure LSAS-SR the within-group pre-post Hedges g effect size was g = 0.98 (CI 95% 0.56–1.39) for the experienced group and g = 1.06 (CI 95% 0.63–1.47) for the inexperienced group, yielding very similar outcomes. However, experienced therapists logged in less frequently. Mean number of log-ins was 25.5 (SD = 12.5) as compared to the inexperienced therapists 33.01 (SD = 13.4). This difference was statistically significant (t100 = 2.84, p = .005).
This trial investigated the effects of ICBT for persons with a DSM-IV diagnosis of SAD and compared the effects of treatment with being part of an online discussion forum. The trial also investigated if knowledge about social anxiety increased with treatment and if novice and experienced therapists were equally effective when guiding the treatment.
Is ICBT for SAD Effective?
The first aim concerned testing guided ICBT against a discussion forum control group. Results clearly showed that the active treatment was superior. The between group effect size was large (g = 0.75), and in line with a range of previous studies on guided ICBT for SAD . Moreover, the findings are consistent with a range of previous studies showing that guided ICBT is an effective treatment for anxiety disorders and that it can be equally effective as face-to-face CBT , , . We also found that the effects were sustained at one year follow-up, which is in line with previous studies , .
Does Knowledge Increase Following ICBT?
The second aim was to determine if ICBT increases knowledge about SAD and its treatment. To be able to answer this question a new test was developed. Knowledge increased in ICBT but not in controls. Small but statistically significant associations between knowledge gain and outcome were found. There is a surprising scarcity of CBT studies investigating knowledge gains, even if there are studies on mental health literacy . This lack of knowledge calls for more research given the role of psychoeducation in CBT in general and in ICBT and bibliotherapy in particular, as the latter involve reading text material and grasping instructions presented over the internet or in text.
Does it Matter if the Therapist is Experienced?
The third aim was to investigate the effects of therapist experience. While there are previous studies suggesting that guidance can be given from a practical and technical point of view , , , this is the first and probably the largest ICBT study for SAD in which therapists with less or more previous experience of ICBT have been randomly allocated to patients. The findings suggest that ICBT does not require experienced therapists to be effective. However, it should be noted that all therapists had been trained in CBT. Moreover, the highly structured ICBT protocol leave less room for therapist effects and it is possible that therapist experience would have been more important in a less structured treatment. However, in a previous trial by our group on depression, inexperienced therapist were found to be effective when the treatment was in the form of e-mail and not structured in advance . Interestingly, the more experienced therapist logged in less often which means they spent less time following the participants. There are suggestions that experienced therapist may drift from treatment protocols , but is probably less likely to make a difference in ICBT as the treatment content remains the same.
The first limitation of the present study was that we recruited a sample via advertisements and not from a clinic. This limits the generalizability of the findings even if previous studies have indicated that ICBT for SAD is effective in regular clinical settings with similar outcomes as in studies with recruitment from the general public , . In addition, we did not see the participants in a live interview, but again this has not been found to yield different patients characteristics when compared with studies on samples who have been assessed in-vivo . The mean scores on the SAD measures at baseline were similar to previous studies , , , including an effectiveness study .
A second limitation concerns the control group and the use of a moderated online discussion forum. We did not check the activity in the discussion forum. It is known that far from all are active in such forums and that many may only watch and not post messages.
themselves . While online discussion forums have been found to be beneficial in some studies , it is not well known if this is beneficial for persons with SAD even if no effects were found in a previous study . We did not collect treatment credibility ratings and the control condition was most likely not perceived as an intervention for SAD.
A third limitation is that the knowledge test was developed and validated in association with this study and not independently. While the controlled design is a strength it is still possible that the knowledge test does not measure the most important aspects of SAD. In addition, the gains were modest in terms of raw score increases, but when we incorporated a rating of how certain the participant was about the response larger differences emerged. The clinical relevance of a modest gain in knowledge can be questioned even if certainty of knowledge may be more clinically relevant, However, certainty of knowledge without consideration of how correct that knowledge is would make no sense and hence we believe that the weighted scores are clinically relevant. We are also aware of the fact that the knowledge test had low internal consistency for the raw scores. This leads to lower statistical power and probably reflect the fact that the test was relatively easy and that certainty about the answers (with good reliability) was the factor we influenced by the treatment.
A fourth limitation concerns the therapists who were either categorized as experienced or inexperienced, which is a categorization that can be questioned. All therapists had basic training in CBT and the difference that emerged relating to the time taken to handle participants may represent a rapid learning curve. It is possible that relatively little experience is needed to learn how to guide clients in ICBT with much structure. In addition, while we had sufficient power to detect main effects of treatment the power to detect small effects of therapist experience was not sufficient. Moreover, our sample of persons with SAD and relatively high educational background may not be representative for patients seen in other settings for whom more experienced therapists may be needed to guide the programs.
The mechanisms of change in ICBT for SAD are not well known, even if it is possible that cognitive aspects are involved as has been found in face-to-face CBT . The ICBT tested in this trial includes both cognitive and behavioral components, but is based on a CBT model that underscores the importance of attention focused on the self, safety behaviors, and beliefs about social situations . Mediators of change in ICBT for SAD have not been investigated and the specificity of the findings can be questioned as other treatments such as applied relaxation  and interpersonal psychotherapy  also lead to reduced symptoms even if they may be less effective than CBT.
The present study was the first to test if ICBT increased knowledge about SAD. This is an important topic as CBT incorporates psychoeducation with the aim to increase knowledge. However, correlations between knowledge gain and improvement on measures of SAD were small and it is likely that knowledge alone is not enough to reduce symptoms of SAD. Future research is needed to validate tests of knowledge regarding SAD and other conditions as it is unclear if lack of knowledge predisposes persons to develop and sustain SAD.
In spite of the limitation that the inexperienced therapists were trained psychologists, our study adds to the literature showing that guidance does not require much therapist experience , , . In the previous studies only a few therapists have been involved. In contrast, the present study had several therapists being randomly assigned to guide the participants during treatment. More studies are needed to investigate the lower limits of competence in guided ICBT and also the conditions under which more expertise is called for.
There are now several controlled trials showing that ICBT for SAD is effective. The present trial revealed that ICBT is better than only being offered to participate in an online discussion forum and that the effects are maintained one year later. Further, the study demonstrated that amount of previous therapist experience does not make a difference in terms of outcome but that experienced therapists may need less time to guide clients through the treatment. Finally, this is the first study to show that knowledge about social anxiety and its management actually increases with treatment.
Members of the SOFIE 6 research group
The SOFIE 6 research group consists of
Project Investigators and Managers
Tomas Furmark (Uppsala University), Gerhard Andersson (Linköping University and Karolinska Institute), and Per Carlbring (Umeå University)
Lisa Ekselius (Uppsala University)
Test Developers and Internet Therapists
Christina Bertholds Felix, Lisa Lundborg (at Uppsala University)
Angelica Rosenström, John Josserand, Kerstin Larm, Lina Sagerlöv Gunnarsson, Sara Hammer, Ika Wahlgren, Cecilia Olsson Lynch, Kajsa Sundling, Sara Engdahl, Sandra Gustafsson, Ann-Sofie Ersson and Emma Grut (at Uppsala University)
Jonas Almöv, Emma Linna, Linda Maurin and Charlotta Törngren (at Linköping University)
Jonas Bjärehed (at Lund University)
Mats Dahlin and Fredrik Malmeus (at Psykologpartners private practice)
Conceived and designed the experiments: TF GA PC. Performed the experiments: TF GA PC. Analyzed the data: PC GA. Wrote the manuscript: GA PC TF.
- 1. Andersson G (2009) Using the internet to provide cognitive behaviour therapy. Behav Res Ther 47: 175–180.G. Andersson2009Using the internet to provide cognitive behaviour therapy.Behav Res Ther47175180
- 2. Barak A, Klein B, Proudfoot JG (2009) Defining Internet-supported therapeutic interventions. Ann Behav Med 38: 4–17.A. BarakB. KleinJG Proudfoot2009Defining Internet-supported therapeutic interventions.Ann Behav Med38417
- 3. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N (2010) Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 5: e13196.G. AndrewsP. CuijpersMG CraskeP. McEvoyN. Titov2010Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis.PLoS One5e13196
- 4. Andersson G, Cuijpers P (2009) Internet-based and other computerized psychological treatments for adult depression: A meta-analysis. Cogn Behav Ther 38: 196–205.G. AnderssonP. Cuijpers2009Internet-based and other computerized psychological treatments for adult depression: A meta-analysis.Cogn Behav Ther38196205
- 5. Cuijpers P, van Straten A-M, Andersson G (2008) Internet-administered cognitive behavior therapy for health problems: A systematic review. J Behav Med 31: 169–177.P. CuijpersA-M van StratenG. Andersson2008Internet-administered cognitive behavior therapy for health problems: A systematic review.J Behav Med31169177
- 6. American Psychiatric Association (2000) Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Press. Association American Psychiatric2000Diagnostic and statistical manual of mental disorders.Washington, DCAmerican Psychiatric Press
- 7. Hofmann SG, Smits JA (2008) Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials. J Clin Psychiatry 69: 621–632.SG HofmannJA Smits2008Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials.J Clin Psychiatry69621632
- 8. Ponniah K, Hollon SD (2008) Empirically supported psychological interventions for social phobia in adults: a qualitative review of randomized controlled trials. Psychol Med 38: 3–14.K. PonniahSD Hollon2008Empirically supported psychological interventions for social phobia in adults: a qualitative review of randomized controlled trials.Psychol Med38314
- 9. Acarturk C, Cuijpers P, van Straten A, de Graaf R (2009) Psychological treatment of social anxiety disorder: a meta-analysis. Psychol Med 39: 241–254.C. AcarturkP. CuijpersA. van StratenR. de Graaf2009Psychological treatment of social anxiety disorder: a meta-analysis.Psychol Med39241254
- 10. Rapee RM, Abbott MJ, Baillie AJ, Gaston JE (2007) Treatment of social phobia through pure self-help and therapist-augmented self-help. Br J Psychiatry 191: 246–252.RM RapeeMJ AbbottAJ BaillieJE Gaston2007Treatment of social phobia through pure self-help and therapist-augmented self-help.Br J Psychiatry191246252
- 11. Abramowitz JS, Moore EL, Braddock AE, Harrington DL (2009) Self-help cognitive-behavioral therapy with minimal therapist contact for social phobia: A controlled trial. J Behav Ther Exp Psychiatry 40: 98–105.JS AbramowitzEL MooreAE BraddockDL Harrington2009Self-help cognitive-behavioral therapy with minimal therapist contact for social phobia: A controlled trial.J Behav Ther Exp Psychiatry4098105
- 12. Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, et al. (2009) Guided and unguided self-help for social anxiety disorder: randomised controlled trial. Br J Psychiatry 195: 440–447.T. FurmarkP. CarlbringE. HedmanA. SonnensteinP. Clevberger2009Guided and unguided self-help for social anxiety disorder: randomised controlled trial.Br J Psychiatry195440447
- 13. Andersson G, Carlbring P, Holmström A, Sparthan E, Furmark T, et al. (2006) Internet-based self-help with therapist feedback and in-vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol 74: 677–686.G. AnderssonP. CarlbringA. HolmströmE. SparthanT. Furmark2006Internet-based self-help with therapist feedback and in-vivo group exposure for social phobia: a randomized controlled trial.J Consult Clin Psychol74677686
- 14. Carlbring P, Gunnarsdóttir M, Hedensjö L, Andersson G, Ekselius L, et al. (2007) Treatment of social phobia: randomized trial of internet delivered cognitive behaviour therapy and telephone support. Br J Psychiatry 190: 123–128.P. CarlbringM. GunnarsdóttirL. HedensjöG. AnderssonL. Ekselius2007Treatment of social phobia: randomized trial of internet delivered cognitive behaviour therapy and telephone support.Br J Psychiatry190123128
- 15. Tillfors M, Carlbring P, Furmark T, Lewenhaupt S, Spak M, et al. (2008) Treating university students with social phobia and public speaking fears: Internet delivered self-help with or without live group exposure sessions. Depression Anxiety 25: 708–717.M. TillforsP. CarlbringT. FurmarkS. LewenhauptM. Spak2008Treating university students with social phobia and public speaking fears: Internet delivered self-help with or without live group exposure sessions.Depression Anxiety25708717
- 16. Hedman E, Andersson G, Ljótsson B, Andersson E, Rück C, et al. (2011) Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: A randomized controlled non-inferiority trial. PloS ONE 6: e18001.E. HedmanG. AnderssonB. LjótssonE. AnderssonC. Rück2011Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: A randomized controlled non-inferiority trial.PloS ONE6e18001
- 17. Tillfors M, Andersson G, Ekselius L, Furmark T, Lewenhaupt S, et al. (2011) A randomized trial of Internet delivered treatment for social anxiety disorder in high school students. Cogn Behav Ther 40: 147–157.M. TillforsG. AnderssonL. EkseliusT. FurmarkS. Lewenhaupt2011A randomized trial of Internet delivered treatment for social anxiety disorder in high school students.Cogn Behav Ther40147157
- 18. Titov N, Andrews G, Schwencke G (2008) Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 42: 595–605.N. TitovG. AndrewsG. Schwencke2008Shyness 2: treating social phobia online: replication and extension.Aust N Z J Psychiatry42595605
- 19. Titov N, Andrews G, Schwencke G, Drobny J, Einstein D (2008) Shyness 1: distance treatment of social phobia over the Internet. Aust N Z J Psychiatry 42: 585–594.N. TitovG. AndrewsG. SchwenckeJ. DrobnyD. Einstein2008Shyness 1: distance treatment of social phobia over the Internet.Aust N Z J Psychiatry42585594
- 20. Titov N, Andrews G, Choi I, Schwencke G, Mahoney A (2008) Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 42: 1030–1040.N. TitovG. AndrewsI. ChoiG. SchwenckeA. Mahoney2008Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia.Aust N Z J Psychiatry4210301040
- 21. Titov N, Andrews G, Schwencke G, Solley K, Johnston L, et al. (2009) An RCT comparing the effects of two types of support on severity of symptoms for people completing Internet-based cognitive behaviour therapy for social phobia. Aust N Z J Psychiatry 43: 920–926.N. TitovG. AndrewsG. SchwenckeK. SolleyL. Johnston2009An RCT comparing the effects of two types of support on severity of symptoms for people completing Internet-based cognitive behaviour therapy for social phobia.Aust N Z J Psychiatry43920926
- 22. Titov N, Andrews G, Schwencke G, Robinson E, Peters L, et al. (2010) Randomized controlled trial of Internet cognitive behavioural treatment for social phobia with and without motivational enhancement strategies. Aust N Z J Psychiatry 44: 938–945.N. TitovG. AndrewsG. SchwenckeE. RobinsonL. Peters2010Randomized controlled trial of Internet cognitive behavioural treatment for social phobia with and without motivational enhancement strategies.Aust N Z J Psychiatry44938945
- 23. Titov N, Andrews G, Choi I, Schwencke G, Johnston L (2009) Randomized controlled trial of web-based treatment of social phobia without clinical guidance. Aust N Z J Psychiatry 43: 913–919.N. TitovG. AndrewsI. ChoiG. SchwenckeL. Johnston2009Randomized controlled trial of web-based treatment of social phobia without clinical guidance.Aust N Z J Psychiatry43913919
- 24. Berger T, Caspar F, Richardson R, Kneubühler B, Sutter D, et al. (2011) Internet-based treatment of social phobia: A randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther 48: 158–169.T. BergerF. CasparR. RichardsonB. KneubühlerD. Sutter2011Internet-based treatment of social phobia: A randomized controlled trial comparing unguided with two types of guided self-help.Behav Res Ther48158169
- 25. Botella C, Gallego MJ, Garcia-Palacios A, Guillen V, Banos RM, et al. (2010) An Internet-based self-help treatment for fear of public speaking: a controlled trial. Cyberpsychol Behav Soc Netw 13: 407–421.C. BotellaMJ GallegoA. Garcia-PalaciosV. GuillenRM Banos2010An Internet-based self-help treatment for fear of public speaking: a controlled trial.Cyberpsychol Behav Soc Netw13407421
- 26. Berger T, Hohl E, Caspar F (2009) Internet-based treatment for social phobia: A randomized controlled trial. J Clin Psychol 65: 1021–1035.T. BergerE. HohlF. Caspar2009Internet-based treatment for social phobia: A randomized controlled trial.J Clin Psychol6510211035
- 27. Carlbring P, Bergman Nordgren L, Furmark T, Andersson G (2009) Long term outcome of Internet delivered cognitive-behavioural therapy for social anxiety disorder: A 30-month follow-up. Behav Res Ther 47: 848–850.P. CarlbringL. Bergman NordgrenT. FurmarkG. Andersson2009Long term outcome of Internet delivered cognitive-behavioural therapy for social anxiety disorder: A 30-month follow-up.Behav Res Ther47848850
- 28. Hedman E, Furmark T, Carlbring P, Ljótsson B, Rück C, et al. (2011) Five-year follow-up of internet-based cognitive behaviour therapy for social anxiety disorder. (2).JMIR 13. e39 p.E. HedmanT. FurmarkP. CarlbringB. LjótssonC. Rück2011Five-year follow-up of internet-based cognitive behaviour therapy for social anxiety disorder.2JMIR 13e39
- 29. Aydos L, Titov N, Andrews G (2009) Shyness 5: the clinical effectiveness of Internet-based clinician-assisted treatment of social phobia. Australas Psychiatry 17: 488–492.L. AydosN. TitovG. Andrews2009Shyness 5: the clinical effectiveness of Internet-based clinician-assisted treatment of social phobia.Australas Psychiatry17488492
- 30. Hunsley J, Lee CM (2007) Research-informed benchmarks for psychological treatments: Efficacy studies, effectiveness studies, and beyond. Professional Psychol: Res Practice 38: 21–33.J. HunsleyCM Lee2007Research-informed benchmarks for psychological treatments: Efficacy studies, effectiveness studies, and beyond.Professional Psychol: Res Practice382133
- 31. Palmqvist B, Carlbring P, Andersson G (2007) Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? Expert Rev Neurother 7: 291–297.B. PalmqvistP. CarlbringG. Andersson2007Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost?Expert Rev Neurother7291297
- 32. Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, et al. (2007) Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 37: 319–328.V. SpekP. CuijpersI. NyklicekH. RiperJ. Keyzer2007Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis.Psychol Med37319328
- 33. Almlöv J, Carlbring P, Källqvist K, Paxling B, Cuijpers P, et al. (2011) Therapist effects in guided Internet-delivered CBT for anxiety disorders. Behav Cogn Psychother 39: 311–322.J. AlmlövP. CarlbringK. KällqvistB. PaxlingP. Cuijpers2011Therapist effects in guided Internet-delivered CBT for anxiety disorders.Behav Cogn Psychother39311322
- 34. Almlöv J, Carlbring P, Berger T, Cuijpers P, Andersson G (2009) Therapist factors in Internet-delivered CBT for major depressive disorder. Cogn Behav Ther 38: 247–254.J. AlmlövP. CarlbringT. BergerP. CuijpersG. Andersson2009Therapist factors in Internet-delivered CBT for major depressive disorder.Cogn Behav Ther38247254
- 35. Robinson E, Titov N, Andrews G, McIntyre K, Schwencke G, et al. (2010) Internet treatment for generalized anxiety disorder: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One 5: e10942.E. RobinsonN. TitovG. AndrewsK. McIntyreG. Schwencke2010Internet treatment for generalized anxiety disorder: a randomized controlled trial comparing clinician vs. technician assistance.PLoS One5e10942
- 36. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, et al. (2010) Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One 5: e10939.N. TitovG. AndrewsM. DaviesK. McIntyreE. Robinson2010Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance.PLoS One5e10939
- 37. Andersson G, Estling F, Jakobsson E, Cuijpers P, Carlbring P (2011) Can the patient decide which modules to endorse? An open trial of tailored Internet treatment of anxiety disorders. Cogn Behav Ther 40: 57–64.G. AnderssonF. EstlingE. JakobssonP. CuijpersP. Carlbring2011Can the patient decide which modules to endorse? An open trial of tailored Internet treatment of anxiety disorders.Cogn Behav Ther405764
- 38. Scogin F, Jamison C, Floyd M, Chaplin WF (1998) Measuring learning in depression treatment: A cognitive bibliotherapy test. Cogn Therap Res 22: 475–482.F. ScoginC. JamisonM. FloydWF Chaplin1998Measuring learning in depression treatment: A cognitive bibliotherapy test.Cogn Therap Res22475482
- 39. Kronmuller KT, Victor D, Schenkenbach C, Postelnicu I, Backenstrass M, et al. (2007) Knowledge about affective disorders and outcome of depression. J Affect Disord 104: 155–160.KT KronmullerD. VictorC. SchenkenbachI. PostelnicuM. Backenstrass2007Knowledge about affective disorders and outcome of depression.J Affect Disord104155160
- 40. Griffiths KM, Calear AL, Banfield M (2009) Systematic review on Internet Support Groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms? J Med Internet Res 11: e40.KM GriffithsAL CalearM. Banfield2009Systematic review on Internet Support Groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms?J Med Internet Res11e40
- 41. Houston TK, Cooper LA, Ford DE (2002) Internet support groups for depression: A 1-year prospective cohort study. Am J Psychiatry 159: 2062–2068.TK HoustonLA CooperDE Ford2002Internet support groups for depression: A 1-year prospective cohort study.Am J Psychiatry15920622068
- 42. Erwina BA, Turk CL, Heimberg RG, Frescoa DM, Hantula DA (2004) The Internet: home to a severe population of individuals with social anxiety disorder? J Anx Dis 18: 629–646.BA ErwinaCL TurkRG HeimbergDM FrescoaDA Hantula2004The Internet: home to a severe population of individuals with social anxiety disorder?J Anx Dis18629646
- 43. Furmark T, Tillfors M, Stattin H, Ekselius L, Fredrikson M (2000) Social phobia subtypes in the general population revealed by cluster analysis. Psychol Med 30: 1335–1344.T. FurmarkM. TillforsH. StattinL. EkseliusM. Fredrikson2000Social phobia subtypes in the general population revealed by cluster analysis.Psychol Med3013351344
- 44. Svanborg P, Åsberg M (2001) A comparison between the Beck Depression Inventory (BDI) and the self-rating version of the Montgomery Asberg Depression Rating Scale (MADRS). J Affect Disord 64: 203–216.P. SvanborgM. Åsberg2001A comparison between the Beck Depression Inventory (BDI) and the self-rating version of the Montgomery Asberg Depression Rating Scale (MADRS).J Affect Disord64203216
- 45. First MB, Gibbon M, Spitzer RL, Williams JBW (1997) Structured clinical interview for DSM-IV Axis I Disorders (SCID-I). In: Washington , editor. D.C: American Psychiatric Press. MB FirstM. GibbonRL SpitzerJBW Williams1997Structured clinical interview for DSM-IV Axis I Disorders (SCID-I).WashingtonD.C: American Psychiatric Press
- 46. Baker SL, Heinrichs N, Kim HJ, Hofmann SG (2002) The Liebowitz Social Anxiety Scale as a self-report instrument: a preliminary psychometric analysis. Behav Res Ther 40: 701–715.SL BakerN. HeinrichsHJ KimSG Hofmann2002The Liebowitz Social Anxiety Scale as a self-report instrument: a preliminary psychometric analysis.Behav Res Ther40701715
- 47. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, et al. (2001) The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med 31: 1025–1035.DM FrescoME ColesRG HeimbergMR LiebowitzS. Hami2001The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats.Psychol Med3110251035
- 48. Mattick RP, Clarke JC (1998) Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther 36: 455–470.RP MattickJC Clarke1998Development and validation of measures of social phobia scrutiny fear and social interaction anxiety.Behav Res Ther36455470
- 49. Furmark T, Tillfors M, Everz P, Marteinsdottir I, Gefvert O, et al. (1999) Social phobia in the general population: prevalence and sociodemographic profile. Soc Psychiatry Psychiatr Epidemiol 34: 416–424.T. FurmarkM. TillforsP. EverzI. MarteinsdottirO. Gefvert1999Social phobia in the general population: prevalence and sociodemographic profile.Soc Psychiatry Psychiatr Epidemiol34416424
- 50. Moher D, Schulz KF, Altman DG, for the Consort Group (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357: 1191–1194.D. MoherKF SchulzDG Altmanfor the Consort Group2001The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.Lancet35711911194
- 51. Beck A, Epstein N, Brown G, Steer RA (1988) An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 56: 893–897.A. BeckN. EpsteinG. BrownRA Steer1988An inventory for measuring clinical anxiety: psychometric properties.J Consult Clin Psychol56893897
- 52. Svanborg P, Åsberg M (1994) A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr Scand 89: 21–28.P. SvanborgM. Åsberg1994A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale.Acta Psychiatr Scand892128
- 53. Frisch MB, Cornell J, Villanueva M, Retzlaff PJ (1992) Clinical validation of the Quality Of Life Inventory: A measure of life satisfaction for use in treatment planning and outcome assessment. Psychol Assess 4: 92–101.MB FrischJ. CornellM. VillanuevaPJ Retzlaff1992Clinical validation of the Quality Of Life Inventory: A measure of life satisfaction for use in treatment planning and outcome assessment.Psychol Assess492101
- 54. Hedman E, Ljótsson B, Rück C, Furmark T, Carlbring P, et al. (2010) Internet administration of self-report measures commonly used in research on social anxiety disorder: a psychometric evaluation Comp Human Behav 26: 736–740.E. HedmanB. LjótssonC. RückT. FurmarkP. Carlbring2010Internet administration of self-report measures commonly used in research on social anxiety disorder: a psychometric evaluation Comp Human Behav26736740
- 55. Guy W (1976) Clinical Global Impressions. In: Manual ECDEU, editor. US Dept of Health and Human Services. Rockville, MD: NIMH. pp. 217–222.W. Guy1976Clinical Global Impressions.ECDEU ManualUS Dept of Health and Human ServicesRockville, MDNIMH217222
- 56. Furmark T, Holmström A, Sparthan E, Carlbring P, Andersson G (2006) Social fobi - Effektiv hjälp med kognitiv beteendeterapi [Social phobia - effective help via CBT]. Stockholm: Liber. T. FurmarkA. HolmströmE. SparthanP. CarlbringG. Andersson2006Social fobi - Effektiv hjälp med kognitiv beteendeterapi [Social phobia - effective help via CBT].StockholmLiber
- 57. Gueorguieva R, Krystal JH (2004) Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Arch Gen Psychiatry 61: 310–317.R. GueorguievaJH Krystal2004Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry.Arch Gen Psychiatry61310317
- 58. Brown H, Prescott R (1999) Applied mixed models in medicine. New York: Oxford University Press. H. BrownR. Prescott1999Applied mixed models in medicine.New YorkOxford University Press
- 59. Jacobson NS, Truax P (1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59: 12–19.NS JacobsonP. Truax1991Clinical significance: A statistical approach to defining meaningful change in psychotherapy research.J Consult Clin Psychol591219
- 60. Andersson G, Carlbring P (2011) Social phobia (Social anxiety disorder). In: O’Donohue WT, Draper CR, editors. Stepped care and e-Health: Practical applications to behavioral disorders. New York: Springer. pp. 99–114.G. AnderssonP. Carlbring2011Social phobia (Social anxiety disorder).WT O’DonohueCR DraperStepped care and e-Health: Practical applications to behavioral disordersNew YorkSpringer99114
- 61. Kiropoulos LA, Klein B, Austin DW, Gilson K, Pier C, et al. (2008) Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT? J Anx Disord 22: 1273–1284.LA KiropoulosB. KleinDW AustinK. GilsonC. Pier2008Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?J Anx Disord2212731284
- 62. Bergström J, Andersson G, Ljótsson B, Rück C, Andréewitch S, et al. (2010) Internet- versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: A randomised trial. BMC Psychiatry 10: 54.J. BergströmG. AnderssonB. LjótssonC. RückS. Andréewitch2010Internet- versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: A randomised trial.BMC Psychiatry1054
- 63. Griffiths KM, Christensen H, Jorm AF, Evans K, Groves C (2004) Effect of web-based depression literacy and cognitive-behavioural therapy interventions on stigmatising attitudes to depression: randomised controlled trial. Br J Psychiatry 185: 342–349.KM GriffithsH. ChristensenAF JormK. EvansC. Groves2004Effect of web-based depression literacy and cognitive-behavioural therapy interventions on stigmatising attitudes to depression: randomised controlled trial.Br J Psychiatry185342349
- 64. Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, et al. (2010) Internet administered guided self-help versus individualized e-mail therapy: A randomized trial of two versions of CBT for major depression. Behav Res Ther 48: 368–376.K. VernmarkJ. LenndinJ. BjärehedM. CarlssonJ. Karlsson2010Internet administered guided self-help versus individualized e-mail therapy: A randomized trial of two versions of CBT for major depression.Behav Res Ther48368376
- 65. Waller G (2009) Evidence-based treatment and therapist drift. Behav Res Ther 47: 119–127.G. Waller2009Evidence-based treatment and therapist drift.Behav Res Ther47119127
- 66. McKenna KYA, Bargh JA (1998) Coming out in the age of the Internet: Identity “demarginalization” through virtual group participation. J Personal Soc Psychol 73: 681–694.KYA McKennaJA Bargh1998Coming out in the age of the Internet: Identity “demarginalization” through virtual group participation.J Personal Soc Psychol73681694
- 67. Hofmann SG (2004) Cognitive mediation of treatment change in social phobia. J Consult Clin Psychol 72: 392–399.SG Hofmann2004Cognitive mediation of treatment change in social phobia.J Consult Clin Psychol72392399
- 68. Clark DM, Wells A (1995) A cognitive model of social phobia. In: Heimberg RG, Leibowitz M, Hope DA, Schneider FR, editors. Social phobia: Diagnosis, assessment and treatment. New York: Guilford press. pp. 63–93.DM ClarkA. Wells1995A cognitive model of social phobia.RG HeimbergM. LeibowitzDA HopeFR SchneiderSocial phobia: Diagnosis, assessment and treatmentNew YorkGuilford press6393
- 69. Stangier U, Schramm E, Heidenreich T, Berger M, Clark DM (2011) Cognitive therapy vs interpersonal psychotherapy in social anxiety disorder: a randomized controlled trial. Arch Gen Psychiatry 68: 692–700.U. StangierE. SchrammT. HeidenreichM. BergerDM Clark2011Cognitive therapy vs interpersonal psychotherapy in social anxiety disorder: a randomized controlled trial.Arch Gen Psychiatry68692700