Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Internet-Delivered Cognitive-Behavioral Therapy for Social Anxiety Disorder in Romania: A Randomized Controlled Trial

  • Bogdan Tudor Tulbure ,

    Affiliation Department of Psychology, West University of Timișoara, Bd. V. Pârvan No. 4, 300223, Timișoara, Romania

  • Aurora Szentagotai,

    Affiliation Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, No. 37 Republici Street, 400015, Cluj-Napoca, Romania

  • Oana David,

    Affiliation Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, No. 37 Republici Street, 400015, Cluj-Napoca, Romania

  • Simona Ștefan,

    Affiliation Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, No. 37 Republici Street, 400015, Cluj-Napoca, Romania

  • Kristoffer N. T. Månsson,

    Affiliation Department of Behavioral Sciences and Learning, Linköping University, SE-581 83, Linköping, Sweden

  • Daniel David,

    Affiliation Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, No. 37 Republici Street, 400015, Cluj-Napoca, Romania

  • Gerhard Andersson

    Affiliations Department of Behavioral Sciences and Learning, Linköping University, SE-581 83, Linköping, Sweden, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden, Karolinska University Hospital Huddinge, SE-141 86, Stockholm, Sweden

Internet-Delivered Cognitive-Behavioral Therapy for Social Anxiety Disorder in Romania: A Randomized Controlled Trial

  • Bogdan Tudor Tulbure, 
  • Aurora Szentagotai, 
  • Oana David, 
  • Simona Ștefan, 
  • Kristoffer N. T. Månsson, 
  • Daniel David, 
  • Gerhard Andersson


Background and Aims

Internet-based cognitive-behavioral therapy (iCBT) for social anxiety disorder has been found effective, as attested by independently conducted randomized controlled trials in four languages. The study aim is to test the efficacy of an iCBT program in a culture where it was not tested before (i.e. Romania).


Participants (n = 76) were recruited, screened and randomized to either a nine-week guided iCBT or a wait-list control group in April and May 2012. Self-report measures were collected before (April 2012) and after the intervention (July 2012), as well as six months later (January 2013). Although social anxiety was assessed with multiple measures, the Liebowitz Social Anxiety Scale - Self Report version (LSAS-SR) and Social Phobia Inventory (SPIN) were used as the primary outcome measures.


A significant difference with a large between-group effect size in favor of iCBT was found (Cohen´s d = 1.19 for LSAS-SR and d = 1.27 for SPIN). Recovery rates show that 36.8% (n = 14) in the treatment group score below the SPIN clinical cut-off compared to only 2.6% (n = 1) in the wait-list control group. Post-intervention clinical interviews also revealed that 34.2% (n = 13) of the treatment group was completely recovered (full remission) while additionally 36.8% (n = 14) retained some social anxiety symptoms (partial remission). However, an important study limitation is that post-intervention interviewers were not blinded to the study conditions. The program also effectively reduced depression and dysfunctional thinking (between-group Cohen´s d = 0.84 for depression and d = 0.63 for dysfunctional thinking). Moreover, the iCBT intervention appears to have a long-term impact for participants’ functioning, as the treatment gains were maintained six months later.


Internet-delivered interventions display a high potential to quickly and widely disseminate effective evidence-based programs around the world. This study provides support for guided iCBT as a promising treatment approach in Romania.

Trial Registration NCT01557894


Social anxiety disorder (SAD), also referred to as social phobia [1], is defined by the DSM-IV as a persistent fear of one or more social or performance situations in which the person is exposed to evaluation or scrutiny by others. The individual fears that he/she will appear anxious or will act in a manner that will be embarrassing or humiliating [2]. Frequently feared situations include public speaking, talking to strangers or people in authority, and attending public events[3]. DSM-IV criteria for SAD were used throughout the study because it was implemented before the release of DSM-V.

SAD varies in severity and a distinction is commonly made between limited/non-generalized (e.g., public speaking) and generalized SAD [4]. The disorder is highly debilitating, with the majority of socially anxious persons reporting numerous problems in individual and social adjustment [5]. SAD is associated with impairment in academic and professional functioning, as well as in romantic and family relationships [5]. People suffering from the disorder report lower quality of life, are less likely to be married, more likely to divorce, are less educated, and of lower socioeconomic status than people who do not have social anxiety [6]. Moreover, SAD often co-occurs with other psychiatric conditions such as mood disorders and substance use disorders, and is significantly associated with suicidal ideation [5, 7, 8]. In Western cultures, the lifetime prevalence of SAD ranges between 7–12% of the population [9], with low rates of spontaneous remission [10], and prevalence appears to be increasing [11].

There are a range of psychological and pharmacological treatments for SAD [12].The most established psychological treatment for SAD is cognitive-behavioral therapy (CBT), which has proven effective both in individually-administered and group-administered formats[13]. Evidence indicates that individually administered CBT is equally effective [14] or more effective [15]than group CBT and superior to standard psychiatric treatment with medication and emotional support [16].

Cognitive models of SAD emphasize the importance of dysfunctional beliefs in generating the disorder, by transforming innocuous social cues into threats [17, 18]. Thus CBT interventions are mainly focused on altering these beliefs using both cognitive and behavioral strategies such as cognitive restructuring, behavioral experiments, exposure, applied relaxation, and social skills training[19].Previous studies investigated mostly the effect of CBT on cognitive distortions (i.e., description and inferences) (e.g., “Everyone will make fun of me”) and largely ignored the impact on appraisal/evaluative beliefs (i.e., rational and irrational beliefs) (e.g., “They should not make fun of me and if they do this is awful and I am a fool”). In his rational-emotive behavior therapy Ellis[20]argued that irrational beliefs represent core cognitive vulnerability factors for various emotional problems. Therefore, it would be helpful to know whether a CBT intervention could alter both cognitive distortions and evaluative beliefs at the same time.

Despite the availability of effective interventions, only a minority of individuals suffering from SAD seek and receive appropriate treatment [7, 21]. Many people live with their disorder for years before turning to mental health services, and treatment is often sought only when symptoms become too severe and disruptive to manage, or when secondary problems, such as depression or substance use occur [21, 22].Embarrassment associated with help-seeking and fear of what others might think or say have been found to prevent individuals with SAD from seeking treatment [7]. In addition to these disorder-specific issues, barriers to accessing professional assistance such as the lack of skilled therapists, lack of evidence-based treatments, long waiting lists, and costs further prevent accessing appropriate treatment. Therefore, increasing the international availability of evidence-based interventions for people suffering from SAD represents an important challenge [23].

In an attempt to increase the accessibility of adequate, cost-effective treatments, researchers have focused on developing internet-based cognitive-behavioral therapy interventions (iCBT). In most studies, a guided self-help approach has been used, where a web-based presented program is combined with minimal, but regular therapist contact by e-mail or phone [24]. Aside from the advantage of widespread and continuously increasing internet access, the internet version of CBT can be an attractive option for people with SAD, who normally fear and avoid social interactions.

The first iCBT trials for SAD were conducted by Andersson and colleagues [25]using a protocol consisting of nine online modules and two therapist-led exposure sessions. Results indicated large effect sizes that were maintained at 1-year follow-up. Internet-based CBT protocols have since been developed and tested by research groups in Australia, Spain, Sweden, and Switzerland, with results supporting the short- and long-term clinical benefits and financial advantages of these interventions[26]. A recent meta-analysis [27]reported large effects of iCBT on social anxiety symptoms (Cohen’s d = .86) and moderate effects on quality of life (d = .53), and comorbid anxiety and depression (d = .43). Considering these encouraging results, it is important that iCBT for SAD are further examined in randomized clinical trials, with major theoretical and practical objectives.

The first study objective was to investigate the efficacy of any CBT program in a culture where it has not been tested before. Our intervention—called Internet Social Phobia (iSOFIE)-was originally written in Swedish[28], being subsequently translated and adapted for use with Romanian SAD clients. Consistent with the principles of parsimony and pragmatism the iSOFIE intervention was reduced, while retaining the main original treatment components. Compared to the previously tested Swedish version, the iSOFIE intervention is shorter in that the amount of information in each module was reduced by half, but the number of modules was unchanged. Such an approach has the potential to effectively treat social anxiety while significantly decreasing the time needed for reading the treatment manual. We assumed that a shorter treatment could reduce the core impairing symptoms in a more parsimonious manner.

Consistent with the evidence-based approach that encourages the effort to test the same program in various contexts and cultures, we investigated the efficacy of an intervention program in a different culture, filling this gap in the literature. Besides enlarging the empirical support of an existing psychological service (an important theoretical contribution), the international availability of an empirically-supported intervention has the potential to alleviate the suffering of many socially anxious individuals in Romania (an important practical contribution).

The second study objective was to look at modifications in appraisal/evaluative beliefs following this brief iCBT intervention. To date, only one study compared the rational-emotive behavior therapy (REBT) with the classical CBT approach and found them equally effective in reducing SAD symptoms[29]. To our knowledge, this is the first study investigating the impact of an iCBT program for SAD on appraisal/evaluative beliefs as secondary outcome measures, making the analysis more nuanced compared to previous studies. Finally, being the first guided iCBT intervention in Romania, an additional objective was to assess participant’s satisfaction with the program.


The protocol for this trial and supporting CONSORT checklist are available as supporting information (see S1 CONSORT Checklist and S1 Protocol).

1.1 Ethics statement

This study was approvedby the Council of Scientific Research Ethics Comission of Babes-Bolyai University, Cluj-Napoca, Romania (Registration No 30273). Written informed consent was obtained from all participants by surface mail.

1.2 Study design and sample size

We designed a single-center, parallel group randomized clinical trial (RCT), with equal randomization for the two conditions (i.e., iCBT, and wait-list control). Stratified randomization with diagnostic status as a factor (i.e., clinical and sub-clinical) allowed the wait-list sub-clinical participants to serve as controls for their counterparts allocated to the iCBT condition.

1.3 Participants

Volunteer applicants from across the country were screened for the study (n = 291). Since the iSOFIE is a disorder-specific program, we only selected participants who could attain the maximum benefits from this intervention (i.e., social anxiety was their major difficulty). After a rigorous check 76participants were recruited for the intervention (see Fig 1). Sixty-six participants (86.84%) meet the DSM-IV criteria for SAD, and ten (13.16%) presented only subclinical symptoms (i.e., they lack one or two criteria to fully meet DSM-IV diagnostic for SAD).Participants’ age ranged from 18 to 54 years, with a mean age of 28.81 (SD = 8.04). The demographic characteristics for each of the two groups as well as for the total sample are summarized in Table 1.

Fig 1. Participants’ recruitment and progress throughout the iSOFIE program.

Eligibility criteria for the study included: (a) being over 18 years old, (b) exceeding the cut-off score on Social Phobia Inventory (SPIN) (i.e., SPIN total score ≥ 19)[30], Social Interaction and Anxiety Scale (SIAS) (i.e., SIAS total score ≥ 24) [31], and Liebowitz Social Anxiety Scale—Self Report version (LSAS-SR) (i.e., LSAS-SR total score ≥ 30) [32],(c) fulfilling the DSM-IV criteria for SAD on Social Phobia Screening Questionnaire (SPSQ) [33], (d) having SAD as the primary diagnostic on Structured Clinical Interview for DSM-IV-TR (SCID) [34], (e) presenting no suicidal ideation (i.e., not exceeding a score of 2 on the suicide item of Beck Depression Inventory-II (BDI-II) [35], and not reporting parasuicidal behavior on the Screening Questionnaire of the SCID), (f) not currently receiving other forms of psychological treatment for SAD, (g) having access to a computer connected to the internet, (h) if on medication, the dose should be constant for at least 1 month, and participants should agree to keep the dosage unchanged for the whole duration of the study, and (i) having no diagnosis of psychoses or borderline personality disorder on the SCID. The accepted cut-off scores for the SAD measure according to the recommendations found for each instrument in their original article were used. However, because SPIN has the most reliable empirical support for its clinical cut-off, the recovery rates were estimated only for SPIN.

To be included in the study, participants were allowed to have prior history of treatment for SAD, but in the treatment group only five participants reported receiving treatment within the last five years (which speaks of the low rate of SAD treatment in Romania).

1.4 The iCBT Intervention

The present intervention was adapted from a manual [28] previously tested in Sweden[25, 3639]. Relevant information about social anxiety is presented in each of the nine modules (see Table 2), and participants are asked to fill in essay questions, provide thought records, build anxiety hierarchies, describe their exposure exercises, and complete a weekly social anxiety measure (LSAS-SR). The iSOFIE participants were encouraged to contact their online psychologist if difficulties in understanding the text or in implementing the practical applications were encountered. A weekly feedback was offered to each participant and questions were answered within a 48-hour interval. In order to proceed through the intervention, participants had to fill in the exercises and essay questions at the end of each module. The access to the next module were given if participants’ responses prove that they had a correct understanding of the main concepts (i.e., they know the difference between thoughts and feelings, they understand the role of safety behaviors etc.) and they used them in at least one real life context. Participant’s subjective understanding was assessed though the weekly essay questions and homework online assignments. The iSOFIE participants were granted one week for each module, thus the nine-week intervention time-frame. The control group received no active treatment during the nine-week interval, but participants were asked to complete a weekly social anxiety measure (LSAS-SR).

Table 2. Brief Description of the Romanian iSOFIE intervention program.

1.5 Procedure

The trial was briefly presented in various national and local newspapers (April, 2012), where a link to the project website ( was provided. Interested participants registered online, read the informed consent, and filled in the screening questionnaires over the internet. Those who fulfilled the minimal inclusion criteria (i.e., high levels of social anxiety, low levels of depression, over 18 years, declared to have no psychosis or personality disorder, and were not currently treated for social anxiety) were contacted for a telephone interview. The interview was conducted within 12 days after the completion of the screening, and was based on the SCID. The purpose of this interview was to assess the applicant’s diagnostic status according to the DSM-IV criteria for SAD, and to briefly check for possible major problems undetected during screening. Sixty-six applicants (out of 121 who were contacted for the interview) meet the DSM-IV criteria for SAD. Ten additional applicants presented subclinical symptoms (i.e., they lack one or two criteria to fully meet DSM-IV diagnostic for SAD). In total, 76 applicants were included in the study as we predicted that they could benefit from this low-intensity psychosocial intervention. The flow of participants throughout the study is diagramed in Fig 1.

Following assessment, the 76 included participants were randomly assigned to either the iSOFIE intervention (n = 38) or to the wait-list control condition (n = 38) via a computerized randomization procedure (May, 2012; see the randomization procedure). Participants in both conditions received an email with information about the assigned group, and were invited to perform a different set of tasks. The wait-list control participants were asked to fill in a weekly social anxiety measure (LSAS-SR) for the following nine weeks, and were informed that in ten weeks they will benefit from the active treatment. The iSOFIE participants were given access to the first module and were also invited to fill in the weekly social anxiety measure (LSAS-SR). Moreover, each iSOFIE participant was randomly assigned to an online psychologist who assisted him/her throughout the intervention. The psychologists were responsible to monitor participant’s activity (i.e., send reminders when no activity was seen on the platform, provide feedback for the homework assignments, answer participant’s questions etc.). All participants were supported throughout the treatment, and positive encouragements were provided for every noticeable progress.

The post-intervention assessment (July, 2012) consisted of a short telephone interview (i.e., the SCID Social Anxiety Module) and the same online questionnaires as the ones completed during the screening phase. In order to assess the satisfaction with the intervention, the iSOFIE participants were further invited to fill in a treatment satisfaction questionnaire. After the post-intervention week (i.e., week ten), participants were no longer able to communicate on the platform, but they were able to access all the modules for the next six months (until the follow-up assessment conducted in January, 2013). Finally, control group participants started the iCBT intervention one week after the iSOFIE group completed the program.

1.6 Measures

1.6.1 Primary outcome measures.

The Liebowitz Social Anxiety Scale—Self-Report version (LSAS-SR) [32] presents 24 commonly anxiety-provoking situations, and asks participants to rate their fear and avoidance for each situation. The psychometric properties of the LSAS-SR are good to excellent [40] and the scale captures symptom changes in both cognitive-behavioral and psychiatric interventions. As a result, the LSAS-SR was recently rated among the evidence-based measures, and recommended for treatment monitoring and treatment outcome in adults[41] and adolescents[42].

The Social Phobia Inventory (SPIN) [30], is a brief (i.e., 17-item) self-report instrument measuring fear in social situations, avoidance of performance/social events, and physiological discomfort in social situations. Each item is rated on a four-point scale, with higher scores corresponding to greater distress. The scale has generally good to excellent psychometric proprieties[30, 43]and was considered an evidence based measure for treatment monitoring and treatment outcome[41].

1.6.2 Secondary outcome measures.

Besides these primary outcome measures, social anxiety symptoms were assessed with two additional measures. The Social Interaction and Anxiety Scale (SIAS) [31], is a 20-item measure that assesses fears of general social interactions. The scale captures both social scrutiny fears and social interaction fears. For each item, respondents are asked to indicate the degree to which they feel the statement is characteristic or true of them on a five-point scale. The SIAS was found to have sound psychometric properties [31].

The Social Phobia Screening Questionnaire (SPSQ)[33], is a diagnostic questionnaire for SAD. The measure presents both dimensional and categorical data, including impairment and duration of reported social anxiety [33]. SPSQ was previously used in various SAD RCTs [25, 37, 38].

Depressive symptoms.

The severity of depressive symptoms according to DSM-IV was measured with the Beck Depression Inventory-II (BDI-II) [35]. Each item consists of four statements indicating increasing symptom severity. Sound psychometric proprieties were reported for BDI-II [35] and it was evaluated among the evidence-based measures for treatment outcome [44].

Cognitive outcomes.

Cognitive factors related to emotional problems were explored to see whether the iCBT program significantly contributes to their reduction (e.g., automatic negative thoughts, irrational thinking).

Participants’ negative thoughts and related cognitive processes (most of them descriptions/inferences) were measured with the Automatic Thoughts Questionnaire (ATQ) [45] and irrational thinking patterns (i.e., all of them evaluative beliefs) were measured with the Attitude and Belief Scale-II (ABS-II) [46, 47]. Although ABS-II allow us to compute various scores (e.g., for rational beliefs, for specific irrational beliefs), we focus here only on the total score of irrational beliefs as a first step to investigate possible changes generated by the iCBT.

1.7 Randomization

Once registered on the treatment platform, participants received a unique study code. An independent person was asked to randomly assign the codes to one of two conditions (using a 1:1 allocation procedure) using a computer-generated randomization procedure with stratification by diagnostic status (clinical and subclinical) as an additional criterion. The interviewers were blind to which group the participants had been randomized. However, because the difference between the iSOFIE and the control condition could not be masked, online therapists and participants were not blinded regarding the condition assignment.

1.8 Statistical Analyses

At baseline, group differences in demographics were analyzed using the t-test and the Chi-square test. Because not all participants completed every weekly, post-test or follow-up measure, the missing data (representing 7.4% of all data) were imputed with multiple imputation procedures [48]. No pattern was established for the relatively small amount of missing data. Therefore, multiple imputation to predict the random missing data was used. Changes in both primary and secondary outcome measures were evaluated using Univariate ANCOVA, with pre-intervention scores as covariates, and group as a fixed factor[49]. Effect sizes were calculated using Cohen’s d for both within- and between-group comparisons, with the pooled standard deviation of the compared groups as denominator. Paired sample t-tests were used to test whether treatment gains were maintained between post-intervention and the six-month follow-up assessment. Finally, 95% confidence intervals (CI) were used to estimate the precision of the effect size for all measures. All analyses were performed in SPSS version 17.0 (SPSS, Inc., Chicago, IL).

1.9 Clinical Responder Definition and Significant Change

Recovery rates based on patient’s self-reported social anxiety was estimated by the proportion of patients who scored below the clinical cut-off at the post-intervention assessment. Participants who did not meet the DSM-IV criteria for SAD at the post-intervention SCID interview (i.e. were evaluated as having less than two SAD symptoms) were considered responders. However, participants who did not meet the DSM-IV criteria for SAD but retained some residual symptoms at post-intervention were considered in partial remission. Clinically significant change was calculated on the main outcome measures using a conservative method suggested by Jacobson and Truax [50]. The participant’s change had to be more than two SDs from the group mean symptom level at baseline (i.e., all included participants), as well as the level had to be within two SDs from a non-clinical reference group [30, 51]. LSAS-SR total scores less than 39.53, and SPIN total scores less than 28.52 defined clinically significant change in the present study.


2.1 Pre-intervention Assessment

The two conditions displayed similar demographic characteristics, and no statistically significant group differences were observed in terms of age, gender, educational level, professional and marital status, and previous psychotherapy attendance at baseline (i.e., all F and χ2displaying a p between. 15 and. 81). Further details about the demographic characteristics are presented in Table 1. The screening self-report measures used for both groups displayed the following internal consistencies (Cronbach’s Alpha): LSAS-SR = .92; SPIN = .85; SIAS = .80; SPSQ = .82; BDI-II = .80; ABS-II = .84; ATQ = .88.

Although social anxiety symptoms were significantly interfering with their normal life routine, none of the 76 included participants were using psychoactive medication nor were they involved in any kind of psychotherapy when assessed for the present study. This alone highlights the lack of SAD treatments in Romania.

2.2 Adherence and attrition

At the end of each module participants could save their answers on the iSOFIE platform. Treatment adherence was estimated by the number of completed worksheets (i.e., ranging between zero and 24). On average, participants completed 2.6 weekly worksheets. Overall, the iSOFIE participants completed 662 worksheets (mean = 17.43, SD = 7.77), and the number of completed modules is presented in the flow diagram (Fig 1). Post-intervention questionnaires were collected from 68/76 participants (89%), while post-intervention SCID interviews were conducted with 72/76 participants (94%). Finally, six-month follow-up questionnaires were collected from 27/36 participants (75%).

2.3 Primary outcome measures

Univariate ANCOVAs on post-intervention LSAS-SR and SPIN, controlling for pre-intervention scores, revealed that the iSOFIE group displayed significantly lower post-intervention scores compared to the control group scores (see Table 3 -first two rows). Separate analyses were conducted only for clinical participants (the 66 participants who meet the SAD diagnostic criteria at pre-intervention interview) to see whether these results are in any way different when compared to the results obtained from all participants. No significant differences could be observed (see Table 3).

Table 3. Results of the two groups presented separately for all / for only clinical participants.

Large between-group effect sizes (Cohen´s d) were observed at post-intervention for all measures (ES ranged between 1.19 and 1.36; see Table 4). The social anxiety levels captured by the weekly LSAS-SR for both groups are depicted in Fig 2A. The 95% CI of the iSOFIE and WLCG do not overlap at post-intervention, illustrating that the interval estimates of the two groups are divided by an important gap (see Fig 2B).

Fig 2. Social anxiety levels for the two groups throughout the iSOFIE program.

Abbreviations: iSOFIE = the iSOFIE intervention group; WLCG = Wait List Control Group; Pre = Pre-intervention assessment, Post = Post-intervention assessment.

Table 4. Results of the two groups before and after the iSOFIE program (all participants).

2.4 Secondary outcome measures

Univariate ANCOVAs were also conducted on the secondary outcome measures, controlling for pre-intervention scores. Results showed that the iSOFIE group displayed lower post-intervention scores when compared with the control group on depression, negative automatic thoughts, and irrational thinking scores (see Table 3). Effect sizes ranged from moderate to large (between-group ES = 0.63 to 0.84, and within-group ES = 0.95 to 1.19; see Table 4).

2.5 Six-month Maintenance

Paired-sample t-tests comparing post-intervention to six-month follow-up scores revealed significant reductions on most primary and secondary outcome measures: SPIN, SIAS, SPSQ, ABS-II, and ATQ (see Table 3- rows three and four). In the absence of any additional interventions participant’s social anxiety and irrational thinking displayed lower levels after six months. No significant improvements from post-intervention to follow-up were observed for LSAS-SR, and BDI-II (see also Table 3). However, pre- to follow-up within-group effect sizes revealed large effects for all measures (Table 4).

2.6 Clinical Significance

As concerning the treatment response, a total of 34.20% (n = 13) of the participants in the iSOFIE group were considered symptom free (i.e., total remission) at the post-intervention SCID interview. Moreover, 14 additional participants (36.80%) significantly improved as they did not meet the DSM-IV criteria for SAD but maintained some residual social anxiety symptoms at the end of the treatment (i.e., partial remission). Compared to the WLCG (n = 4), a significant proportion of the iSOFIE participants (n = 27) were considered responders as they obtained either total or partial remission (i.e., χ2(1) = 28.82; p<.01). Recovery rates show that 14 participants (36.80%) from the iSOFIE group and one participant (2.6%) from the WLCG score below the SPIN clinical cut-off (i.e., below 19 points) at post-intervention assessment. Most participants who scored below the SPIN clinical cut-off were also considered in total remission at the post-intervention SCID interview. Moreover, six-month later, in the absence of any intervention, the recovery rates ascended to 55.60% as 15 participants scored beyond the SPIN clinical cut-off.

We further investigated the impact of the treatment using a clinical significant change criterion. The LSAS-SR outcome measure showed that47.4% (18/38) of the iSOFIE participants (i.e., the first treatment group) reached this criterion, versus 13.2% (5/38) in the WLCG. This difference is statistically significant χ2(1) = 10.53, p<.01.Additionally, the SPIN outcome measure showed that 44.7% (17/38) of the iSOFIE participants scores were clinically and significantly changed, and these results were different from the WLCG (7.9%, 3/38; χ2(1) = 13.30, p<.01).For the second treatment phase (i.e., when the former WLCG participants received the iCBT) 44.7% (17/38) met the clinically significant change criterion for LSAS-SR and 60.5% (23/38) for SPIN.

2.7 Treatment Satisfaction

At the end of the program, participants who completed the treatment satisfaction questionnaire rated the quality of the iSOFIE modules as good and very good (28% and 72% respectively). The overall satisfaction with the program was also high, participants being satisfied or very satisfied with the intervention (40%, 46% respectively). Only a few responders declared to be neither satisfied nor dissatisfied with the program (14%), and none rated it as unsatisfactory. While the intervention was generally considered demanding or very demanding (62%, 10% respectively), all responders viewed the iSOFIE program as helpful or very helpful (28%, 72% respectively). On average participants spent 6.8 hours per week solving the various iSOFIE tasks (e.g., reading the text, completing the homework, writing to their online psychologists etc.) and most of them declared to have had an overall positive experience. Few complaints were made about the limited time available for each module (three participants), the perceived distant communication with her online therapists (one participant) and a remediable technical difficulty (one participant). Finally, participants were asked to indicate their level of agreement on a ten-point scale, where ten indicate the highest agreement. Participants rated the iSOFIE program approach as logical (9/10), and were willing to recommend it to a friend with similar problems (9/10).

2.8 Results for the second treatment group (the former wait-list control group)

After the first group completed the intervention, participants from the wait-list control group were given the chance to receive the same intervention. Paired-sample t-tests comparing pre- to post-intervention scores for the second treatment group revealed significant reductions for all measures (see last two rows of Table 3). Within-group effect sizes were large (ES = 1.78 to 3.23) supporting once again the efficacy of this intervention (see Table 4).Recovery rates show that 30.06% (n = 10) of the second treatment group participants scored below the SPIN clinical cut-off at post-intervention. No follow-up data were collected for this group.


This study was a randomized controlled trial aimed to investigate the efficacy of an internet based cognitive-behavioral intervention compared to a wait-list control group in the treatment of SAD. The Romanian iCBT intervention, named iSOFIE, was based on a Swedish internet-delivered CBT protocol which had been tested in previous studies [25, 39, 52].

Overall, the results suggest that the iSOFIE program is effective not only in terms of treating SAD symptoms, but also in reducing depression and dysfunctional thinking (e.g., both distorted and evaluative cognitions). The intervention had a statistically significant effect on social anxiety symptoms, with large effect sizes for all the social anxiety symptoms measures. This supports previous findings presented in the literature [23, 53]. Furthermore, a 36.8% SPIN recovery rates for the iSOFIE group compared to only 2.6% in the WLCG shows that the differences between the two groups are both statistically and clinically significant.

In terms of the secondary outcomes (i.e., depression, automatic, and irrational thinking), significant changes favoring the iSOFIE group were observed, with moderate to large effect sizes. Since the changes in depression were obtained in the absence of direct focus of the treatment, these effects could show generalized effects of the intervention. Indeed, as depression and anxiety symptoms are often comorbid, they may have common cognitive mechanisms (e.g., inflexible/rigid irrational beliefs) that were addressed in the present study. Moreover, the treatment effects appear to be stable for both primary and secondary outcome measures, as most scores remained unchanged or decreased six months after the program ended. It is possible that, once participants did not meet the primary diagnostic criteria for SAD the secondary/associated depression symptoms were also reduced (i.e., due to the decrease of social anxiety). Similar decreases in secondary outcome measures were found in previous studies[25]. In terms of the treatment satisfaction, participants reported being satisfied with the treatment.

Finally, the original SOFIE interventions conducted in Sweden reveal similar medium to large ES for both primary and secondary outcome measures [25, 37, 38]. In a recent study where the SOFIE program was compared to a WLCG using a big sample (n = 204), a large between group ES at post-intervention was obtained on the LSAS-SR (Hedge’s g = .75;[36]). Interestingly, the parallel delivery of a similar intervention via a smart phone application in Sweden (mSOFIE) also yielded medium to large within-group ES for social anxiety measures (Cohen’s d range between. 71 and. 99;[52]). Considering these preliminary results, it appears that condensing treatment manuals (i.e., presenting shorter versions either online or via smart phones) display the potential to effectively reduce social anxiety and related symptoms in a parsimonious manner.

3.1 Study limitations

This study presents a number of limitations. First, an active control condition matched for clients’ expectations, time, and therapist involvement was not included. Therefore the obtained results could be due to non-specific factors associated with any active treatment and the reported results should be interpreted with caution (i.e., due to the non-existent match in terms of experimental design). Although we could not demonstrate that our iCBT program impacts participant’s life beyond the merely placebo effects, previous studies[2427, 3638, 52, 54]appear to be in line with our conclusions, supporting the idea that iCBT programs are effective in diminishing social anxiety, and modestly effective for comorbid anxiety and depression.Second, due to personnel constraints, post-intervention SCID interviews were not blinded regarding the study conditions. This could have influenced the study results in line with the interviewers’ expectations. However, similar results were obtained using social anxiety self-report measures at post-test. For example, similar recovery rates were obtained for SPIN (36.8%) and the full symptom remission on clinical interviews (34.2%). Furthermore, clinical significance change for the primary outcome measures yielded significant results, certifying that participants’ social anxiety decreased after the intervention. Third, the diagnostic procedure based on SCID interviews was conducted on telephone rather than in-person. However, we tried to overcome these limitations by using evidence-based screening tools with high psychometric properties (i.e., LSAS-SR, SPIN, BDI-II); and similar procedures were previously used in other clinical trials for selecting SAD participants [25, 36].Finally, the iSOFIE intervention was framed as an efficacy (e.g., how it works in well-controlled conditions), rather than as an effectiveness study (e.g., how it works in real clinical practice). However, this step is fundamental to support the internal validity of iCBT.

Conclusions and Future Directions

These findings further support and replicate the efficacy of an iCBT program for SAD called Internet Social Phobia (iSOFIE). The overall results of the iSOFIE group were superior compared to the WLCG, satisfaction with the treatment being high. Comparable ES on both primary and secondary outcome measures were obtained with both the long and the short version of the SOFIE treatment manual. Overall, it seems that the iSOFIE intervention can be effectively used to overcome the problem of high prevalence versus low access to evidence-based treatment for SAD (especially in developing countries).

To further advance the field of internet-delivered interventions future studies could include an active treatment condition, and investigate the external validity of such programs in psychiatric care. In this context, a tailored treatment might be needed in order to address potential co-morbid disorders [55]. A program such as the iSOFIE could be augmented with distinct modules created to respond to these needs. Also, a motivational interviewing could be added before the intervention to enhance treatment adherence [54, 56]. Nevertheless, as internet-delivered interventions display a high potential to widely disseminate effective evidence-based programs around the world, we conclude that iCBT for SAD works in Romania, and this novel treatment format holds promise as a treatment alternative in the future.


The authors are thankful to the iSOFIE master students team who supported us during the screening and intervention phases: Bulea Alexandra, Calo Larisa Cristina, Golita Silvia, Isbășoiu Andreea, Pasca Flavia Sorana, Sofa Redo, Stan Alexandra, Steger Carmine, Seta Darius, Telis Rita Natasha, and Veresezan Emilia.

Author Contributions

Conceived and designed the experiments: BTT AS KNTM DD GA. Performed the experiments: BTT OD SS. Analyzed the data: BTT. Contributed reagents/materials/analysis tools: BTT KNTM DD GA. Wrote the paper: BTT AS OD SS KNTM DD GA.


  1. 1. Liebowitz MR, Heimberg RG, Fresco DM, Travers J, Stein MB. Social phobia or social anxiety disorder: what's in a name? Arch Gen Psychiatry. 2000;57: 191–2. pmid:10665624
  2. 2. Association AP. Diagnostic and statistical manual of mental disorders-IV. Washington: American Psychiatric Press; 1994.
  3. 3. Holt CS, Heimberg RG, Hope DA, Liebowitz MR. Situational domains of social phobia. J Anxiety Disord. 1992;6: 63–77.
  4. 4. Hofmann SG. Cognitive factors that maintain social anxiety disorder: a comprehensive model and its treatment implications. Cogn Behav Ther. 2007;36: 193–209. pmid:18049945
  5. 5. Eng W, Heimberg RG. Social anxiety disorder. In: Freeman C, Power M, editors. Handbook of evidence-based psychotherapies A guide for research and practice. Chichester: John Wiley & Sons Ltd; 2007. pp. 367–83.
  6. 6. Stein MB, Kean YM. Disability and quality of life in social phobia: epidemiologic findings. Am J Psychiatry. 2000;157: 1606–13. pmid:11007714
  7. 7. Olfson M, Guardino M, Struening E, Schneier FR, Hellman F, Klein DF. Barriers to the treatment of social anxiety. Am J Psychiatry. 2000;157: 521–7. pmid:10739410
  8. 8. Schneier FR, Martin LY, Liebowitz MR, Gorman JM, Fyer AJ. Alcohol abuse in social phobia. J Anxiety Disord. 1989;3: 15–23.
  9. 9. Furmark T. Social phobia: overview of community surveys. Acta psychiatrica Scandinavica. 2002;105: 84–93. pmid:11939957
  10. 10. Keller MB. The lifelong course of social anxiety disorder: a clinical perspective. Acta Psychiatr Scand Suppl. 2003: 85–94.
  11. 11. Heimberg RG, Stein MB, Hiripi E, Kessler RC. Trends in the prevalence of social phobia in the United States: a synthetic cohort analysis of changes over four decades. Eur Psychiatry. 2000;15: 29–37. pmid:11520471
  12. 12. Acarturk C, Cuijpers P, van Straten A, de Graaf R. Psychological treatmaent of social anxiety disorder: a meta-analysis. Psychol Med. 2008;39: 241–54. pmid:18507874
  13. 13. Heimberg RG. Cognitive-behavioral therapy for social anxiety disorder: current status and future directions. Biol Psychiatry. 2002;51: 101–8. pmid:11801235
  14. 14. Ledley DR, Heimberg RG, Hope DA, Hayes SA, Zaider TI, Dyke MV, et al. Efficacy of a manualized and workbook-driven individual treatment for social anxiety disorder. Behav Ther. 2009;40: 414–24. pmid:19892086
  15. 15. Stangier U, Heidenreich T, Peitz M, Lauterbach W, Clark DM. Cognitive therapy for social phobia: individual versus group treatment. Behav Res Ther. 2003;41: 991–1007. pmid:12914803
  16. 16. Mörtberg E, Clark DM, Sundin O, Åberg Wistedt A. Intensive group cognitive treatment and individual cognitive therapy vs. treatment as usual in social phobia: a randomized controlled trial. Acta psychiatrica Scandinavica. 2007;115: 142–54. pmid:17244178
  17. 17. Heimberg RG, Brozovich FA, Rapee RM. A cognitive-behavioral model of social anxiety disorder: update and extension. In: Hofmann SG, DiBartolo PM, editors. Social anxiety: Clinical, developmental and social perspectives. Second edition ed. New York: Academic Press; 2010. pp. 395–442.
  18. 18. Clark DM, Wells A. A cognitive model of social phobia. In: Heimberg ML R., Hope D. A., & Schneier F. R., editor. Social phobia: Diagnosis, assessment and treatment. New York: Guilford Press; 1995. pp. 69–93.
  19. 19. Davidson JR, Foa EB, Huppert JD, Keefe FJ, Franklin ME, Compton JS, et al. Fluoxetine, comprehensive cognitive behavioral therapy, and placebo in generalized social phobia. Arch Gen Psychiatry. 2004;61: 1005–13. pmid:15466674
  20. 20. Ellis A. Reason and emotion in psychotherapy. Revised and Expanded Edition ed. New York: Birch Lane Press; 1994.
  21. 21. Fehm L, Pelissolo A, Furmark T, Wittchen HU. Size and burden of social phobia in Europe. Eur Neuropsychopharmacol. 2005;15: 453–62. pmid:15921898
  22. 22. Magee WJ, Eaton WW, Wittchen HU, McGonagle KA, Kessler RC. Agoraphobia, simple phobia, and social phobia in the national comorbidity survey. Arch Gen Psychiatry. 1996;53: 159–68. pmid:8629891
  23. 23. Andersson G, Carlbring P. Social phobia (social anxiety disorder). In: Draper C, O’Donohue WT, editors. Stepped care and e-health: Practical applications to behavioral disorders. New York: Springer; 2011. pp. 99–114.
  24. 24. Berger T, Caspar F, Richardson R, Kneubuhler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther. 2011;49: 158–69. pmid:21255767
  25. 25. Andersson G, Carlbring P, Holmström A, Sparthan E, Furmark T, Nilsson-Ihrfelt E, et al. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol. 2006;74: 677–86. pmid:16881775
  26. 26. Hedman E, Andersson E, Ljotsson B, Andersson G, Ruck C, Lindefors N. Cost-effectiveness of Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial. Behav Res Ther. 2011;49: 729–36. pmid:21851929
  27. 27. Tulbure BT. The efficacy of internet-supported intervention for social anxiety disorder: a brief meta-analytic review. Procedia Soc Behav Sci. 2011;30: 552–7.
  28. 28. Furmark T, Holmström A, Sparthan E, Carlbring P, Andersson G. Social fobi—Effektiv hjälp med kognitiv beteendeterapi [Social Anxiety—Effective treatment using cognitive behavioural therapy]. 2nd ed. Stockholm: Liber; 2006.
  29. 29. DiGiuseppe R, McGowan L, Simon K, Gardner F. A comparative outcome study of four cognitive therapies in the treatment of social anxiety. J Ration Emot Cogn Behav Ther. 1990;8: 129–46.
  30. 30. Connor KM, Davidson JR, Churchill LE, Sherwood A, Foa E, Weisler RH. Psychometric properties of the social phobia inventory (SPIN). New self-rating scale. Br J Psychiatry. 2000;176: 379–86. pmid:10827888
  31. 31. Mattick RP, Clarke JC. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther. 1998;36: 455–70. pmid:9670605
  32. 32. Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatr. 1987;22: 141–73. pmid:2885745
  33. 33. Furmark T, Tillfors M, Everz P, Marteinsdottir I, Gefvert O, Fredrikson M. Social phobia in the general population: prevalence and sociodemographic profile. Soc Psychiatry Psychiatr Epidemiol. 1999;34: 416–24. pmid:10501711
  34. 34. First M, Gibbon M, Spitzer R, Williams J. Structured clinical interview for DSM–IV axis I disorders (SCID–I). Washington: American Psychiatric Press; 1997.
  35. 35. Beck AT, Steer RA, Brown GK. Beck depression inventory. Second Edition ed. San Antonio: TX: Psychological Corporation; 1996.
  36. 36. Andersson G, Carlbring P, Furmark T, Group SOFIER. Therapist experience and knowledge acquisition in internet-delivered CBT for social anxiety disorder: a randomized controlled trial. PloS One. 2012;7: e37411. pmid:22649526
  37. 37. Carlbring P, Gunnarsdottir M, Hedensjo L, Andersson G, Ekselius L, Furmark T. Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry. 2007;190: 123–8. pmid:17267928
  38. 38. Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, Bohman B, et al. Guided and unguided self-help for social anxiety disorder: randomised controlled trial. Br J Psychiatry. 2009;195: 440–7. pmid:19880935
  39. 39. Månsson KNT, Carlbring P, Frick A, Engman J, Olsson C- J, Bodlund O, et al. Altered neural correlates of affective processing after internet-delivered cognitive behavior therapy for social anxiety disorder. Psychiatry Res. 2013;214: 229–37. pmid:24064198
  40. 40. Baker SL, Heinrichs N, Kim HJ, Hofmann SG. The liebowitz social anxiety scale as a self-report instrument: a preliminary psychometric analysis. Behav Res Ther. 2002;40: 701–15. pmid:12051488
  41. 41. Rowa K, McCabe RE, Antony M. Specific phobia and social phobia. In: Hunsley j Mash EJ, editors. A guide to assessments that work. New York: Oxford University Press; 2008. pp. 207–28.
  42. 42. Tulbure BT, Szentagotai A, Dobrean A, David D. Evidence based clinical assessment of child and adolescent social phobia: a critical review of rating scales. Child Psychiatry Hum Dev. 2012;43: 795–820. pmid:22438106
  43. 43. Antony MM, Coons MJ, McCabe RE, Ashbaugh A, Swinson RP. Psychometric properties of the social phobia inventory: further evaluation. Behav Res Ther. 2006;44: 1177–85. pmid:16257387
  44. 44. Persons JB, Fresco DM. Adult depression. In: Hunsley J, Mash EJ, editors. A guide to assessments that work. New York: Oxford University Press; 2008. pp. 96–120.
  45. 45. Hollon SD, Kendall PC. Cognitive self-statements in depression: development of an automatic thoughts questionnaire. Cogn Ther Res. 1980;4: 383–95.
  46. 46. DiGiuseppe R, Leaf R, Exner T, Robin MV, editors. The development of a measure of rational/irrational thinking. World congress of behavior therapy; 1988; Edinburgh, Scotland.
  47. 47. Macavei B. Scala de atitudini si convingeri-II (ABS-II): date preliminare pentru populatia de limba româna (a Romanian adaptation of the attitudes and beliefs scale-II ABS- II). Romanian J Cog Behav Psychother. 2002: 105–22.
  48. 48. Wayman JC. Multiple imputation for missing data: what is it and how can I use it? Annual meeting of the american educational research association; Chicago, Illinois2003.
  49. 49. Vickers AJ, Altman DG. Statistics notes: analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323: 1123–4. pmid:11701584
  50. 50. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59: 12–9. pmid:2002127
  51. 51. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The liebowitz social anxiety scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001;31.
  52. 52. Dagöö J, Asplund RP, Bsenko HA, Hjerling S, Holmberg A, Westh S, et al. Cognitive behavior therapy versus interpersonal psychotherapy for social anxiety disorder delivered via smartphone and computer: a randomized controlled trial. J Anxiety Disord. 2014;28: 410–7. pmid:24731441
  53. 53. Spek V, Cuijpers P, Nyklicek I, Smits N, Riper H, Keyzer J, et al. One-year follow-up results of a randomized controlled clinical trial on internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years. Psychol Med. 2008;38: 635–9. pmid:18205965
  54. 54. Titov N, Gibson M, Andrews G, McEvoy P. Internet treatment for social phobia reduces comorbidity. Aust N Z J Psychiatry. 2009;43: 754–9. pmid:19629797
  55. 55. Carlbring P, Andersson G, Kaldo V. State-of-the-art treatment via the Internet: an optimistic vision of the future. Cogn Behav Ther. 2011;40: 79–81. pmid:25155811
  56. 56. Martins RK, McNeil DW. Review of motivational interviewing in promoting health behaviors. Clin Psychol Rev. 2009;29: 283–93. pmid:19328605