Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Effects of cognitive bias modification on social anxiety: A meta-analysis

  • Haining Liu,

    Affiliations Key Lab of Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China, Psychology Department of Chengde Medical University, Chengde, China, University of Chinese Academy of Sciences, Beijing, China

  • Xianwen Li,

    Affiliation School of Nursing, Nanjing Medical University, Nanjing, China

  • Buxin Han ,

    Affiliations Key Lab of Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, Beijing, China

  • Xiaoqian Liu

    Affiliations Key Lab of Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, Beijing, China



Cognitive bias modification (CBM), a set of techniques for modifying bias in information processing—is considered a novel intervention for social anxiety disorder (SAD), which has drawn considerable interest from researchers. However, the effects of CBM on SAD are not consistent. Some studies have demonstrated significant positive effects compared to control groups, while others have found no such effects.


We conducted a meta-analysis aimed at quantitatively assessing the effects of CBM on SAD at post-test.


Through a systematic literature search by two independent raters, 34 articles (36 randomized studies) including 2,550 participants were identified. A multilevel modeling approach was employed to assess the effects of CBM on SAD, and to explore the potentially crucial procedures and sample characteristics that enhance the effectiveness of benign training.


In general, there were small but significant effects of CBM on the primary symptoms of SAD (g = 0.17), cognitive bias (CB) toward threat (g = 0.32), and reactivity in stressful situations (g = 0.25), but non-significant effects on secondary symptoms. However, the interpretation modification program was more effective than was attentional bias modification in reducing SAD primary symptoms and negative CB. Laboratory training procedures produced larger primary symptom reductions compared to Internet-based training, whereas the percentage of contingency and feedback about training performance boosted cognitive effects only. Finally, the following groups were more likely to benefit from CBM: younger participants (primary symptoms and cognitive effects), women (primary symptom effects), and samples with stronger CB (stressor effects). The quality of the randomized controlled trials was less than desirable, as there was some indication of publication bias in our study.


Current findings broadly supported cognitive theories of SAD that consider a bidirectional or mutually reinforcing relationship between symptoms and CBs. However, the small therapeutic effect observed here indicates that it is necessary to develop more reliable and efficient CBM interventions that are specific to SAD.


Social anxiety disorder (SAD), also referred to as social phobia [1], is a condition marked by sustained fear and avoidance of certain or most social situations because of concerns about evaluation by others. Large-scale epidemiological studies have shown that SAD is one of the most prevalent psychological disorders. The findings of the World Health Organization (WHO) World Mental Health (WMH) Survey Initiative showed that lifelong social phobias are generally prevalent in both developed (15.9%) and developing (14.3%) countries [2]. A greater lifetime risk of SAD was found in people in developing countries as compared to those in developed countries. Additionally, SAD is common among young adults, and it has an early onset age—about 50% of the cases show symptoms by the age of 11 and about 80% of the cases by the age of 20 [3]. Women are disproportionately likely to develop SAD relative to men according to a National Epidemiology Survey on Alcohol and Related Conditions (NESARC) study [46]. Moreover, SAD is usually comorbid with other mental health problems, such as depression, specific phobias, or another anxiety disorder, which are often associated with serious impairment in educational, occupational, and social functioning [7]. Therefore, it is of great importance to develop more effective, accessible, and acceptable treatments for individuals with SAD.

There are diverse explanations for the etiology of SAD from a psychological perspective. For example, psychodynamic conceptualizations of social anxiety encompass numerous schools of thought, including instinctual models, ego-psychology, defense mechanisms, object relationships, and attachment theories [8, 9]. Symptoms of SAD are often believed to originate from childhood traumatic experiences that made the patient feel miserable or abandoned by their parents [10]. Apart from these psychodynamic approaches, researchers adopting an interpersonal approach have assumed that social skill deficits are one of the principal factors underlying SAD [11]. If individuals with SAD feel threatened in a situation, this situation may elicit self-defensive behaviors to guard against negative evaluations. Similarly, the cumulative interpersonal risk model of social anxiety in youth maintains that social skills impairment is one of many interpersonal risk factors for SAD [12]. However, in recent years, emerging cognitive theories—such as the information processing perspective—have maintained that social anxiety is associated with cognitive biases (CBs) in the processing of emotionally congruent information, which in turn play a key role in the onset and development of SAD [1317]. Cognitive models of SAD propose that individuals with higher levels of social anxiety automatically and selectively attend to socially threatening information (attention bias) and interpret emotionally ambiguous events as threatening (interpretation bias) [18,19]. For instance, in novel social scenarios, individuals with SAD are inclined to perceive social threat cues (i.e., someone frowning) and to interpret it as a signal that they are behaving poorly.

Cognitive bias modification (CBM) is a novel experimental technique, built on these cognitive theories of SAD, aimed at reducing negative CBs and thereby diminishing anxiety susceptibility and symptoms. A typical CBM paradigm is the modified dot probe task [20], which is based on MacLeod’s (1986) classic experiment. In a traditional dot-probe task, a pair of stimuli (i.e., a threatening face or word and a neutral counterpart) is presented on a computer screen simultaneously. Immediately following these cueing stimuli, a target stimulus (i.e., one or two dots) appears in the location previously occupied by one of the cueing stimuli. The participants are required to judge the location of the target as quickly and as accurately as possible. Responding faster to the target that replaced the threatening stimulus than to the target that replaced the neutral stimulus indicates attentional vigilance toward threatening information. The Attentional Bias Modification (ABM), paradigm subsequently modified the dot-probe task to direct participants’ attention away from threat, with the probe almost always replacing a benign stimulus (i.e., 80%–100% of the trials).

The interpretation bias paradigm is another typical CBM paradigm; while similar to ABM, it uses more complicated stimulus materials, such as the application of ambiguous sentences or paragraphs. For example, a series of ambiguous sentences are presented to participants, and they are requested to disambiguate the emotional valence of the stimuli towards a positive (or neutral) or negative interpretation. For instance, participants are presented with a word conveying either a threat interpretation (i.e., “embarrassing”) or a positive interpretation (i.e., “amusing”) before the following ambiguous sentence: “After you said something, people laughed.” They are then asked to indicate whether the word and sentence were relevant or irrelevant [21]. The Interpretation Modification Program (CBM-I) encourages participants to develop a benign interpretation of the ambiguous experimental materials. In other words, when participants generate a benign endorsement or refuse threat endorsement of ambiguous material, they are given positive feedback (i.e., ‘‘Correct!”), otherwise negative feedback is presented (i.e., ‘‘Incorrect!”). There are a number of other CBM interventions, such as the approach-avoidance task (AAT) wherein participants push (approach) or pull (avoid) pictorial cues presented on a computer screen using a joystick. This task aims to train participants to approach positive pictures and avoid the negatives, in order to led to more positive mood and lower anxiety [22].

These therapies involve similar implicit cognitive retraining strategies to alter CB. There is growing evidence that CBM might lead to decreases in both self-reported and behavior-based measures of anxiety in socially anxious participants, including both clinically diagnosed [2326] and subclinical samples [21,27,28]. However, over the past few years, other studies have offered some mixed results [2932]. Specifically, findings have shown that compared with controls, the CBM group had no significant difference at post-test in mitigating CB or SAD symptoms [27, 30].

To date, only one previous meta-analysis has been conducted on CBM techniques; the study summarized the findings of 15 randomized studies on the use of ABM to reduce negative attention bias and social anxiety [33]. Their results showed a small reduction in SAD cardinal symptoms (g = 0.27), responses to speech challenges (g = 0.46), and attention bias (AB) (g = 0.30). To explore potential moderators, the researchers found that laboratory-delivered CBM yielded larger effect sizes (ESs) in SAD symptom reduction than it did in Internet-delivered treatments. Additionally, the experimental design and trait anxiety at pre-test moderated these effects. Although this quantitative review took an important step toward the assessment of cognitive theories of social anxiety, its narrow focus on ABM meant that many CBM (i.e., CBM-I) studies were excluded. A more comprehensive quantitative review that covers interpretation, attention biases, and social anxiety is needed to clarify unresolved questions about CBM and its underlying theory.

The present meta-analysis aimed to investigate the clinical effects of CBM on social anxiety systematically, in terms of both treatment effects (i.e., primary/secondary symptoms of SAD andreactivity in stressful situations) and crucial moderators that predict the degree of change. The first goal was to evaluate the degree to which certain variants of CBM can successfully relieve SAD symptoms and CBs. The assumption of the present study was that CBM can modify cognitive biases that contribute to SAD symptoms; indeed, a recent review has concluded that ABM can modify AB toward threat and reduce SAD symptoms [33]. However, it is still unclear to what extent other types of CBs can be modified to produce changes in SAD symptoms. The second goal was to identify the potential moderators of the effectiveness of CBM on SAD. Previous reviews and theories have suggested a number of potential factors affecting the efficacy of CBM, such as the clinical status of the study sample, CB, specific procedural details of the interventions (i.e., training contingency, or whether the target almost always replaced the neutral/positive probe or whether the replacement was random; feedback; training setting; and number of training session), the severity of SAD symptoms at baseline (i.e., baseline score on the Liebowitz Social Anxiety Scale), baseline CB, participants’ characteristics (i.e., age and gender ratio), and impact factor of publications [3437]. The third and final goal was to evaluate the quality of included studies and identify potential publication bias. Specifically, the present study examined whether there is robust empirical evidence with strong methodological quality to support the clinical efficacy of CBM reported in these studies.


Identification and selection of studies

With strict adherence to the PRISMA guidelines [38], the first and second authors searched the literature independently (see S1 Table). Records were identified by searching multiple literature databases (see S1 Text), including ISI Web of Science, PubMed, PsycInfo, EMBASE and the Cochrane Collaboration's register of controlled trials, through December 2015. The following combinations of keywords were used in this search: “cognitive bias modification,” “attention* bias modification,” “interpret* bias modification,” “attention* training,” “bias experimental manipulation,” and “bias training” paired with “social phobi*”or “social anxi*” (see S1 Text). There were no language restrictions. The references within the most recent reviews on CBM for social anxiety disorders were also systematically searched [33, 39]. Unpublished studies were not included.

A total of 1,944 records were identified (see Fig 1 for details). After removal of duplicated records, the titles and abstracts of the remaining articles were screened. After eliminating obviously irrelevant publications, 56 potentially relevant articles were retained and their full texts were reviewed for incorporation into the meta-analysis. Studies were included in the review if: (a) a randomized controlled trial (RCT) was used; (b) the therapeutic effect of CBM was isolated and not merged with other treatments; (c) the study was designed specifically to manipulate CB for the purpose of alleviating SAD symptoms and emotional susceptibility among patients with SAD or in subclinical populations; (d) at least one control group was included (i.e., a placebo control); (e) social-anxiety-relevant symptoms (i.e., SAD symptoms, such as self-reported reactivity in stressful situations or clinician-administered symptom assessment) were assessed at least once after training; (f) positive or negative emotionally relevant stimuli were used as training stimuli (i.e., strongly valenced words, facial expressions); and (g) sufficient data were available to compute ESs.

Some studies included two or even more control groups (i.e., attend threat, the sham intervention, or waiting-list groups) [2628, 32]. We selected the group that was closest to the placebo condition (i.e., sham intervention group) to calculate ESs, as this was the most common approach used for control groups in CBM studies. Two studies used a four-groups design, including two experimental and two control groups, to examine the effectiveness of CBM [40, 41]. In addition, one study reported a six-group design—three experimental and three control groups [42]. For such studies, only the data sets that were related to the standard CBM and control conditions were extracted. Finally, with regard to findings for extended samples [43, 44] based on original studies [23, 45], we only chose the datasets reported in the original studies [23,45], as these offered more information about possible moderators.

Quality assessment and data abstraction

We used the risk of bias assessment tool formulated by the Cochrane Collaboration [46] to evaluate the methodological quality of the studies included in the meta-analysis. Five criteria from this tool were chosen to examine sources of bias in RCTs: (1) generation of adequate random allocation sequence, (2) allocation concealment prior to assignment, (3) blinding of participants to information about the allocated intervention, (4) blinding of outcome assessors to information about the allocated intervention, and (5) dealing with incomplete outcome data. The last criterion was met if there were no missing data or the missing data were managed using an intent-to-treat approach. The quality assessment of the studies was implemented by two independent authors (Liu and Li). Conflicting opinions were resolved via discussion; if this did not resolve the conflict, the authors sought the advice of the senior author (Han).

Information about following characteristics of the included studies was collected (see Table 1):

  • Publication information: including identification information (author and year of publication), and the journal impact factor (through Web of Science) at the time of publication;
  • Characteristics of the study sample: including source (clinical/subclinical), number of participants across different conditions, percentage of women per study, and seriousness of primary and secondary symptoms of social anxiety (i.e., the Liebowitz Social Anxiety Scale or Beck Depression Inventory score at baseline);
  • Characteristics of the CBM procedure: including treatment dose (total number of sessions), type of training materials (i.e., type of threaten and non-threaten cues), training setting (laboratory, internet, or smartphone), type of bias intervention (i.e., ABM or CBM-I), the presence of a stressful task during the training program (yes or no), feedback on accuracy when the tasks were solved (yes or no), and the percentage of trials in the CBM condition in which cues were valid.

The outcome measures were classified into four categories (see S2 Table): (1) self-reported primary social anxiety symptoms at post-test; (2) negative CB at post-test; (3) reactivity in stressful situations (i.e., speech challenge); (4) secondary social anxiety symptoms, both self-reported and clinician-rated, at post-test (i.e., depression and general distress).

Data analysis

For each between-group comparison, the ES (Cohen’s d) was calculated to indicate the differences between the CBM intervention and a control group at post-training. The ESs were calculated by subtracting the mean value of the CBM group from the mean value of the comparison group (at post-training), and dividing by the pooled standard deviations of both groups. Cohen’s d values of 0.2, 0.5, and 0.8 are considered small, medium, and large ESs, respectively [47]. Because some studies had small sample sizes, the corrected ES coefficient—Hedges’ g—was applied [48]. This can be interpreted like Cohen’s d [35].

We calculated the pooled mean ESs using Comprehensive Meta-Analysis (CMA; version 2.2.064 for Windows). As there were a number of measurements used to assess the symptoms, we classified the outcome measures as follows: SAD primary symptoms, CB, reactivity in stressful situations, and secondary symptoms (see S2 Table). When a study included several instruments measuring the same outcome, an average ES was calculated. When calculating averages, we weighted ESs by their sample size. If the studies did not report the means and standard deviations, we used the other statistics recommended by CMA 2.0, such as t values, sample sizes, and precise p values from independent groups. Additionally, we emailed the authors to inquire prudently whether there were any related unpublished data.

We also aimed to investigate how the baseline (i.e., before training) CB influenced the therapeutic benefits yielded by CBM to SAD symptoms, CB, and reactivity in stressful situations. Thus, we imputed a third data set of ESs and coded them as positive values to reflect a larger baseline CB in the CBM group relative to the control group. Only ESs for post-test and post-challenge were reported. Follow-up data were not considered because the duration differed considerably and participants in some studies underwent treatment while others did not. ESs were computed with random-effects models, which assume the observed estimates of treatment effects vary within-study (as with fixed-effects models) as well as between-studies, because this method is generally considered more conservative [49].

Homogeneity of ESs was tested using the Q-statistic and the I2-statistic [50]. A significant Q-statistic suggests that there is true heterogeneity in ESs exceeding the random error. The I2 quantifies the degree of heterogeneity in percentages, with higher values indicating higher heterogeneity. A value of 0% is regarded as no heterogeneity, while those of 25%, 50%, and 75% indicate low, moderate, and high heterogeneity, respectively [51]. When investigating categorical moderators, not only the heterogeneity within each group (Qw) but also the heterogeneity between groups (Qb) were evaluated. Significant between-group heterogeneity indicated that the moderators were significant.

Publication bias was examined using two methods: visually observing the funnel plots for the main outcome classification, and employing the trim and fill procedure of Duval–Tweedie to obtain a corrected ES [52]. Outliers were defined as an ES of ≥2.5 standard deviations (SD) above or below the average ES estimate in each dataset [53]. The robustness of the findings was assessed using sensitivity analyses, which involved comparing results when including and excluding outliers.

Categorical moderators were tested with subgroup analyses using a mixed-effects model. In this model, we investigated pooled studies within subgroups using a random-effects model, whereas significant differences between subgroups were examined using a fixed-effects model. If there were less than three studies within a subgroup, it was not reported. For continuous moderators, unrestricted maximum likelihood meta-regression analyses were conducted.


Thirty-four articles met the selection criteria, which involved a total of 36 RCTs with 2,550 participants. The number of training sessions varied from 1 to 83, with 7 RCTs including just one session. Seventeen RCTs used SAD samples that were diagnosed based on DSM-IV criteria, 18 subclinical samples, and 1 used unselected participants. In total, 25 interventions were carried out in a laboratory setting, 10 via the Internet, and 1 via smartphone. Twenty-two studies examined ABM, 10 examined CBM-I, 2 examined a combination of CBM methods, and 2 examined AAT.

Methodological quality of included studies

Overall, the quality of the included RCTs was heterogeneous. Half of the studies (18/36) met over three of the five quality criteria considered, while 5.6% (2/36) did not satisfy any quality criteria, and only 13.9% (5/36) satisfied all five criteria. Except for handling incomplete outcome data, a relatively large proportion of RCTs (41.7% for generating random sequence, 30.6% for concealing allocation, 19.4% for blinding of participants, and 33.3% for masking of assessors) did not report sufficient information for judging whether high or low risk of biases existed (please see Fig 2 for details).

Fig 2. Risk of bias chart: Each item summarized according to the authors’ judgments of risk of bias presented as a percentage of all the included studies.

Meta-analysis of CBM effect on all outcome categories

SAD primary symptoms.

Main effect sizes. CBM had small but significant effects on measures of SAD symptoms at post-test (g = 0.26, 95% CI: 0.09 to 0.43, N = 25, Z = 2.95, P = 0.003). There was strong evidence of heterogeneity across the included RCTs [Q(24) = 63.94, P < 0.001, I2 = 62.46%]. One study [69] was considered an outlier (using a cut-off of 2.5 SD from the average ES) [53]. With its removal, the effect was still significant but decreased (g = 0.17, 95% CI: 0.04 to 0.30, N = 24, Z = 2.52, P = 0.012) and the heterogeneity became non-significant [Q(23) = 33.92, P = 0.066, I2 = 32.20]. We removed this outlier study from further analyses. Furthermore, we conducted a sensitivity analysis wherein we removed each study at a time and computed the ESs for the remaining studies; overall, the results were not influenced by any one study.

Subgroup and meta-regression analysis. The subgroup analysis was used to test the influence of the categorical moderators. As shown in Table 2, for type of bias, effect sizes for CBM-I were significantly higher than they were for ABM on SAD symptoms. Additionally, studies based on a laboratory environment had significantly larger ESs than did those delivered via the Internet. Heterogeneity remained in the subclinical samples, showing that other possible major sources of heterogeneity might exist. The studies wherein CB was successfully modified had significantly larger ESs than did those wherein CB was not modified.

Table 2. Subgroup analysis for the primary symptoms of social anxiety disorder at post-test.

Meta-regressions were applied to examine the effect of continuous moderators. The results showed that the post-test ESs were significantly moderated by three variables (see Table 3). First, the percentage of women in the sample moderated ES; larger ESs were found in the sample that comprised a greater percentage of women. Second, participants’ age within studies showed a significant moderating effect, such that younger participants benefited the most from CBM. Finally, a significant, negative relationship was found between publication year and ES.

Table 3. Meta-regression for the outcome categories at post-test.

Cognitive bias.

Main effect sizes. The effect of CBM at post-test on CB was small to medium (g = 0.44, 95% CI: 0.23 to 0.64, N = 24, Z = 4.15, P < 0.001). However, there was high and significant heterogeneity among the included studies [Q(23) = 83.84, P = 0.000, I2 = 72.57%]. Four studies [21, 26, 62, 65] were identified as outliers (ESs ≥ 2.5 SD from the mean). However, ESs remained significant after the outliers’ removal (g = 0.32, 95% CI: 0.21 to 0.43, N = 20, Z = 5.94, P < 0.001) and the heterogeneity became nonsignificant [Q(19) = 18.34, P = 0.500, I2 = 0.00%]. These outlier studies were excluded from further analyses. We also performed a sensitivity analysis by removing one study at a time; the results did not change with the removal of any one study.

Subgroup and meta-regression analysis. Regarding the categorical moderators, the subgroup analysis showed that two moderators had significant effects on CB at post-test (see Table 4). First, regarding the type of bias, ESs for CBM-I were significantly higher than they were for ABM for CB. Second, feedback about training performance also moderated post-test ES; the studies wherein participants received feedback were likely to obtain higher ESs than were those that did not.

Regarding the continuous moderators, the meta-regression results indicated that two moderating variables were significant for ESs at post-test: participants’ age and the training contingency (see Table 3). More specifically, the younger the participants and the higher the contingency between the cues and probe were, the larger were the ESs.

Stressor challenge.

Main effect sizes. CBM had a small but significant effect on stressor challenge at post-test (g = 0.25, 95% CI: 0.02 to 0.49, N = 10, Z = 2.14, P = 0.032); there was no evidence of heterogeneity [Q(9) = 15.41, P = 0.080, I2 = 41.60%] and no outlier studies. The sensitivity analysis revealed that no specific study drove the results.

Subgroup and meta-regression analysis. No categorical moderators were significant (see Table 5). The potential moderating effects of the three categorical variables could not be explored: for type of bias and feedback, it was because only two studies examined the effects of CBM-I and feedback [65, 67], while for training setting, it was because only one study was delivered by the Internet [68]. The degree of heterogeneity within the diagnosed samples was significant, which suggests that there were also several other potential sources of heterogeneity.

Table 5. Subgroup analysis for reactivity in stressful situations at post-test.

Meta-regressions revealed that CB at baseline, age of participants, and the year of journal publication moderated post-test ESs for stressor challenge (see Table 3). More specifically, smaller ESs for CB at baseline, and older participants, and more recent publications were associated with smaller ESs.

Secondary symptoms.

Main effect sizes. CBM had a non-significant effect on secondary symptoms at post-test (g = 0.03, 95% CI: -0.10 to 0.17, N = 19, Z = 0.47, P = 0.640). The removal of one outlier [69] did not alter the result (g = -0.02, 95% CI: -0.12 to 0.09, N = 18, Z = -0.28, P = 0.779), and the heterogeneity was non-significant [Q(17) = 14.51, P = 0.779, I2 = 0.00%]. The sensitivity analysis revealed that the results were not altered by the removal of any one study.

Subgroup and meta-regression analysis. We did not conduct moderator analyses because the general effect was not significant.

Publication bias

Publication bias differed for SAD primary symptoms, CB, reactivity in stressful situations, and secondary symptoms at post-test. For SAD primary symptoms, there was no evidence to support publication bias either through examination of the funnel plot shown in Fig 3a or based on the Duval and Tweedie’s trim-and-fill method. Nevertheless, visual inspection of the funnel plots (i.e., Fig 3b, 3c and 3d) showed that there was some evidence of publication bias for the other three dependent measures. For CB, Duval and Tweedie’s trim-and-fill method revealed that one study was missing on the left-hand side. A new mean ES (g = 0.31, 95% CI: 0.21 to 0.42, Q = 19.73) was imputed by estimating a random-effects model. For reactivity in stressful situations, the trim-and-fill method identified two studies that needed to be trimmed, which decreased the ES to non-significant (g = 0.14, 95% CI: -0.09 to 0.38, Q = 23.64). For secondary symptoms, the trim-and-fill method indicated that five studies on the left needed to be omitted, resulting in an ES of g = -0.07 (95% CI: -0.17 to 0.04, Q = 23.56) after adjustment for missing studies.

Fig 3. Funnel plots to assess publication bias.

(a) SAD primary symptoms; (b) cognitive bias; (c) reactivity in stressful situations; and (d) secondary symptoms.


Given the increase in the body of literature on CBM interventions for SAD and the inconsistent findings in past studies, we thought a quantitative review of studies would be needed. First, we assessed the clinical utility of the CBM procedures for the different types of outcome measure described above. Next, we investigated the possible moderators of treatment response that might have an effect on efficiency of CBM training. Finally, we evaluated the quality of the CBM studies using the “gold standard” tool drawn up by the Cochrane Collaboration to evaluate potential sources of bias. We also evaluated publication bias in two ways: a funnel plot and Duval and Tweedie’s trim-and-fill method.

Does benign CBM facilitate a reduction in negative cognitive bias and alleviate SAD symptoms?

Our results partly supported and extended the findings from the previous meta-analysis, which examined only the effects of ABM [33]. We found that CBM had a small-to-medium effect on cognitive bias, while it exerted a positive but small effect on SAD primary symptoms and reactivity in stressful situations at post-intervention. Furthermore, in the meta-analysis including all samples, the effects of CBM intervention showed some heterogeneity for all outcome categories except for reactivity in stressful situations. After removal of outliers, ESs significantly decreased across all outcome categories, but still significant. These results contradict a previous meta-analysis by Cristea et al (2015), who concluded that the ESs for reduction of SAD symptoms became non-significant when outliers were excluded and after adjustment for publication bias [34]. The inconsistent findings were likely partly because the two meta-analyses pooled outcomes from different groups: the meta-analysis by Cristea et al included 9 RCTs on social anxiety, of which 6 focused on a clinical sample (after excluding outliers). In contrast, our meta-analysis included 24 RCTs on SAD primary symptoms, of which 13 focused on a clinical sample after outliers were excluded. Heeren (2015) further suggested that the clinical efficacy and varying moderators (e.g., participants’ age, number of sessions, type of threatening stimuli used during training) might be the main cause of such inconsistencies [13]. The current findings suggested that the clinical therapeutic efficacy for a reduction in the symptoms of SAD produced by CBM was mild. There are several possible explanations for these results. First, in order to enrich cognitive theories of SAD and enhance the availability of CBM programs, researchers frequently test new variations of CBM by varying the tasks, experimental materials, instructions, number of sessions, etc. Such attempts might generate nonsignificant ESs or even negative ones, and consequently, these data might contribute to the attenuation of the total ES when we pool the results from the relevant studies to evaluate the efficacy of CBM treatment for SAD. Second, the reinforcement sensitivity theory (RST) proposes a unified model of SAD that integrates a wide range of factors and suggests that CB is just one of the most important factors; it is not, in other words, a uniquely crucial factor. Indeed, many factors in the model (i.e., genetic, biological, temperamental, and environmental factors) are simultaneously associated with the development and maintenance of SAD [75,76]. This standpoint is at odds with cognitive theories of SAD, which suggest that cognitive bias contributes significantly to the onset, development, and maintenance of SAD [7779]. Third, Clark suggested that social anxiety was characterized by a diverse range of biases in information processing [80]. However, most CBM interventions only target one type of negative CB (i.e. attention bias) and the effects were not optimal, meaning that the other types of CB would not be effectively modified. Furthermore, in some studies, even the target CB could not be successfully modified, despite the CBM having an effect on SAD symptoms by modifying an initial negative CB [81]. As a result, the ESfor the reduction of SAD primary symptoms would be increased by filtering those studies that failed to alter the bias. A recent meta-analysis of interventions for adults with SAD uncovered the interesting finding that a number of psychological treatment methods, including psychodynamic psychotherapy, interpersonal psychotherapy, cognitive behavioral therapy (CBT), mindfulness, and supportive therapy, were all effective to varying degrees in reducing symptoms [82]. This might partly confirm the famous “dodo bird verdict,” which proposes that all psychotherapies have equal effects [83]. Psychological treatment specific to SAD is no exception. It would make sense, then, that each type of psychological intervention for SAD, including CBM, merely addresses certain aspects of a sufferers’ problems. Relatedly, some studies have found that combining CBM with CBT could enhance the training effects on social anxiety [68,84,85]. This hints that CBM might be a beneficial complement to traditional methods of psychotherapy.

How do we enhance the effectiveness of benign CBM?

As previous findings have indicated [34,35], we found that ABM had smaller ESs for SAD symptoms and CB at post-test than did CBM-I; in fact, the effect on SAD symptoms was nonsignificant. This difference might be explained from an information processing perspective, in which AB relates more to automatic processing systems whereas interpretative bias relates to more conscious, strategic processes [39,86]. Furthermore, the content of CBM training materials mostly feature various social situations (i.e., public speaking and meeting strangers), which are more specific to social anxiety. It might be that CBM-I interventions have a more powerful influence on deeper cognitive mechanisms (i.e., dysfunctional cognitive schemas) that possibly underlie these biases, rather than merely temporarily “switching off” CBs. Additionally, a qualitative study about socially anxious individuals’ attitudes toward CBM found that participants reported greater understanding and engagement with the CBM-I program than with the ABM program [87]. Intriguingly, recent research has shown that, for social anxiety, benign ABM did not excel in improving symptoms compared to placebo control procedures [27,32,42,45,56,57,66,68]. A recent study further suggested that adding a negative intervention as a control group might be more beneficial for ABM specific to social anxiety [34]. We agree with them and argue that it was not difficult to find that almost all of the above ABM interventions that did not show an advantage were delivered remotely.

The present meta-analysis showed that ABM programs conducted in a laboratory yielded significantly larger ESs for SAD primary outcomes compared with ABM programs conducted via the Internet. The two previous meta-analyses had similar results [33,37], consistently reporting greater SAD symptom reductions in the laboratory condition than in the remote condition. Previous researchers [57,88] have pointed out that participation in a relatively unconstrained home-based setting might be subject to external distractions, thus reducing the benefits of Internet-delivered procedures. Emmelkamp even suggested that it might be necessary to keep a watchful eye on ABM delivered via the Internet [89]. Later on, two scholars commented on this inference and believed that more research in this field is necessary [90,91]. We agree with the latter and makeseveral recommendations for future CBM studies on SAD that will further enhance our understanding. First, it would be necessary to formulate more rigorous guidance (just as for internet-delivered CBT, see [92]) for participants receiving CBM interventions delivered by the Internet, which would help guarantee consistency and quietness in the treatment environment without distractors. Second, it would help to make the CBM tasks more understandable and engaging in order to offset the adverse effects produced by distractors. Finally, it would be important to evaluate the reliability of CBM delivered remotely in naturalistic settings in future research.

The meta-regression analysis showed that a higher percentage of training contingency between the location of benign cues and the probe in CBM was associated with larger CB reduction. There should be a dose–response gradient between these two variables, supposing that the percentage of contingency is defined as the active dose that might direct attention disengagement from the threatening stimuli delivered to participants. These findings are not quite the same as those reported in previous meta-analyses [33,93,94], that regarded the number of CBM sessions completed as the dose. Although conceptual discrepancies concerning the dose might exist, the current findings suggest that future CBM studies should include a higher training contingency, as this is likely to lead to greater effects on reducing CB and SAD symptoms.

Although studies in which participants were or were not provided feedback during training both have significant effects on CB reduction, the results herein suggest that the presence of feedback to participants yielded larger ESs than the absence did. Consistent with Menne-Lothmann et al.’s meta-analysis [36], feedback on response accuracy might encourage participants to engage in and concentrate on the information presented. In this way, participants might receive greater assistance in decreasing the negative CB compared with the sham condition [25,95].

Do relatively susceptible populations benefit more from CBM?

The meta-regression results suggest that benign CBM was particularly effective for women. In line with a previous meta-analysis [36], studies that included a higher percentage of female participants tended to show higher ESs for the reduction in SAD symptoms. In line with the “gender differences in processing content” hypothesis [96], some studies have showed that gender moderates the relationship between CBs and anxiety [9799], while there is some evidence that women showed greater processing of stimuli than did men at the early attention stage in anxious arousal states [100]. A previous study found gender differences in social anxiety, specifically, more women than men reported greater fear regarding going to a party or speaking at a meeting, whereas more men than women reported greater fear regarding urinating in a public restroom [101]. Current CBM procedures mainly adopt emotional faces or words or social scenarios as experimental materials, which might be more beneficial to female participants. Future CBM interventions could take gender differences into consideration to ensure more obvious therapeutic effects.

The present meta-analysis also examined the relationship between participants’ age and the efficacy of CBM across most outcome categories, and we consistently found that younger participants benefited more from CBM interventions than did older ones (see [33,34]). One researcher concluded that when healthy and educated adults are in their twenties and thirties, they begin exhibiting declines in certain age-related cognitive abilities [102]. It is possible that cognitive reappraisal and executive function, which are associated with the prefrontal cortex, are highly vulnerable to age-related declines [103,104]. Thus, cognitive reappraisal, an important contributor to emotion regulation (which itself requires intact cognitive control ability), might be more successfully used by younger participants. Conversely, age-related deficits in executive processes involving attention, switching, inhibition, and working memory might lead to older participants’ poorer performance compared to that of younger participants on a series of cognitive tasks [105].

Interestingly, we also found that greater CB at baseline within studies seemingly yielded stronger ESs for reactivity in stressful situations at post-test. Similarly, Mogoașe found that AB at baseline correlated significantly with a change in AB post intervention [37]. However, we cannot be sure that there was a “true” baseline CB in the groups receiving CBM intervention. Therefore, these results must be cautiously interpreted.

Did the risk of publication bias exist in the current meta-analysis?

The quality of the included studies, as assessed by the Cochrane Collaboration Risk of Bias Tool (CCRBT), was substandard. More specifically, half of the studies satisfied fewer than three quality standards. Moreover, a varied proportion of the included studies did not provide information necessary for judging the five risks of bias (ranging from 5.6 to 41.7%).

However, positively biased selection was small or even non-existent for some of the outcomes, owing to negative or null findings included in the meta-analysis. The meta-regression results indicated that there were significant negative linear relationships of publication year with SAD symptom reduction and reactivity in stressful situations. Consistent with a previous meta-analysis [34], more recent experiments had smaller ESs than did earlier ones. As in other types of intervention research, CBM is an innovative approach, which means that it is likely subject to publication bias—that is, “positive” and statistically significant studies are likely to be published more rapidly than are studies with “negative” and statistically non-significant results [106108]. Thus, we could not agree more with the suggestion that, to avoid publication bias, psychotherapy researchers should prospectively register all trials before their results are known.

Limitations and future directions

There are several limitations to the present meta-analysis. First, the apples-and-oranges problem persists in this study, which is a common critique of meta-analyses [109]. More specifically, the results from different studies that measured different variables and described different populations were combined statistically in this meta-analysis. Admittedly, there was some evidence of heterogeneity for SAD symptoms and CB at posttest. However, the removal of outliers reduced the heterogeneity to non-significance for the main outcome categories. Nonetheless, when we conducted the subgroup analyses to examine differences between varying study features, I2 remained large for a few outcomes, indicating that heterogeneity remained. This is possibly because of the different types of measures used in CBM studies, which we were only able to classify roughly. Second, there was some evidence-based publication bias in the post-test data set. Often, studies with nonsignificant results are less likely to be published than are studies with significant results. To this end, we contacted authors and requested unpublished data if published articles lacked the essential data to calculate an ES. The results did not change after taking these steps. Third, on the whole, the quality of the included studies was suboptimal, with only half of the included studies meeting three of the five quality criteria and two studies meeting none. Moreover, some studies did not report sufficient information for assessing whether the quality criteria were satisfied, which further increases bias. Finally, it was impossible to compute categorical moderation analyses, as some subgroups only included a small number of studies (i.e., different types of training setting when investigating stressor challenges at post-test).

In view of our finding that CBM had small effect sizes on SAD symptoms, future CBM research could be developed in several ways, each of which also has potentially important theoretical or clinical implications. First, as stated earlier, future research could explore the crucial characteristics that might enhance the effectiveness of CBM, especially in real-world settings, and clarify the functional relationship between CBM and different facets of SAD symptomatology. Recently, a review on the neural effects of CBM for anxiety, addiction, and depression was published in NeuroImage, which included a total of 13 published studies that used (functional) magnetic resonance imaging (MRI) or electroencephalography as an outcome measure; 5 of these studies were on social anxiety [110]. Future studies could thus adopt some techniques commonly used in cognitive neuroscience (i.e., fMRI) to explore the possible neurobiological basis of CB in SAD. Additionally, future research could refine the therapeutic applications of CBM procedures during clinical interventions for SAD in order to optimize their capacity to change target CBs and reduced SAD symptoms. Second, we found that CBM-I was more successful for modifying CB and reducing SAD symptom. Preliminary data indicated that CBM-I could not only guide participants’ interpretation in positive directions, but also has the ability to disengage attention from threat stimuli [55]. This notion is compatible with the combined CB hypothesis [111], which suggested that a change in interpretative bias might influence other aspects of cognitive processing (i.e., attention bias); that is, different forms of CB might interact with each other to maintain social anxiety. Future CBM research specific to SAD must verify the complicated cognitive mechanism underlying SAD and show how variants of CBM program targeting one CB influence other types of bias and then reduces social anxiety. Third, we think that the cultural context of participants exhibiting social anxiety should be taken into account in future CBM interventions. Specifically, Asian individuals living in collectivistic cultures, which emphasize on harmony within the group, tend to have a more interdependent self-construal. In contrast, individuals with a European heritage living in individualistic cultures that value individual achievements and success, tend to possess a more independent self-construal [112,113]. Hence, future studies should examine cultural differences in CBM interventions for social anxiety.

Supporting information

S2 Table. Coding of the four categories for the dependent measures.



We want to thank Tongqi Wei of the National Science Library of the Chinese Academy of Sciences for helping us devise the search strategy.

Author Contributions

  1. Conceptualization: HL.
  2. Data curation: HL.
  3. Formal analysis: HL Xianwen Li.
  4. Funding acquisition: BH.
  5. Investigation: Xiaoqian Liu.
  6. Methodology: HL Xianwen Li.
  7. Project administration: BH.
  8. Resources: Xiaoqian Liu.
  9. Supervision: BH.
  10. Validation: HL.
  11. Visualization: Xianwen Li.
  12. Writing – original draft: HL.
  13. Writing – review & editing: BH.


  1. 1. National Collaborating Centre for Mental Health. Social anxiety disorder: recognition, assessment and treatment. British Psychological Society, 2013.
  2. 2. Steinman SA and Teachman BA. Modifying interpretations among individuals high in anxiety sensitivity. J Anxiety Disord. 2010; 24(1): 71–8. pmid:19747797
  3. 3. Stein MB and Stein DJ, Social anxiety disorder. The Lancet. 2008; 371(9618): 1115–25.
  4. 4. Eaton NR, Keyes KM, Krueger RF, Balsis S, Skodo AEl, Markon KE et al. An invariant dimensional liability model of gender differences in mental disorder prevalence: evidence from a national sample. J Abnorm Psychol. 2012; 121(1): 282–8. pmid:21842958
  5. 5. Schneier F, Goldmark J. Social Anxiety Disorder, in Anxiety Disorders and Gender. Springer. 2015; 49–67.
  6. 6. Xu Y, Schneier F, Heimberg RG, Princisvalle K, Liebowitz MR, Wang S, et al. Gender differences in social anxiety disorder: Results from the national epidemiologic sample on alcohol and related conditions. J Anxiety Disord. 2012; 26(1): 12–9. pmid:21903358
  7. 7. Katzelnick DJ, Kobak KA, DeLeire T, Henk HJ, Greist JH, Davidson JR, et al. Impact of generalized social anxiety disorder in managed care. AM J PSYCHIAT, 2014.
  8. 8. Cramer P. Understanding Defense Mechanisms. Psychodynamic Psychiatry. 2015; 43(4): 523–52. pmid:26583439
  9. 9. Leichsenring F, Beutel M, Leibing E. Psychodynamic psychotherapy for social phobia: a treatment manual based on supportive-expressive therapy. Bulletin of the Menninger Clinic. 2007; 71(1): 56–84. pmid:17484670
  10. 10. Keefe JR, McCarthy KS, Dinger U, Zilcha-Mano S, Barber JP. A meta-analytic review of psychodynamic therapies for anxiety disorders. Clin Psychol Rev. 2014; 34(4): 309–23. pmid:24786897
  11. 11. Angélico AP, Crippa JAdS, Loureiro SR. Social anxiety disorder and social skills: A critical review of the literature. International Journal of Behavioral Consultation and Therapy. 2013; 7(4): 16–23.
  12. 12. Epkins CC, Heckler DR. Integrating etiological models of social anxiety and depression in youth: Evidence for a cumulative interpersonal risk model. Clin Child Fam Psych. 2011; 14(4): 329–76.
  13. 13. Heeren A, Lange WG, Philippot P, Wong QJ. Biased cognitions and social anxiety: building a global framework for integrating cognitive, behavioral, and neural processes. Front Hum Neurosci. 2014; 8: 538. pmid:25100980
  14. 14. Hofmann SG. Cognitive factors that maintain social anxiety disorder: A comprehensive model and its treatment implications. Cogn Behav Ther. 2007; 36(4): 193–209. pmid:18049945
  15. 15. LeMoult J, Joormann J. Attention and memory biases in social anxiety disorder: The role of comorbid depression. Cogn Behav Ther. 2012; 36(1): 47–57.
  16. 16. Morrison AS, Heimberg RG. Social anxiety and social anxiety disorder. Annu Rev Clin Psychol. 2013; 9: 249–74. pmid:23537485
  17. 17. Van Bockstaele B, Verschuere B, Tibboel H, De Houwer J, Crombez G, Koster EH. A review of current evidence for the causal impact of attentional bias on fear and anxiety. Psychol Bull. 2014; 140(3): 682–721. pmid:24188418
  18. 18. Hirsch CR, Mathews A. A cognitive model of pathological worry. Behav Res Ther. 2012; 50(10): 636–46. pmid:22863541
  19. 19. Moser JS, Huppert JD, Foa EB, Simons RF. Interpretation of ambiguous social scenarios in social phobia and depression: evidence from event-related brain potentials. Biol Psychol. 2012; 89(2): 387–97. pmid:22178443
  20. 20. MacLeod C, Rutherford E, Campbell L, Ebsworthy G, Holker L. Selective attention and emotional vulnerability: Assessing the causal basis of their association through the experimental manipulation of attentional bias. J Abnorm Psychol. 2002; 111(1): 107–23. pmid:11866165
  21. 21. Beard C, Amir N. A multi-session interpretation modification program: changes in interpretation and social anxiety symptoms. Behav Res Ther. 2008; 46, 1135–41. pmid:18675400
  22. 22. Rinck M, Telli S, Kampmann IL, Woud ML, Kerstholt M, Te Velthuis S, et al. Training approach-avoidance of smiling faces affects emotional vulnerability in socially anxious individuals. Front Hum Neurosci. 2013; 7: 481. pmid:23970862
  23. 23. Amir N, Beard C, Taylor CT, Klumpp H, Elias J, Bums M, et al. Attention Training in Individuals With Generalized Social Phobia: A Randomized Controlled Trial. J Consult Clin Psychol. 2009; 77(5): 961–73. pmid:19803575
  24. 24. Amir N, Taylor CT. Interpretation Training in Individuals With Generalized Social Anxiety Disorder: A Randomized Controlled Trial. J Consult Clin Psychol. 2012; 80(3): 497–511. pmid:22250851
  25. 25. Beard C, Weisberg RB, Amir N. Combined cognitive bias modification treatment for social anxiety disorder: a pilot trial. Depress Anxiety. 2011; 28, 981–8. pmid:21898705
  26. 26. Heeren A, Reese HE, McNally RJ, Philippot P. Attention training toward and away from threat in social phobia: Effects on subjective, behavioral, and physiological measures of anxiety. Behav Res Ther. 2012; 50(1): 30–9. pmid:22055280
  27. 27. Enock PM, Hofmann SG, McNally RJ. Attention bias modification training via smartphone to reduce social anxiety: A randomized, controlled multi-session experiment. Cognit Ther Res. 2014; 38, 200–16.
  28. 28. Klumpp H, Amir N. Preliminary study of attention training to threat and neutral faces on anxious reactivity to a social stressor in social anxiety. Cognit Ther Res. 2010; 34(3): 263–71.
  29. 29. Asnaani A, Rinck M, Becker E, Hofmann SG. The Effects of Approach-Avoidance Modification on Social Anxiety Disorder: A Pilot Study. Cognit Ther Res. 2014; 38(2): 226–38. pmid:24659832
  30. 30. Boettcher J, Leek L, Matson L, Holmes EA, Browning M, MacLeod C, et al. Internet-Based Attention Bias Modification for Social Anxiety: A Randomised Controlled Comparison of Training towards Negative and Training Towards Positive Cues. PloS one. 2013; 8(9): e71760. pmid:24098630
  31. 31. Bunnell BE, Beidel DC, Mesa F. A Randomized Trial of Attention Training for Generalized Social Phobia: Does Attention Training Change Social Behavior? Behav Ther. 2013; 44(4): 662–73. pmid:24094791
  32. 32. McNally R, Enock P, Tsai C, Tousian M. Attention bias modification for reducing speech anxiety. Behav Res Ther. 2013; 51(12): 882–8. pmid:24211339
  33. 33. Heeren A, Mogoașe C, Philippot P, McNally RJ. Attention bias modification for social anxiety: a systematic review and meta-analysis. Clin Psychol Rev. 2015; 40: 76–90. pmid:26080314
  34. 34. Cristea IA, Kok RN, Cuijpers P. Efficacy of cognitive bias modification interventions in anxiety and depression: meta-analysis. Br J Psychiatry. 2015; 206(1): 7–16. pmid:25561486
  35. 35. Hallion LS, Ruscio AM. A meta-analysis of the effect of cognitive bias modification on anxiety and depression. Psychol Bull. 2011; 137(6): 940–58. pmid:21728399
  36. 36. Menne-Lothmann C, Viechtbauer W, Höhn P, Kasanova Z, Haller SP, Drukker M, et al. How to boost positive interpretations? A meta-analysis of the effectiveness of cognitive bias modification for interpretation. PloS one. 2014; 9(6): e100925. pmid:24968234
  37. 37. Mogoaşe C, David D, Koster EH. Clinical efficacy of attentional bias modification procedures: An updated meta-analysis. J Clin Psychol. 2014; 70(12): 1133–57. pmid:24652823
  38. 38. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2010;8(5):336–41.
  39. 39. Mobini S, Reynolds S, Mackintosh B. Clinical implications of cognitive bias modification for interpretative biases in social anxiety: An integrative literature review. Cognit Ther Res. 2013; 37(1): 173–82.
  40. 40. Heeren A, Lievens L, Philippot P. How does attention training work in social phobia: Disengagement from threat or re-engagement to non-threat? J Anxiety Disord. 2011; 25(8): 1108–15. pmid:21907539
  41. 41. Julian K, Beard C, Schmidt NB, Powers MB, Smits JA. Attention training to reduce attention bias and social stressor reactivity: an attempt to replicate and extend previous findings. Behav Res Ther. 2012; 50, 350–8. pmid:22466022
  42. 42. Boettcher J, Leek L, Matson L, Holmes EA, Browning M, MacLeod C, et al. Internet-based attention bias modification for social anxiety: a randomised controlled comparison of training towards negative and training towards positive cues. PLoS One. 2013; 8(9): e71760. pmid:24098630
  43. 43. Amir N, Taylor CT, Donohue MC. Predictors of response to an attention modification program in generalized social phobia. J Consult Clin Psychol. 2011; 79 (4): 533–41. pmid:21707134
  44. 44. Kuckertz J, Gildebrant E, Liliequist B, Karlstrom P, Vaeppling C, Bodlund O, et al. Moderation and mediation of the effect of attention training in social anxiety disorder. Behav Res Ther. 2014; 53: 30–40. pmid:24373984
  45. 45. Carlbring P, Apelstrand M, Sehlin H, Amir N, Rousseau A, Hofmann SG, et al. Internet-delivered attention bias modification training in individuals with social anxiety disorder--a double blind randomized controlled trial. BMC psychiatry. 2012; 12(1): 66.
  46. 46. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Bmj. 2011; 343: d5928. pmid:22008217
  47. 47. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, 1988.
  48. 48. Hedges C, Olkin I. Statistical techniques for meta-analysis. Academic Press, 1985.
  49. 49. Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. Bmj. 2011; 342: d549. pmid:21310794
  50. 50. Cooper H, Hedges LV, Valentine JC. The handbook of research synthesis and meta-analysis. Russell Sage Foundation, 2009.
  51. 51. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in medicine. 2002; 21(11): 1539–58. pmid:12111919
  52. 52. Duval S. Tweedie R. Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000; 56(2): 455–63. pmid:10877304
  53. 53. Lipsey MW, Wilson DB. Practical meta-analysis. Sage Publications, Inc, 2001.
  54. 54. Amir N, Weber G, Beard C, Bomyea J, Taylor CT. The effect of a single-session attention modification program on response to a public-speaking challenge in socially anxious individuals. J Abnorm Psychol. 2008; 117, 860–8. pmid:19025232
  55. 55. Amir N, Bomyea J, Beard C. The effect of single-session interpretation modification on attention bias in socially anxious individuals. J Anxiety Disord. 2010; 24, 178–82. pmid:19926442
  56. 56. Boettcher J, Berger T, Renneberg B. Internet-based attention training for social anxiety: A randomized controlled trial. Cognit Ther Res. 2012; 36(5): 522–36.
  57. 57. Carleton RN, Sapach MJNT, Oriet C, Duranceau S, Lix LM, Thibodeau MA, et al. A randomized controlled trial of attention modification for social anxiety disorder. J Anxiety Disord. 2015; 33: 35–44. pmid:26047059
  58. 58. De Voogd EL, Wiers RW, Prins PJM, Salemink E. Visual search attentional bias modification reduced social phobia in adolescents. Journal of Behav Ther and Experimental Psychiatry. 2014; 45(2): 252–9.
  59. 59. Heeren A, Lievens L, Philippot P. How does attention training work in social phobia: disengagement from threat or re-engagement to non-threat? J Anxiety Disord. 2011; 25(8): 1108–15. pmid:21907539
  60. 60. Heeren A, Mogoaşe C, McNally RJ, Schmitz A, Philippot P. Does attention bias modification improve attentional control? A double-blind randomized experiment with individuals with social anxiety disorder. J Anxiety Disord. 2015; 29: 35–42. pmid:25465885
  61. 61. Hoppitt L, Illingworth Josephine L, MacLeod C, Hampshire A, Dunn Barnaby D, Mackintosh B. Modifying social anxiety related to a real-life stressor using online Cognitive Bias Modification for interpretation. Behav Res Ther. 2014; 52: 45–52. pmid:24317281
  62. 62. Khalili-Torghabeh S, Fadardi JS, Mackintosh B, Reynolds S, Mobini S. Effects of a multi-session Cognitive Bias Modification program on interpretative biases and social anxiety symptoms in a sample of Iranian socially-anxious students. J Exp Psychopathol. 2014; 5(4): 514–27.
  63. 63. Li S, Tan J, Qian M, Liu X. Continual training of attentional bias in social anxiety. Behav Res Ther. 2008; 46: 905–12. pmid:18538305
  64. 64. Maoz K, Abend R, Fox NA, Pine DS, Bar-Haim Y. Subliminal attention bias modification training in socially anxious individuals. Front Hum Neurosci. 2013; 7: 389. pmid:23888138
  65. 65. Murphy R, Hirsch CR, Mathews A, Smith K, Clark DM. Facilitating a benign interpretation bias in a high socially anxious population. Behav Res Ther. 2007; 45(7): 1517–29. pmid:17349970
  66. 66. Neubauer K, Auer M, Murray E, Petermann F, Helbig-Lang S, Gerlach AL. Internet-delivered attention modification training as a treatment for social phobia: a randomized controlled trial. Behav Res Ther. 2013; 51: 87–97. pmid:23262116
  67. 67. Nowalcowski ME, Antony MM, Koerner N. Modifying interpretation biases: Effects on symptomatology, behavior, and physiological reactivity in social anxiety. Journal of Behav Ther and Experimental Psychiatry. 2015; 49: 44–52.
  68. 68. Rapee RM, MacLeod C, Carpenter L, Gaston JE, Frei J, Peters L, et al. Integrating cognitive bias modification into a standard cognitive behavioural treatment package for social phobia: a randomized controlled trial. Behav Res Ther. 2013; 51(4–5): 207–15. pmid:23435121
  69. 69. Schmidt NB, Richey JA, Buckner JD, Timpano KR. Attention training for generalized social anxiety disorder. J Abnorm Psychol. 2009; 118, 5–14. pmid:19222309
  70. 70. Sportel BE, Hullu Ed, Jong PJ, Nauta MH. Cognitive Bias Modification versus CBT in Reducing Adolescent Social Anxiety: A Randomized Controlled Trial. PLoS One. 2013; 8(5): e64355. pmid:23691203
  71. 71. Taylor CT, Amir N. Modifying automatic approach action tendencies in individuals with elevated social anxiety symptoms. Behav Res Ther. 2012; 50(9): 529–36. pmid:22728645
  72. 72. Vassilopoulos SP, Banerjee R, Prantzalou C. Experimental modification of interpretation bias in socially anxious children: Changes in interpretation, anticipated interpersonal anxiety, and social anxiety symptoms. Behav Res Ther. 2009; 47(12): 1085–9. pmid:19679299
  73. 73. Vassilopoulos SP, Moberly NJ, Zisimatou G. Experimentally modifying interpretations for positive and negative social scenarios in children: a preliminary investigation. Behav Cogn Psychoth. 2013; 41: 103–16.
  74. 74. Vassilopoulos SP, Brouzos A. Cognitive bias modification of interpretations in children: Processing information about ambiguous social events in a duo. J Child Fam Stud. 2016; 25(1): 299–307.
  75. 75. Kimbrel NA. A model of the development and maintenance of generalized social phobia. Clin Psychol Rev. 2008; 28(4): 592–612. A model of the development and maintenance of generalized social phobia. pmid:17959288
  76. 76. Kimbrel NA, Nelson-Gray RO, Mitchell JT. BIS, BAS, and bias: The role of personality and cognitive bias in social anxiety. Pers Indiv Differ. 2012; 52(3): 395–400.
  77. 77. Clark DM, Wells A. A cognitive model of social phobia social phobia:Diagnosis, assessment, and treatment. Guilford Press. 1995: 69–93.
  78. 78. Mellings TM, Alden LE. Cognitive processes in social anxiety: The effects of self-focus, rumination and anticipatory processing. Behav Res Ther. 2000; 38(3): 243–57. pmid:10665158
  79. 79. Rapee RM, Heimberg RG. A cognitive-behavioral model of anxiety in social phobia. Behav Res Ther. 1997; 35(8): 741–56. pmid:9256517
  80. 80. Clark DM, McManus F. Information processing in social phobia. Biol Psychiat. 2002; 51(1): 92–100. pmid:11801234
  81. 81. Clarke PJF, Notebaert L, MacLeod C. Absence of evidence or evidence of absence: reflecting on therapeutic implementations of attentional bias modification[J]. BMC psychiatry, 2014, 14(1): 1–6.
  82. 82. Mayo-Wilson E, Dias S, Mavranezouli I, Kew K, Clark DM, Ades A, et al. Psychological and pharmacological interventions for social anxiety disorder in adults: a systematic review and network meta-analysis. Lancet Psychiat. 2014; 1(5): 368–76.
  83. 83. Mansell W. Core processes of psychopathology and recovery:“Does the Dodo bird effect have wings?”. Clin Psychol Rev. 2011; 31(2): 189–92. pmid:20624664
  84. 84. Butler E, Mobini S, Rapee RM, Mackintosh B, Reynolds SA. Enhanced effects of combined cognitive bias modification and computerised cognitive behaviour therapy on social anxiety. Cogent Psychology. 2015; 2(1): 1011905.
  85. 85. Williams AD, Blackwell SE, Holmes EA, Andrews G. Positive imagery cognitive bias modification (CBM) and internet-based cognitive behavioural therapy (iCBT) versus control CBM and iCBT for depression: study protocol for a parallel-group randomised controlled trial. BMJ Open. 2013; 3(10): e004049. pmid:24171941
  86. 86. Beck AT, Clark DA. An information processing model of anxiety: Automatic and strategic processes. Behav Res Ther. 1997; 35(1): 49–58. pmid:9009043
  87. 87. Beard C, Weisberg RB, Primack J. Socially anxious primary care patients' attitudes toward cognitive bias modification (CBM): a qualitative study. Behav Cogn Psychother. 2012; 40(5): 618–33. pmid:22127022
  88. 88. MacLeod C, Clarke PJF. The attentional bias modification approach to anxiety intervention. Clinical Psychological Science. 2015; 3(1): 58–78.
  89. 89. Emmelkamp PMG. Attention bias modification: the Emperor's new suit? BMC Med. 2012; 10(1): 63.
  90. 90. Clarke PJ, Browning M, Hammond G, Notebaert L, MacLeod C. The causal role of the dorsolateral prefrontal cortex in the modification of attentional bias: evidence from transcranial direct current stimulation. Biol Psychiatry. 2014; 76(12): 946–52. pmid:24690113
  91. 91. Fox E, Mackintosh B, Holmes EA. Travellers’ tales in cognitive bias modification research: A commentary on the special issue. Cognit Ther Res. 2014; 38(2): 239–47.
  92. 92. Cuijpers P, Smit F, Bohlmeijer E, Hollon SD, Andersson G. Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. Brit J Psychiat. 2010; 196(3): 173–8. pmid:20194536
  93. 93. Beard C, Sawyer AT, Hofmann SG. Efficacy of attention bias modification using threat and appetitive stimuli: A meta-analytic review. Behav Ther. 2012; 43(4): 724–40. pmid:23046776
  94. 94. Hakamata Y, Lissek S, Bar-Haim Y, Britton JC, Fox NA, Leibenluft E, et al. Attention bias modification treatment: a meta-analysis toward the establishment of novel treatment for anxiety. Biol Psychiatry. 2010; 68(11): 982–90. pmid:20887977
  95. 95. Hertel PT, Mathews A. Cognitive Bias Modification: Past Perspectives, Current Findings, and Future Applications. Perspect Psychol Sci. 2011; 6(6): 521–36. pmid:26168375
  96. 96. Stewart SH, Taylor S, Baker JM. Gender differences in dimensions of anxiety sensitivity. J Anxiety Disord. 1997; 11(2): 179–200. pmid:9168341
  97. 97. Cannon MF, Weems CF. Cognitive biases in childhood anxiety disorders: Do interpretive and judgment biases distinguish anxious youth from their non-anxious peers? J Anxiety Disord. 2010; 24(7): 751–8. pmid:20554426
  98. 98. Keogh E, Hamid R, Hamid S, Ellery D. Investigating the effect of anxiety sensitivity, gender and negative interpretative bias on the perception of chest pain. Pain. 2004; 111(1): 209–17.
  99. 99. Zhao X, Zhang P, Chen L, Zhou R. Gender differences in the relationship between attentional bias to threat and social anxiety in adolescents. Pers Indiv Differ. 2014; 71, 108–12.
  100. 100. Sass SM, Heller W, Stewart JL, Silton RL, Edgar JC, Fisher JE, et al. Time course of attentional bias in anxiety: emotion and gender specificity. Psychophysiology. 2010; 47(2): 247–59. pmid:19863758
  101. 101. Turk CL, Heimberg RG, Orsillo SM, Holt CS, Gitow A, Street LL, et al. An investigation of gender differences in social phobia. J Anxiety Disord. 1998; 12(3): 209–23. pmid:9653680
  102. 102. Salthouse TA. When does age-related cognitive decline begin? Neurobiol Aging. 2009; 30(4): 507–14. pmid:19231028
  103. 103. Mozolic JL, Long AB, Morgan AR, Rawley-Payne M, Laurienti PJ. A cognitive training intervention improves modality-specific attention in a randomized controlled trial of healthy older adults. Neurobiol Aging. 2011; 32(4): 655–68. pmid:19428142
  104. 104. Opitz PC, Rauch LC, Terry DP, Urry HL. Prefrontal mediation of age differences in cognitive reappraisal. Neurobiol Aging. 2012; 33(4): 645–55. pmid:20674090
  105. 105. Braver TS, Barch DM, Keys BA, Carter CS, Cohen JD, Kaye JA, et al. Context processing in older adults: evidence for a theory relating cognitive control to neurobiology in healthy aging. J Exp Psychol Gen. 2001; 130(4): 746–63. pmid:11757878
  106. 106. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. Jama. 1998; 279(4): 281–6. pmid:9450711
  107. 107. Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Bmj. 1997; 315(7109): 640–5. pmid:9310565
  108. 108. Egger M, Smith GD. Bias in location and selection of studies. Bmj. 1998; 316(7124): 61–6. pmid:9451274
  109. 109. Cortina JM. Apples and oranges (and pears, oh my!): The search for moderators in meta-analysis. Organ Res Methods. 2003; 6(4): 415–39.
  110. 110. Wiers CE,Wiers RW, Imaging the neural effects of cognitive bias modification training, NeuroImage (2016);
  111. 111. Hirsch CR, Clark DM, Mathews A. Imagery and interpretations in social phobia: Support for the combined cognitive biases hypothesis. Behav Ther. 2006; 37(3): 223–36. Imagery and interpretations in social phobia: Support for the combined cognitive biases hypothesis. pmid:16942974
  112. 112. Hofmann SG, Asnaani A, Hinton DE. Cultural aspects in social anxiety and social anxiety disorder. Depress Anxiety. 2010; 27(12): 1117–27. pmid:21132847
  113. 113. Woody SR, Miao S, Kellman-McFarlane K. Cultural differences in social anxiety: A meta-analysis of Asian and European heritage samples. Asian Am J Psychol. 2015; 6(1): 47–55.