Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Characterizing emotional Stroop interference in posttraumatic stress disorder, major depression and anxiety disorders: A systematic review and meta-analysis

  • Marilyne Joyal,

    Roles Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing – review & editing

    Affiliations Medical School, Laval University, Quebec City, QC, Canada, Centre intégré universitaire en santé et services sociaux de la Capitale-Nationale, Quebec City, QC, Canada

  • Tobias Wensing,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing – original draft

    Affiliations Medical School, Laval University, Quebec City, QC, Canada, Centre intégré universitaire en santé et services sociaux de la Capitale-Nationale, Quebec City, QC, Canada

  • Jean Levasseur-Moreau,

    Roles Investigation, Validation, Writing – review & editing

    Affiliations Medical School, Laval University, Quebec City, QC, Canada, Centre intégré universitaire en santé et services sociaux de la Capitale-Nationale, Quebec City, QC, Canada

  • Jean Leblond,

    Roles Supervision, Writing – review & editing

    Affiliation Centre intégré universitaire en santé et services sociaux de la Capitale-Nationale, Quebec City, QC, Canada

  • Alexander T. Sack,

    Roles Writing – review & editing

    Affiliation Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center, Maastricht University, Maastricht, The Netherlands

  • Shirley Fecteau

    Roles Conceptualization, Supervision, Writing – review & editing

    Shirley.Fecteau@fmed.ulaval.ca

    Affiliations Medical School, Laval University, Quebec City, QC, Canada, Centre intégré universitaire en santé et services sociaux de la Capitale-Nationale, Quebec City, QC, Canada

Abstract

Background

Posttraumatic stress disorder is a debilitating psychiatric disorder characterized by symptoms of intrusive re-experiencing of trauma, avoidance and hyper-arousal. Diagnosis and treatment of PTSD is further complicated by concurrently occurring disorders, the most frequent being major depressive disorder and anxiety disorders. Previous research highlights that attentional processing in posttraumatic stress disorder is associated with substantial interference by emotional stimuli, a phenomenon also observed in these concurrently occurring psychiatric disorders. However, the diagnosis-relevance of this interference remains elusive. Here, we investigated the emotional Stroop interference for diagnosis-related stimuli, generally negative stimuli, and generally positive stimuli in posttraumatic stress disorder, major depressive disorder and anxiety disorders.

Methods

We performed a systematic database search in PubMed (Medline), Cochrane Library and PsycINFO on emotional Stroop performance in individuals with a diagnosis of posttraumatic stress disorder, major depressive disorder or anxiety disorders separately. Mean effect sizes, standard errors and confidence intervals were estimated for each clinical group and healthy control group comparison using random effect models.

Results

As compared to healthy control group, the posttraumatic stress disorder group displayed greater interference by diagnosis-related stimuli and positive stimuli but not for generally negative stimuli. The major depressive disorder and anxiety disorders groups showed greater interference by diagnosis-related and negative stimuli, but not by positive stimuli. The age and sex had no significant impact on interference.

Conclusions

These findings highlight the importance of diagnosis-relevant information on attentional processing in all three clinical populations, posttraumatic stress disorder, major depressive disorder and anxiety disorders. Further, the impact of generally negative stimuli but not generally positive stimuli in major depressive disorder and anxiety disorders indicate impaired attentional bias for mood-congruent stimuli but not for general stimuli. Finally, it remains to be studied whether the influence of generally positive stimuli in posttraumatic stress disorder indicate that positive stimuli are perceived as PTSD related.

Introduction

Individuals diagnosed with posttraumatic stress disorder (PTSD) have been exposed to a traumatic event comprising physical or psychological harm. PTSD is a highly debilitating disorder defined by the direct or indirect exposure to a traumatic event, including symptoms of intrusive re-experience, trauma-related thoughts and feelings and hypervigilance [1]. Hence, PTSD can lead to considerable impairments in individual well-being, from everyday functioning up to suicidal ideation and suicide attempts [24].

Symptom severity in PTSD and symptoms of re-experiencing, hypervigilance, avoidance and levels of anxiety have been positively correlated with attentional bias [5,6]. Attentional bias in PTSD usually refers to persistent engagement of attentional processing mechanisms to real or experimentally evoked threatening events. Indeed, attentional processing in PTSD is associated with interference by emotional stimuli [617]. Hence, recent clinical trials implemented the existing knowledge on the mechanisms of attentional bias in PTSD into a novel treatment approach, that is, attentional bias modification (ABM) [18,19]. ABM utilizes the detection-of-object (dot) probe task to elicit attentional bias. Studies on ABM treatment reported a decrease in PTSD symptoms with effect sizes similar to those of placebo-pill pharmacotherapy outcomes. Furthermore, ABM treatment did not improve attentional bias [20], even when the ABM approach was tailored to personally relevant bias-evoking stimuli [21]. Given the lack of evidence for ABM as a treatment for PTSD, there is a need to further understand attentional bias in PTSD.

The Stroop task [22] is a well-established cognitive task to investigate attentional processing of simultaneously occurring sensory information in context of selective attention, cognitive set shifting and response inhibition (see for example MacLeod or Stein et al.) [23,24]. In the original version of the color-word Stroop task (CW-Stroop) [25], participants are presented with three different card templates: the first template contains black-on-white printed words of highly distinguishable colors (“blue”, “green”, “red”, “yellow”); the second template depicts rectangles printed in blue, green, red or yellow color (“color-naming” condition); and the last template shows the same words as the first template but printed in a color that does not represent its actual content (“incongruent” condition; e.g., the word “blue” printed in green). Performance on the first template serves as a measure of reading abilities. Then, a Stroop interference index is calculated by subtracting response times (RT) in the color-naming condition from those in the incongruent condition. This index serves as a measure of the attentional engagement and hence the attentional bias. Nowadays, computerized versions of the Stroop paradigm allow for specific modifications in experimental designs that serve individual study purposes (e.g., stimulus presentation in a trial-by-trial fashion instead of a blocked design using physical template cards).

The emotional Stroop task modifies the CW-Stroop rationale by replacing color-words with neutral and emotionally loaded stimuli (e.g., the word “violence” painted in blue compared to a neutral word in the same color) [26]. In healthy participants, the emotional Stroop task does not elicit attentional bias related to threatening stimuli when stimuli were presented in a single-item fashion (i.e., one stimulus at a time) [18]. Individuals diagnosed with PTSD, however, often display attentional bias for negative stimuli as compared to healthy controls at the emotional Stroop task (i.e., threatening or aversive) [9,13]. Hence, emotional interference appears to be related to the thematic relevance of the personally experienced event and not merely on the exposure to threat per se in both civilians [15] and military personnel [5]. In survivors of sexual violence, for instance, an attentional bias was observed for both intimacy-related trauma words (e.g., “rape”) and intimacy-related positive words (e.g., “love”) [5]. However, recent findings on emotional interference, especially in military personnel with acute PTSD, challenge the view of an attentional bias for threatening or trauma-related information. These studies report an attentional shift away from threatening information, suggesting an avoidance pattern [19,27,28]. Given these inconsistencies, a meta-analysis aimed to synthesize findings on emotional Stroop task performance in PTSD. This meta-analysis indicated that individuals with PTSD, as compared to healthy controls, displayed impairments in the emotional Stroop task when processing trauma-related and generally threatening, but not positive information [29].

Attentional bias is also important in patients with major depressive disorder (MDD) or anxiety disorders (AD). During neuropsychological tests, individuals diagnosed with MDD and AD also display abnormal attentional bias to emotionally loaded stimuli [18,30]. This is of particular importance because MDD and AD are among the most concurrent diagnoses in PTSD. Further, these concurrent psychiatric disorders greatly impact and hamper diagnosis and treatment of PTSD [3133]. For instance, co-occurrence of PTSD and MDD symptoms considerably elevates risk for suicidal ideations and suicide attempts compared to the impact of each disorder on its own [3437].

The aim of this work is thus to investigate attentional bias in individuals diagnosed with PTSD, MDD and / or AD. Hence, we systematically reviewed findings on the emotional Stroop paradigm addressing task performance in individuals with PTSD, MDD, and /or AD, as compared to healthy controls. We used meta-analytic statistical tools to assess whether interference by stimuli with emotional valence in these groups can be attributed to non-specific emotional valence (generally positive or negative) or diagnosis-relevant emotional words.

Materials and methods

Methodological quality assessment

We conducted this review following PRISMA guidelines [38]. The PRISMA checklist is provided in supplementary material (S1 Table).

Search strategy

We conducted a systematic database search in PubMed (Medline), Cochrane Library and PsycINFO on peer-reviewed articles published between 1986 and 30 April 2018, using combinations of the search strings “posttraumatic stress disorder (PTSD)” or “major depressive disorder (MDD)” or “anxiety disorder (AD)” and “modified/emotional Stroop task”. Detailed search terms are provided in supplementary material (S2 Fig). We limited our search to published peer-reviewed articles as they tend to show higher methodological quality than unpublished studies [3940]. In the field of psychiatry research, inclusion of grey literature may also increase risk of bias [41]. We applied a filter to identify relevant literature examining humans only. This database search resulted in a total of 900 abstracts for potential inclusion.

Selection criteria

In a first phase of selection, we excluded abstracts (a) not reporting original data (e.g., reviews, meta-analyses, opinion letters), (b) not assessing a group with a primary diagnosis of PTSD, MDD, or AD, (c) not implementing the modified or emotional Stroop task as a cognitive measurement, and (d) not written in English, leaving a total of 155 original studies. In order to conduct coherent statistical analyses, we further excluded 108 of the 155 studies due to the following reasons: (a) no sufficient behavioral data to calculate effect sizes (neither interference scores nor response times to emotional and neutral conditions), (b) experimental manipulation prior to baseline behavioral testing (e.g., assessment of treatment efficacy), (c) experimental designs that were incomparable to designs of the remaining studies (i.e. stimulus presentation in a multi-item instead of a single-item/trial-by-trial design, stroop task with masked presentation mode or with pictures instead of written words) and (d) overlapping studies. Based on the procedure described above, 47 studies [57,4285] were included in statistical analyses (54 datasets: PTSD: 13; MDD: 12; AD: 29) (Fig 1).

thumbnail
Fig 1. Schematic overview of the meta-analytic study selection procedure in this review.

PTSD = Posttraumatic Stress Disorder, MDD = Major Depressive Disorder.

https://doi.org/10.1371/journal.pone.0214998.g001

Data extraction

Data extraction was performed by two independent reviewers (TW, JLM). Differences regarding outcome variables were resolved by consensus.

Qualitative data.

We collected demographics and experimental design characteristics from all studies including, if available, (a) study type, (b) clinical diagnostics, (c) number of participants, (d) mean age, (e) female/male ratio, (f) medication, (g) comorbidity status, (h) stimulus valence, and (i) response modality. We further assessed potential risk of bias in included studies using the Cochrane risk of bias tool [86]. The Cochrane risk of bias tool allows for inference of studies’ quality via an observer-dependent rating system that scores potential methodological shortcomings as “low”, “high” or “unclear” (S2 Table).

Quantitative data.

We extracted mean response times (RT), mean interference scores, as well as respective dispersion indices (Standard Deviation [SD] and Standard Error [SE]) from all studies.

Outcome measures of the emotional Stroop task

Within the emotional Stroop task, participants are asked to identify the color of either neutral or emotionally loaded stimuli while ignoring its actual content. Interference by a specific type of stimulus is calculated by RT difference scores (i.e., subtracting mean RT of the neutral stimuli from that of emotional stimuli). In this regard, positive values mirror longer RT for emotionally loaded stimuli and negative values indicate shorter RT for emotionally loaded stimuli (see for example El Khoury-Malhame et al.) [13]. Interference scores were our primary outcome measure, calculated on RT measurements. Across studies, RT was commonly defined as the latency from stimulus onset to the participant’s response. RTs of the Stroop tasks were reported in milliseconds (ms) and accompanied by statistical dispersion indices (SD, SE).

Stimulus type and response modalities

Stimuli consisted of words and were presented in a trial-by-trial fashion (i.e., one stimulus at a time) in a randomized or quasi-randomized order on a computer monitor. Participants were instructed to respond verbally (e.g., recorded by microphone on a computer) or by pressing a button that was specific to the color of the displayed word. Twenty-nine group comparisons measured RTs of verbal responses (PTSD: n = 8; MDD: n = 6; AD: n = 15) and 25 group comparisons measured RTs by button press (PTSD: n = 5; MDD: n = 6; AD: n = 14).

Arousal ratings and stimulus valence

This section describes how word stimuli were categorized in the studies included in this review. Ratings of stimuli valence and/or arousal were based on participants’ self-reports prior to or after testing (4 studies), ratings in pilot experiments with independent cohorts of participants (6 studies), evaluations by clinicians or otherwise trained professionals (8 studies) or were taken from previous research and validated stimuli databases (27 studies). In four studies, assessment of arousal or stimulus rating was not further specified. Based on stimulus ratings, and for the sake of comparability across studies and psychiatric diagnostic groups, word stimuli included in the present analysis were categorized as (a) diagnosis-relevant (i.e. PTSD-relevant, e.g. war, abuse; MDD-relevant, e.g. sadness, discouraged; AD-relevant, e.g. panic, embarrassed) (b) generally negative (e.g. fraud, divorce), or (c) generally positive (e.g. pleasant, comedy). Diagnosis-relevant cues had the distinction of being related to the trauma inducing-event or to the symptomatology of clinical groups. In all studies, stimuli with neutral valence were utilized for calculation of interference scores.

Statistical data analysis

Statistical analyses were separately performed for each of the three clinical groups (PTSD-groups, MDD-groups, AD-groups). Effect sizes (bias-corrected standardized mean-differences; Hedge’s g) were calculated with the R Metafor package (R 3.5.0; R Development Core Team) for each study contrasting clinical and healthy control groups. Single-study effect sizes were analyzed using meta-analytic random-effect models and the Hedge’s g estimator to derive a meta-analytic overall effect size (g) and tests for heterogeneity (Q and I2) for each group comparison of interest. We also explored whether some variables had an impact on interference using moderator analyses. There were sufficient available data to test mean age of patients, percentage of women in clinical groups, and response modality at the emotional Stroop task. Separate models were fitted to determine the main effects of each variable. In case of large heterogeneity across studies, outliers were identified based on visual inspection of the meta-analytic forest plots. To further test whether any one study was overly influential on effect size estimates, we conducted leave-one-out cross-validation for these group comparisons of interest.

Results

Study types

Case-control studies was the most common study type. For the PTSD groups, articles described case-control studies (11 studies), controlled before-after study (1 study), and randomized controlled trial (1 study). All articles with MDD groups described case-control studies (12 studies). For AD groups, study types were case-control studies (21 studies), controlled before-after study (1 study), randomized controlled trial (1 study), and crossover study (1 study). One article with AD patients also included three experiments, with both case control and before-after designs. In all cases, we extracted data from the baseline condition to avoid any experimental manipulation prior to behavioral testing.

Study groups

Clinical groups.

Clinical groups comprised individuals with a primary diagnosis of either (a) PTSD, (b) MDD or (c) AD according to administered clinical interviews and scales (Tables 13). Diagnoses of AD included generalized anxiety disorder—not otherwise specified (GAD NOS), panic disorder (PD), social phobia (SoP), specific phobia (SpP) and multiple diagnoses of AD within one study group. To enable statistical group comparisons of interest and provide an overall view of attentional bias in AD in general, all differential AD diagnoses were summarized in a single AD group. In each individual study, clinical assessments were based on the DSM [1] or the International Classification of Disorders (ICD) [87] using the most recent version at the respective time-point of assessment. Across studies, 288 participants were considered in the PTSD-groups, 243 participants in the MDD-groups and 613 participants in the AD-groups.

thumbnail
Table 1. Characteristics of included studies with posttraumatic stress disorder (PTSD) groups.

https://doi.org/10.1371/journal.pone.0214998.t001

thumbnail
Table 2. Characteristics of included studies with major depressive disorder (MDD) groups.

https://doi.org/10.1371/journal.pone.0214998.t002

thumbnail
Table 3. Characteristics of included studies with anxiety disorder (AD) groups.

https://doi.org/10.1371/journal.pone.0214998.t003

Control groups.

All clinical groups were compared to healthy control groups, without psychiatric or neurological disorder. The total number of participants in the healthy control groups was 1068 (260 participants in the PTSD studies; 266 participants in the MDD studies; 542 participants in the AD studies).

Comorbidity and medication status

Comorbidity.

Nineteen out of the 47 studies reported secondary diagnoses of other psychiatric and neurological disorders besides the primary diagnosis according to criteria of the DSM or ICD (Tables 13). For PTSD groups, 7 studies reported comorbid occurring diagnoses including AD (6 studies), MDD (6 studies), obsessive compulsive disorder (2 studies), personality disorder (2 studies), comorbid AD and MDD (1 study), eating disorder (1 study), somatoform disorder (1 study) and mild traumatic brain injury (1 study). For MDD groups, 2 studies reported comorbid diagnoses: AD (2 studies), dysthymic disorder (1 study), obsessive compulsive disorder (1 study), eating disorder (1 study), somatoform disorder (1 study), PTSD (1 study) and substance use disorder (1 study). For AD groups, 10 studies reported co-occurring disorders, including other AD (8 studies), MDD (6 studies), dysthymic disorder (2 studies), somatoform disorder (2 studies), eating disorder (1 study), personality disorder (1 study) and PTSD (1 study). Data on occurrence of comorbidity were insufficient to conduct moderator analyses to assess whether it has an impact on interference.

Medication status.

In 22 out of the 47 selected studies, participants were reported to be undergoing pharmacotherapy (Tables 13). Individuals in the PTSD-groups received antidepressant (4 studies), anxiolytic (3 studies), first- or second-generation antipsychotics (2 study) and anticonvulsant (1 study) medication. One study reported that individuals in the PTSD-group were on current medication, but no specification on the type of drug was provided. In the MDD-groups, individuals received antidepressants (10 studies), anxiolytics (5 studies), first- or second-generation antipsychotics (3 studies). Individuals in the AD-groups were given antidepressants (7 studies), anxiolytics (5 studies), first- or second-generation antipsychotics (2 studies) or other non-specified medication (1 study). Any discrepancies in the number of clinical groups and studies reporting comorbid diagnoses or psychopharmacological treatment are related to individual studies assessing more than one clinical group (see for example Lim & Kim) [57], and clinical groups with individuals receiving customized (poly-) pharmacotherapy.

Emotional Stroop interference

PTSD-groups.

As illustrated in Fig 2, the PTSD group, as compared to healthy control group, showed greater interference for PTSD-relevant and generally positive words (i.e. slowing of responses for PTSD-relevant and generally positive words as compared to neutral words). Effect sizes were strong on interference by PTSD-relevant words (g = .65, p < .001; Table 4) and moderate by generally positive words (g = .38, p < .01). Interference for generally negative words was not significant (g = .33, p = .139). Heterogeneity across studies was large for generally negative words (Q’s p value = .040, I2 = 71.53%). Cross-validation indicated that the interference remained non-significant for generally negative words when the meta-analysis was run multiple times, each time leaving out a different study (all p-values ≥ .068). Moderator analyses indicated that mean age of patients and female proportion did not influence attentional bias (all p-values ≥ .215, S3 Table). Response modality had a significant influence on effect sizes for interference by PTSD-relevant words only (p < .05). Effect sizes were larger for motor than verbal responses.

thumbnail
Fig 2. Effect sizes of emotional vs. neutral stimuli during emotional Stroop task performance for the PTSD-groups.

(A) PTSD-specific words, (B) generally negative words, (C) generally positive words; RE = Random effects, g = Standardized mean-difference, CI95% = 95% confidence interval.

https://doi.org/10.1371/journal.pone.0214998.g002

thumbnail
Table 4. Summary statistics of overall group comparisons for words with diagnosis-relevant, generally negative and generally positive stimulus valence in the emotional Stroop task.

https://doi.org/10.1371/journal.pone.0214998.t004

MDD-groups.

The group composed of individuals with a primary diagnosis of MDD, as compared to healthy control group, displayed an attentional bias with moderate effect sizes for MDD-relevant words (g = .36, p < .01; Table 4; Fig 3) and generally negative words (g = .58, p < .01), but not for generally positive words (g = .11, p = .293). The Q-test of study heterogeneity was significant for generally negative words (Q’s p value = .047, I2 = 63.19%). The attentional bias for this stimulus valence remained significant when each study was removed one at a time in cross-validation analysis (all p-values ≤ 0.027). None of the moderator variables had a significant effect on the attentional bias in MDD-groups (all p-values ≥ 0.239, S3 Table).

thumbnail
Fig 3. Effect sizes of emotional vs. neutral stimuli during emotional Stroop task performance for the MDD-groups.

(A) MDD-specific words, (B) generally negative words, (C) generally positive words; RE = Random effects, g = Standardized mean-difference, CI95% = 95% confidence interval.

https://doi.org/10.1371/journal.pone.0214998.g003

AD-groups.

The group composed of individuals with a primary diagnosis of AD, as compared to healthy control group, showed an attentional bias with moderate effect sizes for AD-relevant words (g = .30, p < .001; Table 4; Fig 4) and generally negative words (g = .32, p < .01), but not for generally positive words (g = .05, p = .570). Between-study heterogeneity was large for generally negative words (Q’s p value = .022, I2 = 38.52). The attentional bias for generally negative words remained significant when we performed leave-one-out cross-validation (all p-values ≤ 0.009). None of the moderator variables had a significant effect on the attentional bias in AD-groups (all p-values ≥ 0.062, S3 Table).

thumbnail
Fig 4. Effect sizes of emotional vs. neutral stimuli during emotional Stroop task performance for the AD-groups.

(A) AD-specific words, (B) generally negative words, (C) generally positive words; Generalized anxiety disorder NOS = Generalized anxiety disorder—not otherwise specified, RE = Random effects, g = Standardized mean-difference, CI95% = 95% confidence interval.

https://doi.org/10.1371/journal.pone.0214998.g004

Risk of bias assessment

Twenty-five out for the 47 studies had a low risk of bias regarding incomplete outcome data, whereas the remaining 22 studies had an unclear risk of bias, due to insufficient reporting of exclusions or attrition. Thirty-three studies had a low risk of bias concerning the allocation concealment, one had a high risk of bias, and the remaining 13 studies had an unclear risk of bias. Regarding the selective reporting of outcomes, 40 studies had a low risk of bias, whereas 7 studies had an unclear risk of bias. All studies had a low risk of bias regarding blinding of participants and blinding of outcome assessments.

Discussion

Attentional bias to emotionally loaded stimuli has been closely linked to PTSD symptom severity and is considered as one of the core cognitive deficits of PTSD. In this meta-analysis, we examined attentional bias in PTSD and in its most frequently concurrently occurring disorders, that is, MDD and AD. We assessed whether studies using the emotional Stroop task showed consensus findings regarding an attentional bias elicited by emotional stimuli in individuals diagnosed with PTSD, MDD and AD as compared to healthy individuals. The results indicate that PTSD, MDD and AD groups showed greater interference by diagnosis-relevant stimuli (i.e. slowing of responses for diagnosis-relevant stimuli in comparison to neutral stimuli) as compared to healthy control groups. Further, the PTSD-groups displayed greater interference by positive stimuli than the healthy groups, but this was not observed in the MDD- and AD-groups. Finally, the MDD- and AD-groups showed greater interference by generally negative stimuli as compared to the healthy groups, but this effect was not observed in the PTSD-groups.

Results of the current meta-analysis show that the PTSD-groups displayed greater interference by PTSD-relevant and positive stimuli, but not by generally negative stimuli, as compared to healthy controls. They support that patients are sensitive to stimuli related to their concern (see for example Bar Haim et al. or Williams et al.) [18,26]. This is of importance as such attentional bias to self-relevant stimuli may play a role in the maintenance of the disorders [8890]. They are also similar to some extent to results from the meta-analysis of Cisler et al. [29] which reported greater interference for PTSD-relevant and threatening stimuli in the PTSD group as compared to the control group. Together, these findings support a hypoactive attentional control mechanism, rather than a hyperactive threat detection mechanism. The results also partially support the idea that a general-purpose defense mechanism can be automatically activated by given stimuli (e.g., PTSD-specific) which impacts seemingly irrelevant stimuli (e.g., positive stimuli) [91]. However, we cannot rule out that positive valence in this meta-analysis was not in some instances processed as PTSD-relevant or threatening (e.g., intimacy, love, sex, heal, brave, honor) by a given patient with PTSD would process this as diagnostic-relevant stimuli (e.g., rape or war). Future work should tailor attention bias tasks to individually relevant stimuli to elucidate this. Such interference by positive stimuli in PTSD was not observed in a previous meta-analysis [29]. The lack of significant group differences for interference by generally negative stimuli may suggest preserved attentional processing of negative emotional content unrelated to the traumatic event in PTSD. Similarly, Cisler et al. [29] found no difference between PTSD and healthy control groups regarding interference for generally negative stimuli. Interestingly, Clausen et al. [92] tested for potential correlations between PTSD symptoms and avoid-approach biases of emotional stimuli (happy, disgust, anger). They found a significant correlation indicating that greater symptom severity was associated with greater bias by happy faces. Together these results suggest that attentional processing of positively valenced stimuli might be more impacted than generally negative stimuli in PTSD. However, the present meta-analysis includes a small sample size with high heterogeneity for the generally negative stimuli which may also account for the lack of differences between the PTSD and healthy groups. As such, cross-validation analysis indicated a p-value of 0.068 when one study was removed [50].

In regards to MDD-groups and AD-groups, the findings show that attentional processing is impaired for diagnosis-relevant and negative stimuli, as interference was greater when compared to healthy controls. There were no significant group differences for positive stimuli. These data indicate that general attentional processing is to some extent intact in MDD and AD that fits with traditional models predicting that depression is associated with an attentional bias for mood-congruent stimuli [93,94] or self-relevant stimuli than general stimuli [26,88,90,95]. In regards to AD, the results are similar to those of a previous meta-analysis [18], which reported that patients with AD had greater interference by diagnostic-relevant and negative stimuli, but not by positive stimuli, at the emotional Stroop task. In regards to depression, a previous meta-analysis reported that patients clinically depressed (including dysthymia and minor depression) displayed greater interference by negative, positive, neutral stimuli at the emotional Stroop task, and even greater interference at the Classic Stroop task, when compared to healthy controls [30]. Effect sizes were greater for negative than positive stimuli and greater for positive than neutral stimuli. They also found greater interference in blocked than randomized design. Such impact of the design on interference was also observed in AD [18,26]. It is thus possible that interferences are more content-specific when using a randomized design than a blocked design (minimizing potential mood induction and not eliciting a generic system).

The impact of age and sex on interference by emotional stimuli

We found that age and sex had no significant role in our moderator analyses for the PTSD, MDD and AD groups. Epp et al. [30] also reported no consistent patterns of moderating effects for age and sex in this meta-analysis in patients with depression. Age only had an impact on interference in healthy controls for incongruent versus control stimuli and sex only had an impact in depressed patients for neutral stimuli. In the current meta-analysis, response modality (verbal vs. button-press) had a significant impact for PTSD-relevant words when comparing the PTSD-groups with the healthy controls. Attentional bias was larger with motor than verbal responses. This difference may result from the modality pairing effect, which corresponds to the processing speed advantage for standard modality pairing (e.g. visual stimuli with motor response) as compared to non-standard modality pairing (e.g. visual stimuli with verbal response) [96]. Additional processes in the non-standard modality pairing condition may have diluted the interference effect. However, the lack of influence of the response modality for other stimulus valences in PTSD-groups as well as in MDD- and AD-groups advise caution about this finding.

Lack of data prevented us to use co-morbidity and pharmacological treatments as moderators. We cannot rule out that these factors had no impact on observed interference by emotional stimuli. Among studies included in the meta-analysis, some reported individuals’ current status of psychopharmacological treatment and diagnoses of comorbid disorders, but the majority did not. We cannot rule out that, simply because these covariates have not been reported, individuals were free of any exposure to psychotropic medications and other psychiatric or neurologic disorders that might have affected emotional Stroop task performance in PTSD. In studies comprising individuals with a primary diagnosis of MDD or AD, medication status was reported more frequently. However, groups were administered mono- or poly-pharmacotherapy, or received no medication at all, exacerbating the disentanglement of potential effects medical drug exposure might have had on behavioral outcomes in these cohorts. The same applies to the status of comorbidity in all three clinical groups of interest. When comorbidity was reported systematically, actual diagnoses were highly heterogeneous ranging from DSM axis I- and II-disorders [1] to neurological disorders. Moreover, our design was unsuited to investigate the effects of attentional bias to emotionally valent stimuli between PTSD and other disorders in explicitly comorbid cohorts (e.g., the impact of trauma-related or depression-related words in a group of individuals with a primary diagnosis of PTSD and comorbid MDD). This might come at the cost of potential interplaying effects on attentional bias when individuals are faced with more than a primary diagnosis of PTSD. However, assessing attentional bias in the emotional Stroop task for psychiatric disorders of interest separately might facilitate to disentangle effects that could, in turn, be responsible for attentional deficits in PTSD with frequent concurrent diagnoses. We also refrained from including a group of trauma-exposed, but PTSD-free individuals in our analysis, since the definition of such a control group was too heterogeneous across selected studies. Characterization of a trauma-exposed group might vary in subclinical scores on psychopathological questionnaires, PTSD in remission, as well as the mere presence of traumatic events during lifetime. It is hard to argue in favor of a trauma-exposed group if differentiation from a PTSD-diagnosed group (in terms of clinical features) in one direction and healthy controls (who might have experienced a traumatic event at least once in their life) in the other, if this border is rather transient than sharply defined. Lack of data prevented us to assess whether chronicity (e.g., time since trauma or diagnosis) was linked to greater interference, but previous work reported that greater severity of depression was linked to greater interference [30].

Methodological considerations

This study has limitations to be considered. One limitation is that 32 studies included multiple effect sizes (either by including multiple conditions of the Stroop, multiple groups, or both). Correlations between effect sizes are likely to be stronger within than between studies, which can lead to partially redundant analyses and incorrect estimation of effects. Thus, our approach does not allow direct comparisons of effect sizes across task conditions. Future meta-analyses including multiple conditions should consider using a multivariate model [97]. Also, our number of group comparisons was considerably smaller than that of other meta-analyses assessing attentional bias in psychiatric disorders. This can be accounted for by our inclusion criteria (e.g., stroop task with words only supraliminal, unmasked stimuli), hence reducing variance in experimental designs across studies. On the one hand, this might account for the lack of attentional bias to negative words in PTSD (e.g., a smaller sample size); on the other hand, it might help to further understand the core mechanisms underlying attentional bias in PTSD and related psychiatric disorders. In addition, studies with smaller sample sizes might have introduced positive bias in some group comparisons, as indicated in the funnel plots (S1 Fig). For instance, it might be the case for generally negative words in PTSD- and MDD-groups, where we find studies with large effect sizes outside the funnel. By visual inspection of the corresponding forest plots (Figs 23), we identified two studies, that may have contributed to the heterogeneity observed in these group comparisons [43,56]. There was, however, no methodological argument to exclude these studies from our analyses. Moreover, these studies did not lead to increased heterogeneity across group comparisons on other emotionally loaded stimuli (i.e., words with generally positive valence). Cross-validation analyses also indicated that the results remained unchanged when we removed these studies. The considerable level of heterogeneity in effect sizes of studies comparing PTSD-groups and healthy controls on words with generally negative valence could partially explain the lack of statistical significance of this group comparison. Finally, some studies reported that the Stroop interference is reliable (see for example Ebersole et al.) [98], but other studies questioned it and suggest that individual differences play a significant role in such reliability [99,100]. The importance of stimuli choice is nicely discussed in the context of substance-related attentional bias by Field & Christiansen [101]. For instance, a given patient with alcohol use disorders might not respond the same to stimuli of heavy spirits than those of rosé wine. In sum, future work may consider using more than a single attentional bias task, which captures only a subset of processes [100], and add physiological measures such as an eye tracker to gather a more complete picture of bias to emotionally loaded stimuli that are relevant to symptoms of PTSD, MDD and AD.

Conclusions

In sum, this review provides valuable synthesized results regarding an attentional bias for emotional stimuli in PTSD and its most frequently concurrent diagnoses of MDD and AD specifically elicited by the emotional Stroop task. However, it remains to be determined if the results presented here are specific to the emotional Stroop task or reflect a generally applicable cognitive deficit in PTSD and concurrent psychiatric disorders. We believe that additional studies are necessary to address the impact of concurrently occurring psychiatric disorders on the development and persistence of attentional bias in PTSD. Accumulated behavioral data might provide a clearer picture on underlying cognitive mechanisms that directly link to future treatment approaches (such as the ABM).

Supporting information

S1 Fig. Funnel plots for each diagnostic category, separately for words with diagnosis-relevant, generally negative and generally positive stimulus valence in the emotional Stroop task.

https://doi.org/10.1371/journal.pone.0214998.s001

(TIF)

S2 Fig. Detailed search terms entered in databases for each clinical group.

https://doi.org/10.1371/journal.pone.0214998.s002

(TIF)

S2 Table. Assessment of risk of bias for each study included in the systematic review based on the Cochrane risk of bias tool.

https://doi.org/10.1371/journal.pone.0214998.s004

(DOCX)

S3 Table. Results of moderator analyses for each clinical group with mean age of patients, percentage of female in clinical groups and response modality as moderator variables.

https://doi.org/10.1371/journal.pone.0214998.s005

(DOCX)

Acknowledgments

The authors would like to thank V. Ashley, R. Gupta and C. E. Wittekind for providing original data sets to be included in the analyses presented here. We also thank Valérie Chénard for her technical help.

References

  1. 1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5 ed. Washington, DC: Author; 2013.
  2. 2. Sareen J, Houlahan T, Cox BJ, Asmundson GJ. Anxiety disorders associated with suicidal ideation and suicide attempts in the National Comorbidity Survey. J Nerv Ment Dis. 2005;193(7):450–4. pmid:15985839
  3. 3. Pagotto LF, Mendlowicz MV, Coutinho ES, Figueira I, Luz MP, Araujo AX, et al. The impact of posttraumatic symptoms and comorbid mental disorders on the health-related quality of life in treatment-seeking PTSD patients. Compr Psychiatry. 2015;58:68–73. pmid:25656798
  4. 4. Wisco BE, Marx BP, Wolf EJ, Miller MW, Southwick SM, Pietrzak RH. Posttraumatic stress disorder in the US veteran population: results from the National Health and Resilience in Veterans Study. J Clin Psychiatry. 2014;75(12):1338–46. pmid:25551234
  5. 5. Ashley V, Honzel N, Larsen J, Justus T, Swick D. Attentional bias for trauma-related words: exaggerated emotional Stroop effect in Afghanistan and Iraq war veterans with PTSD. BMC Psychiatry. 2013;13:86. pmid:23496805
  6. 6. El Khoury-Malhame M, Lanteaume L, Beetz EM, Roques J, Reynaud E, Samuelian JC, et al. Attentional bias in post-traumatic stress disorder diminishes after symptom amelioration. Behav Res Ther. 2011;49(11):796–801. pmid:21924404
  7. 7. Paunovic N, Lundh LG, Öst LG. Attentional and memory bias for emotional information in crime victims with acute posttraumatic stress disorder (PTSD). Anxiety Disorders. 2002;16:675–92.
  8. 8. Elsesser K, Sartory G, Tackenberg A. Attention, heart rate, and startle response during exposure to trauma-relevant pictures: a comparison of recent trauma victims and patients with posttraumatic stress disorder. J Abnorm Psychol. 2004;113(2):289–301. pmid:15122949
  9. 9. Vythilingam M, Blair KS, McCaffrey D, Scaramozza M, Jones M, Nakic M, et al. Biased emotional attention in post-traumatic stress disorder: a help as well as a hindrance? Psychol Med. 2007;37(10):1445–55. pmid:17559703
  10. 10. Hayes JP, Labar KS, Petty CM, McCarthy G, Morey RA. Alterations in the neural circuitry for emotion and attention associated with posttraumatic stress symptomatology. Psychiatry Res. 2009;172(1):7–15. pmid:19237269
  11. 11. Hayes JP, Vanelzakker MB, Shin LM. Emotion and cognition interactions in PTSD: a review of neurocognitive and neuroimaging studies. Front Integr Neurosci. 2012;6:89. pmid:23087624
  12. 12. Mueller-Pfeiffer C, Martin-Soelch C, Blair JR, Carnier A, Kaiser N, Rufer M, et al. Impact of emotion on cognition in trauma survivors: what is the role of posttraumatic stress disorder? J Affect Disord. 2010;126(1–2):287–92. pmid:20417971
  13. 13. El Khoury-Malhame M, Reynaud E, Soriano A, Michael K, Salgado-Pineda P, Zendjidjian X, et al. Amygdala activity correlates with attentional bias in PTSD. Neuropsychologia. 2011;49(7):1969–73. pmid:21440563
  14. 14. Felmingham KL, Rennie C, Manor B, Bryant RA. Eye tracking and physiological reactivity to threatening stimuli in posttraumatic stress disorder. J Anxiety Disord. 2011;25(5):668–73. pmid:21477983
  15. 15. Fleurkens P, Rinck M, van Minnen A. Specificity and generalization of attentional bias in sexual trauma victims suffering from posttraumatic stress disorder. J Anxiety Disord. 2011;25(6):783–7. pmid:21531116
  16. 16. Fani N, Jovanovic T, Ely TD, Bradley B, Gutman D, Tone EB, et al. Neural correlates of attention bias to threat in post-traumatic stress disorder. Biol Psychol. 2012;90(2):134–42. pmid:22414937
  17. 17. Fani N, Tone EB, Phifer J, Norrholm SD, Bradley B, Ressler KJ, et al. Attention bias toward threat is associated with exaggerated fear expression and impaired extinction in PTSD. Psychol Med. 2012;42(3):533–43. pmid:21854700
  18. 18. Bar-Haim Y, Lamy D, Pergamin L, Bakermans-Kranenburg MJ, van IMH. Threat-related attentional bias in anxious and nonanxious individuals: a meta-analytic study. Psychol Bull. 2007;133(1): 1–24. pmid:17201568
  19. 19. Bar-Haim Y, Holoshitz Y, Eldar S, Frenkel TI, Muller D, Charney DS, et al. Life-threatening danger and suppression of attention bias to threat. Am J Psychiatry. 2010;167(6): 694–8. pmid:20395400
  20. 20. Schoorl M, Putman P, Van Der Does W. Attentional bias modification in posttraumatic stress disorder: a randomized controlled trial. Psychother Psychosom. 2013;82(2): 99–105. pmid:23295710
  21. 21. Schoorl M, Putman P, Mooren TM, Van Der Werff S, Van Der Does W. Attentional bias modification in Dutch veterans with posttraumatic stress disorder—a case series with a personalized treatment version. J Trauma Stress. 2014;27(2): 240–3. pmid:24700603
  22. 22. Stroop JR. Studies of Interference in Serial Verbal Reactions. Journal of Experimental Psychology. 1935;18(643–662).
  23. 23. MacLeod CM. Half a century of research on the Stroop effect: an integrative review. Psychol Bull. 1991;109(2):163–203. pmid:2034749
  24. 24. Stein MB, Kennedy CM, Twamley EW. Neuropsychological Function in Female Victims of Intimate Partner Violence with and without Posttraumatic Stress Disorder. Biol Psychiatry. 2002;52:1079–88. pmid:12460691
  25. 25. Golden CJ, Freshwater SM. Stroop Color and Word Test: A Manual for Clinical and Experimantal Uses. Chicago, Illinois: Skoelting; 1978.
  26. 26. Williams JM, Mathews A, MacLeod C. The emotional Stroop task and psychopathology. Psychol Bull. 1996;120(1):3–24. pmid:8711015
  27. 27. Wald I, Lubin G, Holoshitz Y, Muller D, Fruchter E, Pine DS, et al. Battlefield-like stress following simulated combat and suppression of attention bias to threat. Psychol Med. 2011;41(4):699–707. pmid:21108868
  28. 28. Sipos ML, Bar-Haim Y, Abend R, Adler AB, Bliese PD. Postdeployment threat-related attention bias interacts with combat exposure to account for PTSD and anxiety symptoms in soldiers. Depress Anxiety. 2014;31(2):124–9. pmid:23959788
  29. 29. Cisler JM, Wolitzky-Taylor KB, Adams TG Jr., Babson KA, Badour CL, Willems JL. The emotional Stroop task and posttraumatic stress disorder: a meta-analysis. Clin Psychol Rev. 2011;31(5):817–28. pmid:21545780
  30. 30. Epp AM, Dobson KS, Dozois DJ, Frewen PA. A systematic meta-analysis of the Stroop task in depression. Clin Psychol Rev. 2012;32(4):316–28. pmid:22459792
  31. 31. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593–602. pmid:15939837
  32. 32. Stander VA, Thomsen CJ, Highfill-McRoy RM. Etiology of depression comorbidity in combat-related PTSD: a review of the literature. Clin Psychol Rev. 2014;34(2):87–98. pmid:24486520
  33. 33. Gallagher MW, Brown TA. Bayesian Analysis of Current and Lifetime Comorbidity Rates of Mood and Anxiety Disorders In Individuals with Posttraumatic Stress Disorder. J Psychopathol Behav Assess. 2015;37(1):60–6. pmid:26166944
  34. 34. Ramsawh HJ, Fullerton CS, Mash HB, Ng TH, Kessler RC, Stein MB, et al. Risk for suicidal behaviors associated with PTSD, depression, and their comorbidity in the U.S. Army. J Affect Disord. 2014;161:116–22. pmid:24751318
  35. 35. Debeer BB, Kimbrel NA, Meyer EC, Gulliver SB, Morissette SB. Combined PTSD and depressive symptoms interact with post-deployment social support to predict suicidal ideation in Operation Enduring Freedom and Operation Iraqi Freedom veterans. Psychiatry Res. 2014;216(3):357–62. pmid:24612971
  36. 36. Stevens D, Wilcox HC, MacKinnon DF, Mondimore FM, Schweizer B, Jancic D, et al. Posttraumatic stress disorder increases risk for suicide attempt in adults with recurrent major depression. Depress Anxiety. 2013;30(10):940–6. pmid:23893768
  37. 37. Pukay-Martin ND, Pontoski KE, Maxwell MA, Calhoun PS, Dutton CE, Clancy CP, et al. The influence of depressive symptoms on suicidal ideation among U.S. Vietnam-era and Afghanistan/Iraq-era veterans with posttraumatic stress disorder. J Trauma Stress. 2012;25(5):578–82. pmid:23047458
  38. 38. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. pmid:19621072
  39. 39. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technol assess. 2003;7(1):1–4. pmid:12583822
  40. 40. Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synth Methods. 2014;5(3):221–34. pmid:26052848
  41. 41. Martin JL, Pérez V, Sacristán M, Alvarez E. Is grey literature essential for a better control of publication bias in psychiatry? An example from three meta-analyses of schizophrenia. Eur Psychiatry. 2005;20(8):550–3. pmid:15994063
  42. 42. Buckley TC, Galovski T, Blanchard EB, Hickling EJ. Is the Emotional Stroop Paradigm Sensitive to Malingering? A Between-Groups Study With Professional Actors and Actual Trauma Survivors. J Trauma Stress. 2003;16(1):59–66. pmid:12602653
  43. 43. Cassiday KL, McNally RJ, Zeitlin SB. Cognitive Processing of Trauma Cues in Rape Victims with Post-Traumatic Stress Disorder. Cognit Ther Res. 1992;16(3):283–95.
  44. 44. Harvey AG, Bryant RA, Rapee RM. Preconscious Processing of Threat in Posttraumatic Stress Disorder. Cognit Ther Res. 1996;20:613–23.
  45. 45. Herzog JI, Niedtfeld I, Rausch S, Thome J, Mueller-Engelmann M, Steil R, et al. Increased recruitment of cognitive control in the presence of traumatic stimuli in complex PTSD. Eur Arch Psychiatry Clin Neurosci. 2017.
  46. 46. Khanna MM, Badura-Brack AS, McDermott TJ, Embury CM, Wiesman AI, Shepherd A, et al. Veterans with post-traumatic stress disorder exhibit altered emotional processing and attentional control during an emotional Stroop task. Psychol Med. 2017;47:2017–27. pmid:28478767
  47. 47. Martinson AA, Sigmon ST, Craner J, Rothstein E, McGillicuddy M. Processing of intimacy-related stimuli in survivors of sexual trauma: the role of PTSD. J Interpers Violence. 2013;28(9):1886–908. pmid:23266991
  48. 48. McNally RJ, Clancy SA, Schacter DL, Pitman RK. Cognitive processing of trauma cues in adults reporting repressed, recovered, or continuous memories of childhood sexual abuse. J Abnorm Psychol. 2000;109(3):355–9. pmid:11016105
  49. 49. Metzger LJ, Orr SP, Lasko NB, McNally RJ, Pitman RK. Seeking the source of emotional Stroop interference effects in PTSD: a study of P3s to traumatic words. Integr Physiol Behav Sci. 1997;32(1):43–51. pmid:9105913
  50. 50. Thomaes K, Dorrepaal E, Draijer N, de Ruiter MB, Elzinga BM, van Balkom AJ, et al. Treatment effects on insular and anterior cingulate cortex activation during classic and emotional Stroop interference in child abuse-related complex post-traumatic stress disorder. Psychol Med. 2012;42(11):2337–49. pmid:22436595
  51. 51. Wittekind CE, Jelinek L, Kellner M, Moritz S, Muhtz C. Intergenerational transmission of biased information processing in posttraumatic stress disorder (PTSD) following displacement after World War II. J Anxiety Disord. 2010;24(8):953–7. pmid:20655170
  52. 52. Broomfield NM, Davies R, MacMahon K, Ali F, Cross SM. Further evidence of attention bias for negative information in late life depression. Int J Geriatr Psychiatry. 2007;22(3):175–80. pmid:17096465
  53. 53. Constant EL, Adam S, Seron X, Bruyer R, Seghers A, Daumerie C. Hypothyroidism and major depression: A common executive dysfunction? J Clin Exp Neuropsychol. 2006;28:790–807. pmid:16723325
  54. 54. Dozois DJ, Dobson KS. Information processing and cognitive organization in unipolar depression: specificity and comorbidity issues. J Abnorm Psychol. 2001;110(2):236–46. pmid:11358018
  55. 55. Fritzsche A, Dahme B, Gotlib IH, Joormann J, Magnussen H, Watz H, et al. Specificity of cognitive biases in patients with current depression and remitted depression and in patients with asthma. Psychol Med. 2010;40(5):815–26. pmid:19719897
  56. 56. Gupta R, Kar BR. Attention and memory biases as stable abnormalities among currently depressed and currently remitted individuals with Unipolar Depression. Front Psychiatry. 2012;3(99):1–7.
  57. 57. Lim SL, Kim JH. Cognitive processing of emotional information in depression, panic, and somatoform disorder. J Abnorm Psychol. 2005;114(1):50–61. pmid:15709812
  58. 58. Markela-Lerenc J, Kaiser S, Golz T, Fiedler P, Mundt C, Weisbrod M. Attentional bias in depressive patients and the moderating effect of concurrent anxiety. Psychopathology. 2011;44(3):193–200. pmid:21412033
  59. 59. Matsubara T, Matsuo K, Nakashima M, Nakano M, Harada K, Watanuki T, et al. Prefrontal activation in response to emotional words in patients with bipolar disorder and major depressive disorder. Neuroimage. 2014;85 Pt 1:489–97.
  60. 60. McNeely HE, Lau MA, Christensen BK, Alain C. Neurophysiological evidence of cognitive inhibition anomalies in persons with major depressive disorder. Clin Neurophysiol. 2008;119(7):1578–89. pmid:18482863
  61. 61. Mitterschiffthaler MT, Williams SC, Walsh ND, Cleare AJ, Donaldson C, Scott J, et al. Neural basis of the emotional Stroop interference effect in major depression. Psychol Med. 2008;38(2):247–56. pmid:17825123
  62. 62. Mogg K, Bradley BP, Williams R, Mathews A. Subliminal processing of emotional information in anxiety and depression. J Abnorm Psychol. 1993;102(2):304–11. pmid:8315143
  63. 63. Schlosser N, Mensebach C, Rullkotter N, Schaffrath C, Driessen M, Beblo T, et al. Selective attention in depression: influence of emotionality and personal relevance. J Nerv Ment Dis. 2011;199(9):696–702. pmid:21878785
  64. 64. Bradley BP, Mogg K, Millar N, White J. Selective processing of negative information: effects of clinical anxiety, concurrent depression, and awareness. J Abnorm Psychol. 1995;104(3):532–6. pmid:7673577
  65. 65. Chen J, Wang Z, Wu Y, Cai Y, Shen Y, Wang L, et al. Differential attentional bias in generalized anxiety disorder and panic disorder. Neuropsychiatr Dis Treat. 2013;9:73–80. pmid:23326197
  66. 66. Price RB, Eldreth DA, Mohlman J. Deficient prefrontal attentional control in late-life generalized anxiety disorder: an fMRI investigation. Transl Psychiatry. 2011;1:e46. pmid:22833192
  67. 67. De Cort K, Hermans D, Spruyt A, Griez E, Schruers K. A specific attentional bias in panic disorder? Depress Anxiety. 2008;25(11):951–5. pmid:17932961
  68. 68. Deppermann S, Vennewald N, Diemer J, Sickinger S, Haeussinger FB, Dresler T, et al. Neurobiological and clinical effects of fNIRS-controlled rTMS in patients with panic disorder/agoraphobia during cognitive-behavioural therapy. Neuroimage Clin. 2017;16:668–77. pmid:29085773
  69. 69. Dresler T, Ehlis AC, Hindi Attar C, Ernst LH, Tupak SV, Hahn T, et al. Reliability of the emotional Stroop task: an investigation of patients with panic disorder. J Psychiatr Res. 2012;46(9):1243–8. pmid:22770507
  70. 70. Dresler T, Hindi Attar C, Spitzer C, Lowe B, Deckert J, Buchel C, et al. Neural correlates of the emotional Stroop task in panic disorder patients: an event-related fMRI study. J Psychiatr Res. 2012;46(12):1627–34. pmid:23058446
  71. 71. Gropalis M, Bleichhardt G, Hiller W, Witthoft M. Specificity and modifiability of cognitive biases in hypochondriasis. J Consult Clin Psychol. 2013;81(3):558–65. pmid:22563641
  72. 72. Kampman M, Keijsers GPJ, Verbraak MJPM, Näring G, Hoogduin CAL. The Emotional Stroop: A Comparison of Panic Disorder Patients, Obsessive-Compulsive Patients, and Normal Controls, in Two Experiments. Anxiety Disorders. 2002;16:425–41.
  73. 73. Lundh LG, Wikstrom J, Westerlund J, Ost LG. Preattentive bias for emotional information in panic disorder with agoraphobia. J Abnorm Psychol. 1999;108(2):222–32. pmid:10369032
  74. 74. Maidenberg E, Chen E, Craske M, Bohn P, Bystritsky A. Specificity of Attentional Bias in Panic Disorder and Social Phobia. Journal of Anxiety Disorders. 1996;10(6):529–41.
  75. 75. McNally RJ, Riemann BC, Kim E. Selective processing of threat cues in panic disorder. Behav Res Ther. 1990;28(5):407–12. pmid:2256899
  76. 76. McNally RJ, Riemann BC, Louro CE, Lukach BM, Kim E. Cognitive processing of emotional information in panic disorder. Behav Res Ther. 1992;30(2):143–9. pmid:1567343
  77. 77. McNally RJ, Amir N, Louro CE, Lukach BM, Riemann BC, Calamari JE. Cognitive processing of idiographic emotional information in panic disorder. Behav Res Ther. 1994;32(1):119–22. pmid:8135709
  78. 78. Reinecke A, Cooper M, Favaron E, Massey-Chase R, Harmer C. Attentional bias in untreated panic disorder. Psychiatry Res. 2011;185(3):387–93. pmid:20716465
  79. 79. Thomas SJ, Gonsalvez CJ, Johnstone SJ. Neural time course of threat-related attentional bias and interference in panic and obsessive-compulsive disorders. Biol Psychol. 2013;94(1):116–29. pmid:23727542
  80. 80. van den Heuvel OA, Veltman DJ, Groenewegen HJ, Witter MP, Merkelbach J, Cath DC, et al. Disorder-specific neuroanatomical correlates of attentional bias in obsessive-compulsive disorder, panic disorder, and hypochondriasis. Arch Gen Psychiatry. 2005;62(8):922–33. pmid:16061770
  81. 81. Amir N, Freshman M, Foa E. Enhanced Stroop Interference for Threat in Social Phobia. Anxiety Disorders. 2002;16:1–9.
  82. 82. Boehme S, Ritter V, Tefikow S, Stangier U, Strauss B, Miltner WH, et al. Neural correlates of emotional interference in social anxiety disorder. PLoS One. 2015;10(6):e0128608. pmid:26042738
  83. 83. Britton JC, Gold AL, Deckersbach T, Rauch SL. Functional MRI study of specific animal phobia using an event-related emotional counting stroop paradigm. Depress Anxiety. 2009;26(9):796–805. pmid:19434621
  84. 84. Andrews TM, Anderson IM. Information processing in anxiety: a pilot study of the effect of manipulating 5-HT function. J Psychopharmacol. 1998;12(2):155–60. pmid:9694028
  85. 85. Quero S, Baños RM, Botella C. Cognitive biases in panic disorder: a comparison between computerised and card stroop task. Psychology in Spain. 2001;5(1):26–32.
  86. 86. Higgins JP, Altman DG, Gotzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomized trials. BMJ. 2011;343:d5928. pmid:22008217
  87. 87. World Health Organization. The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines: Geneva: World Health Organization; 1992.
  88. 88. Dalgleish T, Watts FN. Biases of attention and memory in disorders of anxiety and depression. Clin Psychol Rev. 1990;10:589–604.
  89. 89. Gotlib IH, Roberts JE, Gilboa E. Cognitive interference in depression. In Sarason I. G., Pierce G. R., & Sarason B. R. (Eds.), Cognitive interference: Theories, methods, and findings. 1996:347–377. Hillsdale, NJ, England: Lawrence Erlbaum Associates Inc.
  90. 90. Mogg K, Bradley BP. Attentional bias in generalized anxiety disorder versus depressive disorder. Cognit Ther Res. 2005;29(1):29–45.
  91. 91. Algom D, Chajut E, Lev S. A rational look at the emotional stroop phenomenon: a generic slowdown, not a stroop effect. J Exp Psychol Gen. 2004;133(3):323–38. pmid:15355142
  92. 92. Clausen AN, Youngren W, Sisante JF, Billinger SA, Taylor C, Aupperle RL. Combat PTSD and Implicit Behavioral Tendencies for Positive Affective Stimuli: A Brief Report. Front Psychol. 2016;7(758):1–7.
  93. 93. Beck AT, Rush A, Shaw B, Emery G. Cognitive Therapy of Depression. New York: The Guilford Press. 1979.
  94. 94. Bower GH. Mood and memory. American Psychologist. 1981;36:129–148. pmid:7224324
  95. 95. Mathews A, MacLeod C. Cognitive approaches to emotion and emotional disorders. Annu Rev Psychol. 1994;45:25–50. pmid:8135504
  96. 96. Hazeltine E, Ruthruff E. Modality pairing effects and the response selection bottleneck. Psychol Res. 2006;70(6):504–13. pmid:16151721
  97. 97. Riley RD, Thompson JR, Abrams KR. An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. Biostatistics. 2008;9(1):172–86. pmid:17626226
  98. 98. Ebersole CR, Olivia E. Atherton OE, Belanger AL, Skulborstad HM, Allend JM et al. Labs 3: Evaluating participant pool quality across the academic semester via replication. J Exp Soc Psychol. 2016;67:68–82.
  99. 99. Ataya AF, Adams S, Mullings E, Cooper RM, Attwood AS, Munafò MR. Internal reliability of measures of substance-related cognitive bias. Drug Alcohol Depend. 2012;121(1–2):148–51. pmid:21955365
  100. 100. Hedge C, Powell G, Sumner P. The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behav Res Methods. 2017;50(3):1166–86.
  101. 101. Field M, Christiansen P. Commentary on Ataya et al. (2012), 'Internal reliability of measures of substance-related cognitive bias' (2012). Drug Alcohol Depend. 2012;124(3): 189– pmid:22386685