Cristea and colleagues have published a critical meta-analysis on the efficacy of Cognitive Bias Modification (CBM) in addiction [1], after previous meta-analyses on CBM in anxiety and depression [2], and on mental health interventions in children and adolescents [3]. I will argue that the meta-analysis on addiction is plagued by methodological issues, most importantly, that the authors mix qualitatively different studies: experimental studies into mechanisms of bias change (typically performed with students with no desire to change their addictive behavior) and true clinical trials in which participants desire to change their addictive behavior. Before we turn to the present meta-analysis, it is good to briefly consider the validity of the previous meta-analyses which have attracted a number of critical commentaries and re-analyses. A number of important observations were made. First, CBM is built on the notion that changing a cognitive bias could have clinically meaningful effects, hence, if a procedure does not succeed in changing the bias, no clinically meaningful effects are to be expected [4,5]. Indeed, a first qualitative analysis corroborated this point: in almost all of the studies in which a bias was successfully changed, a change in psychopathology-related emotional reactions was observed, while studies in which the bias had not been changed did not result in changes in emotional reactions [5]. This point was dismissed by Cristea and colleagues because of the qualitative nature of this observation [3]. Importantly, the original meta-analysis was recently redone with the effectiveness of the CBM intervention as critical moderator [6]. And indeed, in line with the qualitative analysis, across the studies in which the bias was successfully changed (k=12), moderate to large effects were observed on emotional reactions or psychopathology (g=.60 for attentional bias modification, g=.40 for interpretation bias modification), with hardly any heterogeneity, while in studies where the CBM procedure did not result in a change in bias, the effect-size was zero (g=-.01). A related point concerns the delivery mode: while initial studies were done either in a lab or in a clinical context, later on investigators started large trials with online delivery of CBM. The latter studies mostly resulted in negative outcomes, both regarding the change in bias, and in clinical outcomes (which we now know are directly related). Cristea and colleagues attribute this to publication bias, but later availability of online tools for CBM could be a likely alternative explanation. Hence, from the first meta-analysis and the reactions and re-analysis, we can conclude that it is crucial to test to what extent a CBM procedure managed to change a bias, when considering clinically relevant outcomes. A second important moderator concerns the sample: are participants healthy volunteers (typically students) or patients? Interestingly, another meta-analysis was performed in this domain, now only including clinical samples [7]. This more restrictive meta-analysis found a moderate effect size (d=.42) on clinician-rated anxiety outcomes. Crucial moderator was again delivery mode, with no relevant effects in the internet-delivered CBM, and moderately strong effects in clinically delivered CBM. Hence, internet-interventions appear to be less well-suited for changing a bias (and therefore to have clinical effects). Given this state-of-affairs, researchers are testing methods to make bias-change more likely, also in online interventions (e.g. in anxiety by instructing participants first to expose themselves to threatening stimuli to generate a relevant mood-state [8]). In conclusion, we can learn from the field of anxiety that it is crucial to consider 1. whether the CBM-procedure was successful in changing the targeted bias (only then clinical effects can be expected); 2. To whom it was delivered (more evidence in clinical groups); 3. How it was delivered (internet-delivery appears less successful), which is likely related to point 1 (less efficient bias change).

Clinical Trials vs. Lab Experiments

With this in mind, let’s consider the meta-analysis of CBM in addiction by Cristea et al. [1] First, and most importantly, the meta-analysis does not differentiate between experimental lab-studies and clinical studies, but considers them all as RCTs (as also noted in the commentary by Field et al. [9]). That is a crucial shortcoming. In lab-studies, the goal is to investigate experimental procedures to change a bias, and to study (short-lived) effects on behavior. Neither investigators, nor participants have the goal to achieve a clinically relevant change. And the importance of motivation to change in addiction treatment can hardly be overstated, and at the heart of one of the most powerful psychosocial interventions, motivational interviewing [10,11]. Hence, when experimenters recruit participants who do not wish to change their behavior, long-term behavior change is not a relevant outcome-measure. Still, conducting such studies can be useful for other reasons: first to investigate whether a procedure can reliably change a relevant cognitive bias, and second, whether this leads to a short-term behavioral effect, as a way to test the hypothetical causal status of the bias [5,12,13].
Importantly, in lab-studies, the experimenters can attempt to change a bias in two directions: a bias can be increased (which should lead to increased craving and/or consumption) or it can be decreased (which should lead to decreased craving and/or consumption). Obviously, the first manipulation is not used in a clinical context, where typically a continued assessment control-condition is used (for a review, see [13]). In this context, typically students have been tested, who were not selected for a wish to change their addictive behavior, and typically do not have this desire, even when drinking at risky levels [14]. In this non-clinical context, effects on bias have been small (typically smaller for untrained than for trained stimuli, [15–17]), as have been effects on indicators of appetitive motivation such as craving and amount of alcohol consumed directly after the manipulation [15–17]). Arguably, these effects are interesting for two reasons: first, they provide evidence that a bias can be changed using experimental methods, and second, that this can have short-lived effects on appetitive behavior, as others have also concluded in recent reviews [13,18], and a meta-analysis [19]. From a clinical perspective, this might be a way to “nudge” unmotivated participants toward change, but clearly is in itself not sufficient as an intervention. But crucially, these studies were not meant to be studying clinical trials and should not be evaluated as such, that is mixing apples and oranges.

Clinical Trials on the Effectiveness of CBM in Alcohol Use Disorders

When we focus on true clinical trials, the numbers get small. For alcohol, two types of studies can be distinguished: clinical trials where volunteers were recruited who wished to reduce their drinking [20,21], and clinical trials where CBM was provided on top of regular treatment, typically consisting of multiple components including Cognitive Behavior Therapy (CBT) [22–24].* Given these small numbers, others have concluded that no valid meta-analysis can be done yet [25], and would be beyond the scope of a commentary. But let us briefly consider results of the true clinical trials. The first clinical trial [22] was small (N = 43), including both in- and outpatients. Patients were randomly allocated to an active condition of attentional re-training, or to sham-training including the same pictures and general motivating feedback. Results indicated a significant effect on the attentional bias (reduced in the experimental condition compared with the control condition after training), and an indication of a positive clinical effect (a significantly longer time to relapse in the trained group compared with the control group). Results should not be overstated, as it was a small clinical trial, but it does show that an attentional bias can be modified in this context, which might have beneficial clinical effects. Two subsequent large studies tested the effects of a different type of CBM (approach bias re-training) as an add-on to inpatient treatment [23,24]. The first study (N = 214) found a long-term clinical effect on relapse-rate after one year, at borderline significance, which was significant after controlling for gender, with less patients relapsing when they received CBM on top of treatment, compared with either sham-CBM (continued assessment) or no CBM. Supplementary analyses confirmed that the clinical effect was mediated by a change in alcohol-associations [26]. The second study compared CBM with no CBM, again on regular inpatient treatment in a large sample (N = 509) of alcohol-dependent patients [24]. The study replicated a long-term clinical effect (now 9% less relapse one year after treatment-discharge), which was mediated by the change in alcohol-approach bias. Also moderation was found: especially those patients with a strong alcohol-approach bias at pretest profited from this add-on training. As Cristea and colleagues note, most of these trials came from our group, therefore it is important that independent trials are done. And that’s what happened: an independent clinical trial in 83 Australian alcohol-dependent patients just came out [27], which found reduced relapse after real CBM compared with sham-CBM when provided during detox (at borderline significance in intention-to-treat analysis, and significant in per-protocol analysis). Hence, so far four clinical trials have been published with alcohol-dependent patients using varieties of CBM as add-on to treatment, and all four resulted in clinically relevant changes, in the two largest studies with evidence of mediation by the change in bias.
These results can be watered down by a meta-analysis including dozens of non-clinical trials in which students are tested who don’t wish to change their drinking behavior, as the meta-analysis by Cristea et al shows, but that does not take away the significance of the effects in clinical samples. The authors question the clinical significance of these effects, if they exist. Here it is crucial to first acknowledge that it is very difficult to find add-on effects to effective clinical treatment. For example, a large clinical trial combining different types of medication and psychological treatment, medication showed effects in the absence of clinical treatment [28]. If we estimate the long-term effect of adding CBM to psychological treatment to an additional 10% (based on the two largest studies with the same one-year follow-up outcome measure [23,24], NNT = 10), it implies that among the approximate 2000 patients treated so far with CBM in addition to clinical treatment, of the approximate 1000 given real CBM, 100 did not relapse within a year as a result of this add-on. We would argue that this is clinically relevant, and indeed for this reason CBM is now recommended in the German treatment guidelines (most trials took place in Germany) as a promising add-on intervention.
As noted, two other studies tested the efficacy of CBM in volunteers who wanted to reduce drinking [20,21]. This is a markedly different setting for different reasons: first, participants do not receive psychosocial treatment in addition to CBM, and second, the treatment goal in this context is not abstinence but reduction. (All other alcohol studies included students who came for course-credit or a small reward, but not with the goal to reduce their drinking, and therefore should not be considered as clinical trials, see above). The first study was done online [20], and found reduced drinking in several CBM-conditions, but also in the sham-CBM control condition. This suggests that problem-drinkers who are motivated to reduce their drinking are able to do so, with a rather minimal intervention, as was shown for online CBT [29]. It also shows that in this context (unlike the clinical context), there is no differential effect of real CBM and placebo-CBM. Registration of drinking and motivation appear to be enough. The second study compared CBM with a motivational intervention [21] in a 2 x 2 design (with both interventions being present or absent, hence no sham-training condition). Results showed a small short-lived reduction in drinking after CBM, and a slower, but longer-lasting effect of the motivational intervention, in the absence of an interaction effect. In summary, results of the two “sub-clinical” studies appear to be less promising than the results in the true clinical trials, although reduction of drinking was achieved in both studies, effects were small and non-specific. Given the consistent positive effects in true clinical trials (in motivated patients who receive CBM on top of treatment), an important next question in this domain is what minimal conditions are to achieve effects, for example, online CBT, which has been found to reliably produce a small effect [29], which could be combined with online CBM as we are testing [30], or minimal motivating outpatient treatment combined with CBM, as we are currently also testing [31].

Clinical Trials on CBM in Smoking Cessation

The situation for smoking is different than for alcohol use disorders, as there is no inpatient treatment for smoking cessation. We are aware of five clinical trials in smokers on the effectiveness of CBM to help people quit smoking, of which three were included in the meta-analysis [32–34], and two were not [35,36]. Three used attentional re-training [32,34,36], and two approach-bias re-training [33,35]. The first study (N=118) tested attentional re-training as an add-on to nicotine patches and behavioral support [32], and found no change on attentional bias, nor on smoking outcomes. However, no attentional bias was found at pretest, which limits the possibility of finding a training effect, given that the largest alcohol-trial found moderation such that those individuals with a strong bias profited from CBM [24]. A second small trial [34] included 67 smokers who wished to quit, who received 0, 1 or 3 sessions of attentional re-training, or the complementary number of placebo-training sessions (3,2,0, respectively). Here a bias for smoking was detected at pretest, and a stronger reduction was found with more sessions of real re-training. However, no significant effects were found for prolonged abstinence after four weeks. A recent larger trial [36] randomized 434 smokers who were selected because they made an actual quit attempt (not just the intention to quit) over attentional re-training or placebo-training. In the whole sample no significant effects were found. However, post-hoc contrasts indicated that heavy smokers (15+ cigarettes per day) showed a stronger cognitive bias at pretest, and had a higher chance of remaining abstinent after 6 months after CBM, indicating some promise for online attentional re-training in smoking cessation. Two studies tested approach-bias re-training for smoking. The first [33] randomized smokers who wanted to quit to two online interventions, one with and one without motivating feedback regarding reaction times, or to a waiting-list control-group (N=257). Results showed a significant reduction in numbers of cigarettes smoked per day for the standard training group compared with the other two groups. These results could be interpreted as some promise for CBM in smoking, and also as a warning that adding motivating elements may not always work out as planned (see also [37,38]). Finally, a small clinical trial was published targeting American and Dutch adolescents (N=60), with little motivation to quit, who received CBT with motivating elements, and CBM (real or placebo) on top. Here a small add-on effect of CBM was found on 7-day point prevalence abstinence, at statistical trend level. All in all, results for smoking cessation appear to be rather mixed, with some promising and some negative findings, as is the case for the “sub-clinical” trials to reduce drinking.

Publication Bias and Quality of Studies

Cristea and colleagues acknowledge that there is a small effect of CBM for clinical alcohol trials, but water this down further with corrections for publication bias and quality of research. As to publication bias, I would like to add that I was involved in publications of both positive studies and negative studies and we typically publish all. Then the quality of the studies: in all three meta-analysis [1–3], Cristea and colleagues evaluate the quality of the studies, and conclude that quality is wanting, with a high risk of bias in trials. Should this be true, then it is an important message for the field. However, one remarkable feature of the trilogy is that none of the authors have any (published) hands-on experience with CBM (both co-authors are recognized experts in meta-analyses of clinical trials). This could be framed as an asset (independence, no conflict of interest), but could also lead to misunderstandings, as we have seen above (combining clinical trials and experimental studies). In evaluating quality criteria, this appears to also be at least partly the case. For example, the first criterion, random sequence generation, is predominantly rated as “unclear risk of bias”. This criterion is clearly important in a traditional treatment setting, where it is important to know how exactly patients were allocated to treatment A or B. As CBM involves fully automated computer programs, randomization is typically done by a standard randomization sequence generator included in the experimental package. As this is so standard, most researchers and reviewers do not bother to ask for details. Hence, while Cristea et al maintain that this criterion points to low quality, the real state of affairs is that this is standard. (Note that the Cochrane handbook describes “Using a computer random number generator” as an indicator of “yes” for this criterion, and in the accompanying Table add to an example: “Probably done, since earlier reports from the same investigators clearly describe use of random sequences”; hence given that this is standard practice, it should be scored yes, [39], p. 198.) Similarly, allocation concealment was judged almost exclusively as “unclear”, but again, the computer program allocates patients to conditions following a computer random number generator. It is an interesting issue to what extent contingency awareness plays a role in CBM effects, and how to best deal with this issue in future trials. But that does not seem to be a different situation from other treatment research. Two criteria raised more variation: blinding of participants and personnel, and incomplete outcome data. The authors note that there is a relationship between a poor score on these quality parameters, and a positive outcome. That would pose an important challenge to the validity if it was not confounded with another variable: in single session experimental studies it is easy to leave participants and experimenters in the blind, and tests of awareness in this context typically demonstrated that participants were not aware of group assignment. However, in a clinical context this is more difficult, because patients receive multiple sessions of training, with an increasing chance of becoming aware of the purpose of the training. In this context we typically did not let patients guess treatment condition after each session, because it could stir curiosity and further jeopardize blinding. We did take measures to keep the treatment personnel in the blind (e.g., training took place in a separate computer-room, where a research intern started the program, hence the therapists were unaware of condition). We acknowledge that this criterion is important but this situation does not appear to be different from clinical research, where different clinical treatments are compared. Finally, the authors state that the vast majority of studies did not employ methods to include drop-out in the analysis. This is probably true across the selected studies by Cristea and colleagues, because of the initial error to include experimental lab-studies into an analysis of clinical trials (indeed, the vast majority), and in lab-studies, there typically is little or no drop-out, so the point is irrelevant and therefore typically not reported. In the smaller amount of true clinical trials, most studies have used intention-to-treat (itt) analysis, and some have also used more elaborate multiple imputation methods [20]. In conclusion, it is certainly not a bad idea to test the quality of research criteria for clinical trials, but obviously, this is only relevant for clinical trials and unfortunately the authors have chosen to include a majority of non-clinical trials in the present meta-analysis. Also some criteria appear to have been misjudged. Nevertheless, CBM-researchers could profit from considering these criteria for future research, especially the treatment of drop-out and optimally concealing treatment conditions.

Conclusion

In conclusion, the main undermining factor in this meta-analysis on CBM for addiction is the inclusion of non-clinical trials into a meta-analysis of clinical trials. There is no dispute that in unmotivated participants (typically smoking or drinking students), little can be achieved with CBM alone, as we also showed in several studies. As argued above, this type of study can still be interesting, because the efficacy of a CBM procedure can be tested to change a cognitive bias, along with immediate short-lived effects on appetitive behavior. In a true clinical trial, behavior change is the objective. The four published RCTs in which a variety of CBM has been added to treatment for alcohol use disorders, have all produced positive outcomes (either significant or at statistical trend level). These are small in effect-size, which is not surprising because they were added to an effective other treatment (CBT). And we would strongly argue that the approximate 10% gain in one-year outcome is clinically relevant. The six “semi-clinical” trials where people tried to change their addictive behavior using CBM as a stand-alone intervention (two alcohol and four smoking cessation trials), provide a more mixed picture with some negative and some more promising results. In this context the valid conclusion appears to be that it is important to study the boundary conditions under which CBM can produce clinically relevant effects. The combined findings suggest the following: CBM helps people who are motivated to change but do not succeed because of strong automatically-triggered reactions to the cues (as demonstrated by the fact that CBM works best in patients with a strong bias [24], by the effects of CBM on neural cue-reactivity [40] and in heavy smokers, who at a group level showed a stronger bias [36]). As the research in anxiety has demonstrated, it is also crucial that the CBM-procedure indeed changes the targeted bias [6], and indeed in the alcohol-domain, the two largest studies showed mediation of the clinical effect by the change in bias [24,26]. This may be harder to achieve in web-based trials, as was also found in both domains. Important follow-up questions are how these circumstances can be achieved with minimal means, in order to produce the desired clinical effects. Readers should not be distracted from this overarching perspective, based on a meta-analysis that confuses clinical trials and experimental studies.


* The authors list one other study with a clinical population [40]. This was a small-scale study into the neural effects of CBM, where 32 patients were scanned prior to and after either true CBM (approach-bias re-training) or sham-training. Results showed a stronger reduction in Amygdala reactivity to alcohol-cues compared with control pictures after CBM compared with sham-CBM. Given the small number of participants (constrained by the costs of fMRI), this was not a clinical trial on effectiveness of the training on clinical outcomes, but on the working mechanism.

Acknowledgement

Thanks to Matt Field, Elske Salemink, Janna Cousijn and Leroy Snippe for helpful comments on the initial draft of this commentary.

References

1. Cristea IA, Kok RN, Cuijpers P. The efficiency of cognitive bias modification interventions for addictions: A meta-analysis. PLoS One. 2016;11: e0162226. doi:10.1371/journal.pone.0162226
2. Cristea IA, Kok RN, Cuijpers P. Efficacy of cognitive bias modification interventions in anxiety and depression: Meta-analysis. Br J Psychiatry. 2015;206: 7–16. doi:10.1192/bjp.bp.114.146761
3. Cristea IA, Mogoase C, David D, Cuijpers P. Practitioner Review: Cognitive bias modification for mental health problems in children and adolescents: a meta-analysis. J Child Psychol Psychiatry. 2015;56: 723–734. doi:10.1111/jcpp.12383
4. Clarke PJF, Notebaert L, MacLeod C. Absence of evidence or evidence of absence: reflecting on therapeutic implementations of attentional bias modification. BMC Psychiatry. 2014;14: 8. doi:10.1186/1471-244X-14-8
5. Macleod C, Clarke PJF. The attentional bias modification approach to anxiety intervention. Clin Psychol Sci. 2015;3: 58–78. doi:10.1177/2167702614560749
6. MacLeod C, Grafton B. Anxiety-linked attentional bias and its modification: Illustrating the importance of distinguishing processes and procedures in experimental psychopathology research. Behav Res Ther. Elsevier Ltd; 2016; doi:10.1016/j.brat.2016.07.005
7. Linetzky M, Pergamin-Hight L, Pine DS, Bar-Haim Y. Quantitative evaluation of the clinical efficacy of attention bias modification treatment for anxiety disorders. Depress Anxiety. 2015;32: 383–391. doi:10.1002/da.22344
8. Kuckertz JM, Gildebrant E, Liliequist B, Karlström P, Väppling C, Bodlund O, et al. Moderation and mediation of the effect of attention training in social anxiety disorder. Behav Res Ther. Elsevier Ltd; 2014;53: 30–40. doi:10.1016/j.brat.2013.12.003
9. Field M, Jones A, Christiansen P. Questionable classification of studies and choice of outcome measures. PlosOne. 2016;11: e0162226 (comment).
10. Miller W, Rollnick G. Motivational Interviewing: Helping people Change. NY: Guilford; 2013.
11. Miller WR. Rediscovering fire: Small interventions, large effects. Psychol Addict Behav. Educational Publishing Foundation; 2000;14: 6.
12. MacLeod C, Rutherford E, Campbell L, Ebsworthy G, Holker L. Selective attention and emotional vulnerability: assessing the causal basis of their association through the experimental manipulation of attentional bias. J Abnorm Psychol. 2002;111: 107–123. doi:10.1037/0021-843X.111.1.107
13. Wiers RW, Gladwin TE, Hofmann W, Salemink E, Ridderinkhof KR. Cognitive Bias Modification and Cognitive Control Training in Addiction and Related Psychopathology: Mechanisms, Clinical Perspectives, and Ways Forward. Clin Psychol Sci. 2013;1: 192–212. doi:10.1177/2167702612466547
14. Wiers RW, van de Luitgaarden J, van den Wildenberg E, Smulders FTY. Challenging implicit and explicit alcohol-related cognitions in young heavy drinkers. Addiction. 2005;100: 806–19. doi:10.1111/j.1360-0443.2005.01064.x
15. Field M, Duka T, Eastwood B, Child R, Santarcangelo M, Gayton M. Experimental manipulation of attentional biases in heavy drinkers: Do the effects generalise? Psychopharmacology (Berl). 2007;192: 593–608. doi:10.1007/s00213-007-0760-9
16. Schoenmakers T, Wiers RW, Field M. Effects of a low dose of alcohol on cognitive biases and craving in heavy drinkers. 2007; doi:10.1007/s00213-007-1023-5
17. Wiers RW, Rinck M, Kordts R, Houben K, Strack F. Retraining automatic action-tendencies to approach alcohol in hazardous drinkers. Addiction. 2010;105: 279–87. doi:10.1111/j.1360-0443.2009.02775.x
18. Kakoschke N, Kemps E, Tiggemann M. Approach bias modification training and consumption: A review of the literature. Addict Behav. Elsevier Ltd; 2017;64: 21–28. doi:10.1016/j.addbeh.2016.08.007
19. Allom V, Mullan B, Hagger M. Does inhibitory control training improve health behaviour? A meta-analysis. Health Psychol Rev. 2015;7199: 1–38. doi:10.1080/17437199.2015.1051078
20. Wiers RW, Houben K, Fadardi JS, van Beek P, Rhemtulla M, Cox WM. Alcohol Cognitive Bias Modification training for problem drinkers over the web. Addict Behav. Elsevier Ltd; 2015;40: 21–26. doi:10.1016/j.addbeh.2014.08.010
21. Cox WM, Fadardi JS, Hosier SG, Pothos EM. Differential effects and temporal course of attentional and motivational training on excessive drinking. Exp Clin Psychopharmacol. 2015;23: 445–454. doi:10.1037/pha0000038
22. Schoenmakers TM, de Bruin M, Lux IFM, Goertz AG, Van Kerkhof DH a T, Wiers RW. Clinical effectiveness of attentional bias modification training in abstinent alcoholic patients. Drug Alcohol Depend. 2010;109: 30–6. doi:10.1016/j.drugalcdep.2009.11.022
23. Wiers RW, Eberl C, Rinck M, Becker ES, Lindenmeyer J. Retraining automatic action tendencies changes alcoholic patients’ approach bias for alcohol and improves treatment outcome. Psychol Sci. 2011;22: 490–7. doi:10.1177/0956797611400615
24. Eberl C, Wiers RW, Pawelczack S, Rinck M, Becker ES, Lindenmeyer J. Approach bias modification in alcohol dependence: do clinical effects replicate and for whom does it work best? Dev Cogn Neurosci. Elsevier Ltd; 2013;4: 38–51. doi:10.1016/j.dcn.2012.11.002
25. Christiansen P, Schoenmakers TM, Field M. Less than meets the eye: Reappraising the clinical relevance of attentional bias in addiction. Addict Behav. Elsevier Ltd; 2015;44: 43–50. doi:10.1016/j.addbeh.2014.10.005
26. Gladwin TE, Rinck M, Eberl C, Becker ES, Lindenmeyer J, Wiers RW. Mediation of Cognitive Bias Modification for Alcohol Addiction via Stimulus-Specific Alcohol Avoidance Association. Alcohol Clin Exp Res. 2015;39: 101–107. doi:10.1111/acer.12602
27. Manning V, Staiger P, Hall K, Garfield J, Flaks G, Leung D, et al. Cognitive Bias Modification Training During Inpatient Alcohol Detoxification Reduces Early Relapse: A Randomized Controlled Trial. Alcohol Clin Exp Res. 2016;In press. doi:10.1111/acer.13163
28. Anton RF, O’Malley SS, Ciraulo D a, Cisler R a, Couper D, Donovan DM, et al. Combined Pharmacotherapies and Behavioral Interventions for Alcohol Dependence. J Am Med Assoc. 2006;295: 2003–2017. doi:10.1001/jama.295.17.2075
29. Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, et al. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One. 2014;9: e99912. doi:10.1371/journal.pone.0099912
30. van Deursen DS, Salemink E, Smit F, Kramer J, Wiers RW. Web-based cognitive bias modification for problem drinkers: protocol of a randomised controlled trial with a 2x2x2 factorial design. BMC Public Health. BMC Public Health; 2013;13: 674. doi:10.1186/1471-2458-13-674
31. Boffo M, Pronk T, Wiers RW, Mannarini S. Combining cognitive bias modification training with motivational support in alcohol dependent outpatients: study protocol for a randomised controlled trial. Trials. 2015;16: 63. doi:10.1186/s13063-015-0576-6
32. Begh R, Munafò MR, Shiffman S, Ferguson SG, Nichols L, Mohammed MA, et al. Lack of attentional retraining effects in cigarette smokers attempting cessation: A proof of concept double-blind randomised controlled trial. Drug Alcohol Depend. Elsevier Ireland Ltd; 2015;149: 158–165. doi:10.1016/j.drugalcdep.2015.01.041
33. Wittekind CE, Feist A, Schneider BC, Moritz S, Fritzsche A. The approach-avoidance task as an online intervention in cigarette smoking: A pilot study. J Behav Ther Exp Psychiatry. Elsevier Ltd; 2015;46: 115–120. doi:10.1016/j.jbtep.2014.08.006
34. Lopes FM, Pires AV, Bizarro L. Attentional bias modification in smokers trying to quit: A longitudinal study about the effects of number of sessions. J Subst Abuse Treat. Elsevier Inc.; 2014;47: 50–57. doi:10.1016/j.jsat.2014.03.002
35. Kong G, Larsen H, Cavallo DA, Becker D, Cousijn J, Salemink E, et al. Re-training automatic action tendencies to approach cigarettes among adolescent smokers: a pilot study. Am J Drug Alcohol Abuse. 2015;41: 425–32. doi:10.3109/00952990.2015.1049492
36. Elfeddali I, de Vries H, Bolman C, Pronk T, Wiers RW. A Randomized Controlled Trial of Web-based Attentional Bias Modification to Help Smokers Quit. Heal Psychol. 2016;35: 0000–0000.
37. Boendermaker WJ, Boffo M, Wiers RW. Exploring Elements of Fun to Motivate Youth to Do Cognitive Bias Modification. Games Health J. 2015;4: 434–443. doi:10.1089/g4h.2015.0053
38. Boendermaker WJ, Prins PJM, Wiers RW. Cognitive Bias Modification for adolescents with substance use problems - Can serious games help? J Behav Ther Exp Psychiatry. Elsevier Ltd; 2015;49: 13–20. doi:10.1016/j.jbtep.2015.03.008
39. O’Connor D, Green S, Higgins JP. Defining the Review Question and Developing Criteria for Including Studies. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. 2008. doi:10.1002/9780470712184.ch5
40. Wiers CE, Stelzel C, Gladwin TE, Park SQ, Pawelczack S, Gawron CK, et al. Effects of cognitivebias modification training on neural alcohol cue reactivity in alcohol dependence. Am J Psychiatry. 2015;172: 335–343. doi:10.1176/appi.ajp.2014.13111495