Reader Comments

Post a new comment on this article

Limitations that may change findings

Posted by PAJSIJWA on 17 Apr 2019 at 07:28 GMT

Solehinejad et suggest their findings ‘supports a beneficial effect of tDCS on inhibitory control and WM in ADHD with a small-to-medium effect size’ and that ‘TDCS seems to be a promising method for improving neuropsychological and cognitive deficits in ADHD’. However, below are two limitations that can potentially alter the main findings and/or their interpretation

1. Pooling inhibitory and non-inhibitory outcome measures
In the meta-analysis of inhibitory control, the authors include outcome measures that do not specifically measure inhibitory control. For example, effects were drawn from measures that probe non-inhibitory functions, such as inattention (e.g., omission errors), processing speed (e.g., RTs for Go/No-Go Go Trials) and reaction time variability. Also, three included studies used Stroop or Flanker tasks to measure interference inhibition (Breitling et al., 2016; Nejati, Salehinejad, Nitsche, Najian, & Javadi, 2017; Soltaninejad, Nejati, & Ekhtiari, 2019) , but all three studies only reported performance on incongruent trials instead of the key outcome measure for interference inhibition (i.e., incongruent minus congruent trial RTs/errors). These limitations were not made clear to the reader. Mixing inhibitory with non-inhibitory measures confounds the main findings of the paper.

2. Dealing with multiple dependent effects
Another limitation is that effects in both the inhibitory control and working memory meta-analyses might be inflated because the authors pooled together multiple dependent effects. A meta-analysis assumes effects are independent (i.e., effects are drawn from different participant samples). If multiple dependent effects are pooled in the same meta-analysis, variance between effects is underestimated, leading to an overestimation of overall statistical significance (for discussion, see Becker, 2000; Borenstein, Hedges, Higgins, & Rothstein, 2009; Lipsey & Wilson, 2001). Where multiple dependent outcomes are reported, the convention is to calculate the effect size for just one outcome measure (e.g., the most comparable or the one showing the largest effect), but this reduces power and introduces bias (Borenstein et al., 2009). One alternative solution is to average and assume a correlated variance across dependent effects, which can help preserve effect independence and power (for a review and for other alternative solutions, see Becker, 2000; Borenstein et al., 2009; Hedges, Tipton, & Johnson, 2010; Van den Noortgate, López-López, Marín-Martínez, & Sánchez-Meca, 2013). Solehinejad et al included multiple dependent effects in all their meta-analyses, but it is not clear whether or how they controlled for dependent effects. Breaking the assumption of independent effects would not matter so much if the overall effect sizes they reported were large with a narrow confidence interval. However, the largest overall effect in the inhibitory control analysis was small with a borderline confidence interval (see pg 11 Table 3 in paper, anodal tDCS over dlPFC: Hedges’ g of 0.255 [95%CI 0.065 to 0.443]). Thus, it cannot be ruled out that their main finding might be statistically non-significant if the assumption of independence were not broken.

In sum, the main finding of this paper — that anodal tDCS over the dlPFC enhances performance on measures of inhibitory control — is possibly confounded by the above limitations. Their analysis conflates executive function measures and potentially inflates the overall effect size.

References
Becker, B. J. (2000). Multivariate meta-analysis. In Handbook of applied multivariate statistics and mathematical modeling (pp. 499–525). https://doi.org/10.1016/B...
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (Eds.). (2009). Introduction to meta-analysis. Chichester, U.K: John Wiley & Sons.
Breitling, C., Zaehle, T., Dannhauer, M., Bonath, B., Tegelbeckers, J., Flechtner, H.-H., & Krauel, K. (2016). Improving Interference Control in ADHD Patients with Transcranial Direct Current Stimulation (tDCS). Frontiers in Cellular Neuroscience, 10. https://doi.org/10.3389/f...
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/j...
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. In Practical Meta-Analysis. Thousand Oaks, CA, US: Sage Publications, Inc.
Nejati, V., Salehinejad, M. A., Nitsche, M. A., Najian, A., & Javadi, A.-H. (2017). Transcranial Direct Current Stimulation Improves Executive Dysfunctions in ADHD: Implications for Inhibitory Control, Interference Control, Working Memory, and Cognitive Flexibility. Journal of Attention Disorders, 108705471773061. https://doi.org/10.1177/1...
Soltaninejad, Z., Nejati, V., & Ekhtiari, H. (2019). Effect of Anodal and Cathodal Transcranial Direct Current Stimulation on DLPFC on Modulation of Inhibitory Control in ADHD. Journal of Attention Disorders, 23(4), 325–332. https://doi.org/10.1177/1...
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45(2), 576–594. https://doi.org/10.3758/s...

Competing interests declared: I have carried out a meta-analysis on tDCS in ADHD