Re: controversy on "Priming Intelligent Behavior: An Elusive Phenomenon".
From: J. Daunizeau (acting as PLoS ONE academic editor)
Date: the 18th of March 2013

On the 10th of January 2013, I decided to accept for publication the manuscript entitled "Priming Intelligent Behavior: An Elusive Phenomenon", submitted by Shanks and collaborators as a regular article in PLoS ONE. This decision was based upon a thorough review by two independent researchers in the field. In brief, the manuscript reports outstanding difficulties in replicating the so-called "intelligence priming effect", which essentially states that people's intellectual performance is partly determined by prior exposition to intelligence-related concepts. A simple example of this would be that people have a higher IQ score if primarily exposed with the name "Albert Einstein" than with the name "Claudia Schiffer". More precisely, the work by Shanks et al. contains both (i) replications of priming manipulations published in the literature and (ii) additional experiments, which were designed to test hypotheses regarding the lack of significant effect.

As readers will now see, criticism of this work has been raised by Prof. Dijksterhuis, who previously authored a series of papers reporting specific forms of intelligence priming effect (e.g., "behavioural assimilation"). This work was partly based on experiments, whose replication was attempted (but did not succeed) in the work of Shanks and collaborators. The nature of the criticism is related to the "null" nature of the results of Shanks et al.: Pr. Dijksterhuis argues that the experiments were either underpowered or not conducted carefully enough to prevent confounds from masking the effect. We refer the interested reader to Pr. Dijksterhuis comments, as well as to the ensuing response by D. Shanks and B. Newell.

This controversy is important, because notions like the intelligence priming effect are pervasive in the public debate regarding issues as difficult and polemic as the existence of ethnic differences in educational performance at school (c.f. the so-called "racial achievement gap"). In addition, this controversy co-occurs with open calls from renown scientific personalities (such as Nobel-prize winner D. Kahneman) to social psychologists to "restore the credibility of their field by creating a replication ring to check each-other results" (Nature, 3rd of october 2012). Note that the latter call for replication explicitly targets social priming effects. This means that both the paper and its criticism have to be taken very seriously.

First of all, recall that the debate here is mainly of an experimental and statistical nature.
On the one hand, one has to give Pr. Dijksterhuis credit for his experimental skills, which may have been key to detect the intelligence priming effect. On the other hand, if this effect was undoubtedly strong and robust, it would not be confounded by minor variants in the experimental design. In fact, these variants are necessary to assessing the generalizability of experimental results (which was the explicit objective behind the first two experiments of Pr. Shanks and collaborators).
At this stage, I have to reiterate that I (as well as the two independent reviewers) believe that the work by Shank et al. does not consist of "sub-standard experiments", which Pr. Dijksterhuis alleges. This is mainly because, given the reported effect sizes in the literature, it is very unlikely that Shanks et al. found themselves unable to detect the effect above and beyond confounds. This particular aspect of the debate is discussed at length using statistical power considerations in the main article and in the response to Pr. Dijksterhuis's criticism. In brief, I believe the work by Shanks et al. cannot be rejected on methodological grounds. In other words, their conclusions are well supported by their experimental data and analysis.

Second, this debate, essentially questioning the determinants of intelligence priming effects, is healthy, because it triggers new experimental questions. Simply speaking, if both the results by Pr. Dijksterhuis and Shanks hold, then there is something to understand, in terms of what psychological factors potentially gate such effects. An example of such moderator variable was briefly mentioned in Pr. Dijksterhuis' criticism (namely: self-perceived intelligence mediates intellectual performance, cf. Gallinski et al. 2008). It is now in the hands of the academic community to unravel these systematic determinants, if they can be.

Third and last, I would like to remind interested readers that this debate is open to external contributions. More precisely, references to related work, informed suggestions and comments regarding the experimental design, data analysis and/or its criticism, etc..., are welcome and encouraged. This is the raison d'être of open-access journals like PLoS ONE, which have the potential to overcome the limitations of expert peer-review.