Reader Comments

Post a new comment on this article

PLoS, Pipe-Dreams and Peccadillos

Posted by pbio on 07 May 2009 at 22:13 GMT

Author: Stevan Harnad
Position: Canada Research Chair in Cognitive Sciences
Institution: Universite du Quebec a Montreal
Submitted Date: May 16, 2006
Published Date: May 16, 2006
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

I applaud and welcome the results of the Eysenbach (2006) study on 1492 articles published during one 6-month period in one journal (PNAS), showing that the Open Access (OA) articles were more cited than the non-OA ones. I also agree fully that the findings are unlikely to have been an artifact of PLoS's "strong and vested interest in publishing results that so obviously endorse our existence," nor of the fact that "the author of the article is also an editor of an open-access journal" (all quotes are from the PLoS editorial by MacCallum and Parthasarthy, 2006).

However, I am less sure that PLoS's and the author's vested interests are not behind statements (in both the accompanying editorial and the article itself) along the lines that: "solid evidence to support or refute... that papers freely available in a journal will be more often read and cited than those behind a subscription barrier... has been surprisingly hard to find." The online bibliography 'The effect of open access and downloads ('hits') on citation impact' records a growing number of studies reporting precisely such evidence as of 2001, including studies based on data from much larger samples of journals, disciplines and years than the PLoS study on PNAS- and they all find exactly the same effect: freely available articles are read and cited more.

There can be disagreement about what evidence one counts as "solid," but there can be little dispute that prior evidence derived from substantially larger and broader-based samples showing substantially the same outcome can hardly be described as "surprisingly hard to find."

In fact, the only new knowledge from this small, journal-specific sample was (1) the welcome finding of how early the OA advantage can manifest itself, plus (2) some less clear findings about differences between first- and last-author OA practices, plus (3) a controversial finding that will most definitely need to be replicated on far larger samples in order to be credible: "The analysis revealed that self-archived articles are also cited less often than OA articles from the same journal."

The latter (3) is a within-journal (one journal, PNAS) finding; the overwhelming majority of self-archived articles today (on which the prior large-sample OA citation advantage findings are based) do not appear in journals with a paid-OA option. Hence on the present evidence I have great difficulty in seeing this secondary advantage as any more than a paid-OA publisher's pipe-dream at this point.

The following, however, is not a pipe-dream, but a peccadillo: "no other study has compared OA and non-OA articles from the same journal." To be fair, this observation is hedged with "[a]s far as we are aware" (but the OA-advantage bibliography is surely public knowledge - or should be among advocates of public access to science) and the observation is further qualified with: "and [also] controlled for so many potentially confounding factors."

But it has to be stated that of these "potentially confounding" variables -- "number of days since publication, number of authors, article type, country of the corresponding author, funding type, subject area, submission track (PNAS has three different ways that authors can submit a paper)... previous citation record of the first and last authors... [and] whether authors choosing the OA option in PNAS chose to do so for only their most important research (they didn't)" - many are peculiar to this particular short-interval, 3-option, single-journal PLoS study. And several of them (country, subject, year) had already been analyzed in papers that had been published before this 2006 article and were not taken into account despite the fact that both their preprints and their postprints had been freely accessible since well before publication, and that at least one of them (Brody et al. 2005) had been explicitly drawn to the author's attention based on a preprint draft well before the article was submitted to PLoS.

Brody et al. (2005) had found that, alongside the OA citation advantage, more downloads in the first six months after publication are correlated with more citations 18 months later in physics; and Hajjem et al. (2005) had found higher citations for OA articles within the same journal/year for 1,307,038 articles published across 12 years (1992-2003) in 10 disciplines (Biology, Psychology, Sociology, Health, Political Science, Economics, Education, Law, Business, Management).


Brody, T., Harnad, S. and Carr, L. (2005) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 56. http://eprints.ecs.soton....

Eysenbach, G, (2006) Citation Advantage of Open Access Articles. PLoS Biology 4(5): e157 DOI: 10.1371/journal.pbio.0040157

Hajjem, C., Harnad, S. and Gingras, Y. (2005) Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47. http://eprints.ecs.soton....

MacCallum, C.J., and Parthasarathy, H. (2006) Open Access Increases Citation Rate. PLoS Biology 4(5): e176 DOI: 10.1371/journal.pbio.0040176

Competing interests declared: Advocate of Open Access Self-Archiving