Reader Comments

Post a new comment on this article

Complete rubbish

Posted by Phlogiston on 17 Aug 2015 at 02:29 GMT

I don't see how any conclusions can be drawn from such a weak data set. Choosing seven Wikipedia articles, somewhat arbitrarily labelling three of them as "controversial", and counting their edits isn't science. The authors seem to falsely correlate the number of changes to an article and the veracity of those changes. The data set only includes changes through July 2012. Vandalism reversion bots and tools have improved immensely since then, making any conclusions largely irrelevant.

The authors referred to an essay by a Wikipedia editor who wrote "while some articles are of the highest quality of scholarship, others are admittedly complete rubbish." One could say the same thing of PLOS ONE. The authors here claim that their results "reinforce the position that Wikipedia should not be used in academic citations." While no encyclopedia should generally be used in academic citations, this paper's very existence is evidence that Wikipedia is more reliable than peer-reviewed journal articles.

No competing interests declared.

RE: Not complete rubbish

WilsonAdam replied to Phlogiston on 19 Aug 2015 at 19:34 GMT

We do not argue that high edit frequency is inherently problematic or linked directly with decreased accuracy of the content. In this paper, we simply describe how the content in some Wikipedia pages can be surprisingly dynamic and discuss the potential consequences of that dynamism (such as being more difficult for an expert to constantly monitor). In many ways, our results are to be expected, or even tautological: debated topics are debated.

I should add, though, that while regular edits and multiple editors should generally lead to improved page quality, it is clear that some editors are obviously malicious. It is not surprising that this behavior will be more common on politically controversial pages (whether or not they are about scientific topics). For example, see this edit (https://en.wikipedia.org/...) in which the entire global warming page was replaced with "Global Warming is a sham!,” or this one (https://en.wikipedia.org/...) where much of the content was replaced with many repetitions of "THIS IS BULL S*** GLOBAL WARMING IS FAKE YOU HAVE ALL BEEN BRAIN WASHED, THE LIBERAL MEDIA AND AL GORE ARE TRICKING YOU ALL COVER YOUR EARS IN SCHOOL KIDS THIS IS BULL S***.” Granted, most large changes are quickly reverted, while the smaller and more subtle ones sometimes take hours/days before someone else notices. We also note in the paper, as you do in the comment, that vandalism detection software is constantly improving and should result in a reduction of egregious edits over time.

We are aware that analysis of two metrics for seven pages barely scratches the surface (a point we discuss in the article) and we are appropriately cautious in the generality of our conclusions (unlike many of the media reports about this article). However, I would be surprised if a more complete analysis would result in significantly different conclusions (see my first point: debated topics are debated). I’m excited by an idea posted on a wikipedia talk page about this paper (https://en.wikipedia.org/...) to undertake a more thorough evaluation of this topic (including quantifying the "number of editors blocked shortly after editing" to the "geographic distribution of the places listed” and getting "50 editors to look at random articles” to assess quality and bias). I would consider this short analysis an absolute success if it leads to any of those things.

As I am quoted saying here (http://www.washingtonpost...), “I’m not trying to lambaste Wikipedia in any way, because I think that they are doing an amazing thing,” says Wilson. “I just wanted to call attention to this simple idea, really, that the content is so dynamic, so that at any point when you check it, it can be different from when you check it the next time.” To a Wikipedia editor, this is so obvious it sounds silly to even say it (hence the "Captain Obvious" comments on the page linked above). To a casual Wikipedia reader who does not consider how the page came to be, though, it's worth mentioning. And to a science educator responsible for teaching students how to do research, it's critical that they understand the dynamics 'under the hood' of a Wikipedia page about a scientific topic.

Competing interests declared: Author of study