Reader Comments

Post a new comment on this article

Deeply flawed study, from design through to conclusions

Posted by kanderson on 21 Jan 2014 at 16:30 GMT

My full post dissecting this study on the Scholarly Kitchen can be found here: http://scholarlykitchen.s...

This is a quick overview.

The author's assessed the wrong population of journals -- library copies. Most major multispecialty medical journals have a lot of demographic printings for various regions and specialists. Library copies get the fewest ads, except for Canadian journals to Canadian libraries, because there aren't as many demographic splits in Canada. So, their population was destined to reveal the data they saw. But it's erroneous. Their estimates for non-Canadian journal ad revenues are 2.5-3.0 times too low.

There was no need to do estimates, either. Actual, validated, third-party data are available on the market.

They failed to account for the value of controlled circulation copies, which provide thousands of physicians with these journals at no cost.

They calculated pages:revenues as 1:1, which is not true. As pages decline, costs rise, leading to another 20% error in the wrong direction.

The list of problems is long. When faced with the litany of problems, one author claimed that the study "Was meant to start a debate + push for real figures." That reveals two things -- the authors had an axe to grind and wanted to start an argument, not simply present data and conclusions from the data; and the authors were ignorant of how easy it is to get real figures in this area.

From its methods to its conclusions, this paper is misleading and incorrect.

No competing interests declared.