Introduction

The honesty and integrity of scientists is commonly assumed to depend on multiple structural, sociological, and psychological factors [1]. The factors most commonly discussed in the literature include:

1. Policies: the presence, at the institutional or national level, of policies and structures to detect and punish misbehaviour and to promote research integrity is assumed to bolster scientific self-correction and deter scientists from engaging in irresponsible behaviour [2–4].
2. Culture: socio-cultural background is believed to play a role in research misconduct [5]. A particularly elaborate and testable theory was proposed by Lee and Schrank 2010 [6], who argued that the risk of scientific misconduct would be highest in “developmental” states (i.e. countries in which economic growth is prioritized at the expense of regulation) that adopted a German model of higher education, which is more hierarchical and less liberal. The risk should instead be lowest in “regulatory” states that adopted an Anglo-American model of higher education, because in these countries researchers are held accountable for their actions and mutual criticism takes precedence over respect for authority. China, South Korea and other countries would be examples of the former category, the UK and USA of the latter, whilst Germany (which is a regulatory state) and other countries would represent intermediate cases [6].
3. Peer control: at the core of the scientific method lies the principle that transparent, open communication and mutual criticism are essential elements of scientific self-correction. Scholars have therefore suggested that, when such elements are missing, for example when mentorship of trainees is inadequate and/or when collaborators are unable or discouraged from checking and criticising each others’ work, fraudulent or questionable behaviours become more likely [7].
4. Pressures to publish: in most scientifically active countries, competition for jobs and resources is growing and career success is determined to some extent by research performance. This puts virtually all scientists under some “pressures to publish” [8]. However, in several countries, productivity and impact are formally built into promotion criteria, which may force scientists to publish continuously and successfully to maintain their careers [9]. In a growing number of countries, moreover, institutions receive funding in proportion to their publication performance and might therefore make explicit pressures on their employees [10,11]. Finally, in a few countries researchers are rewarded with cash incentives, which are arguably not the source of actual pressures but rather a source of corruption [12].
5. Early-career: young researchers are believed to be particularly at risk of committing scientific misconduct for at least two interconnected reasons. On the one hand, they may not have fully internalized the ethos and principles of science [13]. On the other hand, they have not established a professional reputation yet, so they may have more to gain and less to lose from attempting fraud [14].
6. Gender: at least two independent studies suggested that males were overrepresented in findings of misconduct by the US Office of Research Integrity. This supported speculations that psychological characteristics including higher aggression, competitiveness, status-seeking and risk-taking, made males a high-risk category for scientific misconduct [15,16]. Other interpretations, however, are at least as plausible [17].

A firm belief in the importance of some of these factors underlies major policies and educational initiatives. Major research institutions, for example in Germany and The Netherlands, have revised their research evaluation criteria, following the unquestioned concern that productivity expectations are a major threat to scientific integrity [18,19]. Institutions around the world are being encouraged to establish regulations and structures to deal with cases of misconduct, under the assumption that such structures will have beneficial effects [3]. In the United States and other countries, universities are required to implement training in the responsible conduct of research, following the belief that early-career scientists are most susceptible to misbehaviour and need specific instruction [7,14].

What is the evidence to support such policies and their underlying beliefs? The most explicit connections between scientific misconduct and pressures to publish or other risk factors comes from surveys and focus groups [e.g. 9,10,13,16,20,21], sources that have known limitations. What scientists report when interviewed is of great use in many contexts, but ultimately reflects personal impressions and beliefs, and might not directly echo what scientists actually do. Moreover, results of surveys on research misconduct are significantly influenced by methodological choices and publication bias [22]. Less indirect evidence about scientific misconduct comes from studies that surveyed the literature using proxies of publication bias. These suggest that the ratio of positive to negative results is unequally distributed amongst disciplines [23], is growing in most fields and countries [24,25], and is higher in long-distance collaborations [26] and in scientifically productive countries [27–29]. These patterns lend independent support to conclusions derived by survey data about the prevalence of pressures to publish and other risk factors. However, studies measuring publication bias cannot separate the effects of actual scientific misconduct from those of conscious or unconscious biases, editorial decisions or even factors that are completely unrelated to scientific integrity, such as writing style [22,30].

A more promising source of evidence for scientific misconduct is offered by retractions of scientific papers, because these are usually the consequence of data fabrication, falsification and plagiarism [31]. Existing studies on retractions, however, have failed to control for important confounding factors, thus yielding inconclusive results. Analyses of retraction notices recorded in Medline have led researchers to suggest that scientific misconduct is growing and is particularly common in high-impact journals [32,33] and that data falsification might be more common in long distance collaborations [34]. These conclusions might be misguided, however, because they ignored the effects of policies, structures and cultures in which researchers operate—factors that might vary significantly over time and across countries, institutions, research fields, and journals [35]. The recent growth in retractions, for example, is entirely accounted for by the number of journals that have started to retract papers, so it is not a sign that scientific misconduct has increased [35]. Another major confounding factor in studies on retractions is represented by “prolific retractors”: uncovered cases of misconduct are increasingly likely to yield multiple retractions, and ignoring this fact might have skewed results of past analyses [36].

In addition to retractions, a powerful source of evidence about scientific integrity—and one that has been surprisingly overlooked by scholars—is represented by corrections to the scientific literature. Unlike retractions, corrections carry no stigma and do not affect the publication record, so they have no direct consequence on a scientist’s career. Unlike retractions, which are often accompanied by litigations and lengthy investigations (for current examples see retractionwatch.com), corrections are typically a friendly process, often solicited spontaneously by the authors of the erroneous paper. The stark difference between the connotations of retractions and corrections is well reflected in their history and current prevalence: retractions are an extremely recent phenomenon that has grown in parallel with the strengthening of journal policies on misconduct, whilst corrections have been issued in all disciplines at constant rates for at least half a century and their frequency is still about 30 times higher than that of retractions [35].

What can corrections tell us about scientific integrity? Corrections are the consequence of a mistake, and might therefore reflect some degree of “sloppiness”. However, authors are not strictly obliged to correct their papers, and incur no punishment if they fail to do so. Corrections, in other words, are actions of little consequence, willingly carried out by researchers who wish to perfect their work, protect their reputation, and avoid misleading their colleagues. As such, corrections may be considered manifestations of scientific integrity. It follows that any sociological or psychological factor that increases the risk of scientific misconduct, and therefore the likelihood of retractions, should have, at a minimum, a smaller (null) effect on corrections, and possibly even an opposite effect.

This study verified whether the occurrence of a retraction or a correction could be predicted by study characteristics that reflect the six risk factors discussed above. Fifteen specific predictions were tested, all of which had been proposed explicitly or implicitly in the published literature. To avoid introducing subjectivity in the tests, we followed the classification schemes used in previous publications (Table 1). Sampling and analyses were designed to overcome limitations of past studies on retractions, including the skewing effects of prolific retractors [36] (see Methods). This study, therefore, is a direct and independent test of common beliefs and hypotheses about what might threaten scientific integrity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. General hypothesised risk factors for research misconduct, parameters measured in this study, predicted association of such parameter with the likelihood to retract or correct, and summary of associations observed when selecting from each paper a multi-retracted author or, if this was not available, either the first or the last author (all numerical results are reported in S1 File).

https://doi.org/10.1371/journal.pone.0127556.t001

Methods

Analyses

To approximate normality and/or to report regression effects on a unique scale, all continuous predictor variables except the country-to-author ratio were transformed as where x is the predictor of interest.

Measured characteristics of corrected or retracted papers were compared to those of their matched-control papers using conditional-logistic regression, which is the model best indicated for matched control study design on large samples. Given j groups consisting of corrected papers and matched-controls, if p is the probability of being a corrected paper rather than a control and x is the set of n author characteristics tested as predictors, the log-odds of a paper being corrected are given by:

(1)

The conditional logistic regression model computes implicitly (“conditions out”) the first j terms on the right side of Eq 1, which correspond to the group-specific indicator variables. Estimates reported in all graphs, therefore, can be interpreted as the effects of study and author characteristics on the change in log-odds of being a retracted or corrected paper, adjusting for journal and issue—adjusting, in other words, for discipline, field, subfield, year and month of publication, journal retraction policy, mean journal impact and several other major confounding factors. Analyses were conducted using the package Survival, implemented in the open source statistical software R [40,41].

Effects of author characteristics on the number of retractions per author were instead analysed with a generalized linear model assuming Poisson distribution of errors and a quasi-likelihood function to account for over dispersion of data.

To assess the robustness of results, analyses were repeated identically using the following selection criteria for authors:

• multi-retracted or first author: if one or more co-authors of the included paper had more than one retracted paper amongst their entire literature production we selected the one with the highest number of retractions. If no co-author had more than one retraction, instead, we took the first author.
• multi-retracted or last author: same as above, but taking last author instead of first.
• first author: we always selected first author.
• last author: we always selected last author.
• random author: we selected one of the authors using a pseudo-random number generator

In each case, we avoided multiple inclusions of the same individual in the analysis (for example when the same individual was a first author in two or more papers included in the analysis) by selecting with a pseudo-random number generator one paper from that author and excluding the others.

Results and Discussion

Table 1 summarizes the general hypotheses tested, parameters measured, effects predicted and results obtained by analysing data on multi-retracted and first authors (henceforth, MRF) as well as multi-retracted and last authors (MRL, see Methods for explanations). In this section section we illustrate and discuss primarily results obtained on MRL, and will refer to other analyses only when needed (numerical results for all analyses are reported in the Supporting Information). Compared to MRF, results obtained on MRL showed a greater number of statistically significant effects, although in most cases the direction of observed effects was the same (Table 1, S1 File). The higher statistical significances observed with last authors are likely to be explained by the higher precision and statistical power of these analyses, achieved because last authors tend to be older and more highly ranked than first authors [42] and have published on average a larger number of papers (mean±SD = 29±66 and 75±112, respectively). An alternative interpretation, perhaps applicable to a subset of parameters and worth exploring in future research, is that these differences reflect genuine behavioural asymmetries between last and first authors.

The country of activity of an author (see methods) was a highly significant predictor of retraction. Australia, Germany, China South Korea and Turkey were more likely to host authors of retracted papers compared to the United States and, particularly in the case of MRF, The Netherlands and France were less likely. Countries differed much less in their correction likelihood, and effects were not statistically significant in most main and sensitivity analyses (Fig 1a, S1a Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Retraction and correction likelihood, by country characteristics.

Conditional logistic regression estimates of the association between country of author and likelihood to publish a paper that was later retracted or corrected. Effects are estimated by comparison with matched–control papers. Numbers in parentheses indicate the total number of papers from the specified country that are in the sample, respectively, of corrections and retractions. Each panel represents the results of two multivariable analyses, in which samples for correction and for retraction were analysed using identical models. The indicator reference category was USA (N: 1979 – 449) for panel A, and a generic “other countries” category in all other panels. The “other” category in panel A includes all countries with ≤90 data points in the sample. See Table 1 and Methods for further details.

https://doi.org/10.1371/journal.pone.0127556.g001

The between-country variance in retraction rates was significantly explained by national policies, but not always in the direction predicted. The likelihood of retraction was lower in countries that have policies and structures to handle allegations of misconduct, particularly when such policies are legally defined or institutional (Fig 1b, S1b Fig). The likelihood of a retraction was higher in countries where publication performance is rewarded with cash, in agreement with predictions. Contrary to predictions, however, the likelihood of a retraction was equal or lower in countries in which publication performance determines individual careers or national funding to institutions (Fig 1c, S1c Fig). Since true “pressures” are supposed to occur in these two latter categories and not in the former, these results contradict the pressures to publish hypothesis.

None of the policy categories considered was a significant predictor of corrections, consistent with our assumption that corrections are a spontaneous action that is felt to be inconsequential and is therefore indifferent to policies. Moreover, there was no overlap, in our sample, between authors of retracted and corrected papers, adding further credit to our assumption that corrections and retractions are distinct phenomena.

Countries with a regulatory structure and Anglo-American academic model—i.e. a model believed to optimize institutional control and peer criticism [6]—were significantly less likely to yield retractions, whilst culturally intermediate cases were most likely. German-developmental countries, where the risk of misconduct is predicted to be greatest, exhibited a non-significant tendency to produce more retractions and fewer corrections (Fig 1d S1d Fig). Again, no significant effect was observed for corrections, although the order in which effect sizes are distributed matches predictions almost linearly. Overall, therefore, these trends are in good agreement with the cultural hypothesis (Table 1).

A paper’s number of co-authors increased the likelihood of it being later corrected, but was not a significant predictor of it being retracted, yielding a partial support for the peer-control hypothesis. The countries-to-authors ratio was not associated with corrections or retractions (Fig 2, S2 Fig). This latter finding represents a null support for the peer-control model, but may not be considered a strong refutation, since the proxy used to measure the distance between authors was rather inaccurate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Retraction and correction likelihood, by team and individual characteristics.

Conditional logistic regression estimates of the association between author or team characteristics and likelihood to publish a paper that was later retracted or corrected. Effects are estimated by comparison with matched-control papers. Corrections and retractions were analysed separately using identical univariable analyses, testing each parameter in turn. The gender was analysed in a multivariable model, in which “male” was the reference category. All predictors except gender were log-transformed. Parameters are grouped by the general risk factor of which they are proxies. For further details, see Table 1 and Methods.

https://doi.org/10.1371/journal.pone.0127556.g002

As predicted by the early-career hypothesis, researchers who had a shorter publication history (measured twice, at the time of publishing the original paper and at the time of correcting/retracting it) were more likely to be the authors of a retracted paper, whilst no effect was observed for corrections (Fig 2, S2 Fig).

Against all predictions derived from the pressures to publish hypothesis, retracted papers were less likely, and corrected papers more likely to be authored by researchers that were highly productive and had published frequently in high-impact journals (Fig 2, S2 Fig). Effect sizes were larger, but with wider margins of error, for the size-independent and field-independent measurements (i.e. total number citations showed smaller effects than average citations per paper, which in turn showed smaller effects than average and normalized citations) suggesting the presence of a genuine effect underlying all these measures (Fig 2, S2 Fig).

The sex of the author’s first name was not significantly associated with the likelihood of either retraction or correction, in any of the analyses conducted (Fig 2, S2 Fig, S1 File). Retractions showed a non-statistically significant tendency to be authored less by female first authors, and equally or more by female last authors. These patterns might suggest that a link between gender and misconduct is modulated by career or status. Original speculations on the role of gender in scientific misconduct also suggested a career/status effect, but one that was opposite to the effect noted here [15]. Therefore, whilst these results may not rule out entirely a gender effect, they offer a null or a negative support for the gender hypothesis as formulated in the literature. Future studies should explore in greater detail the link between gender, academic status and scientific misconduct.

Overall, our findings support previous suggestions that national policies, socio-cultural conditions, research environment (including number of co-authors) and situational factors (i.e. career stage) are significant determinants of responsible and irresponsible practices. Effect sizes are in the medium to large range (see S1 File, the “exp(coef)” values correspond to change in odds ratio), which suggests that these risk factors have practical, not just statistical, significance. The hypothesis that males are prone to scientific misconduct, however, was not supported, and the widely held belief that pressures to publish are a major threat to scientific integrity was largely contradicted by analyses at the level of country as well as individual (Table 1).

Perceived problems with pressures to publish might be just a “shared myth” [43]. But, if that is the case, why is the myth so widely believed? Part of the explanation might lie in the disproportionate attention paid to extreme cases of fraud. Within our sample, we observed that extreme cases of “prolific retractors” (authors of several retracted papers) tended to fall in the highest percentiles of productivity, even though their average journal impact was mediocre (Fig 3). Since these cases often represent spectacular examples of fraud, they tend to attract the attention of the scientific community and the mass media, and thus might contribute to reinforcing a stereotype. Interestingly, these authors also tended to be male, so these extreme cases might reinforce gender stereotypes, too (Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Number of retractions per author, by team and individual characteristics.

Total number of retracted papers attributed, in the WOS, to authors included in this study (y axis), plotted against various individual performance parameters (x-axis). All predictor parameters were log-transformed. Authors with no retractions were omitted. For further details, see Fig 2 and Methods.

https://doi.org/10.1371/journal.pone.0127556.g003

Just like the growth in retractions is explained by changes in journal policies [29], the rise of cases of prolific retractors can be explained by the strengthening of national policies against scientific misconduct. Authors from countries that have misconduct policies of any kind had a higher average number of retractions per individual (they were, in other words, more likely to be “prolific retractors”), as would be expected if multiple retractions by one individual followed from thorough investigations. Authors from countries in which performance was incentivized, instead, did not have significantly more retractions per individual, as would have been predicted by the pressures to publish scenario (Fig 4, see S1 File for results on first authors). This analysis, however, was conducted post hoc and is very coarse-grained (see Limitations section). We cannot exclude that extreme cases of prolific retractors might result from stronger incentives or pressures to publish, and future research should test this hypothesis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Number of retractions per author, by country characteristics.

Total number of retracted papers attributed, in the WOS, to authors included in this study, by country of activity (top) and by policy characteristics of those countries (bottom). For further details, see Table 1 and Methods.

https://doi.org/10.1371/journal.pone.0127556.g004

The fundamental conclusions drawn from this study are supported by multiple sensitivity analyses. Effects observed in most parameters were similar in direction or null, but never opposite, when analyses were repeated using multivariable regression (S3 and S4 Figs), in analyses limited to US authors (S4 and S6 Figs) and in tests using different author selection criteria (S1 File). Moreover, results were not significantly affected by name disambiguation errors (see Methods). Our inclusion criteria for corrections were conservative because they included corrections that had not been signed by the authors of the original paper (which might represent errata, i.e. corrections of editorial mistakes). If these non-author-signed correction were excluded (sample size: N = 1054 corrections and N = 2035 matched controls), results were substantially unchanged. Inclusion criteria for retractions were also conservative, because they included retractions due to honest mistakes. We tried limiting analyses to retractions that had not been signed by all authors (which might indicate a serious disagreement and therefore a possible case of misconduct, N = 556) and results obtained were not substantially different. However, if assumptions underlying this study are correct, then retractions due to honest errors should exhibit patterns similar to those observed for corrections and different or opposite to those observed for retractions. This is a new prediction that future studies should test.

Supporting Information

Acknowledgments

This work was funded by a Committee on Publication Ethics (publicationethics.org) Small Research Grant and by the Canada Research Chairs program.