Introduction

Scholarly publishing has been steadily growing for the last two centuries with estimates of more than 3 million articles published per year [1]. In recent times, there has been an increasing focus on the detection and prevention of misconduct and detrimental research practices, in enhancing research rigor and transparency, and in cultivating a research climate best suited to foster these goals [2–5]. Specifically, calls have been made to increase the transparency in conducting and reporting research by registering projects and data analyses plans before data collection, using reporting guidelines when writing up studies, posting preprints, sharing (raw) data, reproducing or replicating studies, as well as rewarding, hiring or promoting researchers based on these practices [6–9]. One such call was made in 2014, when the Transparency and Openness Promotion (TOP) Committee developed eight transparency standards related to: 1) citations, 2) data, 3) analytic methods (code), 4) research materials, 5) data and analysis reporting, 6) preregistration of studies, 7) preregistration of analysis plans, and 8) replication of research [10]. Since then, more than 5,000 journals and 80 organizations became TOP signatories [11]. However, to the best of our knowledge, attitudes related to all aspects of TOP guidelines have not been systematically assessed, and, yet, without agreement from all stakeholders regarding these guidelines, it is unlikely that scholarly practices will change.

It was therefore our goal to assess differences in attitudes and perceptions between authors, reviewers, and editors regarding the TOP guidelines, differences in perceptions of their work climates, and differences in their perceived prevalence of responsible and detrimental research practices.

Materials and methods

We reported our study following the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines [12], as well as the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [13].

Results

Our overall response rate was 4.9% (3,659 out of 74,749 delivered emails) and included responses from 1,389 authors, 1,833 reviewers, and 434 editors. Respondents came from 126 countries, most commonly USA (16%), India (8%), Italy (7%), and Brazil (5%). Respondents’ median age was 44 (IQR 35 to 55, range 23 to 90). The majority worked for universities or colleges (62%), and were male (66%). While respondents came from all scholarly disciplines, most were from Physical (33%) and Life Sciences (25%), followed by Multidisciplinary (18%) and Health Sciences (13%). Similarly, while respondents came from all career stages, most had a publication record of 6 to 25 publications (39%), or more than 50 publications (25%). Finally, most respondents considered themselves to have either basic (44%) or intermediate (45%) knowledge of statistics. Full details on response rate calculation, assignment of respondents to authors, reviewers, and editors, their self-declared statistical knowledge, and sociodemographic characteristics are available in the Appendix. Summary results are presented below, while all answers, as well as percentages of respondents who chose “Don’t know” or “Not applicable” options, are shown in the Appendix.

Attitudes toward transparency in reporting and conducting research

Respondents’ attitudes towards 11 statements on transparency in reporting and conducting research, which were based on TOP guidelines, are shown in Fig 1. The lowest support, across all respondents, was for preregistering studies prior to conducting the research (21%), and the highest for appropriately citing all data, analytic methods (program code) and materials used in the study (95%). Regression analyses (Appendix Table A7 in S1 Appendix) revealed no significant differences between authors, editors and reviewers and significant associations for 3 factors: 1) discipline–Health Sciences researchers had more positive attitudes towards transparency in reporting and conducting research than researchers of other disciplines; 2) number of publications–researchers with less than 6 publications had more positive attitudes than other researchers; 3) country–respondents from India had more positive attitudes than respondents from other countries (Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Respondents’ attitudes toward transparency in reporting and conducting research.

In the numerical comparison differences between groups larger than 5% are in bold. In the graphical comparison highest percentage is darker. * Questions are sorted by the total agreement percentage. Order of questions as they were asked in the survey is presented in the Appendix. Slight variations exist for the number of respondents per question, exact numbers are available on our project’s data repository.

https://doi.org/10.1371/journal.pone.0270054.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Examples of univariate comparisons that were confirmed in regression analyses to be significantly associated with attitudes towards transparency in reporting and conducting research, perceptions of work climate or perceived prevalence of responsible and detrimental research practices.

In the graphical comparison highest percentage is darker. Slight variations exist for the number of respondents per question, exact numbers are available on our project’s data repository.

https://doi.org/10.1371/journal.pone.0270054.g002

Perceptions of work climate

Respondents’ perceptions on 13 statements about their own work climate are shown in Fig 3. Most respondents stated that they share their research data with other researchers unless legal or ethical reasons prevent them from doing so (79%), and that having access to others’ data benefits (or would benefit) them (79%). Two thirds (66%) considered the quality of peer review they received to be generally high, as well as the quality of publications in their field (64%). One fifth of respondents (20%) stated that due to the pressure to publish they sacrifice the quality of their publications for quantity, and one seventh of respondents (14%) stated that funders or sponsors interfered in their study design or study reporting. Regression analyses (Appendix Table A8 in S1 Appendix) revealed no significant differences between authors, editors and reviewers and significant associations with perceptions of work climate with two factors: 1) country–respondents from India or USA had more positive perceptions of their work climate than respondents from other countries; 2) age–younger respondents perceived a worse work climate than older respondents (e.g., younger respondents perceived more pressure to sacrifice quality for quantity, and they more often stated that long peer review times had negatively impacted their careers, Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Respondents’ perceptions toward their work climate.

In the numerical comparison differences between groups larger than 5% are in bold. In the graphical comparison highest percentage is darker. * Questions are sorted by the total agreement percentage. Order of questions as they were asked in the survey is presented in the Appendix. Slight variations exist for the number of respondents per question, exact numbers are available on our project’s data repository.

https://doi.org/10.1371/journal.pone.0270054.g003

Perceived prevalence of responsible and detrimental research practices

Respondents’ perceived prevalence of responsible and detrimental research practices are shown in Fig 4. Among detrimental practices, 38% of respondents perceived undeserved authorship, and 33%, prior relevant research not being cited, as (very) prevalent in their field. Of responsible practices, 32% perceived self-reporting of limitations, 19% perceived sharing of raw data, and 6% publication of studies with null or negative results to be (very) prevalent. Regression analyses (Appendix Table A9 in S1 Appendix) revealed editors perceiving fabrication, falsification, plagiarism, and omitting of references to be more prevalent than authors or reviewers. Additionally, editors also perceived undeclared conflicts of interest, and publication of studies with null or negative results to be more prevalent than reviewers (but not than authors, Appendix Table A9 in S1 Appendix).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Respondents’ perceptions of perceived prevalence of responsible and detrimental research practices.

In the numerical comparison differences between groups larger than 5% are in bold. In the graphical comparison highest percentage is darker. * Questions are sorted by the total agreement percentage. Order of questions as they were asked in the survey is presented in the Appendix. Slight variations exist for the number of respondents per question, exact numbers are available on our project’s data repository.

https://doi.org/10.1371/journal.pone.0270054.g004

Regression analyses also revealed 3 other factors associated with the prevalence of research practices: 1) country–respondents from India or USA perceived detrimental research practices to be more prevalent than respondents from other countries; 2) discipline–Health Sciences respondents perceived responsible practices, i.e., use of reporting guidelines, open peer review, publication of studies with null or negative results, and reporting of study limitations as more prevalent than respondents from other disciplines, but they also perceived one detrimental practice—ghost writing—to be more prevalent than the respondents from other disciplines; 3) age–younger respondents perceived undeserved authorship, as well as prior relevant research not being cited, as more prevalent than older respondents (Fig 2).

Discussion

Our study has shown that authors, reviewers and editors were not supportive of all TOP recommendations for transparency in conducting and reporting of research. For example, while 95% of respondents (strongly) agreed that researchers must appropriately cite study data, methods and materials; a large majority (74%) that authors must follow reporting guidelines, that journals must encourage publication of replication studies (61%), or that authors must share data (60%); only half (50%) felt that journals have to verify that study findings are replicable using the deposited authors’ data and methods of analysis, and 21% that studies must be preregistered. While we found no significant differences in these attitudes between authors, reviewers, and editors, we did observe differences between respondents of different countries, disciplines and research seniority. Overall, younger respondents, those from Health Sciences, or from India, had more positive attitudes towards the TOP recommendations. Direct comparisons of our results with other surveys are difficult, as to the best of our knowledge, no surveys have addressed all aspects of TOP guidelines, nor surveyed respondents from all disciplines or countries represented in our survey since the TOP guidelines were published. Furthermore, year when the survey is conducted, wording differences, use of scales versus single questions to assess attitudes, as well as differences in collected sociodemographic data that might be explored as explanatory variables, often don’t allow direct comparisons [16]. Nevertheless, a survey of 1,015 authors of observational studies, also conducted in 2018, showed that 63% of respondents used reporting guidelines, and that their attitudes towards, awareness, and use of reporting guidelines are influenced by journals’ endorsements [17], which was also echoed in earlier studies [18]. Our previous analysis of journals’ endorsements, however, showed that only 13% of journals across disciplines recommended the use of reporting guidelines, and only 2% required it [15]. Data sharing practices across sciences have not been systematically explored, but recent estimates indicated that data sharing was mentioned in 15% of biomedical [19], and in only 2% of psychological articles [20]. A 2020 systematic review indicated that most researchers have positive attitudes toward data sharing [21]. This discrepancy of positive attitudes versus lack of actual practice of sharing data is influenced by many factors, including requirements for job selection and promotion, dedicated funding and skills, as well as incentives, time and training required to prepare (anonymised) data for sharing [21]. Regarding preregistration, a 2017 survey of 275 authors of systematic reviews or meta-analyses, showed that 37% of participants agreed with making protocols mandatory [22]. While that percentage was higher than in our study, the study’s sample size was smaller, respondents were predominantly from biomedical disciplines, and the questions only pertained to preregistration of systematic reviews.

It is likely that the lack of support of editors in our study towards all aspects of TOP recommendations is one of the contributing factors why these recommendations have not been endorsed by a larger number of journals today. The roles of editors and journals in endorsing or requiring specific practices, the lack of resources they often face, and the lack of proper intervention studies, were discussed extensively in our previous publications [15,23]. Here we would like to add that uptake and attitudes toward different practices are also likely influenced by whom and how recommendations are made. More rigorous methodological steps during recommendation development (similar to those for reporting guidelines or clinical practice guidelines), and open feedback calls might have led to higher initial uptakes. Additionally, published case reports with cost estimates and practical tips from those involved in recommendation development, that then managed to change practices of their journals, departments or institutions, could perhaps lead to additional endorsements.

Our study has also shown that work climates of authors, reviewers, and editors still had room for significant improvements. Approximately two thirds of respondents (66%) found the quality of peer review they received, as well as the quality of publications in their field (64%) to be generally high, one fifth of the respondents (20%) stated that due to the pressure to publish they sacrifice the quality of their publications for quantity, and 14% stated that funders or sponsors interfere in their study design or study reporting. The finding regarding pressure to publish was associated with respondent’s age, with younger respondents feeling more pressure to sacrifice quality for quantity. Younger respondents also felt that long peer review times had a more negative effect on their careers than did the other researchers. Again, direct comparison with previous studies is difficult due to differences in questions, but a recent survey on 7,670 postdoctoral researchers from 93 countries indicated that 56% had a negative outlook on their careers, and that only 46% would recommend to their younger selves to pursue a career in research [24]. Furthermore, as most postdocs in that survey reported being hired for only short periods of time, it is likely that long peer review times, and the number of publications might have more impact on their job prospects than of (tenured) academics. The overall high satisfaction of researchers with peer review we found, has also been reported in previous studies, with a caveat of known differences in reported satisfaction among those that had their papers rejected vs those that had them accepted [25].

Finally, our study showed that most commonly perceived detrimental research practices were undeserved authorship (as was also shown by previous research) [26,27], and prior relevant research not being cited. While there were no significant differences between perceptions of authors, reviewers, and editors regarding the prevalence of undeserved authorship; editors perceived higher prevalence of relevant research not being cited as well as higher prevalences of fabrication, falsification, and plagiarism than authors or reviewers. These findings could be explained by the fact that most researchers often engage in conversations on authorship during their projects and publications, while less than 2% of researchers admitted to having fabricated or falsified data [28], or plagiarised other’s work [29], and therefore the latter practices are more likely to have been encountered or discussed by editors than by other researchers. Additionally, we observed differences in perceived detrimental research practices, with younger researchers finding undeserved authorship, and not citing relevant research to be more prevalent than older researchers. This mimics previous studies which showed that young researchers often felt they were doing all of the work while others were receiving the credit, and that they had more research experience than many starting faculty members [24].

We also found that Health Sciences respondents perceived use of reporting guidelines, open peer review, publication of studies with null or negative results, and reporting of study limitations to be more prevalent than respondents from other disciplines, which is congruent with the frequency of recommendations on those topics in Health Science journals compared to other disciplines [15]. Finally, we also observed that researchers from USA and India perceived detrimental research practices to be more prevalent than respondents from other countries. This could be a consequence of the higher visibility of the Office of Research Integrity, and its legal foothold in the USA [30], as well as role of the Society for Scientific Values in India [31], the number of misconduct cases in those countries, as well as possibly higher competition levels in those countries [32,33].

While our study assessed attitudes and perceptions of a large number of respondents across many countries and disciplines, it is not without limitations. First, due to the low response rate our findings are not necessarily generalizable (we discuss possible influences of self-selection and non-response biases in detail in the Appendix). Our response rate, however, was similar to other recent large online surveys [16,22,34–36], and response rates have been consistently found to be lower in online versus other modes of survey dissemination [37]. The recent exception to this pattern was a 2019 survey of 2,801 researchers from economics, political science, psychology, and sociology regarding open science practices, which had a response rate of 46%. However, each participant in that survey was compensated with either 25 or 40 dollars (randomly) if they were students, or 75 or 100 dollars (randomly) if they were an author.

Second, while we did have respondents from 126 countries, we explored differences between respondents’ attitudes and perceptions for only 4 countries with highest number of respondents in our survey. Further research is warranted to determine national or institutional differences [16,38]. Third, although our survey was confidential, previous research has suggested that researchers often overestimate detrimental research practices of their colleagues [28], but may also underreport such practices in order to protect the reputation of their field or themselves, for being unwilling to report such practices for official investigations [39]. Fourth, to preserve confidentiality, we did not ask information on respondents departments or universities. This precluded taking into account potential clustering of some observations as it is possible that witnessing the same detrimental practice or being aware of the same high-profile cases within a department or a field might have led to overestimation of such practices. Finally, we did not define all terms used in the survey, so some differences between respondents might also stem from their different interpretations of some terms. For example, previous surveys on falsification yielded higher estimates when the term ‘falsified’ was not used but researchers were instead asked if they had ever altered or modified their data [28]. Sixth, while we explored several sociodemographic characteristics, publication practices and knowledge of statistics as factors associated with respondents’ attitudes and perceptions, previous research has also shown influence of respondents personality traits which we did not measure in our study [40].

In conclusion, our study has found that attitudes of authors, reviewers, and editors did not significantly differ regarding the TOP guidelines or their perceptions of their work climates. We also observed differences in the perceived prevalences of detrimental practices between editors and authors or reviewers, which highlights the need to raise awareness of these issues among all stakeholders, and to develop projects where all stakeholders would be working together to eradicate or minimize them. More studies are also needed to showcase the impact of any policy changes, as well as studies that lower the burden of implementing such policies [41]. Finally, recognition and rewarding of responsible practices should move from recommendations to actual practice [9].

Supporting information

Acknowledgments