https://orcid.org/0000-0001-5717-5117
Figures
Abstract
Background
Peer review is widely used in academic fields to assess a manuscript’s significance and to improve its quality for publication. This scoping review will assess existing peer review guidelines and/or checklists intended for reviewers of biomedical journals and provide an overview on the review guidelines.
Methods
PubMed, Embase, and Allied and Complementary Medicine (AMED) databases were searched for review guidelines from the date of inception until February 19, 2021. There was no date restriction nor article type restriction. In addition to the database search, websites of journal publishers and non-publishers were additionally hand-searched.
Results
Of 14,633 database publication records and 24 website records, 65 publications and 14 websites met inclusion criteria for the review (78 records in total). From the included records, a total of 1,811 checklist items were identified. The items related to Methods, Results, and Discussion were found to be the highly discussed in reviewer guidelines.
Conclusion
This review identified existing literature on peer review guidelines and provided an overview of the current state of peer review guides. Review guidelines were varying by journals and publishers. This calls for more research to determine the need to use uniform review standards for transparent and standardized peer review.
Protocol registration
The protocol for this study has been registered at Research Registry (www.researchregistry.com): reviewregistry881.
Citation: Song E, Ang L, Park J-Y, Jun E-Y, Kim KH, Jun J, et al. (2021) A scoping review on biomedical journal peer review guides for reviewers. PLoS ONE 16(5): e0251440. https://doi.org/10.1371/journal.pone.0251440
Editor: Tim Mathes, Witten/Herdecke University, GERMANY
Received: November 24, 2020; Accepted: April 27, 2021; Published: May 20, 2021
Copyright: © 2021 Song et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This work was supported by Korea Institute of Oriental Medicine (KSN2021210 and KSN20212102; awarded to MSL). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1. Background
In academic fields, peer review is a fundamental part of assessing a manuscript’s potential significance to the field and improving its methodological rigor and its overall quality for publishing in scholarly journals. Editors depend on reviewers to make their editorial decisions since the reviewers have expertise in the subject areas of submissions they are reviewing. In terms of writing a paper, there have been many efforts to address inadequate reporting of researches and to achieve the standard in scientific writing through transparent reporting of researches [1]. Besides these guidelines, almost all journals, if not all, provide guidelines for authors at their websites. In recent years, guidelines and checklists have been disseminated by journals and publishers for each of the major study types to ensure consistent and complete reporting [2]. However, guidelines for reviewers are not easily discovered. In a survey of 116 health research journals on the McMaster list, Hirst and Altman found only 41 journals (35%) to be providing online instructions to peer reviewers [3]. Ensuring consistent and complete reviewing is also needed. Publishers like Elsevier, SAGE, Springer Nature, and Wiley provide openly available guidelines for reviewers to assist them through the review process, in order to maintain and support standards in reviewing. Yet, not all publishers provide such guidance to their reviewers openly without any technical barriers such as signing in as a reviewer.
Scoping reviews are similar to systematic reviews in the way they systematically examine evidence and the way they are conducted, but they differ from each other in their purpose. The main differences relate to critical appraisal or risk of bias, results synthesis, and study inclusion. Critical appraisal or risk of bias assessment is necessary for systematic reviews, while it is optional for scoping reviews. Systematic reviews synthesize findings from each included study using meta-analysis or meta-synthesis, while scoping reviews do not synthesize findings [4, 5]. Scoping reviews include more papers broadly to provide an overview of the existing literature regardless of their type or quality [6]. Systematic reviews are used as references for policymaking or clinical practice as they are suitable for clearly defined questions relating to an intervention efficacy, but scoping reviews are descriptive and suitable for examining evidence in a broader sense such as identifying characteristics of the evidence on a topic [7, 8] or identifying available evidence in a field [4]. As peer review is a widely used practice for publication of academic research, there is a need to identify available evidence on peer review guidelines, and to clarify definitions and concepts of peer review process with the evidence.
Currently, there are no widely used standard guidelines for peer reviewing manuscripts which may intimidate or frustrate inexperienced peer reviewers [9]. A lack of standardization may lead to difficulty in reviewing as they are uncertain about what are expected from them. Little has been published about peer review and not much resources have been provided [10]. This scoping review will determine existing peer review guidelines and/or checklists intended for reviewers of biomedical journals. The purpose of this study is to determine whether there are enough evidence and research to set standards on how reviewers of biomedical journals should perform peer reviews and write their review reports, and to provide an overview on the review guidelines.
2. Materials and methods
A scoping review of manuscript peer review guidelines for reviewers of biomedical journals was conducted to identify available evidence on peer review guidelines for reviewers and to determine what is communicated to peer reviewers about how to peer review. In order to improve the methodological and reporting quality of this scoping review, the checklist of PRISMA Extension for Scoping Reviews (PRISMA-ScR) [7], developed in 2018 by the EQUATOR Network was used (S1 File).
2.1. Registration
The protocol for this study has been registered at Research Registry with its unique identifying number, reviewregistry881 in April 2020.
2.2. Databases and search strategy
PudMed, Embase, and Allied and Complementary Medicine Database (AMED) were all searched from the date of inception until February 19, 2021. In the search, there was no date restriction. Reviewer guidelines by some of the major publishers, journals, and non-publishers were additionally hand-searched online and collected.
For comprehensive search on peer review guidelines in biomedical field, the following search terms were used: “reviewing”, “reviewer”, “peer review”, “peer reviewer”, “referee”, “journal”, “manuscript”, “guide”, “guidelines”, “how to”, “lesson”, “tips”, “advice”, “recommendation”, “report form”, and “checklist”. In order to search more publications for comprehensive results, Text Word search was performed, and MeSH term for “peer review” was used for search in PubMed. The search strategy was reviewed by all authors and agreed upon. The complete search strategy can be found in S2 File.
For grey literature, websites of major biomedical journal publishers were searched manually by browsing and clicking through the websites to look for guidelines for reviewers. The publishers were selected based on Simba Information (www.simbainformation.com)’s report, Global Medical Publishing 2019–2023, which provides a list of the leading medical publishers. More publishers were selected from those on Journal Citation Report (jcr.clarivate.com) in the Integrative and Complementary Medicine category. Subsidiary websites were not included in the search of major publishers. The websites searched are as follows: Elsevier (www.elsevier.com), Springer (www.springer.com), Nature (www.nature.com), Wiley (www.wiley.com), Wolters Kluwer (www.wolterskluwer.com), Taylor and Francis (www.tandfonline.com), SAGE (www.sagepub.com), PLOS (www.plos.org), MDPI (www.mdpi.com), Hindawi (www.hindawi.com), Thieme (www.thieme.com), Mary Ann Liebert (www.liebertpub.com), Cognizant Communication Corporation (cognizantcommunication.com), InnoVision Health Media (innovisionhm.com), World Scientific Publishing (www.worldscientific.com), BioMed Central (www.biomedcentral.com), Karger Publishers (www.karger.com), and Lippincott Williams and Wilkins (lww.com). In addition, the websites of non-publishers were searched including Council of Science Editors (www.councilscienceeditors.org), the Committee on Publication Ethics (www.publicationethics.org), the EQUATOR Network (www.equator-network.org), and the ICMJE (www.icmje.org), the World Association of Medical Editors (www.wame.org), the International Congress on Peer Review and Scientific Publication (peerreviewcongress.org) and Publons (www.publons.com). For the EQUATOR reporting checklists published both on their website and some journals, they were considered as website items all under the EQUATOR Network as a non-publisher organization.
2.3. Inclusion and exclusion criteria
The inclusion criteria for this study were any publication relating to reviewer guidelines or checklists that a manuscript reviewer may refer to point by point during peer review, websites of publishers and non-publisher organizations to determine the evidence within the biomedicine field. All types of articles in English (or English translation) were eligible for inclusion in this study without restrictions on date or study design. Thus, editorial materials such as editorial and tutorials were included. Among the websites of above-mentioned publishers and non-publisher organizations in the 2.2 Databases and search strategy section, those with reviewer guidelines or checklists were eligible for inclusion.
The exclusion criteria were statistical review, conference abstracts, non-English articles, checklists primarily for authors, review process related contents, reviewer/editor roles and duties, assessing review quality or reliability, review challenges or issues, conference peer reviews, grant peer reviews, and those not relating to manuscript peer review. Websites that do not provide reviewer guidelines were not eligible for inclusion.
In order to assess and organize the included publications, a modified quality assessment tool was modified from the tool suggested by Hawker et al. [11] and used in this study to assess the publications included.
2.4. Outcomes
The primary outcomes were review checklist items such as a list of questions or points to consider during peer review. The secondary outcomes were the review guidance criteria. Before going through the screening, after the search of literature, the authors randomly pre-screened 10 publications to look for possible review guidance criteria, and based on the discussion by all authors with experience as peer reviewers, the criteria were developed. These criteria aimed to group the contents of reviewer guidelines, such as instructions or recommendations on the reviewer’s attitude, becoming a reviewer, manuscript contents, review resources, reviewer ethics, manuscript structure, study ethics, time management, and other components relating to reviewing manuscripts, and to provide a clear overview of available guidelines for reviewers. Then for each included publication, the criteria were independently selected by two authors.
2.5. Study selection and data extraction
A data extraction form was developed for this scoping review for data charting purposes. The data extracted from the included articles were first author’s last name, publication year, article type, guidance criteria, and number of checklist items.
Two independent review authors searched the databases and assessed the eligibility of the searched records. Any discrepancies on article inclusion were discussed with a third review author and resolved on consensus.
All checklist items for reviewing manuscripts were extracted into an Excel spreadsheet. Each item was identified and categorized by its relevant section in manuscript review. The section categories used were title, abstract, introduction, methods, results, discussion, conclusion, references, tables and figures, presentations, and other. The categories were based on the general sections and components of manuscripts and also modified from the category structure of the PRISMA 2009 checklist [12]. The items from the included records of publications and websites were categorized first by the original category of the checklist if they match. For instance, for the checklist items that were categorized under the methods category on the checklist, we categorized it accordingly under the methods category. The items without any specific category were reviewed by the two authors and their categories were determined.
3. Results
3.1. Search results
After screening and excluding articles and other records that fell under exclusion criteria, a total of 79 records (65 publications on peer review guidelines and 14 publisher and non-publisher websites with reviewer guidelines) were included for this study (Fig 1). Out of 79 records, 24 publications did not provide a checklist as in what to look for in the manuscript (relating to contents) and only provided guidance for reviewers as to how to review. The 65 included publications and the 14 websites contained 1,077 items and 734 items respectively (S3 File), which amounted to 1,811 items in total. Each item is a question or points to consider, such as “Is the title appropriate?” or “Accurate/Adequate Abstract.” The assessment results of each included publication are provided as a supplement (S4 File).
3.2. Characteristics of included articles and websites
The descriptive characteristics of the included articles and websites are summarized in Tables 1 and 2 respectively. Publication types of the included studies were mostly editorial materials (n = 48; 75.0%) such as tutorial, editorial, special articles, perspective, opinion, and commentary. Other types included were review articles (n = 14; 21.9%), research articles (n = 2; 3.1%), and guidelines (n = 1; 1.6%). These article types were extracted as they were presented in the publications. Among the included publications that provided information on how to peer review, 41 publications (63.1%) provided a checklist questions or criteria, but 24 publications (36.9%) did not provide a checklist questions or criteria.
3.3. Criteria of reviewer guidelines
The suggestions and instructions for reviewers provided in the included publications were grouped into 9 criteria: attitude, becoming a reviewer, contents, resources, reviewer ethics, structure, study ethics, time management, and other as presented in 2.4 Outcomes in the Methods section. Publications were determined to fit into one or more of the criteria as appropriate. The frequency of reviewer guides criteria extracted from the publications is provided at the bottom rows in Table 1.
3.4. Categories of reviewer checklist items
The checklist items were extracted from the checklists in the included publications and websites, and the items were grouped into 17 categories (Table 3). The top 3 categories with the largest number of items were the Methods (548 items, 30.3%) category, the Results (231 items, 12.8%) category, and the Discussion (191 items, 10.5%) category from the publication items and website items combined. Looking into them separately by publications and websites, the top 3 categories with the most items were also the same 3 categories of Methods, Results, and Discussion.
The grouped categories of the identified checklist items and their numbers are shown in a graphical representation (Fig 2). In the figure, the darkness of color and the size of diamond shape are associated with the number of items for each category.
Discussion
From an extensive range of sources, this scoping review produced a comprehensive list of items outlining guidance for conducting peer review. Review guidelines and checklists can be important for high quality review, especially since editors rely on reviewers and peer reviewer recommendations have an influence on editorial decisions [76]. Such influence of reviewer recommendations on editorial decisions becomes an issue when the quality of peer review is doubtful or when reviewer bias is present [77]. In order to prevent reviewer bias and better improve the peer review quality, some journals and publishers published and posted reviewer guidelines. However, Davis et al. [78] found the content of the manuscript review evaluation forms to vary considerably and relatively few journals asked reviewers to rate specific components of a manuscript. In their study, the review evaluation form of higher impact factor journals addressed statistical analysis, ethical considerations, and conflict of interest, while lower impact factor journals addressed scientific importance, validity, and novelty. Similarly, Gardner et al. [79] also stated that a reviewer report form was found to be more important in reviewing Methods and Results sections in manuscripts. In addition, in a study by Strayhorn et al. [80], the use of standardized review checklist was found to increase reviewer agreement.
In this review, the review guidance criteria with the two highest frequencies were found to be the Contents (95.3%) criterion and the Structure (75.0%) criterion. These are considered important components to consider during peer review and many of the included publications were highly focused on them. They reflect the quality of the peer review report inside (reviewing on manuscript contents) and outside (organizing review report structure). This is because the main functions of peer review are to assess the manuscript contents and to have the review results clearly delivered to editors and authors. In terms of the manuscript categories among the checklists, the checklist items with the most in number were the Methods, Results, and Discussion categories. This indicates that the authors as well as publishers and non-publisher organizations consider these categories to be the main parts of manuscripts that require careful evaluation over other parts as the manuscript quality depends highly on these parts when reviewed properly. However, bias frequently occurred while reviewing them which eventually affects editors and editorial decisions [81, 82]. The quality of the manuscripts could be improved by providing reviewer guidelines, especially touching on such important components. Although the reporting guidelines by the EQUATOR network (www.equator-network.org) are now widely accepted by major publishers and journals, they are most commonly used by authors in their reporting, since the main purpose of those guidelines was to improve author reporting. For this reason, guidelines by publishers or journals placed much less significance on guidelines for reviewers. Yet, reviewers are expected to cover a wide range of tasks. Glonti et al. [77] conducted a scoping review on the roles and tasks of peer reviewers and found reviewer tasks to include: organizing reviewing, making general comments, assessing and addressing content for each section of the manuscript, addressing ethical aspects, assessing manuscript presentation and providing recommendations. Manuscript peer reviews could be improved by increasing the consistency of ratings, narrative comments, and recommendations [83]. In order to improve the reliability of reviewers, one of the ways suggested by Kravitz et al. [84] was providing more effective guidelines for reviewers, and one suggested by Kwon et al. [85] was providing a checklist to reviewers. In more recent study, Stenfors et al. [66] also suggested that guidelines may help to better identify criteria indicating high-quality research in journal publications. DiDomenico et al. [86] discussed about previous studies that investigated the effectiveness of formal training on review quality and found the training to influence the review quality insignificantly or limitedly; thus, the authors suggested that reviewers may benefit from some guidance.
Primarily, checklists are meant to be mnemonic devices aimed at supporting authors and reviewers through a process with a general outline. The items for the evaluation of the manuscript structure or format had more overlaps than those for the evaluation of the manuscript content such as methods and results. The general items of the review checklist in the biomedical related field are most likely to overlap with non-biomedical field as well. As the checklist can be designed based on the overall purpose, some checklist can be more useful than others. Therefore, in order to improve current peer review conditions, we suggest that more research should be carried out and the various groups involved should come together to discuss about the need of a more standardized checklist to be developed in a systematic format and useful to many groups. This kind of attempt is still in progress as seen in a study protocol of randomized controlled trial by Speich et al. [87]. This trial aims to evaluate the effectiveness or the need of providing a short version of the CONSORT checklist and its explanation to peer reviewers for them to check during peer review.
Getting the various groups involved and developing a more standardized checklist would have a technical merit. The items employed for a checklist should be from a broad range of academic groups and reliable sources. Various perspectives that could represent the reviewer committee in general should be considered and factored into the design of the checklist. This is particularly important if the checklist is designed to decrease errors of omission during peer review. Although reviewer checklists themselves are generally not included for publication even for open peer review, the development of reviewer guidelines should be evidence based and requires a systematic and comprehensive approach. The design of effective checklist should include utilization of published guidelines, formation of expert panels for consensus and extensive pilot-testing of preliminary checklists to ensure that recommendations are valid, reliable, and applicable.
These days, being transparent and open about peer review process has gained more attention and is a critical issue for publishers and editors because it fosters them to reach fair and consistent decisions [88], which would then facilitate the authors to improve their manuscript and to move forward. Having standards in peer reviewing could make reviewers more at ease and simplify their review. As the guidelines for author reporting has encouraged structure standardization for publications, a uniform review guideline checklist would support the transparent peer review to be more transparent and standardized for the journals in the same field with similar scope. It would also help authors to better understand their insufficient parts of the manuscript. However, more evidence is needed to evaluate the effects of peer review checklist. In addition, on top of the general standardized review checklist, a list of specific questions that are tailored for the journal, depending on the study design or topic, could be added on.
Limitation of this study was the inclusion of mainly editorial publication types. Most of the included publications being editorial materials places concerns for the guidelines and checklists suggested, since editorials present the perspective of only the editorial’s author. Scoping reviews cover a wide range of publication types that quality assessment is not mandatory to perform [4, 6]. The publications on “how to review” were mostly editorial materials, including editorials, tutorials, and special contributions, and the generally used quality assessment tools could not be applied to them so the modified quality assessment was used for the assessment of the included publications. In addition, there were limitations in extracting more grey literature. Many journals do not provide reviewer checklists openly and they cannot be accessed by anyone. Thus, the checklists openly provided by publishers were only considered. While the manual search of websites for checklists added more to our data, this may have introduced human error. Other limitations were bias related to search strategies and inclusion criteria. Our preselected keyword search strategies may have introduced some bias or unintended exclusion of publications which should have been included. In addition, biomedical databases with access restrictions were also searched to provide more comprehensive data but this may cause barriers to the publications for researchers. The inclusion of only English language publications may have also excluded some content.
5. Conclusion
This scoping review explored peer reviewer guidelines in biomedical journals. It identified the extent of existing literature on peer review guidelines and provided an overview of peer review guidelines. It demonstrated the current state of peer review guidelines that they were not uniform which may imply the need for more uniform guidelines. This review could provide grounds for future studies related to reviewer guidelines. In order to produce objective criteria for reviewing manuscripts, further research is needed, and various groups involved in publishing academic manuscripts, such as publishers, editors, reviewers, and other experts, should come together to discuss on developing standard guidelines or recommendations for peer reviewers which may be used by itself or along with individual journal’s instructions for reviewers.
Supporting information
S1 File. Preferred Rreporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist.
https://doi.org/10.1371/journal.pone.0251440.s001
(DOCX)
S4 File. Quality assessment of included publications (n = 65).
https://doi.org/10.1371/journal.pone.0251440.s004
(DOCX)
References
- 1. Simera I, Altman DG. Reporting medical research. International journal of clinical practice. 2013;67(8):710–6. pmid:23656235
- 2. Alam M. How to review a manuscript. Dermatologic surgery. 2015;41(8):883–8. pmid:26218722
- 3. Hirst A, Altman DG. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PloS one. 2012;7(4):e35621. pmid:22558178
- 4. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology. 2018;18(1):143. pmid:30453902
- 5. Lockwood C, dos Santos KB, Pap R. Practical Guidance for Knowledge Synthesis: Scoping Review Methods. Asian Nursing Research. 2019;13(5):287–94. pmid:31756513
- 6. Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. International journal of evidence-based healthcare. 2015;13(3):141–6. pmid:26134548
- 7. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Annals of internal medicine. 2018;169(7):467–73. pmid:30178033
- 8. Sucharew H, Macaluso M. Methods for Research Evidence Synthesis: The Scoping Review Approach. J Hosp Med. 2019;7:3. pmid:31251164
- 9. Small HY, Eisner D. Writing a peer review: a primer for junior researchers. Cardiovascular Research. 2019;115(10):e93–e5. pmid:31334806
- 10. Araujo CGSd. Peer review: a constatntly-evolving scientific process. Arq Bras Cardiol. 2012;98:e32–e5. pmid:22378341
- 11. Hawker S, Payne S, Kerr C, Hardey M, Powell J. Appraising the Evidence: Reviewing Disparate Data Systematically. Qualitative health research. 2002;12(9):1284–99. pmid:12448672
- 12. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. pmid:19621072
- 13. Ades AE, Caldwell DM, Reken S, Welton NJ, Sutton AJ, Dias S. Evidence synthesis for decision making 7: a reviewer’s checklist. Medical decision making. 2013;33(5):679–91. pmid:23804511
- 14. Alexander GR. A guide to reviewing manuscripts. Maternal and child health journal. 2005;9(1):113–7. pmid:15880980
- 15. Alexandrov AV, Hennerici MG, Norrving B. Suggestions for reviewing manuscripts. Cerebrovascular diseases. 2009;28(3):243–6. pmid:19602875
- 16. Allen LA, Ho PM. Peer Review of a Manuscript Submission: A How-To Guide for Effective and Efficient Commentary. Circulation Cardiovascular quality and outcomes. 2017;10(12).
- 17. Allen TW. Conducting Proper Peer Review for a Journal. Bariatric Surgical Practice and Patient Care. 2014;9(1):18–20.
- 18. Altieri MS, Pawlik TM. Mentor of the Month Series: How to Review a Manuscript from an Editor’s Perspective? Journal of gastrointestinal surgery 2020;24:1452–1454. pmid:32207076
- 19. Annesley TM. Writing an Effective Manuscript Review: The 6 “Be’s” to Success. Clinical chemistry. 2013;59(7):1028–35. pmid:23741073
- 20. Benos DJ, Kirk KL, Hall JE. How to review a paper. Advances in physiology education. 2003;27(1–4):47–52. pmid:12760840
- 21. Brand RA. Reviewing for clinical orthopaedics and related research. Clinical orthopaedics and related research. 2012;470(9):2622–5. pmid:22752799
- 22. Brown LM, David EA, Karamlou T, Nason KS. Reviewing scientific manuscripts: A comprehensive guide for peer reviewers. The Journal of thoracic and cardiovascular surgery. 2017;153(6):1609–14. pmid:28526112
- 23. Cantor GH, Caswell JL, Crissman JW, Gillette DM, Gunson DE, Hogenesch H, et al. Veterinary Pathology and Peer Review. Veterinary Pathology. 2009;46(2):173–5. pmid:19261628
- 24. Christenbery TL. Manuscript peer review: a guide for advanced practice nurses. Journal of the American Academy of Nurse Practitioners. 2011;23(1):15–22. pmid:21208330
- 25. Crigger NJ. What We Owe The Author: rethinking editorial peer review. Nursing ethics. 1998;5(5):451–8. pmid:9782929
- 26. Currie G, McCuaig C, Di Prospero L. Systematically Reviewing a Journal Manuscript: A Guideline for Health Reviewers. Journal of medical imaging and radiation sciences. 2016;47(2):129–38.e3. pmid:31047176
- 27. Del Mar C, Hoffmann TC. A guide to performing a peer review of randomised controlled trials. BMC Med. 2015;13:248. pmid:26521647
- 28. Dhillon P. How to be a good peer reviewer of scientific manuscripts. The FEBS journal. 2021. pmid:33486891
- 29. Duchesne S, Jannin P. Proposing a manuscript peer-review checklist. NeuroImage. 2008;39(4):1783–7. pmid:18053748
- 30. Duff K, O’Bryant SE, Westervelt HJ, Sweet JJ, Reynolds CR, van Gorp WG, et al. On becoming a peer reviewer for a neuropsychology journal. Archives of clinical neuropsychology. 2009;24(3):201–7. pmid:19640873
- 31. Einarson TR, Koren G. To accept or reject? A guide to peer reviewing of medical journal papers. Journal of population therapeutics and clinical pharmacology. 2012;19(2):e328–33. pmid:22885272
- 32. England JR, Cheng PM. Artificial Intelligence for Medical Image Analysis: A Guide for Authors and Reviewers. American journal of roentgenology. 2019;212(3):513–9. pmid:30557049
- 33. Estrada C, Kalet A, Smith W, Chin MH. How to be an outstanding reviewer for the Journal of General Internal Medicine … and other journals. Journal of general internal medicine. 2006;21(3):281.
- 34. Genter MB. How to Review a Manuscript: Tips and Suggestions. International journal of toxicology. 2020;39(3):263. pmid:32153218
- 35. Halder N, Ramsay R, Tyrer P, Casey P. Peer reviewing made easy. Advances in Psychiatric Treatment. 2011;17(2):150–7.
- 36. Heddle NM, Ness PM. Reviewing manuscripts: tips and responsibilities. Transfusion. 2009;49(11):2265–8. pmid:19761546
- 37. Hill JA. How to Review a Manuscript. Journal of electrocardiology. 2016;49(2):109–11. pmid:26850498
- 38. Hunter J. Rapid Peer Reviewer Checklist for Rapid Reviews—RAPeer (DRAFT). Adv Integr Med. 2020;7(4):183–6. pmid:32837897
- 39. Kelly J, Sadeghieh T, Adeli K. Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide. Journal of the International Federation of Clinical Chemistry. 2014;25(3):227–43.
- 40. Kocak B, Kus EA, Kilickesmez O. How to read and review papers on machine learning and artificial intelligence in radiology: a survival guide to key methodological concepts. Eur Radiol. 2020.
- 41. Kotsis SV, Chung KC. Manuscript rejection: how to submit a revision and tips on being a good peer reviewer. Plastic and reconstructive surgery. 2014;133(4):958–64. pmid:24675196
- 42. Kottner J, Norman I. How to peer review and revise manuscripts submitted for publication in academic nursing journals. International journal of nursing studies. 2016;64:A1–a3. pmid:27692443
- 43. Kyrgidis A, Triaridis S. Methods and Biostatistics: a concise guide for peer reviewers. Hippokratia. 2010;14(Suppl 1):13–22. pmid:21487486
- 44. Lapin BR. Considerations for Reporting and Reviewing Studies Including Health-Related Quality of Life. Chest. 2020;158(1, Supplement):S49–S56.
- 45. Lazarides MK, Georgiadis GS, Papanas N. Do’s and Don’ts for a Good Reviewer of Scientific Papers: A Beginner’s Brief Decalogue. The International Journal of Lower Extremity Wounds. 2020;19(3):227–9. pmid:32525721
- 46. Lippi G. How do I peer-review a scientific article?-a personal perspective. Annals of translational medicine. 2018;6(3):68. pmid:29610756
- 47. Marusić M, Sambunjak D, Marusić A. Guide for peer reviewers of scientific articles in the Croatian Medical Journal. Croat Med J. 2005;46(2):326–32. pmid:15849858
- 48. Moher D. Optimal strategies to consider when peer reviewing a systematic review and meta-analysis. BMC Medicine. 2015;13(1):274. pmid:26521692
- 49. Oerther S, Watson R. Emerging nursing scholars guide to peer reviewing an academic manuscript. Nursing open. 2019;6(4):1284–8. pmid:31660154
- 50. Pai SA. How to referee a paper—and save the world. Indian journal of cancer. 2020;57(1):110–4. pmid:32129302
- 51. Paice E. How to write a peer review. Hospital medicine. 2001;62(3):172–5. pmid:11291469
- 52. Pietrzak WS. A practical guide to effective journal peer reviewing. The Journal of craniofacial surgery. 2010;21(3):631–6. pmid:20485095
- 53. Provenzale JM, Stanley RJ. A systematic guide to reviewing a manuscript. AJR American journal of roentgenology. 2005;185(4):848–54. pmid:16177399
- 54. Rosenfeld RM. How to review journal manuscripts. Otolaryngology—head and neck surgery. 2010;142(4):472–86. pmid:20304264
- 55. Rostami K, Khadjooi K, Abasaeed-Elhag R, Ishaq S. How to evaluate a manuscript for publication? Gastroenterology and hepatology from bed to bench. 2011;4(2):58–62. pmid:24834158
- 56. Rutkowski JL, Cairone JV. How to review scientific manuscripts and clinical case reports for Journal of Oral Implantology. The Journal of oral implantology. 2009;35(6):310–4. pmid:20017649
- 57. Salasche SJ. How to "peer review" a medical journal manuscript. Dermatologic surgery. 1997;23(6):423–8. pmid:9217794
- 58. Sasson A, Okojie O, Verano R, Moshiri M, Patlas MN, Hoffmann JC, et al. How to Read, Write, and Review the Imaging Literature. Current problems in diagnostic radiology. 2021;50(2):109–14. pmid:32024599
- 59. Schüttpelz-Brauns K, Stosch C, Matthes J, Himmelbauer M, Herrler A, Bachmann C, et al. Recommendations for reviewing a manuscript for the GMS Zeitschrift für Medizinische Ausbildung. GMS Zeitschrift fur medizinische Ausbildung. 2010;27(5):Doc75. pmid:21818220
- 60. Seals DR, Tanaka H. Manuscript peer review: a helpful checklist for students and novice referees. Advances in physiology education. 2000;23(1):52–8. pmid:10902527
- 61. Simpson KJ. Reviewing an Original Research Manuscript for the International Journal of Exercise Science: A Guide for Students and Professionals. International journal of exercise science. 2008;1(2):43–9. pmid:27182294
- 62. Smith DV, Stokes LB, Marx K, Aitken SL. Navigating manuscript assessment: The new practitioner’s guide to primary literature peer review. Journal of oncology pharmacy practice. 2019;25(1):94–100. pmid:29498322
- 63. Smolčić V, Simundić AM. Peer-review policy and guidelines for Biochemia Medica Journal. Biochemia medica. 2014;24(3):321–8. pmid:25351350
- 64. Son AR, Kim BG. Guidelines for reviewing manuscripts submitted to Animal Bioscience. Animal bioscience. 2021;34(2):161–2. pmid:33152211
- 65. Stahel PF, Moore EE. How to review a surgical paper: a guide for junior referees. BMC Med. 2016;14:29. pmid:26874834
- 66. Stenfors T, Kajamaa A, Bennett D. How to … assess the quality of qualitative research. The clinical teacher. 2020;17(6):596–9. pmid:32790137
- 67. Stone WJ, Navalta JW, Lyons TS, Schafer MA. From the Editors: A Guide for Peer Review in the Field of Exercise Science. International journal of exercise science. 2018;11(1):1112–9. pmid:30338014
- 68. Sucato GS, Holland-Hall C. Reviewing Manuscripts: A Systematic Approach. Journal of pediatric and adolescent gynecology. 2018;31(5):441–5. pmid:29936302
- 69. Sylvia LM, Herbel JL. Manuscript peer review—a guide for health care professionals. Pharmacotherapy. 2001;21(4):395–404. pmid:11310511
- 70. Talanow R. JRCR—updated author and reviewer guidelines for 2014. J Radiol Case Rep. 2014;8(6):39–40.
- 71. Tandon R. How to review a scientific paper. Asian journal of psychiatry. 2014;11:124–7. pmid:25248566
- 72. Tullu MS, Karande S. Peer reviewing an original research paper. Journal of postgraduate medicine. 2020;66(1):1–6. pmid:31898597
- 73. Venne V. Reviewing manuscripts for the Journal of Genetic Counseling: practical suggestions. Journal of genetic counseling. 2015;24(2):189–92. pmid:25527418
- 74. Walker BF. How to critique an article on therapy. Australasian chiropractic & osteopathy: journal of the Chiropractic & Osteopathic College of Australasia. 1997;6(2):52–7. pmid:17987150
- 75. Wilson JR. Responsible authorship and peer review. Science and engineering ethics. 2002;8(2):155–74. pmid:12092488
- 76. Vintzileos AM, Ananth CV, Odibo AO, Chauhan SP, Smulian JC, Oyelese Y. The relationship between a reviewer’s recommendation and editorial decision of manuscripts submitted for publication in obstetrics. American Journal of Obstetrics and Gynecology. 2014;211(6):703.e1–.e5.
- 77. Glonti K, Cauchi D, Cobo E, Boutron I, Moher D, Hren D. A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals. BMC Med. 2019;17(1):118. pmid:31217033
- 78. Davis CH, Bass BL, Behrns KE, Lillemoe KD, Garden OJ, Roh MS, et al. Reviewing the review: a qualitative assessment of the peer review process in surgical journals. Res Integr Peer Rev. 2018;3:4. pmid:29850109
- 79. Gardner MJ, Machin D, Campbell MJ. Use of check lists in assessing the statistical content of medical studies. British medical journal (Clinical research ed). 1986;292(6523):810–2. pmid:3082452
- 80. Strayhorn J, McDermott JF, Tanguay P. An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry. Am J Psychiatry. 1993;150(6):947–52. pmid:8494074
- 81. Coronel R, Opthof T. The role of the reviewer in editorial decision-making. Cardiovascular Research. 1999;43(2):261–4.
- 82. Lee CJ, Sugimoto CR, Zhang G, Cronin B. Bias in peer review. Journal of the American Society for Information Science and Technology. 2013;64(1):2–17.
- 83. Shattell MM, Chinn P, Thomas SP, Cowling WR 3rd. Authors’ and editors’ perspectives on peer review quality in three scholarly nursing journals. Journal of nursing scholarship. 2010;42(1):58–65. pmid:20487187
- 84. Kravitz RL, Franks P, Feldman MD, Gerrity M, Byrne C, Tierney WM. Editorial peer reviewers’ recommendations at a general medical journal: are they reliable and do editors care? PloS one. 2010;5(4):e10072. pmid:20386704
- 85. Kwon JY, Gonzalez T, Miller C, Cook SP, Briceno J, Velasco BT, et al. Assessment of variables that influence agreement between reviewers for Foot & Ankle International. Foot and Ankle Surgery. 2020;26(5):573–579. pmid:31416682
- 86. DiDomenico RJ, Baker WL, Haines ST. Improving peer review: What reviewers can do. American Journal of Health-System Pharmacy. 2017;74(24):2080–4. pmid:29074482
- 87. Speich B, Schroter S, Briel M, et al. Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial. BMJ Open 2020;10:e035114. pmid:32198306
- 88. Saxe JP. Opening the Black Box of Peer Review. Cell stem cell. 2019;24(6):843–8. pmid:31173714