Dear Ian,

Thank you for submitting your revised manuscript entitled "Analysis of citation dynamics reveals that you do not receive enough recognition for your influential science" for consideration as a Meta-Research Article by PLOS Biology.

Your revisions have now been evaluated by the PLOS Biology editorial staff and I'm writing to let you know that we would like to send your submission out for external peer review.

IMPORTANT: I note that you have not supplied a marked-up "track changes" version of your manuscript. Please provide this when you upload your additional metadata (see next paragraph).

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Once your full submission is complete, your paper will undergo a series of checks in preparation for re-review. After your manuscript has passed the checks it will be sent out for review. To provide the metadata for your submission, please Login to Editorial Manager (https://www.editorialmanager.com/pbiology) within two working days, i.e. by Sep 16 2025 11:59PM.

Feel free to email us at plosbiology@plos.org if you have any queries relating to your submission.

Kind regards,

Roli

Roland Roberts, PhD

Senior Editor

PLOS Biology

rroberts@plos.org

Dear Ian,

Thank you for your patience while we considered your revised manuscript "Analysis of citation dynamics reveals that you do not receive enough recognition for your influential science" for consideration as a Meta-Research Article at PLOS Biology. Your revised study has now been evaluated by the PLOS Biology editors, the Academic Editor and the original reviewers.

You'll see that reviewer #1 is satisfied and has no further requests. Reviewer #2 says that “the empirical findings are of significant interest,” but he disagrees substantially with several aspects of your interpretation. He also thinks that you should explicitly flag that the RCR metric’s merits have been questioned.

In light of the reviews, which you will find at the end of this email, we are pleased to offer you the opportunity to address the remaining points from the reviewers in a revision that we anticipate should not take you very long. We will then assess your revised manuscript and your response to the reviewers' comments with our Academic Editor aiming to avoid further rounds of peer-review, although we might need to consult with the reviewers, depending on the nature of the revisions.

In addition to these revisions, you may need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests shortly. If you do not receive a separate email within a few days, please assume that checks have been completed, and no additional changes are required.

We expect to receive your revised manuscript within 1 month. Please email us (plosbiology@plos.org) if you have any questions or concerns, or would like to request an extension.

At this stage, your manuscript remains formally under active consideration at our journal; please notify us by email if you do not intend to submit a revision so that we withdraw the manuscript.

**IMPORTANT - SUBMITTING YOUR REVISION**

Your revisions should address the specific points made by each reviewer. Please submit the following files along with your revised manuscript:

1. A 'Response to Reviewers' file - this should detail your responses to the editorial requests, present a point-by-point response to all of the reviewers' comments, and indicate the changes made to the manuscript.

*NOTE: In your point-by-point response to the reviewers, please provide the full context of each review. Do not selectively quote paragraphs or sentences to reply to. The entire set of reviewer comments should be present in full and each specific point should be responded to individually.

You should also cite any additional relevant literature that has been published since the original submission and mention any additional citations in your response.

2. In addition to a clean copy of the manuscript, please also upload a 'track-changes' version of your manuscript that specifies the edits made. This should be uploaded as a "Revised Article with Changes Highlighted " file type.

*Resubmission Checklist*

When you are ready to resubmit your revised manuscript, please refer to this resubmission checklist: https://plos.io/Biology_Checklist

To submit a revised version of your manuscript, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' where you will find your submission record.

Please make sure to read the following important policies and guidelines while preparing your revision:

*Published Peer Review*

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:

https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/

*PLOS Data Policy*

Please note that as a condition of publication PLOS' data policy (http://journals.plos.org/plosbiology/s/data-availability) requires that you make available all data used to draw the conclusions arrived at in your manuscript. If you have not already done so, you must include any data used in your manuscript either in appropriate repositories, within the body of the manuscript, or as supporting information (N.B. this includes any numerical values that were used to generate graphs, histograms etc.). For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5

*Protocols deposition*

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for your submission to our journal. We hope that our editorial process has been constructive thus far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Roli

Roland Roberts, PhD

Senior Editor

PLOS Biology

rroberts@plos.org

----------------------------------------------------------------

REVIEWERS' COMMENTS:

Reviewer #1:

The authors have fully addressed my concerns. I congratulate them to their important work.

Reviewer #2:

[identifies himself as Ludo Waltman]

[IMPORTANT: see the fully formatted version at https://prereview.org/reviews/17335296]

My review is available online at https://prereview.org/reviews/17335296.

This is an interesting paper presenting a large-scale comparison between the use of journal-level and article-level citation indicators for assessing biomedical researchers.

While the empirical findings reported by the authors are of significant interest, I disagree with the interpretation the authors give to their findings. In their interpretation of Figure 2, the authors conclude that their results suggest “a substantial improvement in recognition for a large segment of the biomedical research workforce by including article-level indicators as a way of recognizing research”. I disagree with this interpretation because in most contexts, in particular hiring, promotion, and funding allocation, researchers essentially find themselves in a zero-sum setting. If one researcher gets more recognition (e.g., is more likely to be hired, promoted, or funded), this implies that some other researcher will get less recognition (e.g., will be less likely to be hired, promoted, or funded).

In the final subsection in the Results section, the authors recognize that researchers often find themselves in zero-sum settings. Surprisingly, however, the authors claim that even in a zero-sum setting there are “large differences between how many authors are favored using article level metrics rather than journal level metrics”. This is an odd conclusion. In a zero-sum setting, the number of authors favored using one indicator must by definition be equal to the number of authors favored using some other indicator.

This odd conclusion turns out to follow from the specific statistical approach the authors take (without explaining it in full detail) to convert raw indicator scores into percentile ranks, as illustrated in Figure 4. According to the authors, after converting raw indicator scores into percentile ranks, there are still many more researchers who receive recognition using article-level indicators than researchers who received recognition using journal-level indicators. However, this is an artefact of the statistical approach taken by the author. In the real world, the number of opportunities to be hired, to be promoted, or to be funded is fixed, so in the real world these settings are of a truly zero-sum nature. If one researcher receives more recognition and is therefore more likely to be hired, promoted, or funded, there must be some other researcher who receives less recognition and is less likely to be hired, promoted, or funded. The use of article-level indicators instead of journal-level indicators will not result in an increase in the number of researchers who can be offered a job or a promotion, or who can be awarded funding.

There are important reasons to criticize excessive reliance of journal-level indicators, but the argument presented by the authors is not convincing. The empirical insights provided by the authors are valuable, but the authors need to rethink the interpretation they give to their findings.

Finally, a minor comment I have relates to the RCR indicator used by the authors to measure the citation impact of an article. There has been some debate about this indicator. See the following paper by Janssens and colleagues: https://doi.org/10.1371/journal.pbio.2002536. And see also the following blog post by myself: https://www.cwts.nl/blog?article=n-q2u294. In my view, the authors should inform readers that there are different perspectives on the pros and cons of the RCR indicator.

Dear Ian,

Thank you for your patience while we considered your revised manuscript "Analysis of citation dynamics reveals that you do not receive enough recognition for your influential science" for publication as a Meta-Research Article at PLOS Biology. This revised version of your manuscript has been evaluated by the Academic Editor.

Based on the Academic Editor's assessment, we are likely to accept this manuscript for publication, provided you satisfactorily address the following data and other policy-related requests.

IMPORTANT - please attend to the following:

a) Please could you make your Title more explicit for readers who are coming to the topic afresh. We suggest "Analysis of citation dynamics reveals that most researchers would receive more recognition if assessed by article level metrics than by journal level metrics" or "Analysis of citation dynamics reveals that most researchers would be better served if assessed by article level metrics than by journal level metrics"

b) Please address my Data Policy requests below; specifically, we need you to supply the numerical values underlying Figs 1ABCD, 2, 3, 4, 5, 6, S1, S2, either as a supplementary data file or as a permanent DOI’d deposition. I note that you already have associated Figshare depositions, but these seem to contain largely raw data, so please also supply the values directly underlying the Figures.

c) Please cite the location of the data clearly in all relevant main and supplementary Figure legends, e.g. “The data underlying this Figure can be found in S1 Data” or “The data underlying this Figure can be found in https://figshare.com/XXXXXXXX

d) Please make any custom code available, either as a supplementary file or as part of your data deposition.

e) Please include the URLs of your funders in the Financial Disclosure statement.

As you address these items, please take this last chance to review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the cover letter that accompanies your revised manuscript.

In addition to these revisions, you may need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests shortly. If you do not receive a separate email within a few days, please assume that checks have been completed, and no additional changes are required.

We expect to receive your revised manuscript within two weeks.

To submit your revision, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' to find your submission record. Your revised submission must include the following:

- a cover letter that should detail your responses to any editorial requests, if applicable, and whether changes have been made to the reference list

- a Response to Reviewers file that provides a detailed response to the reviewers' comments (if applicable, if not applicable please do not delete your existing 'Response to Reviewers' file.)

- a track-changes file indicating any changes that you have made to the manuscript.

NOTE: If Supporting Information files are included with your article, note that these are not copyedited and will be published as they are submitted. Please ensure that these files are legible and of high quality (at least 300 dpi) in an easily accessible file format. For this reason, please be aware that any references listed in an SI file will not be indexed. For more information, see our Supporting Information guidelines:

https://journals.plos.org/plosbiology/s/supporting-information

*Published Peer Review History*

Please note that you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:

https://plos.org/published-peer-review-history/

*Press*

Should you, your institution's press office or the journal office choose to press release your paper, please ensure you have opted out of Early Article Posting on the submission form. We ask that you notify us as soon as possible if you or your institution is planning to press release the article.

*Protocols deposition*

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please do not hesitate to contact me should you have any questions.

Sincerely,

Roli

Roland Roberts, PhD

Senior Editor

rroberts@plos.org

PLOS Biology

------------------------------------------------------------------------

DATA POLICY:

You may be aware of the PLOS Data Policy, which requires that all data be made available without restriction: http://journals.plos.org/plosbiology/s/data-availability. For more information, please also see this editorial: http://dx.doi.org/10.1371/journal.pbio.1001797

Note that we do not require all raw data. Rather, we ask that all individual quantitative observations that underlie the data summarized in the figures and results of your paper be made available in one of the following forms:

1) Supplementary files (e.g., excel). Please ensure that all data files are uploaded as 'Supporting Information' and are invariably referred to (in the manuscript, figure legends, and the Description field when uploading your files) using the following format verbatim: S1 Data, S2 Data, etc. Multiple panels of a single or even several figures can be included as multiple sheets in one excel file that is saved using exactly the following convention: S1_Data.xlsx (using an underscore).

2) Deposition in a publicly available repository. Please also provide the accession code or a reviewer link so that we may view your data before publication.

Regardless of the method selected, please ensure that you provide the individual numerical values that underlie the summary data displayed in the following figure panels as they are essential for readers to assess your analysis and to reproduce it: Figs 1ABCD, 2, 3, 4, 5, 6, S1, S2. NOTE: the numerical data provided should include all replicates AND the way in which the plotted mean and errors were derived (it should not present only the mean/average values).

IMPORTANT: Please also ensure that figure legends in your manuscript include information on where the underlying data can be found, and ensure your supplemental data file/s has a legend.

Please ensure that your Data Statement in the submission system accurately describes where your data can be found.

------------------------------------------------------------------------

CODE POLICY

Per journal policy, if you have generated any custom code during the course of this investigation, please make it available without restrictions. Please ensure that the code is sufficiently well documented and reusable, and that your Data Statement in the Editorial Manager submission system accurately describes where your code can be found.

Please note that we cannot accept sole deposition of code in GitHub, as this could be changed after publication. However, you can archive this version of your publicly available GitHub code to Zenodo. Once you do this, it will generate a DOI number, which you will need to provide in the Data Accessibility Statement (you are welcome to also provide the GitHub access information). See the process for doing this here: https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content

------------------------------------------------------------------------

DATA NOT SHOWN?

- Please note that per journal policy, we do not allow the mention of "data not shown", "personal communication", "manuscript in preparation" or other references to data that is not publicly available or contained within this manuscript. Please either remove mention of these data or provide figures presenting the results and the data underlying the figure(s).

------------------------------------------------------------------------

Dear Ian,

Thank you for the submission of your revised Meta-Research Article "An analysis of citation dynamics reveals that most researchers would receive more recognition if assessed by article level metrics than by journal level metrics" for publication in PLOS Biology. On behalf of my colleagues and the Academic Editor, Malcolm Macleod, I'm pleased to say that we can in principle accept your manuscript for publication, provided you address any remaining formatting and reporting issues. These will be detailed in an email you should receive within 2-3 business days from our colleagues in the journal operations team; no action is required from you until then. Please note that we will not be able to formally accept your manuscript and schedule it for publication until you have completed any requested changes.

IMPORTANT: You'll see I've had to flip your Title so that "an analysis of citation dynamics" occurs at the beginning. This is because we try to avoid punctuation (and certainly colons) in Titles. Sorry not to have mentioned this earlier, and hope this is OK.

Please take a minute to log into Editorial Manager at http://www.editorialmanager.com/pbiology/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS: We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with biologypress@plos.org. If you have previously opted in to the early version process, we ask that you notify us immediately of any press plans so that we may opt out on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

Thank you again for choosing PLOS Biology for publication and supporting Open Access publishing. We look forward to publishing your study. 

Sincerely, 

Roli

Roland G Roberts, PhD, PhD

Senior Editor

PLOS Biology

rroberts@plos.org