Reader Comments

Post a new comment on this article

On Application Length

Posted by mtaffe on 24 Jul 2008 at 00:41 GMT

"Obviously, the length of the application impacts the number of reviewers that could possibly be used for scoring. More reviewers can be used for shorter applications."

Given that the authors are clearly interested in testing hypotheses regarding the review of NIH grants it is curious that they chose to include such an unsupported comment. It is far from 'obvious' to anyone who is currently involved in reviewing NIH proposals that shortening the application will necessarily reduce the time required to review a proposal.

This may be a common trope of the Review-of-Peer-Review effort but it would be interested to see it supported by experimental studies, such as this one, or at the very least by a decent sampling of opinions from currently active reviewers.

RE: On Application Length

ike9898 replied to mtaffe on 29 Jul 2008 at 21:28 GMT

Are you suggesting that it may not take less time to review a 10 page application that it takes to review a 50 page application? In my experience, a major portion of the time it takes to review proposal involves giving the proposal a careful reading. I don't think the authors are going too far out on a limb with their statement.

RE: On Application Length

dakaplan replied to mtaffe on 01 Aug 2008 at 14:44 GMT

We agree that it is important to collect data on this essential element of peer review. We collected this information for our peer review model system but we haven't seen that NIH has made a similar analysis. In my experience reviewing an application of 15-25 pages is quite difficult and incredibly time-consuming. Also, writing a review adds significantly to the effort required. At the minimum 5 hours is needed. Reviewing a 1-page proposal (as I have previously proposed) to provide a low-precision score without any report has taken me 10-30 minutes. So we still believe that our assumption about the amount of time is correct but agree that the data should be collected.