Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Association of publication record and independent NIH funding

Abstract

Background

Publications may be a modifiable factor toward research project grant (RPG) funding decisions, the objective was to determine the association of publication record with later RPG receipt.

Methods

This was a retrospective cohort study of recipients of K01, K08, or K23 US career development awards (CDAs) starting from 2000–2015. Exposures were CDA awardees’ first-, middle-, and last-author publication counts, and the quartile of awardees’ highest and mean publication impact factors. The independent association of each exposure with time to RPG (R01 or equivalent) was determined using a Cox model, after adjustment for CDA type, awardee change in institution, and institutional CDA count. The proportion of CDA recipients with later independent funding was also determined by publication count.

Results

Among 6744 CDA awardees, 3943 obtained an RPG. The median time to RPG was 5.6 years (interquartile range 4.2–7.5). The number of first-authorships was associated with a shorter time to RPG (1–4 versus 0: hazard ratio [HR] 1.22, 95% confidence interval [CI] 1.10–1.36; 5–9: 1.59, 95% CI 1.40–1.79; 10–24: 1.78, 95% CI 1.54–2.07; 25+: 2.40, 95% CI 1.61–3.56). Last-authorships were associated with a shorter time to RPG (1–4 versus 0: HR 1.99, 95% CI 1.83–2.16; 5–9: 2.72, 95% CI 2.45–3.03; 10–24: 3.17, 95% CI 2.78–3.62; 25+: 3.12, 95% CI 2.17–4.50). Higher maximum impact factor was associated with a shorter time to RPG (Q2 versus lowest: HR 1.28, 95% CI 1.12–1.46; Q3: 1.45, 95% CI 1.24–1.70; Q4: 1.67, 95% CI 1.39–2.02). Mean impact factor and middle-authorships were not associated with time to RPG. Among 687 CDAs with zero associated first- or last-authorships, 158 (23%) achieved later RPG funding. Among those with at least 10 total first- or last-authorships, 1288/1554 (83%) obtained a later RPG.

Conclusions

A higher number and impact of publications was associated with later independent funding.

Introduction

US career development awards (CDAs) are a key step toward independent research careers [1]. These awards provide substantial project and salary support to junior investigators, and provide a pathway to develop the skills and experience to lead large projects. Independent research project grants (RPGs) and individual large project grants (R01 awards) are one of the premier means of supporting biomedical research careers, and are therefore a key outcome of CDAs [2]. These large grants sustain researchers and are a critical source of biomedical innovation in the US [3].

Most CDA recipients will go on to obtain independent research funding, but substantial sex [46] and race [79] disparities in this progression exist. Medical school affiliation, institutional funding track record, citations, and caregiving responsibilities partially mediate these disparities [7, 10]. Additionally, attracting and retaining a younger and more diverse research workforce had become increasingly difficult, as older researchers obtain a larger share of funding and as the total amount of potential funding has grown more slowly than the research workforce [10, 11].

A better understanding of the modifiable characteristics of CDA recipients, such as publication record, that predict later RPG or R01 funding could help provide a blueprint for how CDA recipients might progress to independence.

Materials and methods

I studied US federal CDAs (K01, K08, or K23) with start dates from 2000–2015, excluding any awardees with a prior RPG. The data source was the NIH RePORTER database, 2000–2020.

The main outcome was the receipt of US federal independent research funding, analyzed as either an R01 award or RPG [1] (DP1, DP2, DP3, DP4, DP5, P01, P42, PN1, PM1, R00, R01, R03, R15, R21, R22, R23, R29, R33, R34, R35, R36, R37, R50, R55, R56, R61, RC1, RC2, RC3, RC4, RF1, RL1, RL2, RL9, RM1, U01, U19, U34, UA5, UC1, UC2, UC3, UC4, UC7, UF1, UG3, UH2, UH3, UH5, UM1, or UM2).

Potential predictors of research funding included CDA type, funder Institute and Center (IC) for ICs with at least 100 CDAs during the study, whether the CDA required resubmission to achieve funding, change of awardee institution during the CDA period, number of CDA-associated first-author papers, number of CDA-associated middle-author papers, number of CDA-associated last-author papers, highest impact factor of the CDA-associated papers’ journals, mean impact factor, awardee institution located in the US, and whether the awardee institution was in the top 20, next 30, or outside the top 50 institutions in number of CDAs. Journal impact factors were drawn from the Web of Science 2019 Journal Citation Reports [12].

I determined the association of each predictor with time to independent research funding. To make this comparison, I constructed Kaplan-Meier survival curves for each type of independent research funding stratified on predictor variables and tested each association using a log rank test. All researchers who did not achieve R01 or RPG funding were censored on December 31, 2020. The independent association of each predictor with later funding was evaluated using a Cox proportional hazard models including all predictors except for funder IC (because there were too many ICs to feasibly include). Finally, to examine the role of publishing and of ICs, the proportion of CDA recipients who went on to independent funding was determined by number of first- and last-authorships, and separately by funder IC.

Results

I analyzed 6,744 CDAs after excluding 293 (4.2%) because the awardees had a previous RPG. Demographic features of the cohort are displayed in Table 1. Most CDAs were either 3–4 years (23.2%) or 5 years (63.3%) in duration.

thumbnail
Table 1. Demographic features of career development awards (CDAs) and their recipients between 2000–2015.

https://doi.org/10.1371/journal.pone.0269283.t001

R01s were obtained after 3,150 (46.7%) CDAs and RPGs after 3,943 (58.5%). The median time to R01 among awardees was 5.6 years (interquartile range [IQR] 4.2–7.5). Among RPG awardees it was 5.0 years (IQR 3.5–6.7). The median time to R01 was 0.9 years longer among 5-year compared with 3-4-year CDA awardees. The median time to RPG was 0.8 years longer among 5-year compared with 3-4-year CDA awardees.

Time to R01 and time to RPG were each shorter among CDA awardees who changed institution during their CDA, those with higher impact papers, more first, last, or middle authorships associated with the CDA, and among CDAs awarded to top 20 institutions (p<0.0001 for each comparison, Fig 1). There was no association between CDA type (K01, K08, or K23) and time to R01 or RPG.

thumbnail
Fig 1.

Kaplan-Meier curves depicting time to (A) R01 and (B) RPG award stratified on several characteristics of career development awards.

https://doi.org/10.1371/journal.pone.0269283.g001

The multivariable Cox regression model demonstrated that each predictor except middle authorships were independently associated with time to R01 or RPG (Table 2). For both R01s and RPGs, predictors independently associated with a longer time to independent award were K23, no change of institution during the CDA, lower maximum impact factor of CDA-associated papers, lower mean impact factor of CDA-associated papers, fewer first authorships, fewer last authorships, and affiliation with an institution outside the top 20 CDA-awarded institutions. The strongest predictors were the numbers of first and last authorships.

thumbnail
Table 2. Factors associated with time to R01 or RPG acquisition after a career development award (CDA).

Hazard ratios (HR) and 95% confidence intervals (CI) were determined using a multivariable Cox proportional hazards approach.

https://doi.org/10.1371/journal.pone.0269283.t002

More CDA-associated first or last authorships substantially improved the proportion of applicants who obtained later R01 or RPG funding (Fig 2). Among those with no first or last authorships, 115/687 (17%) obtained a later R01 and 158/687 (23%) obtained a later RPG. By contrast, of those who had at least 10 total first or last authorships, 1108/1554 (71%) obtained a later R01 and 1288/1554 (83%) obtained a later RPG.

thumbnail
Fig 2. Number and proportion of career development awardees who ultimately received R01 (left) or RPG (right) funding, depending on the number of first or last author papers associated with the career development award.

https://doi.org/10.1371/journal.pone.0269283.g002

ICs varied in later funding for CDA recipients (Fig 3). The National Institute on Aging had the highest proportion of later R01s (195/345, 57%) and RPGs (231/345, 67%). The National Institute of Nursing Research had the fewest later R01s (31/98, 32%) and the National Center for Research Resources had the fewest later RPGs (91/219, 42%).

thumbnail
Fig 3. Proportion of career development award (CDA) recipients who received later R01 (grey) or RPG (black) funding, depending on the Institute or Center that funded the CDA.

https://doi.org/10.1371/journal.pone.0269283.g003

Discussion

Among 6,744 CDA recipients from 2000–2015, approximately half later obtained US research project funding. The number and impact of publications was a very strong predictor of success in later independent funding: approximately one fifth of those without first or last authorships go on to later funding, whereas approximately four fifths with at least 10 first or last authorships go on to later funding.

First and last authorships measure different phenomena [13]. First authorships represent work the author leads, and last authorships represent work the author supervises. Middle authorship frequently reflects contributions to a larger team. Middle authorships were not independently predictive of independent research funding success. Taken together, these findings suggest that CDA recipients with a high number of publications reflecting leadership, mentorship, or both will go on to obtain independent funding. These findings could differ in other nations where funding decisions might rely on other criteria.

This study could not evaluate the reasons that more publications are associated with a shorter time to independent research funding. However, regardless of the actual causes, the implications are clear and reflect the adage “publish or perish.” CDA recipients with few publications are unlikely to obtain US federal funding, while those with many are highly likely to do so. As a result, known disparities in early research experience, topic choice, and first- and last-authorships by race [14, 15] and gender [16] may, at least in part, mediate funding disparities [8] by those characteristics.

Whether a grant applicant’s number of publications is a useful barometer of future research impact is unclear. Highly influential scientists may not be funded when publication count is a key metric [17]. Alternative approaches include funding all applicants, funding applicants randomly, automated approaches, rewarding citizenship, or alternative rubrics for evaluation of promising science [18]. However, any single approach has positive and negative features. Absent major changes in the approach to evaluating research grant applications, authorship is likely to continue to be a meaningful contributor to application success. Institutions and leaders seeking to improve local funding rates and narrow disparities in funding should therefore encourage, foster, and monitor leadership and supervision among junior researchers, particularly those funded by CDAs.

Our study has several limitations. First, our study could not account for changes in funding rates and priorities over the 16 years I analyzed. Second, I could not account for CDA recipients who left the workforce prior to the end of the study period, who would therefore have been ineligible for the outcome of RPG funding. I believe workforce departures would be uncommon since most CDA awardees are in their early career. Finally, I could not directly measure the degree of mediation of publications with disparities, as I did not have access to CDA recipient demographics.

Conclusions

Among CDA awardees from 2000–2015, first- and last-authorship and impactful publications were strong independent predictors of a shorter time to R01 or RPG funding. Fostering local research productivity could improve independent grant funding success rates among junior researchers. The effect of improving publication counts on disparities in research funding should be undertaken.

References

  1. 1. Nikaj S, Lund PK. The impact of individual mentored career development (k) awards on the research trajectories of early-career scientists. Acad Med 2019;94:708–14. pmid:30520806
  2. 2. Conte ML, Omary MB. NIH Career Development Awards: Conversion to research grants and regional distribution. J Clin Invest 2018;128:5187–90. pmid:30371504
  3. 3. Li D, Agha L. Big names or big ideas: Do peer-review panels select the best science proposals? Science (80-) 2015;348:434–8.
  4. 4. Jagsi R. Sex Differences in Attainment of Independent Funding by Career Development Awardees. Ann Intern Med 2009;151:804. pmid:19949146
  5. 5. Pohlhaus JR, Jiang H, Wagner RM, Schaffer WT, Pinn VW. Sex differences in application, success, and funding rates for NIH extramural programs. Acad Med 2011;86:759–67. pmid:21512358
  6. 6. Sege R, Nykiel-Bub L, Selk S. Sex differences in institutional support for junior biomedical researchers. JAMA—J Am Med Assoc 2015;314:1175–7. pmid:26372589
  7. 7. Ginther DK, Schaffer WT, Schnell J, Masimore B, Liu F, Haak LL, et al. Race, Ethnicity, and NIH Research Awards. Science (80-) 2011;333:1015–9. pmid:21852498
  8. 8. Ginther DK, Haak LL, Schaffer WT, Kington R. Are Race, Ethnicity, and Medical School Affiliation Associated With NIH R01 Type 1 Award Probability for Physician Investigators? Acad Med 2012;87:1516–24. pmid:23018334
  9. 9. Good M, McElroy SJ, Berger JN, Wynn JL. Name and Characteristics of National Institutes of Health R01-Funded Pediatric Physician-Scientists. JAMA Pediatr 2018;172:297. pmid:29340570
  10. 10. Hartmann KE, Sundermann AC, Helton R, Bird H, Wood A. The scope of extraprofessional caregiving challenges among early career faculty: Findings from a university medical center. Acad Med 2018;93:1707–12. pmid:29596083
  11. 11. Antman MD, Gorelik R, Kennedy A, Liou GF, Billingslea EN, Corrigan JG, et al. Changes in the National Cancer Institute’s R01 workforce: growth, aging, retention, and policy implications. J Clin Invest 2021;131. https://doi.org/10.1172/JCI146925.
  12. 12. Clarivate Analytics. 2019 Journal Impact Factors. J Cit Reports n.d.
  13. 13. Baerlocher MO, Newton M, Gautam T, Tomlinson G, Detsky AS. The Meaning of Author Order in Medical Research. J Investig Med 2007;55:174–80. pmid:17651671
  14. 14. Jeffe DB, Yan Y, Andriole DA. Do Research Activities During College, Medical School, and Residency Mediate Racial/Ethnic Disparities in Full-Time Faculty Appointments at U.S. Medical Schools? Acad Med 2012;87:1582–93. pmid:23018339
  15. 15. Hoppe TA, Litovitz A, Willis KA, Meseroll RA, Perkins MJ, Hutchins BI, et al. Topic choice contributes to the lower rate of NIH awards to African-American/black scientists. Sci Adv 2019;5. https://doi.org/10.1126/sciadv.aaw7238.
  16. 16. Bendels MHK, Müller R, Brueggmann D, Groneberg DA. Gender disparities in high-quality research revealed by Nature Index journals. PLoS One 2018;13:e0189136. pmid:29293499
  17. 17. Nicholson JM, Ioannidis JPA. Research grants: Conform and be funded. Nature 2012;492:34–6. pmid:23222591
  18. 18. Ioannidis JPA. Fund people not projects. Nature 2011;477:529–31.