Introduction

Research is a core component of the mission of U.S. academic medical centers (AMCs), but its viability in the 21^st century has been threatened by a progressive decline in extramural funding. In inflation-adjusted dollars, appropriations from the National Institutes of Health have stagnated since peaking in 2003, and over the same period, the success rate for R01-equivalent awards has declined by about a third [1]. These trends have made it difficult for physician-scientists [2] and other AMC researchers to initiate and advance their careers.

Partly in response to this burdensome extramural funding environment, many AMCs provide intramural research funding to schools, departments, and individual investigators. A recent study reported that for every dollar of external research sponsorship, AMCs spent about 50 cents of internal funds on research. Although almost 70 percent of this AMC investment went toward facilities and administrative subsidies and salary support, about 10 percent was provided directly to investigators in the form of bridge funding or university (intramural) research grants [3].

Beyond deciding whether to provide internal research funds to investigators and how much to spend, however, AMCs must decide how to deploy these supplemental resources. AMCs often design and implement new programs based on a qualitative assessment of the institution’s overall goals, the likely impact of the various options under consideration, and the amount of resources available. But when budgets come up for renewal, program administrators typically face pressure to quantitatively gauge these programs’ effectiveness in order to justify to senior executives which ones should be maintained, expanded, or terminated. The rigor of this assessment is often limited by the quality of the underlying data, small sample sizes, and inherent difficulties in defining relevant outcomes that can be measured on relatively short time scales.

In this study, we report the results of a descriptive, retrospective analysis of internal research funds disbursed by a dedicated center within a large U.S. AMC. In 2005, Brigham and Women’s Hospital (BWH), a top-20 recipient of NIH funding among academic medical centers, launched the Brigham Research Institute (BRI). The BRI’s mandate includes increasing the visibility of research at BWH, bolstering communication within the research community, catalyzing internal and external collaborations, and aiding in fundraising. Starting in 2007, the BRI began directing a substantial portion of its resources toward providing supplemental research funding to internal investigators, using funds from its largely philanthropically supported operating budget. This decision was made partly in response to administrators’ view that the challenging extramural funding environment was restricting the research careers of BWH researchers at all levels, from trainees to senior investigators. Supplemental research funding was also seen as an opportunity to further support the BRI’s main goals according to its mandate (see above). Under the guidance of the institute’s oversight and executive committees, the BRI opted to fund a broad variety of programs, targeting a diverse range of research endeavors and constituencies, instead of a narrowly defined set of grant types, sizes, and/or applicant ranks.

We undertook the study reported here to assess two aspects of the BRI’s portfolio of intramural grants. First, what are the demographics of grant applicants and recipients, and to what extent do they highlight areas where further attention and activity may be warranted? And second, how have application numbers changed over time, and what does this tell us about the demand for intramural funding opportunities? With some caveats (discussed below), both aims were achieved.

Materials and methods

Results

From 2010 to 2017, the BRI awarded a total of 354 grants totaling over $9 million through six intramural funding mechanisms, supported by institutional resources and generous contributions from private donors. These six funding programs reflect an intentional blend of award size and focus (Table 1). The Fund to Sustain Research Excellence (FSRE) provides fairly conventional small bridge grants, intended to cover gaps in research dollars in the window between missed R01 paylines and approval in subsequent NIH grant cycles. Initiated to tackle head-on the challenge of declining R01 funding rates, the FSRE has maintained steady popularity since its inception. Center-specific research grants are offerings (variable in award amount, frequency, and specific goals) by the BRI’s individual centers, each of which is focused on a specific research domain or disease area. These grants aim to seed collaborations within their respective communities or provide pilot funding for special priorities identified within them. The “Shark Tank” program is similar in process to other institutions’ efforts to support “bench to bedside” investigations [4], and further bolsters the BRI’s mandate to increase donor engagement by involving them in determining the final winners. Like the Shark Tank, the BRIght Futures prizes target innovative science with practical ramifications that may fall outside of what is traditionally funded by the NIH. BRIght Futures also serves the BRI’s mission of increasing research visibility, because the final recipient each year is decided by an online vote, open to the lay public worldwide [5]. Finally, the BRI has two programs that target extreme ends of the funding spectrum: microgrants support low-cost endeavors, mainly related to professional development and the seeding of new research directions; whereas the Directors’ Transformative Awards are large grants to ground-breaking collaborative endeavors. Although each program has specific application requirements, most are open to faculty and staff at all levels, with more limited eligibility for postdoctoral researchers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Applicants, awards, and funds awarded through BRI funding programs, 2010–2017.

https://doi.org/10.1371/journal.pone.0241425.t001

Gender and academic rank of BRI grant applicants and awardees

We analyzed the characteristics of applicants and awardees across all of the funding mechanisms except for microgrants, for whom we lacked the associated full demographic information. Over the eight-year analysis period, these programs received 822 applications from 483 unique individuals. Of the applicants, 36% (173/483) applied for more than one grant in the same year and/or different years (range among the 173 “multi-appliers,” 2–11; median, 3; interquartile range, 2–3). “Multi-appliers” were more likely to receive funding than individuals who applied just once (44% vs. 18%, p<0.00001 by Chi-squared test). Compared with one-time applicants, “multi-appliers” were equally likely to be female (66% vs. 64%, p = 0.57 by Chi-squared test), but more likely to be at the instructor or assistant professor level at the time of their first award (92% vs. 68%, p<0.00001 by Chi-squared test).

Over the same period, we awarded 160 grants via these programs to 133 unique individuals. Among the awardees, 83% (110/133) received just one BRI grant, 15% (20/133) received two awards, and the remainder (3/133) received 3–4 grants apiece. Due to the small number of recipients who received more than one award, we elected not to perform statistical tests to compare these individuals to the single-grant awardees.

We performed three analyses to explore the gender and academic rank distributions of applicants and awardees, using a standard two-sided p value cutoff of 0.05 to define statistical significance. First, we compared the characteristics of individuals who submitted applications to those who received awards over the entire eight-year period. There was not a statistically significant difference in the fraction identifying as female between these two groups (285/822 applicants (35%) vs. 53/160 awardees (33%); p = 0.71 by Chi-squared test). Similarly, when we analyzed by faculty rank, excluding research fellows and other non-faculty titles, there was no difference in the percent of faculty at the more junior levels of instructor and assistant professor between the two groups (561/808 applicants (69%) vs. 104/160 awardees (65%); p = 0.27 by Chi-squared test).

Second, we compared the genders and academic ranks of applicants versus awardees in each individual year (Tables 2 and 3). In 2016, we observed a statistically significant difference between applicants and awardees in the percent of individuals identifying as female (23/92 (25%) vs. 8/16 (50%); p = 0.041 by Chi-squared test). No other differences between applicants and awardees with regard to academic rank or gender in individual years were statistically significant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of fraction of females among BRI grant applicants, BRI grant awardees, and U.S. medical school faculty by year, 2010–2017.

https://doi.org/10.1371/journal.pone.0241425.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of proportion of BRI grant applicants, BRI grant awardees, and U.S. medical school faculty at junior academic rank by year, 2010–2017.

https://doi.org/10.1371/journal.pone.0241425.t003

Third, we compared applicants and awardees by year to full-time faculty members of U.S. medical schools, as reported by the American Association of Medical Colleges (see Methods). Compared with the U.S. faculty, the BRI applicant pool had a smaller percentage of females in three of the eight years, and a larger fraction of individuals at junior ranks in four of the years. Among BRI awardees, females were under-represented compared with the national faculty in 2012. All other gender and academic rank comparisons with U.S. medical school faculty were not statistically significant.

Trends in grant applications over time

We examined trends in application numbers by year to the BRI’s funding programs using Spearman’s rank order calculations (Table 4). We excluded center-specific research grants, which had highly variable program execution and design across years that led to vast differences in application numbers. The remaining grant programs accounted for 88% (970/1,106) of the applications received from 2010 to 2017. Our analysis failed to detect a statistically significant trend in applications over time to all programs taken together (r_s = 0.40, p = 0.40) or to any of the programs individually.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Spearman rank-order correlations for BRI funding programs over time.

https://doi.org/10.1371/journal.pone.0241425.t004

Discussion

In this study, we analyzed the number and type of applicants and awardees across several distinct research funding mechanisms at the BRI, a provider of substantial intramural support to investigators at a major AMC. Our goals were to assess (1) the gender and academic rank distribution of grant applicants and awardees in comparison to each other and benchmark data on U.S. medical school faculty, and (2) the growth in applications over time as an indicator of demand for these sorts of funding programs.

With regard to demographics, we did not detect a significant bias by gender or rank when comparing awardees to applicants. Our applicant pool did skew slightly less female and more junior than U.S. medical center faculty overall in several years, but most of these differences were not present when we compared awardees to national faculty overall. With regard to applications over time, we did not observe a statistically significant change in the number of applications over time. Of note, all of these results should be considered exploratory and hypothesis generating only, as they were neither pre-specified nor adjusted for multiple hypothesis testing.

Although the demographic data presented here suggest that the BRI’s grant awards are neither intentionally nor unintentionally biased with regard to female gender, the slight under-representation of women in the applicant pool in several years compared with the national sample is notable and worthy of further investigation. Structural bias is known to impede the career development of female medical faculty, and although mentorship programs and other activities have been somewhat successful in combating it, women remain under-represented in academic medicine, particularly at senior ranks [6, 7]. Although some evidence suggests that grant funding may not be a critical component of the broader gender imbalance [8], data are scant in this area. Furthermore, aside from a recent initiative sponsored by Australia’s National Health and Medical Research Council, there have been few efforts to explicitly promote gender equity in academic medicine with funding programs that specifically target female faculty [9, 10]. Intramural funding groups like the BRI could explore similar programs to boost faculty diversity.

Our analysis of the number of applications over time failed to demonstrate an increase in demand for the BRI’s funding programs. This was somewhat surprising, given that over the same period the BRI increased the diversity of funding mechanisms and engaged in a significant degree of outreach and awareness-building, and the number of faculty grew over this time as well. These results several possibilities that merit investigation: demand may be adequately met by the current intramural grant offerings, and/or outreach efforts to date may have been ineffective at boosting awareness of these funding opportunities, and/or investigators may not find the programs appealing based on the size of the awards, the odds of receiving them, and the effort and time required to apply. In the meantime, our analysis does not currently support the need to increase the breadth, diversity, or number of awards offered by the BRI in order to satisfy demand.

This study illuminates two further areas related to intramural research funding that warrant additional study. First, it is not clear whether grant programs like the BRI’s can and should do more to support broader diversity goals in academic medicine with regard to race and ethnicity. The leadership of BWH is philosophically committed to promoting racial and ethnic diversity within the research community, which has long been known to be lacking [11–13]. In 1989, NIH began offering research supplements to minority investigators as part of a broader suite of initiatives to combat systemic under-representation of particular groups [14]. However, we have been unable to identify any systematic assessment of the impact on diversity of NIH’s minority faculty funding program, and a recent longitudinal study suggested that across most specialties, representation of blacks and Hispanics has actually decreased over the past two decades [15]. To date the BRI has not explicitly focused its funding programs on furthering diversity, and because the BRI has not systematically collected race and ethnicity data from applicants and awardees, we were unable to assess these factors in the current analysis. However, we plan to obtain and analyze these data going forward as we assess how we may be able to deploy intramural funding in a way that promotes faculty diversity.

Second, this work implicitly highlights challenges of evaluating the impact of intramural funding programs in academic medicine. Although the BRI is not required to achieve a specific performance threshold to continue its programs, the question of return on investment (ROI) is top of mind for our leadership team, executives, and potential donors. We have explored applying ROI analysis methods similar to those used by other institutions to assess their intramural career development programs [16–18], but we have been hindered by the low quantity and quality of data we have been able to obtain from awardees, as well as significant outliers that make statistical analyses difficult. We are currently re-evaluating how we track outcomes after grant awards in order to enable more rigorous studies in the future of the impact of these awards on relevant metrics of career success and development.