Introduction

In 2000, the Canadian Medical Research Council and the National Health Research Grant Program were merged to form the Canadian Institutes of Health Research (CIHR), comprising 13 institutes [1–3]. Each institute focuses on a specific scientific healthcare domain (e.g., cancer) and includes biomedical, clinical, health services, and policy research, as well as research addressing the health of populations, including societal, cultural, and environmental determinants of health [4]. Under the guidance of the governing council, scientific directors, and institute advisory boards, each institute forges a health research agenda that reflects the health needs of Canadians, the evolution of the Canadian healthcare system, gaps in science, and the information needs of policy decision makers [5].

In the previous funding scheme at the CIHR, research funding was offered through the Open Operating Grant Program (OOGP) and as strategic funding to conduct research in priority areas identified by institute strategic plans in an approximate 70:30 ratio [6]. Through a well-established peer-review process conducted by various committees with content and methodologic expertise, funding was allocated to address gaps in knowledge [1–3,7,8]. To cultivate the development of scientists, peer-review committees at the CIHR also reviewed and funded New Investigator personnel award applications [9].

Several studies provide evidence that women are less likely than men to receive research funding. A study of grant applications submitted to the CIHR open and strategic grant competitions between 2001 and 2011 by scientists who directed their research to the Institute of Health Services and Policy Research found that men under 45 years old were significantly more likely to be funded than age-matched women (odds ratio [OR] 1.40, 95% confidence interval [CI] 1.01–1.95) [4]. Gender differences in success rates have persisted, with female mid- and senior-career researchers being less likely to secure Foundation Scheme (7-year research program funding) in the second pilot competition of the new CIHR funding scheme [10]. Similarly, studies evaluating grant success at the National Institutes of Health (NIH) have shown significant gender disparity in funding success, favoring men in the rate of receipt of initial Research Project Grants (RO1) awards [11], number of career RO1 awards [12], and in productivity as measured by publications and self-perceived success [13]. Although women at all career stages are more likely to leave academia than men, women who are postdoctoral trainees and early in their careers are recognized to be high-risk groups [14–16].

Although institutes do not directly review applications at the CIHR, they reflect scientific communities. Scientists with expertise in content areas are affiliated with particular institutes and participate in peer-review activities related to their context knowledge and expertise. Investigators have not previously addressed the extent to which gender differences have existed in peer-reviewed funding across research communities reflecting research content areas. To examine whether gender differences existed overall in grant and personnel award funding success rates and whether gender differences in success rates differed across research communities, we summarized OOGP grant and personnel award applications approved for funding at the CIHR over a 15-year period in the previous funding scheme.

Methods

Results

Grant applications

Table 1 presents a summary of the number of OOGP applications received, the number approved for funding, and success rates according to institute. The largest number of applications (on average more than 300 grant applications per year) were directed by scientists to the Institutes of Cancer Research, Circulatory and Respiratory Health, Genetics, Infection and Immunity, Neurosciences/Mental Health and Addiction, and Nutrition/Metabolism and Diabetes. Average success rates were lowest among scientists who directed applications to smaller-volume institutes including the Institutes of Aboriginal Peoples’ Health (17.5%), Gender and Health (18.1%), and Population and Public Health (18.6%) and highest among scientists who directed applications to higher-volume institutes such as Genetics (28.7%), Infection and Immunity (25.5%), and Neurosciences/Mental Health and Addiction (24.3%). There was considerable variation in application volume and success rates over time and across designated institutes (Table 1). Success rates in the OOGP decreased over time across institutes except for the Gender and Health institute, which increased after being launched in 2000/2001 and remained relatively stable over time (Figs 1 and S1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Combined grant success rates by year across CIHR institutes.

CIHR, Canadian Institutes of Health Research; LOESS, locally estimated scatterplot smoothing; SD, standard deviation.

https://doi.org/10.1371/journal.pmed.1002935.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Number of grant applications received, number approved for funding, and success rate by institute.

https://doi.org/10.1371/journal.pmed.1002935.t001

Sex differences in OOGP success

Fig 2 shows the total number of OOGP applications submitted. Although the number of grant applications submitted by men showed greater variation over time, the number of grant applications submitted by women remained consistently low. Compared with men, however, women submitted more grant applications to 3 institutes: Aboriginal Peoples’ Health (179 versus 245), Gender and Health (533 versus 737), and Health Services and Policy Research (1,496 versus 1,591).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Total number of grant applications submitted by institute and gender.

https://doi.org/10.1371/journal.pmed.1002935.g002

In the pooled estimate across institutes, women had significantly lower grant success (RR 0.89, 95% CI 0.84–0.94; p < 0.001; absolute difference 3.2%) compared with men, with substantial heterogeneity (I² = 58%). Women who directed applications to the Institutes of Cancer Research (RR 0.86, 95% CI 0.78–0.96), Circulatory and Respiratory Health (RR 0.74, 95% CI 0.66–0.84), Health Services and Policy Research (RR 0.78, 95% CI 0.68–0.90), and Musculoskeletal Health and Arthritis (RR 0.80, 95% CI 0.69–0.93) were significantly less likely to be funded than men. Conversely, women who directed applications to the Institute of Aboriginal Peoples’ Health (RR 1.67, 95% CI 1.0–2.7) were significantly more likely to be funded than men (Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Forest plot of gender differences in grant success rates by designated research institute.

CI, confidence interval; M-H, Mantel Haenszel.

https://doi.org/10.1371/journal.pmed.1002935.g003

Sex differences in new investigator award success

Compared with men, women directed more New Investigator Award applications to 6 institutes (Aboriginal Peoples’ Health; Aging; Gender and Health; Health Services and Policy Research; Human Development, Child, and Youth; and Population and Public Health). In the pooled estimate, women had significantly lower New Investigator Award success (RR 0.75, 95% CI 0.65–0.86; p < 0.001; absolute difference 6.6%) compared with men, with very little heterogeneity (I² = 0%) (Fig 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Forest plot of gender differences in new investigator success rates by designated research institute.

CI, confidence interval; M-H, Mantel Haenszel.

https://doi.org/10.1371/journal.pmed.1002935.g004

Discussion

Over 15 years of funding at the CIHR, we noted variable success rates among grant applications directed to various institutes and declining success rates over time. Overall, women had significantly lower grant success (RR 0.89, 95% CI 0.84–0.94; p < 0.001; absolute difference 3.2%) compared with men. Across institutes, we found substantial heterogeneity, with variability demonstrated both in the primary analysis for grant applications and in the Poisson regression. Women who directed grants to 4 institutes (Cancer Research, Circulatory and Respiratory Health, Health Services and Policy Research, and Musculoskeletal Health and Arthritis) were significantly less likely to be funded than men. However, women who directed grants to the Institute of Aboriginal Peoples’ Health were more likely to be funded than men. In pooled estimates, women also had significantly lower personnel award success (RR 0.75, 95% CI 0.65–0.86; p < 0.001; absolute difference 6.6%) compared with men, with no heterogeneity. Poisson regression models supported that the effect of gender on grant success rates differed by designated institute and not time.

Our review has several strengths. It is novel in assessing the extent to which gender differences in peer-reviewed funding exist across research communities, represented by CIHR institutes, over a 15-year period in the former funding scheme. We summarized data using both relative and absolute effect measures [21,22] and report our findings using recommendations for cohort studies (S1 Strobe checklist). We identified significant differences in the gradients of grant success in specific research content areas, including Cancer, Circulatory and Respiratory Health, Health Services and Policy Research, and Musculoskeletal Health and Arthritis, favoring men scientists. We found that relative effects of gender were stable over time and, consequently, absolute differences decreased as funding success decreased for both men and women.

Our review also has limitations. First, we analyzed data from the perspective of the applicant. Although applicants did not submit applications directly to CIHR institutes, we used their self-designated research institute as a surrogate identifier to assess whether gender differences existed in grant and personnel award funding overall and across research communities, reflecting research content areas. Notwithstanding, variability may have existed in how NPIs designated a primary institute based on the content of individual grants or award applications. Furthermore, in a small proportion of applications, the primary institute may not have been identified correctly by the NPI or the NPI may have identified more than one institute. Second, we did not have access to metrics that reflected applicant and coapplicant demographics (nationality, race) and quality (training, rank, experience, productivity), peer reviewer demographics and ratings, or features of the peer-review process itself [23,24]. Third, because few grant applications were directed to the Institute of Aboriginal Peoples’ Health, and even fewer personnel award applications were submitted overall, CIs were broad in corresponding summary estimates of effect.

In our study, women submitted 31.1% and 44.7% of grant and personnel award applications, respectively. These findings align with those of other studies conducted in the US and UK, which identified that women submitted fewer grant applications [25,26]. Data from the UK, NIH, Danish Research Council, and the European Research Council indicate that women were less likely than men to secure grant funding [12, 27–29], held fewer large-scale grants [26], and received less funding in both absolute and relative terms [30]. In a similar longitudinal analysis of 34,770 NIH-funded grants, Hechtman and coworkers found that women submitted fewer applications, held fewer grants, and were less likely to have their grants renewed [31].

Compared with women, men have higher training program award success rates [32–35]. Using NIH data, Jagsi and colleagues found that women received significantly fewer Mentored Career Development (K08) (31.4%) and K23 (43.7%) Awards compared with men [13]. Among recipients of career development awards, women were also significantly less likely than men to receive independent grant funding [11]. In their review of 2,823 early-career scientist applications, van der Lee and colleagues found that men had higher funding success rates and received significantly more competitive “quality of researcher” as opposed to “quality of proposal” evaluations [35]. In their study, gender disparities were most prevalent in scientific disciplines with the highest application volume and with similar gender representation [35]. Under these conditions, they hypothesized that reviewers found it challenging to thoroughly process and weigh applications, increasing the chance that they relied on heuristics in formulating their evaluations. Heuristics, in turn, have been associated with implicit bias [36,37]. Finally, through content analysis, van der Lee and colleagues noted the use of “gendered language” in instructional and evaluation materials that may have favored male applicants [35]. Contrary to their findings, we did not find a relationship between application volume and the visibility of woman applicants in either grant or personnel award competitions.

There are 2 potential explanations for the differential success gradients. Either the applications from women are inferior to those of men or bias against women is responsible for the difference. There is no compelling evidence to support that woman and man researchers are not equally capable. A recent study has shown that although differences in “quality of proposal” ratings were not different between grant applications submitted by women and men, women’s applications were awarded significantly lower “quality of researcher” assessments and were significantly less likely to be prioritized compared with those of men [35]. As data reflecting applicants’ track record were not available for analysis, we cannot rule out the possibility that gender disparities in productivity or features related to productivity (age, authorship position, academic rank) [38] resulted in differential applicant evaluations in our study. Although direct comparisons of funding rates between scientific agencies are difficult because of differences in the type of grants awarded and review procedures, at least one feature of the review process has been identified as particularly prone to gender bias—assessments of applicant’s track records of success [39]. Of interest, a gender gap in funding rates has not been demonstrated with NIH RO1 project grants in which reviewers focus solely on the “quality of the proposal.” By contrast, gender disparity disadvantaging women has been demonstrated in funding rates for RO1 renewal and equivalent awards in which applicants’ track records of success were heavily weighted by reviewers [12,40]. Our findings add to this literature by demonstrating gender differences in gradients of grant success in specific research content areas. Compared with man scientists, woman scientists who submit grant applications in these content areas either submit lower-quality grants or are disadvantaged as applicants.

The introduction of review procedures that exclusively focus on the research proposal is a promising mechanism to reduce gender bias in grant reviews. However, the utility of this approach for evaluating personnel award applications, in which productivity has traditionally been a key consideration, is controversial [35,39]. Other forms of bias including intentional or unintentional reviewer bias, unconscious bias, and implicit gender bias may also, individually or collectively, influence the review process [23, 41–43]. To this end, the Swedish Medical Research Council investigators found that woman scientists required 2.5 times as many impactful publications as man scientists to receive an equivalent score for scientific competence by peer reviewers—a finding that has been reproduced in other studies [44–46]. Although authors have proposed several solutions to address the gender disparity in funding, including gender-equal review committees [47,48], mentoring programs [49], gender policy endorsement [35], principal investigator anonymity [50], de-emphasizing researcher quality in grant applications [35,44,50,51], and using gender neutral language [35,52,53], no study has systematically evaluated the reasons for gender differences in grant and personnel award success rates. Without a clear understanding of the reasons for the observed differences in success, the proposed solutions are speculative and may be harmful. For example, repeated emphasis on gender policies to advance women, especially when both genders are equally represented among applicants, may mislead reviewers into thinking that gender bias is not an issue. With this “paradox of equality,” reviewers may be less vigilant for unequal outcomes, thereby increasing the likelihood of “gender imbalance” occurring in the review processes [54,55].

Our data support the call for mandatory reporting of gender in grant and personnel award applications at national funding agencies [56]. Moreover, our findings highlight the need for funding agencies to monitor gender differences in success rates overall and by research content area and explore possible explanations for gender disparity when identified. Additional research is urgently needed to explicate the reasons for gender differences in success rates, overall and by content area, and to identify “bias-enhancing conditions” in the peer-review process.

Conclusions

Gender disparity existed overall in grant and personnel award success rates, especially for grants directed to selected research communities. Funding agencies should monitor gender differences in grant success rates by content and explore possible explanations for gender disparity when identified.

Supporting information

Acknowledgments

The authors gratefully acknowledge Drs. Rachelle Desrochers, Jonathan Whitely, and Matthew Hogel from CIHR for their assistance in reviewing the proposal and in preparing data for analysis. We are extremely grateful to Dr. Anne Martin-Matthews, CIHR Vice President, and Dr. Robyn Tamblyn, Scientific Director for the Institute of Health Services and Policy Research, for their comments and perspective on drafts of this manuscript. We also thank Drs. Alysha Croker (Manager, Science Strategy Team, CIHR) and Ryan Perry (Assistant Scientific Director, Institute of Circulatory and Respiratory Health, CIHR) for their guidance in conducting this study. The CIHR is committed to collaborating with the research community to advance science and knowledge on best practices in peer review without unduly influencing the conclusions drawn by its collaborators. CIHR employees are not currently permitted to be coauthors on papers describing analyses of the agency’s programs. Without the active participation of CIHR employees and their contributions to this study, this manuscript would not have been possible.The views expressed in this paper are those of the authors and do not necessarily reflect those of the CIHR or the government of Canada.