Although women at all career stages are more likely to leave academia than men, early-career women are a particularly high-risk group. Research supports that women are less likely than men to receive research funding; however, whether funding success rates vary based on research content is unknown. We addressed gender differences in funding success rates for applications directed to one or more of 13 institutes, representing research communities, over a 15-year period.
Methods and findings
We retrospectively reviewed 55,700 grant and 4,087 personnel award applications submitted to the Canadian Institutes of Health Research. We analyzed application success rates according to gender and the primary institute selected by applicants, pooled gender differences in success rates using random effects models, and fitted Poisson regression models to assess the effects of gender, time, and institute. We noted variable success rates among grant applications directed to selected institutes and declining success rates over time. Women submitted 31.1% and 44.7% of grant and personnel award applications, respectively. In the pooled estimate, women had significantly lower grant success (risk ratio [RR] 0.89, 95% confidence interval [CI] 0.84–0.94; p < 0.001; absolute difference 3.2%) compared with men, with substantial heterogeneity (I2 = 58%). Compared with men, women who directed grants to the Institutes of Cancer Research (RR 0.86, 95% CI 0.78–0.96), Circulatory and Respiratory Health (RR 0.74, 95% CI 0.66–0.84), Health Services and Policy Research (RR 0.78, 95% CI 0.68–0.90), and Musculoskeletal Health and Arthritis (RR 0.80, 95% CI 0.69–0.93) were significantly less likely to be funded, and those who directed grants to the Institute of Aboriginal People’s Health (RR 1.67, 95% CI 1.0–2.7) were more likely to be funded. Overall, women also had significantly lower personnel award success (RR 0.75, 95% CI 0.65–0.86; p < 0.001; absolute difference 6.6%). Regression modelling identified that the effect of gender on grant success rates differed by institute and not time. Study limitations include use of institutes as a surrogate identifier, variability in designation of primary institute, and lack of access to metrics reflecting applicants, coapplicants, peer reviewers, and the peer-review process.
Why was this study done?
- Research supports that woman applicants are less likely to receive research funding than men.
- Women at all career stages are more likely to leave academia than men, especially women who are postdoctoral trainees and early in their careers.
- We retrospectively reviewed grant and personnel award submissions based at the Canadian Institutes of Health Research based on research content area to assess for gender differences in funding success rates, which are important to identify whether gender disparity existed in funding. This information is important to address potential sources of bias in the review process.
What did the researchers do and find?
- We retrospectively reviewed 55,700 grant and 4,087 personnel award applications submitted to the Canadian Institutes of Health Research over a 15-year period.
- We found that compared with men, women were significantly less likely to be awarded grants and New Investigator personnel awards.
- Additionally, we identified significant differences in the gradients of grant success in specific research content areas including Cancer, Circulatory and Respiratory Health, Health Services and Policy Research, and Musculoskeletal Health and Arthritis favoring men scientists.
- Regression modelling identified that the effect of gender on grant success rates differed by research community and not time.
What do these findings mean?
- These findings call for heightened awareness in applicants, peer reviewers, and funding agencies to the potential for bias to exist in the evaluation of grant and personnel award applications.
- Our findings highlight the need for funding agencies to monitor gender differences in grant success rates overall and by research content area and to explore possible explanations for gender disparity when identified.
- Additional research is urgently needed to explicate the reasons for gender differences in success rates and identify bias-enhancing conditions in the peer-review process.
Citation: Burns KEA, Straus SE, Liu K, Rizvi L, Guyatt G (2019) Gender differences in grant and personnel award funding rates at the Canadian Institutes of Health Research based on research content area: A retrospective analysis. PLoS Med 16(10): e1002935. https://doi.org/10.1371/journal.pmed.1002935
Academic Editor: Bjoern Weiss, Charité, GERMANY
Received: May 22, 2019; Accepted: September 11, 2019; Published: October 15, 2019
Copyright: © 2019 Burns et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: We do not have permission to share de-identified data transmitted from the CIHR - access to data can be requesting CIHR at email@example.com
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: CI, confidence interval; CIHR, Canadian Institutes of Health Research; LOESS, locally estimated scatterplot smoothing; M-H, Mantel Haenszel; NIH, National Institutes of Health; NPI, nominated principal investigator; OOGP, Open Operating Grant Program; OR, odds ratio; RR, risk ratio; TCPS-2, Tri-Council Policy Statement 2: Ethical Conduct for Research Involving Humans
In 2000, the Canadian Medical Research Council and the National Health Research Grant Program were merged to form the Canadian Institutes of Health Research (CIHR), comprising 13 institutes [1–3]. Each institute focuses on a specific scientific healthcare domain (e.g., cancer) and includes biomedical, clinical, health services, and policy research, as well as research addressing the health of populations, including societal, cultural, and environmental determinants of health . Under the guidance of the governing council, scientific directors, and institute advisory boards, each institute forges a health research agenda that reflects the health needs of Canadians, the evolution of the Canadian healthcare system, gaps in science, and the information needs of policy decision makers .
In the previous funding scheme at the CIHR, research funding was offered through the Open Operating Grant Program (OOGP) and as strategic funding to conduct research in priority areas identified by institute strategic plans in an approximate 70:30 ratio . Through a well-established peer-review process conducted by various committees with content and methodologic expertise, funding was allocated to address gaps in knowledge [1–3,7,8]. To cultivate the development of scientists, peer-review committees at the CIHR also reviewed and funded New Investigator personnel award applications .
Several studies provide evidence that women are less likely than men to receive research funding. A study of grant applications submitted to the CIHR open and strategic grant competitions between 2001 and 2011 by scientists who directed their research to the Institute of Health Services and Policy Research found that men under 45 years old were significantly more likely to be funded than age-matched women (odds ratio [OR] 1.40, 95% confidence interval [CI] 1.01–1.95) . Gender differences in success rates have persisted, with female mid- and senior-career researchers being less likely to secure Foundation Scheme (7-year research program funding) in the second pilot competition of the new CIHR funding scheme . Similarly, studies evaluating grant success at the National Institutes of Health (NIH) have shown significant gender disparity in funding success, favoring men in the rate of receipt of initial Research Project Grants (RO1) awards , number of career RO1 awards , and in productivity as measured by publications and self-perceived success . Although women at all career stages are more likely to leave academia than men, women who are postdoctoral trainees and early in their careers are recognized to be high-risk groups [14–16].
Although institutes do not directly review applications at the CIHR, they reflect scientific communities. Scientists with expertise in content areas are affiliated with particular institutes and participate in peer-review activities related to their context knowledge and expertise. Investigators have not previously addressed the extent to which gender differences have existed in peer-reviewed funding across research communities reflecting research content areas. To examine whether gender differences existed overall in grant and personnel award funding success rates and whether gender differences in success rates differed across research communities, we summarized OOGP grant and personnel award applications approved for funding at the CIHR over a 15-year period in the previous funding scheme.
Study design and population
We retrospectively analyzed data to summarize the number of applications submitted by scientists and scientists in training to the spring and fall OOGP competitions and the New Investigator personnel award competition between 2000/2001 and 2014/2015 according to the self-reported gender and institute selected by the nominated principal investigator (NPI). We excluded bridge grants, priority announcements, and grants that were withdrawn after submission.
At the time grants and personnel award applications were submitted, NPIs self-reported their gender and selected one or more primary institute(s) and a peer-review committee reflecting the content of their application. Success rates were based on competition decisions.
After reviewing our brief study proposal (S1 Text), analysts at the CIHR provided data tables in separate, deidentified files for the OOGP grants and the New Investigator Award competitions in Excel (Microsoft Corporation, Redmond, WA, United States) format. Each line in the databases represented a single application and included the year the application was submitted, gender of the NPI, primary institute selected, and funding outcome.
Data were collected by CIHR and held internally by staff at the CIHR as a national funding agency. Research and analytical studies at the CIHR fall under the Canadian Tri-Council Policy Statement 2: Ethical Conduct for Research Involving Humans (TCPS-2) (available from: http://pre.ethics.gc.ca/eng/policy-politique_tcps2-eptc2_2018.html, accessed September 27, 2019). This study had the objective of evaluating gender differences in CIHR funding and falls under Article 2.5 of TCPS-2 and not within the scope of Research Ethics Board review in Canada.
We used descriptive statistics (mean, median, standard deviation, interquartile range, range, and counts) to summarize the number of grant and award applications submitted and approved for funding and to depict success rates (%) for the OOGP and New Investigator Award competitions. We examined success rates by competition, self-reported gender, year, and the primary institute selected by the applicant. Individual applications represented the unit of analysis.
By necessity, we limited analyses pertaining to gender differences in success rates and regression analysis to applications in which the NPI specified his/her gender. We pooled gender differences in relative success rates using random effects models. We derived summary estimates using risk ratios (RRs) across research communities with 95% CIs using Review Manager 5.3 (Cochrane Collaboration, Oxford, United Kingdom) and computed absolute differences from the pooled overall success rates (men–women) over the 15-year period . We evaluated the impact of statistical heterogeneity among pooled studies for each outcome using the I2 measure, with threshold values of 0%–40%, 30%–60%, 50%–90%, and ≥75% representing heterogeneity that might not be important or represent moderate, substantial, or considerable heterogeneity, respectively [18,19].
To assess the adjusted relative effects on success rates by gender, time, institute affiliation, and interaction terms (gender and institute, and gender and time) post hoc, we fitted 3 Poisson regression models using robust variance estimators with the observed number of applications awarded (response variable) and the number of applications received by the CIHR . The first model included both interaction terms, the second model included one interaction term (gender and institute), and the third model included no interaction terms. We used the likelihood ratio test to assess the significance of the interaction terms on model fit by comparing 2 sets of nested Poisson regression models (first and second models; second and third models).
Table 1 presents a summary of the number of OOGP applications received, the number approved for funding, and success rates according to institute. The largest number of applications (on average more than 300 grant applications per year) were directed by scientists to the Institutes of Cancer Research, Circulatory and Respiratory Health, Genetics, Infection and Immunity, Neurosciences/Mental Health and Addiction, and Nutrition/Metabolism and Diabetes. Average success rates were lowest among scientists who directed applications to smaller-volume institutes including the Institutes of Aboriginal Peoples’ Health (17.5%), Gender and Health (18.1%), and Population and Public Health (18.6%) and highest among scientists who directed applications to higher-volume institutes such as Genetics (28.7%), Infection and Immunity (25.5%), and Neurosciences/Mental Health and Addiction (24.3%). There was considerable variation in application volume and success rates over time and across designated institutes (Table 1). Success rates in the OOGP decreased over time across institutes except for the Gender and Health institute, which increased after being launched in 2000/2001 and remained relatively stable over time (Figs 1 and S1).
CIHR, Canadian Institutes of Health Research; LOESS, locally estimated scatterplot smoothing; SD, standard deviation.
New investigator award applications
On average, more than 25 New Investigator Award applications per year were directed to 5 institutes including Cancer Research, Circulatory and Respiratory Health, Health Services and Policy Research, Infection and Immunity, and Neurosciences/Mental Health and Addiction. Average success rates in the New Investigator competition were lowest among scientists who directed applications to the Institutes of Aboriginal Peoples’ Health (9.0%) and Musculoskeletal Health and Arthritis (13.7%) and highest in Genetics (28.1%) and Infection and Immunity (26.9%) (S1 Table).
Sex differences in OOGP success
Fig 2 shows the total number of OOGP applications submitted. Although the number of grant applications submitted by men showed greater variation over time, the number of grant applications submitted by women remained consistently low. Compared with men, however, women submitted more grant applications to 3 institutes: Aboriginal Peoples’ Health (179 versus 245), Gender and Health (533 versus 737), and Health Services and Policy Research (1,496 versus 1,591).
In the pooled estimate across institutes, women had significantly lower grant success (RR 0.89, 95% CI 0.84–0.94; p < 0.001; absolute difference 3.2%) compared with men, with substantial heterogeneity (I2 = 58%). Women who directed applications to the Institutes of Cancer Research (RR 0.86, 95% CI 0.78–0.96), Circulatory and Respiratory Health (RR 0.74, 95% CI 0.66–0.84), Health Services and Policy Research (RR 0.78, 95% CI 0.68–0.90), and Musculoskeletal Health and Arthritis (RR 0.80, 95% CI 0.69–0.93) were significantly less likely to be funded than men. Conversely, women who directed applications to the Institute of Aboriginal Peoples’ Health (RR 1.67, 95% CI 1.0–2.7) were significantly more likely to be funded than men (Fig 3).
Sex differences in new investigator award success
Compared with men, women directed more New Investigator Award applications to 6 institutes (Aboriginal Peoples’ Health; Aging; Gender and Health; Health Services and Policy Research; Human Development, Child, and Youth; and Population and Public Health). In the pooled estimate, women had significantly lower New Investigator Award success (RR 0.75, 95% CI 0.65–0.86; p < 0.001; absolute difference 6.6%) compared with men, with very little heterogeneity (I2 = 0%) (Fig 4).
Using the likelihood ratio test, we did not identify significant differences (p = 0.6) between the first model including both interaction terms (gender and institutes, gender and time) and the second model including only the interaction between gender and institutes, suggesting that removal of the interaction between gender and time does not statistically impact model fit, and the simpler model is preferred. However, the likelihood ratio test between the second model, containing the interaction term between gender and institutes, and the third model, which did not contain interaction terms, was statistically significant (p = 0.02), suggesting that the effect of gender on grant success rates differed by designated institute (research content area) and not time.
Over 15 years of funding at the CIHR, we noted variable success rates among grant applications directed to various institutes and declining success rates over time. Overall, women had significantly lower grant success (RR 0.89, 95% CI 0.84–0.94; p < 0.001; absolute difference 3.2%) compared with men. Across institutes, we found substantial heterogeneity, with variability demonstrated both in the primary analysis for grant applications and in the Poisson regression. Women who directed grants to 4 institutes (Cancer Research, Circulatory and Respiratory Health, Health Services and Policy Research, and Musculoskeletal Health and Arthritis) were significantly less likely to be funded than men. However, women who directed grants to the Institute of Aboriginal Peoples’ Health were more likely to be funded than men. In pooled estimates, women also had significantly lower personnel award success (RR 0.75, 95% CI 0.65–0.86; p < 0.001; absolute difference 6.6%) compared with men, with no heterogeneity. Poisson regression models supported that the effect of gender on grant success rates differed by designated institute and not time.
Our review has several strengths. It is novel in assessing the extent to which gender differences in peer-reviewed funding exist across research communities, represented by CIHR institutes, over a 15-year period in the former funding scheme. We summarized data using both relative and absolute effect measures [21,22] and report our findings using recommendations for cohort studies (S1 Strobe checklist). We identified significant differences in the gradients of grant success in specific research content areas, including Cancer, Circulatory and Respiratory Health, Health Services and Policy Research, and Musculoskeletal Health and Arthritis, favoring men scientists. We found that relative effects of gender were stable over time and, consequently, absolute differences decreased as funding success decreased for both men and women.
Our review also has limitations. First, we analyzed data from the perspective of the applicant. Although applicants did not submit applications directly to CIHR institutes, we used their self-designated research institute as a surrogate identifier to assess whether gender differences existed in grant and personnel award funding overall and across research communities, reflecting research content areas. Notwithstanding, variability may have existed in how NPIs designated a primary institute based on the content of individual grants or award applications. Furthermore, in a small proportion of applications, the primary institute may not have been identified correctly by the NPI or the NPI may have identified more than one institute. Second, we did not have access to metrics that reflected applicant and coapplicant demographics (nationality, race) and quality (training, rank, experience, productivity), peer reviewer demographics and ratings, or features of the peer-review process itself [23,24]. Third, because few grant applications were directed to the Institute of Aboriginal Peoples’ Health, and even fewer personnel award applications were submitted overall, CIs were broad in corresponding summary estimates of effect.
In our study, women submitted 31.1% and 44.7% of grant and personnel award applications, respectively. These findings align with those of other studies conducted in the US and UK, which identified that women submitted fewer grant applications [25,26]. Data from the UK, NIH, Danish Research Council, and the European Research Council indicate that women were less likely than men to secure grant funding [12, 27–29], held fewer large-scale grants , and received less funding in both absolute and relative terms . In a similar longitudinal analysis of 34,770 NIH-funded grants, Hechtman and coworkers found that women submitted fewer applications, held fewer grants, and were less likely to have their grants renewed .
Compared with women, men have higher training program award success rates [32–35]. Using NIH data, Jagsi and colleagues found that women received significantly fewer Mentored Career Development (K08) (31.4%) and K23 (43.7%) Awards compared with men . Among recipients of career development awards, women were also significantly less likely than men to receive independent grant funding . In their review of 2,823 early-career scientist applications, van der Lee and colleagues found that men had higher funding success rates and received significantly more competitive “quality of researcher” as opposed to “quality of proposal” evaluations . In their study, gender disparities were most prevalent in scientific disciplines with the highest application volume and with similar gender representation . Under these conditions, they hypothesized that reviewers found it challenging to thoroughly process and weigh applications, increasing the chance that they relied on heuristics in formulating their evaluations. Heuristics, in turn, have been associated with implicit bias [36,37]. Finally, through content analysis, van der Lee and colleagues noted the use of “gendered language” in instructional and evaluation materials that may have favored male applicants . Contrary to their findings, we did not find a relationship between application volume and the visibility of woman applicants in either grant or personnel award competitions.
There are 2 potential explanations for the differential success gradients. Either the applications from women are inferior to those of men or bias against women is responsible for the difference. There is no compelling evidence to support that woman and man researchers are not equally capable. A recent study has shown that although differences in “quality of proposal” ratings were not different between grant applications submitted by women and men, women’s applications were awarded significantly lower “quality of researcher” assessments and were significantly less likely to be prioritized compared with those of men . As data reflecting applicants’ track record were not available for analysis, we cannot rule out the possibility that gender disparities in productivity or features related to productivity (age, authorship position, academic rank)  resulted in differential applicant evaluations in our study. Although direct comparisons of funding rates between scientific agencies are difficult because of differences in the type of grants awarded and review procedures, at least one feature of the review process has been identified as particularly prone to gender bias—assessments of applicant’s track records of success . Of interest, a gender gap in funding rates has not been demonstrated with NIH RO1 project grants in which reviewers focus solely on the “quality of the proposal.” By contrast, gender disparity disadvantaging women has been demonstrated in funding rates for RO1 renewal and equivalent awards in which applicants’ track records of success were heavily weighted by reviewers [12,40]. Our findings add to this literature by demonstrating gender differences in gradients of grant success in specific research content areas. Compared with man scientists, woman scientists who submit grant applications in these content areas either submit lower-quality grants or are disadvantaged as applicants.
The introduction of review procedures that exclusively focus on the research proposal is a promising mechanism to reduce gender bias in grant reviews. However, the utility of this approach for evaluating personnel award applications, in which productivity has traditionally been a key consideration, is controversial [35,39]. Other forms of bias including intentional or unintentional reviewer bias, unconscious bias, and implicit gender bias may also, individually or collectively, influence the review process [23, 41–43]. To this end, the Swedish Medical Research Council investigators found that woman scientists required 2.5 times as many impactful publications as man scientists to receive an equivalent score for scientific competence by peer reviewers—a finding that has been reproduced in other studies [44–46]. Although authors have proposed several solutions to address the gender disparity in funding, including gender-equal review committees [47,48], mentoring programs , gender policy endorsement , principal investigator anonymity , de-emphasizing researcher quality in grant applications [35,44,50,51], and using gender neutral language [35,52,53], no study has systematically evaluated the reasons for gender differences in grant and personnel award success rates. Without a clear understanding of the reasons for the observed differences in success, the proposed solutions are speculative and may be harmful. For example, repeated emphasis on gender policies to advance women, especially when both genders are equally represented among applicants, may mislead reviewers into thinking that gender bias is not an issue. With this “paradox of equality,” reviewers may be less vigilant for unequal outcomes, thereby increasing the likelihood of “gender imbalance” occurring in the review processes [54,55].
Our data support the call for mandatory reporting of gender in grant and personnel award applications at national funding agencies . Moreover, our findings highlight the need for funding agencies to monitor gender differences in success rates overall and by research content area and explore possible explanations for gender disparity when identified. Additional research is urgently needed to explicate the reasons for gender differences in success rates, overall and by content area, and to identify “bias-enhancing conditions” in the peer-review process.
Gender disparity existed overall in grant and personnel award success rates, especially for grants directed to selected research communities. Funding agencies should monitor gender differences in grant success rates by content and explore possible explanations for gender disparity when identified.
S1 Fig. Combined grant success rates by year across CIHR institutes (actual data).
CIHR, Canadian Institutes of Health Research.
S1 Text. Research proposal.
CIHR, Canadian Institutes of Health Research; OOGP, Open Operating Grant Program.
S1 Table. New investigator award applications received and approved for funding and success rate by institute.
SD, standard deviation.
The authors gratefully acknowledge Drs. Rachelle Desrochers, Jonathan Whitely, and Matthew Hogel from CIHR for their assistance in reviewing the proposal and in preparing data for analysis. We are extremely grateful to Dr. Anne Martin-Matthews, CIHR Vice President, and Dr. Robyn Tamblyn, Scientific Director for the Institute of Health Services and Policy Research, for their comments and perspective on drafts of this manuscript. We also thank Drs. Alysha Croker (Manager, Science Strategy Team, CIHR) and Ryan Perry (Assistant Scientific Director, Institute of Circulatory and Respiratory Health, CIHR) for their guidance in conducting this study. The CIHR is committed to collaborating with the research community to advance science and knowledge on best practices in peer review without unduly influencing the conclusions drawn by its collaborators. CIHR employees are not currently permitted to be coauthors on papers describing analyses of the agency’s programs. Without the active participation of CIHR employees and their contributions to this study, this manuscript would not have been possible.The views expressed in this paper are those of the authors and do not necessarily reflect those of the CIHR or the government of Canada.
- 1. Hoey J. Social sciences meet basic sciences in new era of health-services research. CMAJ. 2001;164:248.
- 2. Gray C. Clinicians will benefit from new research initiative, CIHR promises. CMAJ. 2000;163:586.
- 3. Kondro W. CIHR floats “research-into-action” trial balloon. CMAJ. 2003;168:209.
- 4. Tamblyn R, McMahon M, Girard N, Drake E, Nadigel J, Gaudreau K. Health services and policy research in the first decade at the Canadian Institutes of Health Research. CMAJ open. 2016;4(2):E213–21. pmid:27398366
- 5. Canadian Institutes of Health Research [Internet]. Ottawa: Canadian Institutes of Health Research; c2017 [cited 2018 Feb 16]. About the institutes. Available from: http://www.cihr-irsc.gc.ca/e/7155.html.
- 6. Canadian Institutes of Health Research [Internet]. Ottawa: Canadian Institutes of Health Research; c2018 [cited 2018 Sep 25]. Reports to Parliament. Available from: http://www.cihr-irsc.gc.ca/e/22978.html.
- 7. Kondro W. Correcting the CIHR course. CMAJ. 2009;181:E223–4. pmid:19797264
- 8. Canadian Institutes of Health Research [Internet]. Ottawa: Canadian Institutes of Health Research; c2014 [cited 2017 Jul 26]. Institute of Health Services and Policy Research. Available from: www.cihr-irsc.gc.ca/e/13733.html.
- 9. Golper TA, Feldman HI. New challenges and paradigms for mid-career faculty in academic medical centers: key strategies for success for mid-career medical school faculty. Clin J Am Soc Nephrol. 2008 Nov;3(6):1870–4. pmid:18322049
- 10. Canadian Institutes of Health Research [Internet]. 2016 Foundation Grant Results. Ottawa: Canadian Institutes of Health Research; c2016 [cited 2019 Sep 27]. Available from: http://www.cihr-irsc.gc.ca/e/49854.html.
- 11. Jagsi R, Motomura AR, Griffith KA, Rangarajan S, Ubel PA. Sex differences in attainment of independent funding by career development awardees. Ann Intern Med. 2009;151:804–11. pmid:19949146
- 12. Pohlhaus JR, Jiang H, Wagner RM, Schaffer WT, Pinn VW. Sex differences in application, success, and funding rates for NIH extramural programs. Acad Med. 2011;86:759–67. pmid:21512358
- 13. Jagsi R, DeCastro R, Griffith KA, Rangarajan S, Churchill C, Stewart A, et al. Similarities and differences in the career trajectories of male and female career development award recipients. Acad Med. 2011;86:1415–21. pmid:21952061
- 14. Martinez ED, Botos J, Dohoney KM, Geiman TM, Kolla SS, Olievera A, et al. Falling off the academic bandwagon. Women are more likely to quit at the postdoc to principal investigator transition. EMBO Rep. 2007;8:977–81. pmid:17972894
- 15. Levine RB, Lin F, Kern DE, Wright SM, Carrese J. Stories from early-career women physicians who have left academic medicine: A qualitative study at a single institution. Acad Med. 2011;86:752–758. pmid:21512363
- 16. Lowenstein SR, Fernandez G, Crane LA. Medical school faculty discontent: Prevalence and predictors of intent to leave academic careers. BMC Med Educ. 2007;7:37. pmid:17935631
- 17. Review Manager (RevMan). Version 5.3.0 [software]. 2011. The Cochrane Collaboration. The Nordic Cochrane Centre.
- 18. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21:1539–58. pmid:12111919
- 19. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration. London; 2011 [updated 2011 Mar]. Available from: http://handbook.cochrane.org.
- 20. Frome EL. The analysis of rates using Poisson regression models. Biometrics. 1983;39(3):665–74. pmid:6652201
- 21. King NB, Harper S, Young ME. Use of absolute and relative effect measures in reporting health inequalities: structured review. BMJ. 2012;345:e5774. pmid:22945952
- 22. Low A, Low A. Importance of relative measures in policy on health inequalities. BMJ. 2006;332:967–9. pmid:16627518
- 23. Clohisy DR, Yaszemski MJ, Lipman J. Leadership, Communication, and Negotiation Across a Diverse Workforce*: An AOA Critical Issues Symposium. J Bone Joint Surg Am. 2017 Jun 21;99(12):e60. pmid:28632597
- 24. Girod S, Fassiotto M, Grewal D, et al. Reducing implicit gender leadership bias in academic medicine with an educational intervention. Acad Med. 2016;91:1143–50. pmid:26826068
- 25. National Institutes of Health [Internet]. Research project grants: Competing applications and awards, by gender. Bethesda, MD: National Institutes of Health: 2016 [cited 2018 Jul 25]. Available from: https://report.nih.gov/nihdatabook/report/132?showm=Y&chartId=176&catId=1.
- 26. McAllister D, Juillerat J, Hunter J. Funding: What stops women getting more grants? Nature. 2016;529:466
- 27. National Institutes of Health [Internet]. Research grant investigators: Representation of women, by mechanism. Bethesda, MD: National Institutes of Health; 2016 [cited 2018 May 30]. Available from: https://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=169&catId=15.
- 28. Ginther DK, Schaffer WT, Schnell J, Masimore B, Liu F, Haak LL, et al. Race, ethnicity, and NIH research awards. Science. 2011;333:1015–1019. pmid:21852498
- 29. European Research Council. Gender statistics. European Research Council. Brussels; 2017 [cited 2018 Jul 25]. Available from: https://erc.europa.eu/sites/default/files/document/file/erc_2017_cog_statistics.pdf.
- 30. Head MG, Fitchett JR, Cooke MK, Wurie FB, Atun R. Differences in research funding for women scientists: a systematic comparison of UK investments in global infectious disease research during 1997–2010. BMJ Open. 2013;3:e003362. pmid:24327360
- 31. Hechtman LA, Moore NP, Schulkey CE, Miklos AC, Calcagno AM, Aragon R, et al. NIH Funding Longevity by Gender. Proc Natl Acad Sci USA. 2018 Jul 16. pmid:30012615
- 32. Waisbren SE, Bowles H, Hasan T, Zou KH, Emans SJ, Goldberg C, et al. Gender differences in research grant applications and funding outcomes for medical school faculty. J Womens Health (Larchmt). 2008;17(2):207–14. pmid:18321172
- 33. Eloy JA, Svider PF, Kovalerchik O, Baredes S, Kalyoussef E, Chandrasekhar SS. Gender differences in successful NIH grant funding in otolaryngology. Otolaryngol Head Neck Surg. 2013;149(1):77–83. pmid:23585148
- 34. Gordon MB, Osganian SK, Emans SJ, Lovejoy FH Jr. Gender differences in research grant applications for pediatric residents. Pediatrics. 2009;124(2):e355–61. pmid:19581267
- 35. van der Lee R, Ellemers N. Gender contributes to personal research funding success in The Netherlands. Proc Natl Acad Sci USA. 2015;112(40):12349–353 pmid:26392544
- 36. Nosek BA, Smyth FL, Sriram N, Lindner NM, Devos T, Ayala A, et al. National differences in gender-science stereotypes predict national sex differences in science and math achievement. Proc Natl Acad Sci USA. 2009;106:10593–97. pmid:19549876
- 37. Devine PG. Stereotypes and prejudice: Their automatic and controlled components. J Pers Soc Psychol. 1989;56:5–18.
- 38. van den Besselaar P, SandstroÈm U. Vicious circles of gender bias, lower positions, and lower performance: Gender differences in scholarly productivity and impact. PLoS ONE. 2017 [cited 2019 Sep 27];12(8):e0183301. Available from: https://doi.org/10.1371/journal.pone.0183301 pmid:28841666
- 39. Bornmann L, Mutz R, Daniel HD. Gender differences in grant peer review: A meta-analysis. J Informetrics. 2007;1(3):226–38.
- 40. National Institutes of Health [Internet]. R01-equivalent grants: Success rates, by gender and type of application. Bethesda, MD: National Institutes of Health; 2015 [cited 2015 Apr 10]. Available from: report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=131&catId=13.
- 41. Files JA, Mayer AP, Ko MG, Friedrich P, Jenkins M, Bryan MJ, et al. Speaker Introductions at Internal Medicine Grand Rounds: Forms of Address Reveal Gender Bias. J Womens Health (Larchmt). 2017;26(5):413–19.
- 42. Phillips NA, Tannan SC, Kalliainen LK. Understanding and Overcoming Implicit Gender Bias in Plastic Surgery. Plast Reconstr Surg. 2016;138:1111–6. pmid:27391836
- 43. Mehta S, Burns KEA, Machado FR, Fox-Robichaud AE, Cook DJ, Calfee CS, et al. Diversity Matters–Gender Parity in Critical Care. Am J Respir Crit Care Med. 2017;196:425–9. pmid:28240961
- 44. Wenneras C, Wold A. Nepotism and Sexism in Peer-Review. Nature. 1997;387:341–3. pmid:9163412
- 45. Gannon F, Quirk S, Guest S. Searching for discrimination: Are women treated fairly in the EMBO postdoctoral fellowship scheme? EMBO Rep. 2001;2:655–7. pmid:11493590
- 46. Sandstrom U, Hallsten M. Persistent nepotism in peer-review. Scientometrics. 2008;74:175–89.
- 47. Steinpreis RE, Anders KA, Ritzke D. The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: A national empirical study. Sex Roles. 1999;41(7–8):509–528.
- 48. Marsh HW, Jayasinghe UW, Bond NW. Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability. Am Psychol. 2008;63(3):160–8. pmid:18377106
- 49. Bedi G, Van Dam NT, Munafo M. Gender inequality in awarded research grants. Lancet. 2012;380:474. pmid:22863053
- 50. Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, Handelsman J. Science faculty’s subtle gender biases favor male students. Proc Natl Acad Sci USA. 2012;109:16474–9. pmid:22988126
- 51. Biernat M, Fuegen K. Shifting standards and the evaluation of competence: Complexity in gender-based judgments and decision making. J Soc Issues. 2001;57(4):707–24.
- 52. Schmader T, Whitehead J, Wysocki VH. A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles. 2007;57(7–8):509–14. pmid:18953419
- 53. Kaatz A, Magua W, Zimmerman DR, Carnes M. A quantitative linguistic analysis of National Institutes of Health R01 application critiques from investigators at one institution. Acad Med. 2015;90(1):69–75. pmid:25140529
- 54. Castilla EJ, Bernard S. The paradox of meritocracy in organizations. Adm Sci Q. 2010;55(4):543–76.
- 55. Kaiser CR, et al. Presumed fair: Ironic effects of organizational diversity structures. J Pers Soc Psychol. 2013;104(3):504–19. pmid:23163748
- 56. Sharman Z, Johnson J. Towards the inclusion of gender and sex in health research and funding: an institutional perspective. Social Sci Medicine. 2012;74:1812–16.