Conceived and designed the experiments: ME MS. Analyzed the data: NG MS TB KG. Contributed reagents/materials/analysis tools: NG TB KG. Wrote the paper: ME MS.
The authors have declared that no competing interests exist.
We examined the publication records of a cohort of 168 life scientists in the field of ecology and evolutionary biology to assess gender differences in research performance. Clear discrepancies in publication rate between men and women appear very early in their careers and this has consequences for the subsequent citation of their work. We show that a recently proposed index designed to rank scientists fairly is in fact strongly biased against female researchers, and advocate a modified index to assess men and women on a more equitable basis.
The causes of differences in gender representation within the hierarchical structure of academic science remain contentious. In 2005 the flames of this controversy were fanned by the widely reported comments of Lawrence Summers
There is a clear difference between men and women in science with regard to the quantity of their research output. On average, males publish more papers than their female counterparts, a trend that is consistent across scientific disciplines and exists even when obvious mitigating factors are taken into consideration
One explanation that may account for the productivity puzzle is that female researchers produce fewer but higher quality publications. For example, one survey of biochemists
We examined whether a gender pattern of quality versus quantity holds for researchers in the field of evolutionary biology and ecology. The data were a subset of those used in a previous analysis
Consistent with previous studies
Frequency distributions of the number of publications by male and female researchers in our sample.
Annual productivity of male and female researchers over time.
The graphs also indicate a second dip in productivity rates for females at around the 9–10 year mark. We can only speculate as to its cause, but it may coincide with a time when a number of factors have their greatest impact on female productivity, namely reduced success in grant rounds, time devoted to childcare, and greater administrative burden, as previously suggested. Many strategies implemented by universities to improve representation of women at higher levels in academia focus on mentoring programmes, with the intention of improving their competitiveness for funding, appointment and promotion. However, the implications of these productivity patterns are that, in most cases, such programmes may be offered too late to be useful. We suggest that such schemes need to be implemented at an extremely early career stage (i.e. at graduate student level).
Our analysis covers only researchers from one area of science, but an examination of gender differences in funding success across the arts and sciences suggest that these trends have broader generality. We examined age- and gender-specific success in the Australian Research Council's Discovery Grant awards over six years since 2001 (
Year | % grants funded | % discrepancy males – females age<30 | % discrepancy males – females age 30 and over |
2001 | 25.6 | 6.8 | 3.3 |
2002 | 28.2 | 1.2 | 6.3 |
2003 | 28.7 | 5.4 | 1.1 |
2004 | 34.5 | 2.1 | 5.3 |
2005 | 27.5 | 7.5 | 4.5 |
2006 | 22.4 | 7.5 | 5.0 |
N.B. Men always have a higher % success than women.
There is no difference in the median number of citations per paper for males and females (median = 9 and 10 respectively; Mann-Whitney U = 2830.0, P = 0.237), which argues against a quality versus quantity hypothesis. Nor is there any evidence that men employ a more ‘hit and miss’ strategy for their output, with the variation in citations per paper being similar in males and females (median interquartile range = 15.50 and 13.75 respectively; U = 2653.5, P = 0.603). However, the first quartile of female median citations is significantly higher than that for males (median = 6 and 4 respectively; U = 3225.5, P = 0.007), indicating that there are relatively few females who produce a body of work that is poorly cited. Perhaps males who produce ‘poor quality’ work are more likely to survive in science than females.
However, drawing conclusions about the relationship between quantity and quality of research output is problematic if number of citations is used as the measure of quality because this metric is not independent of our measure of quantity. The median number of citations for our sample of authors is correlated with the number of papers they have published (r = 0.266, n = 168, P<0.001 – using log-transformed values). In other words, more-productive scientists produce more highly cited papers. Kelly & Jennions
We control for non-independence in our analysis by plotting the average number of citations per publication against total number of publications and calculating the y-residuals from the least squares regression line. When we do this (
Relationship between quality of output (median number of citations) and quantity of output for male and female researchers.
One potential complicating factor that we have not considered is self-citation. Researchers are likely to cite their earlier publications to varying extents and this may be more likely if their body of output is larger. The rate of self-citation could influence our analysis if there are gender differences in the propensity to self-cite. We investigated this possibility using the Web of Science's ‘Citation Analysis Report’ option, which provides details of papers that have cited an author's work, with and without self-citations. We found no evidence of gender differences in the rates of self-citation, using a randomly chosen subset of 20 females and 20 males from our original sample (mean percentage of citing papers for an author that are by that author = 5.81% and 6.21% respectively; t32 = 0.310, P = 0.759). Accordingly, our broader analysis is unlikely to be systematically biased by any gender differences in the rates of self-citation.
Given that there are differences between males and females in the quantity, and potentially quality, of research output, how can we establish academic selection systems that do not discriminate on the grounds of gender? Clearly, criteria based solely on quantity of output would favour males, but our results show that even when quality of research is taken into account (through impact of papers) males may be favoured since this measure of quality is correlated with quantity. If we are to ensure that research performance is assessed without such gender bias, then we need a measure that takes into account the relationship between quality and quantity.
The recently proposed
We advocate an alternative metric to
While we believe that our new metric provides a more equitable measure of research performance, it is susceptible in a detrimental way to the addition of just a handful of poorly cited papers. This property might deter scientists from publishing minor works that contain essential but unexciting results. However, it is a moot point whether research that fails to make an impact is actually useful. An alternative view is that this metric might encourage scientists to think more carefully about the quality and potential impact of their research before embarking on a project.
A second problem with our
Clearly, an assessment of a scientific career should not ultimately boil down to a single number
Of course, some will argue that shifting the means by which we assess scientific performance is artificial and undesirable. However, until the career structure of science finds ways to assess females and males on a level playing field that takes into account the prevalent gender differences and imbalances (whatever their causes), we will continue to perpetrate inequality, and fail to maximise our intellectual capital
Publication and citation information for the 168 researchers in our analysis.
(0.32 MB DOC)
We thank E. Van Wilgenburg, K. McNamara, E. Van Lieshout and B.S. Symonds for comments and discussion.