Gender imbalance of eLife reviewing editors

We first quantified the gender ratio among eLife BRE members (Fig 1B). The proportion of RE service contributed by men was significantly larger than the proportion contributed by women throughout the entire study period (2017–2019: N = 12,518 months, women vs. men BRE service: 30.60% vs 69.40%; binomial p<0.001; Cohen’s h = 0.40; see supplementary data and Sup Fig 1A for a comparison of months of service across gender).

We next considered dynamics in gender balance over the three-year window. The gender imbalance observed overall slightly diminished over time due to eLife’s effort to recruit more women to the BRE. The proportion of women in the BRE did not significantly differ between 2017 and 2018 (1.81% difference, χ²₍₁₎ = 3.10, p = 0.078, Cohen’s h = 0.86). By contrast, the proportion in 2019 was significantly greater than that in 2018 (N-1 χ² proportion comparison test; 2018 vs. 2019: 4.42% difference, χ²₍₁₎ = 19.67, p<0.001, Cohen’s h = 0.78). Despite this slight improvement, the BRE gender base rate remained strongly imbalanced (2017: N = 3,715 months, women vs. men BRE service months: 27.64% vs. 72.36%; 2018: N = 4,047 months, 29.45% vs. 70.55%; 2019: N = 4,756 months, 33.87% vs. 66.13%; binomial p-values<0.001; 2017: Cohen’s h = 0.46, 2018: Cohen’s h = 0.42, 2019: Cohen’s h = 0.33). Accordingly, and for all subsequent analyses, the 2017–2019 data were pooled to increase statistical power. Taken together, these results indicate that there exists a pronounced gender imbalance in the BRE gender base rate.

In the next analyses, we used the gender ratio of women BRE members as the base rate to measure if women RE engagement was proportional to what was expected by their representation in the BRE.

External influence in the initial submission–(Action 1)

At the Initial Submission stage, authors suggest potential BRE members that could handle their manuscript (Action 1). We tested if this action was (im)balanced according to gender by comparing the proportion of women REs that were suggested by authors relative to the women BRE member base rate. A N-1 χ2 proportion comparison test revealed that authors suggest significantly fewer women REs than the corresponding proportion among eLife’s BRE (29.08% vs. 30.6%, χ2(1) = 11.65, p<0.001, Cohen’s h = 0.90; Fig 1C). We next sought to determine whether women’s perceived expertise might be a partial explanation for authors’ imbalanced RE suggestions. Specifically, previous research points at potential disparity with the broadness of term women and men use when communicating research [74]. Accordingly, we tested whether women and men REs differed in the number of keywords used to showcase their expertise. We found that women and men REs did not differ in their numbers of associated keywords (Women: 5.51±2.19; Men: 5.32±2.39; t₍₅₈₁₎ = 0.932, p = 0.352). We next sought to determine whether a difference in the scope and reach of the keywords associated with women and men REs could contribute to authors’ imbalanced RE suggestions. Accordingly, we quantified the number of PubMed search results for women and men BRE members’ keywords. Specifically, we queried the number of PubMed publications associated with the string of keywords provided by each RE, using an ‘OR’ operator. This provided us with a simple mean of scope. A permutation Welch’s t-test comparing groups in the number of PubMed search results was not significant (women vs. men search results: 1,755,724±2,979,049 vs. 1,920,643±3,307,501; t(488.9475) = 0.62; p = 0.56; Hedge’s g = -0.052). These data provide no evidence of a gender difference in the overall reach of the keywords provided by BRE members.

Internal processes in the initial assessment stage (Actions 2–3)

We next explored the presence of gender imbalances during Initial Assessments. In this action, the Senior Editor invites one or more REs for an initial consultation in order to assess whether to invite a full submission of the manuscript for peer review. To test whether Senior Editors tend to similarly engage women and men REs (Action 2), we compared the average number of consultation requests per month for individual REs. A permutation Welch’s t-test showed no significant difference in the mean number of requests per month between women and men REs (t_(809.7) = 0.11, p = 0.92; Fig 1D), indicating no evidence for imbalanced engagement solely based on RE gender in this Action. While examining the distributions of requests per month, it appeared that the distribution of the men REs might have a longer tail (kurtosis men RE = 2.36; kurtosis women RE = 1.28). Intuitively, a gender difference in the distribution of requests per month could be due to the increased involvement of selected men REs. To examine this possibility, we selected the BREs who were disproportionally engaged in initial consultations relative to the BRE; that is, the 43 REs defined as the upper outliers of the population (defined as higher than the 75th percentile+1.5×interquartile range), with an average of 6.9 monthly consultations, relative to 2.24 on average. We find that only 10 of these especially engaged REs (23%) were women. However, Levene’s test for equality of variances did not show significant differences between men and women RE request distributions (F_(1,1217) = 0.052, p = 0.82). As such, we find no evidence for gender differences when approaching REs for initial consultation.

We then evaluated the presence of gender differences in RE responses to the initial consultation request (Action 3). Compared to men REs, the response rate of women REs was significantly lower (0.88±0.17 compared to 0.91±0.14; Welch’s t_(651.4) = 3.04, p = 0.001, Hedges’s g = 0.20; Fig 1D). In addition, women REs took longer to respond compared to men REs (1.83±1.55 days vs. 1.54±1.23 days; Welch’s t_(636.3) = -3.24, p = 0.002, Hedges’s g = -0.22; Fig 1D). These data provide converging evidence for longer response time and less frequent responses of women REs when engaging in initial consultations, in comparison to men REs.

Community-facing processes in the full submission stage (Action 4)

For manuscripts that pass the initial assessment, the Senior Editor assigns an RE who handles the reviewing process (Action 4). In order to evaluate the presence of imbalances in RE assignment, we first compared the number of full submissions per month handled by women and men REs. A permutation Welch’s t-test showed that women REs handled slightly, though significantly, fewer submissions per month than men REs (0.40±0.32 vs. 0.44±0.37; t_(869.8) = 2.22, p = 0.026, Hedges’s g = 0.13; Fig 2A). We next explored the effect of the Senior Editor’s gender on manuscript assignment to women and men REs. Using a contingency table analysis, we compared the proportion of manuscripts assigned to women and to men REs as a function of Senior Editor gender. Compared to RE gender base rates of manuscript assignment (6,289 manuscripts; women vs. men RE assigned manuscripts: 30.04% vs. 69.96 %), women Senior Editors assigned significantly more manuscripts to women REs (41.41% for women SEs vs. 30.04% for all SEs) and men Senior Editors assigned significantly more manuscripts to men REs (76.57% for men SEs vs. the 69.96% for all SEs; χ²₍₁₎ = 224.55, p<0.001, contingency coefficient 0.186; Fig 2B). These results demonstrate that both women and men SEs are more likely to assign papers to REs of the same gender relative to the gender base rates.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Gender disparities in eLife during full submission (Action 4).

A. Men REs (blue) handle slightly more full submissions per month than women REs (red). B. Compared to manuscript assignment gender base rates (yellow lines; Base rate of RE manuscript assignment: Women– 30.04%; Men– 69.96 %;), Men Senior Editors (SE, top) assign significantly more manuscripts to men REs (blue; 76.57%) and women Senior Editors (bottom) assign significantly more manuscripts to women REs (red; 41.41%). C. SE-BRE Manuscript assignment homophily is prevalent across disciplines. The effect of Senior Editor gender on the assigned RE’s gender across manuscript disciplines, showing preferential assignment of men REs (blue) by men Senior Editors (left) and of women REs (red) by women Senior Editors (right), compared to the gender base rate of RE manuscript assignment (yellow lines; p values are FDR corrected). D. A scatter plot showing the correlation between the Senior Editor homophily effect (the difference in the rate of manuscripts assigned to men REs when the Senior Editor is a man and when the Senior Editor is a woman) and the Manuscript Assignment Imbalance (the difference in the rate of manuscripts assigned to men REs versus to women REs), across disciplines (Pearson r = -0.47, p = 0.049, BF10 = 1.77). Shaded area depicts the 95% confidence interval. *p≤0.05, **p≤0.01, ***p≤0.001.

https://doi.org/10.1371/journal.pone.0294805.g002

In order to examine how this manifestation of gender homophily might vary across disciplines, we next divided the manuscripts according to the disciplines the authors assigned to their submission (up to 2 out of 18 suggested discipline categories; see Table 1). We repeated the contingency table analysis for each discipline separately and found a significant homophily effect of Senior Editor gender on the gender of the assigned RE in 14 out of the 18 disciplines (78%; Contingency table analysis with FDR correction for multiple comparisons; see Table 1 for details). Fig 2C shows the gender homophily in RE manuscript assignment across all manuscript discipline categories. These results demonstrate that gender homophily in manuscript assignment is a widespread cross-disciplinary effect.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Descriptive statistics and contingency table analysis of senior editor homophily effect across disciplines.

MS–Manuscripts; RE–Reviewing Editor; M–Men; W–Women.

https://doi.org/10.1371/journal.pone.0294805.t001

Previous research suggests that homophily effects negatively associate with the extent of gender imbalance [54]. Accordingly, we next explored associations between homophily in manuscript assignments and gender across disciplines (Fig 2D). For each discipline, we first defined Senior Editor homophily as the difference between (i) the proportion of manuscripts assigned to men REs by men Senior Editors and (ii) the proportion assigned to men REs by women Senior Editors. Intuitively, a value of zero indicates no gender difference between men and women Senior Editor manuscript assignments, whereas a value of unity indicates that Senior Editors only assign manuscripts to REs of their own gender. We similarly defined for each discipline an index that we refer to as the manuscript assignment imbalance, which is calculated as the difference between (i) the proportion of manuscripts assigned to men REs and (ii) the proportion assigned to women REs. Intuitively, a value of zero indicates a fully balanced discipline, whereas a value of unity indicates that manuscripts are assigned exclusively to men REs. Across disciplines, the correlation between Senior Editor homophily and the manuscript assignment imbalance index was negative, albeit borderline in statistical significance (r = -0.47, p = 0.049). A Bayesian correlation analysis also suggested only anecdotal evidence in favour of a negative association (BF₁₀ = 1.77). This result provides preliminary evidence that in disciplines with more equal manuscript assignment, Senior Editor homophily is stronger, in line with previous research [54]. http://www.python.org).

Appeals (Action 5)

In our final analysis, we evaluated the presence of gender imbalances in authors’ appeals (Action 5). In the Initial Assessment stage, only the identity of the Senior Editor is revealed to the authors. The difference in the rates of appeals of manuscripts handled by women and men Senior Editors in the Initial Assessment was marginal; we observed a trend towards fewer appeals over women Senior Editors’ assessments, but this trend did not reach statistical significance (Contingency table analysis, χ²₍₁₎ = 3.781, p = 0.052, Contingency coefficient = 0.013; Fig 1E). Moreover, a Bayesian Contingency table analysis suggested moderate evidence in favour of the null hypothesis (BF₁₀ = 0.28), confirming the lack of difference in Senior Editor gender in Initial Assessment appeals. In the Full Submission stage, the identities of both the Senior Editor and the handling RE are revealed to the authors. Dovetailing with the Initial appeals findings, the gender difference in the rates of appeals for the final decision for both Senior Editors and REs did not reach significance (Senior Editor gender: Contingency analysis, χ²₍₁₎ = 0.34, p = 0.58; RE gender: Contingency analysis, χ²₍₁₎ = 1.69, p = 0.19; Fig 1E). These results suggest that in general, authors’ tendency to appeal does not seem to depend on the gender of the Senior Editor and handling RE. It is important to note, however, that the small rate of appeals limits the robustness of this finding: we observed 809 initial assessment appeals out of 24018 initial submissions (3.4%), and 417 final decision appeals out of 6289 fully submitted manuscripts (6.6%).

The eLife dataset

Before we discuss our key findings, it is important to consider our unique dataset and the potential advantages and limitations inherent to it. As detailed in the Methods section, we used anonymous analytics collected by eLife’s editorial platform for monitoring purposes. This rich dataset reflects a real-life process, and spans a relatively large range of biological disciplines and international contributing scientists and editors. During the investigation period, eLife had a similar fraction of women in their BRE relative to other editorial boards [73], suggesting that the issues identified here are likely to be observed in other journals. However, the specific factors that we could study were not pre-determined based on our experimental needs. Accordingly, we were limited in our explanatory power, both in terms of other relevant factors that might be contributing to the observed effects (e.g., the level of seniority of each RE) and in terms of the statistical power (e.g., authors’ appeals are rather infrequent). To mitigate some of these gaps, we can gain some insight from more recent data relating to the heterogeneity of eLife’s BRE (see Supplemental Section, S1 Fig), although these recent analytics may not fully represent the dataset we analysed here. It is also important to consider the makeup of the BRE; these are invited roles, and as such, all the REs are established in their subfields. However, due to issues we expand upon below, it is possible that women REs are less senior than men REs, as described in the Supplemental data. Unfortunately, the fact that our primary datasets lack direct information on academic attainment levels for all women and men is a limitation. This lack of information should not be interpreted to mean that academic status is equal across genders in our datasets, an assumption that is likely to be incorrect. The sociology of gendered behavior predicts that both academic status and gender likely influenced the outcome of the interaction in which manuscripts were evaluated, as discussed below. We also do not have data on the intersectionality of gender with other primary sources of disparity (e.g., geographic location, race, ethnicity, class, sexual orientation, and ability [79–82]). Yet, the results of a recent eLife self-report survey conducted outside our study period suggest that women serving as editors are more likely to also self-identify as belonging to an underrepresented or minority group based on their race or ethnicity [83]. Finally, as described in the Methods section, perceived gender was assigned as “man” or “woman” (without distinguishing trans from cis) based on the REs’ names and public profiles, and hence may or may not reflect the BRE’s true gender identity. Although eLife recent data suggest that the vast majority of the BRE is cis [83], gender identity was not measured along and outside the binary (e.g., nonbinary, genderfluid, etc.). With these points in mind, our gender effects might be modulated by other contributing factors that should be investigated in future research in greater detail.

Gender disparities

We first considered gender differences in REs bidirectional engagement, including both invitations to contribute to the initial editorial consultation by the Senior Editors and the individuals’ participation in response. We define this process as internally-facing because the identity of the REs involved is only revealed to the other editors engaged in the consultation. We did not find significant differences in the number of invitations of women REs by the Senior Editors to participate in initial consultations relative to men REs. However, we did observe a heavy-tailed, skewed distribution of consultations, such that there is a small group, mostly comprised of men, that disproportionally dominates initial consultations. Even if the differential proportions of these groups are not statistically significant, this small men-dominated group might still skew diversity [84, 85]. To distribute the influence more fairly, a potential solution is to cap the number of consultations per individual RE.

Although the number of initial invitations did not differ between women and men, women engaged less with invitations from the SEs, resulting in the under-involvement of women in editorial activities. Women took only slightly longer to respond relative to men (women were approximately 7 hours slower to answer emailed invitations), but considering the interactive nature of the consultation process, this delay could be meaningful. In the eLife initial consultation process, where editors’ interact in an on-line instant chatting format, this means that men are more likely to set the tone of the discussion by providing their opinion first, making it more difficult for women, on average, to influence the editorial decision (through conformity and anchoring cognitive biases for example [86–88]). It has been previously shown that it is more difficult to voice a different opinion once an opinion has been formed [89, 90]. The delayed response, as well as reduced response rate (by approximately 3%) could potentially be attributed to the fact that women have more duties and responsibilities than men REs. There are multiple reasons to suggest this, depending on women’s specific intersecting identities [43, 91, 92]. For example, senior women are overburdened by administrative responsibilities due to the institutional need to narrow the gender gap [25, 30]. More specifically to our dataset, there is a hint that women REs are at an earlier career stage relative to men (Supplementary section), and hence may be more likely to have children at home than their men colleagues and thus face an added burden on their time [43], or be more laden with obtaining tenure. Another potential contributing factor is the higher standard of communication women are held to in order to receive equal acknowledgment, resulting in an imposed time-consuming quantity/quality trade-off for women, and reducing their productivity [47, 93, 94]. Irrespective of the reasons, our results signal that the journal submission and review process needs to shift away from monitoring decisions based on the decision time, which adds time pressure, and instead could potentially delay discussion and/or decisions about submissions until women have contributed.

We did find a significant difference in the engagement of women REs when considering community-facing duties, particularly when leading the peer-review process. Specifically, women were assigned 9% fewer manuscripts relative to men. This effect is likely exacerbated by the longer response time and less frequent responses we observed during initial consultations, as the assignment of the reviewing RE is often determined during the initial consultation. We are hopeful that if the bias in the previous stage is corrected then the under-assignment of full submissions to women REs will be improved. However, it is also important to consider more carefully other potential sources of bias and how to mitigate them. For example, it is also possible that men might volunteer more readily to take up this time-consuming role–our data does not allow us to shed any light on the inner discussions beyond response time. Regardless, our effect is consistent with other studies showing that women are disproportionately engaged in internal-institutional facing duties, whereas men are disproportionately engaged in community-facing roles, which are also more associated with eminence, networking, and other benefits related to the more visible duties of the reviewing RE leading the peer review [11, 12, 14]. The reasons underlying this pattern should be further studied, however women’s different time allocation may reflect a purposeful choice to contribute to their institutions. Another potential driver could be inherent biases of the Senior Editor assigning the RE; research shows that women are less frequently approached to apply for awards, write invited reviews, etc. [11, 13, 19, 21, 75]. Within the context of editorial assignments, this effect could be potentially corrected by providing gender-specific statistics to the Senior Editors about disproportional engagement by gender. We turn to consider gender-based interactions between the Senior Editor and REs in the next section.

Homophilic behaviours

Homophily is one of the fundamental patterns underlying human relationships across multiple social systems, influencing how communities form, how status is distributed, and how subgroups evolve in occupations and organizations [95]. With respect to the homophily effect of the Senior Editor’s gender on REs assignments, we find that across multiple sub-disciplines, there is a significant tendency for Senior Editors to choose same-gender REs to handle full submission for peer reviews. One might wonder whether the observed homophily effects might be explained by field-specific differences in gender proportions: in a discipline comprised mostly by men, e.g. physics of living systems, the Senior Editor (likely a man) will more often reach out to more men simply because most of the experts are men. To evaluate this possible explanation, we separated our data by discipline. We found that the homophily effect exists quite broadly, across 14 of the 18 disciplines (despite noticeable variability in the proportion of women/men RE across disciplines, see Fig 2C), hence refuting the differently-gendered subdisciplines account. What other drivers could potentially explain the homophily effect? Homophily is driven by various types of associations and dimensions of similarity [96], such as ascribed attributes (e.g. gender [97]), acquired attributes (e.g. occupation [98]), values, attitudes, and beliefs (e.g. activism [99]). Homophily, and gender homophily in particular, are prevalent in academia, for example in shaping interactions in scientific conferences [100], affecting scientific collaboration and scientific societies [11, 101], and biasing the selection of Nobel laureates [13]. Thus, we were not surprised to find that men Senior Editors assign more men REs than the women REs, even after taking into consideration the larger numerical proportion of men in the BRE.

It is possible that homophily in women arises from different drivers than homophily in men [67, 102], due to distinct social processes [103, 104] and the roles they play in intersectional power structures [105]. Considering the current political climate where there is greater awareness for the under-representation of women in STEMM, it is possible that women Senior Editors adopt an informal policy to engage women REs disproportionately. In this respect, the women homophily offsets to some degree the gender bias we see in the editorial process. Activism-driven homophily among women was demonstrated for example in crowdfunding of start-up projects, whereby a small proportion of women backers disproportionately supported women-led projects in areas where women are historically underrepresented [99]. Similarly, gender homophily in reviewer assignment by journal editors was widespread among men editors, while for women only a small number of highly homophilic editors dominated [67]. Our data did not allow us to directly explore the prevalence of homophily among individual REs, yet the fact that homophily was widespread across many fields, involving different REs, suggests women homophily is a broad phenomenon in eLife. Additionally, we find that homophily increases with gender balance across sub-disciplines. This echoes the finding that men homophily in article citations increases as the research field gets more gender balanced with time [54]. However, given that women Senior Editors are outnumbered by men (for example, 36% (30) women vs. 64% (52) men Senior Editors in 2021), on average we see an over-engagement of the men REs, even after accounting for their numerical dominance in the BRE. One simple candidate intervention is to increase the proportion of women in senior roles, which could also potentially serve to address other aspects of gender disparity that we did not study here. However, for the reasons detailed above, simply increasing representation (e.g., the number of women) might not be sufficient to ensure inclusion, equity, and justice [11, 67, 99, 106, 107].

Despite the fact that women display homophilic tendencies that serve to partly balance the homophilic tendencies of men, we do not in general endorse homophily effects as an appropriate solution to the gender bias observed here, as it can have devastating trickle-down consequences. For example, it was previously shown that scientific journal editors of both genders were more likely to appoint reviewers of the same gender as themselves [67]. Moreover, a previous study of eLife editorial decisions focused on how the gender makeup of the participants in the peer-review stage–both editors and reviewers–biases acceptance rates for men and women authors [53]. It was observed that all-men reviewer teams are far more likely to accept men-led manuscripts. Therefore, the homophilic behaviour that we observe among men is likely to exacerbate these effects and increase the gender publishing gap. More generally, it was shown that homophilic groups tend to have similar evaluations and mind-sets [67, 108, 109]. Hence, the uncontrolled effects of homophily may undermine the impartiality of peer-review, and thus undermine science [67, 110]. Instead, solutions should be driven by formal policy that foreground equity and justice. For example, the homophily factor could be monitored to help Senior Editors avoid implicit and explicit biases. Another important candidate intervention for this issue is to diversify the network of the Senior Editors within the BRE.

eLife’s peer review process

eLife holds a unique two-stage evaluation process, as detailed in Fig 1A. The first stage is the initial assessment, and the second stage is peer review. We will describe each in turn, along with the series of actions it comprises.

Data

Data accumulated by eLife’s platform for science publishing over the years 2017–2019 were organised into two datasets, as summarised in Table 3. The first dataset will be referred to as the BRE dataset, and the second will be referred to as the Manuscript dataset. We will describe each in turn. But first we make a note on assigned gender.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. eLife datasets.

Top: BRE Dataset: contains information relating to the engagement of individual BRE members in the editorial process (identified by gender and year). It includes the following fields: The mean number of days until the Reviewing Editor (RE) responded to a Senior Editor‘s request to participate in the Initial Assessment stage (Days to respond); The RE response rate to Initial Assessment consultation requests (Response Rate); The mean number of consultation requests per month each RE received (# Requests per month); The mean number of full submissions per month each RE handled (# Full submissions per month); The keywords associated with each RE to showcase their expertise (Keywords). Note that the number of full submissions may contain papers that the REs had handled as Guest Editors in the year prior to joining the BRE. Also, some REs may have been on leave, and therefore may have not been consulted for a certain period. Bottom: Manuscript Dataset: contains information relating to each manuscript submission, detailing the manuscript’s outcome in each of the reviewing process stages (identified by gender of the Senior and Reviewing Editors). It includes the following fields: The proportion of men BRE members suggested by the authors (% of Men BRE members); The gender of the Senior Editor handling the manuscript throughout the reviewing process (Gender of Senior Editor); The gender of the RE handling the manuscript in the Full Submission stage (Gender of handling RE); The rate of author appeals at the Initial Assessment stage in which only the Senior Editor identity is revealed to the authors (Initial appeal rate); The rate of author appeals at the Full Submission stage in which both the Senior and Reviewing Editors’ identities are revealed to the authors (Initial appeal rate); The two discipline terms the authors chose, out of 18 possible terms (Discipline 1 & Discipline 2; see Table 1 for details).

https://doi.org/10.1371/journal.pone.0294805.t003

Gender assignment. In all cases, Editor gender was assigned by eLife’s staff based on the editor’s name and gender expression. Note that staff (i) assigned a binary “man” or “woman” gender, (ii) did not distinguish between trans and cis identities, and (iii) did not assign other genders such as nonbinary, genderqueer, agender, or genderfluid. Note that any editor could have a gender different from the one that was assigned, and that true gender may or may not be more widely known by the community for several reasons: (i) scientists might be closeted due to the pervasive violence and discrimination faced by gender minorities, (ii) scientists might share their true gender identity only with a few close colleagues or friends, or (iii) scientists might share their identity freely but because of the complexity of the social network landscape in science, that information may not have reached all other scientists in their field. Accordingly, the staff’s assignment of gender therefore reflects not self-identity but rather the perceived binary gender of the person. This perception is likely to also be held by the majority of the broader community, and hence is particularly relevant to understanding how the editor might be treated by that community (e.g., the frequency with which they might be suggested as a Reviewing Editor by authors). We also note that since early 2020, eLife has given all Senior and Reviewing Editors the option of sharing their self-reported gender identity via a confidential survey. However, the current response rate (~40%) precludes a comprehensive analysis of gender disparities using the data at this stage [83].

BRE dataset. This dataset includes anonymous information relating to the engagement of individual REs in the editorial process. This information includes the start and end dates of their editorial contracts, the number of consultations in which they have been invited to participate, how responsive they are to consultation requests (number of responses and response rate), the number of full submissions assigned, and how many days they take to make an editorial decision. In addition, the editorial staff asks REs to provide a set of keywords that reflect the scope of their research, which was also included in this dataset. Note that the terminology was self-generated by the REs (rather than adopted from an existing database), and that there were no limitations on the number of keywords each RE could provide. For some REs, additional keywords are added by the editorial staff based on the information publicly available on the editors’ academic websites.

Manuscript dataset. This dataset includes information relating to each manuscript submission, detailing the manuscript’s outcome in each of the submission stages. This dataset also contains the assigned gender (as described above) of those BRE members that were suggested by the authors, the recorded gender of the handling RE, and the recorded gender of the assigned Senior Editor. Note that here our information regarding gender pertains only to the editorial team handling the manuscripts and not to the manuscript authors, whose identities were not made available for the present study due to ethical considerations (though we note that the authors’ identity, but not necessarily their self-defined gender, was known to the editors involved in the assessment). Manuscripts with appeals received after the Initial Submission and without a Full Submission decision were most likely rejected prior to review. It is possible that a small fraction of manuscripts was withdrawn prior to evaluation; however, we did not have access to such data.

Additionally, this dataset contains up to two (out of 18) disciplines that the authors assigned to their manuscript upon submission. Options included ’Neuroscience’, ’Cell Biology’, ’Developmental Biology’, ’Structural Biology and Molecular Biophysics’, ’Microbiology and Infectious Disease’, ’Biochemistry and Chemical Biology’, ’Chromosomes and Gene Expression’, ’Genetics and Genomics’, ’Computational and Systems Biology’, ’Immunology and Inflammation’, ’Cancer Biology’, ’Medicine’, ’Evolutionary Biology’, ’Physics of Living Systems’, ’Plant Biology’, ’Ecology’, ’Epidemiology and Global Health’, and ’Stem Cells and Regenerative Medicine’ (Manuscript Dataset; see Tables 1 and 2). In order to analyse the manuscript data across disciplines, we assigned to each discipline all the manuscripts in which a discipline was chosen at submission. This process created some overlap between disciplines (6289 fully submitted manuscripts; 1979 manuscripts were assigned to two disciplines out of 8268 assigned manuscripts, or 23.9%).

Data analysis

We applied several exclusion criteria to the data before proceeding with further analysis. In the BRE dataset, we excluded REs who became Senior Editors, or resigned as Senior Editors (and became REs) in a given year, or those who were inactive (i.e., were never contacted on initial submissions). In addition, in the manuscript dataset, we limited the number of author-suggested REs to five per manuscript; and excluded papers handled by guest editors as well as Research Advances, Registered Reports, and formats that go through a different workflow.

Statistical analysis

Results are reported as mean ± standard deviation (StD). Owing to several non-normal distributions in the data, we used non-parametric tests in all analyses. Binomial tests and N-1 χ² proportion comparison tests were performed to compare one or two proportions using JASP (JASP Team (2020) Version 0.14) and MedCalc online tools (MedCalc Software, Ostend, Belgium), respectively. Contingency table analysis was used for testing the interrelation between binary variables using JASP software. When comparing the means of two groups with unequal sample sizes, we used a permutation-based Welch’s independent t-test (10,000 permutations) in MATLAB (PERMUTOOLS package, The Math Works, Inc. MATLAB. Version 2020a, The Math Works, Inc., 2020. Computer Software. www.mathworks.com/). Pearson correlation coefficients were computed using JASP in order to test the association between continuous scale variables, after checking for normality assumption violations using the Shapiro-Wilk test for bivariate normality. When relevant, all tests were conducted using 2-tailed tests. We used Hedges’s g to compute effect size when comparing two means in a permutation Welch’s t-test, and Cohen’s h when comparing two proportions in a N-1 χ2 proportion comparison test. When effects were close to the critical alpha (p<0.05), we conducted equivalent Bayesian analyses, with default prior settings (Bayesian correlation stretched beta prior width = 1; Bayesian Contingency tables, prior concentration = 1) using JASP to test whether there was more evidence for H₀ or for H₁. In order to measure BRE members service contribution (BRE service) as a function of gender, we computed the number of months in which the RE was affiliated with the BRE per year, i.e. the proportion of months of service of women and men REs out of the total number of service months, thus accounting for variability in BRE service contract durations and partial work time (e.g. REs appointed in the middle of the year, being on leave).

To investigate whether gender disparities were associated with REs’ expertise, as advertised by eLife to prospective authors, we conducted an analysis of the relative scope and reach of the REs’ keywords, broken down by the recorded gender of the RE. Keywords for each RE were extracted and strung together using the ‘OR’ operator and then queried against the PubMed database through NCBI’s public API—‘Entrez Programming Utilities (E-utilities)’ (Entrez Programming Utilities Help [Internet]. Bethesda (MD): National Center for Biotechnology Information (US); 2010-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK25501/). The number of search results for each set of RE keywords was recorded and used as a measure of the reach of the keywords provided by the REs, as evidenced by published papers related to the keywords in the literature. The E-Utilities API was accessed through a script in Python (Python Software Foundation. Python Language Reference, version 3.9.6. Available at

Diversity statement

Recent work in several fields of science has identified a bias in citation practices such that papers from women and other minority scholars are under-cited relative to the number of such papers in the field [54, 56–63]. Here we sought to proactively consider choosing references that reflect the diversity of the field in thought, form of contribution, gender, race, ethnicity, and other factors. First, we obtained the predicted gender of the first and last author of each reference by using databases that store the probability of a first name being carried by a woman [54, 111]. By this measure (and excluding self-citations to the first and last authors of our current paper), our references contain 30.62% woman(first)/woman(last), 22.82% man/woman, 18.14% woman/man, and 28.42% man/man. This method is limited in that a) names, pronouns, and social media profiles used to construct the databases may not, in every case, be indicative of gender identity and b) it cannot account for intersex, non-binary, or transgender people. Second, we obtained predicted racial/ethnic category of the first and last author of each reference by databases that store the probability of a first and last name being carried by an author of color [112, 113]. By this measure (and excluding self-citations), our references contain 6.21% author of color (first)/author of color(last), 15.01% white author/author of color, 16.03% author of color/white author, and 62.75% white author/white author. This method is limited in that a) names and Florida Voter Data to make the predictions may not be indicative of racial/ethnic identity, and b) it cannot account for Indigenous and mixed-race authors, or those who may face differential biases due to the ambiguous racialization or ethnicization of their names. We look forward to future work that could help us to better understand how to support equitable practices in science.