Women’s visibility in academic seminars: Women ask fewer questions than men

The attrition of women in academic careers is a major concern, particularly in Science, Technology, Engineering, and Mathematics subjects. One factor that can contribute to the attrition is the lack of visible role models for women in academia. At early career stages, the behaviour of the local community may play a formative role in identifying ingroup role models, shaping women’s impressions of whether or not they can be successful in academia. One common and formative setting to observe role models is the local departmental academic seminar, talk, or presentation. We thus quantified women’s visibility through the question-asking behaviour of academics at seminars using observations and an online survey. From the survey responses of over 600 academics in 20 countries, we found that women reported asking fewer questions after seminars compared to men. This impression was supported by observational data from almost 250 seminars in 10 countries: women audience members asked absolutely and proportionally fewer questions than male audience members. When asked why they did not ask questions when they wanted to, women, more than men, endorsed internal factors (e.g., not working up the nerve). However, our observations suggest that structural factors might also play a role; when a man was the first to ask a question, or there were fewer questions, women asked proportionally fewer questions. Attempts to counteract the latter effect by manipulating the time for questions (in an effort to provoke more questions) in two departments were unsuccessful. We propose alternative recommendations for creating an environment that makes everyone feel more comfortable to ask questions, thus promoting equal visibility for women and members of other less visible groups.


Introduction
Women account for 59% of undergraduate degrees, but only 47% of PhD graduates, 45% of fixed-term contract postdoctoral researchers, 37%  1]. The decreasing representation of women in academia as careers progress is frequently referred to as the "leaky pipeline" [2]. Many factors have been proposed to explain the attrition of women as academic careers progress, including innate differences in ability; differences in the career preferences of men and women; the assessment of women's CVs for hiring, tenure and promotion; differences in males' and females' salaries for equivalent positions; parenting; imposter syndrome; and a lack of appropriate role models and mentors for women, all of which lead to reduced visibility of women in academic science [reviewed in 1,3].
Social role theory provides a framework to understand how various factors might influence individuals' decisions to choose an academic career. According to social role theory [4], people tend to make inferences about which characteristics are needed to be successful in a given role by examining the characteristics of the people who most predominantly occupy that role. Because women are often underrepresented in the later career stages in academic science, it is possible that women (and other underrepresented minority groups) might infer that they do not possess or want to express the relevant characteristics for senior faculty positions and therefore do not belong in those particular careers, as has been shown in the medical field [5]. Furthermore, when people do not have first-hand knowledge of their own level of performance in a given domain, they look to the performance of similar others (i.e., ingroup members; in this case other women) to gauge their own potential likelihood of success in that domain [e.g. [6][7][8]. For these reasons, observing successful models, with whom one can easily relate, is critical for encouraging larger numbers of underrepresented group members to enter and remain in that field [9]. In the case of the "leaky pipeline" for women in academic science, then, the degree to which other women are visible becomes an important problem that needs to be addressed.
In addition to a general pattern of gender inequality in academic posts, women and men-and their contributions-may not be equally visible or equally valued. For example, men are overrepresented in terms of authorship, especially first, senior, and sole authorship [10][11][12][13] and men's papers are cited more often [10,14]. In addition, when considering contributions to individual papers, women were more likely than men to be credited with performing the experiments (i.e., the more physical part of the process), whereas men were more likely than women to be credited with data analysis, experimental design, contributing tools, and writing (i.e., the more conceptual parts of the process) [15]. Just as many factors have been proposed to explain the leaky pipeline, various factors have been cited to explain these differences in the representation of women and men in academia.
For example, the difference in citations has been explained in part by the fact that women cite themselves less often than men do, and men cite other men more than they cite women [14].
Although publications represent one form of conceptual "visibility" for scientists, there are many other forms, including some more literal. Direct interactions involving groups of scientists are likely to have a stronger influence on shaping an individual's impression of the academic community.
One forum where this occurs is at international conferences, where differences in visibility are known to occur: women are less likely than men, and less likely than expected given their proportional representation in a field, to give talks at conferences, and more likely to contribute to less prestigious (and less visible) alternatives, such as posters [16][17][18]. Although some part of this underrepresentation may be due to selection bias, other explanations have been proposed; for example, women are more likely to decline invitations to give a talk [18], and more likely to seek out shorter rather than longer talks [19]. Another way in which women are less visible at conferences is in their question-asking behaviour: a small number of studies have reported that women ask proportionally fewer questions than men at these events [20][21][22].
In this study, we examine a form of visibility that is more common and frequent, and apparent earlier in the pipeline (i.e., to junior academics): question-asking behaviour at local departmental academic seminars (i.e., talks, presentations, colloquia, etc.). Social role theory suggests that women should benefit from being exposed to successful ingroup role models at all points along the leaky pipeline. Before attending academic conferences and seeing women present their work, and before gaining a familiarity with the authors of papers in a particular research area, undergraduate and postgraduate students are exposed to the role-modelled behaviours of the women and men who work in their department. Given social role theory explanations for how gendered expectations of certain roles develop based on who is seen occupying those roles, we argue that the behaviour of the local community may play a formative role in identifying ingroup role models at an early career stage. Few studies have investigated such local phenomena, but these reveal a potential bias against women.
For example, female undergraduate students are less likely to volunteer to answer an instructor's questions in class, and somewhat less likely to pose their own questions [23]. Such differences in behaviour might emerge through reinforcement: during the early years of schooling girls are slightly more likely than boys to raise their hands to ask a question but teachers are less likely to choose them to answer [24].
Our aims were to determine whether women and men differ in their visibility at academic seminars and which factors might underlie any differences. With regards to the first aim, we tested the hypothesis that women would ask fewer questions at departmental seminars, thus limiting their potential visibility to others. We were interested in individuals' actual question-asking in seminars, to quantify directly any disparity that might exist. With regard to the second aim, we were interested in perceptions of question-asking in seminars, to understand the motivations and beliefs that underlie any disparity. Thus, our data collection also took two approaches. First, we ran an online survey that collected data on over 600 academic respondents' self-reported attendance and question-asking in seminars, their perceptions of others' question-asking behaviour in seminars, and their beliefs about why they themselves and others do and do not ask questions in seminars. Second, we collected observational data at almost 250 seminars in 10 countries to quantify the attendance and questionasking behaviour of women and men in departmental seminars.
Using these two data sets, we asked three questions. First, we asked whether there was a gender disparity in the question-asking of audience members in academic seminars (Question 1).
Using data collected in the survey, we asked academics whether they perceived a disparity in women's question-asking in seminars (Q1a). We also used our observational data to describe women's and men's actual question-asking behaviour at seminars (Q1b). Second, we aimed to understand why there is a disparity in women's question-asking in academic seminars (Q2). Using the survey data, we asked both women and men why they did not ask questions when they wanted to, and for those that thought there was a gender disparity in question-asking, we asked why they believed there to be a disparity (Q2a). Next, we used our observational data to identify factors associated with the disparity (Q2b). Finally, we aimed to explore ways of addressing the disparity (Q3). Based on preliminary findings from the first year of our observational data collection, we ran an experiment in two departments to manipulate the time given to questions, in an attempt to promote a gender balance in the audience's question-asking (Q3a). We also asked the survey respondents what they thought could be done to ameliorate the gender disparity (Q3b).

Online survey: Seminar participation and perceptions
The survey received ethical approval from the Science and Health Faculty Ethics Sub-Committee of the University of Essex. Participants declared their consent prior to participation and could withdraw from the survey at any time or leave any question unanswered. After completion, participants were briefed about the purpose of the study and provided with contact information in case they wanted further details. No identifying information was collected during the survey, and all data were pooled prior to analyses. To ensure data privacy, the survey was administered through Qualtrics (from an institutional account at the University of Cambridge).
The survey was advertised via social media (Twitter, Facebook) and emails to relevant academic groups, and was active between 16th June 2016 and 22nd August 2016.
The survey asked for details on the participants (gender, academic subject, career stage, country), the structure of academic seminars at their institution (e.g., typical length of time for questions), and their own attendance and question-asking behaviour at seminars. Finally, we asked for their impression of any gender disparity in question-asking and potential reasons for it (for the full survey design see Supplementary Material 1). We disguised our specific interest in a gender disparity by also asking whether question-asking behaviour was related to seniority, confidence, extraversion, and competence. Data on these distractor questions were not analysed in this study.

Observation of seminar participation
To determine the extent of the gender disparity in question-asking during academic seminars, we observed seminars and recruited colleagues through personal contact to do the same. Because these data were collected passively at public events, ethical approval was not needed (following https://memforms.apa.org/apa/cli/interest/ethics1.cfm). Observers were in the same fields as the authors (biology or psychology), chosen to represent as much geographic distribution as possible; they were based in 10 different countries and 35 different institutions. We solicited observers' help by explaining the motivation for the study and our preliminary findings (see Supplementary Material 2).
In the end, more than 90% of people that were invited to act as observers reported observations. Data were collected opportunistically during seminars that the observers normally attended in their institutions and these seminars are therefore likely to be a representative sample of the broader experiences of academics.
We provided all observers with written guidelines prior to the start of their observations (see Supplementary Material 2). During the initial period of observations at the University of Cambridge, two of us (AJC and DL) attended six seminars together but independently scored them. This yielded identical observations regarding the gender of the first person to ask a question and the total number of questions asked by each gender, and the counts of the audience numbers were within 0-2 people, suggesting that the guidelines are sufficiently specific for comparison across observers.
For each seminar, observers recorded: whether the speaker was an external visitor or affiliated with the hosting institution; the gender of the speaker; the start and end time of the presentation, and the start and end time of the question period after the presentation; the number of women and men in the audience; the number of questions asked by women and by men; and the gender of the person asking the first question. Each observer recorded the number of women and men among the faculty of the hosting department based on the teaching staff listed on the institution's official website.
We recorded gender as perceived by the observer. This is likely to reflect the perception of other audience members, but we acknowledge that this may not match the target's gender identity.
As we wanted a measure of the potential opportunities for the visibility of each gender, observers recorded the total number of questions (including multiple questions from the same person), rather than the total number of different people asking questions. This is because after most talks, there is a limited amount of time for questions; multiple questions asked by the same questioner therefore raises the visibility of that particular gender in proportion to the number of questions asked.

Experimental manipulation of time given to questions
We (AJC and DL) collected preliminary observations of question-asking from the University of Cambridge during the 2014-2016 academic years (N = 62, comprising 18, 18, and 26 seminars in each of three departments). These data indicated a correlation between the number of questions asked and the imbalance in questions, with the imbalance approaching 0 as more questions were asked (linear mixed effects model with department as a random effect: β ± S.E. = 0.02 ± 0.009, t = 2.02). Based on this preliminary finding, we hypothesised that we could increase the number of questions asked by women by increasing the amount of time devoted to questions after seminars.
We thus designed a manipulation at two institutions to test whether decreasing the length of talks (and thus, theoretically, increasing the time allotted to questions) would lead to more equal questionasking from male and female audience members. While these seminar series previously had indicated to speakers that presentations should last for about 45 minutes, during the manipulation we asked speakers in the invitation email to plan for their talk "to last for 40 min with 20 min for questions. This format is designed to encourage a more discursive and inclusive question session in our department."

Data and analyses
Our data and analysis scripts are available in the institutional repository of the Max Planck Society at https://dx.doi.org/10.17617/3.12. Our analyses were conducted in R v3.2.2. For each, we list the approach and specifications in the results below. Generalised and linear mixed models were analysed using the lme4 package [25]; because this package does not report p-values for linear mixed models, we considered t-values over 1.94 as statistically significant and report these below.

Descriptives
In total, 600 people provided consent and started our online survey, and 518 (90%) recorded a response when asked their gender (the last question in the survey), including 303 (58%) women, 206 (40%) men, 4 transgender/non-binary, and 5 who preferred not to report their gender. We restricted our analyses to the responses of women and men given the small number of respondents who did not consider themselves within these categories, resulting in a sample of 509 responses for our analyses. Survey respondents were from the academic community: 2% were undergraduates (N = 12), 38% were post-graduates (N = 192), 20% were postdoctoral researchers (N = 102), 5% were research fellows (N = 26), 29% were faculty (N = 150), 5% were "other" (N = 27). The participants who completed the online survey were from 19 different countries (9 participants did not provide information about country) and 28 fields of study (28 participants did not provide information about field of study). The majority of respondents who indicated their field of study (74%; N = 356) were from the same fields as the authors of this study: biology and psychology.
Observational data were collected at 247 seminars, from 42 departments of 35 institutions in 10 countries. We retained the pilot data collected at the University of Cambridge and the seminars that were subject to the experimental manipulation, since we found no effect of our manipulation on the time given to questions (see below).

The current culture of academic seminars
We first aimed to describe the general patterns of academics' attendance at and questionasking in departmental seminars. Overall, most people reported in the online survey that they

Gender differences in attendance and question-asking behaviour
There was no difference between men and women in self-reported frequency of attendance, In general, men and women did not differ in their motivations for asking questions; approximately equal proportions of men and women reported being motivated by an interest in the subject (92% of men; 92% of women), the need for clarification (67% of men; 64% of women), the desire to act as a model for more junior academics (32% of men; 31% of women), or to establish a connection with the speaker (26% of men; 30% of women), t's < 1.10, p's > 0. 25

Q1: Is there a gender disparity in participation in academic seminars?
We aimed to quantify whether academics perceive a gender disparity in the proportions of men and women who ask questions in seminars, and whether this perception differs according to gender. Most respondents reported that gender played a role in who asked questions at seminars, reporting that they believed that men were more likely to ask questions (N = 279 (58%); see Fig 1C). However, men and women differed in their endorsement of this belief; women reported more frequently than men that they believed there was a bias towards men asking questions (N = 182 women (60%) vs. 97 men (47%); ! 2 (2) = 8.40, p = 0.01).
These perceptions about a gender disparity in question-asking were borne out by the selfreport data. Men and women differed in how frequently they reported asking questions, ! 2 (4) = 21.71, p < 0.001: women self-reported asking questions less frequently than men (see Supplementary Material 4 results, Fig S4.1B). Despite this, the vast majority of respondents of both genders reported that they sometimes did not ask a question when they had one (N = 277 women (91%); 189 men (92%); overall 92%).
We next examined whether the observational data substantiated these perceptions and selfreports of a disparity in women's question-asking after seminars. To test whether the proportion of questions asked by women differed from the proportion of women present in the audience, we ran a two-tailed t-test comparing the difference in these proportions to 0 (no difference). Survey respondents' (especially women's) general belief that men ask more questions than women was supported by the actual behaviour observed in seminars: proportionally fewer women asked questions after seminars than would be expected given the proportion of women in the audience (M = -0.19, 95% CI = -0.16, -0.22, t(245) = -12.55, p < 0.001, Fig 1A,B). Put another way, male attendees were over two and half times more likely to ask a question than women attendees (odds ratio = 2.57) during the seminars that we observed.

Fig 1 title:
The gender "disparity" in question-asking points falling in the upper green half indicate a disparity towards women audience members. Indicated are two seminars that fall in different categories. The green arrow indicates a seminar with a bias towards questions from women, in which the proportion of women in the audience was 0.38, but the proportion of questions asked by women was 0.67. Conversely, the orange arrow indicates a seminar with a bias towards questions from men, in which the proportion of women in the audience was 0.78 but the proportion of questions asked by women was 0.40. Panel (b) shows the frequency at which the disparities were observed, with orange bins indicating seminars with questions disproportionately asked by male audience members and green bins indicating seminars with questions disproportionately asked by female audience members. In both panels, the red line indicates no disparity (i.e., the proportion of the women in the audience matched the proportion of questions asked by women). Panel (c) shows the proportions of female (green) and male (orange) respondents who indicated that they believed that men or women asked more questions in seminars, or that questions were asked equally by men and women.

Q2: Why is there a disparity in question-asking behaviour?
We next aimed to understand why there is a disparity in women's question-asking at seminars. The vast majority of our online survey respondents (91% of women and 92% of men) reported sometimes not asking a question when they had one. We asked them what prevented them from asking a question in these cases on a Likert scale from 1 (not at all important) to 5 (extremely important). The results are summarised by gender in Table 1 (for detailed results, see Table 1 Table 1; Fig 2). Women rated all the reasons as more important than men did (except for a lack of time, which men judged as more important than women) suggesting that women rated 'internal' factors as more limiting than men.
We  Presented are the factors; the results of a Welch two-sample t-test, including the t-value (t), degrees of freedom (df) and the p-value (p); and the means and standard deviations (M ± SD) of the responses of respondents who identify as women and men.

Fig 2 title:
Mean importance assigned by women and men to (1) each reason why they themselves have not asked a question in a seminar when they wanted to, and to (2) each reason men and women believe women do not ask questions when they want to

Fig 2 caption:
Shown are the mean values for women (green) and men (orange) rating how important each factor is in restricting why they themselves did not ask questions when they wanted to (circles). For the respondents who reported a belief that women ask fewer questions than men, shown are the mean values for women (green) and men (orange) rating how important each factor is in restricting women from asking questions when they wanted to (triangles).
We also asked respondents who had indicated a belief that women ask fewer questions than men why they believe that women do not ask more questions. Women rated each reason we asked them about as more important than men did, except being intimidated by the speaker (Table 1; Fig 2; for detailed results, see Table 2 in Supplementary Material 3). For example, women not feeling clever enough to ask a question was rated as more important by women (M = 3.21, SD = 1.13), than by men (M = 2.57, SD = 1.17) ( Table 1).
Next, using our observational data, we examined potential predictors of a gender disparity in the questions asked after seminars. We used generalised linear mixed effects models with a binomial response, with questions from female audience members coded as cases, and questions from male audience members coded as noncases. To control for repeated measures, all models included the country, and the department nested within the institution as random effects. We did not include the observer as a random effect because most observers collected data within only one department within an institution.
We aimed to test the following fixed effects: the proportion of women in the audience (centred at 0.50), to estimate whether differences in the number of questions asked by women and men reflect differences in individual contributions rather than just their share of the audience; the gender of the speaker (female or male), to understand, for example, whether attendees might feel more comfortable asking a question of a person of the same gender; the gender of the first person to ask a question (female or male) to understand whether a social role model effect might occur within sessions (see below); the total number of questions asked (centred at the median of 6 questions) and the duration of the question time (centred at the median of 12 min) to understand whether perceived or real competition over asking one of the questions limited some individuals; the hour of the day that the seminar started (integer ranging from 10 to 18) as childcare needs differ throughout the day; the proportion of the permanent staff (faculty) in the host department who were female (centred at 0.50) to understand whether gender biases among individuals asking questions were associated with seniority; the number of attendees to understand whether the genders differed in their response to the size of the audience for their question; the field of study (broadly characterised as biology, psychology or philosophy, based on the department in which the talk took place) to understand whether differences in norms or gender roles in different fields influenced participation; and whether the speaker was internal (i.e., from within the department) or not to understand whether familiarity with the speaker influenced who asked a question. Unsurprisingly, there was covariation between the duration of the question time and the number of questions that were asked (generalised linear model with the number of questions as the response and a Poisson link: β ± S.E. = 0.029 ± 0.0022, z = 13.03, p <0.001); we thus used the number of questions rather than the duration for questions, but found qualitatively similar results when using the number of questions asked (see below).
We also included a number of interactions that we predicted a priori could contribute to the disparity. Because gender differences in the speakers' behaviour may induce different behaviour from the audience members, we tested whether the speaker's gender also interacted with (a) the total number of questions asked and (b) the number of attendees to affect the gender disparity in the questions asked. In addition, because the first person to ask a question may set the "tone" for the subsequent (disparity in) questions asked, we investigated the interaction between the gender of the first person to ask a question and (a) the total number of questions asked and (b) the gender of the speaker. Such social influence biases have been found in online interactions, where, for example, the tone of the first comment posted influences the tone of subsequent comments [26]. This resulted in a total of four interactions.
Because we had a large number of a priori predictors and our modelling approach was exploratory in nature, we used stepwise model simplification to obtain minimal models whose retained components significantly explained the variation in the response (the probability that a question was asked by a female audience member). We thus started with models that included a set of predictors (from those listed above, described below) and interaction terms, and then used backwards elimination of non-significant terms until a minimal model remained that explained the variation in the gender disparity in questions. Then, each dropped term was added back to the final model, one at a time, to check that it remained a non-significant predictor of the gender disparity.
In predicting the proportion of questions asked by women, we could not include the gender of the first person asking a question, since the first person biases the overall gender ratio of questions, in particular when only few questions were asked. We thus ran two sets of analyses using slightly different data and predictors in the starting models to account for this. The first model included the complete dataset and all fixed effects and interactions not including the gender of the first attendee to ask a question. The second model used a reduced dataset, with the first question removed, and included the gender of the first person asking a question as an additional predictor.
Using the complete dataset, we found that the probability that a question was asked by a female audience member was predicted by the proportion of the audience that was female, the proportion of female faculty in the department, the number of questions asked, the gender of the speaker, and whether the speaker was internal or not (    The values for the non-significant terms (i.e., that were dropped during the model simplification procedure), representing the effect size of the terms when they were added back individually to the minimal model, are reported for completeness.

Fig 3 title:
The effects predicting the probability of question asked by a female audience member after departmental seminars

Q3: Is there a way to address the gender disparity in question-asking behaviour?
We asked the survey participants who had indicated that they sometimes do not ask questions how important several factors could be in encouraging them to ask their questions at seminars (Table   3; for detailed results, see Table 3 in Supplementary Material 3). Respondents indicated that the factors most likely to encourage them to ask more questions were having more confidence (M = 3.53) and having an opportunity to ask the question in person (M = 3.48). The factors they thought least likely to encourage them to ask more questions were having a moderator (M = 2.29), or having a better moderator that engages the audience (M = 2.60). Women were more likely than men to think that all of the factors we listed would encourage them to ask more questions (Table 3).  Presented are the factors; the means and standard deviations (M ± SD) of the responses (ordered from highest mean to lowest); the results of a Welch two-sample t-test for differences between the genders in their responses, including the t-value (t), degrees of freedom (df) and the p-value (p); and the means and standard deviations of the respondents who identified as women and men.
It is possible, however, that people are not aware of factors that might actually be helpful. In order to uncover factors that could potentially be targeted to increase the number of women asking questions, we ran a series of multiple linear regressions on women survey respondents only, predicting how often they reported asking questions. First, we ran a model in which we entered . This result suggests that manipulating the talk duration would not result in a change in the time dedicated to questions. Therefore, the manipulation may have been more successful had we aimed to manipulate directly the time dedicated to questions rather than indirectly trying to affect this by manipulating the talk duration.

Discussion
The visibility of women role models at all career stages is important for redressing problems of the leaky pipeline. Our results add to a growing body of evidence showing that women are less visible than men, both conceptually and literally, in various scientific domains. Other studies have reported a similar bias in visibility, with men participating more already in school classrooms [23,27,28], at conferences [21,29], and public events [30]. Here, we report an underrepresentation in the literal visibility of women in a new domain: asking questions at departmental seminars. Our data show that a given question after a departmental seminar was more than 2.5 times more likely to be asked by a male than a female audience member, significantly misrepresenting the gender-ratio of the audience which was, on average, equal. These results are important because this gender disparity is observable particularly early in the career pipeline: junior academics are likely to observe the question-asking behaviour of the men and women in their department before they ever attend a conference, or become familiar with the researchers publishing in their area of interest. Below we briefly discuss the implications of our findings for women's attrition in academia, before addressing some limitations of our study and recommendations for increasing women's visibility at these events.
The lack of visible female role models asking questions at departmental seminars is likely to be both a symptom of the leaky pipeline and a cause of that same problem. As we explained earlier in this paper, research on role modelling suggests that having access to successful ingroup role models (e.g., women in senior levels of the academy) can be a key factor in determining what course of study or occupation a person will pursue [6,9], and, when people do not have first-hand experience in a particular domain, ingroup role models can signal whether a person would also be likely to achieve success in that domain [7,8]. In the case of academic seminars, then, the fact that our data show women asking disproportionately fewer questions than men necessarily means that junior scholars are encountering fewer visible female role models in the field. This lack of visibility of women during this type of regular academic interaction (the departmental seminar) is further compounded by women giving fewer talks at, and asking fewer questions at conferences [16,18,19], and women being less visible in the scientific literature as first and senior authors of scientific papers [10][11][12][13]. Given the importance of successful ingroup role modelling, we maintain that examining the visibility of female academics at local, departmental seminars is perhaps even more valuable than examining women's visibility at later levels of the academic trajectory (e.g., publications or conference presentations) because junior scholars are much more likely to attend these departmental seminars, as a way of "seeing what it is like" in order to make the choice of whether to pursue an academic career. Following from social role theory, from early on in their academic trajectory, scholars may encode the relative lack of female role models as an indicator that the academy is not a place where women are successful or represented, and subsequently choose to opt-out of academic careers as a result. When this happens, it perpetuates the original problem of the leaky pipeline by causing women who might have otherwise advanced to senior level positions in academia to take alternate career paths, which means there will continue to be fewer women than men in those positions.
One possible alternative interpretation of the low proportion of questions asked by women in our observational data is that more senior audience members are more willing to ask questions after seminars, and the data could accurately reflect the gender discrepancy in the proportions of senior audience members. That is, there could be a confound between seniority and gender, and the effect we observe is an effect of seniority, not of gender. Because we did not expect our observers to be familiar with the seniority of the members of the audience of all of the seminars they attended, we did not collect data on the seniority of the attendees asking questions. However, two lines of evidence suggest that the disparity we observed is not due only to this. First, in our observational data we controlled for the proportion of female faculty members in the host department and, while this proportion significantly predicted the proportion of questions asked by women, variation remained that was explained by other factors in the models. Additionally, this effect was "shallower" than a direct relationship would predict, with a 5% increase in the proportion of women faculty predicting only a 1.5% increase in the proportion of questions from women. This may suggest that senior women asked proportionally fewer questions than their senior male counter-parts, which is supported by our second line of evidence from the survey data. Men self-reported asking questions after seminars at higher frequencies than women at every career stage, suggesting that even amongst senior faculty men ask questions after seminars more frequently than women (Supplementary Material 4, Fig S4.1C). This finding is also consistent with one study of question-asking behaviour at conferences, which found that younger male attendees asked more questions than younger female attendees at the same rate as the entire sample of questions asked [21].Together, these patterns suggest that seniority does not completely explain the pattern we observed in the gender disparity.
Our observational data suggested that, in addition to the proportion of women faculty mentioned above, several factors affected the proportion of women asking questions after seminars.
The proportion of women in the audience had a significant positive correlation with the proportion of questions asked by women. Although this result is unsurprising, the magnitude of the effect was relatively small, with only a ~1.6% higher share of questions asked by women for a 5% increase in women in the audience. Based on the results of the survey that showed that women rated internal factors as more important in preventing them from asking a question than men, we suggest that the weakness of this effect may stem from women's lower self-reported confidence when asking questions. Such an interpretation is further supported by the finding that a greater proportion of women asked questions when the speaker was from the department, suggesting that familiarity with the speaker may make asking a question less intimidating.
Contrary to our prediction, we found that when the speaker was male, a greater proportion of questions asked after the seminar were from women. We had predicted that the proportion of questions from women would be higher when the speaker was female. However, our results suggest that this was not the case and that men ask proportionally more questions of female speakers and/or women ask proportionally more questions of male speakers. One interpretation may be that men are less intimidated by female speakers than women are, and thus ask more questions when the speaker is female. Alternatively, or in addition to this interpretation, women may avoid "challenging" a female speaker, but may be less concerned for a male speaker.
The gender of the first person to ask a question was also correlated with the proportion of questions asked by women, with a greater proportion of women asking subsequent questions when the first question was asked by a woman compared to when the first question was asked by a man.
A similar effect has also been observed at astronomy conferences [29]. We had included the gender of the first person to ask a question as a predictor because we believed that it may "set the tone" for subsequent questions. Our results suggest that this could be the case and may be an example of gender stereotype activation-where an individual behaves in a gender-stereotype consistent manner when a gender stereotype is activated [31,32]-with a male-first question immediately reinforcing gender stereotypes. This could affect not only women's but also men's behaviour after seminars, with women asking fewer questions and men asking more because of gender stereotypes in assertiveness and confidence. Alternatively, this association could arise because aspects we did not measure might have set an overall environmental tone influencing women and men to ask questions, with the first question being representative of any systematic bias in the subsequent questions. For example, it could be that because of internal factors women are only willing to ask questions in particularly stimulating situations, and in these situations, women will be both more likely to ask the first question, and to ask a greater-than-average proportion of questions. These alternative hypotheses result in the same prediction; an experimental approach is needed to tease them apart.
Several of these interpretations make connections between the self-report results, which focus on psychological factors, and the observational results, which focus on contextual factors. For example, we suggest that women's self-reported lower confidence might explain why they ask more questions when the speaker is internal. It is important to note that, despite research showing that people generally know their own personality best [33], they may lack self-knowledge [34,35], be inaccurate [36]. or may not wish to reveal their true feelings. On one hand, men's ratings of selfconfidence may be low simply because they do not wish to report that they lack confidence. On the other hand, women may also not want to confirm stereotypes by reporting that they lack confidence, and their self-reports might be higher than reality. Thus, any comments on connections between the self-report and observational data are necessarily speculative.

Some Recommendations
Given the problem of the leaky pipeline and the importance of the visibility of women for addressing this, we hope to provide some recommendations that could increase women's visibility during these common events. First, however, we would like to make it clear that we do not place any blame on any party for the disparity that we observed in question-asking after seminars. Many men are not aware that men are asking proportionally more questions and most women identify internal factors as holding them back from asking questions. To the extent that participants' self-reports are accurate, our results suggest that internalised gender stereotypes may be at least partly responsible for the observed disparity [37] , both in men's participation and women's lack of it, and the problem can only be addressed by lasting changes in the academic culture that can help to break gender stereotypes and provide an environment which anyone can feel part of. However, until that time, our data suggest ways we could encourage more equal visibility of men and women, although we note that these recommendations have not yet been empirically tested.
Several of the factors that we identified as important to the proportion of women asking questions after seminars are not easily under a department's control, and we therefore do not consider them as actionable recommendations. These include the proportion of women in the audience and the proportion of women in the department, the latter of which could be changed only over the longerterm. While the characteristics of the speaker are difficult to manipulate, we would encourage seminar organisers not to neglect inviting internal speakers and for moderators to be particularly conscious of bias when the speaker is female. However, it may be possible to change the number of questions asked and the gender of the first person to ask a question. Increasing the number of questions increased the proportion of questions asked by women. Given our manipulation, which failed to increase question time by decreasing the seminar duration, we recommend that, where possible, the question time be unlimited, to encourage more questions. This could be achieved through, for example, booking a seminar room for longer than one hour so that the next event in the room does not cut short the question time. Having said this, a longer time for questions may be a taxing requirement for the speaker after having given a seminar, and may also be undesirable to the audience members. Alternatively, keeping questions and answers short (e.g., through an explicit statement of this as the new department culture, or with the help of a skilled moderator) will allow more questions to be asked during a given question period, and could be an alternative method to allow a greater proportion of questions from women. Although we cannot be sure of the causative relationship between the gender of the first questioner on the disparity in the subsequent questions asked, we would recommend that, should the opportunity arise, a female-first question be prioritised.
This is because (a) a female-first question was a good predictor of low disparity in the questions asked in our observational data and it is possible that gender stereotype activation is responsible for the observed difference and (b) by choosing a female-first question, a female-friendly environment may be fostered over time.
Generally, we feel that more could be done through active changes in speakers', attendees' and particularly moderators' behaviour. Having an active, trained moderator may avoid those situations where one audience member seems to be "showing off" (which survey respondents claim to be the case quite often; N male = 9, N female = 10) or is going off-topic, or a speaker who goes over time. In addition, moderators could be trained to see the whole room (location was mentioned as a factor by N male = 2, N female = 2), and to maintain as much balance as possible with respect to gender and seniority of question-askers. In the open-ended survey questions, respondents complained that moderators call on people they know or more senior people, overlooking the rest (N male = 3, N female = 6). Although it may seem fair to call on people in the order that they raise their hands, doing so may inadvertently result in fewer women and junior academics asking questions, since they often need more time to formulate questions and work up the nerve. In our observational data, we did not record whether a moderator was present, and we did not record the gender of people who attempted to ask a question; our data cannot elucidate whether women asked fewer questions because fewer women raised their hand or because fewer women were chosen to ask a question. It is likely that the discrepancy results from a combination of both, supporting the potential benefits of an active moderator.
Women rated internal factors as more important in holding them back from asking a question, compared to men. To counteract this low confidence, it could help both women and men to provide a small break between the talk and the question period, which would give people time to formulate a question and try it out on a colleague, as well as providing the general benefits of allowing people who need to leave a chance to do so, and giving the speaker a small break.
Although these recommendations (which have yet to be tested empirically) were generated with the idea of increasing women's visibility, they are likely to benefit everyone. It is not only women who are underrepresented in academia; aspiring and early career academics would also benefit from ethnic minorities being more visible.
Our results have implications for redressing the leaky pipeline in academia and indicate that without active steps, the various factors that contribute to women choosing other careers over academia are unlikely to change. Our results support a self-perpetuating feedback loop, where the absence of visible role models influences the behaviour of women in a way that is likely to increase their decision to leave, further reducing their visibility. However, our data show that women are not inherently less likely to ask questions when the conditions are favourable-there is no gender discrepancy when a woman asks the first question. Our suggestions should be seen as aims to create favourable conditions that remove the barriers that restrain anyone from speaking up and being visible.

S1 Qualtrics_Survey_Participation_in_seminars. Survey questions and survey flow.
Caption: The list of questions and the flow of the Qualtrics survey S2 methodology for collecting data in seminars. Instructions for collecting observational data.
Caption: The instructions that were distributed to the observers who agreed to collect data at their local seminars. also included that reflect these data.

Acknowledgements
We Participation in seminars Q1 Thank you for taking the time to participate! In this study, you will be asked some questions about your attendance and participation in academic seminars and the culture around participation in seminars in your department, before we ask some questions about your demography. The study will take fewer than 10 minutes to complete. You can withdraw your participation at any time during the study and are never obliged to answer a question. Your privacy is very important. We will ask you for some demographic information, but nothing that could identify you. We are committed to open science, so the data that we collect will be made available online, for use by other researchers (e.g., at https://osf.io/). Results from this study will be presented in academic publications. Results are normally presented in terms of groups of individuals. If any individual data were presented, the data would be anonymous, without any means of identifying the individuals involved. This project has received ethical approval from the Departmental Psychology Ethics Committee of the University of Essex.
If you have any questions you'd like to ask before starting the survey, please feel free to contact Dr. Gillian Sandstrom at gsands@essex.ac.uk, Dr Alecia Carter at ac854@cam.ac.uk, Dr Dieter Lukas at dl384@cam.ac.uk or Dr Alyssa Croft at alyssac@email.arizona.edu. Consent to Participate: I have read and understood the consent form. I have had sufficient time to consider the information provided and to ask for advice if necessary. I have had the opportunity to ask questions and have had satisfactory responses to my questions. I understand that all of the information collected will be kept confidential and that the results will be made publically available. I understand that my participation in this study is voluntary and that I am completely free to refuse to participate or to withdraw from this study at any time. I understand that I am not waiving any of my legal rights as a result of agreeing to this consent form.
If you consent, please click the 'I consent' button below and then click the arrow to begin the survey.
I consent (1) If I consent Is Not Selected, Then Skip To End of Survey Q2 For this survey, we define a seminar as a public presentation at an academic institution attended by students and faculty.
On average, how many of these types of seminars have you attended? >1 per week (1) Weekly ( (5) Not enough time (1) Worried that I had misunderstood the content (2) Couldn't work up the nerve (3) Not sure whether the question was appropriate (4) Not my field (5) The speaker was too eminent/intimidating (6) Worried that I was not clever enough to ask a good question (7) I was meeting the speaker later / asked after the talk had ended (8) Other (please specify): (9) Q7 To what extent would each of these factors encourage you to ask more questions?
Wouldn't help at all (1) Wouldn't help much (2) Might help a bit (3) Would help a lot (4) Would make a huge difference (5) A longer time to formulate the question (1) A chance to ask in person (2) Nicer speakers (3) More welcoming host (4) Confidence (5) Seniority (6) Having a moderator to ask the questions (7) Moderator doing a better job engaging whole audience (8) Other (please specify): (9) Q6 If / when you ask questions at seminars, what has been your main motivation? (Check all that apply.) Interested in subject (1) Feel you spotted a mistake (2) Need for clarification (3) I feel it's part of my role (e.g., to act as a model for more junior academics) (4) To establish a connection with a particular speaker (5) Q28 What factors do you think play a role in who asks questions?Does Seniority play a role in who asks questions in seminars? Senior audience members ask more questions (1) Junior audience members ask more questions (2) Senior and Junior audience members ask about the same amount of questions (3) Q30 Does Confidence play a role in who asks questions in seminars? Confident people ask more questions (1) Not confident people ask more questions (2) Confident and Not confident people ask about the same amount of questions (3) Q27 Does Extraversion play a role in who asks questions in seminars? Introverted people ask more questions (1) Extraverted people ask more questions (2) Introverted and Extraverted people ask about the same amount of questions (3) Q29 Does Gender play a role in who asks questions in seminars? Women ask more questions (1) Men ask more questions (2) Men and Women ask about the same amount of questions (3) Q31 Does Competence play a role in who asks questions in seminars? Competent people ask more questions (1) Incompetent people ask more questions (2) Competent and Incompetent people ask about the same amount of questions (3) Q28 Do other factors play a role in who asks questions during seminars? No (4) Maybe (5) ____________________ Yes (6) ____________________ Answer If Does Gender play a role in who asks questions in seminars? Men ask more questions Is Selected Or Does Gender play a role in who asks questions in seminars? Women ask more questions Is Selected Q34 You indicated that gender plays a role in who asks questions. How important do you think each of these factors is in preventing the gender asking fewer questions from asking more questions?
Not at all important (1) Slightly important (2) Moderately important (3) Very important (4) Extremely important (5) Worry that they misunderstand the content (1) Can't work up the nerve (2) Are unsure that their questions are appropriate (3) Feel they are not an expert (4) Feel intimidated by the speaker (5) Believe that they are not clever enough to ask a good question (6) Ask questions after the seminar is over (7) Other (please specify): (8) Q14 How much time is usually provided for questions after seminars?  (4) It's not possible (5) Q17 What is the culture around meeting speakers in your department? Speakers meet only with the host (1) Speakers meet with relevant faculty (2) Anyone can sign up to meet a speaker (3) Everyone is actively encouraged to meet speakers, and speakers have organized events (e.g. lunch with PhD students) (4) Q13 In your department, what percentage of the permanent faculty are women? (1) 10-25% (2) 26-50% (3) 51-75% (4) 76-100% (5) I don't know (6) Q28 In your department, what percentage of the graduate/PhD students are women? (1) 10-25% (2) 26-50% (3) 51-75% (4) 76-100% (5) I don't know (6) Q11 What is your subject? (Please choose one from the list.) Accounting and Finance (1) Anthropology (2) Archaeology (3) Art History (4) Biochemistry (5) Biological Sciences (6) Biomedical Sciences (7) Business and Management (8) Chemistry (9) Classics and Ancient History (10) Computer Science and IT (11) Criminology (12) Drama (13) Earth Sciences (14) Economics (15) Education (16) Engineering (17 Q22 The study's aims: This study was designed to help us understand why there is a bias in the gender ratio of academics that attend and ask questions during seminars. Our preliminary research shows that more women attend seminars than men, but they ask fewer questions. From your answers, we would like to make recommendations that will lead to an improved visibility of women in academia through fostering an environment that promotes women's participation in regular academic events. The last thing that we want to ask you is not to share your knowledge about the true purpose of this study. We will be running this study for several weeks. As you can imagine, it would be very difficult for us to collect accurate information if people knew about the true purpose of this study beforehand. Consequently, we would appreciate if you do not discuss the true aim of this survey with others. Thank you so much for participating in this research. Without your help we would be unable to test our hypotheses and gather the necessary data. In case you are interested in the findings of the survey, we will be updating this website once the survey is completed http://academicseminarparticipation.strikingly.com/ If you have any questions, please contact any of the investigators on the project: Dr. Gillian Sandstrom (gsands@essex.ac.uk); Dr Alecia Carter (ac854@cam.ac.uk); Dr Dieter Lukas (dl384@cam.ac.uk); Dr Alyssa Croft (alyssac@email.arizona.edu).
Presented are the questions; the numbers of respondents of each gender who answered the question (N); the numbers and proportions of each gender who responded that the indicated factor was not at all, slightly, moderately, very, and extremely important (N, %) for preventing women from asking questions; and the results of a Kruskal-Wallis test (in all cases, df = 1) indicating whether there was a difference between the genders' responses, including the test statistic (χ 2 ) and significance (p).

Supplementary Table 3:
Responses of a sample of academics who identify as male and female about what factors would encourage them to ask more questions after a seminar