Closing the gender gap at academic conferences: A tool for monitoring and assessing academic events

The importance of participation in academic conferences is well known for members of the scientific community. It is not only for the feedback and the improvement of the work, it is also about career development, building networks and increasing visibility. Nevertheless, women continue to be under-represented in these academic events and even more so in the most visible positions such as speaking roles. This paper presents the development of a tool based on performance indicators, which will allow monitoring and evaluating gender roles and inequalities in academic conferences in order to tackle the underrepresentation of women. The study identifies relevant perspectives (participation, organizational structure and attitudes) and designs specific lists of performance indicators for each of them. The tool is based on a combination of two multicriteria techniques, Analytic Hierarchy Process and Analytic Hierarchy Process Sort, and a qualitative analysis based on in-depth interviews and information gathered from a focus group. The use of the AHP multi-criteria decision technique has allowed us to weight the indicators according to the opinion of several experts, and with them to be able to generate from these weightings composite indicators for each of the three dimensions. The most relevant indicators were for the participation dimension. Additionally, the tool developed has been applied to an academic conference which has been monitored in real time. The results are shown as a traffic light visualization approach, where red means bad performance, yellow average performance and green good performance, helping us to present the results for each indicator. Finally, proposals for improvement actions addressed to the red indicators are explained. The work carried out highlights the need to broaden the study of gender equality in academic conferences, not only regarding the participation but also the performance of different roles and functions.


Reviewer #1.
Be careful when using the abbreviation for AHP Sort because 3 abbreviations AHP Sort, AHPSort, AHP-Sort are used in the work. Choose one and use it throughout the paper.
Every different abbreviation was changed to AHPSort throughout the paper Explain the results obtained by the AHP method.
The most relevant results obtained by the AHP method are explained so that the objective of the prioritization of indicators is better understood. We have added an explanation about these results in paragraph 397-409 (manuscript version) Reviewer #2.
In the first part, it is written that the interview was conducted with 8 experts. In the part with the AHP method, 11 experts are mentioned. Where does this difference come from? Are they the same people or has a new group been formed for the second round?
There were two different groups of experts: one for the interviews and the other one for the focus group and the AHP technique. The first group was composed of 7 experts. In this case, interviews were conducted online and face-to-face. For the focus group and the AHP technique, a new set of experts has been selected, since in-person participation was required. In addition, a wider variety of academic profiles were included in this case. We have clarified this issue in the paper including a better description of the different participant groups in paragraph 253-258 and 283-291.

In Table 2. The numbers are not arranged in order
We have corrected that and the numbers are now arranged in order.
The list of 11 focus group participants states different disciplinary backgrounds. It is not clear on the basis of which criteria the participants were selected? Why there was a difference in relation to the group from the beginning of the research. It is necessary to supplement the work.
We have added an explanation about the criteria for the selection for both interviews and focus group in paragraph 287-289 and 252-257. The main selection criterion was academic relevance. In this sense, we understand academic relevance as a combination between the quality of research publications and projects and the attendance to academic conferences. In addition, interviewees were selected on the basis of opinions and recommendations from members of the corresponding scientific community regarding the most linkage with gender topics in their academic discipline. For the focus group, the need to attend in person was a decisive factor.

The difference in the number and types of disciplinary origin of the interview participants (initial examination) and the focus group is not explained.
The interviews sought to offer a first approach to the phenomenon studied. They focused mainly on the discussion of the dimensions and indicators related to gender issues. But also, on collective and individual strategies and ideas to tackle the gender gap in conferences according to each individual's trajectory and experience. In this sense, a variety of profiles is needed but we consider more relevant the expertise related to the topic of gender. For this reason, interviewees were also selected on the basis of opinions and recommendations from members of the corresponding scientific community regarding the most linkage with gender topics in their academic discipline.
In the contrary, the objective in the focus group was to deepen and refine the definition of indicators. In this sense, it was necessary to cover all areas of knowledge with social and hightech visions. The emphasis on the presence of the different areas of knowledge is fundamental for the correct and varied redefinition and prioritization of indicators.

Complement the work with a selection of works from 2020. Year
We have reviewed different papers from 2020 and have also included them in our references. A short review of the latest applications of traditional AHP and AHP in conjunction with other methods has been carried out and presented in paragraph 343-344. We have also included references of year 2020. Table 6.

Explain how you calculated the value of "Group (AIJ)" in
We have explained it right before Table 6.
Aggregation of individual judgments (AIJ) were performed using the geometric mean for all the experts (Saaty 2001; Saaty and Peniwati 2008). When individuals act in concert and pool their judgments in such a way that the group becomes a new 'individual' and behaves like one, there is a synergistic aggregation of individual judgments. Furthermore, since the group becomes a new 'individual' and behaves like one, the reciprocity requirement for the judgments must be satisfied and the geometric mean rather than an arithmetic mean must be used. Treating the group as a new 'individual' with AIJ requires satisfaction of the reciprocity condition for the judgments. Aczel and Saaty (1983), have shown that when aggregating the judgments of n individuals where the reciprocal property is assumed, only the geometric mean satisfies the Pareto principle (unanimity condition) and the homogeneity condition (if all individuals judge a ratio n times as large as another ratio, then the synthesized judgment should also be n times as large). Thus, for AIJ, the geometric mean must be used. the geometric mean is more consistent with the meaning of both judgments and priorities in AHP. In case the group structure is homogenous and decision makers are willing to act like one single individual, a synergistic AIJ is possible. Each decision maker conducts the pairwise comparisons by himself. Afterwards the geometric mean method (GMM) could be used to obtain the group judgment for each entry of the comparison matrices. The AIJ procedure (Saaty 1989) is conducted by determining the mean of the individual judgments for each entry of the pairwise comparison matrices. If a r i. j are the individual judgments of the group members DMr with r = 1,..., R by comparing element i with element j the aggregated pairwise comparison judgment A (i, j) is computed by the weighted geometric mean method (GMM): A GMM (i, j) = ∏ R r=1 (a r i. j ) Thank you for receiving our manuscript and considering it for review. We appreciate your time and look forward to hearing from you.
The authors.