Effects of gender sensitive language in job listings: A study on real-life user interaction

Dominik Hetjens; Stefan Hartmann

doi:10.1371/journal.pone.0308072

Peer Review History

Original SubmissionFebruary 6, 2024
14 May 2024 Decision Letter - Montserrat Comesaña Vila, Editor PONE-D-24-04132Effects of gender sensitive language in job listings: a study on real-life user interactionPLOS ONE Dear Dr. Hetjens, Thank you for submitting your manuscript to PLOS ONE. Your manuscript has been reviewed, and the comments of the reviewers are included at the bottom of this letter. Both are positive and recommend its publication pending minor reviews. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jun 28 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Montserrat Comesaña Vila Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. In your Methods section, please include additional information about your dataset and ensure that you have included a statement specifying whether the collection and analysis method complied with the terms and conditions for the source of the data. 3. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The entire manuscript is exceptionally well-written, featuring a meticulously structured case study section, clear objectives, and a robust methodology. The study demonstrates a strong motivation, drawing from previous research, and the questions posed within this manuscript are pertinent. They have the potential to yield significant insights into the impact of gender-sensitive language, particularly in the context of job listings, which is an important and timely topic. This paper is poised to make substantial contributions across various fields, with the potential for significant impact. Moreover, it introduces a novel perspective by analyzing real language usage with a large sample size to explore gender-sensitive language, while also providing a methodology that can pave the way for further research inquiries. However, navigating through the manuscript, especially the introduction, can prove challenging. I believe that reorganizing and simplifying the content is necessary to facilitate comprehension for a wider audience. I recommend that the author begins with a more general review of gender and gender sensitivity, along with their implications, before delving into the specific situation in German language. The current introduction may deter some readers, as it appears overly focused on the German context, whereas I believe it holds broader implications. While I appreciate the authors' thorough introduction, I suggest trimming it down and focusing on more specific points to avoid repetition. For instance, the method of specifying the feminine is mentioned multiple times through the addition of suffixes. Additionally, some of the reviewed studies contain excessive detail, such as including the professions of the authors, which may fatigue external readers. The introduction would benefit from being streamlined and more concise overall. It currently resembles a chapter from a book rather than an introduction to a paper. The subsections on "Systemic Approach" and "Corpus-based Approach" incorporate various studies, theoretical frameworks, and underlying philosophies of each approach. While it's important to explain these to readers, they don't have a direct link to the main topic. However, the "Experimental Approach" includes specific studies relevant to job advertisements, which is the primary focus. It might be more effective for the author to begin with literature directly related to job advertisements and the experimental approach. Currently, readers have to wait until page 22 to encounter specific literature on the main topic. I recommend introducing this literature on job advertisements earlier, followed by a brief overview of how systemic and corpus-based approaches can aid in investigating this topic, emphasizing their relevance to gender studies. Each approach could be presented in a condensed and more focused manner. Additionally, since the study combines corpus-based and experimental approaches, less emphasis should be placed on systemic approaches. In the case study section, it's challenging to discern which aspects of the current study and the analysis the authors consider part of the experimental approach versus the corpus-based approach. Since the main objective of the study is to capitalize on the strengths of both methodologies, it's crucial to clarify this explicitly. While it's apparent that classifying job listings based on genderism within the corpus analysis, and tracking the number of views by real users aligns more with the experimental approach, the distinction isn't sufficiently clear. This becomes especially important given the absence of additional data such as reaction times, acceptability tests, or eye-tracking, which are typically integral components of the experimental approach. Therefore, I suggest strengthening the separation between the two methodologies to provide greater clarity and understanding for readers. Reviewer #2: The study investigated the effects of gender-sensitive language (in German, which is a language with three grammatical genders masculine, feminine and neuter) in job advertisements on user interaction. It gathered anonymous data on user interactions, specifically click counts, from a company's website, employing a novel approach that blends corpus-based and experimental methods. Strengths of the paper include its innovative use of real-world interaction data and its timely focus on gender-sensitive language in job postings. I also commend the authors for their transparent methodological reporting. However, the study has some methodological limitations, most of which are aknowledged by the authors. It draws data from a specific recruitment platform, potentially introducing biases based on user demographics and behavior, thus limiting generalizability. The study recognizes extralinguistic factors like job sector, location, and company reputation, which are challenging to control in a naturalistic setting. While the study identifies correlations between gender-sensitive language and user behavior, it cannot definitively establish causation, as other unmeasured variables may influence observed patterns. Acknowledging these limitations, the authors suggest avenues for future research to enhance the generalizability and robustness of findings, including diversifying datasets and contexts. As such, I think the authors may find this recent paper on ideologies relevant, as it delves into gender-inclusive language in German across diverse user groups, including those who learned German as adults exclusively and German speakers residing outside German-speaking regions. Truan, Naomi. "Whose language counts?: Native speakerism and monolingual bias in language ideological research: Challenges and directions for further research" European Journal of Applied Linguistics, 2024. https://doi-org.ezproxy.leidenuniv.nl/10.1515/eujal-2024-0006 Additionally, considering multilingual societies where speakers often default to either the masculine or feminine gender when code-switching languages could offer valuable insights about gender inclusiveness. For instance, Balam (2016) and Valdés Kroff (2016) discuss examples from Spanish/English in Miami and Belize (' el.masc Virgin Mary' ) at a rate between 97 and 99% of all switched nominal constructions. Parafita Couto et al. (2015) provide examples of a feminine deafault from Spanish-Basque ('la.fem osaba' - ' the uncle') . Bellamy and Parafita Couto (2022) offer an overview of such strategies. Reflecting on these phenomena across diverse communities, where multilingual speakers navigate languages with varying gender categories, could significantly enhance future studies on gender sensitivity and gender-inclusive language within these contexts. Balam, O. (2016) Semantic categories and gender assignment in contact Spanish: Type of code-switching and its relevance to linguistic outcomes. Journal of Language Contact, 9, 405–435. Bellamy, K and Parafita Couto, MC (2022) Gender assignment in mixed noun phases: State of the art. In Ayoun, D (ed), The Acquisition of Gender: Crosslinguistic perspectives. Amsterdam: John Benjamins. pp. 13–48. https://doi.org/10.1075/sibil.63.02bel Parafita Couto, M. C., Munarriz, A., Epelde, I., Deuchar, M., & Oyharçabal, B. (2015) Gender conflict resolution in Spanish-Basque mixed DPs. Bilingualism: Language and Cognition, 18(2), 304–323. Valdés Kroff, J. R. (2016) Mixed NPs in Spanish-English bilingual speech: Using a corpus-based approach to inform models of sentence processing. In R. E. Guzzardo Tamargo, C. M. Mazak & M. C. Parafita Couto (Eds.), Spanish-English code-switching in the Caribbean and the US (pp. 281–300). John Benjamins. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Antonio Iniesta Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. Attachments Attachment Submitted filename: Effects of gender sensitive language in job listings_review.pdf https://doi.org/10.1371/journal.pone.0308072.r001
Revision 1
4 Jul 2024 Author Response Response to reviewers We would like to thank the reviewers for their helpful and thorough feedback. Following the reviewers’ advice, we have restructured the paper and contextualized the discussion of gender-sensitive language much more. While we have tried to incorporate all suggestions put forward by Reviewer 1, some of Reviewer 2’s suggestions go beyond the scope of our paper, although they are definitely very interesting and relevant. As we are focusing on the empirical question of the potential effects of gender-sensitive language on user interactions with online job advertisements, we cannot say much about potential effects of multilingualism on the use of grammatical genders. However, we agree that this is an aspect that should be pursued in future research, and we have added some ideas to the outlook. Below we respond to the individual issues raised by the reviewers in more detail. Reviewer #1: Point 1.1 The entire manuscript is exceptionally well-written, featuring a meticulously structured case study section, clear objectives, and a robust methodology. The study demonstrates a strong motivation, drawing from previous research, and the questions posed within this manuscript are pertinent. They have the potential to yield significant insights into the impact of gender-sensitive language, particularly in the context of job listings, which is an important and timely topic. This paper is poised to make substantial contributions across various fields, with the potential for significant impact. Moreover, it introduces a novel perspective by analyzing real language usage with a large sample size to explore gender-sensitive language, while also providing a methodology that can pave the way for further research inquiries. However, navigating through the manuscript, especially the introduction, can prove challenging. I believe that reorganizing and simplifying the content is necessary to facilitate comprehension for a wider audience. Reply 1.1 Thanks for the positive assessment and for the constructive feedback! We hope that our revisions sufficiently address the very legitimate concerns regarding the readability and accessibility of the paper. In particular, we have considerably shortened and condensed the introduction as well as the research overview, and contextualized the specific discussion more in a general context. Point 1.2 I recommend that the author begins with a more general review of gender and gender sensitivity, along with their implications, before delving into the specific situation in German language. The current introduction may deter some readers, as it appears overly focused on the German context, whereas I believe it holds broader implications. Reply 1.2 This is indeed a very relevant aspect. We have added a short discussion about language and gender more generally, abstracting away from the specifics of German. At the same time, we have refrained from providing an in-depth overview of the (linguistic and/or public) debates on language and gender, as this topic is not only too broad to be adequately discussed in such a short paper, but addressing it in too much detail might also invite the assumption that we are overestimating the implications of our study. Point 1.3 While I appreciate the authors' thorough introduction, I suggest trimming it down and focusing on more specific points to avoid repetition. For instance, the method of specifying the feminine is mentioned multiple times through the addition of suffixes. Additionally, some of the reviewed studies contain excessive detail, such as including the professions of the authors, which may fatigue external readers. The introduction would benefit from being streamlined and more concise overall. It currently resembles a chapter from a book rather than an introduction to a paper. Reply 1.3 We have restructured the paper to keep the introduction significantly shorter, and we have followed the advice to trim down the literature review. In particular, we have substantially shortened the overview of corpus-based approaches, as they are not as relevant to our own study as previous experimental work on the topic. To still provide some of the information included in the original introduction, we have added a short additional section between the introduction and the research overview, in which we provide a short overview of grammatical genders in German. This way, we intend to keep the introduction short and more concise, while at the same time providing the most important information necessary to understand the study. Point 1.4 The subsections on "Systemic Approach" and "Corpus-based Approach" incorporate various studies, theoretical frameworks, and underlying philosophies of each approach. While it's important to explain these to readers, they don't have a direct link to the main topic. However, the "Experimental Approach" includes specific studies relevant to job advertisements, which is the primary focus. It might be more effective for the author to begin with literature directly related to job advertisements and the experimental approach. Currently, readers have to wait until page 22 to encounter specific literature on the main topic. I recommend introducing this literature on job advertisements earlier, followed by a brief overview of how systemic and corpus-based approaches can aid in investigating this topic, emphasizing their relevance to gender studies. Each approach could be presented in a condensed and more focused manner. Additionally, since the study combines corpus-based and experimental approaches, less emphasis should be placed on systemic approaches. Reply 1.4 To address these points, we have condensed the research overview into one section, mentioning the systemic approach only briefly, and introducing the more relevant experimental studies earlier. The whole section now focuses strongly on research that has a more direct link to the main topic. Point 1.5 In the case study section, it's challenging to discern which aspects of the current study and the analysis the authors consider part of the experimental approach versus the corpus-based approach. Since the main objective of the study is to capitalize on the strengths of both methodologies, it's crucial to clarify this explicitly. While it's apparent that classifying job listings based on genderism within the corpus analysis, and tracking the number of views by real users aligns more with the experimental approach, the distinction isn't sufficiently clear. This becomes especially important given the absence of additional data such as reaction times, acceptability tests, or eye-tracking, which are typically integral components of the experimental approach. Therefore, I suggest strengthening the separation between the two methodologies to provide greater clarity and understanding for readers. Reply 1.5 We have added a few remarks to make it more clear in which way parts of our study can be considered part of the corpus-based approach. Overall, we believe that our approach is - to the best of our knowledge - new, and cannot be classified according to the classification of approaches we have introduced in the research overview. Reviewer #2: Point 2.1 The study investigated the effects of gender-sensitive language (in German, which is a language with three grammatical genders masculine, feminine and neuter) in job advertisements on user interaction. It gathered anonymous data on user interactions, specifically click counts, from a company's website, employing a novel approach that blends corpus-based and experimental methods. Strengths of the paper include its innovative use of real-world interaction data and its timely focus on gender-sensitive language in job postings. I also commend the authors for their transparent methodological reporting. However, the study has some methodological limitations, most of which are acknowledged by the authors. It draws data from a specific recruitment platform, potentially introducing biases based on user demographics and behavior, thus limiting generalizability. The study recognizes extralinguistic factors like job sector, location, and company reputation, which are challenging to control in a naturalistic setting. While the study identifies correlations between gender-sensitive language and user behavior, it cannot definitively establish causation, as other unmeasured variables may influence observed patterns. Acknowledging these limitations, the authors suggest avenues for future research to enhance the generalizability and robustness of findings, including diversifying datasets and contexts. Reply 2.1 Thanks for the positive assessment! Point 2.2 As such, I think the authors may find this recent paper on ideologies relevant, as it delves into gender-inclusive language in German across diverse user groups, including those who learned German as adults exclusively and German speakers residing outside German-speaking regions. Truan, Naomi. "Whose language counts?: Native speakerism and monolingual bias in language ideological research: Challenges and directions for further research" European Journal of Applied Linguistics, 2024. https://doi-org.ezproxy.leidenuniv.nl/10.1515/eujal-2024-0006 Additionally, considering multilingual societies where speakers often default to either the masculine or feminine gender when code-switching languages could offer valuable insights about gender inclusiveness. For instance, Balam (2016) and Valdés Kroff (2016) discuss examples from Spanish/English in Miami and Belize (' el.masc Virgin Mary' ) at a rate between 97 and 99% of all switched nominal constructions. Parafita Couto et al. (2015) provide examples of a feminine deafault from Spanish-Basque ('la.fem osaba' - ' the uncle') . Bellamy and Parafita Couto (2022) offer an overview of such strategies. Reflecting on these phenomena across diverse communities, where multilingual speakers navigate languages with varying gender categories, could significantly enhance future studies on gender sensitivity and gender-inclusive language within these contexts. Balam, O. (2016) Semantic categories and gender assignment in contact Spanish: Type of code-switching and its relevance to linguistic outcomes. Journal of Language Contact, 9, 405–435. Bellamy, K and Parafita Couto, MC (2022) Gender assignment in mixed noun phases: State of the art. In Ayoun, D (ed), The Acquisition of Gender: Crosslinguistic perspectives. Amsterdam: John Benjamins. pp. 13–48. https://doi.org/10.1075/sibil.63.02bel Parafita Couto, M. C., Munarriz, A., Epelde, I., Deuchar, M., & Oyharçabal, B. (2015) Gender conflict resolution in Spanish-Basque mixed DPs. Bilingualism: Language and Cognition, 18(2), 304–323. Valdés Kroff, J. R. (2016) Mixed NPs in Spanish-English bilingual speech: Using a corpus-based approach to inform models of sentence processing. In R. E. Guzzardo Tamargo, C. M. Mazak & M. C. Parafita Couto (Eds.), Spanish-English code-switching in the Caribbean and the US (pp. 281–300). John Benjamins. Reply 2.2 Thanks for these suggestions! They are very much in line with the first reviewer’s proposal to delve a bit more deeply into the issue of gender-sensitive language beyond the specifically German-language context. While the issue of multilingualism goes beyond the scope of the present paper, we have added a more thorough theoretical discussion that contextualizes our study in the more general debates about gender-sensitive language and language ideologies. Attachments Attachment Submitted filename: Response to Reviewers.docx https://doi.org/10.1371/journal.pone.0308072.r002
17 Jul 2024 Decision Letter - Montserrat Comesaña Vila, Editor Effects of gender sensitive language in job listings: a study on real-life user interaction PONE-D-24-04132R1 Dear Dr. Dominik Hetjens, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Montserrat Comesaña Vila Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0308072.r003
Formally Accepted
22 Jul 2024 Acceptance Letter - Montserrat Comesaña Vila, Editor PONE-D-24-04132R1 PLOS ONE Dear Dr. Hetjens, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Montserrat Comesaña Vila Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0308072.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .