Inequality of opportunity in selection procedures limits diversity in higher education: An intersectional study of Dutch selective higher education programs

Lianne Mulder; Eddymurphy U. Akwiwu; Jos W. R. Twisk; Andries S. Koster; Jan Hindrik Ravesloot; Gerda Croiset; Rashmi A. Kusurkar; Anouk Wouters

doi:10.1371/journal.pone.0292805

Peer Review History

Original SubmissionMarch 7, 2023
4 Sep 2023 Decision Letter - Stefan Grosek, Editor PONE-D-23-06576Inequality of opportunity in selection procedures limits diversity in higher education: An intersectional study of Dutch selective higher education programsPLOS ONE Dear Dr. Mulder, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Oct 19 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Stefan Grosek, Ph.D., M.D., Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating in your Funding Statement: "This work was supported by Nationaal Regieorgaan Onderwijsonderzoek (NRO), grant number 40.5.18650.007 (https://www.nro.nl/) (awarded to GC as consortium leader), and by a Microdata Access Discount award from ODISSEI, the Open Data Infrastructure for Social Science and Economic Innovations (https://ror.org/03m8v6t10) (awarded to LM)." Please provide an amended statement that declares all the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now. Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement. Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf. 3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. 4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide. 5. Please include a caption for figure 1. 6. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. Additional Editor Comments: Dear Authors Thank you for submitting your article to the PLOS ONE. It took a while to find appropriate reviewers. Finally we got two very experienced and knowledgable reviewers on the matter you submitted the article. The first author wants to clarify some statistical issues, so please read it carefully and reply accordingly and send it back ASAP. Kind regards [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors aim to examine how the selection process for higher education programs affects student diversity and representativeness. Specifically, they address: (i) the difference between students in higher education programs compared to the applicant pool, the higher general population, the pre-university population, and their age cohort; (ii) what demographic variables are associated with an applicant's non-admission to selective higher education programs; and (iii) what are the admission rates of applicants with all, some, or none of the background characteristics that are positively associated with the odds of admission. To answer the research questions, the authors analysed national data from the Netherlands on more than 85,000 applicants for 2019 and 2020. The first research question was answered by comparing the distributions of the variables of interest between different groups. They highlighted differences between applicants from different clusters of study programs in terms of gender and socioeconomic status. The second research question was answered using univariable and multivariable logistic regression, separately for different clusters of higher education programs. The authors identified several demographic variables that affect the odds of being accepted into a particular (cluster of) higher education programs and interpreted the differences between clusters of educational programs. Since the authors test a very large number of hypotheses, I wonder if they considered the multiple testing problem and if they evaluated the assumptions of logistic regression. Also, it would be beneficial to include some sort of quasi-R2 to compare the models between clusters of higher education programs. The third research question was addressed by conducting the "intersectional analysis." For each cluster of study programs, the authors calculated the share of students with matching characteristics with (all/some/none) statistically significant impacts who received an offer of admission. Such a share is called the acceptance rate. Here I do not see much added value of this analysis, and the interpretation of the corresponding results is also limited. For a given example on page 23, it is not clear which groups were considered (for Figure 2; which should perhaps be called Figure 3), and I also think that some figures are not available "insert link after acceptance". Minor comments: - The heading levels of, e.g., Migration background and Variables associated with odds of admission should be different. - The »Variables associated with odds of admission« section should be more clearly separated. - Could the effect of the covid pandemic be stronger of the more vulnerable groups, and consequently on the selection for HE? - Missing bracket on page 21, after »final model«. - Fig. 2 should be Fig. 3 on page 22? - The term »gender« is used in S3 and »sex« elsewhere. - Did the authors had in mind »missing completely at random« on page 28? Reviewer #2: This is a well written contribution on higher education accessibility and unequal opportunities. Data is clearly presented in the text with supporting documents and the text of the contribution is well written and clearly presented. It should be published as it is. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0292805.r001
Revision 1
18 Sep 2023 Author Response Our Response to Reviewers document can be found in the files attached to this resubmission. The Table format (with two columns) enables easier reading of our point-by-point response. We have copied the contents of the table here as well: Reviewer #1 1. Please include a caption for figure 1. The caption for Figure 1 can be found on page 10: Fig 1. Study size and exclusion criteria 2. The second research question was answered using univariable and multivariable logistic regression, separately for different clusters of higher education programs. The authors identified several demographic variables that affect the odds of being accepted into a particular (cluster of) higher education programs and interpreted the differences between clusters of educational programs. Since the authors test a very large number of hypotheses, I wonder if they considered the multiple testing problem and if they evaluated the assumptions of logistic regression. We thank the reviewer for the attention given to our manuscript and their detailed feedback. With regard to this question, we would like to respond that our paper is not really about testing hypotheses, it is more dealing with explorative analyses to see which variables are more and which variables are less related to the admission to selective higher education programs. Because of the explorative analyses, we did not adjust the p-values for multiple testing. We added this explanation to the discussion of the paper: “Because of the explorative nature of our study, in which our goal was to find out which variables are more or less related to gaining admission to selective programs, we did not adjust the p-values for multiple testing.” Regarding the assumptions for (multiple) logistic regression analyses, we evaluated collinearity between the independent variables. Furthermore, all independent variables were either dichotomous or categorical, so the assumption whether the relationship with the independent variable is linear or not, is not applicable. 3. Also, it would be beneficial to include some sort of quasi-R2 to compare the models between clusters of higher education programs. Because we were interested in the variables which are related to the admission to selective higher education programs, we did not provide R2 values or any other fit measure for the different multivariable models. As you mentioned, model fit measures provide information about the question which multivariable model is the best. However, which model is the best, does not provide an answer to our research question. We have therefore not included R2 values or any other fit measure for the different multivariable models. 4. The third research question was addressed by conducting the "intersectional analysis." For each cluster of study programs, the authors calculated the share of students with matching characteristics with (all/some/none) statistically significant impacts who received an offer of admission. Such a share is called the acceptance rate. Here I do not see much added value of this analysis, and the interpretation of the corresponding results is also limited. We thank the reviewer for giving us the opportunity to explain the relevance of the intersectional analyses in more detail. The added value of these analyses, is that they make the inequality in opportunity easier to understand for everyone who does not know how to interpret a multivariable logistic regression model. The differences between ‘Group All’ and ‘Group None’ show the level of inequality based on two percentages. In Supplemental Information 5, figures can be found for each cluster, and for the total dataset (2019 and 2020 combined, and separately). We have added a more extensive interpretation to our results with reference to S5 with the following text: “The intersectional acceptance rates in Supplemental Information 5 add value to our understanding of inequality of opportunity in the selection procedures for higher education. They show that different groups of applicants (based on a combination of e.g. sex, migration background, and other variables) have very different chances of success in different programs. For example, in cluster SH5_2020 (Creative Media and Game Technologies; Fashion & Textile Technologies; Industrial Design Engineering; and Art and Economics), female applicants from (very) strongly urban areas without a migration background had an acceptance rate of 80.8% (198/245). In comparison, the same cluster only accepted 56.0% of female applicants from (very) strongly urban areas with a TMSDI migration background (28/50). In cluster SU1_2019 (Medicine), female applicants without a migration background had an acceptance rate of 48.4% (1504/3107), while this was only 32.0% for male applicants with a TMSDI migration background (81/253). The latter percentage is significantly lower than the average acceptance rate of the cluster: 45.4%. Combined with the fact that male TMSDI applicants comprised the smallest group of the applicant pool (with only 253 applicants out of 6571 applicants, 3.9%), their low odds of admission contribute to their underrepresentation in the student population. This is part of a pattern found in many clusters. The result is that the student population of many selective programs does not reflect the diversity of the age cohort, the higher general or the pre-university population (see Supplemental Information 3).” 5. For a given example on page 23, it is not clear which groups were considered (for Figure 2; which should perhaps be called Figure 3) We only have two figures in our manuscript, so we are not able to change this. Figure 1 contains the flow chart with our study size and exclusion criteria. Figure 2 provides an example of the intersectional acceptance rates in the cluster SU4_2020 (Biomedical Sciences and Clinical Technology in 2020). 6. and I also think that some figures are not available "insert link after acceptance". We have now provided all the intersectional figures for all clusters in the file in Supplemental Information 5. In this file, we show the acceptance rates of different groups of applicants, based on the variables that were significantly associated with receiving an offer of admission in the corresponding multivariable logistic regression models (see Supplemental Information 4). Within individual clusters, groups are divided based on these characteristics until a division is no longer allowed due to at least one category having less than 10 persons. Statistics Netherlands regulations do not allow publication of frequencies <10. Since a number of clusters had a large number of significant variables in their multivariable models, their figures had to be split up into multiple figures due to limited space on the page. In figures where this is the case, we have added an a, b, or c in the description of the figure. Example: SH1_2019a is an extension of the figure for SH1_2019. Since this file contains 47 figures in total, we decided that it would be easier for the reader to use the file as it is, rather than having to open 47 individual figures. In the Word file, it is easy to scroll through the figures by using the ‘Navigation pane’ on the left. 7. The heading levels of, e.g., Migration background and Variables associated with odds of admission should be different. The »Variables associated with odds of admission« section should be more clearly separated. We thank the reviewer for their attention to detail and have changed the heading levels accordingly. The headings related to the three research questions (‘Representativeness’, ‘Variables associated with odds of admission’ and ‘Intersectional acceptance rates’) are now in heading level 2, and the variables (Sex; Socio-economic status indicators; migration background) are now in heading level 3. 8. Could the effect of the covid pandemic be stronger of the more vulnerable groups, and consequently on the selection for HE? While it is possible that in individual cases or study programs, the pandemic has played a role in selection outcomes, we have seen no evidence of this in the data on a national level. We made a comparison for all clusters between 2019 and 2020. In 7 clusters, there were more variables significantly associated with the outcome in 2019 than in 2020, meaning that during the pandemic (2020) there were fewer variables of influence in the selection. These clusters are SH1, SH2, SH6, SU1, SU4, SU5, and SU6. There were 3 clusters with an equal number of variables influencing the outcome. These clusters are SU3, SU7, and SU8. Only in 5 clusters, there were more variables associated with the outcome in 2020 than in 2019. These clusters are SH3, SH4, SH5, SU2, and SU9. It is also important to note that in The Netherlands, the lockdown started on March 12, 2020. For a number of selective programs, (some of) the selection instruments (e.g. knowledge tests) had already been completed. This means that the pandemic would have had limited influence on the odds of admission for candidates who had partially or fully completed the procedure. We have therefore not included the potential effect of the pandemic in our manuscript. 9. Missing bracket on page 21, after »final model«. We have added the bracket and thank the reviewer for their attention to detail. 10. Fig. 2 should be Fig. 3 on page 22? We only have two figures in our manuscript, so we are not able to change this. Figure 1 contains the flow chart with our study size and exclusion criteria. Figure 2 provides an example of the intersectional acceptance rates in the cluster SU4_2020 (Biomedical Sciences and Clinical Technology in 2020). 11. The term »gender« is used in S3 and »sex« elsewhere. We have changed the term in S3 to ‘sex’. 12. Did the authors had in mind »missing completely at random« on page 28? We have changed the text to ‘missing completely at random’ Reviewer #2: This is a well written contribution on higher education accessibility and unequal opportunities. Data is clearly presented in the text with supporting documents and the text of the contribution is well written and clearly presented. It should be published as it is. We thank the reviewer for their compliments. Attachments Attachment Submitted filename: Response to Reviewers.docx https://doi.org/10.1371/journal.pone.0292805.r002
29 Sep 2023 Decision Letter - Stefan Grosek, Editor Inequality of opportunity in selection procedures limits diversity in higher education: An intersectional study of Dutch selective higher education programs PONE-D-23-06576R1 Dear Dr. Mulder, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Stefan Grosek, Ph.D., M.D., Academic Editor PLOS ONE Additional Editor Comments (optional): Dear Authors Thank you for your patience and response to the reviewers' comments. I found your article very interesting and imprtant because it addresses a "hidden" problem, not only in your country but also in many others, i.e. inequality of opportunity in selective procedures which limit diversity in higher education as you nicely stated already in your title. I'll send recommendation to the editor to accept your your article. Kind regards Academic Editor Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: (No Response) ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) Reviewer #2: The manuscript is well prepared and all the issues raised by the reviewers adequately addressed. It should be published. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** https://doi.org/10.1371/journal.pone.0292805.r003
Formally Accepted
5 Oct 2023 Acceptance Letter - Stefan Grosek, Editor PONE-D-23-06576R1 Inequality of opportunity in selection procedures limits diversity in higher education: An intersectional study of Dutch selective higher education programs Dear Dr. Mulder: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Stefan Grosek Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0292805.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .