PONE-D-21-27097How Does Moving Public Engagement with Research Online Affect Audience Diversity? Comparing Inclusion Indicators for 2019 & 2020 European Researchers’ Night EventsPLOS ONE

A Dhuine Uasail Dr. Roche,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

 Please see the more detailed responses below. The responses were wide ranging and it's important to acknowledge the views of all the reviewers and respond to each, of which I look forward to receiving.

Please submit your revised manuscript by Dec 11 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Le gach dea-ghu,

Dylan A Mordaunt, MB ChB, FRACP, FAIDH

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please amend your authorship list in your manuscript file to include author

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.  

In your revised cover letter, please address the following prompts:

   a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

    b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

Thank you for your submission. We had widely ranging feedback, all of which I've included below. The article presents what some might call a case study and one of the authors has referred to as a natural experiment, each of which are used in different social sciences contexts, which was part of the challenge in editing this paper. With specific reference to the publication criteria (https://journals.plos.org/plosone/s/criteria-for-publication):

1. The study appears to present the results of original research.

2. Results do not appear to have been published elsewhere.

3. Experiments, statistics, and other analyses are require some elaboration and further detail, as outlined by the reviewers below.

4. Conclusions are presented in an appropriate fashion and are supported by the data.

5. The article is presented in an intelligible fashion and is written in standard English.

6. The research meets all applicable standards for the ethics of experimentation and research integrity.

7. The article adheres to appropriate reporting guidelines and community standards for data availability.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Comment 1: This paper adopted a case, the 2019 & 2020 European Researchers’ Night Events, to investigate the digital divide of online and offline participants, however, relying on survey methods. The limitations are obvious, conclusion and implications can only be understood in a narrow context, such as the European Researchers’ Night Events. Hence, from my perspective, a qualitative method seems more appropriate for this type of research.

Comment 2: The research questions including two hypotheses: audience participation in online events seem to have higher educational attainment and subjective economic well-being when compared to those engaged in offline activities, seems can not provide this field with enough contribution and implications. It is more like a course paper than a journal paper, and I recommend you rethink the RQs and Hypos based on what theoretical and practical implications you can provide to this field.

Comment 3: The sample method for this second hand survey data, which seems to have adopted a convenience sampling method, can not guarantee representativeness, which in turn undermines the conclusion and generality of your study.

Comment 4: In the data analysis part, more advanced statistical methods could be applied, instead of too descriptive evidence, to show direct and robust evidence for the reader’s understanding.

Reviewer #2: Summary

A descriptive review of the change in audience attending the ERN, an annual European science festival, as it transforms from physical to digital delivery in a pre and post-COVID-19 climate. The study compares two social inclusion metrics within nations Malta and Ireland, namely levels of education attained and self-reported economic well-being. The manuscript clearly delineates between the evening expo events and how online content has compensated with mixed media interactions available in the digital space. The author articulates their hypotheses, methods, and results understandably.

Hypotheses of increased educational attainment levels and greater self-reported economic well-being were both upheld and not. In the context of diversity, questions arise whether COVID-19 potentiated skews in education and or economic well-being could be explored further. The manuscript could be strengthened with other measures of diversity and additional examination into reasons for hypotheses not being upheld; if the data is available to support more investigation. A title change to remove diversity without any revision of the data would be less misleading.

Title: Diversity

Diversity and inclusion metrics could be disaggregated for gender or age as a key measure of social inclusion. Ethnicity and linguistics may or may not be relevant or easy to compare given the two nations lack of direct comparison; however, the survey was entirely completed in English. Differences between events, and those speaking English as a first or second language at both events may reveal other influences on audience participation, socio-economic status and or other barriers to accessing the event eg. disability, isolation.

Title: Inclusion Indicators

Quantify the number of inclusion metrics as measures are not exhaustive. 2 inclusion indicators would frame the body of work better.

Abstract: Digital Divide

Income and education are often cited with age, linguistics, race and other factors in context of digital divide. Would help to acknowledge the communication infrastructure necessary to support digital delivery, comparison between the two locations Malta/Ireland and why other indicators have not been included in the statistical measure.

Introduction: Literature

Digital divide literature review methodology would be helpful to include.

ERN Night: Marketing campaigns

Aside from spend, were the demographic targeting settings for digital marketing the same for both? Could this have impacted the diversity of the audience / outcomes of the study?

Discussion: COVID-19

Given the pre and post-pandemic setting for the study, what relevance does it bring to bear on the digital divide and it’s impacts on diverse audiences’ engagement with science?

Reviewer #3: This study provides an interesting ‘natural experiment’ comparing in-person and online attendance at research engagement events pre- and mid-COVID pandemic (2019 vs 2020). This is a well-presented manuscript which will be of interest to many event organisers who continue to be impacted by COVID-19 in their ability to fully present in-person events and may have valid concerns about their ability to reach certain audiences online. The authors have done well to connect their results to the so-called ‘digital divide’, and with a few minor changes will be able to publish a study that has wide significance across numerous fields.

Minor comments:

The keywords provided are limited and repetitive: I recommend adding “science communication” or science/research engagement/outreach or “public engagement in science (or research)” or likewise to help researchers in those fields find this study when it is published.

Tables 1, 2 and 3 use the terms ‘enrolled’ and ‘achieved’ but these are not easy to understand out of context and do not appear well defined in text or in the table captions. Perhaps it would be cleared to use ‘recruited’ or ‘invited’ and ‘responded’, which would align more closely with the description in the methods.

From the in-text description it is clear which city ‘Case 1’ and ‘Case 2’ refers to, so I am unclear why they cannot simply be referred to by the city in the table, i.e. Dublin 1 and Dublin 2. It seems to introduce an unnecessary element of confusion to use different labels in the table when they are clearly identified elsewhere.

For the chi-square test of hypothesis 2, no significant effect has been detected and I think the current wording ‘an extremely weak statistical relationship’ is misleading. Please rephrase to make it clear no relationship was detected between economic wellbeing and year. However, you might consider running the test on Ireland alone to see if there is a difference when you separate country. Is there any economic difference between Ireland and Malta that would contribute to a different result?

In response to Q3 – have the authors made data full available: In the authors’ responses they have indicated data from the study will be available on request, whereas the PLOS Data Policy requires data to be fully available as part of the manuscript, supporting information or in a public repository. The authors should make their data available in such a manner, or explain why they are unable to meet the policy’s requirements.

Ethics statement – is there an approval number from the Trinity College ethics committee? This statement is quite vague: “an ethics committee” – is there more than one?

Major comments:

The introduction to the ERN events is generally very good and provides sufficient background to non-European readers. However, it might be useful to know if the European Commission has particular aspirations in terms of who is reached through these events, for instance those with less formal education or scientific qualifications. If such goals exist, it would be useful context in which to read this study’s findings and could be linked in the discussion in terms of whether a shift to online helped or hindered the reaching of these particular audiences.

It is not clear from the manuscript whether ERN events are free or ticketed. This is relevant information since moving events online would have a likely impact on cost (if there is any cost to attendees). Please amend the manuscript to make this distinction clear: if events are free, it can be mentioned early and dismissed as a potential factor influencing attendance. If there is a cost to attendees, then this should be considered as part of the implications of moving online and should be canvassed in the discussion.

The discussion section is limited and based mostly around the ‘digital divide’, however, there are several interesting avenues the authors could discuss their findings in light of. For instance, for the Ireland events in particular it appears there was a focus on in-person events held on university campuses: there is currently no discussion on whether the shift to online may have facilitated a greater breadth of attendees by democratising the venue and being more open to attendees with less formal education who otherwise might feel uncomfortable at a university-based event.

Currently missing from the discussion is any form of recommendation or reassurance for those running such outreach events. It would be useful to comment on such events in the context of these results, especially as some event organisers may choose to operate ‘dual mode’ or provide some online events to help attract a wider audience. It would also be interesting to discuss whether delivering events online was cost-effective to organisers and funders as a way of reaching a broader range of attendees perhaps in addition to in-person events.

Overall, I think the authors haven’t done their study justice in the discussion: their findings raise several interesting points about whether online delivery might attract a wider range of attendees to such events and better achieve goals to connect with audiences with less formal education. At least this should be discussed, potentially in the form of recommendations for event organisers or further research examining events that were forced online in 2020/21. This could also be better tied back to science/research outreach and the goal of connecting with a broader range of audiences.

Reviewer #4: The manuscript is technically sound, well-written, and all data discussed are clearly presented. The statistics need some further explanation to ensure that they have been rigorously tested. As an example, you refer to four 'cases' but it is not clear how these were tested statistically, and how classifying these into 4 cases is important. Overall, the paper demonstrates an important finding, that shifting public science events online can polarise the audience by strengthening attendance from higher educated groups whilst also attracting those with no formal qualification. I felt that the manuscript was rather too simplistic in this analysis however, as there was no discussion about other interesting demographics and how these may be affected by a move to an online delivery. It was also not explained why Malta and Ireland were chosen for this study, and no recommendations were given for improving online events to attract a wider demographic. I would be interested to see a revised version that presents evidence for future event organisers (and other science communication practitioners) to use to ensure greater representation of diverse society at these events.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Katy Thomas

Reviewer #3: No

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Reviewers comments.docx

How does moving Public Engagement with Research Online Change Audience Diversity? Comparing Inclusion Indicators for 2019 & 2020 European Researchers’ Night events.

PONE-D-21-27097R1

Dear Dr. Roche,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dylan A Mordaunt, MB ChB, MPH, MHLM, FRACP, FAIDH

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thank you for your resubmission. I accept the responses and this now meets the criteria for publication.

Reviewers' comments: