I’m fine with collecting data: Engagement profiles differ depending on scientific activities in an online community of a citizen science project

Till Bruckermann; Hannah Greving; Milena Stillfried; Anke Schumann; Miriam Brandt; Ute Harms

doi:10.1371/journal.pone.0275785

Peer Review History

Original SubmissionApril 1, 2022
2 Sep 2022 Decision Letter - Alice Coles-Aldridge, Editor PONE-D-22-09613 I’m fine with collecting data: Engagement profiles differ depending on scientific activities in an online community of a citizen science project PLOS ONE Dear Dr. Bruckermann, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. The manuscript has been evaluated by three reviewers, and their comments are available below. The reviewers have raised a number of concerns that need attention. They request additional information and discussion on how citizen engagement profiles can differ in different types of research to provide context to interpret the results with. They also request additional clarity on the definitions of data collection and data analysis, and note concerns with the small sample sizes for the data analysis profiles and that not all claims may be supported. Could you please revise the manuscript to carefully address the concerns raised? Please submit your revised manuscript by Oct 17 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Alice Coles-Aldridge Editorial Office PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1.Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly Reviewer #3: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: I Don't Know ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for exploring the extent to which citizens are engaged in different tasks of the scientific process. Many times, citizen involvement in research is based around tokenism, and does not result in meaningful participation of citizens in the work. Revision suggestions: Adding some discussion (2-3 lines, maybe a paragraph) around how citizen engagement profiles can differ in different types of research (e.g., health sciences vs. zoological sciences etc) in the background could provide the reader with some context to interpret the results with. Reviewer #2: The paper builds on an important issue under examination in the citizen science field-- how to deepen participant-engagement to benefit the citizens and the science. One concern I have is the authors' definition of "data collecting" . Here, it is described as the act of setting out camera traps and identifying/labeling the images captured by the cameras. That actually could be considered data analysis. They are analyzing the data collected by camera traps. The act of making and sharing the observations (by going into nature to take and upload the pictures, for example), is what I expected to see in this paper. Perhaps the authors can address this in the paper a bit? There are thousands of citizen science projects that rely on people to collect and share data and typically that means that the participants make and share observations...not retrieve, analyze and label the data. Again, that seems to be a form of data analysis. Reviewer #3: Thank you for conducting this work examining engagement patterns across projects. In general, I think the paper is very strong and is need of minor revisions. This statement is unclear, please explain: The standard deviation of days between pairs of sequential active days was the variation in periodicity (v). The sample sizes for the data analysis profiles are so small - can we really say anything about these profiles? Unsure that this claim is warranted: "We suppose as a post-hoc explanation that ownership (i.e., an individuals’ feeling of possessing a concrete or abstract object or entity; [38]) for the data, that they themselves collected and contributed, might explain why highly active participants also engaged in data analysis." There may be many other factors at play besides ownership. It would be important for you to include alternate explanations (e.g., not interested/motivated to do analysis, feeling "uncredentialed," lack of educational support for analytical skills. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No Reviewer #3: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0275785.r001
Revision 1
8 Sep 2022 Author Response Dear Alice Coles-Aldridge, dear Reviewers, thank you very much for your positive and helpful feedback and for investing your time and effort in reviewing our manuscript. We appreciate your constructive and valuable feedback that has helped us very much in improving the paper on its way to publication. We have considered all of your suggestions, and we are convinced that we have made appropriate improvements. In this letter, we address all of your comments from the reviews and explain our revisions in detail. To ensure the transparency of our revisions, we have responded to each comment separately and explain our responses to each comment below. In some cases, we have also pasted the new or changed paragraphs in quotation marks and indented them. For a full account of our changes, please refer to the revised manuscript in which we have highlighted the changes in blue color. Comments from the Editor and Reviewers: Editor: Comment 1: Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Response 1: Thank you very much for this positive feedback. We are happy to resubmit a revised version of our manuscript. Comment 2: The manuscript has been evaluated by three reviewers, and their comments are available below. The reviewers have raised a number of concerns that need attention. They request additional information and discussion on how citizen engagement profiles can differ in different types of research to provide context to interpret the results with. They also request additional clarity on the definitions of data collection and data analysis, and note concerns with the small sample sizes for the data analysis profiles and that not all claims may be supported. Could you please revise the manuscript to carefully address the concerns raised? Response 2: These are valid points from the reviewers. And yes, of course, we have carefully revised the manuscript and have addressed all of the reviewers’ comments accordingly (for our detailed responses, see below). Reviewer #1: Comment 3: Thank you for exploring the extent to which citizens are engaged in different tasks of the scientific process. Many times, citizen involvement in research is based around tokenism, and does not result in meaningful participation of citizens in the work. Revision suggestions: Adding some discussion (2-3 lines, maybe a paragraph) around how citizen engagement profiles can differ in different types of research (e.g., health sciences vs. zoological sciences etc) in the background could provide the reader with some context to interpret the results with. Response 3: We thank Reviewer 1 very much for the positive feedback regarding our submission. To provide context for the interpretation of our results, we have now discussed how participants’ engagement potentially differs between projects from specific disciplines in the Introduction section (p. 6). “However, even though citizens should be objectively able to participate in the whole scientific process in CS projects [25], it is unclear whether citizens equally engage in all levels of scientific activities or whether they are even interested in doing so [4,9,20]. First, CS projects often take place in informal learning settings [26] which provide citizens with different opportunities to learn from different scientific activities [27]. In this case, their choice of activities may not correspond to the ideal engagement pattern of the highest level [9,19]. Second, the spreading of CS in various disciplines [28] has led to CS projects offering discipline-specific activities for citizens to participate in [21]. For example, in the biological sciences, CS projects often engage participants in observation and monitoring activities [21], while in the health sciences, CS projects may involve their participants in more gamified activities such as digitally folding, and modifying proteins and their structures [29]. Such discipline-specific activities of CS projects might promote that participants’ engagement differs between projects from different disciplines [1].” Reviewer #2: Comment 4: The paper builds on an important issue under examination in the citizen science field-- how to deepen participant-engagement to benefit the citizens and the science. One concern I have is the authors' definition of "data collecting" . Here, it is described as the act of setting out camera traps and identifying/labeling the images captured by the cameras. That actually could be considered data analysis. They are analyzing the data collected by camera traps. The act of making and sharing the observations (by going into nature to take and upload the pictures, for example), is what I expected to see in this paper. Perhaps the authors can address this in the paper a bit? There are thousands of citizen science projects that rely on people to collect and share data and typically that means that the participants make and share observations...not retrieve, analyze and label the data. Again, that seems to be a form of data analysis. Response 4: We thank the Reviewer very much for the positive feedback and for bringing this point to our attention. We regret that we have not been clear enough about the reasons why we included species identification in the data collection phase and not in the data analysis phase. To address this point, we have explained our considerations in the Introduction section and the Methods section. “The mere collection of data in a CS project is regarded as the lowest level of participation (contributory projects: [19]); collecting data, data analysis, and the interpretation of results in a CS project is seen as an intermediate level (collaborative projects: [19]), because some projects allow for data analyses that go beyond species identification activities [21] and may include statistical data analyses (e.g., [22]); and being involved in the whole research process, from generating research questions to drawing conclusions, is regarded as the highest level (co-created projects: [19]; see also ‘extreme CS’: [20]).” (p. 5) “During the data collection phase, participants (1) uploaded camera trap photographs, (2) learned how to identify the wildlife species in the photographs from a tutorial, and (3) identified the species. Participants also (4) identified species in pictures from other participants in order to validate other participants’ identification of species. These tasks of the data collection phase correspond to a task typology that clearly distinguishes observations, species identification, and data entries from data analysis [21] (e.g., statistical data analysis: [22]), although species identification could be seen as some kind of initial data analysis [31].” (p. 9) Reviewer #3: Comment 5: Thank you for conducting this work examining engagement patterns across projects. In general, I think the paper is very strong and is need of minor revisions. Response 5: We thank the Reviewer very much for the positive feedback. Comment 6: This statement is unclear, please explain: The standard deviation of days between pairs of sequential active days was the variation in periodicity (v). Response 6: To be clearer about the calculation of the variation in periodicity, we have rephrased the definition accordingly. “We also counted the number of days between each of a participant’s active days and then calculated the standard deviation of these days in between. Thus, we estimated how regularly a participant actively contributed to the project which was the variation in periodicity (v).” (p. 12) Comment 7: The sample sizes for the data analysis profiles are so small - can we really say anything about these profiles? Response 7: We can understand the Reviewer’s comment. We believe that the strength of our research approach is that we followed all participants of the Wildlife Researcher projects across the two phases of the project, that is, the data collection phase and the data analysis phase. While we agree that the number of the participants in the data analysis profiles is smaller than in the data collection profiles, we also think that it is eminent for our research to show how many participants stayed active after data collection and thus were included in the data analysis profiles. Still, to better highlight the limitations that come with smaller sample sizes, we have added a discussion of the sample sizes for the data analysis profiles to the Limitations section. “A second limitation concerns the sample size in our field studies. Especially, the sample sizes for the engagement profiles in the data analysis phase were smaller than the sample sizes of those profiles in the data collection phase. While this difference may limit the conclusion about participants’ engagement in the data analysis, the quantitative and qualitative changes between the profiles also highlight the differences in engagement between the data collection and the data analysis phase. In general, the number of participants was limited to the camera traps available in the CS project.” (p. 32) Comment 8: Unsure that this claim is warranted: "We suppose as a post-hoc explanation that ownership (i.e., an individuals’ feeling of possessing a concrete or abstract object or entity; [38]) for the data, that they themselves collected and contributed, might explain why highly active participants also engaged in data analysis." There may be many other factors at play besides ownership. It would be important for you to include alternate explanations (e.g., not interested/motivated to do analysis, feeling "uncredentialed," lack of educational support for analytical skills. Response 8: This is a valid point. To make up for it, we have added further considerations on alternate post-hoc explanations such as that those participants who were highly engaged were overall more motivated to participate in both phases. Furthermore, those participants who were highly engaged in the data collection possibly held a specific set of knowledge and skills that also helped them to engage in the data analysis more strongly. “We suppose as a post hoc explanation for why highly active participants also engaged in data analysis that those participants felt ownership (i.e., an individuals’ feeling of possessing a concrete or abstract object or entity; [45]) for the data, that they themselves collected and contributed. This could well be the case, as research on project ownership has indicated that high project ownership promotes deeper involvement in problem solving in the inquiry process [46]. Other possible explanations for why highly active participants in the data collection also engaged in data analysis could be that those participants were in general more motivated to participate in all phases [47] or even held a specific set of knowledge and skills that helped them to analyze the data as well [48].” (p. 29) Thank you very much again for your positive feedback. We hope that we have revised the manuscript in accordance with what you had in mind. The revisions have improved the manuscript and we hope you agree with us that it is ready for publication. All the best and kind regards, The authors Attachments Attachment Submitted filename: Response to the Editors and Reviewers_final.docx https://doi.org/10.1371/journal.pone.0275785.r002
26 Sep 2022 Decision Letter - Andrea Fronzetti Colladon, Editor I’m fine with collecting data: Engagement profiles differ depending on scientific activities in an online community of a citizen science project PONE-D-22-09613R1 Dear Dr. Bruckermann, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Andrea Fronzetti Colladon, Ph.D. Academic Editor PLOS ONE Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed Reviewer #3: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: I Don't Know Reviewer #3: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for addressing the comments provided in the first round of revisions. I think this is a strong manuscript and discuss citizen science in an interesting light. Reviewer #2: (No Response) Reviewer #3: Thank you for the revision. You have adequately addressed all of my comments and I have no further recommendations. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No Reviewer #3: No ******** https://doi.org/10.1371/journal.pone.0275785.r003
Formally Accepted
28 Sep 2022 Acceptance Letter - Andrea Fronzetti Colladon, Editor PONE-D-22-09613R1 I’m fine with collecting data: Engagement profiles differ depending on scientific activities in an online community of a citizen science project Dear Dr. Bruckermann: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Prof. Andrea Fronzetti Colladon Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0275785.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .