A method for campus-wide SARS-CoV-2 surveillance at a large public university

Terren Chang; Jolene M. Draper; Anouk Van den Bout; Ellen Kephart; Hannah Maul-Newby; Yvonne Vasquez; Jason Woodbury; Savanna Randi; Martina Pedersen; Maeve Nave; Scott La; Natalie Gallagher; Molly M. McCabe; Namrita Dhillon; Isabel Bjork; Michael Luttrell; Frank Dang; John B. MacMillan; Ralph Green; Elizabeth Miller; Auston M. Kilpatrick; Olena Vaske; Michael D. Stone; Jeremy R. Sanford

doi:10.1371/journal.pone.0261230

Peer Review History

Original SubmissionJuly 19, 2021
2 Sep 2021 Decision Letter - David M. Ojcius, Editor PONE-D-21-23418 A method for campus-wide SARS-CoV-2 surveillance at a large public university PLOS ONE Dear Dr. Sanford, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. In particular, the reviewers request clarification of some experimental procedures and statistical analysis. Please submit your revised manuscript by Oct 17 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, David M. Ojcius Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following financial disclosure: “R35GM130361 GM095850 The work was funded by the Office of Research at UC Santa cruz” At this time, please address the following queries: a) Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution. b) State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.” c) If any authors received a salary from any of your funders, please state which authors and which funders. d) If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.” Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 3. Thank you for stating the following in the Competing Interests section: “I have read the journal's policy and the authors of this manuscript have the following competing interests: Jeremy Sanford and Michael Stone are paid consultants and have an ownership interest in SummerBio, a commercial COVID testing laboratory that specializes in institutional surveillance.” We note that one or more of the authors are employed by a commercial company: name of commercial company. a. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form. Please also include the following statement within your amended Funding Statement. “The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.” If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement. b. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf. 4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: -There are several references that are not pre-prints or peer-reviewed publications (e.g., 27) and Ref. 12 has no information besides a link. Also, please check formatting for things like FDA documents and laws (CLIA). It is unlikely that they are best cited as a URL. -Many aspects of your introduction remind me of a paper out of UC Berkeley from June 2020 (https://www.nature.com/articles/s41587-020-0583-3), also covered in Francis Collin's Blog https://directorsblog.nih.gov/2020/05/12/pop-up-testing-lab-shows-volunteer-spirit-against-deadly-pandemic/. Nevertheless, this paper is scarcely cited. It looks like others have been published as well... https://www.jbc.org/article/S0021-9258(17)50379-8/fulltext & https://www.nature.com/articles/s41564-020-00818-3 & https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251296. This manuscript would benefit from a more thorough literature review and diligent referencing. -Instead of referring to "Many studies" in "Many studies have suggested that oral swabs represented a facile self-collection alternative to NP swabs compatible with mass testing [24]", it may help to clarify that it was a meta-analysis (e.g., "A meta-analysis of __ studies suggests that..." -We have practically no information regarding how the comparison studies were carried out. Were these tests carried out under an IRB? If not, what medical justification was there to collect both samples from patients? If so, what is the protocol #? What is the time difference between OP/NP samples? What are the characteristics of this population? -What sort of statistical analysis did you do to arrive at the statement "After careful consideration of the literature and further validation studies in our lab, we implemented a supervised self-collected anterior nares (AN) nasal specimen." Are the "further validation studies" described anywhere? -Who were the "healthy donors" in "We performed a comparator study using healthy donors to determine if sample integrity was maintained in the 1.4 mL tubes compared to the 15 mL sample collection tubes." What sorts of protocols were used for recruitment, informed consent, etc? -Since there are already "blueprint" papers out there like this, it would be helpful to see how the methods outlined here compare to the previously published efforts. Currently, the manuscript reads almost like a methodology paper without comparison to existing methods. -Please make sure that the claim "Although there are many reports of the theoretical implementation of sample pooling approaches for COVID-19 surveillance testing [33]; [34]; [35], there are few reports showing how this process has been put into practice on an institutional level." is accurate. There are reports I'm aware of that describe this. -Is the LIMS you built truly the result of only a collaboration between your group and Third Wave Analytics? ("A custom laboratory information management system (LIMS) was developed in collaboration with Third Wave Analytics (San Francisco) using the Salesforce Lightning Platform and Experience Cloud.") Reading over the June 2020 paper from UC Berkeley, it looks like they also built such a system with Third Wave Analytics. If the system you built was at all an adaptation of this or a system another group built, it would be prudent to cite that group. -The section entitled "Impact of surveillance on campus-wide SARS-CoV2 transmission" is somewhat misleading. No epidemiological methods were employed in this paper to demonstrate that surveillance done affected overall transmission. We are not provided with any of the surveillance parameters for non-student groups (e.g., frequency, % of population of interest), nor prevalence in the local community. -Turn around time is mentioned in the introduction, but we are provided with no actual data about it in the body of the manuscript. Please provide analysis with the raw data. -Please check figure #s before resubmission. As a final note, the document has inconsistent formatting with paragraph breaks an indentation. It would be appreciated if revisions were properly formatted for ease of navigation. Reviewer #2: Overall this is a well written and clearly presented paper that does not require major revisions. It describes the practical implementation of a pooled testing strategy for automated handling, testing and tracking of a high volume of self-collected COVID-19 nasal swab samples, and should be of interest to audiences planning to implement or optimize strategies for high volume viral testing. While pooled strategies for campus COVID-19 surveillance have been widely mentioned in nontechnical news reports, this manuscript provides both technical details of implementation and useful supporting research that was conducted in order to validate the strategy and to optimize protocols. In the results section, the statement "Surprisingly we observed a slight, but significant difference in Ct values for RP from negative individual surveillance samples and negative clinical samples (mean Ct 24.77 and 25.10, respectively)." remains as a bit of a loose end that needs to be explained more completely. The expectation (and observation) in the pooled surveillance samples was that there would be a difference in the two Ct values due to dilution -- why is there a different expectation for RP and what might be an explanation for the observed results? The abbreviation RP does not seem to be defined at its first occurrence. Figure 1 might be less cluttered if presented as box and whisker plots, and I don't think the explicit connection between sample pairs is necessary to the point of the figure. Figure 2 labels contain misspellings of the word "Individual". Are automation programs for the robotics and custom LIMS code available anywhere e.g. on a GitHub? It seems like those might be useful for other teams trying to implement similar protocols or using the same hardware for related purposes. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0261230.r001
Revision 1
23 Nov 2021 Author Response Specific points 1. Reviewer #1: -There are several references that are not pre-prints or peer-reviewed publications (e.g., 27) and Ref. 12 has no information besides a link. Also, please check formatting for things like FDA documents and laws (CLIA). It is unlikely that they are best cited as a URL. All citations have been updated and are now in the correct format. 2. Many aspects of your introduction remind me of a paper out of UC Berkeley from June 2020 (https://www.nature.com/articles/s41587-020-0583-3), also covered in Francis Collin's Blog https://directorsblog.nih.gov/2020/05/12/pop-up-testing-lab-shows-volunteer-spirit-against-deadly-pandemic/. Nevertheless, this paper is scarcely cited. It looks like others have been published as well... https://www.jbc.org/article/S0021-9258(17)50379-8/fulltext & https://www.nature.com/articles/s41564-020-00818-3 & https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251296. This manuscript would benefit from a more thorough literature review and diligent referencing. We indeed drew inspiration from our colleagues at UC Berkeley. We now include a broader array of references for other campus testing programs as well. 3. Instead of referring to "Many studies" in "Many studies have suggested that oral swabs represented a facile self-collection alternative to NP swabs compatible with mass testing [24]", it may help to clarify that it was a meta-analysis (e.g., "A meta-analysis of __ studies suggests that... 4. We have practically no information regarding how the comparison studies were carried out. Were these tests carried out under an IRB? If not, what medical justification was there to collect both samples from patients? If so, what is the protocol #? What is the time difference between OP/NP samples? What are the characteristics of this population? All of the work was conducted by our diagnostic lab, operating under the license of the UCSC Student Health Center, which is subject to the regulatory framework that governs diagnostic laboratories, including CLIA guidelines. The comparison studies do not qualify as university research. IRB review was not necessary for this work as it focused on validating and improving the quality of our diagnostic test and was performed according to CLIA guidelines. The data was not generated for and cannot be used for university or other research; summary data is presented in this paper with the sole purpose of describing the process of validation and test improvement that we underwent in order to maximize text reliability, accuracy and value. Sample collection was performed by our clinical partners from health care providers (UCSC Student Health Center, Salud Para La Gente and Santa Cruz Community Health Centers) after review by their respective ethics committees. Because the comparisons of OP and NP were not directly relevant to the description of our surveillance program, we removed this figure from our revised manuscript. 5. What sort of statistical analysis did you do to arrive at the statement "After careful consideration of the literature and further validation studies in our lab, we implemented a supervised self-collected anterior nares (AN) nasal specimen." Are the "further validation studies" described anywhere? To improve the clarity of the manuscript we deleted this rationale. Because the FDA treats NP and AN as equivalent upper respiratory samples, we opted for AN due to the ability to perform supervised self-collection. The only validation studies performed are described in the previous figure 2, which directly compared AN to NP swabs, and OP to NP swabs from symptomatic individuals. 6. Who were the "healthy donors" in "We performed a comparator study using healthy donors to determine if sample integrity was maintained in the 1.4 mL tubes compared to the 15 mL sample collection tubes." What sorts of protocols were used for recruitment, informed consent, etc? To improve clarity, this comparison is omitted in the final version of the manuscript. However, volunteers were recruited from the laboratory staff and patients visiting the UC Santa Cruz Student Health Services for COVID—19 tests. Participants provided informed consent for the study. 7. Since there are already "blueprint" papers out there like this, it would be helpful to see how the methods outlined here compare to the previously published efforts. Currently, the manuscript reads almost like a methodology paper without comparison to existing methods. We now provide detailed comparisons between the UC Berkeley IGI blueprint paper and our approach. This text is part of the discussion. We describe how our choice of sample collection tubes and liquid handlers enabled pooling and required a custom LIMS. 8. Please make sure that the claim "Although there are many reports of the theoretical implementation of sample pooling approaches for COVID-19 surveillance testing [33]; [34]; [35], there are few reports showing how this process has been put into practice on an institutional level." is accurate. There are reports I'm aware of that describe this. We revise the text according to the reviewers suggestion. 9. Is the LIMS you built truly the result of only a collaboration between your group and Third Wave Analytics? ("A custom laboratory information management system (LIMS) was developed in collaboration with Third Wave Analytics (San Francisco) using the Salesforce Lightning Platform and Experience Cloud.") Reading over the June 2020 paper from UC Berkeley, it looks like they also built such a system with Third Wave Analytics. If the system you built was at all an adaptation of this or a system another group built, it would be prudent to cite that group. We followed UC Berkeley’s blueprint by contracting with Thirdwave, but our process (sample collection tubes, accessioning, liquid handling) was sufficiently different that developing a custom LIMS was necessary. This was particularly true for the pooled sample test. 10. The section entitled "Impact of surveillance on campus-wide SARS-CoV2 transmission" is somewhat misleading. No epidemiological methods were employed in this paper to demonstrate that surveillance done affected overall transmission. We are not provided with any of the surveillance parameters for non-student groups (e.g., frequency, % of population of interest), nor prevalence in the local community. We revised this section of the manuscript by highlighting a recently published epidemiology study by Pollock et al. that demonstrates that COVID—19 incidence in 19-29 year old demographic is significantly lower on UC Campuses than within the surrounding county. These data suggest that the multilayered mitigation strategies implemented by the UC, including asymptomatic surveillance, have an impact on SARS-CoV-2 transmission. Pollock BH, Kilpatrick AM, Eisenman DP, Elton KL, Rutherford GW, Boden-Albala BM, et al. Safe reopening of college campuses during COVID-19: The University of California experience in Fall 2020. PLoS One. 2021;16: e0258738. We also provide a comparison of the positivity rates for our campus surveillance plan and the clinical testing we performed for the safety-net health car providers (Figure 5D). 11. Turn around time is mentioned in the introduction, but we are provided with no actual data about it in the body of the manuscript. Please provide analysis with the raw data. We now include a plot and turnaround time statistics for all samples received by the diagnostic lab (Figure 2F). 12. Please check figure #s before resubmission. We double checked all call outs to all figures in the revised manuscript 13. As a final note, the document has inconsistent formatting with paragraph breaks an indentation. It would be appreciated if revisions were properly formatted for ease of navigation. We reformatted the manuscript prior to resubmission. Reviewer #2: Overall this is a well written and clearly presented paper that does not require major revisions. It describes the practical implementation of a pooled testing strategy for automated handling, testing and tracking of a high volume of self-collected COVID-19 nasal swab samples, and should be of interest to audiences planning to implement or optimize strategies for high volume viral testing. While pooled strategies for campus COVID-19 surveillance have been widely mentioned in nontechnical news reports, this manuscript provides both technical details of implementation and useful supporting research that was conducted in order to validate the strategy and to optimize protocols. 1. In the results section, the statement "Surprisingly we observed a slight, but significant difference in Ct values for RP from negative individual surveillance samples and negative clinical samples (mean Ct 24.77 and 25.10, respectively)." remains as a bit of a loose end that needs to be explained more completely. The expectation (and observation) in the pooled surveillance samples was that there would be a difference in the two Ct values due to dilution -- why is there a different expectation for RP and what might be an explanation for the observed results? This result is somewhat hard to explain. The individual surveillance samples refer to tests completed before we implemented pooling, so dilution is unlikely to be an explanation for this source of variation. Indeed, the pooled test only dilutes the SARS-CoV-2 signal, whereas RP is an endogenous gene and present at similar levels in all samples. The most logical explanation of this variability is in a clinician collected vs self-collected surveillance sample. We now raise this possibility in the revised manuscript. 2. The abbreviation RP does not seem to be defined at its first occurrence. We now define RP at the first occurrence in the revised manuscript. 3. Figure 1 might be less cluttered if presented as box and whisker plots, and I don't think the explicit connection between sample pairs is necessary to the point of the figure. During the process of revising the manuscript, we decided to remove this figure because it distracts from the clarity of the paper. Instead, we replaced this figure with an overview of our processes. 4. Figure 2 labels contain misspellings of the word "Individual". We corrected this typo in Figure 2. We also improved the labeling by referring to pooled or unpooled samples. 5. Are automation programs for the robotics and custom LIMS code available anywhere e.g. on a GitHub? It seems like those might be useful for other teams trying to implement similar protocols or using the same hardware for related purposes. We created a github account and uploaded all automation scripts. We refer to this repository in the methods section. The github account is available here: https://github.com/UCSC-CCDL/Bravo-protocol-files Attachments Attachment Submitted filename: Reviewer_Response.pdf https://doi.org/10.1371/journal.pone.0261230.r002
25 Nov 2021 Decision Letter - David M. Ojcius, Editor A method for campus-wide SARS-CoV-2 surveillance at a large public university PONE-D-21-23418R1 Dear Dr. Sanford, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, David M. Ojcius Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0261230.r003
Formally Accepted
9 Dec 2021 Acceptance Letter - David M. Ojcius, Editor PONE-D-21-23418R1 A method for campus-wide SARS-CoV-2 surveillance at a large public university Dear Dr. Sanford: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. David M. Ojcius Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0261230.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .