PONE-D-22-13832Diagnostic accuracy of a SARS-CoV-2 rapid antigen test among military and civilian personnel of an Air Force airport in central ItalyPLOS ONE

Dear Dr. Spada,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The work described in this manuscript is valid even though the study is not original and the results were predictable based on previous literature. However, the manuscript should be revised for clarity, internal congruency and completeness of details and information. The reviewers identified critical issues and concerns that should be addressed point-by-point in the revised manuscript. If you decide to re-submit the manuscript, I'd also invite you to consider reducing the number of tables as suggested by one reviewer.   

Please submit your revised manuscript by Sep 05 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Luisa Gregori

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

4. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors tested from November 2020 to April 2021, 1294 nasopharyngeal swab samples from 1183 participants with the rapid antigen test AFIAS COVID-19 Ag, a fluorescence-based rapid antigen test. This is a not wide-spread rapid antigen test, not easy to work with as the development of the final reaction is fluorescence and a little instrumentation is required for reaction reading. Results were compared with standard RT-PCR (gene E and N, by Altona) according to the presence of symptoms and the level of viral load as expressed by the Ct value as a proxy. Testing was performed within the military and civilian personnel of a military airport, located in the metropolitan area of Rome, who underwent a screening program to control transmission of SARS-CoV-2 infection, Prevalence rate of Covid-19 in the studied population: 3.78%.

Forty-nine samples (3.78%) were positive by RT-PCR (32 of them with a high viral load: Ct range 12.12-24.86), while 54 samples were positive by AFIAS COVID-19 Ag; 18 samples were false negative by AFIAS and 23 were false positive. AFIAS overall sensitivity, specificity, positive and negative predictive values were 0.633, 0.981, 0.574, 0.985, respectively with only a moderate level of concordance with RT-PCR. These figures increased when comparison was made in samples with a high viral load (Ct ≤22). Similar results were achived if AFIAS COVID-19 Ag was compared with RT-PCR results in symptomatic patients and this is certainly a results of higher viral loads in these patients. Instead, and as expected by the scientific literature, test sensitivity was poor for samples from asymptomatic participants and for viral load CT >30.

In conclusion, the authors stated that AFIAS COVID Ag showed high specificity but only moderate sensitivity in the screened population where the prevalence of COVID-19 was, by the way, low. However, and again as expected, the assay showed good sensitivity for samples with high viral load and in participants with COVID-19 compatible symptoms. Thus, in high prevalence setting this test can be useful for COVID-19 case identification and management at a point-of-care level.

The work is in line with many other papers on rapid antigen testing in COVID-19 pandemic showing that the sensitivity of these type of tests is higher in symptomatic than in asymptomatic patients and that using the Ct value as a proxy for viral load, the sensitivity increases with the increasing of viral loads. All these data have already been acknowledged by the scientific and medical community, the only reason to ask a re-submission from the authors is that there are not so many data on the type of rapid antigen test used by them, e.g. AFIAS COVID-19 Ag and it would be certainly usefull to add more data to the existing ones. The good correlation with RT-PCR is only for Ct <22, which means that the technique of this rapid test e.g. fluorescence, is not as sensitive as the one of other rapid test working with fluorescence in the same context (Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2020 Aug 26;8(8). There are no comments by the authors on this important issues.

Therefore, my suggestion is to re-write it as short report, making the study more synthetic, technical and readable, cutting down the number of tables to the essential (just one as the overall evaluation and one more according to the Ct level). The Ct level chosen for statistical analysis should be the same across all the study (e.g. abstract ct<25, table Ct< 22.34, text Ct <22), as there are only 49 positive samples by RT-PCR, 18 of them negative by the antigen test (the authors are working on 31 positive concordant samples, which is not that much). Therefore, there is no clue in making several subsets of positive samples/patients (and each one corresponds to a table!) since they are just few positive samples by the gold standard RT-PCR. This gives the work a strong reading difficulty and hard to follow the analysis. Moreover, it is not clear if the statistical analysis is run on sample or patients (1294 vs 1183, text vs. tables).

Reviewer #2: The authors evaluated a SARS-CoV-2 rapid antigen detection test (RADT) while using RT-PCR as the gold standard. The works were scientifically sound. However, mistakes and confusions were found and I have several queries for the current version of the manuscript. I hope the authors will find them useful to revise it so that the quality could be improved.

- the research gap was not well defined:

(1) lines 95-100: the authors have to explain the reasons for choosing ‘military and civilian personnel of a military airport in the metropolitan area of Rome’ as the subjects

(2) lines 69, 87-88, 114-121: the authors stated that RADTs are easy to use since equipment was not required, however, the authors employed the fluorescence based RADT in the study. This test requires an equipment to read the test results. It seems that the introduction was discordant to the method used.

- line 172: you have to mention the meaning of the cut-off value Ct 22.34, readers will not know that unless they went through the table 1b

- lines 220-221, 272-273, table 3, footnote a: you have to define group B and group C clearly. I still do not know the differences between groups B and C after going through these explanations. I only know that participants in groups B and C had contact history with COVID-19 patients within the last 14 days. Participants in groups B will be more likely to be detected by RADT than those in group C. If I understand correctly, groups B participants had contact history with COVID-19 patients tested positive by RADT while groups C participants had contact history with COVID-19 patients tested positive by RT-PCR. Please confirm my speculation.

- the authors analyzed the RADT results according to the four different parameters: (1) EU HSC (2) RT-PCR results (3) presence of symptoms (4) contact history with COVID-19 cases. It means that many different cut-offs were used:

(1) EU HSC cut-offs: <25, 26-30, 30-36, >36 (lines 141-142)

(2) median RT-PCR: 22.34

(3) symptomatic patients: 18.08

(4) asymptomatic patients: 26.00

For (1), a table is preferred rather than just describing the results in text only (lines 179-182).

Minor comments:

- avoid creating unnecessary abbreviations if the fluency is not improved, it is not inconvenient to spell the terms SS (line 73) and SP (line 75) in full as sensitivity and specificity respectively. I cannot see those terms will either save the word counts or make the manuscript more neat and tidy. In addition, these two terms are not commonly used by other research groups. Readers have to memorize them throughout the manuscript. It is easy to create confusion. On the other hand, the term ‘N’ referring nucleocapsid protein created (line 117) has not been used in subsequent texts. All these kinds of arrangements make this manuscript quite unprofessional.

- there were only two different assays, RT-PCR and RADT, it is not necessary to create another term, index test in lines 34, 41, 44, 49, 169, 200, 211, 254, 260. As you define RADT at the beginning, you can either use this term throughout the manuscript or make a short form for the RADT that your performed in your study, ‘AFIAS’ to refer ‘AFIAS COVID-19 Ag’.

Reviewer #3: In this manuscript, Authors report the results of a screening program to control transmission of SARS-CoV-2 infection in the workplace. The study was conducted from November 2020 to April 2021 on the personnel of a military airport in Rome. Tests were performed with immunochromatographic fluorescence-based rapid antigen test designed to detect the nucleocapsid protein (N) of SARS-CoV-2 in nasopharyngeal swab specimens.

The study was conducted appropriately. Nevertheless, an important limitation is that the study was carried out almost a year and a half ago: the epidemiological situation and the variants circulating today are different. Moreover, the topic of the article appears to be of limited interest since it has been extensively covered in similar published works regarding the same rapid antigen test and others similar. In addition to the two studies already mentioned in the manuscript’s discussion, some other examples are reported below:

- Baccani I, Morecchiato F, Chilleri C, Cervini C, Gori E, Matarrese D, Bassetti A, Bonizzoli M, Mencarini J, Antonelli A, Rossolini GM. Evaluation of Three Immunoassays for the Rapid Detection of SARS-CoV-2 antigens. Diagn Microbiol Infect Dis. 2021 Oct;101(2):115434. doi: 10.1016/j.diagmicrobio.2021.115434. Epub 2021 May 21. PMID: 34174523; PMCID: PMC8137375.

- Parvu, V.; Gary, D.S.; Mann, J.; Lin, Y.C.; Mills, D.; Cooper, L.; Andrews, J.C.; Manabe, Y.C.; Pekosz, A.; Cooper, C.K. Factors that influence the reported sensitivity of rapid antigen testing for SARS-CoV-2. Front. Microbiol. 2021, 12, 714242.

- Filchakova, O.; Dossym, D.; Ilyas, A.; Kuanysheva, T.; Abdizhamil, A.; Bukasov, R. Review of COVID-19 testing and diagnostic methods. Talanta 2022, 244, 123409.

- Bruzzone, B.; De Pace, V.; Caligiuri, P.; Ricucci, V.; Guarona, G.; Pennati, B.M.; Boccotti, S.; Orsi, A.; Domnich, A.; Da Rin,G.; et al. Comparative diagnostic performance of rapid antigen detection tests for COVID-19 in a hospital setting. Int. J. Infect.Dis. 2021, 107, 215-218.

Here below, Authors can find a list of revisions that need to be addressed in order to improve the quality of the manuscript.

Abstract

- line 38: median age is missing from results

- line 41-44: it is not so clear what concept the authors want to express

Methods

- the method of enlistment is not specified (they were volunteers?)

- It is not specified how was determined the number of people to be enlisted (only time criteria?)

- line 104-106: we suggested to specify the use that will be made of the data contained in the questionnaires

- viral variants circulating at the time of the study are never mentioned in the study. Therefore, data on their detection capabilities for the test under consideration are missing.

- line 139-140: it would be helpful to specify the number of samples with inconclusive results found, whether they were included in the study and the AFIAS COVID-19 test result, if any

- line 140-142: this section reports viral load cut-offs that are not met in subsequent sections

Results

- Table 1b reported a classification of analyzed samples that does not meet the content of Methods section; moreover, there is no explanation of how and why Authors selected and calculated the cut-off and to which of target gene it refers (E or S or both target genes of molecular assay)

- Table 1a: an asterisk is reported near the values of prevalence, but there is no explanation of its meaning

- Table 1a: the number of negative samples is too high compared with PCR positives

- line 185-188: the population appears disproportionate between symptomatic and asymptomatic participants

- Table 2: Same considerations of Table 1b

- Table 3: Same considerations of Table 1b

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Andrea Orsi

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-22-13832R1Diagnostic accuracy of a SARS-CoV-2 rapid antigen test among military and civilian personnel of an Air Force airport in central ItalyPLOS ONE

Dear Dr. Spada,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please re-consider the description of fluorescence-based RADT. 

Please submit your revised manuscript by Dec 15 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Etsuro Ito

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The authors addressed all of my queries, the revised version is better than the previous one. I went through the reviewer 1 and reviewer 3 comments and I shared similar views for some of them. I have the following three comments, the quality of the manuscript can be improved further.

1. The authors should elaborate more about the background of the study by copy and paste your response ‘The study was part of the public health response to control as soon as possible any outbreak occurring in the military airport (as reported in the Scientific Collaboration Protocol signed by the Experimental Flight Center, Italian Air Force Logistic Command and Istituto Superiore di Sanità on 30 November 2020), with simultaneous evaluation of the RADT used for the screening.’ into line 95, after the sentence ‘………….transmission of SARS-CoV-2 infection in the workplace.’

2. Reviewer 1 and I shared similar concerns of using fluorescence based RADT in your study, it is worthwhile to justify the reasons for selecting this RADT instead of traditional RADT without using fluorescence instrument.

3. Both reviewers 1 and 3 raised the concerns of similar studies have been published between 2020 and 2022. I also shared this view when I reviewed the first version, however, the PLOS journal did not focus on the novelty of the research. As I am not the new reviewer in PLOS, that’s why I did not raise this out. I suggest you should go through your manuscript thoroughly, try to write more about the background and the reasons to launch this study. The above comments 1 and 2 are aimed at filling this gap.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Diagnostic accuracy of a SARS-CoV-2 rapid antigen test among military and civilian personnel of an Air Force airport in central Italy

PONE-D-22-13832R2

Dear Dr. Spada,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Etsuro Ito

Academic Editor

PLOS ONE