PONE-D-21-03249

External quality assessment of Rift Valley fever diagnosis in countries at risk of the disease: African, Indian Ocean and Middle-East regions

PLOS ONE

Dear Dr. Cetre-Sossah,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please explain some confusing information for the counting in the positive or negative sera. As asked include some detailled computation for the sensitivity or  CP/IP. Please also take in account that even if belonging to France, as stated by reviewer 2, the La Réunion Island and Mayotte cannot be classified in the very same geographical context, as there is some differences considering these island either related to Est Africa mainland or to the Indian Ocean island.

Please submit your revised manuscript by Apr 25 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Pierre Roques, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating in your Funding Statement:

'This study was partly funded by the SURE project, Préfecture de la Réunion, INTERREG FEDER TROI 2015-

2017 in the framework of the DP One Health Indian Ocean (www.onehealth-oi.org).'

a. Please provide an amended statement that declares *all* the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement.

b. Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

3. We note that Figure 1 in your submission contains map images which may be copyrighted.

All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (a) present written permission from the copyright holder to publish this figure specifically under the CC BY 4.0 license, or (b) remove the figure from your submission:

a. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license. 

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish this figure under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

4. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The aim of this study was to assess the capacity of laboratories to detect antibodies against RVFV in 13 countries, improving the reliability of RVF sero-diagnostics with an inter-laboratory proficiency test. ELISA kit and in house ELISA were used and the specificity and sensitivity of each of them were determined. 20 samples were send for this proficiency test. The status of each sera was determined by the seroneutralisation test considered as the reference. Most of the laboratories find a very good sensitivity and sensitivity. The repeatability of the various ELISA assays was also very high. The dose-response relationship was also followed.

The objectives of this study are clear and de study design is sound. This manuscript needs minor modifications. The most confusing is that authors consider 3 pool of sera, but positive sera are sometime considered in the positive pool, sometime not (ID. 8-10, ID 16-20, specially for the ID 9 -10 - 16)

1 I was confused by the sensitivity calculation. This value must be calculated using independent sera. Here the number of positive independent sera is 8, not 10 (ID 8, 9, 10 are identical and not independent). Please recalculate the sensitivity for each assay.

2 The authors claimed that sera 8 and 16-20 are the same. Why the date of collection vary (15/11/2016 – 07/02/2018). Was the serum collected on two different dates?

3 Table 1

The number of sera and their associated status are highly confusing: 11 are positive but ID8, ID9 and ID10 are the same and must be only considered only in the repeatability test. Then, only 8 sera constitute the positive pool. Seven sera are considered as negative but following the gold standard SNT, I counted 9. Moreover, ID 17 is considered as negative in table2 and did not alter the calculated specificity. Here’s the limit of the SNT can be noticed or the presentation should be changed. Authors cannot change the performance criteria following the sera. Sometime the repeatability assay is included in the results, sometime, dose response result, is not included in the calculation. I recommend that each pool of sera must be considered only for their purpose and not be mixed. The ID of sera at limit of detection must be given for more clarification.

4 table 3

The D sens (%) of the lab1 is false, the Neg (the ID5 being negative) should be inserted for the sensitivity calculation, moreover, the number of positive sera must be 8, not 11 (ELISA pos/ SNT Pos = 7/8). The sensitivity cannot be 100%. Please recalculate sensitivity with 8 positive sera. The authors did not use false positive status (regarding the SNT result) of sera ID 17 in the specificity calculation; I think it is wright if the number of positive sera is 8. I don’t understand the threshold (L206-208) mean + 2SD. (Max-min) is the amplitude of the difference of values, when mean+2SD is a value. The equation could be (max-min)/2 < 2SD ? for both half of the Gaussian. The best criteria must be : Value < Mean +/- 2 SD for the sera.

The ‘SD mean’, ‘SD’ and ‘mean +2 SD’ are calculated for all data. This is not clear as these values are presented in the table. The ‘SD mean’ is the mean? In the legend authors wrote ‘OD values’. Are they really OD value or CP and IP (then %, not OD)

5 L195

The authors must explain briefly in materials and methods how the CP and IP were obtained.

In figure 4 why the authors did not mention the value of ID8 within this graph, as written L196? It should be added if it is the same sera (but because the date of collection is not identical, sample could be different, this must be clarified). The X abscissa must be dilution factor, and log, else, authors must show histogram. The ID of laboratories must be added in the legend, and the positive threshold must be added on the graph.

6 the figure 2 could be removed, because it is non-informative.

7 The figure 3 should be shown in full scale (X abscissa). The positive threshold must be added.

8 Figure 5

Because the sera ID8, ID9 and ID10 are the same, they must be presented in only one box. This figure presented the inter laboratory repeatability, which is different from the intra laboratory repeatability. Authors should divide the two exercises. Then, authors could add and present the qualitative results obtained for all sera. In conclusion, intra laboratory and inter laboratory tests must be clarified.

Reviewer #2: In this study, the authors conducted a serological inter-laboratory proficiency test (PT) to assess the capacity of 18 veterinary laboratories in 13 countries (4 considered to be at risk and 10 to be endemic for RVF) to detect IgG antibodies against RVF. Sixteen laboratories used commercial kits (13 used only the ID screen RVF competition multi-species ELISA kit, 2 used only the INgezim RVF Compac ELISA kit, and 1 used both kits), and 2 used their own in-house ELISA kits. A panel of 20 samples was tested by each laboratory and different criteria were analyzed (sensitivity, specificity, detectability, dose-response relationship, and repeatability). This PT showed that 3 countries need to improve their detection capacities for RVF.

Despite the limitations of the study (described in the discussion), these findings are important from an epidemiologic and public health perspective as efficient diagnostic tools are necessary for the early detection of emerging infectious diseases. Nevertheless, there are minor issues that should be addressed before paper can proceed to publication.

ABSTRACT:

- Lines 54: I suggest adding "the capacity of veterinary laboratories"

INTRODUCTION:

- Line 103: remove parenthesis after “Reunion Island”. Do you consider mainland France and the Reunion Island as one country?

- Lines 102-105: You state that 13 countries participated in the RVF PT. However, 4 countries at risk + 10 countries endemic for RVF = 14 countries. I presume that Mayotte is considered as France but I do not think that it is evident for all readers.

RESULTS:

- Line 189: Are you sure about the kappa value of 0.08 for the in-house tests? Could you provide the detailed calculation?

- Lines 210-212: Details of CP and IP calculation should be previously given at lines 200-201.

DISCUSSION:

- General comment: Since the aim of this kind of study is to improve the diagnostic capacity of laboratories, have the 3 laboratories that provided the worst results planned to use another kit? This should be discussed.

- Line 253: close the parenthesis after "(dilution 1:8"

FIGURES:

Figure 1: I suggest writing on the map the name of the countries and the code of the labs participating in the RVF PT.

In the text at line 113, it is indicated that 13 countries participated in the RVF PT whereas on the map 15 stars are represented. I suppose that Mayotte and Reunion islands are considered as belonging to France. Consequently I suggest indicating on the map "Mayotte (France)" and "Reunion (France)".

Figure 2: Information given in the Figure 2 are already given in the Table 2. I suggest removing this figure and replacing in the text "(Fig 2)" by "(Table 2)".

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

External quality assessment of Rift Valley fever diagnosis in countries at risk of the disease: African, Indian Ocean and Middle-East regions

PONE-D-21-03249R1

Dear Dr. Cetre-Sossah,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Pierre Roques, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No