Responses to the editor and reviewers

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

Answer: Thank you for directing us to the relevant links. We have made the required formatting changes across the manuscript. Please also note that the earlier submitted file “S1 Appendix” has now been uploaded separately as “S1 Table” and “S1 figure” with the requested format.

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

Answer: Thank you for highlighting this. In the Ethics statement and details of informed consent section, we mentioned that “no human subjects were involved for this hypothetical costing exercise”. This can be found in line 143-144 of the document. The observation only captured the workflow for each of the constituent tests during a testing cycle and did not observe any human subjects.

3. Thank you for stating the following in the Acknowledgments Section of your manuscript:

“This work was funded through multi-donor core funding to Foundation for Innovative New Diagnostics (FIND). We would like to also acknowledge the contributions of Wendy Susan Stevens and Lesley E Scott, University of Witwatersrand, who oversaw the time-and-motion study that assisted us with certain cost estimates. This work was also supported by the New Faculty Startup Fund from Seoul National University (HS)”

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“The author(s) received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

Answer: Thank you for the appropriate guidance. We have now removed all funding information from the Acknowledgements or other sections of the manuscript. We have reflected this information in our new cover letter. We thank you for editing the online submission on our behalf.

4. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

Answer: We have now uploaded, as supplementary information, the minimal data set underlying the results described in our manuscript. We have reflected this detail in our revised cover letter and thank you for updating the Data Availability Statement.

5. Please amend the manuscript submission data (via Edit Submission) to include author Lesley E Scott.

Answer: Thank you for pointing this out. Lesley E. Scott had requested to be removed as co-author and we thanked them in the Acknowledgements Section. We had erroneously left their name in the author list and have now removed the same.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this paper Malhotra et al conducted a cost analysis of two groups of moderate complexity NAATs with different testing throughput: Lower Throughput (LT, < 24 tests per run) and Higher Throughput (HT, upto 90+ tests per run). This cost analysis study is of great importance to guide and assist the options and placement of assays and systems in resource limited settings. Authors used per-test cost as the main indicator to assess the drivers of cost by resource types and optimized levels of annual testing volumes for the respective NAATs. The authors reached base-case per test costs of $18.52 (range: $13.79 - $40.70) for LT test and $15.37 (range: $9.61 - $37.40) for HT test. Authors assumed equivalent performance and infrastructural needs, for group-comparative purposes.

Its a well grounded, well preformed and consistent work. The conclusions are supported by the results. This work provides important information especially for LMIC TB countries and TB programs in the decision making regarding new molecular technology implementation. The paper brings relevant information to science, especially implementation science applied to TB laboratories in LMIC and for the management of M/XDRTB.

There are few aspects that need to be improved:

Answer: We thank you for your encouraging comments and overall feedback. Please see our responses to your specific queries below.

- In the introduction the advantages and limitations of molecular biology based testing incorporation in TB programs needs to be addressed and reviewed, especially accuracy/performance limitations, connection with clinical outcomes and program aims and cost-effectiveness. The criteria to place these systems in primary care vs centralized labs and TB networks, related with the market target of LT avd HT systems ins not explored nor presented. This subject is relevant and is very well addressed in the WHO guidelines from where this study is a helpfull spin-off.

Answer: We appreciate your thoughtful comment. We made necessary revisions to reflect your comments and provided advantages and drawbacks of molecular testing for the centralized assays in question. We have also referenced the WHO guidelines and relevant literature for the same. These revisions are made in lines 56-62: “These automated NAATs offer higher throughput capacity (single-run capacity: 24 to 96 samples) with comparable performance and lower infrastructural needs compared to WHO endorsed DSTs, thus being advantageous to both patients and programs (9,10). These NAATs are faster and less complex than phenotypic culture based drug susceptibility testing (DST) and line-probe assays (LPA), being largely automated after the sample preparation step(9). Placement of these NAATs strategically in relation to the point-of-care can help improve DST coverage and DR-TB patient management(11). However, successful implementation of these NAATs is dependent on a robust sample transport network and a high population density(9). This may potentially lead to exclusion of low population communities with poor infrastructure.”

- In the materials and methods the authors should clearly define which molecular systems endorsed by WHO were included in the Low Throughput (LT) group and which were included in the High Throughput (HT). For a non expert its difficult to differentiate the inclusion criteria for each of the groups, the available and endorsed systems and another concept also used in the paper- MC-NAATS. The paper as it is, is difficult to connect the results with, at least, the canonical examples of WHO endorsed LT and HT systems, as defined by the authors. Please define better the groups inclusion criteria and give examples of systems included.

Answer: Thank you for your comment. In the Intervention sub-section under Methods (lines 75-78) we have defined what we mean by low throughput and high throughput sections. Since this is a hypothetical costing exercise, and is manufacturer agnostic, we did not want to name the specific device(s). Our intention is to only comment on the drivers of cost and placement strategies of these MC NAATs. Revisions are made as the following:

“We defined Low Throughput (LT) system as those test platforms with a capacity to test up to 24 samples per batch (i.e., singe test run). High Throughput (HT) test was defined as a system with a capacity of test up to 96 samples per batch, and which required reflex testing for confirmatory diagnosis.”

- The references in the text use different formats. Please check and correct.

Answer: Thank you for catching this. We noticed a superscript error for one of the references and have corrected the same.

- The quality of the graphs is poor and should follow PlosOne authors guidelines for figures.

Answer: Thank you for directing us to the PlosOne guidance for the figures. We have upgraded the quality of our figures (minimum pixel and appropriate size requirements) and have saved them using the correct extension (TIF). As recommended by the editor, these images were uploaded to the PACE digital diagnostic tool for images and now meet the PlosOne requirements.

Reviewer #2: In this article, the authors studied the low- and high throughput MC-NAAT for TB DST. This is an assumption-based study and the authors report that the cost effectiveness depends on the testing volume. I think this is a pre-mature study, and the real-world trend could be very different. Hence, it is advised to test their assumptions at least at a minimal TB testing centers and compare the cost, turnaround time, reliability of the results with available standard DST testing techniques.

Answer: We thank you for your thoughtful comment. Despite being a hypothetical study, our analyses are based on time estimates and laboratory resources used for the entire testing workflow directly observed for various testing volumes. from our time and motion sub-study. We have performed a range of sensitivity analyses to reflect operations at smaller (or larger) laboratories that has low (minimal testing) and high testing volumes. These scenarios reflect a range of laboratory operations from minimal to high volume testing (see our sensitivity analysis section). As our study was carried as part of an initial evaluation study (i.e., proof of concept study) by FIND, we did not have a comparative data on turn-around-time. Furthermore, our aim was to estimate and compare per-test cost of low and high throughput MC-NAATs, not compare these costs to conventional DST. Therefore, we did not have empiric cost estimates for conventional DST (e.g., LPA or culture). However, diagnostic accuracy of these MC-NAATs were assessed as part of the main study and these results are reported in (REF – FIND MC-NAAT proof of concept study results in JCM). To address the reviewer’s concerns related to this matter, we have added texts in the discussion section as the following:

Limitations in not being able to compare test performance, turn-around-time, and costs of MC-NAAT to conventional DSTs:

Lines 248-257: “Third, we did not account for costs associated with differences in test performance (accuracy), rate of repeat testing (due to failed or indeterminate test results), and other operational factors such as turn-around-times against the current standard DST. Regarding the test performance, the parent study reported equivalent test accuracies for MC-NAATs compared to existing molecular DST tests (e.g., Xpert MTB/RIF and Line Probe Assays, LPA). Turn-around-times for MC-NAAT, while theoretically being a same-day test (i.e. full testing time is less than 4 hours in total from sample processing to result), can depend heavily on how frequently can the laboratory run MC-NAAT tests (per week or per month). In very low volume laboratories (i.e. once weekly or bi-weekly testing, corresponding to annual testing volumes less than 1,500 samples), there may not be any benefits in turn-around-times nor in terms of per-test costs relative to the conventional DST.”

Costs associated with other operations (e.g., sample transport network)

Lines 258-264: “Placement for MDR-TB diagnostics should be informed by a strategic cost-effective approach that considers logistics outside of the laboratory as well, including but not limited to sample transport related costs, presence of staff with requisite training, access to favorable reagent deals, multiplexing capabilities, and service volume. There is a push to make TB-NAAT testing accessible via single module machines at lower level/decentralized health facilities, to increase point of care testing and reduce loss to follow up at the early diagnosis stage(19). However, such an intervention may not be cost-effective for all TB programs, especially in a low population setting with small testing volumes.

Centralized testing requires an efficient sample transportation network which would include costs attributed to staff time, fuel, and sample storage, and costs for implementation of such a network in regions where no such infrastructure currently exists. In addition, the need to transport samples to the central facility and wait for results can lead to gaps in the TB care cascade, leading to patients being lost to follow up at initial stages of TB diagnosis and delaying treatment. This results in poor health outcomes such as under detection or delayed detection of both drug-susceptible and drug-resistant TB.”

Reviewer #3: Overall comments

This research paper deals with an important subject of determining the conditions under which per-test costs can be optimized for each NAAT and evaluating the drivers of the cost. Overall this is a good work but the authors have to address few queries and provide few clarifications.

Specific Comments

• The authors have used two hypothetical laboratories in a resource-limited setting, is it the standard methodology followed in studies of this kind? Authors can give references for this to strengthen the methodology.

Answer: Thank you for your comment. While ours is a hypothetical study, we structured our cost analysis based on the costing guideline for TB (and other) diagnostic tests that was developed by one of our co-authors (H Sohn). We have revised the text in the beginning of the ‘Cost analysis’ section in the methods to reflect this:

Line 83-84: “We conducted our costing study based on activity-based, bottom-up costing method as suggested by by Sohn et al., 2009”

As such, we follow-up activity-based, bottom-up costing method where we use time and motion data to define duration of key resource use (e.g., laboratory equipment, human resource, and laboratory space) and actual use of key laboratory (test kits, reagents, etc.) and general consumables (laboratory gloves, pipet tips, etc.). As the parent study was conducted in a single laboratory in South Africa, we were not able to ascertain overhead costs of different ‘types’ of laboratories in a range of low-and-middle-income country contexts. As such, we generated a range of assumptions on costs of overall laboratory overheads, human resource costs, and other operational costs of different types of laboratories where MC-NAATs could be placed in the LMIC context. We test these range of assumptions to demonstrate the robustness of our primary per-test costs and suggest different scenarios in which lower or higher per-test cost estimates may be relevant.

• I suggest the authors to provide some more details on automated NAATs for a proper understanding to the readers

Answer: Thank you for your suggestion. We have expanded on our description of automated NAATs in following lines:

Lines 56-59: “thus being advantageous to both patients and programs (9,10). These NAATs are faster and less complex than phenotypic culture based drug susceptibility testing (DST) and line-probe assays (LPA), being largely automated after the sample preparation step(9)”

And…

Lines 75-78: “We defined Low Throughput (LT) system as those test platforms with a capacity to test up to 24 samples per batch (i.e., singe test run). High Throughput (HT) test was defined as a system with a capacity of test up to 96 samples per batch, and which required reflex testing for confirmatory diagnosis.”

• It has been mentioned that the current study assessed the costs of two moderate complexity automated NAATs, what were these two NAATs?

Answer: Since this is a hypothetical costing exercise, and is manufacturer agnostic, we did not want to name the specific device(s). Our intention is to only comment on the drivers of cost and placement strategies of MC NAATs which were featured in the 2021 WHO Diagnostics Guidelines. Instead, we have provided specifics of these two test platforms based on test throughput (low and high):

Lines 75-78: “We defined Low Throughput (LT) system as those test platforms with a capacity to test up to 24 samples per batch (i.e., singe test run). High Throughput (HT) test was defined as a system with a capacity of test up to 96 samples per batch, and which required reflex testing for confirmatory diagnosis.”

• Capital asset useful life years - 10 years for laboratory equipment and 30 years for building spaces- provide reference for this statement

Answer: Thank you for catching this. We have provided a reference for these values.

• Why have the authors used different discount rates 3% 1% 10% in the study as the discount rates will not change.

Answer: Thank you for your comment. In the real-world context, discount rates may vary. However, for analytical purposes, economic evaluations, including cost analyses, use fixed discount rate for primary estimates. For our analysis, we have used a discount rate of 3% for our primary estimate then varied the discount rates between 1 and 10% to test for uncertainties in our per-test cost estimate, which did not change considerably.

• All costs were captured in 2019 US dollars; if possible, authors can convert it to the current value for the years 2022 or 2023.

Answer: Thank you for your suggestion. With all due respect, while it is possible to cover our estimates to 2022 or 2023 US dollars, generally it would be best to provide cost estimates that align with the dates/years of the primary study. Hence, we would like to keep our cost estimates as 2019 US dollar figure.

• It would be good if the authors provide methodology with sub-titles such as model structure, parameters used, data collection and source, and analysis

Answer: We appreciate your suggestion and have added sub-titles throughout.

• Primary estimate (PE) to be estimated in the figures

Answer: Thank you for your comment. In our figure captions for all 3 of our figures, we have specified which values/symbols refer to the primary estimate or base case. Please let us know if we are missing anything and we will be able to rectify the same.

• In figure two, the lower value for testing days seems to be very high, please check

Answer: Thank you for requesting us to check this. The “Low” value for testing days seems high because of the relative difference in the number of testing days. In the “Low” scenario, we assumed testing for one day a week whereas in the primary estimate we assumed testing for 5 days a week, as specified in table 1.

________________________________________

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

________________________________________

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step

Answer: Thank you for this tip. The images were uploaded and then downloaded from the PACE tool after correction.

Attachments

Attachment

Submitted filename: MC NAAT Response to Reviewers_R1.docx