PONE-D-23-23819A new QRS detector stress test combining temporal jitter and accuracy (JA) reveals significant performance differences amongst popular detectorsPLOS ONE

Dear Dr. Porr,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Reviewers have suggested several revisons. Authors are enocuraged to revise upon the comments and submit again.

Please submit your revised manuscript by Feb 16 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Noman Naseer, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

3. Thank you for stating the following in the Competing Interests section:

“B.P. is CEO of Glasgow Neuro LTD which manufactures the Attys DAQ board.”

We note that one or more of the authors are employed by a commercial company: Glasgow Neuro LTD

a.        Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

b. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. 

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Additional Editor Comments (if provided):

Reviewer's have suggested several revisons. Authors are enocuraged to revise upon the comments and submit again.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: However, some problems still exist, and the manuscript need to be improved by considering the following comments:

1. The Abstract is clear and comprehensive.

2. The figures are clear.

3. Key terms of equations must be defined.

4. Give more detaile aboute the Dataset. Also, the authors have specified the reasons for selecting the dataset.

5.The results section is clear.

6. The authors should mention to "where the proposed system is applicable".

7.The references should be the last five years.

Reviewer #2: The aim of this paper is to demonstrate a new quality measure for the evaluation of QRS detectors. The proposed measure combines temporal jitter with accuracy, which makes it possible to assess the impact of all errors that arise during the detection of QRS complexes, especially in the presence of noise caused by patient movement. In this study, six popular QRS detectors were compared taking into account the proposed IAM quality index. The Glasgow University Database (GUDB) which consists of two-minute two lead ECG 100 recordings from 25 subjects each performing five different tasks, for a total of 125 records, was used for comparison.

The paper is interesting and addresses a problem with clinical applicability. However, I recommend rearranging the manuscript in relation to the research steps, for a better fluency of the information presented.

Thus, overall, the structure of the paper is difficult to follow in relation to the dedicated sections.

The proposed methodology design should be more clarified.

1. The presentation of the database and of the performance measures for detection efficacy should be presented in experimental setup.

2. Computation of accuracy depends on the knowledge of the true negative (TN), namely (equation 15):

Accuracy = (TP + TN) / (TP + TN + FP + FN)

However, in the context of QRS detection, the concept of true negatives is not applicable. The authors currently use an alternative definition of the accuracy which does not include the true negative. However, these alternative definitions do not conform to the standard definitions provided in the literature, and their interpretation is unclear. Therefore, this indicator should be removed from the manuscript or rearrenged.

3. Table 1. The description of the table suggests that all results presented are from the GUDB database, whereas the first column contains results obtained from the MIT-BIH AR database, which is not mentioned in any way and is misleading. Furthermore, what is meant by the phrase unspecified tolerance (%). In the following columns, it is specified that these are results obtained (I think) from the work on the basis of signals from GUDB records. However, combining the results in this way may mislead readers. Another inaccuracy is the value given after the plus/minus sign. What is this value? How was it determined? In addition, Table 1 of Hamilton's paper is assigned the wrong bibliographic number, 31 instead of 24.

4. The Discussions should be oriented towards summarizing the results and comparing them with those of similar studies and presenting the advantages and novelty of the proposed algorithm. In addition, the limits of the research carried out should be identified and briefly presented. Nevertheless, are important to mention the perspectives for the application in clinical practice of the new algorithm, and new research directions derived from the research carried out.

5. There is no conclusion section.

6. No translation of abbreviations, e.g. EMG.

7. What is the relevance to the manuscript of the information that the Pan&Tompkins algorithm was originally written for the Z80 microprocessor?

8. There is a lot of formulations in the paper that are not supported by the results (e.g. line 353-354).

9. No description of the numerical experiment in which a standard deviation value of 35.8% is obtained (line 367). How many times were the calculations performed?

10. It seems to me that for the quality of the work as a whole, it would be advisable to use the MIT-BIH AR database and then compare the results. By which the work would be undoubtedly better appreciated.

11. Line 172: The paper says that the annotation of heartbeats in ECG signals was performed with a Python script using an interactive plot. Were the determined QRS positions verified by an expert cardiologist? From the information available, it appears that the development of this software ended in 2018/2019 and furthermore, no information is provided about the signal processing used.

Reviewer #3: This paper presents an ECG testing database with precise annotations under realistic noise levels and a new benchmarking score (JA) that equally accounts for all error types, including temporal inaccuracies, and is application-independent to truly assess different detector algorithms. The paper idea is interesting. But there are many issues in the paper that need further clarification:

1. The contribution of the paper needs to be properly specified in the abstract.

2. The figures should be place with in the text for better readability of the article during review process.

3. What are the key technical problems that this paper tries to solve and what are the drawbacks of the existing QRS detectors?

4. In the introduction, I suggest that the author rearrange the introduction section.

5. The paper needs to discuss recent developed QRS detection methods and include in the comparison. Some of the recent methods are in the following works:

Design of A Biorthogonal Wavelet Transform Based R-Peak Detection and Data Compression Scheme for Implantable Cardiac Pacemaker Systems

Efficient QRS complex detection algorithm based on Fast Fourier Transform

Design of Efficient Fractional Operator for ECG Signal Detection in Implantable Cardiac Pacemaker Systems

Hardware Emulation of a Biorthogonal Wavelet Transform Based Heart Rate Monitoring system

6. The availability of dataset should be provided in the paper before reference section via link.

7. Why post processing is required.

8. Is the annotation was done through software or by medical practitioners.

9. Discuss the computational complexity of proposed model and compare with the existing methods.

10. There are lots of typo error and grammatical error. That should be corrected.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Ali Noori kareem

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

A new QRS detector stress test combining temporal jitter and F-score (JF) reveals significant performance differences amongst popular detectors

PONE-D-23-23819R1

Dear Dr. Porr,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Noman Naseer, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

All concerns of reviewers have been addressed.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: Overall, the manuscript does seem well-put together now, particularly after the revisions. They have established well-defined parameters to gauge JF in their draft. I am happy with the amendments they have made and would recommend it for publication now.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

**********