Interpretation of continuously measured vital signs data of COVID-19 patients by nurses and physicians at the general ward: A mixed methods study

Harriët M. R. van Goor; Martine J. M. Breteler; Lisette Schoonhoven; Cor J. Kalkman; Kim van Loon; Karin A. H. Kaasjager

doi:10.1371/journal.pone.0286080

Peer Review History

Original SubmissionDecember 8, 2022
21 Feb 2023 Decision Letter - Sampson Twumasi-Ankrah, Editor PONE-D-22-33707Interpretation of continuously measured vital signs data of COVID-19 patients by nurses and physicians at the general ward: a mixed methods studyPLOS ONE Dear Dr. van Goor, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Apr 07 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Sampson Twumasi-Ankrah, PHD Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information. 3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. In your revised cover letter, please address the following prompts: a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. We will update your Data Availability statement on your behalf to reflect the information you provide. 4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: No Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Abstract; The conclusion should speak directly to the three clearly stated aims of finding consensus regarding vital sign interpretation, assess ability to recognize respiratory insufficiency and evaluation of the interpretation process. Semi-structured interview Use of untrained interviewer for the interviews present with inherent limitations and authors are to indicate remedies for minimizing this effect. Authors mention they aimed to have a representative study population from the hospital, this must be explicitly stated under the sample population. Participant inclusion The inclusion criteria of experience with COVID-19 need further information as the frequency and how recent are likely to influence responses Results Under quantitative analysis, authors need to provide a legend on categorizing agreement as low, moderate or high to aid readers appreciate why the choice of these words. Under table 2, authors should indicate at least a general estimate of p value informing the non-agreements e.g. p values less than ……… No analysis on PPV in results section but estimates are stated under the results narration. Discussion First paragraph under discussion do not report on key findings of all three objectives. Line 258-263 Conclusion See above on abstract Minor Comments Line 42 includes should be included Line 43 focused and not focussed Line 44 entailed and not entails Line 282; a collaboration that is should be replaced with “which” Line 287; authors should consider “uniform” instead of unify Line 319; described instead of describe Line 335; focused instead of focussed Reviewer #2: The robust nature of the study with good description of the methods. However the sampling for the quantitative arm of the study did not come out well. The discussion could have been more detailed capturing every set objective. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0286080.r001
Revision 1
14 Mar 2023 Author Response Reviewer's Responses to Questions First of all, let us thank you for your time and effort in reviewing our work. We are very grateful for the chance to improve our report. Secondly, we would like to inform you that during revision, we found a small error in one of the R-scripts: we found that instead of a lower cutoff of 0.025, we accidentally used 0.25, which impacts the lower limit of the 95% CI for Krippendorff’s alpha. We have corrected the script and corrected the values throughout the manuscript. We have thoroughly checked all conclusions drawn based on these values, but none of the conclusions was affected by the new lower limit of the 95%CI. Below you will find our point by point answer to the questions of the reviewers. Reviewer #1: Abstract The conclusion should speak directly to the three clearly stated aims of finding consensus regarding vital sign interpretation, assess ability to recognize respiratory insufficiency and evaluation of the interpretation process. Thank you for pointing this out. We have added the recognition of respiratory insufficiency as a separate finding to the conclusion. All three aims of the study are now (subsequently) included in the conclusion. Semi-structured interview Use of untrained interviewer for the interviews present with inherent limitations and authors are to indicate remedies for minimizing this effect. Although the interviewer did not undergo formal training, she was coached and guided by three members of the research group with experience in interviews studies, to limit the effect of inexperience. This included designing the interview together and thoroughly assessing the interview before it was held. We have added information about coaching to the methods section in line 137/138. Authors mention they aimed to have a representative study population from the hospital, this must be explicitly stated under the sample population. This statement is included in the methods section under ‘Participant inclusion’ in line 100-101. Participant inclusion The inclusion criteria of experience with COVID-19 need further information as the frequency and how recent are likely to influence responses We did not include information as to the frequency and how recent the experience with COVID-19 patients was, since this was not part of our inclusion criteria (any experience was considered sufficient). However, since the study was performed in 2022, all experience was gained during the previous two years (the pandemic) and all participants turned out to have had extensive experience as a consequence of the high case load during the pandemic. We have added this information to the results. Results Under quantitative analysis, authors need to provide a legend on categorizing agreement as low, moderate or high to aid readers appreciate why the choice of these words. The categorization as stated in the methods was added to the title of Table 2. Under table 2, authors should indicate at least a general estimate of p value informing the non-agreements e.g. p values less than ……… We have considered the use of a p-value in this context extensively, however we do not find it to be added value. We will explain why. We have 3 types of results: those with an 95%CI for the Krippendorff’s alpha entirely below 0.67, those with a value below 0.67 but an upper 95%CI limit above 0.67, and those with a value above 0.76 but a lower limit below 0.67. For the first category, we can say with much certainty that the agreement is low, since 95% of all bootstrap results falls below 0.67. For the second category, the value is ‘non agreement’, but the true agreement might still be ‘moderate’ since the upper limit is above 0.67. However, we have no certainty about this. For the last category, the value indicates agreement, but the lower limit is below 0.67, so again we have no certainty about this. These last two categories can be interpreted as ‘non-significant’. The H1 hypothesis of this study was that there would be agreement among professionals regarding interpretation. Since none of these values have a confidence interval entirely above 0.67, we cannot prove this hypothesis, and therefore concluded that we could not find agreement among hospital professionals. Since the 95%CI gives more information about the amount of uncertainty around the agreement value than a p-value, we choose to report the 95%CI. And since all conclusions as described above can be drawn using only a 95%CI, we did not think a p-value would be of added value. To help interpret these findings, we have added a sentence to the results explaining the uncertainty of the respiratory insufficiency expectation results. We hope this aids in the interpretation of the 95%CI. No analysis on PPV in results section but estimates are stated under the results narration. We do not quite understand the point being made in this comment. We have included all results of our analyses, as stated in the methods, in the results section. Quantitative analyses (agreement and positive/negative predicting value) are stated under the quantitative analysis subheading, the results of the interview study are reported under the qualitative analysis subheading. Results including a large number of values are clarified using tables (table 1 and 2). Since the PPV an NPV only have 2 values, these were not summarized in a table. Hopefully this clarifies our structure of the results section. Discussion First paragraph under discussion do not report on key findings of all three objectives. Line 258-263 The objective that was missing was the assessment of the ability to recognize respiratory insufficiency. We have added the findings of this objective to the first paragraph of the discussion. Conclusion See above on abstract Minor Comments Line 42 includes should be included Line 43 focused and not focussed Line 44 entailed and not entails Line 282; a collaboration that is should be replaced with “which” Line 287; authors should consider “uniform” instead of unify Line 319; described instead of describe Line 335; focused instead of focused Thank you for pointing out these errors, we have corrected them throughout the manuscript. Reviewer #2: The robust nature of the study with good description of the methods. However the sampling for the quantitative arm of the study did not come out well. The discussion could have been more detailed capturing every set objective. We thank you for the feedback and are happy to read that the reviewer finds our study robust with a good description of methods. We are a little uncertain whether the ‘sampling for the quantitative arm of the study’ includes the sample size of the study, or the sample selection, or both. We have tried to clarify certain point of the sample selection, hopefully improving the information regarding sampling. The discussion has been made to include every set objective in the first paragraph. Attachments Attachment Submitted filename: Response to reviewers.docx https://doi.org/10.1371/journal.pone.0286080.r002
2 May 2023 Decision Letter - Sampson Twumasi-Ankrah, Editor PONE-D-22-33707R1Interpretation of continuously measured vital signs data of COVID-19 patients by nurses and physicians at the general ward: a mixed methods studyPLOS ONE Dear Dr. van Goor, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jun 16 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. We look forward to receiving your revised manuscript. Kind regards, Sampson Twumasi-Ankrah, PHD Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: 1) For Table 1, include percentages to the frequencies for all categorical variables. e.g. Gender (Female 14(....%); Male (10(....%) 2) Kindly check and correct "where" in line 208 [Note: HTML markup is below. Please do not edit.] Reviewers' comments: [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0286080.r003
Revision 2
7 May 2023 Author Response Dear dr. Twumasi-Ankrah, Thank you again for revising our manuscript. We are happy to see the previous corrections have been accepted. We have corrected the two additional points made: - We have added percentages to table 1 - We have corrected the word ‘where’ to ‘were’ twice between lines 203-210 Hopefully our work is now up to your publication standards. We hope to hear from you soon, Sincerely, On behalf of the research team, Harriët van Goor https://doi.org/10.1371/journal.pone.0286080.r004
9 May 2023 Decision Letter - Sampson Twumasi-Ankrah, Editor Interpretation of continuously measured vital signs data of COVID-19 patients by nurses and physicians at the general ward: a mixed methods study PONE-D-22-33707R2 Dear Dr. van Goor, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Sampson Twumasi-Ankrah, PHD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0286080.r005
Formally Accepted
16 May 2023 Acceptance Letter - Sampson Twumasi-Ankrah, Editor PONE-D-22-33707R2 Interpretation of continuously measured vital signs data of COVID-19 patients by nurses and physicians at the general ward: a mixed methods study Dear Dr. van Goor: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Sampson Twumasi-Ankrah Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0286080.r006

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .