Are clinically unimportant findings qualified as benign in lumbar spine imaging reports? A content analysis of plain X-ray, CT and MRI reports

Caitlin Farmer; Romi Haas; Jason Wallis; Denise O’Connor; Rachelle Buchbinder

doi:10.1371/journal.pone.0297911

Peer Review History

Original SubmissionMay 16, 2023
4 Sep 2023 Decision Letter - Aloysius Gonzaga Mubuuke, Editor PONE-D-23-14641Are clinically unimportant findings qualified as benign in lumbar spine imaging reports? A content analysis of plain X-ray, CT and MRI reportsPLOS ONE Dear Dr. Farmer, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Oct 19 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Aloysius Gonzaga Mubuuke Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information. If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information. 3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. In your revised cover letter, please address the following prompts: a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. We will update your Data Availability statement on your behalf to reflect the information you provide. 4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: The paper addresses an important area trying to show how radiologists make their decisions using different imaging modalities in lumbar images. In addition to the comments from reviewers, the authors should strengthen the discussion by clearly explaining the implication of the findings to clinical practice in patients presenting with lumbar issues for imaging. In addition, proof-read the entire paper to refine the language. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for the opportunity to review this article. The research question is if radiologists label normal findings in imaging (X-Ray, CT, and MRI) as such in their report. For this, 200 of each modality from patients in which the imaging was indicated due to lower back pain, were acquired. The reports analyzed with the question, if the normality of likely clinically unimportant findings was reported. The main finding is that only a minority of “most likely clinically unimportant” findings were not labeled as such, which, as they argue, could lead clinicians and patients to overestimate the severity of findings and lead to potentially unwarranted escalation of therapies. The introduction clearly introduces the topic and explains the clinical relevancy in an encompassing manner. The methods are clearly described. The reporting of the is generally very detailed and transparent of the findings, although some aspects require additional explanation. In table 1 an extra row should be added, regarding how many percent of the clinically unimportant findings were on average reported as benign. Although this is similar to “Number of reports that qualified all reported clinically unimportant findings as benign, N(%)”, it does not allow conclusions about average percent of findings that were reported as benign. In table 2 a brief explanation should be added, that cases of potentially clinically significant findings were excluded. Although it is described in the methods, the table adds ambiguity which gradings of the reported features were included. For example, just mentioning Foraminal stenosis, might lead readers to the conclusion that all grades of foraminal stenosis were evaluated as likely clinically unimportant. The mentioning of “Lumbar spine alignment” and “Lordosis” or” Scoliosis” leads to confusion, what exactly is meant with the term “Lumbar spine alignment”. The discussion could be improved by discussing potential reasons for the percentage differences in labeling of features as benign. For example, normal lumbar spinal alignment and Pedicles/pars defect were report as benign 100% of the time, while no occurrence “Spondylolisthesis” was reported as benign. From an orthopedics surgeons’ point of view, the assertion that changes like Modic Typ3, Nerve root irritation, Facet joint arthropathie, or Spondylolisthesis are benign or clinically unimportant is difficult to grasp. These might be not significantly associated with low back pain on a population’s levels, as shown in some studies, but for individual patients, they can be indicative of underlying pathology that may require intervention (conservative or otherwise). In clinical practice, a patient’s history, physical examination findings, and imaging studies must be considered together to make informed decisions on management. Furthermore, some of these changes, might be associated with mechanical instability or degenerative processes that could progress over time. As such, dismissing them as benign or clinically unimportant without a thorough evaluation and considering the patient's unique circumstances could potentially lead to missed opportunities for early intervention and management, which in turn could impact the quality of life and functional outcomes for the patient. Given that radiologist often have limited clinical data about the patient, it seems reasonable, that radiologist would not reported these findings explicitly as benign (or normal, etc.). The discussion should aim to address these concerns or better specify the population of patients this study is aimed to cover. An additional limitation of this study that should be reported is the missing of clinical data. Overall the manuscript is well written and answers that set out research question. After the comments are adressed, I can recommend this manuscript for publication. Reviewer #2: Comments Abstract line 35; qualifying that they are benign 103-104 and this study was conducted over the 104 following year 114 To ensure anonymity of the reporting radiologist, - add coma 125; serious finding is vague; quantify or substantiate on the meaning of vague. 143-146; why did physiotherapists and rheumatologists independently determined whether finding was likely clinically unimportant or important? Why not the physicians such as orthopedic surgeons or referring clinicians? Was this appropriate? 199: The 95 were excluded; Reasons for exclusion? 212; use age range instead of median age 217; use mean/average instead of median. 264; The most common finding across imaging modalities was fracture, while for CT and MRI it was foraminal and canal stenosis and nerve root impingement; So was it fracture for MRI and CT too? 297; This study found lumbar spine X-Ray, CT and MRI reports include a large number of findings.; write; This study found that; also large number of findings is vague; do you mean “irrelevant” findings or? 383; Avoid the word- serious 408: Provided; not provide. Overall: Try to summarise This study found lumbar spine X-Ray, CT and MRI reports include a large number of findings. the discussion; it is too long. Check all grammatical errors and correct them before submitting. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Rita Nassanga ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0297911.r001
Revision 1
12 Dec 2023 Author Response Thank you for your constructive and considered feedback, which we have incorporated into our manuscript. We have reviewed grammar and edited for clarity, and significantly reduced the discussion length. We have also refined formatting and labelling in accordance with PlosOne policies. Please find our full response below and in the attached 'Response to reviewers' document. We thank the reviewers for their insightful comments on our manuscript, and answer their queries below. Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ________________________________________ 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ________________________________________ 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ________________________________________ 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ________________________________________ 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for the opportunity to review this article. The research question is if radiologists label normal findings in imaging (X-Ray, CT, and MRI) as such in their report. For this, 200 of each modality from patients in which the imaging was indicated due to lower back pain, were acquired. The reports analyzed with the question, if the normality of likely clinically unimportant findings was reported. The main finding is that only a minority of “most likely clinically unimportant” findings were not labeled as such, which, as they argue, could lead clinicians and patients to overestimate the severity of findings and lead to potentially unwarranted escalation of therapies. The introduction clearly introduces the topic and explains the clinical relevancy in an encompassing manner. The methods are clearly described. The reporting of the is generally very detailed and transparent of the findings, although some aspects require additional explanation. In table 1 an extra row should be added, regarding how many percent of the clinically unimportant findings were on average reported as benign. Although this is similar to “Number of reports that qualified all reported clinically unimportant findings as benign, N(%)”, it does not allow conclusions about average percent of findings that were reported as benign. We have added a row in Table 1, “Mean percent of clinically unimportant findings described as benign per report” as requested. We also reordered one row on the table. In table 2 a brief explanation should be added, that cases of potentially clinically significant findings were excluded. Although it is described in the methods, the table adds ambiguity which gradings of the reported features were included. For example, just mentioning Foraminal stenosis, might lead readers to the conclusion that all grades of foraminal stenosis were evaluated as likely clinically unimportant. The mentioning of “Lumbar spine alignment” and “Lordosis” or” Scoliosis” leads to confusion, what exactly is meant with the term “Lumbar spine alignment”. Thank you for this feedback. We have added descriptors to the relevant terms (for example, ‘Scoliosis (mild)’ to make this clearer to readers, and added a note regarding alignment terms, which were reproduced as they were written in the report. The discussion could be improved by discussing potential reasons for the percentage differences in labeling of features as benign. For example, normal lumbar spinal alignment and Pedicles/pars defect were report as benign 100% of the time, while no occurrence “Spondylolisthesis” was reported as benign. We have revised the discussion and added a paragraph regarding the potential reasons for the range in labelling of findings as benign as suggested (lines 346-350 in track changes manuscript). From an orthopedics surgeons’ point of view, the assertion that changes like Modic Typ3, Nerve root irritation, Facet joint arthropathie, or Spondylolisthesis are benign or clinically unimportant is difficult to grasp. These might be not significantly associated with low back pain on a population’s levels, as shown in some studies, but for individual patients, they can be indicative of underlying pathology that may require intervention (conservative or otherwise). In clinical practice, a patient’s history, physical examination findings, and imaging studies must be considered together to make informed decisions on management. Furthermore, some of these changes, might be associated with mechanical instability or degenerative processes that could progress over time. As such, dismissing them as benign or clinically unimportant without a thorough evaluation and considering the patient's unique circumstances could potentially lead to missed opportunities for early intervention and management, which in turn could impact the quality of life and functional outcomes for the patient. Given that radiologist often have limited clinical data about the patient, it seems reasonable, that radiologist would not reported these findings explicitly as benign (or normal, etc.). The discussion should aim to address these concerns or better specify the population of patients this study is aimed to cover. We agree that it is important to comprehensively assess an individual patient and we would expect that the GP/primary care practitioner would do so. We have made it clearer that in primary care most of these findings would be considered benign. Our judgements about relevance of individual findings were based upon the available published evidence that these findings are usually benign – as we indicated, both cross-sectional and longitudinal population-based studies have found that the changes we included as benign do not appear to be associated with either current or future back pain. This is outlined in the methods (lines 143-158) and discussion (lines 375-381). In the following paragraph in the discussion (lines 384-389) we also indicated that we were intentionally conservative in our judgements. As well we provided these data in full (Supplementary Table 1) so that the reader can also make their own judgments about our assessments. We agree that our judgements were based upon population-based data, which is the highest form of evidence for looking at these types of associations and together with clinical judgement can be therefore extrapolated to individual patients in most instances. We agree that in a specialist setting, the specialist can review the films and make their own interpretation for individual patients. An additional limitation of this study that should be reported is the missing of clinical data. We have added a sentence clarifying this (lines 392-394). Overall the manuscript is well written and answers that set out research question. After the comments are adressed, I can recommend this manuscript for publication. Thank you for these helpful comments. Reviewer #2: Comments Abstract line 35; qualifying that they are benign Corrected, thank you. 103-104 and this study was conducted over the following year Added as requested. 114 To ensure anonymity of the reporting radiologist, - add coma Added as requested. 125; serious finding is vague; quantify or substantiate on the meaning of vague. We have expanded the examples of serious findings. 143-146; why did physiotherapists and rheumatologists independently determined whether finding was likely clinically unimportant or important? Why not the physicians such as orthopedic surgeons or referring clinicians? Was this appropriate? Findings were determined as likely clinically unimportant or important based on the available evidence (Supplementary table 1). The research authors were physiotherapists and a rheumatologist. Their background was included as context about who was performing the study and the judgements. All are expert clinicians with expertise in back pain so we consider this appropriate. 199: The 95 were excluded; Reasons for exclusion? We have clarified in the text that the reasons for exclusion are provided in Figure 1 (top box, right side of figure). 212; use age range instead of median age We prefer to report both median and range for age. 217; use mean/average instead of median. We prefer to report median rather than mean as the data were not normally distributed. 264; The most common finding across imaging modalities was fracture, while for CT and MRI it was foraminal and canal stenosis and nerve root impingement; So was it fracture for MRI and CT too? We have clarified that the most common finding for x-ray was fracture, for CT it was canal stenosis, and for MRI it was foraminal stenosis. 297; This study found lumbar spine X-Ray, CT and MRI reports include a large number of findings.; write; This study found that; We have amended this as suggested. also large number of findings is vague; do you mean “irrelevant” findings or? We have clarified this sentence to “This study found that lumbar spine X-Ray, CT and MRI reports describe a large number of imaging findings overall.” 383; Avoid the word- serious We prefer to retain this word as it is based upon its use in clinical guidelines and our prior cited papers. 408: Provided; not provide. Amended as suggested. Overall: Try to summarise This study found lumbar spine X-Ray, CT and MRI reports include a large number of findings. the discussion; it is too long. Check all grammatical errors and correct them before submitting. We have shortened the discussion and checked and corrected grammatical errors. https://doi.org/10.1371/journal.pone.0297911.r002
16 Jan 2024 Decision Letter - Aloysius Gonzaga Mubuuke, Editor Are clinically unimportant findings qualified as benign in lumbar spine imaging reports? A content analysis of plain X-ray, CT and MRI reports PONE-D-23-14641R1 Dear Dr. Farmer, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Aloysius Gonzaga Mubuuke Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0297911.r003
Formally Accepted
4 Mar 2024 Acceptance Letter - Aloysius Gonzaga Mubuuke, Editor PONE-D-23-14641R1 PLOS ONE Dear Dr. Farmer, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Aloysius Gonzaga Mubuuke Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0297911.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .