A comparison of software for analysis of rare and common short tandem repeat (STR) variation using human genome sequences from clinical and population-based samples

John W. Oketch; Louise V. Wain; Edward J. Hollox

doi:10.1371/journal.pone.0300545

Peer Review History

Original SubmissionJanuary 2, 2024
15 Feb 2024 Decision Letter - Paul Aurelian Gagniuc, Editor PONE-D-24-00164A comparison of software for analysis of rare and common short tandem repeat (STR) variation using human genome sequences from clinical and population-based samplesPLOS ONE Dear Dr. Hollox, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Mar 31 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Paul Aurelian Gagniuc, PhD Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating in your Funding Statement: “JWO is funded by a Wellcome Trust PhD studentship as part of the Wellcome Trust Genetic Epidemiology and Public Health Genomics Doctoral Training Programme by grant number 218505/Z/19/Z . LWV holds a GSK/Asthma+Lung UK Chair in Respiratory Research (C17-1). The research was partially supported by the National Institute for Health Research (NIHR) Leicester Biomedical Research Centre Please provide an amended statement that declares all the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now. Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement. Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf. 3. Thank you for stating the following in the Acknowledgments Section of your manuscript: “JWO is funded by a Wellcome Trust PhD studentship as part of the Wellcome Trust Genetic 515 Epidemiology and Public Health Genomics Doctoral Training Programme by grant number 516 218505/Z/19/Z . LWV holds a GSK/Asthma+Lung UK Chair in Respiratory Research (C17-1). The 517 research was partially supported by the National Institute for Health Research (NIHR) Leicester 518 Biomedical Research Centre; the views expressed are those of the author(s) and not necessarily 519 those of the National Health Service (NHS), the NIHR or the Department of Health.” We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: “JWO is funded by a Wellcome Trust PhD studentship as part of the Wellcome Trust Genetic Epidemiology and Public Health Genomics Doctoral Training Programme by grant number 218505/Z/19/Z . LWV holds a GSK/Asthma+Lung UK Chair in Respiratory Research (C17-1). The research was partially supported by the National Institute for Health Research (NIHR) Leicester Biomedical Research Centre” Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 4. Please update your submission to use the PLOS LaTeX template. The template and more information on our requirements for LaTeX submissions can be found at http://journals.plos.org/plosone/s/latex. 5. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. 6. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This study is important as it identifies an approach to investigate the role of genomic STR variants in polygenic human diseases. It is also a study that genotypes common STRs with the software used and identifies rarer STR expansions genome-wide. Therefore, it can be published in the journal. Tables must be written in the appropriate format. There is duplicate text content in the article writing, it was uploaded twice. It should be reviewed as a writing language. Reviewer #2: The authors present a benchmarking of open-source software for the analysis of repeat expansion from short-read genomic data. The comparison is made considering: computational resources, genotyping accuracy (expansion length and variant calling). Four cases of application at genome wide detection, mendelian inheritance (allele differentiation), rare disease (known pathogenic expansions) and in forensic analysis (variant calling) are described. The experiments performed and the well detailed results are very useful for other scientists working with repeat expansion detection and encourage the use of good practices. The research topic fits the current scenario, as genomic sequencing is becoming cheaper and the availability of genomic data for analysis in different research fields (large population studies, clinical diagnostics, etc.) is increasing. Comparative evaluation of tools for detection of repeat expansions has not been widely published in the past. The comparative evaluation of such variants is of great importance to apply correct bioinformatics approaches in routine laboratory work, due to their genomic variability at the sequence level and their importance in diseases. The abstract and introduction are well presented addressing the current state of the art of the capabilities of detection tools. The experiments used are adequate and cover all the performance-based questions that the authors aim to address. The experimental design is well conceived control samples are grouped according to the questions. The design of the study considers differences between software that could imply restrictions in the comparison of the results. The number of samples used is sufficient to obtain meaningful results and the reference materials chosen (coriell and 1000genomes) are adequate. For the comparison in each category, performance parameters were well chosen. The tables and figures are understandable and help other scientists to consider which software might be best applied to specific scientific questions. The study conforms to ethical standards. The data are freely available in repositories (web links work) and the methods are available for replication of the results by any other user. Data collection and interpretation is well done. The presentation of the results is well structured and explained. It supports the conclusions and discussion. Supplementary tables provide all the data produced in the study at sample and variant level and provide sufficient evidence to support the benchmarking goals. The discussion is conducted in the context of the results presented and at the current or research level, limitations are discussed. The statistical analysis and parameters chosen for benchmarking conform to current guidelines for quantitative and qualitative variant detection. Figures and tables are well presented and support the results. The use cases in Mendelian disorders and forensic medicine focus on current issues in STR detection (i.e. allele-specific genotyping and call accuracy, respectively). The clinical samples FMR1, HTT and FXN are good examples, as they have proven to be tricky in accurate detection. The authors point out the limitations of the study, such as the number of samples was not significant enough (e.g. FMR1) and the type of variation they do not include (e.g. 1 bp). Overall, this study does not present a groundbreaking advance in the field but presents a comparative assessment for a type of genomic variant that is difficult to determine and for many years overlooked in population and clinical studies. Such benchmarking studies are urgently needed to improve best practice in selecting strategies for genomic analysis, especially in clinical settings. Importantly, the authors also addressed computational resources, an issue that is often underestimated and results in unexpectedly high costs. Comments for the authors: The authors state in the Abstract: “Their contribution to common disease is not well-understood, but recent software tools designed to genotype STRs using short read sequencing data are beginning to address this.” Referring to repeats. Software help to address questions related to repeat contribution to common disease, but the software itself does not address contribution. The sentence is misleading and should be changed to improve the meaning within the context of the abstract. Authors should not refer to Tables (Table 1 is mentioned twice) or detailed results in the introduction. Please delete. Authors should more clearly specify in the introduction that the manuscript focuses exclusively on short read sequencing data (and not long read sequencing). Figure 3 resolution is pixelized in the review document. Please make that figures have the required resolution for publication. Authors assume as a gold standard capillary electrophoresis for the genotyping of known forensic STR loci to benchmark error rates. The accuracy of this technique should be stated (sensitivity specificity) and discussed in case this might have an influence in the cases where calls do not match with the software. Authors show underestimation of repeat length in clinical cases with very large expansions. Please briefly discuss which are the limitations in the software or short read techniques that cause these underestimations. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0300545.r001
Revision 1
26 Feb 2024 Author Response Response to reviewers Reviewer #1: This study is important as it identifies an approach to investigate the role of genomic STR variants in polygenic human diseases. It is also a study that genotypes common STRs with the software used and identifies rarer STR expansions genome-wide. Therefore, it can be published in the journal. Tables must be written in the appropriate format. There is duplicate text content in the article writing, it was uploaded twice. It should be reviewed as a writing language. Thanks to the reviewer for their positive comments. Indeed, the entire article was presented twice in the package to reviewers – this was a mistake. We will follow the publisher’s guidance on table formatting. Reviewer #2: The authors present a benchmarking of open-source software for the analysis of repeat expansion from short-read genomic data. The comparison is made considering: computational resources, genotyping accuracy (expansion length and variant calling). Four cases of application at genome wide detection, mendelian inheritance (allele differentiation), rare disease (known pathogenic expansions) and in forensic analysis (variant calling) are described. The experiments performed and the well detailed results are very useful for other scientists working with repeat expansion detection and encourage the use of good practices. The research topic fits the current scenario, as genomic sequencing is becoming cheaper and the availability of genomic data for analysis in different research fields (large population studies, clinical diagnostics, etc.) is increasing. Comparative evaluation of tools for detection of repeat expansions has not been widely published in the past. The comparative evaluation of such variants is of great importance to apply correct bioinformatics approaches in routine laboratory work, due to their genomic variability at the sequence level and their importance in diseases. The abstract and introduction are well presented addressing the current state of the art of the capabilities of detection tools. The experiments used are adequate and cover all the performance-based questions that the authors aim to address. The experimental design is well conceived control samples are grouped according to the questions. The design of the study considers differences between software that could imply restrictions in the comparison of the results. The number of samples used is sufficient to obtain meaningful results and the reference materials chosen (coriell and 1000genomes) are adequate. For the comparison in each category, performance parameters were well chosen. The tables and figures are understandable and help other scientists to consider which software might be best applied to specific scientific questions. The study conforms to ethical standards. The data are freely available in repositories (web links work) and the methods are available for replication of the results by any other user. Data collection and interpretation is well done. The presentation of the results is well structured and explained. It supports the conclusions and discussion. Supplementary tables provide all the data produced in the study at sample and variant level and provide sufficient evidence to support the benchmarking goals. The discussion is conducted in the context of the results presented and at the current or research level, limitations are discussed. The statistical analysis and parameters chosen for benchmarking conform to current guidelines for quantitative and qualitative variant detection. Figures and tables are well presented and support the results. The use cases in Mendelian disorders and forensic medicine focus on current issues in STR detection (i.e. allele-specific genotyping and call accuracy, respectively). The clinical samples FMR1, HTT and FXN are good examples, as they have proven to be tricky in accurate detection. The authors point out the limitations of the study, such as the number of samples was not significant enough (e.g. FMR1) and the type of variation they do not include (e.g. 1 bp). Overall, this study does not present a groundbreaking advance in the field but presents a comparative assessment for a type of genomic variant that is difficult to determine and for many years overlooked in population and clinical studies. Such benchmarking studies are urgently needed to improve best practice in selecting strategies for genomic analysis, especially in clinical settings. Importantly, the authors also addressed computational resources, an issue that is often underestimated and results in unexpectedly high costs. We thank the reviewer for their careful review and positive comments. Changes are highlighted under “track changes” in the manuscript document. Comments for the authors: The authors state in the Abstract: “Their contribution to common disease is not well-understood, but recent software tools designed to genotype STRs using short read sequencing data are beginning to address this.” Referring to repeats. Software help to address questions related to repeat contribution to common disease, but the software itself does not address contribution. The sentence is misleading and should be changed to improve the meaning within the context of the abstract. We have changed “Their contribution to common disease is not well-understood, but recent software tools designed to genotype STRs using short read sequencing data are beginning to address this.” to “Their contribution to common disease is not well-understood, but recent software tools designed to genotype STRs using short read sequencing data will help address this.” Authors should not refer to Tables (Table 1 is mentioned twice) or detailed results in the introduction. Please delete. We have deleted references to table 1 in the introduction, and have cut out two sentences that discuss the results in detail (lines 124-128). Authors should more clearly specify in the introduction that the manuscript focuses exclusively on short read sequencing data (and not long read sequencing). We agree, and have made two clarifications in the introduction : line 111, line 118, and one in the abstract (line 35), and twice in the methods (lines 165, 205). Figure 3 resolution is pixelized in the review document. Please make that figures have the required resolution for publication. Resolution of figures will be publication quality, following the publisher’s requirements. Authors assume as a gold standard capillary electrophoresis for the genotyping of known forensic STR loci to benchmark error rates. The accuracy of this technique should be stated (sensitivity specificity) and discussed in case this might have an influence in the cases where calls do not match with the software. STR calling using capillary electrophoresis will, of course, have an error rate. Sensitivity and specificity will be very dependent on sample origin, quality and assay. However, these data had been generated using a forensic genotyping kit (Promega’s Powerplex Fusion) which will be very robust, and optimised for use on poor-quality forensic samples. Therefore we believe that, when used on laboratory quality samples, we are justified in using these genotypes as an error-free gold standard. We have emphasised this in the manuscript by including “forensic-standard” (line 392). Authors show underestimation of repeat length in clinical cases with very large expansions. Please briefly discuss which are the limitations in the software or short read techniques that cause these underestimations. We echo Dolzhenko et al 2017, in suggesting that GC-bias of Illumina sequencing (leading to a lower coverage than expected of GC-rich repeats), or somatic mosaicism may lead to this effect. We now have mentioned this (lines 409-410). Attachments Attachment Submitted filename: Response to reviewers.docx https://doi.org/10.1371/journal.pone.0300545.r002
29 Feb 2024 Decision Letter - Paul Aurelian Gagniuc, Editor A comparison of software for analysis of rare and common short tandem repeat (STR) variation using human genome sequences from clinical and population-based samples PONE-D-24-00164R1 Dear Dr. Hollox, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Paul Aurelian Gagniuc, PhD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0300545.r003
Formally Accepted
20 Mar 2024 Acceptance Letter - Paul Aurelian Gagniuc, Editor PONE-D-24-00164R1 PLOS ONE Dear Dr. Hollox, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Paul Aurelian Gagniuc Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0300545.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .