Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network

Fahad Khalil Peracha; Muhammad Irfan Khattak; Nema Salem; Nasir Saleem

doi:10.1371/journal.pone.0285629

Peer Review History

Original SubmissionNovember 28, 2022
26 Dec 2022 Decision Letter - Abdullah M. Mutawa, Editor PONE-D-22-32395Causal Speech Enhancement using Dynamical-Weighted Loss and Attention Encoder-Decoder Recurrent Neural NetworkPLOS ONE Dear Dr. Saleem, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Feb 09 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Abdullah M. Mutawa, Ph.D Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. New software must comply with the Open Source Definition. 3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. 4. Please upload a new copy of Figure 1 as the detail is not clear. Please follow the link for more information: " ext-link-type="uri" xlink:type="simple">https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/" https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/ 5. Please include a separate caption for each figure in your manuscript. Additional Editor Comments: Dear Author, Please answer all points mentioned by the reviewers and prepare a point-by-point table with reply to every point showing the location of change in the manuscript. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: No Reviewer #2: N/A ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: No ******** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Causal Speech Enhancement using Dynamical-Weighted Loss and Attention EncoderDecoder Recurrent Neural Network" is a good topic for paper but having some suggestions 1) Compare results with some current developed methods and use those references in your introduction. 2) use one more different input data for analysis and evaluation. 3) use proper references Reviewer #2: Summary: In this work, a causal data-driven model is proposed for single-microphone SE operating in real-time systems. The proposed system utilizes attention encoder-decoder long short-term memory (LSTM) to estimate the time-frequency mask from noisy speech. The manuscript is interesting; however, the following comment should be addressed : Abstract : - - - - - - - - - - - 1 – Please include problem statement . 2 - Improvement ratio between the proposed and existing works should be included . Introduction Section : - - - - - - - - - - - - - - - - - - - - - - 3 – In the Introduction, the authors need to refer to other speech enhancement algorithms such as : i) doi: 10.1088/1757-899X/1090/1/012102, ii) doi: 10.3390/s21217025, and ii) doi: 10.1109/ACCESS.2019.2929864. 4 – The contribution should be included as a list for better readability. Proposed Speech Enhancement Section : - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 5 – Please check the numbering of the subsections . 6 – Define the functions used such as “\|⋅\|” , “\|\|⋅\|\|”, “” , etc . Experiments Section : - - - - - - - - - - - - - - - - - - - - - - 7 – The authors utilize STFT; however, there are different types of transforms which are based on orthogonal polynomials. The authors need to refer to the difference between the Fourier Transform and the following transforms Krawtchouk transform (doi: 10.3390/e23091162), Hahn Transform (doi: 10.1109/ACCESS.2022.3170893), and Meixner transform (doi: 10.1007/s11554-021-01093-z). This will help the researchers to utilize other transforms for SE. Conclusion Section : - - - - - - - - - - - - - - - - - - - - - - 8 – This section is fine. No comments . General Comments: - - - - - - - - - - - - - - - - - 9 - There are some grammatical error should be checked and corrected . - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ******* 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. quillbot-extension-portal/quillbot-extension-portal https://doi.org/10.1371/journal.pone.0285629.r001
Revision 1
23 Jan 2023 Author Response Reviewer #1: Causal Speech Enhancement using Dynamical-Weighted Loss and Attention Encoder Decoder Recurrent Neural Network" is a good topic for paper but having some suggestions 1- Compare results with some current developed methods and use those references in your introduction. Response: Thank you for the important suggestion, the results are compared to the recently developed methods and the reference methods are reflected in the introduction part of the revised manuscript. The Tables are revised to address the suggestion of respected reviewer. Table captions are highlighted with Red to show the changes. 2- Use one more different input data for analysis and evaluation. Response: Thank you for the important suggestion. This algorithm was intended for the additive noisy backgrounds; however, to address the concern of the respected reviewer, the authors have included noisy reverberation as other input data. Action: The following text and results are added to the revised paper. This section examined the dereverberation performance of the proposed SE. To train the SE model, three reverberation times (0.4 sec, 0.6 sec, and 0.8 sec) are considered. A total of 100 anechoic speech utterances from the IEEE dataset [40] are used to create the training dataset. The testing dataset contains 40 reverberant speech utterances. There is no overlapping between the speech utterances used during model training, and testing. The proposed method with reverberant speech utterances are compared and examined for the dereverberation. The results are compared with study of Wu and Wang [49], where estimated inverse filters and spectral subtraction are used to reduce the reverberation. Table 11 shows the results using STOI and PESQ. The proposed method delivered the best STOI and PESQ scores, i.e., STOI≥78.3%, and PESQ≥2.45 at RT≥4 sec. The spectrograms are provided in Fig. 8 where the smearing energy produced by reverberation is considerably reduced, showing that the reverberation performance of the proposed method. 3- Use proper references. Response: Thank you for the suggestion, the references is arranged in the proper manner to address the concern Reviewer#2: The manuscript is interesting; however, the following comment should be addressed 1 – Please include problem statement. Response: Thank you for the important suggestion, the problem statement is included in the revised manuscript. Action: Speech enhancement (SE) reduces background noise signals in target speech and is applied at the front end in various real-world applications, including robust ASRs and real-time processing in mobile phone communications. SE systems are commonly integrated into mobile phones to increase quality and intelligibility. As a result, a low-latency system is required to operate in real-world applications. On the other hand, these systems need efficient optimization. This research focuses on the single-microphone SE operating in real-time systems with better optimization. 2 - Improvement ratio between the proposed and existing works should be included. Response: Thank you for the suggestion, the improvement ratios between the proposed and related studies are included in the revised manuscript. 3 – In the Introduction, the authors need to refer to other speech enhancement algorithms such as : i) doi: 10.1088/1757-899X/1090/1/012102, ii) doi: 10.3390/s21217025, and ii) doi: 10.1109/ACCESS.2019.2929864. Response: The introduction part is modified with the references suggested. Thank you 4 – The contribution should be included as a list for better readability. Response: The contributions are listed in the revised manuscript to address the concern of the reviewer. Thank you 5 – Please check the numbering of the subsections. Response: Thank you for the correction, the sections and subsections are corrected in the revised manuscript. 6 – Define the functions used such as “\|⋅\|” , “\|\|⋅\|\|”, “” , etc . Response: Thank you for the important correction, the typos in equations is corrected and these notations are defined in the revised manuscript. 7 – The authors utilize STFT; however, there are different types of transforms which are based on orthogonal polynomials. The authors need to refer to the difference between the Fourier Transform and the following transforms Krawtchouk transform (doi: 10.3390/e23091162), Hahn Transform (doi: 10.1109/ACCESS.2022.3170893), and Meixner transform (doi: 10.1007/s11554-021-01093-z). This will help the researchers to utilize other transforms for SE. Response: Thank you for the important suggestion, the authors have used STFT transform widely used in the speech signal processing. The other mentioned transforms are associated to the different applications and the authors will conduct a separate study based on these transforms. The mentioned transforms are added with the references to the revised manuscript. Attachments Attachment Submitted filename: Response to Reviewers.pdf* https://doi.org/10.1371/journal.pone.0285629.r002
27 Apr 2023 Decision Letter - Abdullah M. Mutawa, Editor Causal Speech Enhancement using Dynamical-Weighted Loss and Attention Encoder-Decoder Recurrent Neural Network PONE-D-22-32395R1 Dear Dr. Saleem, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Abdullah M. Mutawa, Ph.D Academic Editor PLOS ONE Reviewers' comments: Reviewer's Responses to Questions Comments to the Author Reviewer #2: All comments have been addressed Reviewer #3: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes Reviewer #3: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes Reviewer #3: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes Reviewer #3: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes Reviewer #3: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: Summary: In this work, a causal data-driven model is proposed for single-microphone SE operating in real-time systems. The proposed system utilizes attention encoder-decoder long short-term memory (LSTM) to estimate the time-frequency mask from noisy speech. The authors have addressed the raised comments. No further comments. Comments: Abstract : - - - - - - - - - - - 1 – The abstract is fine. No further comments. Introduction Section : - - - - - - - - - - - - - - - - - - - - - - 2 – This section is fine. No further comments. Proposed Speech Enhancement Section : - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 3 – This section is fine. No further comments. Experiments Section : - - - - - - - - - - - - - - - - - - - - - - 4 – This section is fine. No further comments. Conclusion Section : - - - - - - - - - - - - - - - - - - - - - - 5 – This section is fine. No further comments. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Reviewer #3: I noticed that the revised version corrects all recommendations form the reviewers, I accept the paper in this form. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: No Reviewer #3: No ******** https://doi.org/10.1371/journal.pone.0285629.r003
Formally Accepted
2 May 2023 Acceptance Letter - Abdullah M. Mutawa, Editor PONE-D-22-32395R1 Causal Speech Enhancement using Dynamical-Weighted Loss and Attention Encoder-Decoder Recurrent Neural Network Dear Dr. Saleem: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Abdullah M. Mutawa Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0285629.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .