SIFusion: Lightweight infrared and visible image fusion based on semantic injection

Song Qian; Liwei Yang; Yan Xue; Ping Li

doi:10.1371/journal.pone.0307236

Peer Review History

Original SubmissionJanuary 31, 2024
9 Apr 2024 Decision Letter - Anas Bilal, Editor PONE-D-24-04252SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic InjectionPLOS ONE Dear Dr. Li, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by May 24 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Anas Bilal, Ph.D. Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. Thank you for stating the following financial disclosure: "Xinjiang Uygur Autonomous Region Tianchi Talent Innovation Leading Talent Project" Please state what role the funders took in the study. If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" If this statement is not correct you must amend it as needed. Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf. 4. We note that your Data Availability Statement is currently as follows: All relevant data are within the manuscript and its Supporting Information files. Please confirm at this time whether or not your submission contains all raw data required to replicate the results of your study. Authors must share the “minimal data set” for their submission. PLOS defines the minimal data set to consist of the data required to replicate all study findings reported in the article, as well as related metadata and methods (https://journals.plos.org/plosone/s/data-availability#loc-minimal-data-set-definition). For example, authors should submit the following data: - The values behind the means, standard deviations and other measures reported; - The values used to build graphs; - The points extracted from images for analysis. Authors do not need to submit their entire data set if only a portion of the data was used in the reported study. If your submission does not contain these data, please either upload them as Supporting Information files or deposit them to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories. If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. If data are owned by a third party, please indicate how others may request data access. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: 1. Expand on how SIFusion is novel compared to existing methods, highlighting unique features and improvements. 2. Provide more detailed explanations of the Semantic Injection Module (SIM) and Edge Convolution Module (ECB), including mathematical formulations. 3. Include comparative analysis with existing image fusion methods in terms of performance metrics and computational efficiency. 4. Address the practical applicability of the method, particularly in terms of computational cost and deployment in real-world scenarios. 5. Enhance the experimental section with more diverse datasets to validate the effectiveness of SIFusion in various conditions. 6. Clarify how the method improves the processing of semantic information in high-level visual tasks, with specific examples or case studies. 7. Discuss limitations and potential areas for improvement in the method to provide a balanced view. 8. Include a section on potential future work, exploring how the method can be extended or adapted to other applications. 9. Improve the clarity and structure of the paper, ensuring that technical terms and concepts are well-explained for readers unfamiliar with the topic. 10. Strengthen the conclusion by summarizing key findings and their implications for both human and machine vision applications. Reviewer #2: Given the limitations of existing image fusion algorithms, this paper proposes a lightweight infrared and visible light image fusion network SIFusion based on semantic injection. The method has achieved good results. Here are some suggestions: （1）Why do the authors need to stitch together infrared and visible light images before inputting them into the model instead of extracting features separately? What is the difference between these two approaches? （2）The author extracted fusion features and semantic features from the source image separately, but the extraction module is the same in terms of framework. Please explain how the same module extracts two different features separately. （3）The ablation experiment is insufficient, and it is not enough to simply remove the module for comparison. It should be compared with other typical modules to demonstrate the effectiveness of the proposed module. （4）The author claims in “Contributions” that the ECB module is their first introduction to the field of image fusion, which is incorrect. The ECB module was introduced in 2023 for infrared and visible light image fusion and was publicly published that year. The reference is: CHEN Zhaoyu, FAN Hongbo, MA Meiyan, et al. FECFusion: Infrared and visible image fusion network based on fast edge convolution. Mathematical Biosciences and Engineering: MBE, 2023, 20(9): 16060-16082. （5）The bold values described in Table 7 are not bolded. （6）Because the author describes the network proposed as a lightweight network, but the paper does not compare network parameters and network computational complexity（FLOPs）, it is recommended to add this data comparison. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0307236.r001
Revision 1
26 Apr 2024 Author Response To Senior Editor and Reviewers， Thank you for giving us the opportunity to submit a revised draft of the manuscript "SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic Injection" for publication in the Journal of "PLOS ONE". We appreciate the time and effort that editors and the reviewers dedicated to providing feedback on our manuscript and are grateful for the insightful comments and valuable improvements to our paper. We have incorporated most of the suggestions made by the reviewers. Those changes are highlighted with red color in the manuscript. Please see below, for a point-by-point response to the reviewers' comments and concerns. Reviewer #1： 1.Expand on how SIFusion is novel compared to existing methods, highlighting unique features and improvements. Response: Thank you for your suggestion. We have made revisions to the section on semantic guidance in the related work of this paper and have marked the changes with red color. 2.Provide more detailed explanations of the Semantic Injection Module (SIM) and Edge Convolution Module (ECB), including mathematical formulations. Response: Thank you for your suggestion. We have added the mathematical formulas for the Semantic Injection Module (SIM) and the Edge Convolutional Block (ECB), along with corresponding explanations. 3.Include comparative analysis with existing image fusion methods in terms of performance metrics and computational efficiency. Response: Thank you for your suggestion. We have added comparative analysis with existing image fusion methods in terms of performance metrics and computational efficiency. The data can be viewed in Table 5. It should be specifically noted that only FLFuse, TarDAL, UMF-CMGR, and the method presented in this paper are implemented based on the Pytorch architecture, while the other comparative algorithms are based on the Tensorflow 1.x architecture. The model data for these methods cannot be provided directly. Therefore, the model data is obtained through a model reconstructed based on the Pytorch architecture. 4.Address the practical applicability of the method, particularly in terms of computational cost and deployment in real-world scenarios. Response: Thank you for your suggestion. We have made revisions to address this issue in the content of the efficiency comparison experiment. 5.Enhance the experimental section with more diverse datasets to validate the effectiveness of SIFusion in various conditions. Response: Thank you for your suggestion. In this paper, the training set of the MSRA dataset is not directly used as the training set for the network model. Instead, data augmentation is performed by segmenting the dataset, which to some extent allows the network model to have stronger generalization capabilities. Therefore, we first conduct experimental comparisons using the LLVIP dataset. Subsequently, we also perform generalization tests on the TNO dataset and the LLVIP dataset. These datasets contain a variety of different scenarios, especially day and night scenes. Thus, we believe that the existing experiments in this paper can, to a certain extent, verify the effectiveness of SIFusion under various conditions. 6.Clarify how the method improves the processing of semantic information in high-level visual tasks, with specific examples or case studies. Response: Thank you for your suggestion. Firstly, we illustrate the importance of semantic information for image fusion tasks and the limitations of existing methods in handling semantic information through the examples in Figure 1. Then, we introduce the design of semantic information processing in our method in the Methods section, such as the loss function for the semantic feature branch and the combination of semantic features with fusion features through SIM. Next, we demonstrate the superiority of our method by comparing it with 10 mainstream algorithms. Lastly, we validate the effectiveness of SIM through ablation experiments. In addition, we also confirm the facilitative effect of our method on advanced visual tasks through semantic segmentation experiments. Overall, we analyze the research on semantic information in our method through the above content. 7.Discuss limitations and potential areas for improvement in the method to provide a balanced view. Response: Thank you for your suggestion. We have revised the summary section and have expanded on the content within it. 8.Include a section on potential future work, exploring how the method can be extended or adapted to other applications. Response: Thank you for your suggestion. We have revised the summary section and have expanded on the content within it. 9.Improve the clarity and structure of the paper, ensuring that technical terms and concepts are well-explained for readers unfamiliar with the topic. Response: Thank you for your valuable suggestion. We believe that the current manuscript has already struck a balance between professional depth and accessibility as much as possible. Specialized technical terms are all referenced with citations, allowing readers who are not familiar with the subject to gain a deeper understanding. In order to maintain the professionalism of the paper and to avoid redundancy that could arise from over-simplifying the foundational knowledge, we have decided not to make further modifications to the current version regarding this issue. If you still have difficulties understanding any of the proprietary terms, we will make further revisions to the manuscript. 10.Strengthen the conclusion by summarizing key findings and their implications for both human and machine vision applications. Response: Thank you for your suggestion. We have revised the summary section and have expanded on the content within it. Reviewer #2: 1.Why do the authors need to stitch together infrared and visible light images before inputting them into the model instead of extracting features separately? What is the difference between these two approaches? Response: In the field of infrared and visible image fusion, both single-branch and dual-branch structures are among the mainstream network architectures in existing fusion methods. The dual-branch structure can extract features from two different modalities separately before fusing them, offering better interpretability. In contrast, the single-branch structure obtains shared feature information with similar information domains through the same feature extraction approach. To further achieve a lightweight network model and in conjunction with the characteristics of the semantic injection architecture, the method proposed in this paper selects a single-branch structure with fewer parameters, while the unique feature information is combined with the shared feature information through a semantic injection approach. 2.The author extracted fusion features and semantic features from the source image separately, but the extraction module is the same in terms of framework. Please explain how the same module extracts two different features separately. Response: The two encoders used in this paper for extracting fusion features and semantic features, although they have the same network structure, do not share weight parameters. Therefore, encoders with the same structure but different parameters can learn different parameters through specific loss guidance. The encoder of the semantic branch is guided by the target loss to learn the parameters for extracting semantic features, thereby enabling these two encoders to perform different functions. 3.The ablation experiment is insufficient, and it is not enough to simply remove the module for comparison. It should be compared with other typical modules to demonstrate the effectiveness of the proposed module. Response: Thank you for your suggestion. We have added a comparative experiment between ECB and other typical modules in the ablation experiment section. 4.The author claims in “Contributions” that the ECB module is their first introduction to the field of image fusion, which is incorrect. The ECB module was introduced in 2023 for infrared and visible light image fusion and was publicly published that year. The reference is: CHEN Zhaoyu, FAN Hongbo, MA Meiyan, et al. FECFusion: Infrared and visible image fusion network based on fast edge convolution. Mathematical Biosciences and Engineering: MBE, 2023, 20(9): 16060-16082. Response: Thanks for pointing out the issues in our paper, and we have revised the wording of the manuscript. 5.The bold values described in Table 7 are not bolded. Response: Thank you very much for pointing out the issue with Table 7 in our paper. We apologize for the oversight and promise to correct this error immediately. In the revised version of the paper, we will ensure that all mentioned important data and values are displayed in bold to clearly distinguish and highlight this key information. 6.Because the author describes the network proposed as a lightweight network, but the paper does not compare network parameters and network computational complexity(FLOPs), it is recommended to add this data comparison. Response: Thank you for your suggestion. We have added experimental comparison data between SIFusion and ten comparative algorithms regarding runtime, operational memory space, parameter count, weight size, and FLOPs. The data can be viewed in Table 6. It should be specifically noted that only FLFuse, TarDAL, UMF-CMGR, and the method presented in this paper are implemented based on the Pytorch architecture, while the other comparative algorithms are based on the Tensorflow 1.x architecture. The model data for these methods cannot be provided directly. Therefore, the model data is obtained through a model reconstructed based on the Pytorch architecture. Thank you again for the reviewers' suggestions. The revised content is marked in red within the manuscript, and the file is named "Revised Manuscript with Track Changes.pdf". Attachments Attachment Submitted filename: Response to Reviewers.pdf https://doi.org/10.1371/journal.pone.0307236.r002
22 May 2024 Decision Letter - Anas Bilal, Editor PONE-D-24-04252R1SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic InjectionPLOS ONE Dear Dr. Li, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jul 06 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Anas Bilal, Ph.D. Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed Reviewer #3: (No Response) ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes Reviewer #3: Partly ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes Reviewer #3: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes Reviewer #3: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes Reviewer #3: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: (No Response) Reviewer #3: This paper introduces SIFusion, a lightweight infrared and visible light image fusion based on semantic injection. SIFusion proposes a semantic injection module to integrate semantic information. Experimental results demonstrate its superior performance both qualitatively and quantitatively. However, I still have the following concerns about this paper: 1. In the comparison experiments, the authors compare the proposed SIFusion with other ten competitors on the MSRS and LLVIP datasets, but select six competitors on the TNO dataset. To avoid subjective bias, the authors should add experiments and propose the same comparable methods on the TNO dataset. 2. A major contribution of this work is to design a semantic injection module, thus ensuring that the fused image has rich semantic information. Fig.1 gives a schematic comparison of SeAFusion and SIFusion. Therefore, to comprehensively verify the effectiveness of the method the authors should compare some semantic-aware methods, such as SeAFusion and PSFusion, in the experiments. 3.In the fusion experiment with the MSRS dataset, the fusion performance of GANMcC better than CSF. However, it's performance of object detection is inferior to CSF. How can we explain this inconsistency between fusion performance and downstream task performance? 4. There are some errors in the references. For example, Ref.14 and Ref.37 were published in 2019 and 2022, respectively. Please double-check the whole manuscript. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: No Reviewer #3: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0307236.r003
Revision 2
8 Jun 2024 Author Response To Senior Editor and Reviewers， Thank you for giving us the opportunity to submit a revised draft of the manuscript "SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic Injection" for publication in the Journal of "PLOS ONE". We appreciate the time and effort that editors and the reviewers dedicated to providing feedback on our manuscript and are grateful for the insightful comments and valuable improvements to our paper. We have incorporated most of the suggestions made by the reviewers. Those changes are highlighted with red color in the manuscript. Please see below, for a point-by-point response to the reviewers' comments and concerns. Reviewer \\#3: This paper introduces SIFusion, a lightweight infrared and visible light image fusion based on semantic injection. SIFusion proposes a semantic injection module to integrate semantic information. Experimental results demonstrate its superior performance both qualitatively and quantitatively. However, I still have the following concerns about this paper: 1.In the comparison experiments, the authors compare the proposed SIFusion with other ten competitors on the MSRS and LLVIP datasets, but select six competitors on the TNO dataset. To avoid subjective bias, the authors should add experiments and propose the same comparable methods on the TNO dataset. Response: Due to the TNO dataset being one of the earliest public datasets for infrared and visible light images in the military field, its content is mainly dominated by buildings, with only a small portion of images containing common semantic targets such as pedestrians and vehicles. Therefore, the proposed method did not perform well on this dataset. To better demonstrate the generalization performance of the proposed method, we replaced the TNO dataset with the newer M3FD dataset, which contains multi-scene and multi-target semantic information. We compared SIFusion with twelve comparative algorithms on the MSRS, M3FD, and LLVIP datasets, and SIFusion showed good fusion performance across all of them. 2.A major contribution of this work is to design a semantic injection module, thus ensuring that the fused image has rich semantic information. Fig.1 gives a schematic comparison of SeAFusion and SIFusion. Therefore, to comprehensively verify the effectiveness of the method the authors should compare some semantic-aware methods, such as SeAFusion and PSFusion, in the experiments. Response: We have added comparative experiments related to SeAFusion and PSFusion, and conducted an analysis of the experiments. However, the models of SeAFusion and PSFusion require a larger number of parameters and greater consumption of computational resources. In contrast, the method proposed in this paper, while maintaining a lower computational complexity, can effectively integrate information from the source images, thereby generating fused images that are more in line with the visual effects of human vision. 3.In the fusion experiment with the MSRS dataset, the fusion performance of GANMcC better than CSF. However, it's performance of object detection is inferior to CSF. How can we explain this inconsistency between fusion performance and downstream task performance? Response: Fusion performance metrics are used to evaluate the image quality of the fusion results and do not have a direct connection with the performance of downstream tasks. It is not possible to reflect the performance of downstream tasks through the fusion evaluation metrics. Therefore, we indirectly assess the performance of the semantic expression of the fusion results through the performance of the semantic segmentation model. From the experimental results, although the metric results of the method in this paper are not as good as PSFusion, the model parameters and computational resources required for inference of the method in this paper are far less than those of PSFusion. 4.There are some errors in the references. For example, Ref.14 and Ref.37 were published in 2019 and 2022, respectively. Please double-check the whole manuscript. Response: Thank you for the reviewer's comments; we have re-examined the manuscript and made corrections to the relevant issues. Attachments Attachment Submitted filename: Response to Reviewers.pdf https://doi.org/10.1371/journal.pone.0307236.r004
28 Jun 2024 Decision Letter - Xiaoyong Sun, Editor PONE-D-24-04252R2SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic InjectionPLOS ONE Dear Dr. Li, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 12 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Xiaoyong Sun Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed Reviewer #3: (No Response) ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: (No Response) Reviewer #3: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: (No Response) Reviewer #3: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: (No Response) Reviewer #3: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: (No Response) Reviewer #3: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: (No Response) Reviewer #3: The authors have addressed most of my concerns, but there are still some minor issues that need to be addressed. 1、The response reveals that the proposed method performed inadequately on the TNO dataset. However, concerning the MSRS, M3FD, and LLVIP datasets, the proposed SIFusion still exhibits suboptimal performance in comparison with SeAFusion and PSFusion. Hence, we suggest that the authors incorporate a Discussion section to elucidate this reason, thereby providing readers with more space for reflection. 2、There is a lack of introduction and analysis of some key works in this field. [1] Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation [2] Infrared and visible image fusion via interactive compensatory attention adversarial learning [3]CDDFuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: No Reviewer #3: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0307236.r005
Revision 3
28 Jun 2024 Author Response To Reviewers, Thank you for giving us the opportunity to submit a revised draft of the manuscript "SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic Injection" for publication in the Journal of "PLOS ONE". We appreciate the time and effort that editors and the reviewers dedicated to providing feedback on our manuscript and are grateful for the insightful comments and valuable improvements to our paper. We have incorporated most of the suggestions made by the reviewers. Those changes are highlighted with red color in the manuscript. Please see below, for a point-by-point response to the reviewers' comments and concerns. Reviewer \\#3: The authors have addressed most of my concerns, but there are still some minor issues that need to be addressed. 1. The response reveals that the proposed method performed inadequately on the TNO dataset. However, concerning the MSRS, M3FD, and LLVIP datasets, the proposed SIFusion still exhibits suboptimal performance in comparison with SeAFusion and PSFusion. Hence, we suggest that the authors incorporate a Discussion section to elucidate this reason, thereby providing readers with more space for reflection. Response: Thank you for the reviewer's suggestions. We have added a section titled "Performance Discussion on the TNO Dataset" following the "Application of Semantic Segmentation" subsection to further discuss the shortcomings and reasons of the algorithm presented in this paper when applied to the TNO dataset. 2. There is a lack of introduction and analysis of some key works in this field. [1] Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation. [2] Infrared and visible image fusion via interactive compensatory attention adversarial learning. [3]CDDFuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion. Response: We have added introductions and analyses of these three key works in the section of related work. Attachments Attachment Submitted filename: Response to Reviewers.pdf https://doi.org/10.1371/journal.pone.0307236.r006
2 Jul 2024 Decision Letter - Xiaoyong Sun, Editor SIFusion: Lightweight Infrared and Visible Image Fusion Based on Semantic Injection PONE-D-24-04252R3 Dear Dr. Li, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Xiaoyong Sun Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0307236.r007
Formally Accepted
4 Jul 2024 Acceptance Letter - Xiaoyong Sun, Editor PONE-D-24-04252R3 PLOS ONE Dear Dr. Li, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Xiaoyong Sun Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0307236.r008

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .