PONE-D-24-54905

Convolutional neural network isotope identifier created with hybrid simulated and experimental gamma spectra

PLOS ONE

Dear Dr. Lee-Brewin,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

In this manuscript, Luke et al. present their research on the "Convolutional Neural Network Isotope Identifier Created with Hybrid Simulated and Experimental Gamma Spectra." This research project is both timely and crucial for safety and security considerations, particularly as it advances real-time isotope identification in urban environments. Such capability is essential for providing critical information regarding the nature of potential threats. The authors encounter difficulties associated with isotope identification, given the extensive range of possible isotopes, activities, and shielding factors in uncontrolled urban settings.

The research has generated significant insights, warranting publication in PLOS ONE. The following points support this recommendation:

- First, the research is both original and relevant. The findings hold substantial importance for researchers and regulators within the nuclear security sector. The study could attract considerable interest if the dataset, primarily composed of background radiation with sporadically distributed medical alarms, were expanded beyond the isotopes 18F and 99mTc.

- Second, the research has been conducted with a commendable level of scientific rigor.

Nonetheless, the absence of isotopes such as 40K, 232Th, 226Ra, or U in the background radiation dataset raises questions about the environmental radiation context of the study area and the applied research protocols. It would be beneficial for the authors to address this concern.

Overall, the linearity and clarity of the exposition merit review. The title appears misaligned with the primary objectives articulated throughout the manuscript, including the abstract and the introduction. Additionally, the quality of the article could be enhanced by elucidating the rationale behind the selection of the radioisotopes considered in this research, as well as examining the quality control and quality assurance measures of the presented data.

==============================

Please submit your revised manuscript by Feb 13 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Cebastien Joel Guembou Shouop, Ph.D., ME, MS

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1.Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. Please note that funding information should not appear in any section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript.

4. Thank you for stating the following financial disclosure:

“Initials of Authors: LLB, CST

Partially funded by Nusec (Nuclear Security Science Network)

URL: https://www.nusec.uk/nnsa

LLB: PDRA award 80%, remainder University of Surrey

Data collected as part of the SIGMA data challenged. Formally released to selected academic partners October 2022”

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

5. We note that you have indicated that there are restrictions to data sharing for this study. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

Before we proceed with your manuscript, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of recommended repositories, please see

https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible.

We will update your Data Availability statement on your behalf to reflect the information you provide.

6. Please ensure that you refer to Figure 6 in your text as, if accepted, production will need this reference to link the reader to the figure.

7. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

8. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

In this manuscript, Luke et al. present their research on the "Convolutional Neural Network Isotope Identifier Created with Hybrid Simulated and Experimental Gamma Spectra." This research project is both timely and crucial for safety and security considerations, particularly as it advances real-time isotope identification in urban environments. Such capability is essential for providing critical information regarding the nature of potential threats. The authors encounter difficulties associated with isotope identification, given the extensive range of possible isotopes, activities, and shielding factors in uncontrolled urban settings.

The research has generated significant insights, warranting publication in PLOS ONE. The following points support this recommendation:

- First, the research is both original and relevant. The findings hold substantial importance for researchers and regulators within the nuclear security sector. The study could attract considerable interest if the dataset, primarily composed of background radiation with sporadically distributed medical alarms, were expanded beyond the isotopes 18F and 99mTc.

- Second, the research has been conducted with a commendable level of scientific rigor.

Nonetheless, the absence of isotopes such as 40K, 232Th, 226Ra, or U in the background radiation dataset raises questions about the environmental radiation context of the study area and the applied research protocols. It would be beneficial for the authors to address this concern.

Overall, the linearity and clarity of the exposition merit review. The title appears misaligned with the primary objectives articulated throughout the manuscript, including the abstract and the introduction. Additionally, the quality of the article could be enhanced by elucidating the rationale behind the selection of the radioisotopes considered in this research, as well as examining the quality control and quality assurance measures of the presented data.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Convolutional neural network isotope identifier created with hybrid simulated and experimental gamma spectra

Here are some constructive comments and questions for the authors:

• The title is not appropriate at all, it should be revised!

• There is no comparison is clear between experimental and simulated results in the Abstract

• The abstract is not appropriate at all, it should be revised and rewritten (Background, methodology, results, conclusion)

• The methodology for combining simulated and experimental gamma spectra is intriguing but needs clearer explanation. For instance, how were discrepancies between simulated and experimental data handled during training?

• While the SIGMA dataset is mentioned, additional details about its composition, diversity of isotopes, and representativeness of real-world scenarios would strengthen the study.

• The study would benefit from benchmarking the CNN against traditional machine learning or other state-of-the-art deep learning models for isotope identification.

• Were any pre-processing techniques applied to the gamma spectra before feeding them into the CNN?

• I recommend to cite the next article that more appropriate for your work (Measurement 168, 108456, 2021)

• What steps were taken to ensure that the CNN generalizes well to unseen datasets?

• The presentation of the work including heading titles should be well revised

These comments should be revised before final decision taken

Reviewer #2: Reviewer’s comments report

First of all, I would like to thank the editor of PLOS ONE for giving this opportunity to revise the manuscript of a entitle of “Convolutional neural network isotope identifier created with hybrid simulated and experimental gamma spectra.”, please kindly find the comments.

General comment:

This manuscript proposes an innovative approach to the identification of radioactive isotopes in complex urban environments, combining real and simulated spectra in order to overcome the lack of representative training data. The database, comprising 750 million spectra collected in London, ensures the requisite realism and applicability. The incorporation of simulated spectra enables the effective management of the inherent variability associated with uncontrolled environments. The convolutional neural network achieves 96% accuracy for medical isotopes such as 18F and 99mTc, thereby demonstrating the effectiveness of the method. The approach is adaptable to non-medical alarms and has strong operational potential for security purposes, offering a high-performance, scalable solution for real-time radiological detection. The paper is well written and all the target points are clear and understood. It may be accepted, after minor revision.

Line 94: “This subset is will contain a higher ratio of alarms to.” The grammatical issue should be correct for more clarity.

Reviewer #3: I understand that the primary purpose of this manuscript is to show a proof of concept for developing an isotope identification system that is trained primarily using simulated data that has been modified to resemble realistic scenarios–such as temperature-dependent gain change, etc.–and real-world background data. The authors then tested the applicability of this network using a subset of the SIGMA dataset. I believe this manuscript contains useful information and should be published, but I have a few concerns and suggestions:

(1) On page 8, the authors state “... more than 5 standard deviations above the average number of counts for a given day”.

What is the method of calculating the average number of counts for a given day? For example, is it a rolling average that continues until the alarm occurs, or is it a total average across the entire day? If it is the latter, how may this method “be effectively deployed to identify isotopes in real time”?

(2) Regarding the Geant4 simulation: Are more details available regarding how the simulation was performed? E.g., was the distance between the source and the detector always stationary? Were there any other obstructions other than the aluminum shielding? Were there any possibilities of gamma ray scattering within the environment?

(3) Would it be possible for the authors to comment on how the manual identification-related errors in the k-means clustering procedure could potentially influence the reported accuracies of the identification network?

(4) It may be beneficial to provide insights on a more impactful method other than the 5-sigma threshold technique for identifying alarms: For example, an unsupervised autoencoder architecture (such as ARAD: https://doi.org/10.1016/j.engappai.2022.104761, among others). I would recommend providing brief information on alternatives and whether or not they would be applicable in this case.

(5) The first sentence of page two states: “ In 2018 the Atomic Weapons Establishment (AWE) created using a subset of the SIGMA dataset.”

I presume this should read “... created a dataset using a subset …”; but, in any case, I believe this sentence should be revised.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Mohamed El Tokhy

Reviewer #2: No

Reviewer #3: Yes: Randall W. Gladen

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Validation of simulated training sets using a convolutional neural network for isotope identification in urban environments

PONE-D-24-54905R1

Dear Dr. Lee-Brewin,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Cebastien Joel Guembou Shouop, Ph.D., ME, MS

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

As previously mentioned, the quality of the article could be enhanced by elucidating the rationale behind the selection of the radioisotopes considered in this research. But this is left to the author to decide whether to include such details or not.

Reviewers' comments:

Reviewer #1: All comments have been addressed

Reviewer #2: Accept

Reviewer #3: All comments have been addressed