PONE-D-24-48304Recognition of flight cadets brain functional magnetic resonance imaging data based on machine learning analysisPLOS ONE

Dear Dr. Yan,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. In particular, the reviewers raised concerns about the control group, and requested additional details in the methods and rationale for the study. 

Please submit your revised manuscript by Mar 22 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Dzung Pham

Academic Editor

PLOS ONE

Journal requirements:   When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf. 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.  When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section. 4. We note that you have indicated that there are restrictions to data sharing for this study. For studies involving human research participant data or other sensitive data, we encourage authors to share de-identified or anonymized data. However, when data cannot be publicly shared for ethical reasons, we allow authors to make their data sets available upon request. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.  Before we proceed with your manuscript, please address the following prompts: a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible. Please update your Data Availability statement in the submission form accordingly. 5. In the online submission form, you indicated that [The data from this study are available upon request. However, the Civil Aviation Administration of China prohibits the public sharing of pilots’ physiological data due to legal constraints. Researchers interested in accessing these data may contact:yandongfeng@cafuc.edu.cn.]. All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information.This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. 

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Overall, this is a very interesting manuscript that I would suggest is published with minor edits. The manuscript is well written with clear explanation and logic. I suggest minor edits for consideration below.

Materials and Methods

Please provide an explanation on the exposure of the control group. Was this group only involved in ground school training without any flight experience? Are these trainees in another carrier field? This will help the reader understand what is meant by ‘trainees from the ground program.’ I see that 0 hours of flight training was recorded in Table 2.

Table 2 – I am not clear what 39/0 and 37/0 mean in the Sex category. I would suggest reporting the n instead of the %, or put the % within parentheses.

On line 271, I might suggest adding “where feature is defined as…” to add clarity for the reader.

Line 295 – I would suggest changing the word ‘hue’ to ‘color.’

I would suggest Figure 7 and 8 (with explanation) be within the results section with interpretation in the discussion section.

The explanation of the BOLD signal was informative for the rest of the manuscript. I might recommend the authors provide a similar concise explanation of how the reader should conceptualize FC, ALFF, and ReHo as there is no explanations of what these functions are.

Lines 552 – 557:

Other limitations – only trainee pilots in one country with limited flying experience. I would suggest commenting on gender and handedness (see comment about Table 2). Additionally, each ML model had <100% sensitivity and specificity. I would suggest adding a line discussing how there are factors not identified in these models, which suggest they are incomplete.

I am not clear how any conclusions can be drawn from this project about how this ML model could be used to aid pilot selection. This study did not aim to identify who did/did not pass flight training. Instead, this study aimed to identify whether ML models can identify whether someone did or did not complete a certain degree of flight training. I would suggest removing this line. A line could be added about how these data could inform future working aiming to predict pilot performance in flight training/selection.

Line 569, I would recommend changing the word ‘indicating’ to ‘suggesting’

Line 571 – I would suggest another term besides ‘regular’ trainees

Line 572-576 – again, I am not sure how any conclusions can be drawn about ‘pilot screening and training optimization.’ I would suggest removing this line. This study demonstrated that a ML model can identify those who did and did not complete flight training and provides insight on neural connectivity. It would be reasonable suggest a line how future research could employ these data to support those objectives.

A line will need to be added stating whether the authors will provide their primary data upon request

Reviewer #2: The manuscript submitted by Le et al. examines the neural profile of flight cadets and how it differs from ground program cadets. All participants underwent a resting state fMRI scan which was used by the authors to extract four different metrics of brain activity (BOLD signal, functional connectivity, amplitude of low frequency fluctuation, and regional homogeneity). The authors developed five different classification models using machine learning to determine the best model(s) to analyze the resting state fMRI data. Overall, the BOLD signal was the best metric to classify flight cadets while the logistic regression and Gaussian Naïve Bayes models exhibited higher classification accuracy relative to other model types. Moreover, specific brain regions in the default mode, frontoparietal, somatomotor, and visual networks possess different connectivity patterns in flight cadets. In general, the manuscript examines an interesting topic that may guide future studies in personnel selection for certain occupations. However, there are a few points that the authors should address in the manuscript before acceptance can be considered:

Major:

1. Introduction (page 4, lines 67-68; page 5, lines 98-99): There are many terms that the authors mention but do not define or describe. Terms such as SN, DMN, CEN, ALFF, ReHo are all technical terminology that brain imaging researchers would understand, but for the average reader, which would likely be the case for a PLoS One audience, these terms should be defined. Some of these terms, such as DMN, are not defined until the Discussion (page 25, lines 434-439) when it would be more appropriate to define them in the Introduction instead. The aforementioned terms only require 1-2 sentences each to describe in the Introduction while a more detailed description of the relevant terms, such as DMN, can be elaborated on in the Discussion.

2. Introduction (page 5, lines 95-97): The study rationale seems incomplete. The authors do not elaborate why it is important to study brain function in healthy individuals. For example, an explanation of what constitutes flight training and how it is unique from other occupations (e.g., ground cadets) would be helpful. Similarly, the authors mention drug/alcohol use (lines 106-109) but it is unclear how this even relates to the study question.

3. Introduction (page 6, lines 110 – 119): This seems like an exploratory study, which should be made clear in the introduction. Furthermore, what were the authors’ hypotheses for the study questions/objectives?

4. Methods (page 8, 163-165): The authors removed the signal from cerebrospinal fluid, white matter, and head motion. It is common procedure for the resting state fMRI data to also remove physiological signals, such as heart beat and respiration. Could the authors explain why this was not collected and/or regressed out and how this would impact their resting state signal metrics?

5. Methods (Feature Extraction, page 8): The authors only explained how they calculated BOLD signal and FC. Where is the description for how ALFF and ReHo were computed?

6. Methods (Feature extraction, pages 8-10): Have the authors considered using graph theory as another resting state fMRI metric to be compared in the study? Graph theory allows resting state fMRI data to be analyzed with more granularity which is particularly relevant in this study when various metrics are being compared head-to-head [see Medaglia (2017) Neuroimaging Clin N Am, 27(4):593-607; Lv et al., (2018) AJNR Am J Neuroradiol, 39(8):1390-1399]. Given that the authors already have 116 brain regions outlined in Table 1 that they used in calculating the BOLD signal and FC, they can perform graph theory on these same regions to extract values such as clustering, path length, or efficiency.

7. Methods (page 9, lines 183-186) and Results (page 15, Table 3): It appears that the resting state fMRI metrics with the most features that have been extracted (29,000 for BOLD signal and 6,670 for FC) and selected (40 for BOLD signal and 141 for RC) are ultimately the features that will perform the best in the ML algorithms. While I recognize the authors applied more stringent p-value thresholds for the BOLD and RC metrics, the ALFF and ReHo metrics appear to already be at a disadvantage from the beginning as fewer features were extracted to be used in the ML model comparison and evaluation. In other words, how would the results differ if all 4 resting state fMRI metrics used the same number of features (e.g., only the top 10 features for each metric) for ML model comparison and evaluation?

8. Methods (pages 12-13, lines 221-223): The choice of the 5 classification models (GNB, SVM, RF, LR, and XGBoost) appears arbitrary. Please explain the rationale for these 5 model types over other machine learning models. Would the authors obtain similar results if they used k-nearest neighbor (KNN) or principal component analysis (PCA), as an example?

9. Discussion (pages 24-25, lines 416-425): The authors discuss the results from the perspective of neural changes due to cadet training. While this is a plausible explanation, this seems a bit misleading for this study since the authors did not compare flight cadets before and after a training course nor did they compare separate groups of flight cadets (experienced vs. novice). Instead, the authors compared a group of flight cadets versus ground cadets, the latter being a different occupation altogether. One cannot rule out the possibility that the neural profile for flight cadets is already different from ground cadets at the outset and may (or may not) be related to performance/success in flight cadet training, which of course requires further research to investigate.

Minor:

1. Introduction, page 5, line 106: Please add the word “functional” in front of magnetic resonance imaging.

2. Methods, page 7 (line 146) and page 8 (line 152): Please explicitly state the voxel size for the T1 and resting state fMRI scans. Are these 4 × 4 × 4 mm voxels?

3. Results, page 14, Table 2: Please clarify the number of participants in the flight cadet group. Table 2 says N = 26, but on page 14, line 243 it indicates that N = 39.

4. Discussion, page 26-27, lines 458-462: Both sentences seem redundant, please consider rewording or removing one of these sentences.

5. Please check for minor spelling typos throughout the manuscript (e.g., page 5, line 98 reads as: “… resting-state brain function, including Amplitude of ALFF, functional connectivity …” but either the words “Amplitude of” is not required or the letters “FF” should be spelled out).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Review Comments.docx

Recognition of flight cadets brain functional magnetic resonance imaging data based on machine learning analysis

PONE-D-24-48304R1

Dear Dr. Yan,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dzung Pham

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The authors have done an admirable job in addressing my comments. The manuscript is much improved with a complete study rationale, definition of terms at the first instance, and clear hypothesis, all of which help the reader in understanding the logic behind the study. I also appreciate that the authors conducted a supplementary analysis to further validate that BOLD and FC indicators perform the best in ML algorithms. I have no additional concerns and recommend this manuscript for publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #2: No

**********