PONE-D-22-11894CVR-MRICloud: an online processing tool for CO2-inhalation and resting-state cerebrovascular reactivity (CVR) MRI dataPLOS ONE

Dear Dr. Liu,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 05 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yen-Yu Ian Shih, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Peiying,

My apology for the delay in making a decision. As you can see, both reviewers are enthusiastic about this work and suggest the manuscript to be suitable for publication pending a minor revision. Please feel free to let me know if there’s anything I can help clarify further.

Best,

Ian

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this manuscript Liu et al report their development and publication of a cloud-based tool for processing of CVR MRI data. The authors outline their rationale of expanding availability of CVR processing, both for a hypercapnic based and resting-state MRI paradigms, the implemented methodology and provide some validation in a healthy lifespan cohort of 203 datasets, three patient datasets and a healthy young subject. Liu et al report example outputs from the processing, results of the alignment and quantitative metrics amongst others. I am grateful to the editor and authors for the opportunity to comment on this interesting and useful work.

Major

I agree with the authors that there is considerable scope for broadening availability of CVR for clinical studies. However, there may be a risk of such a resource becoming a ‘black box’ for some prospective users, potentially leading to false inferences, and/or missing key details e.g. the reported ‘CO2 Switching’ phenomena. Further the authors make no mention of any evaluations of the software/user friendliness of the software by individuals with no experience of CVR which from the introduction appear to be the target audience.

The authors describe the benefits of cloud computing, which are significant, but not the challenges. Notably patient confidentiality, data protection and secure transfer/deletion are key. I believe that these should be considered in the Discussion, further detail on the technical setup and upload/download procedures of the server may also be appropriate for the Methods e.g. are uploads deleted after a set time? Do they only exist as long as a console is open?, How are results downloaded?, Who has access to/how secure is uploaded data? etc?

The authors present the toolbox as a way of measuring CO2- and RS-CVR – however no comparison between the two is given despite data being available in the same population. As a validation of the toolbox this would seem a useful validation i.e. to show that both approaches show similar associations, and I would encourage the authors to add such an analysis, along with more detailed discussion of the results in the Discussion.

Minor

The data availability statement is contradicted by the description of where data is to be found which indicates data will only be released case-by-case upon request after funding agency approval. Clearly there can be valid funding or ethics/consent restrictions which may limit how much data can be shared. However, if there is an exceptional reason it is not currently presented which should be corrected before publication.

Line 56 – The authors reference the Mild Stroke Study 3 (Clancy et al), however as a single centre study it does not seem to fit with their phrasing. The authors perhaps meant to refer to Investigate-SVDs 10.1016/j.cccb.2021.100020 which was multi-site and led by the same group?

Lines 68-69 – The authors draw an analogy between task/resting state fMRI and CO2-CVR/RS-CVR, however no citation is provided, I would suggest the authors either remove this sentence or add relevant references.

Lines 70-71 – The authors state most CVR studies have used in-house processing methods but do not supply a relevant reference, one such example would be 10.3389/fphys.2021.643468. The authors may also want to note that despite many CVR studies depending on in-house code few authors have published their code, creating challenges for reproducible and repeatable research.

Line 138 and 147 – What was the reason for choosing a window size of 10? And combining the results based on the higher of Algorithm I and II?

Line 169 – Delays of 0 to EtCO2 time course duration are allowed, and in line 175 a narrower search range of +/- 5 s is used. Are the authors confident this can handle the full range as pathology? Longer/altered delays have been reported in patient e.g. WMH in SVD patients etc.

Line 227 –Given the temporal filtering frequency in ref. 33 seem to come from healthy patients or moyamoya how transferable is the range used e.g. for stroke or MS?

Line 288-9 – Commenting on consistency of values with previous work may be better suited to the discussion than results.

Line 302-3 – Comments on reliability and reproducibility seem to be based on a single healthy subject, evaluations in other groups with larger sample size would be useful to determine how consistent this pattern is.

Line 337-9 – If no published data is available then it may be valuable to other groups to include some in the supplementary information to support this point.

Lines 207-211, Line 363 – Add reference. In general there were several areas with unsupported statements which may benefit from further additions, not least as it makes it more accessible for a wider readership.

Line 367 – internal reference models have significant limitations cf SUV models for PET, the drawbacks should be considered in my opinion. As although superficially the maps may look similar the lack of directly comparable values is a significant impediment.

Line 383 – Is there any error protection against the CO2 switching phenomena i.e. does MRICloud label it/throw an error?

Figure 9 – It would be useful to have the spread of the CVR as points instead of only the mean and CI.

Reviewer #2: In this work, the authors developed and describe a cloud-based cerebrovascular reactivity (CVR) processing pipeline. The authors comprehensively describe all of the processing steps, including extraction of end-tidal CO2 curves and its alignment with BOLD data, the calculation of a variety of CVR metrics from CO2-challenge and resting-state CVR data and a quality control metric. The authors also evaluate the performance of the processing pipeline from a population comprising a variety of ages, appropriate distribution of sex and MRI data from a wide range of imaging parameters. The manuscript is very clearly written and the work is valuable as it has the potential to broaden the uptake of the CVR technique by removing the data processing skill entry barrier, facilitates reproducible research and meticulously describes the logic behind each choice of processing step. Some minor questions and comments are listed below:

1) As the BOLD signal comprises a mixture of a variety of different hemodynamic and metabolic parameters, could the choice of MR imaging parameters affect the alignment of BOLD data to EtCO2, or the quality of CO2 and resting-state CVR maps (e.g. short TR, high flip angle increasing the sensitivity to in-flow effect)?

2) Could the authors clarify whether patient health information is automatically removed from image metadata upon upload to the cloud server or if the user is responsible for removing patient health information?

3) Could the authors describe what MRI image file formats are compatible with CVR-MRICloud?

4) If available, could the authors include a citation to the Smooth-Cross algorithm used for peak detection (algorithm 2, page 7, line 137) for the interested reader?

5) On page 18, line 370 the authors state that the CVR-MRICloud lacks the flexibility of allowing user inputs. Could the authors clarify whether this extends to the ability of the user to upload brain masks and the ability to modify the FWHM of the spatial-smoothing Gaussian kernel?

6) Do the authors plan to update the software, such as addition or modification of processing steps, as the CVR-mapping technique further develops? If so, will older versions still be available to users to ensure continuity of in-progress studies?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

CVR-MRICloud: an online processing tool for CO2-inhalation and resting-state cerebrovascular reactivity (CVR) MRI data

PONE-D-22-11894R1

Dear Peiying,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Yen-Yu Ian Shih, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewer 1 left a note. Please kindly take a look. 

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I am grateful to the authors for their response which has addressed all the main substantive points.

The one caveat I would raise is in relation to point 2. While the authors have clearly considered the issues for their own legislative domain other regions have different standards, I think in particular of the General Data Protection Regulations (GPDR) standards which apply in most European countries. While previously data transfer to the USA was possible under the Privacy Shield this is unfortunately no longer the case: https://gdpr-info.eu/issues/third-countries. My (limited) understanding is at a minimum transfer of even anonymised data to a web server in the USA for processing would ideally be included in the participant consent form and ethics, and many institutions would have their own processes to approve transfer, which may include considering who else could hypothetically access the data e.g. server admins, law enforcement etc and on what basis.

Clearly any detailed discussion of these points is beyond the scope of this paper, and it would be impossible to account for local rules in all jurisdictions. However, the authors may wish to note somewhere that researchers should consider whether they need to include cloud processing in the USA in their consent process and ensure compliance with local data protection regulations before uploading data.

I congratulate the authors again on a valuable piece of work which I am sure will be extremely useful to the community going forward.

Reviewer #2: All questions and comments from the original review have been answered very well. Happy to recommend the manuscript for publication in its current form.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********