Assessment of complementary health approaches use in pediatric oncology: Modification and preliminary validation of the "Which Health Approaches and Treatments Are You Using?" (WHAT) questionnaires

Mohammad R. Alqudimat; Karine Toupin April; Lindsay Jibb; Charles Victor; Paul C. Nathan; Jennifer Stinson

doi:10.1371/journal.pone.0294393

Peer Review History

Original SubmissionNovember 12, 2022
3 Jan 2023 Decision Letter - Erik Loeffen, Editor PONE-D-22-31203How to assess and discuss complementary health approaches use in pediatric oncology: Adaptation and preliminary validation of the "Which Health Approaches and Treatments Are you Using?" (WHAT) questionnairesPLOS ONE Dear Dr. Alqudimat, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Feb 17 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Erik Loeffen, M.D., Ph.D. Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf. 2. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: No Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: A validated tool for the assessment of complementary health approaches by patients and their family is much needed. The project involved included international providers through the SIOP working group which is commended. Title: The title is too long, suggestion below Assessment of complementary health approaches in pediatric oncology: Modification of the Which Health Approaches and Treatment Are you Using (WHAT) questionnaire Keywords: Too many Abstract: The abstract should draw in the interest of the reader; this abstract is too wordy. First sentence: Complementary health approaches (CHA) are commonly used by pediatric patients with cancer; however, few health care providers (HCP) inquire regarding the use of CHA for symptom alleviation. Reframe from using we in the method and results of the abstract and in these sections of the manuscript The manuscript is too long, needs editing and sentences are run on Design Suggest you delete: This article describes In adapting and validating the WHAT questionnaire, a Delphi survey was conducted. This survey was distributed electronically to pediatric patients with cancer, parents, pediatric oncology HCPs, pediatric complementary care HCPs, and researchers. Phase 1 allowed consensus and adaption of the questionnaire domains and items for use in pediatric oncology. Phase 2 explored the face and content validity of the questionnaire, assessing relevance, comprehensiveness, and comprehension in the context of pediatric oncology. Line 120: states semi-structured interview… using cognitive interviews. This sentence is not clear Line 122: Interviews focused instead of were focused Line 132 Participants Section A convenience sample Line 134-5 Patient eligibility included children diagnosed with cancer ≥ 3 months, English-speaking, 8-18 years of age, with a history of CHA during cancer therapy. Line 139-143 describing HCP recruitment is not clear and too wordy. It is unclear who comprises “researchers” Paragraph Procedure Line 161. “We” used 5 times in one paragraph and starts 3 of the sentences On the following page, “we” overused and distracting. Line 173: Participants completed a demographic survey and were then asked to suggest modifications in the questionnaire domains and items, as well as recommendations for domains and items that might be added. The data analysis should not include pronouns, just describe how the analysis was conducted The result section is too long, and the authors have inferred opinions and comments that should not be included in a result section For example, Line 277; These questions would be best for research purposes Page 13 the first paragraph is too wordy and hard to follow Line 256-258 needs editing for clarification Line 261: The median rating by HCPs was 9 for all domains and approximately (the end of the sentence is not clear) they rated half the items at a 9? Line 270: Only three participants suggested new domains… These suggestions included…. Line 295: All participants endorsed understanding the content of the question, expect two 10 year old children. Line 335: results should not describe that redcap was used for the project Line 435 speaks of two children 8 and 9, but previously they were 10. Are these different patients? The results in many sections appear repetitive. Last paragraph page 25 Reviewer #2: Thank you for the opportunity to review this paper. In general, the manuscript is well-written and organized. The topic is important, and the questionnaire is a valuable tool to facilitate discussions among pediatric oncology patients, their parents, and healthcare providers. I have some minor comments: Pg 8. Line 146. The authors state they are targeting a response rate of 17%; however, they don’t provide the population size (e.g., the number of members of the organization or children at the Hospital from sick children). What was the denominator? Did they reach their 17% response rate target? Pg11, line 219. The author states that MA and BW did the content analysis, but BW is not on the list of authors. Pg 13. Lines 247-252 These percentages add to 99%; who was the other 1%? (The numbers add up, but the percentages do not) Pg6, line 106 the sentence states: “Both the child- and parent proxy-report versions of the original WHAT questionnaires include four domains: child CAH use.” I believe the authors meant to write CHA; please double-check. In table 2, under the communication domain, check spelling; now it reads; “communication about CAH” I think it is CHA. Discussion: The authors interviewed HCP from different regions of the world, but they don’t discuss how this tool can be used or the implications it would have in high vs. low-income countries with a difference in survival rates. Also, if HCP from different countries were interviewed, why were only Canadian patients included in the study? Wouldn’t cultural differences affect the understanding and interpretations of the questions? This might be something worth addressing in the discussion. What about immigrant populations in Canada? Perhaps this is a limitation of the development of the questionnaires. I think this is important to address because, in your study, you argue that it is important to adapt and tailor questionnaires to enrich HCP knowledge about CHA, and you might be missing the input of an often-underserved population that HCP has little knowledge about. Why were no complementary HCPs included in phase 2: testing face and content validity of the WHAT questionnaires? Could this be another limitation of the study? ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0294393.r001
Revision 1
8 Feb 2023 Author Response The revised version of our manuscript has been formatted using PLOS ONE's style requirements. We have also added complete REB information, including the full name of the REB, the REB protocol numbers, and more details describing the obtained informed written and implied consent. The details in the manuscript have been alighted to the ethical statement. We have carefully reviewed the feedback and revised the manuscript accordingly. Please see below for an itemized response to the reviewers’ comments. We hope the paper is now in an acceptable form for publication in PLOS ONE. All page and line numbers included in our responses below refer to the unmarked version of our manuscript without tracked changes (i.e., labeled “manuscript”) Reviewers' comments: Reviewer #1: A validated tool for the assessment of complementary health approaches by patients and their family is much needed. The project involved included international providers through the SIOP working group which is commended. Title: The title is too long, suggestion below Assessment of complementary health approaches in pediatric oncology: Modification of the Which Health Approaches and Treatment Are you Using (WHAT) questionnaire Response: We appreciate your suggestion, and we considered changing the title accordingly. We condensed the first part of the title using your suggestion. For the second part, we replaced the word "Adaptation" with "Modification", but we kept the phrase "preliminary validation" since this paper is about both modifying and testing the validity of the WHAT questionnaires. The new title is "Assessment of complementary health approaches use in pediatric oncology: Modification and preliminary validation of the "Which Health Approaches and Treatments Are you Using?" (WHAT) questionnaires". Keywords: Too many Response: We reduced the number of keywords and kept the essential ones (i.e., six keywords). Abstract: The abstract should draw in the interest of the reader; this abstract is too wordy. First sentence: Complementary health approaches (CHA) are commonly used by pediatric patients with cancer; however, few health care providers (HCP) inquire regarding the use of CHA for symptom alleviation. Response: We followed the PLOS ONE submission guidelines and included 300 words in the original abstract. Indeed, we considered your comment and suggestion. We have revised the first sentence according to your suggestion. Also, the other parts of the abstract have been revised and condensed. The revised abstract now includes 289 words (see pages 2 and 3). Reframe from using we in the method and results of the abstract and in these sections of the manuscript Response: Thank you for the comment. We used the active voice "as much as possible to create direct, clear, and concise sentences" (Section 4.13 of the APA publication manual 7th edition). However, we agree that we overused the active voice in our first submission. Thus, the abstract and the manuscript have been revised in line with your comment. The manuscript is too long, needs editing and sentences are run on Response: To address this comment, we have substantially shortened our manuscript by 51 lines. This was the net reduction after condensing the method, streamlining the results, and adding further details to the limitation sections (see below for more information). We also improved the presentation of Table 2 (i.e., results of the Delphi survey) by re-grouping the findings. So, the domain ratings have been moved to be presented first, followed by item ratings. Design: Suggest you delete: "This article describes" In adapting and validating the WHAT questionnaire, a Delphi survey was conducted. This survey was distributed electronically to pediatric patients with cancer, parents, pediatric oncology HCPs, pediatric complementary care HCPs, and researchers. Phase 1 allowed consensus and adaption of the questionnaire domains and items for use in pediatric oncology. Phase 2 explored the face and content validity of the questionnaire, assessing relevance, comprehensiveness, and comprehension in the context of pediatric oncology. Response: As per the reviewer's suggestions, we have revised this paragraph accordingly (see lines 109-114). Line 120: states semi-structured interview… using cognitive interviews. This sentence is not clear Response: To improve clarity, this sentence has been revised, and the phrase "using cognitive interviews" has been replaced with the phrase "using think-aloud and probing techniques" (lines 117-118) to clarify the two sub-types of cognitive interviewing methods that have been used in Phase 2. Line 122: Interviews focused instead of were focused Response: As suggested, this sentence has been modified accordingly (line 118). Line 132 Participants Section A convenience sample Response: We have revised this statement (line 131). Line 134-5: Patient eligibility included children diagnosed with cancer ≥ 3 months, English-speaking, 8-18 years of age, with a history of CHA during cancer therapy. Response: These lines have been revised following your suggestion (lines 132-135). We kept one part of the original sentence: "[…] having used at least one type of CHA since cancer diagnosis according to self- or proxy-report." to clarify how we identified the CHA users since this would not be a documented piece of information in the medical record. Line 139-143 describing HCP recruitment is not clear and too wordy. It is unclear who comprises "researchers" Response: This part includes long names of two groups (i.e., SIOP and PedCAM) that were mentioned for the first time, making it wordy. However, those lines have been revised to enhance clarity (lines 137 to 142). The word "researchers" has been clarified by adding "CHA" to be "CHA researchers" (line 139). Paragraph Procedure Line 161. "We" used 5 times in one paragraph and starts 3 of the sentences On the following page, "we" overused and distracting. Response: We have modified the writing style for this procedure paragraph and for the following page. The text has been edited according to your comment (see pages 8 and 9), and sentences started with "We" have been minimized significantly throughout the manuscript. Line 173: Participants completed a demographic survey and were then asked to suggest modifications in the questionnaire domains and items, as well as recommendations for domains and items that might be added. Response: This sentence has been modified according to your suggestion with a slight change to show the correct order of our procedure steps. Here is the edited sentence (lines 167 – 169): "participants were asked to suggest modifications in the domains and items, as well as recommendations for domains and items that might be added and were then asked to complete a demographic survey." The data analysis should not include pronouns, just describe how the analysis was conducted Response: We revised the data analysis section accordingly. Pronouns have been removed and we shortened this section (see lines 170-172 and 201-219). The result section is too long, and the authors have inferred opinions and comments that should not be included in a result section For example, Line 277; These questions would be best for research purposes Response: We share your concern that the results section in the first submission was too long. The results section has been shortened significantly. Like previous studies on instrument validation and following COSMIN guidelines, we used qualitative data as evidence to guide our decisions about keeping, deleting, or modifying the WHAT items. For the Delphi study, quantitative data was described in the results section without including our opinion. For the short answers by participants that included suggestions for adding some domains, we analyzed these short responses and used them as evidence to decide whether to adjust the WHAT content accordingly. The analysis process for cognitive interviews included transcribing interviews, documenting field notes, summarizing participants' interpretations/understanding of items, identifying face and content validity concerns, and then deciding about each item (see data analysis paragraph on page 10). Also, we aimed to weigh the input from different data sources (i.e., triangulation) and make the decision for each item – whether to retain, omit, or revise. Thus, decision-making about items is considered the final step of our analysis, leading to the final results (i.e., the modified WHAT versions). Therefore, we included how we made the decision to modify the original WHAT in the results section. Page 13 the first paragraph is too wordy and hard to follow Response: This paragraph has been edited to clarify the numbers and the proportions (see lines 234 – 238). Also, the CHA definition has been revised to improve the reading flow. Line 256-258 needs editing for clarification Line 261: The median rating by HCPs was 9 for all domains and approximately (the end of the sentence is not clear) they rated half the items at a 9? Response: We appreciate your suggestion. This section has been revised. All findings in the text that overlapped with information found in Table 2 have been removed (see lines 239 to 247). We also reorganized Table 2 to make it easier to follow. First, the ratings of the domains were presented, followed by the ratings of each item. Line 270: Only three participants suggested new domains… These suggestions included…. Response: We appreciate your suggestion. This line has been edited accordingly (lines 239 and 242). Line 295: All participants endorsed understanding the content of the question, expect two 10 year old children. Response: Thank you for the suggestion. This line has been edited, and extra words have been removed (lines 262-263). Line 335: results should not describe that redcap was used for the project Response: Agreed, we revised and moved this part about REDCap from the section about the results of the first cycle of Phase 2 to the "Resulting modified WHAT questionnaires" section (page 24). We wanted to highlight that switching from paper-based to electronic versions was done using a research tool (i.e. REDCap). Once the modified WHAT is ready to be used in clinical practice, it will ideally be integrated into an electronic medical record platform to be part of the clinical assessment. Line 435 speaks of two children 8 and 9, but previously they were 10. Are these different patients? Response: Our response to this question is yes; these are different participants. The two 10 years old children participated in cycle 1 of Phase 2 (see line 262 under cycle 1 findings). In cycle 2, we had a new sample of participants (lines 363-364), of which two children 8 and 9 years old participated (line 377). The results in many sections appear repetitive. Last paragraph page 25 Response: The results sections have been revised and shortened substantially. Reviewer #2: Thank you for the opportunity to review this paper. In general, the manuscript is well-written and organized. The topic is important, and the questionnaire is a valuable tool to facilitate discussions among pediatric oncology patients, their parents, and healthcare providers. I have some minor comments. Pg 8. Line 146. The authors state they are targeting a response rate of 17%; however, they don't provide the population size (e.g., the number of members of the organization or children at the Hospital from sick children). What was the denominator? Did they reach their 17% response rate target? Response: We appreciate your suggestion. The response rate for each invited group has been added (lines 224-226): "The Delphi survey was completed by 61 participants with response rates per group as 68% (15/22) of children, 59% (13/22) of parents, 24% (10/41) of PedCAM pediatric HCPs and complementary HCPs and researchers, and 30% (23/77) of SIOP oncology HCPs." Pg11, line 219. The author states that MA and BW did the content analysis, but BW is not on the list of authors. Response: Thank you for the comment. Yes, Brittany Wiles (BW) is not one of the authors. We meant to add the acknowledgement section in the first submission. Thus, the acknowledgment section has been added to the revised version (page 31) and we acknowledged BW efforts in the content analysis. Pg 13. Lines 247-252 These percentages add to 99%; who was the other 1%? (The numbers add up, but the percentages do not) Response: We agree that this 1% has been missed by rounding up the percentages to zero decimal places. Thus, we re-rounded the numbers to one decimal place. Now, these percentages add to 100% (i.e., 80.3%, 16.4%, 3.3%); see lines 234-238. Pg6, line 106 the sentence states: "Both the child- and parent proxy-report versions of the original WHAT questionnaires include four domains: child CAH use." I believe the authors meant to write CHA; please double-check. In table 2, under the communication domain, check spelling; now it reads; "communication about CAH" I think it is CHA. Response: Yes, we meant to say "CHA". These typos have been corrected. Discussion: The authors interviewed HCP from different regions of the world, but they don't discuss how this tool can be used or the implications it would have in high vs. low-income countries with a difference in survival rates. What about immigrant populations in Canada? Perhaps this is a limitation of the development of the questionnaires. I think this is important to address because, in your study, you argue that it is important to adapt and tailor questionnaires to enrich HCP knowledge about CHA, and you might be missing the input of an often-underserved population that HCP has little knowledge about. Response: We agree that testing the WHAT in low-middle-income countries and by a sample of pediatric oncology immigrants in Canada is a limitation of our study and requires future research. This limitation has been added to the limitation section (see lines 488 - 494). Also, if HCP from different countries were interviewed, why were only Canadian patients included in the study? Wouldn't cultural differences affect the understanding and interpretations of the questions? This might be something worth addressing in the discussion. Response: We only invited international HCPs from the International Society of Pediatric Oncology to participate in a Delphi survey (no interviews) to get diverse perspectives and reach consensus on the domains and items of the WHAT questionnaires that meant to be used in the multi-cultural Canadian pediatric oncology populations. The Delphi survey is not a validity testing study; it is an important step in instrument development/adaptation (i.e., to identify the pool of domains and items that would be tested for validity and reliability). So, the first validity test for the WHAT was done in Phase 2, as we tested the content of the questionnaires in the Canadian context (i.e., no international participants). The results of the Delphi survey of Canadian children/parents, and Canadian/International HCPs showed that all domains and items of the WHAT reached consensus after one cycle – this might highlight that the original WHAT included the key core set of domains and items that a sample of participants from different cultural backgrounds agreed to include after only one cycle of Delphi survey. Also, this might reflect that we included culturally diverse sample of Canadian children and parents in our study. We added some background details to Tables 1 and 2 to show the ethnic diversity of the children and parents who participated in Phases 1 and 2 (see pages 11-12 and 16-18). Why were no complementary HCPs included in phase 2: testing face and content validity of the WHAT questionnaires? Could this be another limitation of the study? Response: We agree that including complementary HCPs would enrich our understanding of the important items to include in the WHAT questionnaires. This point has been added to the limitations section (see lines 483-488). Attachments Attachment Submitted filename: 1 - Response to Reviewers.docx https://doi.org/10.1371/journal.pone.0294393.r002
2 May 2023 Decision Letter - Erik Loeffen, Editor PONE-D-22-31203R1Assessment of complementary health approaches use in pediatric oncology: Modification and preliminary validation of the "Which Health Approaches and Treatments Are You Using?" (WHAT) questionnairesPLOS ONE Dear Dr. Alqudimat, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jun 16 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Erik Loeffen, M.D., Ph.D. Academic Editor PLOS ONE Journal Requirements: 1. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: First of all I want to congratulate the authors with an interesting an well-conducted study. The authors have incorporated the good suggestions from the reviewers, and as a result of this the paper has improved substantially. Content-wise this paper is ready for acceptance and publication in Plos ONE. Nevertheless before I make a decision to accept I would urge the authors to shorten the manuscript substantially. From what I can see the text body is between 5000 and 6000 words, and although Plos ONE has no strict word limit this feels much too long for a Delphi&interview/validation study. For instance, line 279-300 is essentially "multiple participants did not find the paper format and the use of tables user-friendly, therefore we chose to adjust...". All the results and numbers can either be removed, put into a table or (my preference) put in detail in the supplementary data. I think shortening the manuscript would increase readability in a great manner. (Especially given the nice, large tables there is no need for an abundance of results/numbers in the text.) For a swift proces, at resubmission I will handle the manuscript personally, sending for peer review would then not be necessary (as we have completed the step already). [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: (No Response) Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) Reviewer #2: Thank you for addressing all my comments. All the comments were appropriately answered and the included in the text. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0294393.r003
Revision 2
12 Jun 2023 Author Response Dr. Erik Loeffen Academic Editor PLOS ONE June 12, 2023 Dear Dr. Loeffen, Thank you for the constructive feedback on our manuscript “Assessment of complementary health approaches use in pediatric oncology: Modification and preliminary validation of the “Which Health Approaches and Treatments Are You Using?” (WHAT) questionnaires. We have carefully reviewed your comments and revised the manuscript accordingly. Please see below for an itemized response to your comments. We hope the paper is now in an acceptable form for publication in PLOS ONE. Thank you for the opportunity to revise and resubmit our manuscript. Sincerely, Mohammad R. Alqudimat (Corresponding Author) Ph.D. Candidate - University of Toronto m.alqudimat@mail.utoronto.ca Thank you for the comments and suggestions. We have substantially shortened our manuscript by 89 lines (1,339 words)—the revised text body is now 4,491 words. The following is a summary of this revision. All page and line numbers included in our response below refer to the unmarked version of our manuscript without tracked changes (labelled “manuscript”): 1) We have condensed the results section, as suggested, and moved the details to the existing Table 1 and two new Tables 4 and 6. a. The details about participants’ geographical locations have been removed from the first paragraph of the results (page 11) and moved to Table 1 “Characteristics of participants who participated in the Delphi survey” (page 12). b. The findings of the first cycle of the face and content validity study have been condensed, and specific details about comprehensiveness, comprehensibility, and relevance have been moved to a new Table 4 “Findings of the first cycle of the individual interviews regarding the comprehensiveness, comprehensibility, and relevance” (page 18). Only high-level findings have been kept in the text body (page 17). c. The section titled “Resulting modified WHAT questionnaires” has been condensed to describe the modified WHAT. d. The findings of the second cycle of the face and content validity study have been condensed, and details about comprehensiveness, comprehensibility, and relevance were moved to a new Table 6 “Findings of the second cycle of the individual interviews regarding the comprehensiveness, comprehensibility, and relevance” (page 22). Only high-level findings were kept in the text body (page 21). 2) Minor changes to parts of the design, discussion, limitation, and conclusion have been made. We have condensed or clarified these parts, and no new ideas have been added. a. The procedure of Phase 1: i. Lines 160-161: The sentence “Children and parents were asked to rate the importance of each domain and item of the child self-report and parent proxy-report versions, respectively” has been changed to “Children were asked to rate the importance of each domain and item of the self-report version, and parents were asked to rate the proxy-report.” b. The procedure and data analysis of Phase 2: i. Lines 191-197: Three sentences describing the interview procedure have been reorganized to ease the readability and to reduce the word count. Here is the revised part: “During the interviews, children were asked to read over the child self-report, parents read the parent proxy-report. Children and parents were then asked to describe what they understood about the meaning of the item and whether it was easy to understand (comprehensibility), any essential items were missing (comprehensiveness), and all items and response options were relevant to the questionnaires’ purpose. HCPs were asked to review both versions of the WHAT questionnaires and to answer open-ended questions about item relevance and comprehensiveness of the questionnaires.” ii. Lines 217-218: The word “retaining” has been clarified by adding two words, “modifying, removing, or retaining”. c. Discussion: i. Lines 309-311: The sentence “We introduce the first preliminary face and content validity tested electronic disease-specific CHA questionnaires (i.e., the modified WHAT) that are designed to assess CHA use and initiate clinical dialogue about CHA use in pediatric oncology” has been condensed and simplified to be “We introduce the first electronic disease-specific CHA questionnaires (i.e., the modified WHAT) that have evidence of face and content validity and are designed to assess CHA use and initiate clinical discussions about CHA in pediatric oncology.” ii. Lines 370-372: The sentence has been clarified by adding “Agreement between the self- and proxy-report is not granted unless confirmed by inter-rater reliability testing; for example,” to the original sentence “previous research has shown that proxy-report results are not equivalent to the self-report in a study on perceptions of quality of life in children under chiropractic care.” d. Limitations: i. Lines 392-393: Minor changes to clarify the original two sentences have been made. Here are the edited sentences “In Phase 2, children, parents, and oncology HCPs were recruited to determine the face and content validity of the WHAT. This was because we aimed to test the face and content validity of the WHAT by a sample of the populations who will use the WHAT in clinical settings: the respondents (children and parents), and data users (oncology HCPs who will use the data to initiate clinical discussion).” ii. Lines 398-401: A few words were replaced to clarify one study limitation. Here is the edited sentence “We were also not able to recruit participants from pediatric oncology newly arrived immigrant populations in Canada; thus, future research should be conducted to ask for their perspectives, and tailor the WHAT content to their CHA use experience.” e. Conclusions: i. Lines 407-410: Two sentences have been edited to clarify and simplify the points. The edited sentences are shown as “Children, parents, and HCPs agreed on the content of both versions of the WHAT questionnaires after one cycle of Delphi survey. Two iterative cycles of individual interviews and modifications made to the WHAT questionnaires according to suggestions from children, parents and HCPs have helped ensure adequate face and content validity for use in pediatric oncology.” 3) No changes to the reference list have been made. 4) The revised version of our manuscript has been formatted using PLOS ONE’s style requirements. Attachments Attachment Submitted filename: 1 - Response to Editor.pdf https://doi.org/10.1371/journal.pone.0294393.r004
2 Nov 2023 Decision Letter - Marianne Clemence, Editor Assessment of complementary health approaches use in pediatric oncology: Modification and preliminary validation of the "Which Health Approaches and Treatments Are You Using?" (WHAT) questionnaires PONE-D-22-31203R2 Dear Dr. Alqudimat, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Marianne Clemence Staff Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) Reviewer #2: (No Response) ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** https://doi.org/10.1371/journal.pone.0294393.r005
Formally Accepted
24 Feb 2024 Acceptance Letter - Marianne Clemence, Editor PONE-D-22-31203R2 PLOS ONE Dear Dr. Alqudimat, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr Marianne Clemence Staff Editor PLOS ONE https://doi.org/10.1371/journal.pone.0294393.r006

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .