A power approximation for the Kenward and Roger Wald test in the linear mixed model

Sarah M. Kreidler; Brandy M. Ringham; Keith E. Muller; Deborah H. Glueck

doi:10.1371/journal.pone.0254811

Peer Review History

Original SubmissionJanuary 7, 2021
3 Mar 2021 Decision Letter - Lei Shi, Editor PONE-D-21-00404 A power approximation for the Kenward and Roger Wald test in the linear mixed model PLOS ONE Dear Dr. Ringham Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Apr 17 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols We look forward to receiving your revised manuscript. Kind regards, Lei Shi Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following in the Competing Interests section: "The authors have declared that no competing interests exist." We note that one or more of the authors are employed by a commercial company: Sunrun Inc. 2.1. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form. Please also include the following statement within your amended Funding Statement. “The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.” If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement. 2.2. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf. Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests 3. Thank you for stating the following in the Acknowledgments Section of your manuscript: Funding for this work was provided by NIDCR 1 R01 DE020832-01A1 (Keith E. Muller, PI; Deborah H. Glueck, University of Colorado site PI), NIGMS 9R01GM121081-05 (Deborah H. Glueck, Keith E. Muller, Dana Dabelea, PIs), and OD 5UG3OD023248-02 (Dana Dabelea, PI). We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "KEM, DHG NIDCR 1 R01 DE020832-01A1 The National Institute of Dental and Craniofacial Research www.nih.gov No KEM, DHG NIGMS 9R01GM121081-05 The National Institute of General Medical Sciences www.nih.gov No The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." Please include your amended statements within your cover letter; we will change the online submission form on your behalf. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: In this paper, a power approximation of Kenward and Roger test is derived. Via Monte Carlo simulation, author’s demonstrate that the new power approximation is accurate for cluster randomised trials and longitudinal study designs. The paper is well written and addresses an interesting problem. My major issues are listed below.Major comments: Comment 1: On line 171, it is claimed that if Σs and β are estimated using multivariate techniques, independence would follow. Provide a reference for this or give a detailed explanation in support of this claim. Comment 2: The comparison of empirical and proposed powers is done assuming intraclass correlation(ICC) 0.04. This value of ICC is very small and in practice it can vary up to 0.5. Make the comparison of powers for higher values of ICC as well (say 0.1, 0.2, 0.5). Comment 3: In the spirit of the longitudinal studies, how efficient is the power approximation when the correlation structure is assumed to be auto–regressive? Comment 4: In Section 5 (Applied Example), rather than assuming the values of standard deviation and intraclass correlation, it is more reasonable to use the estimates of the parameters obtained from the data. Minor comments: Comment 1:Check line 122 Reviewer #2: The authors here present a noncentral F power approximation for the Kenward and Roger test. This work is innovative, and the organization of the manuscript is clear and comprehensive. Below are some minor comments that could help further streamline the text. My review comments has been uploaded as an attachment. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Huang Lin [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. Attachments Attachment Submitted filename: Review_Report.pdf Attachment Submitted filename: Comments.docx https://doi.org/10.1371/journal.pone.0254811.r001
Revision 1
10 May 2021 Author Response Response to Reviewers We thank the reviewers for their kind comments. We have responded to all comments below. Reviewer comments are in italics and our response is in plain type. General Comments 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf Done. 2. Thank you for stating the following in the Competing Interests section: "The authors have declared that no competing interests exist." We note that one or more of the authors are employed by a commercial company: Sunrun Inc. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form. Please also include the following statement within your amended Funding Statement. “The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.” If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf. Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests We have removed Dr. Kreidler’s affiliation with SunRun, Inc. Previously we indicated that Dr. Kreidler was affiliated with SunRun Inc. After further review, and examining the policies of PLOS One, we realized that SunRun did not play any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and, in addition, did not provide support in the form of salaries for any author, including SMK. We have updated Dr. Kreidler’s affiliation to be the Department of Biostatistics and Information, University of Colorado Denver. Although Dr. Kreidler is currently employed by SunRun, Inc., all of the work on this manuscript was completed while Dr. Kreidler was a doctoral student at the University of Colorado Denver. Current revisions are being done during Dr. Kreidler’s personal time. No SunRun, Inc. resources were used in the revisions of this manuscript for submission. The publication of this manuscript will not affect SunRun Inc., nor Dr. Kreidler in any financial way. 3. Thank you for stating the following in the Acknowledgments Section of your manuscript: Funding for this work was provided by NIDCR 1 R01 DE020832-01A1 (Keith E. Muller, PI; Deborah H. Glueck, University of Colorado site PI), NIGMS 9R01GM121081-05 (Deborah H. Glueck, Keith E. Muller, Dana Dabelea, PIs), and OD 5UG3OD023248-02 (Dana Dabelea, PI). We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "KEM, DHG NIDCR 1 R01 DE020832-01A1 The National Institute of Dental and Craniofacial Research www.nih.gov No KEM, DHG NIGMS 9R01GM121081-05 The National Institute of General Medical Sciences www.nih.gov No The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." Please include your amended statements within your cover letter; we will change the online submission form on your behalf. We have provided an amended funding statement in the cover letter for this resubmission. Reviewer 1 Comments 1. On line 171, it is claimed that if Σs and β are estimated using multivariate techniques, independence would follow. Provide a reference for this or give a detailed explanation in support of this claim. We now provide a citation to support the statement above where it is mentioned in the manuscript. For reference, the citation is Anderson (1984, pg. 291, Theorem 8.2.2). 2. The comparison of empirical and proposed powers is done assuming intraclass correlation(ICC) 0.04. This value of ICC is very small and in practice it can vary up to 0.5. Make the comparison of powers for higher values of ICC as well (say 0.1, 0.2, 0.5). We have re-run the simulation after incorporating the reviewer’s suggestions. The new results appear in the revised manuscript. The different ICC’s do not appear to change the accuracy of the results. 3. In the spirit of the longitudinal studies, how efficient is the power approximation when the correlation structure is assumed to be auto–regressive? We used an auto-regressive correlation structure for the simulation studies described in Section 4.1.2 Longitudinal Designs. The median deviation between the approximate power and the true power was 0.003 (range: -0.010 to 0.016; 1st quartile: 0.00, 3rd quartile: 0.009). 4. In Section 5 (Applied Example), rather than assuming the values of standard deviation and intraclass correlation, it is more reasonable to use the estimates of the parameters obtained from the data. The example is a synthetic example designed to demonstrate the utility of the calculations for a simple, cluster randomized study with different sized clusters. While the example was inspired by the trial described by Hennrikus et al., the example was so simplified that the reference to the Hennrikus et al. paper was confusing, rather than helpful. We have removed the reference to Hennrikus et al., clarified that the example is synthetic, and added a sentence to describe how a researcher might extract the inputs for the power analysis from review of the literature. 5. Check line 122 We reviewed the line and removed the duplicated equation. Thank you for finding the error. Reviewer 2 Comments 1. The homoscedasticity assumption (line 136) the author made could be a key limitation of the approximation method with the presence of different random coefficients. Even though it has been mentioned that there are no repeated covariates (line 82), it is worth mentioning it in the Discussion section. We now describe limitations due to model assumptions in the Discussion section. “The method does not apply to repeated covariates, which often appear in biomedical studies. However, the method does apply to baseline covariates, a common study design. We make a strong homoscedasticity assumption of equal variance for each independent sampling unit. This assumption means that the power computations are not appropriate for random regression, for models with group differences in variance, or for certain spatial-temporal applications. Nevertheless, the assumption of homoscedasticity is widely made for randomized controlled clinical trials, laboratory studies, and observational studies, which makes the method useful for a variety of cases.” 2. Theorem 3 in the Appendix lacks the reference regarding “the sum of the inverse Wishart distribution asymptotically or approximately follows an inverse Wishart distribution.” We have now clarified the proof for Theorem 3 to explicitly state what we set out to prove. We added the following text to the end of the proof. “The method of moments approximation yields an asymptotic approximation for the sum, as desired.” 3. Using p_i to denote the number of observations is counter-intuitive, especially given you define n=∑_(i=1)^N▒p_i . Replacing p_i with n_i would be easier to read. We recognize the value of using n to denote different sample sizes. However, for this particular application, we use multivariate modeling theory to develop our results. In multivariate notation, it is standard to refer to the repeated measures using p. When the data are stacked, the vector of observations for independent sampling unit i is then given pi. We recognize that the notation may appear slightly awkward, but respectfully request that we may keep it as written so that it aligns with standard notation in the field. See, for example, Muller and Stuart (2006). 4. Duplicated terms appeared in line 122. Thank you for finding the error. We have corrected it. 5. The notation for noncentrality parameter of F distribution is very similar to the notation of Wald statistic, please consider changing one of them. We agree and have now adjusted the notation for the non-centrality parameter that appears in Section 2.1 Notation. 6. Box-plot elements should be defined (e.g. center line, median; box limits, upper and lower quartiles; whiskers, 1.5x interquartile range; points, outliers) in the legend. Done. Attachments Attachment Submitted filename: Repsonse to Editor Requests 04.pdf https://doi.org/10.1371/journal.pone.0254811.r002
5 Jul 2021 Decision Letter - Lei Shi, Editor A power approximation for the Kenward and Roger Wald test in the linear mixed model PONE-D-21-00404R1 Dear Dr. Brandy Ringham, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Lei Shi Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: (No Response) ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: (No Response) ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: (No Response) ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: (No Response) ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: In the revised version authors successfully incorporated all the suggestions and corrections. All comments are addressed adequately. Reviewer #2: (No Response) ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Huang Lin https://doi.org/10.1371/journal.pone.0254811.r003
Formally Accepted
9 Jul 2021 Acceptance Letter - Lei Shi, Editor PONE-D-21-00404R1 A Power Approximation for the Kenward and Roger Wald Test in the Linear Mixed Model Dear Dr. Ringham: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Lei Shi Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0254811.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .