Analysing trajectories of a longitudinal exposure: A causal perspective on common methods in lifecourse research

Sarah C. Gadd; Peter W. G. Tennant; Alison J. Heppenstall; Jan R. Boehnke; Mark S. Gilthorpe

doi:10.1371/journal.pone.0225217

Peer Review History

Original SubmissionAugust 2, 2019
16 Oct 2019 Decision Letter - Stephen Z Levine, Editor PONE-D-19-21843 Analysing trajectories of a longitudinal exposure: a causal perspective on common methods in lifecourse research PLOS ONE Dear Ms Gadd, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. We would appreciate receiving your revised manuscript by Nov 30 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. We look forward to receiving your revised manuscript. Kind regards, Prof Stephen Z Levine, PhD Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. In your Methods section, it is reported that "Simulation code is available as supplementary material"; please correct this statement, as we note that the code is not provided as a supplementary file, but is instead deposited in GitHub. Additional Editor Comments (if provided): [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This is an interesting paper, addressing a very important methodological issue in lifecourse epidemiology. I have some suggestions to improve the accessibility of this manuscript: 1. To make the comparison across different models/methodologies more explicit, I think it would be useful to spell out the model specifications in equations. Although the directed acyclic diagrams are useful for visualizing the causal relations between variables, equations help readers with the understanding of the statistical models used to estimate the causal relations. 2. Since DAGs are used to show the causal structure in the simulated data, I think it would be equally useful to use DAGs to show the implied causal structures within the statistical models (e.g. multilevel models etc). SO readers could compare the DAGs to apprehend why some models are more "correct" than others in different scenarios. 3. Although the causal relations between the variables in the simulated data were shown in DAGs, I feel it would useful to provide the R codes in the appendix; so readers could examine the details of the simulations. Reviewer #2: This is a clear, nicely written paper on an important topic. My comments are below: Major comment: 1. The methods that condition on the outcome didn't perform as badly as I expected. My understanding is that this is because of the restriction to settings with no time-varying confounder-treatment feedback as stated in the discussion. I think it would be helpful to introduce this restriction / assumption earlier in the paper -- perhaps in the introduction -- so that the results don't get over-generalized. Minor comment: 1. The simulated data creates a scenario in which no one is lost to follow-up or experiences a competing event. While this is a reasonable simplification to demonstrate the problems with the outcome-control methods, some readers may miss that there are further potential dangers associated with LTFU or competing events. A short review of these issues in the discussion would make the paper a bit more complete. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0225217.r001
Revision 1
28 Oct 2019 Author Response Reviewer #1 Questions 1. Is the manuscript technically sound, and do the data support the conclusions? Reviewer #1: Yes 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes 3. Have the authors made all data underlying the findings in their manuscript fully available? Reviewer #1: No Author response: The data underlying findings in this manuscript was simulated using code made available on GitHub which we appreciate was not correctly indicated in the manuscript. Following on from this, we decided to include code as an appendix as GitHub hosting may not be permanent. Author change: Changed “as supplementary material.” to “in the S1 appendix.” in Methods [p5]; Added “Supporting Information Captions¶ S1 Appendix: Code used to simulate and analyse data.” at end of manuscript [p19]. Supplementary material has been submitted in the file “S1_Appendix.docx”. 4. Is the manuscript presented in an intelligible fashion and written in standard English? Reviewer #1: Yes Reviewer #1 Comments to the Author This is an interesting paper, addressing a very important methodological issue in lifecourse epidemiology. I have some suggestions to improve the accessibility of this manuscript: 1. To make the comparison across different models/methodologies more explicit, I think it would be useful to spell out the model specifications in equations. Although the directed acyclic diagrams are useful for visualizing the causal relations between variables, equations help readers with the understanding of the statistical models used to estimate the causal relations. Author response: Thank you for this suggestion, we agree that this will be a useful addition to the paper to clarify the models used. We note that no equations have been provided for the Z-score plots as these do not use models. Author change: Equation 1 added following the changed sentence: “The multilevel model (outcome as covariate) analysis involved fitting multi-level models of weight over time, with covariates for age, diabetes status, and an age-diabetes interaction term, defined by the following equations (where i indexes observations and j individuals):“ in Methods [p6]; Equation 2 added following the altered sentence: “The multilevel model (two-step) approach involved fitting two models, defined by the following equations (where i indexes observations and j individuals):” in Methods [p6-7]. References to Equation 2.1 and 2.2 added in Methods [p7]: “The first (Equation 2.1) was a multilevel…” and “In the second step (Equation 2.2), a logistic…” 2. Since DAGs are used to show the causal structure in the simulated data, I think it would be equally useful to use DAGs to show the implied causal structures within the statistical models (e.g. multilevel models etc). SO readers could compare the DAGs to apprehend why some models are more "correct" than others in different scenarios. Author response: Thanks for this useful suggestion. While we agree that visual representations of models are often useful for explaining them, in this case, we have chosen not to include DAGs to represent the implied causal structures of the models for the following reasons: 1. A single model specification could correctly estimate a causal effect for a multitude of different DAGs. That is, a DAG (given a chosen exposure and outcome) implies one (or several) appropriate models, but a given model may also be appropriate for many DAGs. For example, the four DAGs shown in the Response to Reviewers document all imply the model D~A to find the total causal effect of A on D, but the model implies any of these DAGs (and many more with these four variables). We thought to include many implied DAGs for each model could potentially lead to greater confusion than clarity and would certainly warrant a more detailed explanation that might only detract from the main message of the paper. 2. Some features of the models are not commonly understood as a feature of DAGs. In particular, this relates to the multilevel model with an outcome as covariate which contains an interaction term, indicating “effect modification” between the time and diabetes variables. While effect modification in DAGs has been discussed (https://www.ncbi.nlm.nih.gov/pubmed/17700242) it is not commonly implemented and so would not be a familiar component of a DAG to most readers. In the presence of the newly included model equations to clarify their specification (as per comment 1) we felt that the inclusion of implied DAGs may create more confusion than the clarity it would seek to bring. 3. Although the causal relations between the variables in the simulated data were shown in DAGs, I feel it would useful to provide the R codes in the appendix; so readers could examine the details of the simulations. Author response: Thank you for highlighting this issue – we have made appropriate changes to include the code as supplementary material as discussed in our response to Reviewer #1 Questions #3. Reviewer #2 Questions 1. Is the manuscript technically sound, and do the data support the conclusions? Reviewer #2: Yes 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes 3. Have the authors made all data underlying the findings in their manuscript fully available? Reviewer #2: Yes 4. Is the manuscript presented in an intelligible fashion and written in standard English? Reviewer #2: Yes Reviewer #2 Comments to the Author This is a clear, nicely written paper on an important topic. My comments are below: Major comment: 1. The methods that condition on the outcome didn't perform as badly as I expected. My understanding is that this is because of the restriction to settings with no time-varying confounder-treatment feedback as stated in the discussion. I think it would be helpful to introduce this restriction / assumption earlier in the paper -- perhaps in the introduction -- so that the results don't get over-generalized. Author response: Thank you for this useful comment, we agree that it is important to introduce this earlier in the paper for the reasons stated and we have made changes to do so. Author changes: In the Introduction section [p4] the sentence “Approaches are suggested to avoid these biases” has been changed to “Such methods are compared to an alternative approach that avoids these biases. This method, however, is not suitable for all situations; other methods, such as g-methods, would be necessary in the presence of time-varying confounding, which is not examined in this paper.” Minor comment: 1. The simulated data creates a scenario in which no one is lost to follow-up or experiences a competing event. While this is a reasonable simplification to demonstrate the problems with the outcome-control methods, some readers may miss that there are further potential dangers associated with LTFU or competing events. A short review of these issues in the discussion would make the paper a bit more complete. Author response: This is another important restriction that should be highlighted and we are grateful for pointing it out. Author changes: We have added the following paragraph to the discussion section [p15]: “It is important to note that, for illustrative purposes, this paper presents a simplified scenario in which there are no competing events or loss to follow up, both of which would be present in reality. Any differential loss to follow up or occurrence of competing events would (further) bias the results from all three methods examined in this paper [20].” Reference number 20 has been added to the reference list [p19]. Attachments Attachment Submitted filename: Response to reviewer comments.docx https://doi.org/10.1371/journal.pone.0225217.r002
31 Oct 2019 Decision Letter - Stephen Z Levine, Editor Analysing trajectories of a longitudinal exposure: a causal perspective on common methods in lifecourse research PONE-D-19-21843R1 Dear Dr. Gadd, We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements. Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication. Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. With kind regards, Stephen Z Levine, PhD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0225217.r003
Formally Accepted
7 Nov 2019 Acceptance Letter - Stephen Z Levine, Editor PONE-D-19-21843R1 Analysing trajectories of a longitudinal exposure: a causal perspective on common methods in lifecourse research Dear Dr. Gadd: I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. For any other questions or concerns, please email plosone@plos.org. Thank you for submitting your work to PLOS ONE. With kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Stephen Z Levine Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0225217.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .