PONE-D-24-14717

Using triangulation to evaluate findings from random-intercept cross-lagged panel models: An application with data on curiosity and creativity

PLOS ONE

Dear Dr. Sorjonen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We look forward to receiving your revised manuscript.

Kind regards,

Vanessa Carels

Staff Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

Response: We hope the amendments have increased the technical soundness of the manuscript.

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

Response: As we say in the manuscript, data used by Ma and Wei, and reanalyzed by us, is available at the Open Science Framework (https://osf.io/d48xa/) (line 135). Our analytic script, which downloads data used by Ma and Wei and then conducts all analyses reported in our manuscript, is available at the Open Science Framework (https://osf.io/rk4ne/) (line 205). Anyone can download our script, run it, and receive the same results as reported in our manuscript.

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1:

1. General Assessment

The manuscript presents a relevant contribution to the literature by examining the applicability of the random-intercept cross-lagged panel model (RI-CLPM) in analysing the longitudinal relationship between curiosity and creativity. The study proposes critically examining this model, employing triangulation with different analytical approaches, such as latent change models and multilevel regression. The study's main conclusion challenges previous findings that suggest a bidirectional causal relationship between curiosity and creativity, arguing that such associations may be spurious due to common latent factors. Despite the topic's relevance, methodological and interpretation aspects of the results can be improved to strengthen the robustness of the conclusions.

Response: We hope our amendments have strengthened the robustness of the conclusions.

2. Theoretical Foundation and Justification

The literature review presents a solid coverage of cross-lagged models, particularly the RI-CLPM, and their limitations in terms of causal inference. However, it could be expanded to include a more in-depth discussion on the suitability of the triangulation model in psychological research, exploring previous examples of its application. In addition, a greater alignment between the theoretical review and the formulation of hypotheses is recommended, making the relationship between the triangulated approaches and the gaps the study aims to fill more explicit.

Response: We have added the following (lines 109-114):

We have previously used triangulation, i.e., analyzed data with alternative models, to challenge conclusions based on findings from analyses with the traditional CLPM [e.g., 9–11]. Moreover, in a recent study we used triangulation to scrutinize and challenge conclusions of causal effects of academic self-concept (i.e., self-perceived academic competence) on academic achievement based on findings from analyses with the RI-CLPM [12].

3. Methodology

The choice of triangulation with multiple statistical models is a strength of the manuscript, as it increases the reliability of the findings. However, there is a need for greater clarity in the justification for selecting auxiliary models, such as the latent change model and multilevel regression with mean-centered scores. In addition, the description of the data used, which came from a sample of employees from three companies in Hong Kong, could be expanded to consider possible cultural and sampling biases that may impact the generalizability of the findings.

Response: We have added the following (lines 193-202):

Analyzing models 2 and 5 were in line with proposals that time-reversed analyses may be used to identify potential statistical artifacts [19,20]. Previous analyses of data including an unquestionable causal effect (of adding stones in a container on the total weight of the container) found consistent effects in models 1-6 (i.e., effects in line with the predictions outlined above), suggesting that these models may be used to discriminate between true causal and spurious effects [21]. Model 7 (the MoSLA) was used in two recent studies and suggested that longitudinal data on academic self-concept and achievement [12] and on executive deficits and psychopathology [22], respectively, may have been generated without any direct effects between these constructs and, consequently, that previous causal claims could be challenged.

As we say in the manuscript (lines 393-396):

all participants were employees at three firms in Hongkong, China, and it is unclear to what degree the present findings, that longitudinal associations between curiosity and creativity probably were spurious, generalizes to other cultural, social, and economical contexts.

We have now added the following (lines 396-402):

Hence, our findings may have been affected by cultural and sampling biases. However, we see no reason to believe that such biases would selectively have affected all models except the original RI-CLPM. Therefore, we do not believe that it would be tenable to argue that due to possible cultural and sampling biases, we should trust findings from the RI-CLPM, suggesting increasing prospective within-individual effects between curiosity and creativity, and dismiss contradictory findings from the other analyzed models.

4. Analysis of the Results

The results indicate that only the RI-CLPM suggests significant cross-lagged effects between curiosity and creativity, while the other models provide contrary or null evidence. The introduction of the model of spurious longitudinal associations (MoSLA) as an alternative explanatory tool is an interesting innovation, but could be better supported. Furthermore, the interpretation of the findings could include a more detailed discussion of the possible impacts of variability in individual scores and the sensitivity of the models to the characteristics of the data used.

Response: We have added the following (lines 198-202):

Model 7 (the MoSLA) was used in two recent studies and suggested that longitudinal data on academic self-concept and achievement [12] and on executive deficits and psychopathology [22], respectively, may have been generated without any direct effects between these constructs and, consequently, that previous causal claims could be challenged.

Variability in individual scores is a prerequisite for associations and effects. Without variability, i.e., if all individuals would have the same score, it would not be possible to estimate associations or effects.

Moreover, as we say in the manuscript:

we know that a positive effect (indicating a decreasing effect) in models 2 and 5 is exactly what could be expected if curiosity and creativity were measured with less than perfect reliability (which should be very likely) and, therefore, susceptible to regression to the mean. Moreover, we also know that if measures of two constructs, e.g., curiosity and creativity, are affected by some common state factor (which also should be very likely), concurrent correlations (rX1,Y1) will tend to be stronger than cross-lagged correlations (rX1,Y2) which, according to Equation 1 [27], would result in a negative effect of X1 (e.g., initial curiosity) on the Y2-Y1 (e.g., subsequent - initial creativity) difference. (lines 348-356)

The used measures, self-rated for curiosity and supervisor-rated for creativity, may not have been optimal and susceptible to measurement errors, bias, etc. However, it is important to bear in mind that the same data were used across analyzed models in the present study. Hence, possible shortcoming in measurements were constant across the analyzed models and could, consequently, not explain why the models indicated simultaneous and paradoxical increasing, decreasing, and null effects between curiosity and creativity. (lines 403-408)

5. Discussion and Contribution

The discussion section correctly highlights the challenges in causal inference in longitudinal models and reinforces the importance of triangulated approaches. However, it would be beneficial to contextualize the findings within the psychological research field and apply these results in practice to future studies on creativity and curiosity. Furthermore, the suggested research agenda could include more specific recommendations on when and how triangulation should be used to analyze longitudinal data.

Response: We have added the following (lines 289-295):

The present divergent findings suggest that the data analyzed by Ma and Wei, and reanalyzed by us, offer no support to Ma and Wei’s hypothesis that curiosity fosters creativity because it improves individuals’ ability to develop original and useful ideas. Neither does the data appear to offer support to Ma and Wei’s hypothesis, based on the information-gap theory [27], that creativity promotes curiosity because creativity prompts individuals to gather and scrutinize diverse information, make novel associations, gain fresh perspectives, and yearn for new knowledge.

Moreover, as we say in the manuscript (lines 372-374):

We believe that the present study, and the available analytic script (https://osf.io/rk4ne/), could be used as a model/tutorial by researchers wishing to scrutinize results from the RI-CLPM,

We have now added the following (lines 374-378):

i.e., when analyzing within-individual effects between two longitudinally measured constructs. The method of triangulation could be used with data on curiosity and creativity as well as observational (i.e., non-experimental) data on other constructs, both in psychology and other areas of research. Interested researchers are welcome to contact the corresponding author (KS) for advice and assistance.

6. Conclusion and Recommendations

The manuscript represents a valuable contribution to understanding the limitations of cross-lagged models and promoting triangulation in longitudinal analyses. However, the following improvements are recommended before acceptance for publication:

- Expand the theoretical review to contextualize the use of triangulation in psychology better;

Response: We have added the following (lines 109-114):

We have previously used triangulation, i.e., analyzed data with alternative models, to challenge conclusions based on findings from analyses with the traditional CLPM [e.g., 9–11]. Moreover, in a recent study we used triangulation to scrutinize and challenge conclusions of causal effects of academic self-concept (i.e., self-perceived academic competence) on academic achievement based on findings from analyses with the RI-CLPM [12].

- Justify in more detail the choice of auxiliary models used in the analysis;

Response: We have added the following (lines 193-202):

Analyzing models 2 and 5 were in line with proposals that time-reversed analyses may be used to identify potential statistical artifacts [19,20]. Previous analyses of data including an unquestionable causal effect (of adding stones in a container on the total weight of the container) found consistent effects in models 1-6 (i.e., effects in line with the predictions outlined above), suggesting that these models may be used to discriminate between true causal and spurious effects [21]. Model 7 (the MoSLA) was used in two recent studies and suggested that longitudinal data on academic self-concept and achievement [12] and on executive deficits and psychopathology [22], respectively, may have been generated without any direct effects between these constructs and, consequently, that previous causal claims could be challenged.

- Deepen the discussion on the implications of the findings for research on creativity and curiosity;

Response: We have added the following (lines 289-295):

The present divergent findings suggest that the data analyzed by Ma and Wei, and reanalyzed by us, offer no support to Ma and Wei’s hypothesis that curiosity fosters creativity because it improves individuals’ ability to develop original and useful ideas. Neither does the data appear to offer support to Ma and Wei’s hypothesis, based on the information-gap theory [27], that creativity promotes curiosity because creativity prompts individuals to gather and scrutinize diverse information, make novel associations, gain fresh perspectives, and yearn for new knowledge.

- Explore more clearly the limitations and potential biases of the sample used.

Response: As we say in the manuscript (lines 393-396):

all participants were employees at three firms in Hongkong, China, and it is unclear to what degree the present findings, that longitudinal associations between curiosity and creativity probably were spurious, generalizes to other cultural, social, and economical contexts.

We have now added the following (lines 396-402):

Hence, our findings may have been affected by cultural and sampling biases. However, we see no reason to believe that such biases would selectively have affected all models except the original RI-CLPM. Therefore, we do not believe that it would be tenable to argue that due to possible cultural and sampling biases, we should trust findings from the RI-CLPM, suggesting increasing prospective within-individual effects between curiosity and creativity, and dismiss contradictory findings from the other analyzed models.

Reviewer #2:

This study re-examines prior findings that curiosity and creativity mutually reinforce each other using the random-intercept cross-lagged panel model (RI-CLPM). Acknowledging potential bias in RI-CLPM results, the authors apply triangulation with alternative methods, including a latent change score model (LCSM), multilevel regression with person-mean centering, and the novel Model of Spurious Longitudinal Associations (MoSL

Attachments

Attachment

Submitted filename: Triangulation_Response_R1.docx

PONE-D-24-14717R1

Using triangulation to evaluate findings from random-intercept cross-lagged panel models: An application with data on curiosity and creativity

PLOS ONE

Dear Dr. Sorjonen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The paper is poorly written; many points and claims are incomprehensible.

1. The triangulation problem and circle illustration in Fig. 1 make little sense and are disconnected from the statistical model used by the authors. For example, if the points are in a three-dimensional space, the author’s conclusion is wrong. The authors do not solve a location problem, and therefore, this illustration does not work. I guess the authors meant that more information implies a more accurate answer or solution— a trivial statement.

Response: Although the point that more information allows more accurate answers may seem trivial, it appears not to be appreciated by many researchers. For example, it is quite common to conclude, as in the study by Ma and Wei (2023) that we challenge, causal effects based on cross-lagged effects without requiring any additional information. The point we are making with the triangulation example is that just as we should not infer a location based on its distance from another (known) location (something we assume/hope that all readers would agree with), we should not infer causality from a cross-lagged effect in observational data. We hope that this may open some eyes among researchers who have been lured into believing that cross-lagged panel models (with or without random intercepts) may, magically, reveal causality in observational data. We have added the following (line 41);

(in a two-dimensional space)

2. The statistical model, the heart of the discussion, is not defined. It is impossible to judge the validity of the conclusions and results until the model, along with all the involved variables depicted in Fig. 3, is mathematically defined in equation form (if the authors feel the model is too complicated, it may be presented in the Supplementary Materials file). The reader does not know what exactly the model is. The table with all estimated coefficients, along with standard errors and p-values, must be presented as well.

Response: We believe it to be standard practice to present structural equation models (SEM) as path diagrams rather than as equations. See, for example, Sorjonen et al. (2025) for a recent example by us and Toyoshima et al. (2020) for an example published in PLOS ONE. Such path-diagrams can also be found in very prestigious journals (e.g., Gosling et al., 2022). However, we have included documents with all effects in equation form as well as all output from the analyzed models (including standard errors and p-values) in our supplementary. We have added the following:

(regression equations for effects included in these models can be found in the document “Supplementary_Equations” available at https://osf.io/rk4ne/) (lines 148-150).

(full output for all models, including standard errors, p-values, etc., can be found in the document “Supplementary_Output” available at https://osf.io/rk4ne/) (lines 221-222).

As a general critique, the paper lacks transparency regarding its modeling assumptions, making it difficult to assess the validity of the results.

Response: We have made the analytic script publicly available (see line 216). Anyone with access to a computer and the internet can download the script, run it (with the statistical freeware R), and receive the same results as we did. This was confirmed in the previous round by reviewer # 3. Now we have also included documents with all effects in equation form as well as all output from the analyzed models (including standard errors and p-values) in our supplementary. We have added the following:

(regression equations for effects included in these models can be found in the document “Supplementary_Equations” available at https://osf.io/rk4ne/) (lines 148-150).

(full output for all models, including standard errors, p-values, etc., can be found in the document “Supplementary_Output” available at https://osf.io/rk4ne/) (lines 221-222).

Please submit your revised manuscript by Dec 12 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Eugene Demidenko, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise.

Additional Editor Comments:

The paper is poorly written; many points and claims are incomprehensible.

1. The triangulation problem and circle illustration in Fig. 1 make little sense and are disconnected from the statistical model used by the authors. For example, if the points are in a three-dimensional space, the author’s conclusion is wrong. The authors do not solve a location problem, and therefore, this illustration does not work. I guess the authors meant that more information implies a more accurate answer or solution— a trivial statement.

Response: Although the point that more information allows more accurate answers may seem trivial, it appears not to be appreciated by many researchers. For example, it is quite common to conclude, as in the study by Ma and Wei (2023) that we challenge, causal effects based on cross-lagged effects without requiring any additional information. The point we are making with the triangulation example is that just as we should not infer a location based on its distance from another (known) location (something we assume/hope that all readers would agree with), we should not infer causality from a cross-lagged effect in observational data. We hope that this may open some eyes among researchers who have been lured into believing that cross-lagged panel models (with or without random intercepts) may, magically, reveal causality in observational data. We have added the following (line 41);

(in a two-dimensional space)

2. The statistical model, the heart of the discussion, is not defined. It is impossible to judge the validity of the conclusions and results until the model, along with all the involved variables depicted in Fig. 3, is mathematically defined in equation form (if the authors feel the model is too complicated, it may be presented in the Supplementary Materials file). The reader does not know what exactly the model is. The table with all estimated coefficients, along with standard errors and p-values, must be presented as well.

Response: We believe it to be standard practice to present structural equation models (SEM) as path diagrams rather than as equations. See, for example, Sorjonen et al. (2025) for a recent example by us and Toyoshima et al. (2020) for an example published in PLOS ONE. Such path-diagrams can also be found in very prestigious journals (e.g., Gosling et al., 2022). However, we have included documents with all effects in equation form as well as all output from the analyzed models (including standard errors and p-values) in our supplementary. We have added the following:

(regression equations for effects included in these models can be found in the document “Supplementary_Equations” available at https://osf.io/rk4ne/) (lines 148-150).

(full output for all models, including standard errors, p-values, etc., can be found in the document “Supplementary_Output” available at https://osf.io/rk4ne/) (lines 221-222).

As a general critique, the paper lacks transparency regarding its modeling assumptions, making it difficult to assess the validity of the results.

Response: We have made the analytic script publicly available (see line 216). Anyone with access to a computer and the internet can download the script, run it (with the statistical freeware R), and receive the same results as we did. This was confirmed in the previous round by reviewer # 3. Now we have also included documents with all effects in equation form as well as all output from the analyzed models (including standard errors and p-values) in our supplementary. We have added the following:

(regression equations for effects included in these models can be found in the document “Supplementary_Equations” available at https://osf.io/rk4ne/) (lines 148-150).

(full output for all models, including standard errors, p-values, etc., can be found in the document “Supplementary_Output” available at https://osf.io/rk4ne/) (lines 221-222).

References

Gosling, W. D., Miller, C. S., Shanahan, T. M., Holden, P. B., Overpeck, J. T., & Van Langevelde, F. (2022). A stronger role for long-term moisture change than for CO2 in determining tropical woody vegetation change. Science, 376(6593), 653–656. https://doi.org/10.1126/science.abg4618

Ma, J. (Yonas), & Wei, W. (2023). Curiosity causes creativity? Revealing the reinforcement circle between state curiosity and creativity. The Journal of Creative Behavior, jocb.606. https://doi.org/10.1002/jocb.606

Sorjonen, K., Melin, B., & Nilsonne, G. (2025). Inconclusive evidence for a prospective effect of academic self-concept on achievement: A simulated reanalysis and comment on Marsh et al. (2024). Educational Psychology Review, 37(2), 30. https://doi.org/10.1007/s10648-025-10008-4

Toyoshima, K., Inoue, T., Masuya, J., Fujimura, Y., Higashi, S., Tanabe, H., & Kusumi, I. (2020). Structural equation modeling approach to explore the influence of childhood maltreatment in adults. PLOS ONE, 15(10), e0239820. https://doi.org/10.1371/journal.pone.0239820

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

________________________________________

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

________________________________________

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

________________________________________

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

________________________________________

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

________________________________________

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: This revision satisfactorily addresses all my comments. This paper represents a good reanalysis of existing data.

________________________________________

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Curtis Atkisson

Attachments

Attachment

Submitted filename: Triangulation_Response_R2.docx