https://orcid.org/0000-0002-8795-7352
Figures
Abstract
Objective
Randomised controlled trials (RCTs) are considered the gold standard in clinical research. However, randomization isn’t always feasible, e.g., due to ethical considerations. As a result, quasi-experimental designs, such as the regression-discontinuity design (RDD), have been advocated as valuable alternatives and are increasingly employed in health sciences. So far, however, there is no way to take repeated measures into account in an RDD analysis, although repeated measures are not uncommon in medical research. This article proposes an extension of the RDD methodology to incorporate repeated measures within the statistical analysis.
Methods
The article introduces consistent mathematical notation for combining a univariate RDD with a linear mixed model (LMM) to account for repeated measures. The application is presented using data from the nFC-isPO (N = 1,417), where the Hospital Anxiety and Depression Scale (HADS) was employed to assess anxiety and depression in newly diagnosed cancer patients over a 12-month treatment period. The HADS scores were measured at baseline (T1), after 4 months (T2) and after 12 months (T3). Patients were assigned to control (psychosocial care) or treatment (psycho-oncological-psychotherapeutic care) group based on a predetermined HADS threshold at T1.
Results
The average treatment effect (ATE) was similar in both univariate RDD and LMM-RDD when applied to complete cases. Including all available data (T2 and/or T3), univariate RDD analyses were based on different samples for each time point. At T2, the ATE sign reversed from negative to positive, suggesting a change in discontinuity direction. At T3, the ATE magnitude nearly doubled compared to the complete case analysis. LMM-RDD estimated treatment effects were higher than those from univariate RDD. Nevertheless, none reached statistical significance. The time effect (), representing the difference in treatment effects between two time points (complete case:
=0.577; T2 and/or T3 available:
=0.491), was not significant (p = 0.855; 95%-CI: [−5.589; 6.743]; p = 0.869, 95%-CI: [−5.328; 6.310]).
Conclusion
Extending the RDD to incorporate repeated measures within a linear mixed model (LMM-RDD) provides a more robust and comprehensive analytical approach when an RCT is not feasible. This approach not only addresses the challenges of missing data and an unbalanced number of outcome measures, but also allows for the identification of time-varying treatment effects that may not be discernible using traditional RDD.
Citation: Hagemeier A, Rosenberger KD, Hellmich M (2026) Extension of Regression Discontinuity Designs (RDD) to evaluate time effects – A quasi-experimental approach using the data from the new form of care “Integrated cross-sectoral psycho-oncology (nFC- isPO)”. PLoS One 21(2): e0343414. https://doi.org/10.1371/journal.pone.0343414
Editor: Mohammed Misbah Ul Haq, Deccan School of Pharmacy, INDIA
Received: March 1, 2025; Accepted: January 16, 2026; Published: February 26, 2026
Copyright: © 2026 Hagemeier et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets generated and/or analysed during the current study are not publicly available due to ethical and legal restrictions. Data are available from the Institute of Medical Statistics and Computational Biology (IMSB) of the Medical Faculty of the University of Cologne and the University Hospital Cologne, Germany (contact via imsb-kontakt@uni-koeln.de) for researchers who meet the criteria for access to confidential data.
Funding: The preparation of this manuscript was partially funded by the Innovation Fonds of the Joint Federal Committee (Innovationsausschuss beim Gemeinsamen Bundesausschuss) in the form of the project “isPO—Integrated, cross-sectoral psycho-oncology” (grant number: 01NVF17022-IsPo). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. There was no additional external funding received for this study.
Competing interests: The authors have declared that no competing interests exist.
Abbrevations:: ATE, average treatment effect; CAPSYS2020, computer-assisted assistance system in isPO; HADS, Hospital Anxiety and Depression Scale; LMM-RDD, linear mixed model RDD; nFC-isPO, new form of care integrated, cross-sectoral psycho-oncology; NRW, North Rhine-Westphalia; RCT, Randomised controlled trial; RDD, Regression discontinuity design
1. Introduction
Although the randomised controlled trial (RCT) is the gold standard in clinical research, it cannot be done in every setting. For example, if the intervention to be evaluated has already been implemented in standard practice withholding from certain patients may not be an ethical option anymore [1]. However, a quasi-experimental design, mimicking randomization of individuals, might still provide valid information to evaluate the intervention. Specifically, in the regression discontinuity design (RDD), which in its simplest form is based on ordinary linear regression, a predefined threshold is used to assign cases to treatment groups. The control group receives standard care, while the intervention group receives standard care plus extended care. This assignment is based on a continuous parameter measured before the intervention. A quasi-randomisation is created around the threshold within a certain pre-defined range (bandwidth). Within this range, the treatment effect can be estimated, visible as the discontinuity of two regression lines at the threshold [2]. The design has its origins in 1960 [3], however was rarely used in the health sciences until 2010, and thereafter has become increasingly popular, especially in the last five years [4]. Despite the increasing use of this design and the ability to examine causal relationships without random assignment, there is currently no way to account for repeated measures within the RDD. Repeated measures are common in medical research and many studies use them to estimate treatment effects over time [5], including the new Form of Care and integrated cross-sectoral Psycho-Oncological (nFC-isPO) study. The nFC-isPO ran from 2017 to 2022 and aimed to reduce anxiety and depression in newly diagnosed cancer patients during 12 months of care. The Hospital Anxiety and Depression Scale (HADS) was used to measure the patient’s burden at time of diagnosis (baseline, T1), after 4 months (T2) and after 12 months of treatment (primary outcome, T3) [6,7]. The study was conducted and analysed using the RDD where both the continuous assignment variable and the outcome variable are HADS scores measured at different time points. The assignment into control (HADS≤14) and intervention (HADS≥15) group was done using the baseline HADS value (T1, assignment variable) with c = 14.5 as the cut-off value [8,9]. This threshold value is established in psycho-oncological care and was used in the isPO study due to its clinical relevance for the classification of ‘moderate to high psychological stress’ [7,10,11]. For final evaluation of the study, the time points T2 and T3, were analysed separately, as with the standard RDD approach it is not possible to take into account time effects. In order to obtain additional information on temporal relationships, mixed models for repeated measures were applied in secondary analyses [12]. However, an integrated approach considering time effects within the RDD analysis was not yet described in literature.
The aim of this article is to oppose and formally describe mixed model repeated measures approach to analyse the RDD for a scenario with continuous outcome measured at 3 time points (baseline T1, T2, T3) and to exemplify this novel approach with recent data from the nFC-isPO.
2. Methods
2.1 Model extension
To address the effect over time, the functional form of the RDD, which is based on a simple linear regression, needs to be extended to a linear mixed model for repeated measures.
Defining Y as the continuous outcome for n observations, X as the covariate that is the continuous assignment variable for each observation and c as the cut-off value for treatment assignment based on the covariate X. Furthermore, define as the measure of the distance to the cut-off and let 1 denote the indicator function for the treatment allocation, that equals
(control group) or
(treatment) (the so-called “sharp RDD”).
Then the univariate (i.e., considering only one time point) RDD equation [2] is given as
with patients and residuals
with mean of
and variance of σ² as the random error. Equation (1) corresponds to Fig 1 where
is the intercept,
denotes the slope
of the regression line below the cut-off (control),
denotes the change in slope (
for the regression line above the threshold (treatment) compared to the control group and
denotes the average treatment effect (ATE, discontinuity/ “jump” at cut-off).
A more detailed description of the RDD can be found in [2]. Equation (1) is then extended to a linear mixed model (LMM) for repeated measures over time:
where is defined as the continuous outcome for patient
and time points
.
is a dummy variable to model the time effect with running index
.
denotes the change in intercept at time point
compared to time point
.
denote the changes in slopes at time point
compared to time point
and
is the change in the treatment effect for time point
compared to time point
.
, with a mean of
and a variance
, is the random intercept for patient
to account for stochastic dependence over time (repeated measures) and
with a mean of
and a variance σ denotes the vector of error terms for patient
at time point
. A random slope could be defined in a similar way for each coefficient, but is not considered further below for the purpose of simplicity.
For illustration, equation (3) shows the example for just two outcome measures, i.e., two time points with (
and
) for
patients:
With and
a schematic representation of equation (3) is given in Fig 2.
The extended model now considers the correlation between repeated measures for each observation. Moreover, allows the inclusion of cases with only one follow-up outcome measure, assuming that these are missing at random. The extension of the RDD equation to a linear mixed model RDD for repeated measures allows the evaluation of the time effect within an RDD analysis. This is achieved by estimating the difference between two regression coefficients corresponding to the treatment effects at individual time points.
2.2 Calculation of the time effect
To evaluate the time effect, the standard error, the corresponding confidence interval as well as the corresponding t-statistic- and p-value can be calculated for the difference between two regression coefficients [13,14]. First, the difference between the coefficients of interest is calculated, where these coefficients represent the treatment effect at different time points . The difference corresponds to the time effect and is given by
with
and where the head notation denotes an estimate. The corresponding standard error can be derived using Bienaymé’s theorem [15]:
The standard error and the estimated difference of the coefficients (time effect) can then be used to calculate the corresponding t-value: and via the z-score (which approximates the t-value for large samples [16]) the p-value can be calculated by
. Finally, a confidence interval (
) for the difference between the coefficients (time effect) can be determined:
In the following, both the simple linear RDD and the LMM-RDD for repeated measures (with unstructured variance-covariance matrix over time) are exemplified using the data from the isPO study. All analyses are exploratory and were done using R 4.4.0 [17]. Results were considered statistically significant at a significance level of α = 0.05 when p ≤ 0.05.
2.3 Data collection and analysis population
Between January 15, 2019 and March 31, 2021, N = 1,756 newly diagnosed cancer patients were enrolled in four isPO care networks (Cologne, Troisdorf, Mönchengladbach and Neuss) in North Rhine-Westphalia (NRW) and documented in the CAPSYS2020 computer assistance system developed specifically for the project. Each patient received individualized care and was therefore assigned to different treatment groups based on their needs as determined by their HADS scores (baseline, T1) at study enrolment. Patients with HADS scores ≤ 14 received a psychosocial intervention (control group) and patients with scores ≥ 15 received psychooncological, psychotherapeutical treatment (intervention group). After 4 (T2) and 12 months (T3), the HADS was measured again, and the treatment effect for the primary analysis in isPO was analysed separately for the two measurement time points (T2, T3) using standard RDD within a predefined bandwidth (HADS between 13–16) around the cut-off value [9]. Since a HADS result had to be available at least after 4 or 12 months for evaluation, 339 cases were excluded. This results in an N of 1,417 cases for the analysis population to estimate the treatment and time effects via LMM-RDD (Fig 3). All patients gave their written informed consent.
2.4 Ethics statement
The study was conducted in accordance with the Declaration of Helsinki and the Ethics Committee of the Medical Faculty of the University of Cologne has approved the isPO project (18–092). The isPO study is registered in the German Clinical Trials Registry (DRKS00015326). All participants have been informed about the aim of the study and about the voluntary nature of participation and have signed a written informed consent form for participation in the isPO care programme and data processing.
3. Results
3.1 Characteristics of the patients
N = 1,417 nFC-isPO patients had at least one HADS follow-up outcome available at 4 (T2) or 12 months (T3). Limited to the relevant bandwidth of 13–16 points on the HADS scale at baseline (T1), N = 247 cases remain for primary analysis. Around two thirds of them were female. 61.5% reported to live in a relationship and in the same household with their respective partners. Detailed information about main characteristics of the patients can be found in Table 1.
3.2 Results of the univariate and mixed model RDD
Table 2 shows the results of the univariate and linear mixed model RDD for the cases with complete follow-up data and for the case where either T2 and/or T3 are available. To illustrate the difference in results, depending on the data used, both modelling approaches were applied to cases with complete follow-up data to show that the result, the , is the same (
;
) regardless of whether the RDD is performed in its usual univariate or extended linear mixed model form. When the models are applied to patients with at least one follow-up visit (T2 and/or T3 available), univariate RDD estimates are based on (slightly) different samples of cases for each time point. In the univariate case for T2, the sign changes from negative to positive and the discontinuity changes direction. Although the sign of the
estimate changes, both estimates were very close to zero with wide confidence intervals. For T3, the
is almost twice as high as in the model applied to the cases with complete follow-up data, but again the estimate is very small with a wide confidence interval. The treatment effects estimated by the LMM-RDD are even higher than the effects calculated in the univariate models, but still with large confidence intervals. None of the results reached statistical significance.
3.3 Effect over time
Within the mixed model, it is now possible to calculate the time effect as the difference between the two treatment effects at the different time levels. The time effect with cases with complete follow-up data was with a standard error of 3.146 and a t-value of 0.183. Fig 4 is the graphical representation of the LMM-RDD in which the HADS value of T2 or T3 are not missing (see last row of Table 2). In order to obtain an overview of the data, the upper scatter plots show the relationship between the HADS values at time T1 and the respective outcome measurements at T2 and T3 over the entire value range of the HADS, whereby it can be seen that the respective outcome value increases with increasing initial value. Below is the representation of the LMM-RDD within the specified bandwidth with an
of
for T2 and an
of
for T3 (both not significant; s. Table 2). The table below all four plots contains the estimated time effect (
), which is also not significant (
).
4. Discussion
The RDD gives a promising alternative for drawing causal conclusions in case random group assignment is not feasible. Previously, it was not possibility to include repeated measures in the RDD. In order to close this gap, the simple linear RDD was extended to a repeated measure linear mixed model RDD by including the factor time and a random intercept (see equation 2). To demonstrated the advantages and differences of the linear mixed model RDD compared to the univariate RDD, data from the nFC-isPO study were reanalysed. First, only cases with complete follow-up data and, second all data (T2 and/or T3 are available) were considered. With the univariate method, a direct comparison between the time points on the basis of the calculated effects for cases with complete follow-up data is not possible. A valid 95% CI needs to account for the statistical dependence of both time points (repeated measures). Therefore, only a graphical interpretation or comparison of the RDD lines is possible. When using all available data, a graphical analysis is still possible, but even more limited, as the calculations of the treatment effect are based on (slightly) different samples. This implies that the database should be limited to cases with complete follow-up data or cases with valid primary endpoint, thus information is deliberately excluded from the analysis. With a linear mixed model a restriction to complete follow-up data is not required. When effects are estimated, particularly due to treatment and time, the correlation structure of repeated measures is accounted for [12].
4.1 Limitations and open issues
A major limitation of conducting a study in the medical field is the often limited sample size, which poses a problem when using an RDD. Simulation studies have shown that RDDs require a larger amount of data than RCTs in order to detect (moderate) treatment effect sizes and achieve comparable statistical power. The reason for this is that only units that are close to the threshold value can be compared – i.e., in the pre-specified range where quasi-randomisation can be assumed [18–20]. This might be one explanation for the overall insignificant results, including the rather wide confidence intervals of the analyses performed. Moreover, with regard to the application of the methods to the nFC-isPO study data, there was the limitation that the cases within the bandwidth were either too similar in terms of the individual need for psycho-oncological care or the burden above the defined threshold was too low to be able to demonstrate a relevant treatment effect.
This article extends a current method and demonstrates its feasibility with empirical data. For clarity, the analysis was limited to a random intercept model; however, further research could incorporate random slopes to gain valuable insights into individual differences over time. Further investigation is also needed to systematically compare the performance of the univariate and LMM-RDD approaches in a simulation study, considering various sample sizes, effect sizes, and missing data patterns.
5. Conclusion
Applying a linear mixed model (LMM) to a regression discontinuity design (RDD) with repeated measures is a powerful approach to analyse longitudinal data in case a RCT cannot be conducted. The presented combined model (LMM-RDD) enables the estimation of time effects, which has not been possible with the existing approaches, yet.
Acknowledgments
We thank all participating patients, all the clinical staff involved, and the cooperating health insurance companies for their support in the isPO project. We also thank all other consortium partners in the isPO project.
References
- 1. Bor J, Moscoe E, Mutevedzi P, Newell M-L, Bärnighausen T. Regression discontinuity designs in epidemiology: causal inference without randomized trials. Epidemiology. 2014;25(5):729–37. pmid:25061922
- 2. Hagemeier A, Samel C, Hellmich M. The regression discontinuity design: methods and implementation with a worked example in health services research. Z Evid Fortbild Qual Gesundhwes. 2022;172:71–7. pmid:35718728
- 3. Thistlethwaite DL, Campbell DT. Regression-discontinuity analysis: an alternative to the ex post facto experiment. J Educ Psychol. 1960;51(6):309–17.
- 4. Hilton Boon M, Craig P, Thomson H, Campbell M, Moore L. Regression discontinuity designs in health: a systematic review. Epidemiology. 2021;32(1):87–93. pmid:33196561
- 5. Cleophas TJ, Zwinderman AH, van Ouwerkerk B. Clinical research: a novel approach to the analysis of repeated measures. Am J Ther. 2012;19(1):e1-7. pmid:21048432
- 6. Snaith RP. The hospital anxiety and depression scale. Health Qual Life Outcomes. 2003;1:29.
- 7. Mitchell AJ, Meader N, Symonds P. Diagnostic validity of the Hospital Anxiety and Depression Scale (HADS) in cancer and palliative settings: a meta-analysis. J Affect Disord. 2010;126(3):335–48. pmid:20207007
- 8. Jenniches I, Lemmen C, Cwik JC, Kusch M, Labouvie H, Scholten N, et al. Evaluation of a complex integrated, cross-sectoral psycho-oncological care program (isPO): a mixed-methods study protocol. BMJ Open. 2020;10(3):e034141. pmid:32156765
- 9. Kusch M, Labouvie H, Schiewer V, Talalaev N, Cwik JC, Bussmann S, et al. Integrated, cross-sectoral psycho-oncology (isPO): a new form of care for newly diagnosed cancer patients in Germany. BMC Health Serv Res. 2022;22(1):543. pmid:35459202
- 10.
Kusch M, Labouvie H, Hein-Nau B. Klinische Psychoonkologie. Springer-Verlag; 2012.
- 11. Vodermaier A, Millman RD. Accuracy of the Hospital Anxiety and Depression Scale as a screening tool in cancer patients: a systematic review and meta-analysis. Support Care Cancer. 2011;19(12):1899–908. pmid:21898134
- 12. Detry MA, Ma Y. Analyzing repeated measurements using mixed models. JAMA. 2016;315(4):407–8.
- 13. Clogg CC, Petkova E, Haritou A. Statistical methods for comparing regression coefficients between models. Am J Sociol. 1995;100(5):1261–93.
- 14.
Altman D, et al. Statistics with confidence: confidence intervals and statistical guidelines. John Wiley & Sons; 2013.
- 15.
Loeve M. Probability theory. 3rd ed. Dover Publications; 2017.
- 16.
Hogg RV, Tanis EA, Zimmerman DL. Probability and statistical inference. Boston: Pearson; 2015.
- 17.
Team RC. A language and environment for statistical computing. R Foundation for Statistical Computing; 2021.
- 18.
Schochet PZ. Technical methods report: statistical power for regression discontinuity designs in education evaluations. NCEE 2008-4026. National Center for Education Evaluation and Regional Assistance; 2008.
- 19.
Jacob R, et al. A practical guide to regression discontinuity. MDRC; 2012.
- 20.
Cattaneo MD, Idrobo N, Titiunik R. A practical introduction to regression discontinuity designs: foundations. Cambridge University Press; 2020.