https://orcid.org/0000-0003-3319-1125
Abstract
Remote longitudinal studies are on the rise and promise to increase reach and reduce participation barriers in chronic disease research. However, maintaining long-term retention in these studies remains challenging. Early identification of participants with different patterns of long-term retention offers the opportunity for tailored survey adaptations. Using data from the online arm of the Swiss Multiple Sclerosis Registry (SMSR), we assessed sociodemographic, health-related, and daily-life related baseline variables against measures of long-term retention in the follow-up surveys through multivariable logistic regressions and unsupervised clustering analyses. We further explored follow-up survey completion measures against survey requirements to inform future survey designs. Our analysis included data from 1,757 participants who completed a median of 4 (IQR 2–8) follow-up surveys after baseline with a maximum of 13 possible surveys. Survey start year, age, citizenship, MS type, symptom burden and independent driving were significant predictors of long-term retention at baseline. Three clusters of participants emerged, with no differences in long-term retention outcomes revealed across the clusters. Exploratory assessments of follow-up surveys suggest possible trends in increased survey complexity with lower rates of survey completion. Our findings offer insights into characteristics associated with long-term retention in remote longitudinal studies, yet they also highlight the possible influence of various unexplored factors on retention outcomes. Future studies should incorporate additional objective measures that reflect participants’ individual contexts to understand their ability to remain engaged long-term and inform survey adaptations accordingly.
Figures
Citation: Daniore P, Yan C, Stanikic M, Iaquinto S, Ammann S, Kamm CP, et al. (2024) Real-world patterns in remote longitudinal study participation: A study of the Swiss Multiple Sclerosis Registry. PLOS Digit Health 3(11): e0000645. https://doi.org/10.1371/journal.pdig.0000645
Editor: Cleva Villanueva, Instituto Politécnico Nacional Escuela Superior de Medicina: Instituto Politecnico Nacional Escuela Superior de Medicina, MEXICO
Received: June 7, 2024; Accepted: September 17, 2024; Published: November 6, 2024
Copyright: © 2024 Daniore et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The underlying data for this analysis are human research participant data and in combination potentially identifiable. The data that supports the findings of this study are available upon reasonable request. Requests should be directed to the Swiss Multiple Sclerosis Registry at the University of Zurich; Epidemiology, Biostatistics & Prevention Institute; Hirschengraben 84, CH-8001 Zurich (ms-register@ebpi.uzh.ch).
Funding: The author(s) received no specific funding for this work.
Competing interests: CPK has received honoraria for lectures as well as research support from Biogen, Novartis, Almirall, Teva, Merck, Sanofi Genzyme, Roche, Janssen, Eli Lilly, Celgene and the Swiss MS Society (SMSG). MS reports employment by Roche branch in Serbia, Roche d.o.o., from February 2019 to February 2020. CZ receives financial support for speaking, educational, research or travel grants from Abbvie, Almirall, Biogen Idec, Celgene, Sanofi, Merck, Novartis, Teva Pharma, Roche.
Introduction
Multiple sclerosis (MS) is a chronic neurodegenerative autoimmune condition characterized by a wide range of symptom patterns and disease courses that remain incompletely understood [1]. Longitudinal epidemiological studies can offer a more comprehensive understanding of the associations between risk factors and certain symptom patterns or disease courses in people living with MS (pwMS), enabling more tailored treatment approaches [2]. Remote longitudinal studies, or longitudinal epidemiological studies delivered entirely online, hold promise for further advancing MS research by increasing reach and reducing participation barriers for pwMS related to time, accessibility and in-person presence [3–5]. In turn, studies can benefit from higher external validity, which enhances their real-world application.[6]
Remote longitudinal studies have gained considerable traction in chronic disease research due to their efficient and innovative approaches to engage participants [7,8]. Nevertheless, such studies are still prone to low long-term retention, which increases the risk of selection bias in the case of a systematic difference between participants lost to follow-up and those left in the study [9,10]. The resulting methodological challenges in such studies [11,12] call for the identification of participant subgroups associated with varying levels of long-term retention in order to inform novel survey designs [13]. In MS research, recent digital health intervention studies suggest that subgroups defined by sociodemographic characteristics, including age and gender, increased symptom burden from disease progression, and daily life responsibilities such as employment, can influence study participation [2,3,14,15].
However, there is currently limited experience in identifying subgroups of pwMS with varying retention levels in remote longitudinal studies. This highlights the need to explore new methods for identifying subgroups that can help ensure long-term retention in such studies [16]. In turn, these findings can guide the design of future online surveys for MS research by allowing for tailored study designs specific to different subgroups.
Aims
We aimed to apply well-established statistical methods in health research to explore (1) baseline participant characteristics associated with long-term retention in the online arm of the Swiss Multiple Sclerosis Registry (SMSR) and (2) the influence of survey design elements on participants’ survey completion. Through these, we aimed to demonstrate different analyses that can be performed at baseline to identify population subgroups with distinct response patterns and inform survey adaptations accordingly. The overarching goal of this study was to inform future remote longitudinal study design to facilitate long-term retention in MS as well as other chronic disease research.
Methods
Setting
This study uses data from participants registered in the SMSR, an observational study initiated in 2016, with ongoing enrollment. The SMSR recruits adult pwMS residing or receiving treatment in Switzerland. Participation in the SMSR is voluntary and requires completion of a signed consent form along with a confirmation of MS or clinically isolated syndrome (CIS) diagnosis. Upon enrollment, all participants complete a baseline questionnaire and continue their participation by providing self-reported data through follow-up questionnaires provided approximately twice a year, accessible both online and in paper format. All participants who fill out a baseline questionnaire receive an invitation to fill out follow-up and thematic surveys. The SMSR was approved by the Ethics Committee of the Canton of Zurich (PB-2016–00,894; BASEC-NR 2019–01,027). Additional information about the SMSR can be found in prior publications [17,18].
This study refers to data from the online format of the baseline survey, along with all online follow-up and thematic surveys from the SMSR, hereafter referred to as follow-up surveys. Participants receive regular invitations and reminders to fill out the questionnaires, and are given the option to pause and recontinue the completion of the surveys on their own schedules. The first follow-up survey (FU-6) was released in June 2017, and the last follow-up survey considered in this analysis (FU-48) was released in October 2022. We refer to data from June 2016 to May 2023. Participants with complete baseline sociodemographic information, including age, sex, living situation, education, citizenship, and partnership are included in the study analysis (S1 Fig).
Outcome variables
We developed two primary outcome variables from responses to online follow-up surveys. The variables were informed by previous studies [19–21] and were used to create a binary classification of online participation in the SMSR. The first primary outcome variable categorized participants under high or low retention based on whether they completed at least one follow-up survey each calendar year, after participating in the SMSR for two years. We selected a two-year minimum for participation to ensure an adequate observational period for individuals with chronic conditions [22]. The second primary outcome variable categorized participants under lower or higher response than median based on whether the number of completed surveys was equal to or above, or below the median number of surveys completed by participants who joined the SMSR in the same year. Sensitivity analyses were done using a variation of the first outcome variable. This variation was based on whether the participants completed at least one follow-up survey each calendar year until the end of the study period (2022 or 2023), to assess trends in active participation from the start to the end of the survey period.
Variables of interest at baseline
This study evaluated three categories of baseline data identified in previous studies on MS and chronic diseases [23–28] that can contribute to different patterns in retention, namely sociodemographic, health-related and daily-life variables. The sociodemographic variables include age, sex, Swiss language regions, whether participants have children, whether they received a university degree, partnership status, and whether they have a Swiss citizenship. The health-related variables include variables that define the health status and indicators of disease development at baseline. These include MS type, years since MS diagnosis, whether participants have relatives with MS, symptoms of fatigue, symptoms of paraesthesia, symptoms of depression and self-reported disability status scale [29] (SRDSS). The daily-life variables provide contextual indicators for partaking in daily activities. These include whether participants independently drive a car, use public transport, employment status and whether they received assistance to fill out the surveys. Sensitivity analyses were performed to assess whether the timing of participants receiving disease-modifying therapies, or not receiving them at all, influenced the main analyses.
Statistical analyses
In this study, we describe participant characteristics and primary outcomes of interest using counts and percentages, as well as median and interquartile range (IQR). To identify predictors for long-term retention at baseline, univariable and multivariable logistic regression analyses were conducted for both primary outcomes. All a priori selected baseline variables were included in these analyses. Additional interactions were explored and kept if they decreased the Akaike information criterion (AIC) score by 2 points or more [30–32]. Events per variable (EPV) were calculated to ensure a minimum of 10 EPV for sample size considerations in logistic regression analyses [33]. Multicollinearity was evaluated using the Variance Inflation Factor (VIF), with VIF values above 5 indicating collinearity [34]. Finally, variables with perfect separation were recategorized into fewer categories to balance the responses [35]. Since 155 participants had partially missing data, complete case analyses as well as multiple imputed data derived from 5 multiple imputed datasets were analysed for comparison (S1 Fig) [36].
We conducted an exploratory k-means clustering analysis to identify possible participant subgroups at baseline with varying long-term retention. The clustering analysis was conducted in two steps: (1) unsupervised clustering using all baseline variables for participants with complete data used in the primary analysis, and (2) associating the identified clusters of participants with their respective long-term retention outcomes. For the clustering analysis, categorical variables were converted into binary variables for each category level, and continuous variables were standardized to minimize the impact of scale differences on model fitting. The elbow method was used to determine the optimal number of clusters (S2 Fig). We finally conducted exploratory descriptive analyses on survey design factors, including question length and Flesch-Kinkaid scores [37] for text readability, and their associations with follow-up survey completion. Survey completion is defined by responses to the first and last survey questions, as well as responses to an equal or greater number of questions compared to the median number of responses from all survey participants.
All analyses were conducted in R (version 4.2.2) and Python (version 3.12.0).
Results
Study population and response
A total of 1,757 participants were included in the analyses (flowchart available in S1 Fig). Sociodemographic, health, and daily life related characteristics of the participants for both primary outcomes of interest are reported in Table 1. Overall, 671 (38%) participants filled out at least one survey per year during the first two years of study participation and 948 (54%) participants filled out an equal number or more surveys compared to those who completed the baseline questionnaire in the same starting year. During the study period, 41 (2.3%) left the SMSR by explicitly deregistering or dying (S1 Table). Median age [IQR] for participants was 45 [36–54] years. Most participants were female (n = 1,254, 71%), resided in the German/Romansh language region of Switzerland (n = 1,357, 77%), had relapsing-remitting MS (n = 1,257, 72%) and were employed at baseline (n = 1,173, 67%). Characteristics of participants for the sensitivity analysis with an alternate outcome variable are available in S2 Table.
Overall, participants completed a median of 4 (IQR 2–8) follow-up surveys after baseline, with a maximum of 13 surveys, which corresponds to the total number of follow-up surveys in the study period (Fig 1).
Baseline predictors for long-term participation: yearly retention
Multivariable regression analyses for the complete case data (Fig 2 and S3 Table) reveal similar results to univariable regression analyses for the complete case data and multivariable regression analyses for multiple-imputed data (S3 Table). Compared to participants who joined the MS Registry in 2016, participants were less likely to actively participate on a yearly basis if they filled out the baseline questionnaire between 2017 and 2019 (OR 0.77, 95% CI [0.61, 0.97]) or from 2020 onwards (OR 0.40, 95% CI [0.28, 0.57]). For the sociodemographic characteristics, compared to participants aged between 18–35 years old at baseline, participants in the 36–45 year old age group (OR 1.60, 95% CI [1.16, 2.21]) and 46–55 year old age group (OR 1.58, 95% CI [1.13, 2.20]) were more likely to actively participate on a yearly basis, while Swiss citizens revealed more likely to actively participate on a yearly basis that non-Swiss citizens (OR 1.60, 95% CI [1.11, 2.34]). For the health-related characteristics, compared to participants with relapsing remitting MS, participants with the more advanced secondary progressive MS or in transition were more likely to actively participate on a yearly basis (OR 1.57, 95% CI [1.08, 2.28]). Sensitivity analyses that included the timing variable for participants receiving disease-modifying therapies showed no differences in the results (S4 Table).
The three identified clusters of participants (Figs 3 and S3 and S5 Table) also revealed subgroups based on age groups, with one subgroup (cluster 1, n = 456) consisting primarily of participants aged 56+ (48%), another subgroup of participants (cluster 3, n = 608) between the ages of 36–55 (72%), and the last subgroup (cluster 2, n = 538) with participants between the ages of 18–35 (52%). Cluster 1 primarily comprises individuals with the secondary progressive or in transition form of MS (40%), while clusters 2 and 3 (89% and 86%, respectively) predominantly consist of individuals with the less advanced relapsing remitting MS. Counts of yearly high retention outcomes were 41% (95% CI [36%,45%]) for cluster 1, 37% (95% CI [33%,41%]) for cluster 2 and 39% (95% CI [35%,43%]) for cluster 3. As such, differences in long-term retention between the clusters were not observed (S6 Table).
The bar chart illustrates the primary contributing variables to each cluster for the pre-defined characteristics represented as within-cluster percentages. Overall, cluster 1 is mostly comprised of participants in the older age groups, who have a more advanced form of MS, have higher symptom and disability burden, and are unemployed; cluster 2 is mostly comprised of participants from the youngest age groups, who are not in a partnership, do not have children, have a less advanced form of MS, have low symptom and disability burden and are unemployed; and cluster 3 is mostly comprised of participants in the middle age-group, who are in a partnership, have children and have similar health-related and daily-life characteristics as participants in cluster 2. These results are displayed as Sankey plots in S3 Fig.
Baseline predictors for long-term participation: starting year-based retention
Multivariable regression analyses for the complete case data (Fig 4 and S7 Table) revealed similar results to univariable regression analyses for the complete case data and multivariable regression analyses for multiple-imputed data (S7 Table). For the sociodemographic characteristics, compared to participants aged between 18–35 years old at baseline, participants in the 36–45 year old age group (OR 2.01, 95% CI [1.48, 2.74]), in the 46–55 year old age group (OR 2.11, 95% CI [1.53, 2.92]), in the 56–65 year old age group (OR 2.17, 95% CI [1.44, 3.30]) and in the 66 and over age group (OR 2.04, 95% CI [1.08, 3.87]) were more likely to participate than participants of the same starting year. For the health-related characteristics, participants with symptoms of fatigue were less likely to participate than those without symptoms of fatigue (OR 0.75, 95% CI [0.57, 0.99]). For the daily life-related characteristics, participants who regularly drove a car at baseline were more likely to participate than participants who did not drive a car (OR 1.38, 95% CI [1.04, 1.82]). Sensitivity analyses with an alternate outcome variable (S8 Table) further highlight the relevance of these characteristics on long-term retention. Sensitivity analyses that included the timing variable for participants receiving disease-modifying therapies showed no differences in the results (S9 Table).
The three identified clusters of participants (Figs 3 and S3 and S5 Table) also revealed subgroups based on age groups, with one subgroup consisting primarily of participants aged 56+ (cluster 1, 48%), another cluster of participants between the ages of 36–55 (cluster 3, 72%), and the last cluster with participants between the ages of 18–35 (cluster 2, 52%). Cluster 1 includes participants with a high burden of fatigue symptoms (79%) and a higher prevalence of individuals who do not drive (35%) or use public transport (28%) within the cluster compared to clusters 2 and 3. Counts of high starting year-based retention outcomes were 52% (95% CI [48%, 57%]) for cluster 1, 55% (95% CI [51%, 59%]) for cluster 2 and 57% (95% CI [52%, 61%]) for cluster 3. As such, differences in long-term retention between the clusters were not observed (S6 Table).
Complexity of follow-up surveys and impact on survey non-completion
Overall, the median length of the follow-up surveys was 29,468 characters (IQR: 17,712–36,673), 3,807 words (IQR: 2,271–4,918) and 70 questions (IQR: 59–78) (survey details and measures presented in S10 Table). Median readability of the surveys based on the Flesch-Kinkaid score was around the 7th grade school level (i.e., ages 12–13) (IQR: 6.2–7.6) (S4 Fig). The median percentage of participants who completed the surveys among those who started the surveys was 51% (IQR: 49–55%, Fig 5). A negative trend was observed between the completion of surveys for these participants and the total number of characters in a survey (correlation = -0.44), or the total number of questions in a survey (correlation = -0.33). However, both correlations were not statistically significant. The median percentage of participants who completed the surveys among those who were invited to fill out the surveys was 25% (IQR: 24–31%, Fig 6). A negative trend was also observed between the completion of surveys for these participants and the total number of characters in a survey (correlation = -0.36), or the total number of questions in a survey (correlation = -0.51). However, both correlations were also not statistically significant.
Survey complexity, displayed by number of characters (left) and number of questions (right) in the survey, vs percentage of participants who completed the survey among those who started the survey.
Survey complexity, displayed by number of characters (left) and number of questions (right) in the survey, vs percentage of participants who completed the survey among those who were invited to complete the survey.
Discussion
This study reports findings on long-term participant retention using data from the online arm of the SMSR, a real-world longitudinal study on pwMS in Switzerland [17,18]. Our findings reveal that start year, age, citizenship, MS type, symptom burden and whether participants independently drive a car on a regular basis are associated with long-term retention at baseline. These findings were consistent for two different primary outcomes of interest reflective of long-term retention, with associations in the same direction of effect among variables within the three baseline categories for both analyses. Exploratory clustering of the baseline data revealed three distinct subgroups, however no differences were found in long-term retention among the clusters. Exploratory assessments of follow-up survey design characteristics and survey completion suggest the potential for adjusting survey components to enhance overall long-term survey participation.
Our analysis revealed that participants experiencing higher fatigue symptom burden and receiving disability insurance at baseline were less likely to be retained in the long-term. Participants with progressive MS and who drive a car independently showed a greater likelihood of long-term retention. These findings suggest that participants with progressive MS who retain independence may be more motivated to participate in research, likely due to reduced work commitments at this stage, as observed in a previous SMSR study.[38] Yet, these participants may be at risk of lower retention due to higher disability burden. Recent digital health interventions [2,39,40] and longitudinal surveys [41–43] assessing pwMS and other chronic diseases report similar lower retention for individuals with more pronounced symptom and disability severity. Notably, a recent analysis of a SMSR follow-up survey identified a subgroup of participants with low-frequency adoption of health technology with higher disability and disease burden, suggesting they may be more vulnerable to digital exclusion.[44] These individuals may be intrinsically motivated to join a study, likely due to the progression of MS, however could require adaptations to participate in the long-term [4,45,46]. Some approaches to facilitate their inclusion in remote longitudinal studies include to provide participants the option to receive support to fill out surveys, which is already offered by the SMSR, invest additional communication efforts to engage these individuals in data collection and offer timely assistance, and adapt formatting, such as text size, to enhance survey accessibility [45,46]. Nevertheless, there remains a distinct need for future studies to develop solutions that accommodate these individuals and address these disparities effectively.
Despite our study’s large sample size, with available data spanning up to thirteen surveys over an eight-year follow-up period, our findings do not offer major new insights on predictors of long-term retention. The limited selection of variables used to define the clusters may have contributed to these outcomes. Nevertheless, the variables we assessed extend beyond those in similar studies on retention in pwMS, with the most comprehensive study to our knowledge being a systematic review, which found that assessments are limited to patient-related (sociodemographic data, MS type, years since MS diagnosis, disability level) and study design variables [2]. In the context of MS, considering additional contextual variables related to individuals’ life and disease course could provide additional insights on long-term retention. For example, whether participants receive long-term care could offer insights into their motivation to participate or their access to support for study participation. While we did not assess this due to the small size of this group in our study, it could be valuable to explore in future research. There is also a growing body of literature highlighting the need to consider additional contextual and behavioral variables that extend beyond typical assessments of retention in chronic disease research [47–49]. For instance, a study on retention indicators across multiple remote digital health studies found that contextual factors, such as clinical referral to join a study, helped identify participant subgroups based on retention [50]. Another study found that contextualizing data of participants’ mental health with the time required to respond to and complete surveys in a one year span contributed in identifying engagement subgroups [51]. An in-person study also identified behavioral and psychological predictors of loss to follow-up that go beyond sociodemographic characteristics [10]. As such, survey response patterns may be influenced by various unexplored factors. Therefore, instead of focusing solely on the analysis methods used to assess participation, attention should be directed toward developing objective metrics to predict participation based on new participant profiles, contexts, and behaviors.
In designing long-term longitudinal surveys, particularly when delivered online with limited participant-researcher interactions, it is crucial to assess factors that may increase participant burden to prevent counterproductive effects. Exploratory assessments from our study suggest a possible trend, although not statistically significant, indicating that increased survey complexity may be associated with lower survey completion. This finding is supported by the earlier mentioned systematic review of self-management interventions for pwMS, which reported a significant negative association between attrition and the length of the intervention [2]. A recent meta-analysis further highlights that design factors, such as study duration and survey length, can adversely affect participation and the usefulness of retention strategies [52]. In response, adaptive survey strategies are encouraged in the literature, where retention strategies are reviewed periodically by assessing survey non-response and collecting user feedback [53,54]. These strategies can then be modified accordingly, using engagement techniques, such as gamification or adjusting the timing and approach to data collection, to align with participants’ preferences [55,56]. However, there is limited information available to apply such methodologies due to the scarcity of quantitative reports on long-term retention, which is likely a result of longitudinal studies mainly reporting missing data based on item non-response rather than long-term response and follow-up rates [57]. As remote data collection becomes more common in longitudinal studies, they continue to pose a higher risk of attrition, especially among individuals with chronic conditions who require lower-burden tasks. It is, therefore, crucial to document and apply shared experiences on methodological adaptations for improving long-term retention effectively.
Limitations and future research
This study provides valuable and detailed insights into patterns of long-term retention in a large-scale, remote longitudinal study by exploring participant subgroups and study design elements. However, this study presents some limitations. Firstly, given the limited number of data collection points available after baseline, binary outcome variables for low and high long-term retention were derived based on approaches suggested in the literature on longitudinal studies. These derived measures may not be fully reflective of long-term retention in remote longitudinal studies. Secondly, reasons for non-response were not collected, which may limit the contextualization of our study’s findings. Thirdly, specific survey design characteristics, such as the time of day when surveys were responded to, were unavailable, which limits the ability to assess possible contextual factors for survey non-response. Nevertheless, the complexity measures used in this study provide an initial context to assess possible survey design elements that may affect survey non-response.
Conclusion
Our assessment of participants in the online arm of the Swiss Multiple Sclerosis Registry (SMSR) provides new insights into the application of well-recognized statistical methods to identify predictors of long-term retention in remote longitudinal studies. However, our findings also underscore the current difficulties in identifying participant subgroups using the limited measures generally employed in chronic disease research to assess long-term retention in studies. As our findings suggest a link between survey completion and complexity, it is crucial to identify participant subgroups and tailor survey elements to accommodate their characteristics and preferences to facilitate long-term retention. In future remote longitudinal studies, integrating additional contextual and psychological measures into surveys, and assessing them against survey response outcomes may help define these subgroups. Researchers can then adapt survey elements using various engagement strategies in response to these observations, ideally iterating with regular revisions until an optimal strategy is identified.
Supporting information
S4 Fig. Flesch-Kinkaid score and impact on non-completion, with filtered out survey completers who answered less than the median of survey questions.
https://doi.org/10.1371/journal.pdig.0000645.s004
(DOCX)
S1 Table. Yearly retention and starting year-based retention outcomes for participants who deregistered from the SMSR or died during the study.
https://doi.org/10.1371/journal.pdig.0000645.s005
(DOCX)
S2 Table. Sensitivity analysis with alternate outcome variable, participant characteristics, yearly retention until the end of the study period.
https://doi.org/10.1371/journal.pdig.0000645.s006
(DOCX)
S3 Table. Univariate and multivariable logistic regression, yearly retention.
https://doi.org/10.1371/journal.pdig.0000645.s007
(DOCX)
S4 Table. Sensitivity analysis with therapy timing variable, univariate and multivariable logistic regression, yearly retention.
https://doi.org/10.1371/journal.pdig.0000645.s008
(DOCX)
S5 Table. Clustering task, baseline participants.
https://doi.org/10.1371/journal.pdig.0000645.s009
(DOCX)
S6 Table. Retention outcomes based on identified clusters.
https://doi.org/10.1371/journal.pdig.0000645.s010
(DOCX)
S7 Table. Univariate and multivariable logistic regression, starting year-based retention.
https://doi.org/10.1371/journal.pdig.0000645.s011
(DOCX)
S8 Table. Sensitivity analysis with alternate outcome variable, univariate and multivariable logistic regression, yearly retention until the end of the study period.
https://doi.org/10.1371/journal.pdig.0000645.s012
(DOCX)
S9 Table. Sensitivity analysis with therapy timing variable, univariate and multivariable logistic regression, starting year-based retention.
https://doi.org/10.1371/journal.pdig.0000645.s013
(DOCX)
S10 Table. Descriptions of the complexity variables for SMSR surveys.
https://doi.org/10.1371/journal.pdig.0000645.s014
(DOCX)
References
- 1. Gironi M, Arnò C, Comi G, Penton-Rol G, Furlan R. Chapter 4 ‐ Multiple Sclerosis and Neurodegenerative Diseases. In: Boraschi D, Penton-Rol G, editors. Immune Rebalancing Academic Press; 2016. p. 63–84.
- 2. Arafah AM, Bouchard V, Mayo NE. Enrolling and keeping participants in multiple sclerosis self-management interventions: a systematic review and meta-analysis. Clin Rehabil SAGE Publications Ltd STM; 2017 Jun 1;31(6):809–823. doi: https://doi.org/10.1177/0269215516658338 pmid:27401492
- 3. Jongen PJ, Kremer IEH, Hristodorova E, Evers SMAA, Kool A, Noort EM, Hiligsmann M. Adherence to Web-Based Self-Assessments in Long-Term Direct-to-Patient Research: Two-Year Study of Multiple Sclerosis Patients. J Med Internet Res 2017 Jul 21;19(7):e6729. pmid:28733272
- 4. Daniore P, Nittas V, von Wyl V. Enrollment and Retention of Participants in Remote Digital Health Studies: Scoping Review and Framework Proposal. J Med Internet Res 2022 Sep 9;24(9):e39910. pmid:36083626
- 5. Giovannoni G a, Rhoades RW b. Individualizing treatment goals and interventions for people with MS. Curr Opin Neurol 2012 Feb; pmid:22398661
- 6. Tofthagen C. Threats to Validity in Retrospective Studies. J Adv Pract Oncol 2012;3(3):181–183. pmid:25031944
- 7. Peyroteo M, Ferreira IA, Elvas LB, Ferreira JC, Lapão LV. Remote Monitoring Systems for Patients With Chronic Diseases in Primary Health Care: Systematic Review. JMIR MHealth UHealth 2021 Dec 21;9(12):e28285. pmid:34932000
- 8. Inan OT, Tenaerts P, Prindiville SA, Reynolds HR, Dizon DS, Cooper-Arnold K, Turakhia M, Pletcher MJ, Preston KL, Krumholz HM, Marlin BM, Mandl KD, Klasnja P, Spring B, Iturriaga E, Campo R, Desvigne-Nickens P, Rosenberg Y, Steinhubl SR, Califf RM. Digitizing clinical trials. Npj Digit Med Nature Publishing Group; 2020 Jul 31;3(1):1–7. pmid:32821856
- 9. Abshire M, Dinglas VD, Cajita MIA, Eakin MN, Needham DM, Himmelfarb CD. Participant retention practices in longitudinal clinical research studies with high retention rates. BMC Med Res Methodol 2017 Feb 20;17(1):30. pmid:28219336
- 10. Wees DA van, Daas C den, Kretzschmar MEE, Heijne JCM. Who drops out and when? Predictors of non-response and loss to follow-up in a longitudinal cohort study among STI clinic visitors. PLOS ONE Public Library of Science; 2019 Jun 19;14(6):e0218658. pmid:31216341
- 11. Kahn KL, Liu H, Adams JL, Chen W-P, Tisnado DM, Carlisle DM, Hays RD, Mangione CM, Damberg CL. Methodological Challenges Associated with Patient Responses to Follow-up Longitudinal Surveys Regarding Quality of Care. Health Serv Res 2003;38(6p1):1579–1598. pmid:14727789
- 12. Chatfield MD, Brayne CE, Matthews FE. A systematic literature review of attrition between waves in longitudinal studies in the elderly shows a consistent pattern of dropout between differing studies. J Clin Epidemiol 2005 Jan 1;58(1):13–19. pmid:15649666
- 13. Nielsen LK, King M, Möller S, Jarden M, Andersen CL, Frederiksen H, Gregersen H, Klostergaard A, Steffensen MS, Pedersen PT, Hinge M, Frederiksen M, Jensen BA, Helleberg C, Mylin AK, Abildgaard N. Strategies to improve patient-reported outcome completion rates in longitudinal studies. Qual Life Res 2020 Feb 1;29(2):335–346. pmid:31549365
- 14. Claflin S, Klekociuk S, Campbell J, Taylor B. Reasons for non-completion of a massive open online course about multiple sclerosis: A mixed methods study. Mult Scler Relat Disord 2022 Nov 1;67:104092. pmid:35977440
- 15. Bevens W, Weiland T, Gray K, Jelinek G, Neate S, Simpson-Yap S. Attrition Within Digital Health Interventions for People With Multiple Sclerosis: Systematic Review and Meta-analysis. J Med Internet Res 2022 Feb 9;24(2):e27735. pmid:35138262
- 16. Osborne LA, Middleton RM, Jones KH, Ford DV, Noble JG. Desirability and expectations of the UK MS Register: Views of people with MS. Int J Med Inf 2013 Nov 1;82(11):1104–1110. pmid:24021929
- 17. Puhan MA, Steinemann N, Kamm CP, Müller S, Kuhle J, Kurmann R, Calabrese P, Kesselring J, von Wyl V, Smsr SMSR. A digitally facilitated citizen-science driven approach accelerates participant recruitment and increases study population diversity. Swiss Med Wkly Switzerland; 2018;148:w14623. pmid:29767828
- 18. Steinemann N, Kuhle J, Calabrese P, Kesselring J, Disanto G, Merkler D, Pot C, Ajdacic-Gross V, Rodgers S, Puhan MA, von Wyl V, Anderseck B, Calabrese P, Chan A, Disanto G, Engelhardt B, Gobbi C, Häussler R, Kamm CP, Kägi S, Kesselring J, Kuhle J, Kurmann R, Lotter C, Luyckx K, Merkler D, Monin P, Müller S, Nedeltchev K, Pot C, Puhan MA, Rapold I, Salmen A, Schippling S, Vaney C, von Wyl V, the Swiss Multiple Sclerosis Registry. The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research. BMC Neurol 2018 Aug 13;18(1):111. pmid:30103695
- 19. Kocar S, Biddle N. The power of online panel paradata to predict unit nonresponse and voluntary attrition in a longitudinal design. Qual Quant 2023 Apr 1;57(2):1055–1078. pmid:35493336
- 20. Radler BT, Ryff CD. Who Participates? Accounting for Longitudinal Retention in the MIDUS National Study of Health and Well-Being. J Aging Health SAGE Publications Inc; 2010 Apr 1;22(3):307–331. pmid:20103686
- 21. Chan Y-FY, Wang P, Rogers L, Tignor N, Zweig M, Hershman SG, Genes N, Scott ER, Krock E, Badgeley M, Edgar R, Violante S, Wright R, Powell CA, Dudley JT, Schadt EE. The Asthma Mobile Health Study, a large-scale clinical observational study using ResearchKit. Nat Biotechnol 2017 Apr;35(4):354–362. pmid:28288104
- 22. Goodman RA, Posner SF, Huang ES, Parekh AK, Koh HK. Defining and Measuring Chronic Conditions: Imperatives for Research, Policy, Program, and Practice. Prev Chronic Dis 2013 Apr 25;10:E66. pmid:23618546
- 23. Newland P, Oliver B, Newland JM, Thomas FP. Testing Feasibility of a Mobile Application to Monitor Fatigue in People With Multiple Sclerosis. J Neurosci Nurs 2019 Dec;51(6):331. pmid:31688283
- 24. Kok RN, Beekman ATF, Cuijpers P, van Straten A. Adherence to a web-based pre-treatment for phobias in outpatient clinics. Internet Interv 2017 Sep 1;9:38–45. pmid:30135835
- 25. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, Berger T, Botella C, Breton JM, Carlbring P, Christensen H, Graaf E de, Griffiths K, Donker T, Farrer L, Huibers MJH, Lenndin J, Mackinnon A, Meyer B, Moritz S, Riper H, Spek V, Vernmark K, Cuijpers P. Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’ meta-analysis. Psychol Med 2015 Oct;45(13):2717–2726. pmid:25881626
- 26. Bamer AM, Johnson KL, Amtmann DA, Kraft GH. Beyond fatigue: Assessing variables associated with sleep problems and use of sleep medications in multiple sclerosis. Clin Epidemiol Dove Medical Press; 2010 Aug 9;2:99–106. pmid:20838467
- 27. Huda S, Cavey A, Izat A, Mattison P, Boggild M, Palace J. Nurse led telephone assessment of expanded disability status scale assessment in MS patients at high levels of disability. J Neurol Sci 2016 Mar 15;362:66–68. pmid:26944120
- 28. Griffin N, Kehoe M. A questionnaire study to explore the views of people with multiple sclerosis of using smartphone technology for health care purposes. Disabil Rehabil Taylor & Francis; 2018 Jun 5;40(12):1434–1442. pmid:28322588
- 29. Kaufmann M, Salmen A, Barin L, Puhan MA, Calabrese P, Kamm CP, Gobbi C, Kuhle J, Manjaly Z-M, Ajdacic-Gross V, Schafroth S, Bottignole B, Ammann S, Zecca C, D’Souza M, von Wyl V, for the SMSR (SMSR). Development and validation of the self-reported disability status scale (SRDSS) to estimate EDSS-categories. Mult Scler Relat Disord 2020;42. pmid:32371376
- 30. Bozdogan H. Model selection and Akaike’s Information Criterion (AIC): The general theory and its analytical extensions. Psychometrika 1987 Sep 1;52(3):345–370.
- 31.
Faraway JJ. Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. Boca Raton, Fla.: Chapman & Hall/CRC; 2006. ISBN:978-1-58488-424-8
- 32. Marck CH, De Livera AM, Brown CR, Neate SL, Taylor KL, Weiland TJ, Hadgkiss EJ, Jelinek GA. Health outcomes and adherence to a healthy lifestyle after a multimodal intervention in people with multiple sclerosis: Three year follow-up. Aktas O, editor. PLOS ONE 2018 May 23;13(5):e0197759. pmid:29791509
- 33. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996 Dec 1;49(12):1373–1379. pmid:8970487
- 34. Dormann CF, Elith J, Bacher S, Buchmann C, Carl G, Carré G, Marquéz JRG, Gruber B, Lafourcade B, Leitão PJ, Münkemüller T, McClean C, Osborne PE, Reineking B, Schröder B, Skidmore AK, Zurell D, Lautenbach S. Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography 2013;36(1):27–46.
- 35. Heinze G, Schemper M. A solution to the problem of separation in logistic regression. Stat Med 2002;21(16):2409–2419. pmid:12210625
- 36. Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, Wood AM, Carpenter JR. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ British Medical Journal Publishing Group; 2009 Jun 29;338:b2393. pmid:19564179
- 37. Jindal P, MacDermid JC. Assessing Reading Levels of Health Information: Uses and Limitations of Flesch Formula. Educ Health 2017 Apr;30(1):84. pmid:28707643
- 38. Sieber C, Chiavi D, Haag C, Kaufmann M, Horn AB, Dressel H, Zecca C, Calabrese P, Pot C, Kamm CP, Wyl V von, Registry SMS. Electronic Health Diary Campaigns to Complement Longitudinal Assessments in Persons With Multiple Sclerosis: Nested Observational Study. JMIR MHealth UHealth 2022 Oct 5;10(10):e38709. pmid:36197713
- 39. Bentley CL, Powell L, Potter S, Parker J, Mountain GA, Bartlett YK, Farwer J, O’Connor C, Burns J, Cresswell RL, Dunn HD, Hawley MS. The Use of a Smartphone App and an Activity Tracker to Promote Physical Activity in the Management of Chronic Obstructive Pulmonary Disease: Randomized Controlled Feasibility Study. JMIR MHealth UHealth 2020 Jun 3;8(6):e16203. pmid:32490838
- 40. Meyerowitz-Katz G, Ravi S, Arnolda L, Feng X, Maberly G, Astell-Burt T. Rates of Attrition and Dropout in App-Based Interventions for Chronic Disease: Systematic Review and Meta-Analysis. J Med Internet Res 2020 Sep 29;22(9):e20283. pmid:32990635
- 41. Boeschoten RE, Nieuwenhuis MM, van Oppen P, Uitdehaag BMJ, Polman CH, Collette EH, Cuijpers P, Beekman ATF, Dekker J. Feasibility and outcome of a web-based self-help intervention for depressive symptoms in patients with multiple sclerosis: A pilot study. J Neurol Sci 2012 Apr 15;315(1):104–109. pmid:22133479
- 42. Kessler D, McCutcheon T, Rajachandrakumar R, Lees J, Deyell T, Levy M, Liddy C. Understanding barriers to participation in group chronic disease self-management (CDSM) programs: A scoping review. Patient Educ Couns 2023 Oct 1;115:107885. pmid:37473604
- 43. Okpara C, Adachi J, Papaioannou A, Ioannidis G, Thabane L. Exploring participant attrition in a longitudinal follow-up of older adults: the Global Longitudinal Study of Osteoporosis in Women (GLOW) Hamilton cohort. BMJ Open British Medical Journal Publishing Group; 2023 Jul 1;13(7):e066594. pmid:37491101
- 44. Nittas V, Zecca C, Kamm CP, Kuhle J, Chan A, Wyl V von. Digital health for chronic disease management: An exploratory method to investigating technology adoption potential. PLOS ONE Public Library of Science; 2023 Apr 13;18(4):e0284477. pmid:37053272
- 45. Daniore P, Nittas V, Gille F, von Wyl V. Promoting participation in remote digital health studies: An expert interview study. Digit Health SAGE Publications Ltd; 2023 Jan 1;9:20552076231212063. pmid:38025101
- 46. Simblett S, Greer B, Matcham F, Curtis H, Polhemus A, Ferrão J, Gamble P, Wykes T. Barriers to and Facilitators of Engagement With Remote Measurement Technology for Managing Health: Systematic Review and Content Analysis of Findings. J Med Internet Res 2018 Jul 12;20(7):e10480. pmid:30001997
- 47. Aboumatar H, Pitts S, Sharma R, Das A, Smith BM, Day J, Holzhauer K, Yang S, Bass EB, Bennett WL. Patient engagement strategies for adults with chronic conditions: an evidence map. Syst Rev 2022 Mar 5;11(1):39. pmid:35248149
- 48. Santos-Vijande ML, Gómez-Rico M, Molina-Collado A, Davison RM. Building user engagement to mhealth apps from a learning perspective: Relationships among functional, emotional and social drivers of user value. J Retail Consum Serv 2022 May 1;66:102956.
- 49. Savioni L, Triberti S, Durosini I, Sebri V, Pravettoni G. Cancer patients’ participation and commitment to psychological interventions: a scoping review. Psychol Health Routledge; 2022 Aug 3;37(8):1022–1055. pmid:33966548
- 50. Pratap A, Neto EC, Snyder P, Stepnowsky C, Elhadad N, Grant D, Mohebbi MH, Mooney S, Suver C, Wilbanks J, Mangravite L, Heagerty PJ, Areán P, Omberg L. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. Npj Digit Med Nature Publishing Group; 2020 Feb 17;3(1):1–10. pmid:32128451
- 51. Zhang Y, Pratap A, Folarin AA, Sun S, Cummins N, Matcham F, Vairavan S, Dineley J, Ranjan Y, Rashid Z, Conde P, Stewart C, White KM, Oetzmann C, Ivan A, Lamers F, Siddi S, Rambla CH, Simblett S, Nica R, Mohr DC, Myin-Germeys I, Wykes T, Haro JM, Penninx BWJH, Annas P, Narayan VA, Hotopf M, Dobson RJB. Long-term participant retention and engagement patterns in an app and wearable-based multinational remote digital depression study. Npj Digit Med Nature Publishing Group; 2023 Feb 17;6(1):1–13. pmid:36806317
- 52. Teague S, Youssef GJ, Macdonald JA, Sciberras E, Shatte A, Fuller-Tyszkiewicz M, Greenwood C, McIntosh J, Olsson CA, Hutchinson D, Bant S, Barker S, Booth A, Capic T, Di Manno L, Gulenc A, Le Bas G, Letcher P, Lubotzky CA, Opie J, O’Shea M, Tan E, Williams J, the SEED Lifecourse Sciences Theme. Retention strategies in longitudinal cohort studies: a systematic review and meta-analysis. BMC Med Res Methodol 2018 Nov 26;18(1):151. pmid:30477443
- 53. Zhang S, West BT, Wagner J, Gatward R. Incorporating Adaptive Survey Design in a Two-Stage National Web or Mail Mixed-Mode Survey: An Experiment in the American Family Health Study. J Surv Stat Methodol 2023 Sep 12;smad035. pmid:38911251
- 54. Gates E, Hole B, Hayward S, Chesnaye NC, Meuleman Y, Dekker FW, Evans M, Heimburger O, Torino C, Porto G, Szymczak M, Drechsler C, Wanner C, Jager KJ, Roderick P, Caskey F, for the EQUAL investigators. Converting from face-to-face to postal follow-up and its effects on participant retention, response rates and errors: lessons from the EQUAL study in the UK. BMC Med Res Methodol 2022 Feb 11;22(1):44. pmid:35148682
- 55. Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform 2017 Jul 1;71:31–48. pmid:28536062
- 56. Chun AY, Heeringa SG, Schouten B. Responsive and Adaptive Design for Survey Optimization. J Off Stat SAGE Publications; 2018 Sep 1;34(3):581–597.
- 57. Bu F. Non-response and attrition in longitudinal studies. J Epidemiol Community Health BMJ Publishing Group Ltd; 2022 Dec 1;76(12):971–971. pmid:36307201
