https://orcid.org/0000-0002-5567-9300
Figures
Abstract
Digital health technology (DHT) holds the potential to improve health services, and its adoption has proliferated in recent decades owing to technological advancement. Optimal evaluation methodologies appropriate for generating quality evidence on DHT have yet to be established; traditional comparative designs present several limitations. This study aimed to scope the literature to highlight common methodological approaches used and their limitations to inform considerations for designing robust DHT evaluation studies. A scoping review was conducted following the Joanna Briggs Institute (JBI) scoping review guidelines. A systematic search was conducted using the CINAHL (EBSCO), MEDLINE (EBSCOhost), PsycINFO (EBSCO), EMBASE (Elsevier) and Web of Science (Clarivate Analytics) databases using iteratively developed search terms. We selected studies published in English between January 2016 and March 2022 and focussed on primary research evaluating the effectiveness of DHT with technology-user interactive or asynchronous features for adults (≥18 years) with cancer, diabetes or cardiovascular conditions. The final number of articles, after the screening and selection process, comprised 140 records. Data were analysed descriptively (frequency and percentages) and summarised thematically. Results showed most studies (n = 104, 74.3%) employed the standard two-arm parallel RCT design, with usual/standard care as the preferred comparator in nearly half (n = 65, 47.1%) of all included studies. Of the 104 comparative studies reviewed, limitations in recruitment were most frequently reported (n = 70, 37%), followed by limitations in evaluation/measurement techniques (n = 57, 27%), presence of confounding factors (n = 50, 24%) and short duration of studies (n = 24, 11%). The review highlights the need to consider inclusive approaches to recruitment and adoption of the emerging methodological approaches that account for the fast-paced, multi-component and group contamination problem resulting from the unconcealable nature of DHT interventions.
Author summary
There is growing potential for digital health technology to improve healthcare delivery and access. However, digital health interventions are usually complex creating methodological challenges in their evaluation. In this article, we present findings of a scoping review on common methodological approaches and limitations and/or issues in studies evaluating digital health interventions for three common long-term conditions (i.e., diabetes, cancer and cardiovascular disease). Our findings suggest that the standard, two-arm parallel randomised controlled trials with usual/standard care as comparator, remain the preferred methodological approach. The common methodological limitation reported in the comparative study designs was bias in selection and recruitment, either due to biased eligibility criteria or recruiting highly motivated and technology-savvy participants. Other issues reported are the unreliability of the measurement techniques used, and inability of investigators to fully control the environment in which the digital health intervention was tested and/or control participants’ access to the intervention. This reduces the generalisability and credibility of the studies’ findings. Based on our findings, researchers need to advertise their studies widely using varied communication channels and provide support to less technical participants to ensure the recruitment of a diverse and representative sample. Also, there is a need to consider emerging methodological approaches that allow multi-phase and multiple-group comparisons in an attempt to address these limitations.
Citation: Gityamwi N, Armes J, Harris J, Ream E, Green R, Ahankari A, et al. (2025) Methodological approaches and author-reported limitations in evaluation studies of digital health technologies (DHT): A scoping review of DHT interventions for cancer, diabetes mellitus, and cardiovascular diseases. PLOS Digit Health 4(4): e0000806. https://doi.org/10.1371/journal.pdig.0000806
Editor: Haleh Ayatollahi, Iran University of Medical Sciences, IRAN, ISLAMIC REPUBLIC OF
Published: April 24, 2025
Copyright: © 2025 Gityamwi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting information files.
Funding: The author(s) received no specific funding for this work. The authors’ time was covered by their respective institutions. Nyangi Gityamwi’s salary is paid by NIHR Applied Research Collaboration - Kent, Surrey and Sussex (ARC- KSS). Jo Armes receives funding from the NIHR Applied Research Collaboration - Kent, Surrey and Sussex (NIHR ARC KSS). The views expressed are those of the author(s) and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Digital health technology (DHT) has the potential to improve health and social care by widening access for individuals [1]. DHT has sparked public interest as it can significantly help increase patients’ engagement, empower them to manage their conditions [2] and ensure continuity of care, as was the case during the COVID-19 pandemic lockdowns.
Nevertheless, as with any healthcare service or product, the use of DHT needs to be supported by robust evidence to show it is accurate, reliable, safe and able to demonstrate effectiveness in producing the intended outcome [2]. Despite the potential of DHT interventions, evaluation faces challenges due to their complex nature and evolving technology. The randomisation approaches and tight control of the research environment [3] make randomised controlled trials (RCTs) the accepted gold standard methodology for the unbiased demonstration of causal relationships between intervention and outcome [4]. Consequently, RCTs have been widely employed when testing digital health interventions [5,6]. RCTs are both recommended in the World Health Organisation’s (WHO) practical guideline for monitoring and evaluating digital health interventions [7] and endorsed by the National Institute for Health & Care Excellence (NICE) [2].
However, since RCT protocols require a static and tightly controlled environment to minimise bias and confounding factors [3], this poses challenges regarding the appropriateness of RCTs as an evaluation design for DHT because of their continuous technological evolution [8]. Moreover, long timeframes required for RCTs [9] mean a DHT may be outdated when the evaluation of its efficiency has been completed. Therefore, the utility of RCTs to evaluate the complex and multifaceted nature of DHT interventions is a subject of debate [10], particularly when the technology is to be scaled up and tested in real-world complex health systems. Instead, methodological approaches that allow flexibility and context-dependent evaluation are gaining importance [11,12]. The UK Medical Research Council has proposed a range of other designs that can be used as an alternative to the standard RCT such as stepped-wedge or N-of-1 designs [13]. These and other designs can provide greater flexibility regarding measurements and their timing, dosage and length of intervention/exposure duration [14,15]. Examples include the sequential multiple assignment randomised trial (SMART) and multiphase optimisation strategy (MOST), including factorial designs to measure effects as presented by Collins and colleagues [16]. Indeed, few studies have used adaptive study designs for the evaluation of DHTs, as noted by both Pham and colleagues [17] and the more recent scoping review by Hrynyschyn and colleagues [18] of designs used other than RCTs. Our scoping review aimed to contribute to this body of literature by reviewing and critically analysing practical implications of the comparative study designs, highlighting limitations and issues from the researchers’ reported experience. We aimed to answer two research questions:
- i. What study designs are used to evaluate the impact of DHT on clinical health outcomes among adults diagnosed with diabetes, cancer and cardiovascular diseases?
- ii. What are the author-reported methodological challenges in comparative study designs evaluating DHT?
Methods
This review was conducted in accordance with the Joanna Briggs Institute (JBI) scoping review guidelines [19]. Scoping reviews are useful for examining emerging and complex fields of study and when the research focus or question is broad [20,21]; thus, very relevant for systematically collating research evidence with varied study designs used in the evaluation of digital health interventions. The protocol is registered with Open Science Framework (OSF) https://doi.org/10.17605/OSF.IO/R24D5.
This paper reports the findings from the scoping review by following the reporting guidelines of the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist [22].
Eligibility and inclusion criteria
The review was conducted at a time of high proliferation of DHT and research studies evaluating them were numerous and heterogeneous in terms of their focus. To address this, our review focussed on studies conducted to evaluate the impact of digital health interventions on three long-term conditions namely diabetes, cancer and cardiovascular disease. These conditions were selected due to their chronic nature, and the significant burden they place on individuals and the healthcare system globally [23]. Additionally, their importance in the digital health landscape was a key factor in their selection [24]. Furthermore, only studies that evaluated the effectiveness or efficacy of DHT interventions were eligible for inclusion in the final data synthesis. We excluded proof of concept studies and those which only assessed acceptability or usability without clinical or treatment outcomes. Table 1 presents the inclusion and exclusion criteria that were applied in the study screening and selection.
Because of the breadth of DHT, we narrowed the review to include only those DHTs with technology-user interactive (asynchronous) features. The search was limited to articles published from January 2016 with the last search conducted in March 2022. This time frame was selected to capture the recent studies technology at the time of the review.
Information sources and search strategy
An initial limited search was undertaken in MEDLINE (EBSCOhost) in consultation with a research librarian to identify the database-specific index terms related to two key concepts—“digital health” and “disease management”. The search strategy was then adapted for each included database and first search was performed between March and June 2020 in CINAHL (EBSCO), MEDLINE (EBSCOhost), PsycINFO (EBSCO), EMBASE (Elsevier) and Web of Science (Clarivate Analytics). In March 2022, we conducted updated search in EBSCOhost, a collection of MEDLINE, CINAHL and PsycINFO databases (Table 2). The complete search strategies and results for both searches are provided in S1 File.
Reference screening and selection
All identified citations were retrieved and imported into the reference manager software EndNoteX9 (Clarivate Analytics, PA, USA). Duplicates were removed by the lead researcher and author (NG). All titles and abstracts were screened independently for eligibility against the inclusion and exclusion criteria by NG, and then a random sub-sample of 20% of papers was reviewed by other members of the research team (SS, JA, JH) to ensure consistency in applying the eligibility criteria. Where there were discrepancies in study selection, full articles were collectively reviewed by members of the research team and discussed to reach a consensus. The full text of eligible citations was then assessed in detail against the inclusion criteria by two reviewers independently (NG and JH). Any disagreements that arose between the reviewers were resolved through discussion with a third (JA or SS). Fig 1 shows the study selection process using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR). The description of studies included in the review and data extracted is presented in S2 File.
Data charting
Data charting guide was created in Microsoft Excel by the primary author (NG) and then discussed and verified by two co-authors (JA and SS). Pilot data charting (20% of the included articles) was performed by three investigators independently (JH, JA and ER) to test the guide, any discrepancies were discussed until consensus was reached and the guide was revised as appropriate. The final full data charting guide (presented in S3 File) includes but is not limited to a description of the references, condition investigated, aim of the digital health intervention, DHT classification, study design used and comments on methodology. All investigators participated in data extraction using a Microsoft Excel spreadsheet.
Data synthesis
Data were extracted from 140 articles and the initial analysis was performed to broadly describe the characteristics of the included studies and study designs adopted to evaluate DHT for the selected long-term conditions (research question 1). Of these studies, 118 studies were controlled/comparative studies, and these were selected to summarise author-reported methodological challenges (research question 2). Data were presented in percentages in general and according to the condition investigated and categories of the DHT aims. The NICE Evidence Standards Framework [25] for digital health technologies was used to define and classify DHTs according to their aim. We reported the challenges in the context of the author’s reported limitations.
Results
Search results
The two searches identified a total of 23,473 records. Following the removal of duplicates (n = 8,796), the titles and abstracts of 14,677 articles were screened and 584 were selected for full-text review. A further 444 articles were excluded following the full-text review with the reasons outlined (Fig 1) and the final articles included were 140.
Characteristics of the reviewed references
Of the 140 articles, 11 (7.9%) were published in 2016, 21 (15%) in 2017, 19 (13.6%) in 2018, 15 (10.7%) in 2019, 42 (30%) in 2020, 30 (21.4%) in 2021, with only 2 (1.4%) published in 2022. Studies were most frequently conducted in the United States of America (n = 29, 20.7%) followed by the Netherlands and China each with 12 (8.6%) publications. Of the three included conditions, more than a third of the reviewed studies focussed on diabetes mellitus (n = 52, 37.1%). When grouped studies into three categories according to the aim of the DH interventions following the NICE classification framework [25], those aimed at providing treatment and/or therapies were 42.9%, (n = 60); aiding disease self-management were 35.7% (n = 50), and those aimed at promoting preventive behavioural change 21.4% (n = 30).
Study designs used to evaluate the impact of DHT on clinical health outcomes
Six study designs were identified from the reviewed studies. These are RCTs; controlled, non-randomised quasi-experimental studies; pre-post single-arm follow-up; cross-sectional observation; mixed methods; and retrospective studies. Generally, RCTs were the most common research design, adopted in 104 of the 140 (74.3%) included studies (Table 3). This was the case regardless of the condition as we found RCTs comprised 76.2% of the cancer studies, 63.5% of the diabetes mellitus, 84.2% of the cardiovascular, and 87.5% of the studies with multiple conditions (Table 3). When classified according to intervention aim (i.e., DHT for treatment/therapy, self-management, preventive/behavioural change), RCTs were still the most commonly design, found in more than half of studies in all three groups.
Other study designs adopted included pre-post/prospective single-arm follow-ups (n = 14), retrospective studies (n = 5), mixed methods (n = 2), and a cross-sectional observational study (n = 1) (Table 3). Of the 14 pre-post studies, 9 evaluated DHT aimed at providing treatment/therapies, 4 evaluated DHT for patient self-management, and 1 study evaluated DHT for preventive/behavioural change. All five retrospective studies evaluated DHT for treatment/therapy. Of the two mixed-methods studies, one evaluated DHT for treatment/therapy, and the other evaluated DHT for preventive/behavioural change. The cross-sectional evaluation study evaluated DHT aimed at aiding patient self-management.
Study designs according to health condition
Study designs used in cancer studies.
Of the 42 cancer-focussed studies, 76.2% (n = 32) adopted RCT designs, whereas 14.2% (n = 6) were non-randomised studies with controls and 9.5% (n = 4) were pre-post single-arm studies. More than half of the cancer studies which used RCT designs (63%, n = 20) were interventions aimed at providing treatment/therapies.
Other RCTs focussed on patients’ self-management (31%, n = 10) and preventive/behavioural change (25%, n = 8). Of the six non-randomised cancer studies, three focussed on self-management and the remaining three were studies aimed at providing treatment/therapies. Of the four pre- and post-studies, two focussed on treatment/therapies, one on self-management, and one on preventive/behavioural change category.
Study designs used in studies on diabetes mellitus.
Of the 52 studies involving patients with diabetes mellitus, 63.5% (n = 33) adopted RCT designs, whereas 9.6% (n = 5) were non-randomised studies with controls and 15.3% (n = 8) were pre-post single-arm studies. Additionally, there was one cross-sectional study, one study that employed a mixed methods design and four retrospective studies.
More than half of diabetes studies which adopted RCT designs (52%, n = 17) were interventions aimed at aiding self-management. Those that provided treatment/therapies numbered 10 (30%) while preventive/behavioural change interventions constituted 19.4% (n = 6) of the diabetes RCTs.
All five non-randomised studies with controls targeted self-management. Of the eight studies which employed a pre-post design, six were categorised as disease treatment/therapy whereas the remaining two were preventive/behavioural change interventions. All four retrospective studies focussed on providing treatment/therapies. The study that employed a cross-sectional design was a self-management intervention while the mixed-method study aimed to provide treatment/therapy.
Study designs used in studies for cardiovascular diseases.
Of the 38 cardiovascular disease studies, 84.2% (n = 32) adopted RCT designs, whereas 7.9% (n = 3) were non-randomised studies with controls. Finally, there was a pre-post single-arm trial, a retrospective study, and one with a mixed method.
Of the 32 studies that adopted RCT designs, 41% (n = 13) evaluated interventions for treatment/therapy, 34% (n = 11) studies investigated preventive/behavioural change interventions and the remaining 25% (n = 8) evaluated self-management interventions. Of the three cardiovascular non-RCTs, two evaluated treatment/therapies whereas one evaluated a self-management intervention. The study which employed a pre-post single arm evaluated self-management intervention, the retrospective study evaluated a treatment/therapies intervention, and the mixed methods study evaluated a preventive/behavioural change intervention.
Study designs used in studies with multiple conditions.
Eight studies involved patients with multiple conditions. Seven of these adopted RCT designs (88%), whereas one employed a pre-post single-arm design (12%). Of the seven RCTs, four (57.1%) evaluated interventions for self-management and the remaining three studies evaluated preventive/behavioural change interventions. The pre-post study evaluated a treatment/therapy intervention.
Comparative designs used in studies evaluating the effectiveness of DHT interventions
To address the question regarding the appropriateness of standard recommended comparative approaches such as RCTs for digital health evaluation, the present review presents limitations reported in comparative studies only, to inform other researchers on issues to consider when planning to use similar approaches.
A total of 118/140 (84%) of the included studies adopted a controlled design either using randomised (n = 104, 88%) [26–129] or non-randomised quasi-experimental approaches (n = 14, 12%) [130–143]. These were reviewed further and described as presented in Table 4 illustrating the type of controls used and how the study groups were allocated. A list of studies in each category is presented in S2 File.
Randomised controlled designs.
The majority of the reviewed RCTs used usual care (n = 63, 60%) as comparators. Other types of comparators comprised active control/alternative interventions (n = 28, 27%) and waitlist/delayed interventions (n = 13, 13%).
When grouping according to conditions, usual care was the most common comparator employed in controlled studies for all conditions, present in 36%, (n = 15) of cancer studies, 46% (n = 24) of diabetes mellitus studies, 55% (n = 21) cardiovascular studies and 75% (n = 6) of studies with multiple conditions. Additionally, waitlist/delayed treatment was the second most common comparator for cancer studies, employed in 24% (n = 10) studies whereas active/alternative intervention was the second comparator of choice for diabetes and cardiovascular studies. Historic controls were used little in studies across all conditions; 10% (n = 4) in cancer studies, 6% (n = 3) in diabetes mellitus studies, 8% (n = 3) in cardiovascular diseases, and none in studies with multiple conditions.
Of all the 63 RCT studies that employed usual care controls, 92% (n = 58) had equal (1:1) and random allocation of participants to two parallel groups, i.e., between intervention and control. Five of the sixty-threee RCTs were three-arm studies, with equal and random allocation of participants to two interventions and one usual care control group.
Of all the 28 RCTs which used active/alternative intervention control, 89% (n = 25) equally and randomly allocated participants to two parallel groups, i.e., to either intervention or an active/alternative intervention control group. The remaining three RCTs were parallel arm trials2014two of which compared two interventions against an active control, and one study compared an intervention against two active controls (Table 4).
Of the 13 RCTs which used waitlist/delayed intervention control, participants were equally and randomly allocated between the intervention or waitlist/delayed intervention in 11 studies. Two studies were three-arm trials comparing two interventions against one waitlist control (Table 4).
Non-randomised controlled designs.
For the non-randomised quasi-experimental studies, historic controls were the predominant comparators, used in more than half of the studies (8/14). Among these, six (Table 4) compared intervention groups with 1:1 matched historic controls, while two studies compared an intervention group with matched historic controls that were twice as large as the intervention groups (i.e., 1:2 ratio). One study used a second setting as the comparator group providing an alternative non-DH intervention (active control). Two studies had usual care as comparators. One study was conducted in a real-life clinical setting, and the groups were formed sequentially with a control group formed after the clinic stopped the DH intervention—thus patients were receiving usual care. In a second study, two interventions were compared to a usual care control group in a ratio of 1:2:2 (the first intervention group was smaller than the second intervention and the control groups). Two studies used control groups which had delayed access to an intervention. One of these studies used 1:1 allocation and both groups were drawn from an ill population, while in the second study, a healthy population group was used as a comparator and was further divided and randomly assigned to either receive an intervention or to a waitlist group making it a three-arm trial. The last study used an equal number of the intervention’s non-responders as a control group.
Authors reported methodological challenges in comparative studies evaluating the effectiveness or efficacy of DHT interventions
We present the author-reported limitations in the context of methodological challenges found in the pilot and full studies included in the review.
Author-reported limitations in the pilot trials
Fourteen of the one hundred eighteen (12%) comparative studies included in the review were either pilot or feasibility studies. Eight author-reported limitations were reported, and these were summarised into four themes including: (1) issues pertaining length of the study, (2) limitations in recruitment ,(3) limitations in evaluation and measurement techniques, and (4) the presence of Confounding variables. Table 5 presents a list of the limitations reported grouped by theme and their frequency of reporting.
S4 File presents a list of pilot/feasibility studies reporting each limitation grouped according to the aim of DHT intervention.
Author-reported limitations in full comparative studies included in the review
We reviewed the limitations reported in 104 full studies that adopted a comparative approach in their designs and 13 limitations were found. These were assigned to four themes similar to those in pilot studies namely: (1) recruitment, (2) study length, (3) confounding variables, (4) evaluation and/or measurement techniques. Table 6 shows the limitations grouped into the four themes and their frequency of reporting. S5 File presents a list of full comparative studies reporting the limitations grouped according to the condition investigated and the aim of DHT intervention.
Issues in recruitment were the most commonly reported limitation (37%). This was followed by confounding variables, limitations in evaluation/measurement techniques used, and the length of the study (Table 6). The same trend was observed when the data was grouped according to the condition investigated and the aim of the DHT intervention with few exceptions. The highest proportion of cardiovascular studies reported limitations in recruitment and evaluation/measurement techniques whereas those reported confounding variables and length of the study limitations were lower than the proportions for the studies overall. Another difference was seen in studies involving multimorbidity whereby the proportion that reported the presence of confounding variables was above the average whereas those reporting limitations in recruitment and evaluation and/or measurement techniques were below the average for the studies overall (Fig 2).
Limitations in recruitment.
The reported limitations in recruitment include bias in selection and recruitment of participants, which appeared in just over a third (38%) of the reviewed full comparative studies. Other limitations were small sample size due to under-recruitment (33%, n = 34) and high attrition rate (4%, n = 4). When grouped according to the conditions investigated, compared to the overall sample, a greater proportion of studies that recruited participants with multimorbid conditions reported bias in selection and recruitment (71%, n = 5). The proportion of diabetes (39%, n = 13) and cardiovascular (38%, n =11) studies that reported small sample size due to under-recruitment was also slightly higher than the overall percentage of 33% (Fig 3). When grouped according to the aim of DHT, studies of behaviour change and/or disease prevention interventions we identified a greater proportion reporting all three limitations in recruitment (Fig 4).
Presence of confounding variables.
Confounding variables reported include the presence of expert/researcher influence (13%, n= 14), uncontrollable environmental factors that influenced the findings (13%, n = 13), inherent systemic differences between study groups (3%, n = 3), and multi-component intervention (8%, n = 8). Eighteen percent (n = 19) of the studies acknowledged that their methodologies did not account for confounding variables. When grouped according to the condition investigated, studies with multimorbidity reported a greater proportion of all four individual confounding limitations when compared to the proportion for the overall sample (Fig 3). When grouped according to the aim of DHT intervention, the proportion of behaviour change and/or disease prevention intervention studies that reported the presence of multi-component intervention, inability to control study environment (contamination), and expert/researcher influence were higher than that reported by the studies overall. Also, the proportion of studies of behaviour change and/or disease prevention intervention and those of disease self-management that did not account for confounding variables in their evaluation were greater than those in the overall sample (Fig 4)
Limitations in evaluation and/or measurement techniques.
The reported limitations in evaluation and/or measurement techniques include the use of unvalidated/unreliability measurement techniques, reported by nearly a quarter of the studies (22%, n = 23); unstandardised study procedure or change in protocol or technical faults during the study (6%, n = 6); low adherence to intervention (16%, n = 17) and incomplete/missing data (4%, n = 4). When grouped according to the conditions investigated, the proportion of studies that reported low adherence to the intervention was higher in multimorbidity studies compared to the proportion of the studies overall. All conditions in a similar proportion (above the average) reported being limited by unreliable/unvalidated measurement tools except diabetes studies which reported in lower proportion than average (Fig 3). When grouped according to the aim of the DHT intervention, a greater proportion of incomplete/missing data and low adherence to interventions was reported for self-management intervention studies. Unreliability/validity of measurement tools was most frequently reported for behaviour change/disease prevention interventions whereas a greater proportion of unstandardised study procedure/changes in protocol/technical faults were reported for therapy/treatment intervention studies than in the overall sample (Fig 4).
Length of the study.
Nearly a quarter of the studies (23%, n = 24) reported that the interventions or their studies were short and/or had inadequate follow-up periods. When grouped according to the condition investigated, a greater proportion of multimorbidity and diabetes studies reported this limitation than the studies overall (Fig 3). When grouped according to the aim of the DH intervention, the proportion of disease self-management and Behaviour change/disease prevention studies that reported being of short duration was slightly above that of the studies overall (Fig 4).
Discussion
Methodologies used in DHT evaluation studies
Our review found that RCTs are the primary methodological choice for most researchers evaluating DHT interventions, independent of the health condition investigated or the purpose of the DHT intervention, i.e., providing treatment/therapy, aiding self-management, or preventive/behavioural change. This is consistent with the findings of recent reviews [17,18] indicating that, despite its perceived limitations, the RCT design is still the preferred approach for evaluating DHT interventions. The majority of studies used the standard two-arm parallel design in which participants are randomly allocated between the intervention or control group. Again, these approaches to evaluating DHT interventions are perhaps surprising given that in most cases there may be multiple elements influencing their effectiveness.
Multi-arm trials have been suggested as an alternative design, offering the possibility of conducting multiple comparisons simultaneously [144]. In the present review, there were ten RCTs with more than two trial arms. Those with multiple arms compared either two interventions with one control or one intervention compared with two control groups. Nevertheless, it is important to note that where multiple comparisons are made simultaneously within the same study (e.g., subgroup comparisons, comparisons across multiple treatment arms, analysis of multiple outcomes, and multiple analyses of the same outcome at different times), the probability of rejecting at least one null hypothesis given that all nulls are true increases (type I error), misleading the findings (false-positive findings) [145]. In this case of multiplicity, adjustments need to be made in terms of the design and statistical analysis plan. For example, a single outcome/comparison needs to be identified as the primary and treat the remaining outcomes/comparisons as extensively discussed in studies by Li and colleagues and Odutayo and colleagues [144,146]. These did not feature in the studies with multiple controls included in the present review.
Some studies used cluster randomisation by grouping the individuals into clusters (groups) and then randomising these groups to different study arms [32]. Cluster randomisation is increasingly being adopted when evaluating complex interventions, particularly those with multiple individual elements that are likely to interact or influence each other, an issue that is significant in most DHT interventions [147]. However, cluster randomised trials tend to be complex to design, deliver and even interpret, compared to individual RCTs due to the interplay between the similarities and differences within and between clusters respectively [148,149]. Therefore, care needs to be taken when deciding on the sample size to ensure studies are sufficiently powered, avoid bias in selection and recruitment, and consider suitable data analysis approaches and reporting [150,151].
Our review also highlighted the small number of non-comparative studies, such as pre-post single-arm follow-up, cross-sectional observation, mixed methods, and retrospective studies. Temporal variability is thought to be more problematic in these types of designs as, without a control group, there is no way of knowing what the outcome would have been without an intervention [152]. To account for the lack of control, multiple measures of the outcome of interest at baseline and after intervention are suggested [153].
Type of control/comparator group
The present review also illustrated that usual/standard care is the preferred comparator for most researchers. While it is recognised that a placebo control is an ideal comparator when designing high-quality RCTs, there have been ethical debates regarding the use of a placebo in health intervention studies as it would mean denying some of the study participants care/treatment and possibly exposing them to further harm or risk [154]. Also, because of the nature of the DH intervention, i.e., inability to mask the intervention, it is questionable whether a placebo control could be feasibly employed and/or be of benefit. Active controls have increasingly been adopted in clinical intervention studies to address the ethical concerns about placebo control. In the present review, 21.4% of studies compare the intervention of interest with an alternative intervention/active control. Nevertheless, the use of active control is not without its own limitations. Researchers have highlighted the lack of sensitivity in such trials which measure the relative efficacy of the intervention of interest relative to the alternative, unlike in placebo-controlled designs which enable assessment of the absolute efficacy of the intervention [155].
We also found waiting list/delayed treatment controls were used, but this was not common (in 13 of the 104 RCTs). The use of waitlist controls is ethically advantageous since it allows the provision of the same care/treatment to both study arms. However, it is argued that such designs run the risk of overestimating the intervention effect as unexpectedly lower levels of the variable of interest tend to be observed in waitlist control groups. This is particularly true in studies where participants are aware of their group allocation as they are likely to consciously not show/inform any changes until they start the intervention [156]. Considering the nature of DHT interventions (i.e., difficult to mask), it is also often likely that participants in the control group will be exposed to or access the intervention either intentionally or unintentionally, contaminating the study and creating biased results. This concern was acknowledged by some of the reviewed studies [40,94,95,120]. Finally, the use of a waitlist control group precludes assessment of the long-term effect of the intervention.
Fourteen studies were controlled but not randomised, with eight of them using historic groups/data as a comparator. Historic controls have commonly been used in situations where there are ethical concerns or where it is impractical to have a concurrent control group [157]. However, the use of historic controls can lead to selection bias or failure to account for systematic differences between the groups or data and conditions/standards of the studies [158,159], importantly in the fast-paced technology era as acknowledged in two studies included in our review [134,140]. Six of eight historically controlled trials included in the present review used matched historic control groups whereas the remaining two studies used historic comparison groups that were twice as large as the intervention groups. The use of non-responders and healthy individuals as control groups was also noted, confounding the results due to possible inherent differences between groups.
Author-reported limitations in comparative studies
Limitations in recruitment.
Nearly half of the pilot studies and more than half of full trials reported limitations in recruitment, particularly inadequate sample size due to under-recruitment and bias in selection and recruiting participants. While it is common for pilot or feasibility studies to have a small sample size since they are usually conducted to inform the design of future full trials rather than to measure effectiveness directly themselves, in our review, over a quarter of the full trials also reported under recruitment and over one-third reported bias in selection and recruitment compromising the internal and external validity of the results. This was particularly seen in a higher proportion of studies evaluating behaviour change/disease prevention studies and those studying multimorbidity respectively. Under recruitment in online studies has been reported elsewhere and the recruitment strategy employed seems to play a role in recruitment outcomes [160,161]. Online recruitment methods for example through social media are increasingly used in intervention studies including those in the present review as they have the potential for a wide reach. However, this does not always translate to a positive response. Online messages can be perceived as fraudulent information and unsafe and people can be reluctant to respond or ignore the adverts [162]. Moreover, although online recruitment might be a reasonable recruitment strategy for interventions delivered and accessed through technology, it carries the risk of creating a sample that is not representative as participants recruited online are subject to greater self-selection and may disproportionately comprise people from a particular socio-economic group, technology savvy or those who are health conscious or motivated to seek health solutions [163,164]. Recruitment strategies must consider individual differences and tailor the recruitment strategy depending on the targeted population [165]. Where online recruitment methods are used, it is worth using trusted and/or verified sources for example institutional platforms, and ensure the data protection strategy is clear and communicated [166]. One could also consider using offline approaches (such as word of mouth through community networks or letters) in parallel to online recruitment [162]. Budgeting for and promising provision of technology and/or technology skills and ongoing technical support throughout the intervention might help to increase confidence of research volunteers and widen participation and inclusion [166].
Length of study.
Some of the reviewed studies had short intervention periods, in some cases as short as 4–8 weeks. This was the case for both pilot and full trials, while this might not be a concern for pilot trials, it is more problematic in full trials aiming to demonstrate impact or whether an effect dissipates over time, particularly for lifestyle or behavioural change interventions, which require time to work. In our review, a quarter of the studies which evaluated DHT for behavioural change or disease prevention and more than one-third of multimorbidity studies reported being of short duration. Longer follow-up of at least 1 year would generally be expected for behavioural change or risk prevention interventions to enable impact as recommended by NICE guidance on individual approaches to behaviour change [167]. However, designing long DHT interventions might be challenging due to the fast-paced nature of technology. Nevertheless, the level of engagement or exposure to interventions is equally important and critical for the effectiveness of DHT, and this extends beyond the study duration encompassing the amount, frequency, depth, and length of usage of the intervention [168]. Thus, engagement is an important measure in DHT evaluation studies despite the challenges to accurately capture such data due to its manifaced nature [169,170]. In the present review, although user engagement was not among the reported limitations, it is possible that this was not well evaluated in the studies limiting the understanding of the interventions’ impact.
Evaluation and measurement techniques.
A good proportion of studies reported being limited by the use of measurement tools/scales that are unvalidated and unreliable or that they collected participants’ self-reporting data which can be unreliable. This is particularly the case for studies on behaviour change/disease prevention and those with multimorbidity. For example, despite the use of validated standard outcome measures such as distress (e.g., Perceived Stress Scale [171]), anxiety (e.g., Generalized Anxiety Disorder 7-item Scale (GAD-7) [172]) and depression (e.g., Patient Health Questionnaire-9 (PHQ-9) [173]), health-related quality of life (e.g., Assessment of Quality of Life 8-item Questionnaire [174]), participants self-report is thought to be subject to bias, particularly when participants are aware of the intervention they are receiving and the expected outcome. Participants’ technology skills and awareness of their condition (e.g., those on long-term conditions) may contribute to bias in reporting. Additionally, other outcome measures like dietary intake and physical activities rarely used standardised scales [175]. Training of participants to accurately record and report data, employing blinding techniques and incorporating objective measures where possible capturing data directly by the digital devices might help reduce systemic error in reporting data.
Confounding variables.
Nearly a quarter of full comparative trials reported being confounded by different factors such as the inability to control the study environment and the influence of a researcher or experts involved in the delivery of the intervention to the outcome of the intervention. In particular, this was highly reported in studies evaluating behaviour change/disease prevention intervention, perhaps because varied individual environmental factors may unwittingly influence their behaviour and findings. It is also difficult to blind behavioural interventions [176]. Authors reported contamination where the participants from control groups gained access to the intervention either intentionally or unintentionally [177]. This is also common for study designs using waitlist and/or standard care as usual controls and not blinded to intervention allocation [178]. In the present review, of the 13 studies that reported contamination as a limitation, 9 used usual care as the control condition and 4 employed a waitlist control. Psychotherapy research has demonstrated small effect sizes in studies with waitlist control groups [179] suggesting that caution needs to be taken when using this type of control to account for the waiting effect. Participants’ awareness of their allocation might also influence how they respond within a trial and so potentially leading to biased responses [180]. Few considerations have been suggested to mitigate the highlighted limitations such as the adoption of pragmatic randomised control trials by employing appropriate alternative intervention choices as a control group rather than no-treatment or usual care [181]. The use of cluster randomisation and mantling designs like MOST and the SMART which allows variations of intervention content delivered to participants helps to evaluate the isolation effects of different components of interventions and different times [16].
Strengths and limitations of the study
The strength of our scoping review is boosted by the fact that we conducted an extensive search and reviewed a large body of literature. Narrowing our review focus to three long-term conditions of global relevance in terms of disease prevalence and impact, and with high adoption of DHT has enabled us to synthesise the literature rigorously and produce evidence that is more relevant and with wide impact. Our review is limited in the sense that, due to the fast-paced proliferation of DHT and the associated evaluation studies, the evidence presented might not be the latest at the time of publication. Nevertheless, we believe that the conclusion and recommendations made will remain relevant and applicable to research contexts similar to those included in the present review.
Conclusion
Our review suggests that the standard, two-arms parallel RCT design remains the preferred methodological approach with the usual/standard care group commonly used as the control arm. The reported methodological limitations including selection and recruitment bias suggest it is worth considering recruitment modalities that are inclusive using online and offline credible channels. Additionally, RCTs can be seen as restrictive and time-consuming, unable to react to the fast pace of development of technology-based solutions; particularly those involving artificial intelligence or machine learning [182]. Adoption of inclusive approaches to recruitments and emerging pragmatic approaches to RCTs including mantling designs such as the MOST and SMART methods holds considerable promise in addressing the fast-paced, multi-component and group contamination problem resulting from the unconcealable nature of DHT interventions.
Acknowledgments
We would like to thank the following researcher(s) for their assistance in data extraction: Dr Renske Visse.
References
- 1. Willis VC, Thomas Craig KJ, Jabbarpour Y, Scheufele EL, Arriaga YE, Ajinkya M, et al. Digital health interventions to enhance prevention in primary care: scoping review. JMIR Med Inform. 2022;10(1):e33518. pmid:35060909
- 2.
NICE. Evidence standards framework for digital health technologies. 2019 [cited 2024 Oct 15. ]. Available from: https://www.nice.org.uk/Media/Default/About/what-we-do/our-programmes/evidence-standards-framework/digital-evidence-standards-framework.pdf
- 3. Thiese MS. Observational and interventional study design types; an overview. Biochem Med (Zagreb). 2014;24(2):199–210. pmid:24969913
- 4.
Piantadosi S. Clinical trials: a methodologic perspective. Wiley; 2024 [cited 2024 Oct 15. ] Available from: https://books.google.co.uk/books?id=oCL_EAAAQBAJ
- 5. Lau F, Kuziemsky C. Handbook of eHealth evaluation: an evidence-based Approach [Internet]. 2017 [cited 2024 Oct 15. ] Available from: https://www.ncbi.nlm.nih.gov/books/NBK481590/
- 6. Zanaboni P, Ngangue P, Mbemba GIC, Schopf TR, Bergmo TS, Gagnon M-P. Methods to evaluate the effects of internet-based digital health interventions for citizens: systematic review of reviews. J Med Internet Res. 2018;20(6):e10202. pmid:29880470
- 7. WHO. Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. 2016 [cited 2024 Oct 15. ]. Available from: https://saluddigital.com/wp-content/uploads/2019/06/WHO.-Monitoring-and-Evaluating-Digital-Health-Interventions.pdf
- 8. Murray E, Hekler E, Andersson G, Collins L, Doherty A, Hollis C. Evaluating digital health interventions: Key questions and approaches. Am J Prev Med. 2016;51(5):843–51.
- 9. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA. 1998;279(4):281–6. pmid:9450711
- 10. Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism. Int J Med Inform. 2001;64(1):39–56. pmid:11673101
- 11. Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015;17(7):e166. pmid:26155878
- 12. Bonten TN, Rauwerdink A, Wyatt JC, Kasteleyn MJ, Witkamp L, Riper H, et al. Online guide for electronic health evaluation approaches: systematic scoping review and concept mapping study. J Med Internet Res. 2020;22(8):e17774. pmid:32784173
- 13. Skivington K, Matthews L, Simpson S, Craig P, Baird J, Blazeby J, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374(1):1–10.
- 14. Murphy SA. An experimental design for the development of adaptive treatment strategies. Stat Med. 2005;24(10):1455–81. pmid:15586395
- 15. Almirall D, Nahum-Shani I, Sherwood NE, Murphy SA. Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research. Transl Behav Med. 2014;4(3):260–74. pmid:25264466
- 16. Collins LM, Murphy SA, Strecher V. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med. 2007;32(5 Suppl):S112-8. pmid:17466815
- 17. Pham Q, Wiljer D, Cafazzo JA. Beyond the randomized controlled trial: a review of alternatives in mHealth clinical trial methods. JMIR mHealth uHealth. 2016;4(3):e107. pmid:27613084
- 18. Hrynyschyn R, Prediger C, Stock C, Helmer SM. Evaluation methods applied to digital health interventions: what is being used beyond randomised controlled trials?—a scoping review. Int J Environ Res Public Health. 2022;19(9):5221. pmid:35564616
- 19. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119–26. pmid:33038124
- 20. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143. pmid:30453902
- 21. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
- 22. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. pmid:30178033
- 23. WHO. Non communicable diseases. 2024 [cited 2024 Oct 15. ]. Available from: https://www.who.int/en/news-room/fact-sheets/detail/noncommunicable-diseases
- 24. Bashi N, Fatehi F, Mosadeghi-Nik M, Askari MS, Karunanithi M. Digital health interventions for chronic diseases: a scoping review of evaluation frameworks. BMJ Health Care Inform. 2020;27(1):e100066. pmid:32156751
- 25. NICE. Evidence standards framework for digital health technologies. 2022 [cited 2022 Dec 01. ]. Available from: https://www.nice.org.uk/corporate/ecd7/chapter/section-b-classification-of-digital-health-technologies
- 26. Abrahams HJG, Gielissen MFM, Donders RRT, Goedendorp MM, van der Wouw AJ, Verhagen CAHHVM, et al. The efficacy of internet-based cognitive behavioral therapy for severely fatigued survivors of breast cancer compared with care as usual: a randomized controlled trial. Cancer. 2017;123(19):3825–34. pmid:28621820
- 27. Absolom K, Warrington L, Hudson E, Hewison J, Morris C, Holch P. Phase III randomized controlled trial of eRAPID: eHealth intervention during chemotherapy. J Clin Oncol. 2021;39(7):734–47.
- 28. Akinci B, Yeldan I, Satman I, Dirican A, Ozdincler AR. The effects of Internet-based exercise compared with supervised group exercise in people with type 2 diabetes: a randomized controlled study. Clin Rehabil. 2018;32(6):799–810. pmid:29417832
- 29. Anzaldo-Campos MC, Contreras S, Vargas-Ojeda A, Menchaca-Díaz R, Fortmann A, Philis-Tsimikas A. Dulce wireless Tijuana: a randomized control trial evaluating the impact of project dulce and short-term mobile technology on glycemic control in a family medicine clinic in Northern Mexico. Diabetes Technol Ther. 2016;18(4):240–51. pmid:26914371
- 30. Araya R, Menezes PR, Claro HG, Brandt LR, Daley KL, Quayle J, et al. Effect of a digital intervention on depressive symptoms in patients with comorbid hypertension or diabetes in brazil and peru: two randomized clinical trials. JAMA. 2021;325(18):1852–62. pmid:33974019
- 31. Ariza-Garcia A, Lozano-Lozano M, Galiano-Castillo N, Postigo-Martin P, Arroyo-Morales M, Cantarero-Villanueva I. A web-based exercise system (e-CuidateChemo) to counter the side effects of chemotherapy in patients with breast cancer: randomized controlled trial. J Med Internet Res. 2019;21(7):e14418. pmid:31342907
- 32. Baer HJ, Rozenblum R, De La Cruz BA, Orav EJ, Wien M, Nolido NV, et al. Effect of an online weight management program integrated with population health management on weight change: a randomized clinical trial. JAMA. 2020;324(17):1737–46. pmid:33141209
- 33. Baldwin PA, Sanatkar S, Clarke J, Fletcher S, Gunn J, Wilhelm K, et al. A web-based mental health intervention to improve social and occupational functioning in adults with type 2 diabetes (the springboard trial): 12-month outcomes of a randomized controlled trial. J Med Internet Res. 2020;22(12):e16729. pmid:33258790
- 34. Barnason S, Zimmerman L, Schulz P, Pullen C, Schuelke S. Weight management telehealth intervention for overweight and obese rural cardiac rehabilitation participants: a randomised trial. J Clin Nurs. 2019;28(9–10):1808–18. pmid:30667588
- 35. Baron JS, Hirani S, Newman SP. A randomised, controlled trial of the effects of a mobile telehealth intervention on clinical and patient-reported outcomes in people with poorly controlled diabetes. J Telemed Telecare. 2017;23(2):207–16. pmid:26880694
- 36. Baron JS, Hirani SP, Newman SP. Investigating the behavioural effects of a mobile-phone based home telehealth intervention in people with insulin-requiring diabetes: results of a randomized controlled trial with patient interviews. J Telemed Telecare. 2017;23(5):503–12. pmid:27377790
- 37. Compen F, Bisseling E, Schellekens M, Donders R, Carlson L, van der Lee M, et al. Face-to-face and internet-based mindfulness-based cognitive therapy compared with treatment as usual in reducing psychological distress in patients with cancer: a multicenter randomized controlled trial. J Clin Oncol. 2018;36(23):2413–21. pmid:29953304
- 38. Cohen LB, Taveira TH, Wu W-C, Pirraglia PA. Pharmacist-led telehealth disease management program for patients with diabetes and depression. J Telemed Telecare. 2020;26(5):294–302. pmid:30691328
- 39. Bellens A, Roelant E, Sabbe B, Peeters M, van Dam PA. A video-game based cognitive training for breast cancer survivors with cognitive impairment: a prospective randomized pilot trial. Breast. 2020;53:23–32. pmid:32554133
- 40. Bendig E, Bauereiss N, Schmitt A, Albus P, Baumeister H. ACTonDiabetes-a guided psychological internet intervention based on acceptance and commitment therapy (ACT) for adults living with type 1 or 2 diabetes: results of a randomised controlled feasibility trial. BMJ Open. 2021;11(7):e049238. pmid:34244277
- 41. Boels AM, Vos RC, Dijkhorst-Oei L-T, Rutten GEHM. Effectiveness of diabetes self-management education and support via a smartphone application in insulin-treated patients with type 2 diabetes: results of a randomized controlled trial (TRIGGER study). BMJ Open Diab Res Care. 2019;7(1):e000981.
- 42. Børøsund E, Ehlers SL, Varsi C, Clark MM, Andrykowski MA, Cvancarova M, et al. Results from a randomized controlled trial testing StressProffen; an application-based stress-management intervention for cancer survivors. Cancer Med. 2020;9(11):3775–85. pmid:32243717
- 43. Buscemi J, Oswald LB, Baik SH, Buitrago D, Iacobelli F, Phillips SM, et al. My health smartphone intervention decreases daily fat sources among Latina breast cancer survivors. J Behav Med. 2020;43(5):732–42. pmid:31970652
- 44. Chiang S-L, Shen C-L, Chen L-C, Lo Y-P, Lin C-H, Lin C-H. Effectiveness of a home-based telehealth exercise training program for patients with cardiometabolic multimorbidity: a randomized controlled trial. J Cardiovasc Nurs. 2020;35(5):491–501. pmid:32511110
- 45. Coombes JS, Keating SE, Mielke GI, Fassett RG, Coombes BK, O’Leary KP, et al. Personal activity intelligence e-health program in people with type 2 diabetes: a pilot randomized controlled trial. Med Sci Sports Exerc. 2022;54(1):18–27. pmid:34334715
- 46. Delrieu L, Anota A, Trédan O, Freyssenet D, Maire A, Canada B, et al. Design and methods of a national, multicenter, randomized and controlled trial to assess the efficacy of a physical activity program to improve health-related quality of life and reduce fatigue in women with metastatic breast cancer: ABLE02 trial. BMC Cancer. 2020;20(1):622. pmid:32620149
- 47. Deshpande S, Pinsker JE, Church MM, Piper M, Andre C, Massa J, et al. Randomized crossover comparison of automated insulin delivery versus conventional therapy using an unlocked smartphone with scheduled pasta and rice meal challenges in the outpatient setting. Diabetes Technol Ther. 2020;22(12):865–74. pmid:32319791
- 48. Dos Santos M, Hardy-Léger I, Rigal O, Licaj I, Dauchy S, Levy C, et al. Cognitive rehabilitation program to improve cognition of cancer patients treated with chemotherapy: a 3-arm randomized trial. Cancer. 2020;126(24):5328–36. pmid:32996583
- 49. Duan YP, Liang W, Guo L, Wienert J, Si GY, Lippke S. Evaluation of a Web-based intervention for multiple health behavior changes in patients with coronary heart disease in home-based rehabilitation: pilot randomized controlled trial. J Med Internet Res. 2018;20(11):e12052. pmid:30455167
- 50. Egbring M, Far E, Roos M, Dietrich M, Brauchbar M, Kullak-Ublick GA, et al. A mobile app to stabilize daily functional activity of breast cancer patients in collaboration with the physician: a randomized controlled clinical trial. J Med Internet Res. 2016;18(9):e238. pmid:27601354
- 51. Egede LE, Williams JS, Voronca DC, Knapp RG, Fernandes JK. Randomized controlled trial of technology-assisted case management in low income adults with type 2 diabetes. Diabetes Technol Ther. 2017;19(8):476–82. pmid:28581821
- 52. Engelen MM, van Dulmen S, Puijk-Hekman S, Vermeulen H, Nijhuis-van der Sanden MW, Bredie SJ, et al. Evaluation of a web-based self-management program for patients with cardiovascular disease: explorative randomized controlled trial. J Med Internet Res. 2020;22(7):e17422. pmid:32706708
- 53. Eyles H, McLean R, Neal B, Jiang Y, Doughty RN, McLean R, et al. A salt-reduction smartphone app supports lower-salt food purchases for people with cardiovascular disease: findings from the SaltSwitch randomised controlled trial. Eur J Prev Cardiol. 2017;24(13):1435–44. pmid:28631933
- 54. Franc S, Hanaire H, Benhamou P-Y, Schaepelynck P, Catargi B, Farret A, et al. DIABEO system combining a mobile app software with and without telemonitoring versus standard care: a randomized controlled trial in diabetes patients poorly controlled with a basal-bolus insulin regimen. Diabetes Technol Ther. 2020;22(12):904–11. pmid:32407148
- 55. Galiano-Castillo N, Cantarero-Villanueva I, Fernández-Lao C, Ariza-García A, Díaz-Rodríguez L, Del-Moral-Ávila R, et al. Telehealth system: a randomized controlled trial evaluating the impact of an internet-based exercise intervention on quality of life, pain, muscle strength, and fatigue in breast cancer survivors. Cancer. 2016;122(20):3166–74. pmid:27332968
- 56. Garnweidner-Holme L, Henriksen L, Torheim LE, Lukasse M. Effect of the pregnant+ smartphone app on the dietary behavior of women with gestational diabetes mellitus: secondary analysis of a randomized controlled trial. JMIR mHealth uHealth. 2020;8(11):e18614. pmid:33146620
- 57. Gimbel RW, Rennert LM, Crawford P, Little JR, Truong K, Williams JE, et al. Enhancing patient activation and self-management activities in patients with type 2 diabetes using the US Department of Defense Mobile Health Care Environment: feasibility study. J Med Internet Res. 2020;22(5):e17968. pmid:32329438
- 58. Golsteijn RHJ, Bolman C, Volders E, Peels DA, de Vries H, Lechner L. Short-term efficacy of a computer-tailored physical activity intervention for prostate and colorectal cancer patients and survivors: a randomized controlled trial. Int J Behav Nutr Phys Act. 2018;15(1):106. pmid:30376857
- 59. Handa S, Okuyama H, Yamamoto H, Nakamura S, Kato Y. Effectiveness of a smartphone application as a support tool for patients undergoing breast cancer chemotherapy: a randomized controlled trial. Clin Breast Cancer. 2020;20(3):201–8. pmid:32201165
- 60. Hauffman A, Alfonsson S, Bill-Axelson A, Bergkvist L, Forslund M, Mattsson S, et al. Cocreated internet-based stepped care for individuals with cancer and concurrent symptoms of anxiety and depression: Results from the U-CARE AdultCan randomized controlled trial. Psychooncology. 2020;29(12):2012–8. pmid:32691455
- 61. Hilmarsdóttir E, Sigurðardóttir ÁK, Arnardóttir RH. A digital lifestyle program in outpatient treatment of type 2 diabetes: a randomized controlled study. J Diabetes Sci Technol. 2021;15(5):1134–41. pmid:32680441
- 62. Höchsmann C, Infanger D, Klenk C, Königstein K, Walz SP, Schmidt-Trucksäss A. Effectiveness of a behavior change technique-based smartphone game to improve intrinsic motivation and physical activity adherence in patients with type 2 diabetes: randomized controlled trial. JMIR Serious Games. 2019;7(1):e11444. pmid:30758293
- 63. Holtdirk F, Mehnert A, Weiss M, Mayer J, Meyer B, Bröde P, et al. Results of the optimune trial: a randomized controlled trial evaluating a novel Internet intervention for breast cancer survivors. PLoS One. 2021;16(5):e0251276. pmid:33961667
- 64. Huang C-C, Kuo H-P, Lin Y-E, Chen S-C. Effects of a web-based health education program on quality of life and symptom distress of initially diagnosed advanced non-small cell lung cancer patients: a randomized controlled trial. J Cancer Educ. 2019;34(1):41–9. pmid:28780685
- 65. Huang F, Zhang S, Tian Y, Li L, Li Y, Chen X, et al. Effect of mobile health based peripartum management of gestational diabetes mellitus on postpartum diabetes: a randomized controlled trial. Diabetes Res Clin Pract. 2021;175:108775. pmid:33771645
- 66. Hummel SB, van Lankveld JJDM, Oldenburg HSA, Hahn DEE, Kieffer JM, Gerritsma MA, et al. Efficacy of internet-based cognitive behavioral therapy in improving sexual functioning of breast cancer survivors: results of a randomized controlled trial. J Clin Oncol. 2017;35(12):1328–40. pmid:28240966
- 67. Ionov MV, Zhukova OV, Yudina YS, Avdonina NG, Emelyanov IV, Kurapeev DI, et al. Value-based approach to blood pressure telemonitoring and remote counseling in hypertensive patients. Blood Press. 2021;30(1):20–30. pmid:32954832
- 68. Iversen MM, Igland J, Smith-Strøm H, Østbye T, Tell GS, Skeie S, et al. Effect of a telemedicine intervention for diabetes-related foot ulcers on health, well-being and quality of life: secondary outcomes from a cluster randomized controlled trial (DiaFOTo). BMC Endocr Disord. 2020;20(1):157. pmid:33087074
- 69. Jia W, Zhang P, Zhu D, Duolikun N, Li H, Bao Y, et al. Evaluation of an mHealth-enabled hierarchical diabetes management intervention in primary care in China (ROADMAP): a cluster randomized trial. PLoS Med. 2021;18(9):e1003754. pmid:34547030
- 70. Johnston N, Bodegard J, Jerström S, Åkesson J, Brorsson H, Alfredsson J, et al. Effects of interactive patient smartphone support app on drug adherence and lifestyle changes in myocardial infarction patients: a randomized study. Am Heart J. 2016;178:85–94. pmid:27502855
- 71. Kanera IM, Willems RA, Bolman CAW, Mesters I, Verboon P, Lechner L. Long-term effects of a web-based cancer aftercare intervention on moderate physical activity and vegetable consumption among early cancer survivors: a randomized controlled trial. Int J Behav Nutr Phys Act. 2017;14(1):19. pmid:28187725
- 72. Kario K, Morisawa Y, Sukonthasarn A, Turana Y, Chia Y-C, Park S, et al. COVID-19 and hypertension-evidence and practical management: guidance from the HOPE Asia Network. J Clin Hypertens (Greenwich). 2020;22(7):1109–19. pmid:32643874
- 73. Kerfoot BP, Gagnon DR, McMahon GT, Orlander JD, Kurgansky KE, Conlin PR. A team-based online game improves blood glucose control in veterans with type 2 diabetes: a randomized controlled trial. Diabetes Care. 2017;40(9):1218–25. pmid:28790131
- 74. Kim DY, Kwon H, Nam K-W, Lee Y, Kwon H-M, Chung YS. Remote management of poststroke patients with a smartphone-based management system integrated in clinical care: prospective, nonrandomized, interventional study. J Med Internet Res. 2020;22(2):e15377. pmid:32130140
- 75. Koehler F, Koehler K, Deckwart O, Prescher S, Wegscheider K, Kirwan B-A, et al. Efficacy of telemedical interventional management in patients with heart failure (TIM-HF2): a randomised, controlled, parallel-group, unmasked trial. Lancet. 2018;392(10152):1047–57. pmid:30153985
- 76. Kraai I, de Vries A, Vermeulen K, van Deursen V, van der Wal M, de Jong R, et al. The value of telemonitoring and ICT-guided disease management in heart failure: results from the IN TOUCH study. Int J Med Inform. 2016;85(1):53–60. pmid:26514079
- 77. Lee DY, Yoo S-H, Min KP, Park C-Y. Effect of voluntary participation on mobile health care in diabetes management: randomized controlled open-label trial. JMIR Mhealth Uhealth. 2020;8(9):e19153. pmid:32945775
- 78. Li A, Del Olmo MG, Fong M, Sim K, Lymer SJ, Cunich M, et al. Effect of a smartphone application (Perx) on medication adherence and clinical outcomes: a 12-month randomised controlled trial. BMJ Open. 2021;11(8):e047041. pmid:34373299
- 79. Lim SL, Ong KW, Johal J, Han CY, Yap QV, Chan YH, et al. Effect of a smartphone app on weight change and metabolic outcomes in asian adults with type 2 diabetes: a randomized clinical trial. JAMA Netw Open. 2021;4(6):e2112417. pmid:34081137
- 80. Ling D, Wang R, Chen Q, Liu X, Qi X, Chen C, et al. Assessment of chronic disease management mode (CDMM) on participants with primary hypertension. Trop Med Int Health. 2021;26(7):829–37. pmid:33780099
- 81. Lleras de Frutos M, Medina JC, Vives J, Casellas-Grau A, Marzo JL, Borràs JM, et al. Video conference vs face-to-face group psychotherapy for distressed cancer survivors: a randomized controlled trial. Psychooncology. 2020;29(12):1995–2003. pmid:32618395
- 82. Lozano-Lozano M, Martín-Martín L, Galiano-Castillo N, Fernández-Lao C, Cantarero-Villanueva I, López-Barajas IB, et al. Mobile health and supervised rehabilitation versus mobile health alone in breast cancer survivors: randomized controlled trial. Ann Phys Rehabil Med. 2020;63(4):316–24. pmid:31454561
- 83. Lu Y, Wang L, Wang H, Gu J, Ma ZJ, Lian Z, et al. Effectiveness of an impedance cardiography guided treatment strategy to improve blood pressure control in a real-world setting: results from a pragmatic clinical trial. Open Heart. 2021;8(2):e001719. pmid:34580169
- 84. Maguire R, McCann L, Kotronoulas G, Kearney N, Ream E, Armes J, et al. Real time remote symptom monitoring during chemotherapy for cancer: European multicentre randomised controlled trial (eSMART). BMJ. 2021;374:n1647. pmid:34289996
- 85. Mayberry LS, Mulvaney SA, Johnson KB, Osborn CY. The messaging for diabetes intervention reduced barriers to medication adherence among low-income, diverse adults with type 2. J Diabetes Sci Technol. 2017;11(1):92–9. pmid:27595710
- 86. McDermott MM, Spring B, Berger JS, Treat-Jacobson D, Conte MS, Creager MA, et al. Effect of a home-based exercise intervention of wearable technology and telephone coaching on walking performance in peripheral artery disease: the HONOR randomized clinical trial. JAMA. 2018;319(16):1665–76. pmid:29710165
- 87. Meltzer JA, Baird AJ, Steele RD, Harvey SJ. Computer-based treatment of poststroke language disorders: a non-inferiority study of telerehabilitation compared to in-person service delivery. Aphasiology. 2017;32(3):290–311.
- 88. Mihuta ME, Green HJ, Shum DHK. Web-based cognitive rehabilitation for survivors of adult cancer: a randomised controlled trial. Psychooncology. 2018;27(4):1172–9. pmid:29266524
- 89. Bohingamu Mudiyanselage S, Stevens J, Watts JJ, Toscano J, Kotowicz MA, Steinfort CL, et al. Personalised telehealth intervention for chronic disease management: a pilot randomised controlled trial. J Telemed Telecare. 2019;25(6):343–52. pmid:29793387
- 90. Nolan RP, Feldman R, Dawes M, Kaczorowski J, Lynn H, Barr SI, et al. Randomized controlled trial of E-counseling for hypertension: REACH. Circ Cardiovasc Qual Outcomes. 2018;11(7):e004420. pmid:30006474
- 91. Or CK, Liu K, So MKP, Cheung B, Yam LYC, Tiwari A, et al. Improving self-care in patients with coexisting type 2 diabetes and hypertension by technological surrogate nursing: randomized controlled trial. J Med Internet Res. 2020;22(3):e16769. pmid:32217498
- 92. Pekmezaris R, Nouryan CN, Schwartz R, Castillo S, Makaryus AN, Ahern D, et al. A Randomized controlled trial comparing telehealth self-management to standard outpatient management in underserved black and hispanic patients living with heart failure. Telemed J E Health. 2019;25(10):917–25. pmid:30418101
- 93. Peng X, Su Y, Hu Z, Sun X, Li X, Dolansky MA, et al. Home-based telehealth exercise training program in Chinese patients with heart failure: a randomized controlled trial. Medicine (Baltimore). 2018;97(35):e12069. pmid:30170422
- 94. Plotnikoff RC, Wilczynska M, Cohen KE, Smith JJ, Lubans DR. Integrating smartphone technology, social support and the outdoor physical environment to improve fitness among adults at risk of, or diagnosed with, type 2 diabetes: findings from the “eCoFit” randomized controlled trial. Prev Med. 2017;105:404–11. pmid:28887192
- 95. Poppe L, De Bourdeaudhuij I, Verloigne M, Shadid S, Van Cauwenberg J, Compernolle S, et al. Efficacy of a self-regulation-based electronic and mobile health intervention targeting an active lifestyle in adults having type 2 diabetes and in adults aged 50 years or older: two randomized controlled trials. J Med Internet Res. 2019;21(8):e13363. pmid:31376274
- 96. Prendergast HM, Petzel-Gimbar R, Kitsiou S, Del Rios M, Lara B, Jackson M, et al. Targeting of uncontrolled hypertension in the emergency department (TOUCHED): Design of a randomized controlled trial. Contemp Clin Trials. 2021;102:106283. pmid:33484897
- 97. Ramallo-Fariña Y, García-Bello MA, García-Pérez L, Boronat M, Wägner AM, Rodríguez-Rodríguez L, et al. Effectiveness of internet-based multicomponent interventions for patients and health care professionals to improve clinical outcomes in type 2 diabetes evaluated through the INDICA study: multiarm cluster randomized controlled trial. JMIR Mhealth Uhealth. 2020;8(11):e18922. pmid:33136059
- 98. Sarfo FS, Treiber F, Gebregziabher M, Adamu S, Nichols M, Singh A, et al. Phone-based intervention for blood pressure control among Ghanaian stroke survivors: A pilot randomized controlled trial. Int J Stroke. 2019;14(6):630–8. pmid:30465630
- 99. Still C, Margevicius S, Wright Jr J, Ruksakulpiwat S, Moore S. A pilot study evaluating the effects of a technology-based and positive psychological training intervention on blood pressure in African Americans with hypertension. J Prim Care Commun Health. 2021;1:1–11.
- 100. Sun C, Sun L, Xi S, Zhang H, Wang H, Feng Y, et al. Mobile phone-based telemedicine practice in older chinese patients with type 2 diabetes mellitus: randomized controlled trial. JMIR Mhealth Uhealth. 2019;7(1):e10664. pmid:30609983
- 101. Sunil Kumar D, Prakash B, Subhash Chandra BJ, Kadkol PS, Arun V, Thomas JJ. An android smartphone-based randomized intervention improves the quality of life in patients with type 2 diabetes in Mysore, Karnataka, India. Diabetes Metab Syndr. 2020;14(5):1327–32. pmid:32755831
- 102. Syrjala KL, Yi JC, Artherholt SB, Romano JM, Crouch M-L, Fiscalini AS, et al. An online randomized controlled trial, with or without problem-solving treatment, for long-term cancer survivors after hematopoietic cell transplantation. J Cancer Surviv. 2018;12(4):560–70. pmid:29730827
- 103. Tang YH, Chong MC, Chua YP, Chui PL, Tang LY, Rahmat N. The effect of mobile messaging apps on cardiac patient knowledge of coronary artery disease risk factors and adherence to a healthy lifestyle. J Clin Nurs. 2018;27(23–24):4311–20. pmid:29777560
- 104. Taylor AH, Taylor RS, Ingram WM, Anokye N, Dean S, Jolly K, et al. Adding web-based behavioural support to exercise referral schemes for inactive adults with chronic health conditions: the e-coachER RCT. Health Technol Assess. 2020;24(63):1–106. pmid:33243368
- 105. Thatthong N, Sranacharoenpong K, Praditsorn P, Churak P, Ponprachanuvut P, Srisangwan N, et al. Innovative tool for health promotion for at-risk Thai people with hypertension. J Public Health (Berl). 2019;28(4):437–43.
- 106. Thielbar KO, Triandafilou KM, Fischer HC, O’Toole JM, Corrigan ML, Ochoa JM. Benefits of using a voice and EMG-driven actuated glove to support occupational therapy for stroke survivors. IEEE Trans Neural Syst Rehabil Eng. 2017;25(3):297–305.
- 107. Thompson D, Al-Lamee R, Foley M, Dehbi HM, Thom S, Davies JE, et al. Achieving optimal adherence to medical therapy by telehealth: findings from the ORBITA medication adherence sub-study. Pharmacol Res Perspect. 2021;9(1):e00710. pmid:33570248
- 108. Tiede M, Dwinger S, Herbarth L, Härter M, Dirmaier J. Long-term effectiveness of telephone-based health coaching for heart failure patients: a post-only randomised controlled trial. J Telemed Telecare. 2017;23(8):716–24. pmid:27605214
- 109. Tobe SW, Yeates K, Campbell NRC, Maar MA, Perkins N, Liu PP, et al. Diagnosing hypertension in Indigenous Canadians (DREAM-GLOBAL): a randomized controlled trial to compare the effectiveness of short message service messaging for management of hypertension: main results. J Clin Hypertens (Greenwich). 2019;21(1):29–36. pmid:30474909
- 110. Uhm KE, Yoo JS, Chung SH, Lee JD, Lee I, Kim JI, et al. Effects of exercise intervention in breast cancer patients: is mobile health (mHealth) with pedometer more effective than conventional program using brochure? Breast Cancer Res Treat. 2017;161(3):443–52. pmid:27933450
- 111. Urech C, Grossert A, Alder J, Scherer S, Handschin B, Kasenda B, et al. Web-based stress management for newly diagnosed patients with cancer (STREAM): a randomized, wait-list controlled intervention study. J Clin Oncol. 2018;36(8):780–8. pmid:29369731
- 112. van der Hout A, van Uden-Kraan CF, Holtmaat K, Jansen F, Lissenberg-Witte BI, Nieuwenhuijzen GAP, et al. Role of eHealth application Oncokompas in supporting self-management of symptoms and health-related quality of life in cancer survivors: a randomised, controlled trial. Lancet Oncol. 2020;21(1):80–94. pmid:31838009
- 113. Vogel J, Auinger A, Riedl R, Kindermann H, Helfert M, Ocenasek H. Digitally enhanced recovery: Investigating the use of digital self-tracking for monitoring leisure time physical activity of cardiovascular disease (CVD) patients undergoing cardiac rehabilitation. PLoS One. 2017;12(10):e0186261. pmid:29020079
- 114. Vos JAM, Duineveld LAM, Wieldraaijer T, Wind J, Busschers WB, Sert E, et al. Effect of general practitioner-led versus surgeon-led colon cancer survivorship care, with or without eHealth support, on quality of life (I CARE): an interim analysis of 1-year results of a randomised, controlled trial. Lancet Oncol. 2021;22(8):1175–87. pmid:34224671
- 115. Wang Y, Li M, Zhao X, Pan X, Lu M, Lu J, et al. Effects of continuous care for patients with type 2 diabetes using mobile health application: a randomised controlled trial. Int J Health Plann Manage. 2019;34(3):1025–35. pmid:31368137
- 116. Watson M, White C, Lynch A, Mohammed K. Telephone-delivered individual cognitive behavioural therapy for cancer patients: an equivalence randomised trial. Psychooncology. 2017;26(3):301–8. pmid:27943570
- 117. Wen M-C, Kau K, Huang S-S, Huang W-H, Tsai L-Y, Tsai T-Y, et al. Smartphone education improves embarrassment, bowel preparation, and satisfaction with care in patients receiving colonoscopy: a randomized controlled trail. Medicine (Baltimore). 2020;99(46):e23102. pmid:33181675
- 118. Wentink MM, Berger MAM, de Kloet AJ, Meesters J, Band GPH, Wolterbeek R, et al. The effects of an 8-week computer-based brain training programme on cognitive functioning, QoL and self-efficacy after stroke. Neuropsychol Rehabil. 2016;26(5–6):847–65. pmid:27184585
- 119. Widmer RJ, Allison TG, Lennon R, Lopez-Jimenez F, Lerman LO, Lerman A. Digital health intervention during cardiac rehabilitation: a randomized controlled trial. Am Heart J. 2017;188:65–72. pmid:28577682
- 120. Willems RA, Bolman CAW, Mesters I, Kanera IM, Beaulen AAJM, Lechner L. Short-term effectiveness of a web-based tailored intervention for cancer survivors on quality of life, anxiety, depression, and fatigue: randomized controlled trial. Psychooncology. 2017;26(2):222–30. pmid:26988800
- 121. Wolf A, Fors A, Ulin K, Thorn J, Swedberg K, Ekman I. An eHealth diary and symptom-tracking tool combined with person-centered care for improving self-efficacy after a diagnosis of acute coronary syndrome: a substudy of a randomized controlled trial. J Med Internet Res. 2016;18(2):e40. pmid:26907584
- 122. Wong EM-L, Leung DYP, Chair S-Y, Sit JWH. Effects of a web-based educational support intervention on total exercise and cardiovascular risk markers in adults with coronary heart disease. Worldviews Evid Based Nurs. 2020;17(4):283–92. pmid:32772509
- 123. Wu C-JJ, Sung H-C, Chang AM, Atherton J, Kostner K, McPhail SM. Cardiac-diabetes self-management program for Australians and Taiwanese: a randomized blocked design study. Nurs Health Sci. 2017;19(3):307–15. pmid:28544230
- 124. Yaron M, Sher B, Sorek D, Shomer M, Levek N, Schiller T, et al. A randomized controlled trial comparing a telemedicine therapeutic intervention with routine care in adults with type 1 diabetes mellitus treated by insulin pumps. Acta Diabetol. 2019;56(6):667–73. pmid:30783823
- 125. Yasmin F, Nahar N, Banu B, Ali L, Sauerborn R, Souares A. The influence of mobile phone-based health reminders on patient adherence to medications and healthy lifestyle recommendations for effective management of diabetes type 2: a randomized control trial in Dhaka, Bangladesh. BMC Health Serv Res. 2020;20(1):520. pmid:32513164
- 126. Yu C, Liu C, Du J, Liu H, Zhang H, Zhao Y, et al. Smartphone-based application to improve medication adherence in patients after surgical coronary revascularization. Am Heart J. 2020;228:17–26. pmid:32745732
- 127. Zachariae R, Amidi A, Damholdt MF, Clausen CDR, Dahlgaard J, Lord H, et al. Internet-delivered cognitive-behavioral therapy for insomnia in breast cancer survivors: a randomized controlled trial. J Natl Cancer Inst. 2018;110(8):880–7. pmid:29471478
- 128. Zha P, Qureshi R, Porter S, Chao Y-Y, Pacquiao D, Chase S, et al. Utilizing a mobile health intervention to manage hypertension in an underserved community. West J Nurs Res. 2020;42(3):201–9. pmid:31057081
- 129. Zhou W, Chen M, Yuan J, Sun Y. Welltang—a smart phone-based diabetes management application—improves blood glucose control in Chinese people with diabetes. Diabetes Res Clin Pract. 2016;116:105–10. pmid:27321324
- 130. Batch BC, Spratt SE, Blalock DV, Benditz C, Weiss A, Dolor RJ, et al. General behavioral engagement and changes in clinical and cognitive outcomes of patients with type 2 diabetes using the Time2Focus mobile app for diabetes education: pilot evaluation. J Med Internet Res. 2021;23(1):e17537. pmid:33470947
- 131. Chiu C-J, Yu Y-C, Du Y-F, Yang Y-C, Chen J-Y, Wong L-P, et al. Comparing a social and communication app, telephone intervention, and usual care for diabetes self-management: 3-arm quasiexperimental evaluation study. JMIR Mhealth Uhealth. 2020;8(6):e14024. pmid:32484448
- 132. Chung I, Jung M, Park Y, Cho D, Chung H, Min Y. Exercise promotion and distress reduction using a mobile app-based community in breast cancer survivors. Front Oncol. 2020;9:132.
- 133. Jung H, Lee J-E. The impact of community-based eHealth self-management intervention among elderly living alone with hypertension. J Telemed Telecare. 2017;23(1):167–73. pmid:26678063
- 134. Kamoen O, Maqueda V, Yperzeele L, Pottel H, Cras P, Vanhooren G, et al. Stroke coach: a pilot study of a personal digital coaching program for patients after ischemic stroke. Acta Neurol Belg. 2020;120(1):91–7. pmid:31701472
- 135. Grašič Kuhar C, Gortnar Cepeda T, Kovač T, Kukar M, Ružić Gorenjec N. Mobile app for symptom management and associated quality of life during systemic treatment in early stage breast cancer: nonrandomized controlled prospective cohort study. JMIR Mhealth Uhealth. 2020;8(8):e17408. pmid:32427567
- 136. Lemelin A, Godbout A, Paré G, Bernard S. Improved glycemic control through the use of a telehomecare program in patients with diabetes treated with insulin. Diabetes Technol Ther. 2020;22(4):243–8. pmid:31657625
- 137. Mihuta ME, Green HJ, Shum DHK. Efficacy of a web‐based cognitive rehabilitation intervention for adult cancer survivors: a pilot study. Eur J Cancer Care. 2018;27(2):1–11.
- 138. Montero AR, Toro-Tobon D, Gann K, Nassar CM, Youssef GA, Magee MF. Implications of remote monitoring technology in optimizing traditional self-monitoring of blood glucose in adults with T2DM in primary care. BMC Endocr Disord. 2021;21(1):222. pmid:34758807
- 139. Nelson LA, Mulvaney SA, Gebretsadik T, Johnson KB, Osborn CY. The MEssaging for Diabetes (MED) intervention improves short-term medication adherence among low-income adults with type 2 diabetes. J Behav Med. 2016;39(6):995–1000. pmid:27488604
- 140. Paruchuri K, Finneran P, Marston NA, Healy EW, Andreo J Jr, Lynch R, et al. Outcomes of a smartphone-based application with live health-coaching post-percutaneous coronary intervention. EBioMedicine. 2021;72:103593. pmid:34657825
- 141. Shah M, Douglas J, Carey R, Daftari M, Smink T, Paisley A, et al. Reducing ER visits and readmissions after head and neck surgery through a phone-based quality improvement program. Ann Otol Rhinol Laryngol. 2021;130(1):24–31. pmid:32567391
- 142. Sundberg K, Lindström V, Petersson L-M, Langius-Eklöf A. Supporting health literacy using an interactive app for symptom management during radiotherapy for prostate cancer. Patient Educ Couns. 2020.
- 143. Sundberg K, Wengström Y, Blomberg K, Hälleberg-Nyman M, Frank C, Langius-Eklöf A. Early detection and management of symptoms using an interactive smartphone application (Interaktor) during radiotherapy for prostate cancer. Support Care Cancer. 2017;25(7):2195–204. pmid:28236145
- 144. Li G, Taljaard M, Van den Heuvel E, Levine M, Cook D, Wells G. An introduction to multiplicity issues in clinical trials: the what, why, when and how. Int J Epidemiol. 2017;46(2):746–55.
- 145. Lazzeroni LC, Ray A. The cost of large numbers of hypothesis tests on power, effect size and sample size. Mol Psychiatry. 2012;17(1):108–14. pmid:21060308
- 146. Odutayo A, Gryaznov D, Copsey B, Monk P, Speich B, Roberts C, et al. Design, analysis and reporting of multi-arm trials and strategies to address multiple testing. Int J Epidemiol. 2020;49(3):968–78. pmid:32176282
- 147. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337.
- 148. Campbell MK, Piaggio G, Elbourne DR, Altman DG, CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661. pmid:22951546
- 149. Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials. Am J Public Health. 2004;94(3):416–22. pmid:14998805
- 150. Christie J, O’Halloran P, Stevenson M. Planning a cluster randomized controlled trial: methodological issues. Nurs Res. 2009;58(2):128–34. pmid:19289934
- 151. Puffer S, Torgerson DJ, Watson J. Cluster randomized controlled trials. J Eval Clin Pract. 2005;11(5):479–83. pmid:16164589
- 152. Chidambaram AG, Josephson M. Clinical research study designs: the essentials. Pediatr Investig. 2019;3(4):245–52. pmid:32851330
- 153. Schmidt W-P. Randomised and non-randomised studies to estimate the effect of community-level public health interventions: definitions and methodological considerations. Emerg Themes Epidemiol. 2017;14:9. pmid:28912825
- 154. Glass KC, Waring D. Effective trial design need not conflict with good patient care. Am J Bioeth. 2002;2(2):25–6. pmid:12189068
- 155. Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 1: ethical and scientific issues. Ann Intern Med. 2000;133(6):455–63. pmid:10975964
- 156. Cunningham JA, Kypri K, McCambridge J. Exploratory randomized controlled trial evaluating the impact of a waiting list control design. BMC Med Res Methodol. 2013;13:150. pmid:24314204
- 157. Lim J, Walley R, Yuan J, Liu J, Dabral A, Best N, et al. Minimizing patient burden through the use of historical subject-level data in innovative confirmatory clinical trials: review of methods and opportunities. Ther Innov Regul Sci. 2018;52(5):546–59. pmid:29909645
- 158. Hall KT, Vase L, Tobias DK, Dashti HT, Vollert J, Kaptchuk TJ, et al. Historical controls in randomized clinical trials: opportunities and challenges. Clin Pharmacol Ther. 2021;109(2):343–51. pmid:32602555
- 159. Marion JD, Althouse AD. The use of historical controls in clinical trials. JAMA. 2023;330(15):1484–5. pmid:37768654
- 160. Forbes CC, Finlay A, McIntosh M, Siddiquee S, Short CE. A systematic review of the feasibility, acceptability, and efficacy of online supportive care interventions targeting men with a history of prostate cancer. J Cancer Surviv. 2019;13(1):75–96. pmid:30610736
- 161. GOLIȚĂ S, BĂBAN A. “A systematic review of the effects of internet-based psychological interventions on emotional distress and quality of life in adult cancer patients”. JEBP. 2019;19(2):47–78.
- 162. Iflaifel M, Hall CL, Green HR, Willis A, Rennick-Egglestone S, Juszczak E, et al. Widening participation–recruitment methods in mental health randomised controlled trials: a qualitative study. BMC Med Res Methodol. 2023;23(1):211.
- 163. Bethlehem J. Selection bias in web surveys. Int Statistical Rev. 2010;78(2):161–88.
- 164. De Man J, Campbell L, Tabana H, Wouters E. The pandemic of online research in times of COVID-19. BMJ Open. 2021;11(2):e043866. pmid:33622948
- 165. Akers L, Gordon JS. Using facebook for large-scale online randomized clinical trial recruitment: effective advertising strategies. J Med Internet Res. 2018;20(11):e290. pmid:30409765
- 166. Bybee S, Cloyes K, Ellington L, Baucom B, Supiano K, Mooney K. Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychol Sex. 2022;13(4):901–11. pmid:36439051
- 167. NICE. Behaviour change: individual approaches. 2014 [cited 2024 Oct 15]. Available from: https://www.nice.org.uk/guidance/ph49/chapter/Recommendations
- 168. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017;7(2):254–67. pmid:27966189
- 169. Kelders SM, van Zyl LE, Ludden GDS. The concept and components of engagement in different domains applied to eHealth: a systematic scoping review. Front Psychol. 2020;11:926. pmid:32536888
- 170. Short CE, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, et al. Measuring engagement in ehealth and mhealth behavior change interventions: viewpoint of methodologies. J Med Internet Res. 2018;20(11):e292. pmid:30446482
- 171. Jiang JM, Seng EK, Zimmerman ME, Sliwinski M, Kim M, Lipton RB. Evaluation of the reliability, validity, and predictive validity of the subscales of the perceived stress scale in older adults. J Alzheimers Dis. 2017;59(3):987–96. pmid:28671128
- 172. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med Care. 2008;46(3):266–74. pmid:18388841
- 173. Negeri ZF, Levis B, Sun Y, He C, Krishnan A, Wu Y, et al. Accuracy of the patient health questionnaire-9 for screening to detect major depression: updated systematic review and individual participant data meta-analysis. BMJ. 2021;375.
- 174. Richardson J, Sinha K, Iezzi A, Khan MA. Modelling utility weights for the assessment of quality of life (AQoL)-8D. Qual Life Res. 2014;23(8):2395–404. pmid:24719017
- 175. Desroches S, Lapointe A, Ratté S, Gravel K, Légaré F, Turcotte S. Interventions to enhance adherence to dietary advice for preventing and managing chronic diseases in adults. Cochrane Database Syst Rev. 2013;2013(2):CD007200.
- 176. Boutron I, Guittet L, Estellat C, Moher D, Hróbjartsson A, Ravaud P. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med. 2007;4(2):e61. pmid:17311468
- 177.
Keogh-Brown M, Bachmann M, Shepstone L, Hewitt C, Howe A, Ramsay CR, et al. Contamination in trials of educational interventions. 2007.
- 178. Richards D, Enrique A, Eilert N, Franklin M, Palacios J, Duffy D, et al. A pragmatic randomized waitlist-controlled effectiveness and cost-effectiveness trial of digital interventions for depression and anxiety. NPJ Digit Med. 2020;3:85. pmid:32566763
- 179. Steinert C, Stadter K, Stark R, Leichsenring F. The effects of waiting for treatment: a meta-analysis of waitlist control groups in randomized controlled trials for social anxiety disorder. Clin Psychol Psychother. 2017;24(3):649–60. pmid:27445199
- 180. Gunnarsson KU, McCambridge J, Bendtsen M. Reactions to being allocated to a waiting list control group in a digital alcohol intervention trial. Patient Educ Couns. 2023;107:107572. pmid:36442435
- 181. Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med. 2007;32(5 Suppl):S119-26. pmid:17466816
- 182. Park SH, Choi J-I, Fournier L, Vasey B. Randomized clinical trials of artificial intelligence in medicine: why, when, and how? Korean J Radiol. 2022;23(12):1119–25. pmid:36447410