Introduction

Randomised controlled trials (RCTs) are designed to minimise bias in evaluating healthcare interventions. When well executed, RCTs can provide credible evidence to help researchers determine the effectiveness of an intervention [1]. While the number of trials is growing [2], concerns are consistently raised about the quality of trials [3]. RCTs are often expensive and complex, and paediatric trials bring additional obstacles [4], including added ethical challenges [5], scarcity of funding [5,6], and recruitment barriers [2]. In 2015, a retrospective study of 559 paediatric RCTs found that 19% (n = 104) were discontinued early [6]. A 2022 review of 13,259 trials in people under 18 found that 11% (n = 903) were discontinued before completion due primarily to recruitment barriers [4]. Not only are paediatric trials more challenging to navigate and complete, but fewer are also conducted [2]. In an analysis of 4,146 RCTs, Groff et al. identified 14.2% (n = 591) of trials enrolled paediatric participants, 67.3% (n = 2794) enrolled adults, and 18.3% (n = 761) enrolled both [2]. The U.S. government and European Parliament have issued policies in the last two decades calling for more high-quality paediatric clinical trials to increase medicinal safety and efficacy for children [5,7,8]. Despite these initiatives, paediatric trials are still underfunded [6], conducted less frequently [2], and published less often [4,6].

One strategy increasingly used to address the challenges of conducting randomised trials is decentralising the trial [9–11]. A decentralised trial is an umbrella term that captures trial methods that occur away from a central point of trial conduct, i.e., collecting data via multiple mobile clinics instead of one large hospital, leveraging digital technologies (e.g., wearable devices), or moving a trial entirely online [12]. Decentralised trials offer wider-reaching recruitment opportunities [11], diversify a population sample [10], decrease the burden of involvement on participants [12], and research waste and trial costs [11,12]. These trials have existed for many years [9–12]; however, the COVID-19 pandemic has emphasised their potential. Many clinical trials were halted in early 2020, but Price et al. [13] found that by May 2020, participant recruitment in decentralised trials not only recovered from this forced pause but exceeded pre-pandemic recruitment. In a survey of 1,002 rheumatology patients in 2020, Mirza et al. [14] reported that over 75% of patients responded positively to decentralised interventions such as remote trial visits.

Despite their advantages, researchers and reviewers have expressed apprehension about the ability of decentralised, particularly entirely online, trial methods to produce reliable results [15]. Online trials require careful and critical planning to sample appropriate populations, avoid fraudulent data, and ethically incentivise participants [15]. While decentralised strategies promise to improve trial processes, not all trials lend themselves to complete decentralisation. Partially decentralised or ‘hybrid’ trials have grown in numbers in the last two decades [9]. Hybrid trials include an aspect of decentralisation combined with traditional methods [10,13].

In this paper, we are interested in fully online RCTs with children to inform the design and conduct of our own online trial with children. Given the absence of guidance for decentralised paediatric trials, this scoping review aims to (a) identify what methods and tools were used to conduct online trials with children and (b) determine the gaps in the knowledge. We describe recruitment methods, consent, data collection, compensation, loss to follow-up, and public and patient involvement (PPI) in online trials within paediatric populations. We also assess the risk of bias (RoB) for each trial and identify facilitators in the conduct of these trials.

Methods

The Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for Scoping Reviews (PRISMA-ScR) [16], the Joanna Briggs Institute’s Manual for Evidence Synthesis [17] and Peters et al. [18] guided the review process. The completed PRISMA-ScR checklist is available in S1 Checklist.

Results

Search results

We screened 6,957 titles and abstracts and excluded 6,550 records. We screened 407 full texts. Twenty full texts met the inclusion criteria. These records’ references were manually searched, and one additional full text was added, resulting in 21 studies identified that met the inclusion criteria. Fig 1 captures the information sources, search results, justifications for excluding full texts, and identification strategy for relevant studies in the PRISMA flow diagram [27].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA 2020 flow diagram.

https://doi.org/10.1371/journal.pone.0280965.g001

Characteristics of included studies

Table 1 displays the basic characteristics of all included trials [28–48]. The years of publication of the studies ranged from 2014 to 2022, where the majority (n = 14) were published in the last two years. There were four pilot trials. The number of participants ranged from 45 to 2,452. The pilot trials had, on average, fewer participants ( = 86) than the full trials ( = 560.7). The mean age of children who participated was 14.6 years. Of the 21 included trials, seven were single-session interventions, and one was unclear (Table 1). The remaining 13 trials ranged in duration from three- to ten- weeks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Overview of characteristics of included studies.

https://doi.org/10.1371/journal.pone.0280965.t001

Because of the breadth of our scoping review, the included studies are heterogeneous in design, aims, and topic of study. We categorised the trials into four groups: mental health (n = 13), substance use issues (n = 2), sexual health (n = 1), and public health and education (n = 5). Four trials collected data contributed by both caregivers and children. Four of the 21 trials reported on gender or sexual minority identification. Two trials intentionally enrolled only female participants, and one intentionally enrolled only males. Sixteen trials reported race or ethnicity (S1 Table). All 16 reported that most participants were White, the largest racial group in their respective countries of conduct [49–55].

Recruitment methods

Social media was the most used platform for recruiting participants (n = 15), and nine trials relied on it exclusively. Seven trials leveraged multiple methods for their recruitment. Four trials only used external, professional recruitment panels for their online recruitment, and one used email alone. Detailed recruitment methods for all trials are available in S2 Table.

Of the social media platforms, Facebook, or a combination of Facebook and one or more additional social media platforms, was used in 53.3% (n = 8/15) of trials. Seven of the 15 trials (46.7%) using social media used Instagram, exclusively or in combination with others. Four trials did not identify which social media platforms they used.

All seven trials using multiple recruitment methods employed an alternative form of online advertising to social media, i.e., banner ads on different web pages, search engine advertising, live-streaming events, and email listservs. Of these seven trials, six combined other online advertising with social media. Three trials used a dedicated website for recruitment, combined with at least one other method [29,34–35] (S2 Table). Table 2 illustrates the online platforms leveraged by the included trials and the social media platforms used.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Online platforms and specific social media platforms used for recruitment to trials.

https://doi.org/10.1371/journal.pone.0280965.t002

Eight trials combined their online recruitment with offline advertising of the trial [29,30,34,35,39,40,43,46]. Offline recruitment included advertisements in communities, through healthcare professionals, and schools. Offline recruitment details are available in S2 Table.

Compensation

Table 3 summarises the compensation offered to participants in the 14 trials that reported on compensation. The type of compensation differed among trials, but half reported using an e-gift card or e-voucher (n = 7). Five trials did not report the form of compensation, only the amount, and two offered a chance to win an e-voucher or other prize. Compensation ranged from a US $3.50 one-time payment to US $120 to complete all study requirements. None of the trials that required caregiver consent or included data submitted by both caregivers and children specified who received the compensation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Compensation type and amounts reported.

https://doi.org/10.1371/journal.pone.0280965.t003

Risk of bias (RoB) assessment

The complete RoB tables for each trial are available in S3 Appendix. Fig 2 displays the RoB summary data. The three domains where most assessments were ‘low’ risk were “random sequence generation” (n = 12), “selective reporting” (n = 14), and “other” bias (n = 17). ‘Unclear’ risk assessments were most common in “allocation concealment” (n = 11). The three domains with the most ‘high’ risk assessments were “blinding of participants and personnel” (n = 13), “blinding of outcome assessment” (n = 11), and “incomplete outcome data” (n = 11). The trial with the least risk of bias assessments was Schleider et al. [47] with six ‘low’ risk assessments. Four trials had five ‘low’ risk assessments across domains (Arnaud et al. [29], Ghaderi et al. [34], Nelson et al. [42], and O’Dea et al. [44]). The trial with the most (n = 5) uncertainty in risk of bias across domains was Schwinn et al. [48]. The trials with the highest risk of bias across domains (n = 5) were Manicavasagar et al. [39] and O’Connor et al. [43].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Risk of bias (RoB) assessment summary for all included trials.

https://doi.org/10.1371/journal.pone.0280965.g002

Discussion

We identified 21 trials with children conducted entirely online that met our inclusion criteria from 6,957 screened, highlighting the scarcity of published research in this area. We found that the reporting of recruitment, retention and public involvement methods in the included studies were not always sufficient to assess or replicate. Consent acquisition, data collection, and compensation methods were reported more thoroughly.

The use of social media as a recruitment tool was most prevalent among the studies, with specific studies noting that this approach significantly sped up their recruitment process [33,34,43,44,47]. However, apart from these five trials, there was no distinct information regarding the effectiveness of the recruitment methods used in the other studies. To facilitate better planning and implementation in future research, it would be advantageous if researchers could include details on the efficiency of their recruitment strategies in their reports, whenever feasible. Ten trials created dedicated study websites, but only three [29,34,35] reported using them in their recruitment efforts. Newly created websites only receive traffic from word-of-mouth, seed recruitment (where one participant is responsible for recruiting other participants), advertising via other websites or search engines, and offline media. Relying on dedicated study websites or emailing listservs exclusively generated the least amount of enrolment. Four trials used commercial recruitment platforms, which advertise that they can recruit rapidly and cost-effectively, but there are concerns they lack transparency on cost, bias, and potential over-use of a narrow participant pool [9,15].

A recent scoping review of 33 studies (1/33 was with children) examining recruitment to clinical trials via social media, traditional methods, and other online media reported that seven out of 20 trials had the highest recruitment rates with social media compared to other methods [56]. Since most social media platforms have a lower age limit of 13 [57], and the digital age of consent varies across countries [58], researchers need to consider both the legal and ethical implications of targeting social media recruitment toward children. Five trials in our review included children under 13 and reported using social media for recruitment [31,35,40,44,45]. Three of these trials [31,35,40] reported targeting parents on social media, and two [44,45] did not. Considering that all studies in our review received ethical approval, it is possible that social media advertising was targeted toward caregivers, but we are unable to make that assessment. Social media is an effective recruitment tool to reach adolescents who have higher social media usage than older adults [59], but not always appropriate for all paediatric populations. We identified a gap in the knowledge in the reporting of recruitment methods by many of the studies included in this review. There is a need for researchers to outline their recruitment plans and recruitment processes in more detail to contribute to the knowledge base.

Very little was reported on retention methods, and no analyses were done on the participant retention rates and engagement efforts. Online trials are often less burdensome for participants as they do not impose in-person interactions that may require travel, increased time commitments, and scheduling problems. This could result in higher participant retention; however, based on this review’s findings, we cannot tease out which methods used to engage and retain child participants might have most potential. Delineating recruitment and retention in trials can be challenging. When a website or app is part of a trial, their ease of use and visual impression affects how participants view the trial. Because of this, the technology and methods used for recruitment and delivering the trial intervention are linked intricately with a trial’s retention rates. There were trials in our study that had high-quality data and high retention of participants showing that choosing suitable strategies to undertake online trials with children is possible and appropriate. Nonetheless, there is a noticeable gap in the literature regarding the specific methods that facilitated high retention rates in these studies. Identifying and understanding these facilitators is crucial to enhancing future research and improving participant retention.

Our review found a lack of PPI input, and that input was not always sufficiently reported. Of the eight trials that sought feedback or input from children in the design of their trials, only one reported on how they recruited for their PPI group. Online advisory groups may be difficult to navigate logistically and therefore present a challenge to trial design; however, the involvement of PPI is key to increasing inclusivity and relevance of trial outcomes [60]. Further research is needed on how online children’s PPI groups are best compensated so that it can be integrated into trial design.

The trials in our review did not report on cost analyses or comparisons, so we cannot make any assessments regarding the cost of different recruitment methods in online trials with children and have therefore identified this as another gap in the knowledge. Cost comparisons would be useful for trialists, although it may be difficult to determine if the cost of different online recruitment and retention methods affect those outcomes. However, beyond relying on the public, professional and advocacy groups to ‘share’ the trial details, all social media advertising incurs a cost, as do advertisements on search engines, websites, and the use of external recruitment panels. In two recent reviews that examined cost-effectiveness of online recruitment methods [56,61], social media had a lower cost per enrolee than traditional enrolment methods.

Consent invitation processes varied, with e-consent links emailed or sent direct messages via social media to interested or potentially interested participants as the most common tool. Automated re-direction links to external websites were the second most common method. Ten trials [30–32,35,38,40,42,43,46,48] in our review created consent validation checks demonstrating creative and varied methods to increase their participants’ comprehension of the trial. Nelson et al. [42] confirmed capacity to consent using questions that asked participants to identify what the study was expecting of them, if they understood how their group allocation was decided, how to handle any distress they might experience, and to identify what risks might be present during the study. O’Connor et al. [43] added True/False questions about the study before consent/assent was given and incorrect answers activated a pop-up box that explained the correct answer.

A 2020 scoping review of 69 papers describing electronic informed consent in medical centres (three paediatric trials, one trial with both children and adults, 65 with adults) found no universal approach to gaining consent online, and the legal requirements for consent by country or region varied [62]. Nine of the 69 trials found that participants understood online consent more easily than traditional, paper consent forms and that online or e-consent forms may present a more streamlined approach in paediatric trials because in-person parental consent appointments often challenge parents and researchers [62].

Data collection methods were reported thoroughly in most of the trials included in our review. No challenges were reported in achieving adequate data protection measures, and external host websites appeared to fit this need for several studies. Where studies addressed fraudulent or duplicate entries (when a participant submits data or enters the trial more than once for incentive purposes or purely by mistake) this was addressed comprehensively and creatively, demonstrating several ways to decrease the likelihood of their occurrence [28,30,32,37]. Amsalem et al. [28] combined CAPTCHA questions, timers on questions and videos, and scanned for repeat or suspicious IP addresses. Bragg et al. [30] checked participants’ attentiveness by asking them to type a word into a box and excluded incorrect answers. Dobias et al. [32] required participants to answer data-quality questions that checked for attentiveness, English fluency, and at least three-word answers. Kelleher et al. checked for repeat IP addresses to identify multiple entries to their study [37]. These methods likely facilitated a decrease in the incidence of fraudulent and duplicate entries.

No barriers were reported regarding the online compensation of participants. E-gift cards or vouchers were the most common and acceptable compensation in the trials included here.

Based on the findings of this review, we have collated the following considerations when designing and conducting online trials with children:

• social media and ‘other’ online advertising expedite recruitment efforts in these populations, especially if parents or caregivers are the target recruitment pool
• use social media for paediatric populations above the age of 13
• combine methods of online recruitment to widen the participant pool, i.e. search engine advertising, social media advertising, school-, community-, healthcare facility-, and child-centric- website advertising
• consider conducting a SWAT (Study Within A Trial [63]) to help determine which recruitment methods best fit the needs of the trial
• consider transparency of sampling, randomisation and recruitment methods if employing commercial recruitment platforms
• involve PPI input as early as possible
• PPI group recruitment and involvement for online trials should occur online
• PPI group meetings should accommodate both children and parents’/caregivers’ schedules
• implement appropriate compensation for PPI partners, consider hourly rates for both parents/caregivers keeping local tax and labour laws in mind
• embed consent validation methods to ensure informed consent/assent (e.g., questions to participants to check for understanding, using timers on screens to encourage all information has been read or heard, double-checking IP addresses to decrease duplicate entries)
• check country and regional laws for online consent guidance
• fit-for-purpose external host websites store data securely and may assist in efficient data collection.

Conclusions

The methods, facilitators, and gaps in the knowledge of how online, randomised trials with children are conducted were identified and described; most trials used social media for multiple phases of their trials, although this may not always be the most suitable platform for these methods. More in-depth consideration of target age groups, population, and PPI input should inform the design and conduct of these trials, and more thorough reporting of the methods is needed in future studies.

Supporting information

Acknowledgments

This work contributes to one of the authors’ (SL) doctoral projects. The findings and conclusions in the document are those of the authors and not necessarily those of the authors employing organisations.