Introduction

Retention is a considerable challenge in randomised controlled trials [1–3]. Retention issues have been identified as one of the top research priorities in trials methodology [4] and require a solution that can be implemented globally. Poor participant retention impacts the quality and validity of the trial results, makes it difficult to draw accurate trial conclusions, causes trial delays and increases trial costs [1,5–8].

We previously conducted a scoping review [9] which highlighted the lack of communication of plans to promote participant retention in trial protocols. In that, we identified inconsistent reporting of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) item 18b. SPIRIT is a set of guidelines introduced in 2013 that provide a framework for designing and reporting randomised trials, and item 18b is specifically concerned with participant retention. While SPIRIT is widely endorsed as an international standard for trial protocols [10], no prior research exists on the effectiveness of the completion of SPIRIT item 18b in the protocol on participant retention in the corresponding trial report.

The purpose of this study was to examine if the completion of SPIRIT item 18b, i.e., evidence of planning for retention, in the protocol improves the retention of participants in the subsequent trial. We were also interested in knowing if those that don’t complete SPIRIT in their trial protocol report its use in their trial publication.

Materials and methods

As part of a previous scoping review [9] we collected trial protocols, some of which mentioned the use of the SPIRIT guidelines when developing the protocol and some of which did not. To compare whether the reporting of SPIRIT item 18b improves retention, we selected all the protocols that reported the use of the SPIRIT guidelines and randomly selected a matching group of protocols that did not report the use of SPIRIT guidelines. We searched PubMed and Medline for the corresponding trial results papers, published as a journal article. Where journal articles could not be accessed, the corresponding author of the trial protocol was contacted in an attempt to obtain the publication. In some instances, the trials were still actively recruiting or had not yet been completed, so, these journal articles could not be included. Fig 1 shows the PRISMA flow diagram for the selection of protocols and corresponding trial publications.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flow diagram of protocol and corresponding trial publication selection.

https://doi.org/10.1371/journal.pone.0327110.g001

Data extraction

All authors met and agreed the variables to be extracted from the trial publications. This included trial publications that mentioned using the SPIRIT guidelines (hereafter called ‘SPIRIT trials’) and those that did not mention using SPIRIT (hereafter called ‘non-SPIRIT trials). These are shown in Table 1. We defined a retention strategy as per our definition in our scoping review “an action/activity that is conducted with the purpose of retaining participants in a trial, reducing missing data or improving data completeness”(9:3). We were not concerned with extracting information regarding activities to improve adherence or compliance with an intervention.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Variables extracted from the journal articles.

https://doi.org/10.1371/journal.pone.0327110.t001

AOC trialled the data extraction from the trial publication with five ‘SPIRIT trials’ and five ‘non-SPIRIT’ trials. EM double extracted the same trials. Five discrepancies were identified and resolved by discussion. FS reviewed and approved the discussion on the five discrepancies. The process was repeated for a further ten trials (five each of the ‘SPIRIT trials’ and ‘non-SPIRIT’ trials). Six discrepancies were identified; these discrepancies were related to how to calculate the numbers recruited. It was agreed by the three authors that for the numbers recruited to the trial, we would capture the numbers present at the point of consent, but where that wasn’t available, the numbers present at the point of randomisation would be used. These numbers were located on the Consolidated Standards of Reporting Trials (CONSORT) [11] flow diagram presented in the trial publication. Where the CONSORT diagram was not present, the text was referred to, to obtain the relevant figures. Where discrepancies were identified between the text and the CONSORT flow diagram, the numbers in the CONSORT flow diagram took precedence. Due to data being imputed in many of the trials for the analysis, the figures for the ‘numbers retained’ were extracted from the component indicating direct follow-up of participants at the given time point for the primary outcome (or for further time points attributed to secondary or tertiary outcomes).

Results

In total we had 222 protocols and their corresponding trial publication, 115 of which mentioned the use of SPIRIT, ‘SPIRIT trials’, and 107 of which did not mention the use of SPIRIT, ‘non-SPIRIT’ trials.

The characteristics of the 506 trial protocols (n = 253 ‘SPIRIT’; n = 253 ‘non-SPIRIT’) are shown in Table 2. Of the 506 protocols published between 2014–2019, 56.5% (n = 286) were non-commercial and most tested non-drug interventions (68.8%, n = 348). The majority were individually randomised trials (83.6%, n = 423) and included a vast range of clinical specialities (n = 37), but the field of public health was the most common (15.8%, n = 80). Trial size also varied, ranging from 58 to 15,000 participants. The primary outcome was patient-reported in 35.6% (n = 180) of the protocols.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Protocol characterises.

https://doi.org/10.1371/journal.pone.0327110.t002

SPIRIT item 18b reporting in the trial publication

Table 3 reports the key findings concerning the use of SPIRIT items 18b (i), (ii) and (iii). Of the 222 trial publications (n = 115 ‘SPIRIT’; n = 107 ‘non-SPIRIT’) no publication included all aspects of item 18b (18b (i), (ii) and (iii) – “plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols”(10:3). Thirty-one-point-three percent (36/115) of ‘SPIRIT trials’ reported on item 18b(i), “plans to promote participant retention” in comparison to 1.9% (2/107) in the ‘non-SPIRIT trials’. Three-point-five percent (4/115) of ‘SPIRIT trials’ reported on item 18b(ii), “plans to complete follow-up including list of any outcome data to be collected for participants who discontinue from intervention protocols”, in comparison to 0% in the ‘non-SPIRIT trials’. Finally, no trials in either the ‘SPIRIT trials’ or ‘non-SPIRIT trials’ reported on item 18b(iii), “plans to complete follow-up including list of any outcome data to be collected for participants who deviate from intervention protocols”.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Key SPIRIT characteristics observed in trial publications.

https://doi.org/10.1371/journal.pone.0327110.t003

Trial retention

The logistic regression results show that the use of SPIRIT guidelines, item 18b, in the protocol did not improve participant retention for all reported trial outcomes as the confidence interval included the value of no association. When considered separately, findings for retention to the primary outcome were similar. For commercial trials, and academic trials, the results were no different as can be seen from the confidence intervals in Table 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Logistic regression.

https://doi.org/10.1371/journal.pone.0327110.t004

Discussion

Our analyses suggest that the reported use of the SPIRIT guidelines does not improve participant retention rates in trials, but this hides the reality of what is happening in practice. Despite 115 ‘SPIRIT trials’ (protocols that mentioned the use of SPIRIT guidelines) [10] when developing the trial protocol, often there was no evidence of SPIRIT item 18b in the resulting trial publication. Among the identified trial publications corresponding to protocols in the SPIRIT group, 31.3% (36/115) reported item 18b(i), 1.7% (2/115) reported item 18b(ii) and none of the journal articles reported item 18b(iii). This could be because the trial team did not consider participant retention during protocol development and therefore did not include a “plan to promote participant retention” (SPIRIT item 18b(i)) despite the guidance outlined in the SPIRIT guidelines, or that SPIRIT item 18b was considered but not reported by the trial team in the resulting trial publication. While SPIRIT is endorsed as an international standard for clinical trial protocols, its use is not mandatory, so thorough adherence to the guidelines may not be prioritised. Due to the lack of reporting of SPIRIT item 18b we are not able to determine if trial teams have in-fact used strategies to promote retention of participants in the trials. Neither can we thus determine if proactive strategies (plans outlined in the protocol that aim to actively promote participant retention in the trial) or reactive strategies (plans outlined by the trial team to complete outcome data collection and to complete follow-up of participants who have withdrawn/discontinued or deviated from the intervention protocols) were used, an issue we discussed in our scoping review previously [9]. This contributes to the issue of incomplete reporting and causes a lack of transparency in trial conduct [10,13–15]. This lack of reporting ultimately limits trial methodology researchers when attempting to carry out replications and evaluations of retention strategies which are very much needed due to the lack of evidence to support the effectiveness of these activities [1].

Further reporting issues were also identified in this paper. We found a disparity in the reporting of recruitment and retention figures, where seemingly, researchers interpret the ‘point of enrollment’ differently across several trials. We also found several disparities between the consort diagram and the text in the trial publication. Therefore, improved reporting is crucial for transparency and to facilitate readers in obtaining an unambiguous comprehension of what occurred throughout the trial. Ensuring consistency in reporting standards across trials will enhance the reliability and reproducibility of research findings, ultimately strengthening the evidence base for clinical practice.

It might also be the case that the SPIRIT guidelines for trial retention (item 18b) may lack specificity. Item 18b states “plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols”(10:3), however, it’s possible that more inexperienced trial teams are unsure what constitutes a retention strategy. For example, it might be standard practice in their clinical trial unit to phone trial participants to remind them of their upcoming visit. If it is an activity that is routine, perhaps they do not view it as a ‘retention strategy’. Both SPIRIT 2013 [10] and CONSORT (consolidated standards of Reporting Trials) 2010 [16] have been updated simultaneously with the aim of aligning reporting in both checklists and to provide unified guidance for reporting trial design, conduct and analysis from protocol development to final publication. The initial thought given the overlap between SPIRIT and CONSORT, particularly in methodological aspects of trial design, was that alignment would enhance practicality, facilitate implementation and improve efficiency [17]. The new guidelines have been published and while both SPIRIT 2025 [18] and CONSORT 2025 [19] have increased in length, little has changed to harmonise the planning (SPIRIT) and reporting (CONSORT) of recruitment strategies. The language has remained identical between item SPIRIT 2013 and SPIRIT 2025. SPIRIT 2025 Item 25b (18:38) “Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” states that planning is desirable, rather than essential, but acknowledges the effect on the trial analysis and interpretation of results “…desirable to plan for how retention will be promoted to prevent missing data and avoid the associated complexities in both study analysis (item 27c) and interpretation”. CONSORT 2025 has no reporting requirement for retention strategies used during the trial. Consort Item 22b requires information on losses and exclusions “For each group, losses and exclusions after randomisation, together with reasons” but does not expand this to seek information on efforts made to encourage trial participants to stay in the trial. It is perhaps a missed opportunity.

While a lack of SPIRIT guideline usage and/or reporting is a possible explanation of the resulting non-significance in this research, it is also important to consider the possibility that the retention strategies used in the trials were not effective enough to overcome the challenges associated with participant retention. There were a variety of different strategies mentioned across both the SPIRIT group and the non-SPIRIT group, for example, offering the control group active treatment following trial culmination [20], offering monetary incentives to participants at the last patient visit [21] or reminding participants of appointments via phone call or text message [22]. These commonly used strategies may not be the most effective. In fact, evidence from the Cochrane systematic review of retention strategies in trials shows that there is no high-certainty evidence to support any existing documented retention strategy [1]. This is quite stark.

Finally, we did not see any instances of all three items being completed (18b(i), (ii), and (iii)). It would be interesting to assess if all these elements of SPIRIT item 18b were completed, which would indicate forward planning for retention to the trial, what the impact on the overall retention to the trial would be. We cannot answer that now, but we call on all trial teams to plan strategies to improve retention to their trial early and document these plans. This will facilitate replication and evaluation in the future.

Conclusions

High levels of attrition in randomised trials can result in both poor internal and external validity [1,3,7,8,23,24], which can be detrimental to trial outcomes. While we cannot conclude from this research that SPIRIT item 18b has the capability of improving participant retention in trials, we believe this is attributable to the lack of evidence on the existing strategies being used, rather than a reflection on SPIRIT itself. We strongly encourage all trial teams to use the SPIRIT guidelines [10] as this contributes to the overall quality, completeness, transparency, and consistency of trial reporting [9,17,24,25]. Raising awareness of the need for early planning for participant retention along with the increased reporting of item 18b may well help to address the considerable challenge, that is participant retention [4].