Ethics and consent

The study received ethical approval from the London – Bromley Research Ethics Committee (22/LO/0846) and Health Research Authority Confidentiality Advisory Group approval (22/CAG/0165) for data use, both on 19^th December 2022. Consent was not required for the analysis of pseudonymised data for staff or residents, but individuals could request to opt-out and have their data omitted from transfer to UCL and subsequent trial analyses. Staff testing was not mandatory.

Data sources

In order to minimise the burden on care home staff, much of the data for analysis were obtained from routinely collected healthcare information held within the UK COVID-19 Datastore. This included results of LFD and Polymerase Chain Reaction (PCR) tests for SARS-CoV-2 and information on resident hospital admissions (UK National Health Service’s Hospital Episode Statistics) and deaths (UK Office for National Statistics). Data within the COVID-19 Datastore are linked to a pseudonymised ID code for each individual, which could be linked to unique codes (Care Quality Commission IDs (CQC-IDs)) for participating care homes and associated staff or resident status through SARS-CoV-2 test records. We carried forward data-linkage of residents to specific homes for 1 year from the date of their latest SARS-CoV-2 test (whether or not this predated the trial period). Demographic data including age, sex, care home, and care home role were available from SARS-CoV-2 testing records which were entered by requestor during the test registration process (self or by care home staff member).

Data on hospital admissions and deaths are linked to ICD10 diagnostic codes within the COVID-19 Datastore. However, there is a lag of several months in the assignment of ICD10 codes to hospital admission data. To allow timely monitoring of the primary outcome and avoid the risk of missing relevant admissions, care providers were asked to upload records of COVID-associated hospital admissions at participating care homes to the COVID-19 Datastore. These data were also linked to pseudonymised ID codes for each individual.

The management teams of care home providers were asked to provide weekly aggregate data for each participating care home including the total number of residents, total number of staff and number of staff opting out of asymptomatic testing (for intervention arm), staff sickness absence and employment of agency staff, and number of residents with COVID-19-linked hospital admission (i.e., the sum of primary outcome events). Data from the UK COVID-19 Datastore were processed in the secure NHS Foundry system, within which relevant data records were identified for participating care homes and opt-out requests applied. Datasets for analysis were exported from Foundry to the UCL Data Safe Haven (a walled garden secure environment to ISO27001 standards) and collated with the aggregate data from Providers.

Outcome definitions

The trial team checked provider-supplied data on hospital admissions against routinely collected data prior to final analyses and without viewing intervention allocations. It was confirmed that primary outcome events would be defined, using routine data, as hospital admissions with ICD10 code for confirmed COVID-19 (‘U071’, not limited to primary code) and/or any admissions in residents who test positive for COVID-19 within 24h following admission or 7 days before. Repeat hospital admissions within 30 days were not included. COVID-associated mortality events were defined as death with any ICD10 code (not limited to primary) of confirmed COVID-19 and/or within 28 days of a positive SARS-CoV-2 test. All analyses of resident outcomes used weekly numbers of care home residents from Providers to inform the ‘exposure’ variable of person-days at risk, and incidence rates were expressed per 1000 person-years (/1kPY).

Sample size

Based on observational data from the VIVALDI study [18], we assumed that we would observe a cumulative incidence for the primary outcome of approximately 3.0% in the control arm, requiring a trial duration of 5–6 months, in combination with a conservative intraclass correlation coefficient (ICC) value of 0.01 and average care home size of 35 residents. We planned to recruit 280 homes randomised 1:1 to trial arms and, using a two-sided test at a 5% significance level, this would provide 84% power to detect a reduction in primary outcome due to the intervention to 1.9% (relative risk 0.63).

Randomisation and masking

The randomisation process was conducted separately for each Provider joining the study or for batches of homes joining the study from a Provider if required. Restricted randomisation was used to ensure a balance between arms by region and in the mean number of residents per home for each Provider; in this process a set of intervention allocations is selected at random from amongst the subset of possible allocation in which balance is within a pre-specified tolerance level (full details in randomisation plan). Randomisation was conducted by the Trial Statistician (O.S.) using the cvcrand package for R.

Staff and residents of participating care homes were not blinded to their intervention allocation. The Trial Statistician remained blinded to intervention allocation up until the final stages of analyses for the interim report. The same analysis code formed the basis for reporting of results after completion of the study. The final decision regarding the coding of the primary outcome based on the different data sources for the final analysis was taken based on a dataset with information on care homes and the associated intervention allocation for each resident removed.

Statistical analysis

Analyses for all outcomes were carried out on an ‘intention-to-treat’ basis according to the allocated trial arm of each home. If a Provider or individual care home entirely dropped out of the trial and provided no further data, then we considered the relevant homes to be no longer under follow-up. Analyses were conducted using Stata V18.0.

Incidence outcomes relating to hospitalisations and positive SARS-CoV-2 tests of individual residents (including primary outcome) were analysed on a weekly aggregate basis. Mortality incidence outcomes were analysed on a monthly basis, as only month of death was exported for analysis. Mixed effects negative binomial models, or Poisson models if convergence failed, were used for all these outcomes, with cluster-robust standard errors. Data for incidence outcomes in the first 2 weeks of involvement for each site were considered to represent a transition period for intervention sites and were omitted from the analysis. The main effect estimate was given by incidence rate ratios (IRRs), and the marginal incidence rate difference was also calculated.

For outcomes expressed as a proportion (e.g., staff testing per week), the analysis used mixed effects logistic regression applied to weekly data from each home, with cluster-robust standard errors. The main effect estimate was given by adjusted odds ratios (aORs), with marginal risk difference also calculated.

Analysis models for primary and secondary outcomes included adjustment for care home provider (grouping any independent homes), region, size (number of residents) at study entry, and calendar time. Care home size and calendar time were included as continuous variables for most outcomes, allowing for potentially non-linear adjustment using a 5-knot restricted cubic spline [24]. For mortality outcomes, adjustment for calendar time used a categorical variable for calendar month. For adjustments related to both calendar time and care home size, a simpler relationship (e.g., linear) was considered if the spline model did not converge. All mixed effects models included normal random intercept terms for each care home.

A single interim report prepared by the Trial Statistician was planned after 3–6 months of operating the intervention to allow the trial’s Data Monitoring and Ethics and Trial Steering Committees (DMEC and TSC) to assess the emerging benefit: risk ratio, the level of testing in each trial arm and projected power based on site recruitment and primary outcome event rate. The TSC reviewed site recruitment and staff testing data by arm alongside primary and key secondary outcomes in the control arm only, with outcome data by arm also provided to the DMEC. Further details of the planned analyses are given in supplementary S1–S3 Files.

Interpretation of trial outcomes

There was a nominally statistically significant reduction in all-cause hospital admissions between control and intervention homes. However, this should be interpreted with appropriate caution because weekly tests were logged by only 14% of staff in intervention homes, implying a level of testing too low to cause an appreciable impact, and the neutral primary analysis means that this result should be considered exploratory. Low uptake of testing was likely due to a combination of testing fatigue and the fact that the salience of the behavioural intervention, which primarily relied on financial incentives (sickness pay) to encourage staff to test, was reduced by lower than anticipated rates of COVID-19 in the community. There have also been increasing concerns about the adverse impacts of the continuing use of NPIs, more generally, on the physical and mental health of care residents and staff [28].

Despite the vulnerability of the care home population, with the exception of interventions to improve hand hygiene [29–31], there have been few trials of NPIs to reduce transmission of COVID-19 or other respiratory infections in this setting [14,32–35]. Substantial uncertainty remains regarding the effectiveness, cost-effectiveness, benefits, and harms of different disease control measures, and this inevitably contributed to delays in responding to the COVID-19 pandemic. To date, the most comprehensive evaluation of the effectiveness of SARS-CoV-2 testing for staff to prevent infections and severe outcomes in residents within the scientific literature is an observational study using data from 13,424 nursing homes in the USA [16]. In this study, asymptomatic testing was associated with a significant reduction in infections and COVID-19 related deaths in residents in the period before vaccines were available. It had no effect on resident outcomes in the period between vaccination rollout and the emergence of the Omicron (B.1.1.529) variant but was associated with a reduction in rates of infection in residents during the period of Omicron dominance [16]. Whilst this suggests a potential role for asymptomatic testing in staff to protect residents when there is a substantial risk of severe infection-related outcomes, the authors were unable to account for concurrent usage of other NPIs (except personal protective equipment), which makes it difficult to discern the direct effect of asymptomatic testing. This cohort study was also unable to evaluate the potential negative impacts of asymptomatic testing on resident’s well-being, physical and mental health. A report commissioned by UKHSA evaluated the SARS-CoV-2 testing programme in England between October 2020 and March 2022 within adult social care settings [17] and also concluded that higher levels of staff testing were associated with reduced incidence of infections and subsequent deaths among residents. Cost-effectiveness in preventing deaths was found to be highest in the period prior to completion of primary vaccination.

There was a discrepancy between the number of staff at intervention sites who reported opting out of testing based on information provided by the care home (mean 16.3%) and the number of test results that were logged in routinely collected data (mean weekly value 14.4% of staff), with these combined values falling far short of 100%. We do not know whether this was primarily due to staff testing but not logging their results or due to a lack of testing without staff members formally stating that they had opted out. It is likely that behaviours changed over the course of the trial. To the end of March 2022 fewer than 25% of LFDs distributed to care home settings resulted in a reported test result, so under-reporting of LFD use seems to have been a long-standing issue [17]. Care homes were willing to provide weekly aggregate data on staff and residents but they appear to have missed a substantial number of COVID-19-related resident hospital admissions when collating these data, and the planned data pipeline for secure upload of individual-level data on these events was not used by most providers.

Epidemiological context

When our trial was conceived, asymptomatic testing for staff was still recommended nationally in the UK. By the time we started, only those who were symptomatic were eligible for testing, and policy was further revised on 3^rd April 2023 to restrict symptomatic testing to only the minority of staff eligible for COVID-19 therapeutics. This change coincided with a marked decline in asymptomatic testing in intervention homes, highlighting the challenges of conducting research in a changing epidemiological and policy context. Hypothetically, it may have been preferable to have undertaken this trial 6–12 months earlier when the incidence of COVID-19 within care homes was higher [36], asymptomatic testing was still recommended as national policy, and the request to care providers would have been to opt out of rather than re-instate asymptomatic testing. This approach may have been more palatable to care home staff, but it would have raised ethical and practical challenges by contravening the public health guidance that was in place at the time of the trial and potentially leaving care homes exposed to claims of negligence, for example in the event of an outbreak.

Lessons learnt

Our successful ascertainment of trial outcomes demonstrates the feasibility of our novel use of routine health data for a cluster randomised trial in the context of residential and nursing care homes. This is an area where more high-quality randomised research is urgently needed to inform policy, but where such studies are very difficult to set up due to difficulties relating to recruitment of care homes [37] and individual residents, and a lack of embedded research staff and infrastructure. A key barrier to the delivery of any kind of research in care homes is a lack of staff capacity to support study delivery because staff are overstretched with high vacancy rates, and research is not typically part of the role for front-line carers. In England, studies conducted in the NHS benefit from support and direct financial incentives for recruitment via the NIHR Research Delivery Network, but this infrastructure is undeveloped in care homes. Cluster trials typically require large number of individual participants [38], making it very challenging to collect all of the data needed for a trial. The creation of a reliable pipeline for routine healthcare data of care home residents is therefore crucial for the development of future large-scale trials in this setting in the UK.

The preceding Vivaldi observational study (ISRCTN 14447421) began as a SARS-CoV-2 sero-prevalence study [18], but we realised that there was a unique opportunity to support the COVID-19 response in care homes by integrating routinely collected individual-level testing and vaccination data from residents and staff, linked to healthcare outcomes. This made it possible—for the first time ever in England—to establish an accurate database of care home residents and staff that could be used for COVID-19 research to directly inform policy in near real-time. The resulting cohort study included >300 care homes and individual-level data from >70,000 staff and residents, which we used to generate evidence on disease burden and severity [36], variants [39] and vaccine effectiveness [12,21]. For the present trial, we re-purposed the care home network and data infrastructure established through Vivaldi to rapidly set-up a pragmatic, cluster randomised trial.

We are developing a new pilot study, Vivaldi Social Care [40], with the goal to make it easier for care homes to participate in research studies (by keeping the workload associated with study participation to a minimum), to increase research impact and efficiency by enabling studies to progress at pace and scale, and to deliver research in partnership with the care sector. We are expanding the Vivaldi network to c.1500 care homes, and establishing new processes to access infection-related routinely collected data from residents in these homes. Through collaboration with care sector organisations (e.g., The Outstanding Society, Care England and others), and extensive engagement with residents, relatives, charities and care homes staff, we are gradually changing the culture and understanding of research in homes that are participating in this project. A key technical development for the Vivaldi Social Care project [40] is that the data pipeline will be based on identification codes (i.e., NHS numbers) of residents exported daily directly from the electronic record systems of participating care homes; this will provide an improved level of accuracy regarding admission and exit dates of residents, and will also function reliably without the need for regular SARS-CoV-2 testing records.

Most care homes are unfamiliar with research practices and procedures, and whilst the use of routinely collected data can greatly reduce the need for prospective data collection, it is nonetheless preferable to combine these approaches in a clinical trial where possible. A greater amount of researcher-led primary data collection within the homes would have given us a better understanding regarding, for example, true testing and reporting levels. However, this would have required a substantial increase in resource need for the trial and would have made recruitment and retention of homes more difficult. Overall, our findings underscore the need for innovative approaches to care home research that enable the rapid generation of evidence to inform policy. This should include investment in training and capacity building for care home staff, so they have the knowledge, skills and ring-fenced time to support both intervention delivery and data collection. Such investment is required to bring the sector in line with other areas of health care delivery.