Introduction

Source and contact tracing by public health professionals (manual contact tracing or MCT) is a well-known method to control the spread of communicable diseases, and was used during the 2020–2022 SARS-CoV-2 pandemic. However, for the first time, digital contact tracing (DCT) using smartphone applications was also introduced in many countries worldwide [1–3]. These apps differed in their levels of privacy-by-design, functionalities, and management of detected exposures. But they all had in common that a notification was triggered if the app user had been exposed to a person who had tested SARS-CoV-2-positive. In the Netherlands, the app was dubbed CoronaMelder. It was launched in the 8th month of the Dutch epidemic on 10 October 2020 (see S1 Text for a detailed history of the Dutch epidemic) [4,5].

DCT has several theoretical advantages over MCT and over testing because of having symptoms or having received a warning by an index case. These include identification and notification of more contacts (including contacts whom the app user does not know personally), reducing delays in becoming aware of an exposure or presymptomatic infection and getting tested, and increasing general awareness which might stimulate preventive behaviour [6]. Modelling studies from various countries in the first year of the pandemic suggested that DCT might have a prominent epidemiological impact, but that the impact depends on user uptake, use coverage, percentage of exposures detected, testing capacity and policies, and the specifics and delays of the tracing process [7–10]. However, one modelling group concluded that when less optimistic values for these parameters are used, the impact is expected to be in the single digits [11]. More empirical evidence is needed to properly quantify these factors for use in future mathematical transmission models, which are needed to guide policy decisions regarding future investments in DCT and MCT [1,2]. In this paper, we assessed whether DCT and MCT had added value in SARS-CoV-2 control during the 2020–2021 SARS-CoV-2 epidemic in the Netherlands. We focused on reasons for testing (including having received an exposure notification via DCT or MCT) and exposure-test intervals by reason for testing.

Materials and methods

In the Netherlands, the general public could get tested at public health service (PHS) test sites (from 1 June 2020 onwards), commercial test sites (between February 2021 and March 2022), and by self-testing (from 31 March 2021 onwards). Public testing was free-of-charge for specific indications, and initially included symptomatic individuals only. Free-of-charge testing of exposed but asymptomatic individuals with close contacts became possible from 1 December 2020 onwards: the initial recommendation was to test on the fifth day after exposure, but from 18 February 2021 onwards to also test as soon as possible after the exposure. Individuals who tested positive at public or commercial test sites were contacted by a PHS employee and asked about their recent contacts; until February 2022, individuals with a positive self-test were asked to do a repeat test at a PHS test site to confirm the result and to enter the PHS MCT programme. During times when the MCT programme was fully operational, PHS employees notified each reported close contact. However, MCT was regularly scaled down due to the programme being overwhelmed, and at those times, cases were asked to notify their close contacts themselves (S1 Text).

The CoronaMelder app was introduced on 10 October 2020 (technical explanations in S1 Text and S1 Fig). By the end of May 2021, almost 30% of the Dutch population had downloaded the CoronaMelder app but only around 18% were still actively using it [5]. For DCT to have epidemiological impact, all steps of the exposure-notification-action cascade should be optimised (Fig 1 and S1 Text): app uptake and active use, detection of exposures, quarantine and testing once an exposure is detected, and isolation and notification of contacts after testing positive. App users who tested positive could voluntarily upload that information (via a so-called PHS-key) from the app to a backend server, which subsequently resulted in notifications being sent to all phones with CoronaMelder in active mode that had been within 1.5 meters for at least 15 minutes within the last 14 days. PHS-keys could initially only be uploaded to the backend server after a PHS employee had entered the date of symptom onset or, if no symptoms present, the date of the positive test into the PHS-key. From 11 November 2021 onwards (which is after the three datasets used in this study had already been collected), app users could upload their PHS-key themselves.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The exposure-notification-action cascade during the SARS-CoV-2 epidemic in the Netherlands, 2020–2021.

https://doi.org/10.1371/journal.pdig.0000396.g001

See S1 Text for detailed explanations of each of these steps.

Statistical analyses

In all datasets, individuals could report multiple reasons for testing (Table 1 for an overview of these reasons). Symptoms could have been reported as a reason for testing, but anyone requesting a test or participating in one of the studies was also asked if they had any symptoms as a standalone question. These questions were asked at the time of test request in the PHS Amsterdam dataset and at the time of testing in the two RDT studies. We used a hierarchy to limit the number of reasons for testing categories in all analyses except the Weibull and tobit models (see below). The hierarchy in the PHS Amsterdam dataset was DCT notification, MCT notification, having symptoms without notification, and unknown. The hierarchy in the first and second RDT studies, was DCT notification, MCT notification, followed by other types of notifications and other types of reasons for testing (S2 Text). We used the last date of exposure/contact, and the date a test sample was taken, to calculate the exposure-testing interval in all datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Reasons for testing in the PHS Amsterdam, first RDT, and second RDT studies in the Netherlands, 2020–2021.

https://doi.org/10.1371/journal.pdig.0000396.t001

Reasons for testing over time were displayed graphically. Population characteristics by reason for testing were tabulated per data source and compared using Pearson’s Chi-squared test for categorical variables and Kruskal-Wallis test for continuous variables. Adjusted standardized residuals of statistically significant categorical variables (p<0.05) were subsequently compared to the critical value, with p-values adjusted by the Bonferroni method. Continuous variables that differed statistically significantly between groups were subsequently analysed by Dunn Kruskal-Wallis multiple comparison test, with p-values adjusted by the Bonferroni method. Mean exposure-testing intervals with standard deviations (SD) were calculated by reason for testing and by other population characteristics. Univariable and multivariable Weibull regression and tobit censored regression models were performed to estimate associations between characteristics and exposure-testing intervals [14,15]. Reason for testing was included in these models as an indicator variable to allow for reporting of multiple reasons per person. The outcome data were skewed, because testing peaked on day five after exposure, but were not censored. We therefore considered Weibull models the most appropriate but also present the tobit models (with 0 days as the lower value and 14 days as the upper value) as sensitivity analyses. In the PHS Amsterdam regression models, all individuals for whom an exposure-testing interval could be determined had been part of the MCT programme, and MCT was therefore not included as a variable. All statistical analyses were performed in R versions 3.6.3 or 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Reasons for testing

The main reason for testing over time was having symptoms, except in the first RDT study that only included asymptomatic close contacts (Table 1, Fig 2). In the PHS Amsterdam dataset, the overall percentage of test requests due to symptoms was 70.2% over the entire time period, but declined from a daily percentage between 73–97% in the period prior to 1 December 2020 to 60% on 31 May 2021; the overall percentage in the second RDT study that took place in the second quarter of 2021 was 45.2%. Having received a DCT notification, or having been contacted by the MCT programme, were minor reasons for testing in all three datasets over time. The percentages of test requests that listed DCT as one of the reasons was 1.5% in the PHS Amsterdam dataset, 2.4% in the second RDT study, and 7.1% in the first RDT study in asymptomatic close contacts. These percentages were 4.9%, 1.9%, and 12.3% for MCT, respectively. Direct notification by a housemate who tested positive or another index case (any other person who tested positive) was much more common than DCT and MCT notifications: 30% and 58.3% in the second and first RDT studies, respectively. In the first RDT study among asymptomatic close contacts, direct notification by index cases was consistently the main reason for testing over time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Reasons for testing over calendar time in the PHS Amsterdam, first RDT, and second RDT datasets in the Netherlands, 2020–2021.

Abbreviations: DCT = digital contact tracing; Index = a person who tested SARS-CoV-2 positive; MCT = manual contact tracing; PHS = public health service; RDT = rapid diagnostic test; Self = testing at one’s own initiative. In Fig 2A, the first vertical dashed line represents the launch of the CoronaMelder app and the second one the change in testing policy, allowing asymptomatic individuals to get tested after exposure to a close contact. Fig 2A is based on 908,060 tests by 560,775 individuals; 2B on 4,126 tests by 4,126 individuals; and 2C on 7,925 tests by 7,925 individuals.

https://doi.org/10.1371/journal.pdig.0000396.g002

Most test requests in the PHS Amsterdam dataset (72.2%) and the second RDT study (62.3%) included symptoms as one of the reasons for testing (S1 Table and S3 Table). In the first RDT study, 9.2% of asymptomatic close contacts developed symptoms by the time of testing (S2 Table). Furthermore, a substantial proportion of test requesters who reported a DCT or MCT notification also indicated the presence of symptoms (36% or 31% in the PHS Amsterdam dataset and 20% or 28% in the second RDT study, respectively). Test population characteristics (such as age and gender) by reasons for testing are shown in S1–S3 Tables and described in S3 Text.

Weibull regression models with exposure-testing interval as outcome

The univariable Weibull models showed that testing after having received a DCT notification was associated with a statistically significantly 4–7% longer exposure-testing interval compared to all other reason for testing groups in all three datasets, but this only remained significant in the multivariable models of the PHS Amsterdam MCT subset and second RDT study (8% and 10%, respectively; Tables 2–4). In uni- and multivariable Weibull models, testing after MCT notification was associated with a statistically significantly longer interval in the second RDT study (6% and 11%, respectively), testing after index case notification with a shorter interval in the first RDT study (5% and 5%, respectively), testing after a household notification with a shorter interval in the second RDT study (14% and 14%, respectively), and testing at one’s own initiative with a longer interval in the second RDT study (9% and 11%, respectively). In the PHS Amsterdam MCT subset, individuals who were registered as cases or as household contacts had shorter exposure-testing intervals than non-household close contacts with long or short duration of contact (Table 2). In the multivariable Weibull model excluding cases, testing after a household contact was associated with a significantly shorter interval than testing after a non-household close contacts with long or short duration of contact (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results of Weibull regression models for the interval between last exposure and testing in days–PHS Amsterdam MCT subset.

https://doi.org/10.1371/journal.pdig.0000396.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Results of Weibull regression models for the interval between last exposure and testing in days–first RDT study.

https://doi.org/10.1371/journal.pdig.0000396.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Results of Weibull regression models for the interval between last exposure and testing in days–second RDT study.

https://doi.org/10.1371/journal.pdig.0000396.t004

In the PHS Amsterdam MCT subset, testing because of symptoms was associated with statistically significantly shorter exposure-testing intervals (13–16%) in all regression models (Table 2). In the second RDT study, intervals were not available for individuals who tested because of having symptoms. In the group who tested because of having been exposed to a positive individual, reporting symptoms was not significantly associated with a shorter interval in the univariable and/or multivariable models but trends in that direction were apparent (Table 4).

In the PHS Amsterdam MCT subset, individuals who ended up testing positive had a 11% shorter exposure-test interval than those who ended up testing negative (Table 2). Similar trends were seen in the first and second RDT study (Tables 3–4), but only reaching significance in the second RDT study (7% shorter in univariable and 5% shorter in the multivariable models). In the second RDT study, COVID-19 vaccination status and previous SARS-CoV-2 infection were not significantly associated with exposure-testing intervals (Table 4). The tobit censored regression model results are shown in S7–S9 Tables and described in S3 Text.

Discussion

We found that the epidemiological impact of both DCT and MCT were limited during the Dutch 2020–2022 SARS-CoV-2 epidemic. Only around 2% of all tests took place after a DCT notification, and 2–5% after a MCT notification depending on MCT capacity at the time. Additionally, test positivity among those testing after a DCT notification was significantly lower than among individuals testing for other reasons, the exposure-test intervals were longer after a DCT or MCT notification than for other reasons for testing, and about 20–36% of those who had received a DCT or MCT notification had symptoms at the time of test request and might have tested anyway even without having received the notification. These findings are in line with a self-evaluation performed by the Dutch national PHS (GGD-GHOR) that found that the majority of tests at PHS test sites were not triggered by MCT or DCT notification, and test positivity was lower than average after DCT notification [16].

It has been estimated that almost 30% of the Dutch population had downloaded the CoronaMelder app, but that only around 18% were still actively using it, by the end of May 2021 [5]. These percentages were similar to those in other countries that rolled out voluntary DCT: DCT apps were estimated to have been downloaded by 28% of the Norwegian population [17], 28% of the UK population [18], 26% of the Swiss population (declining to 22% active users over time) [19], 17% of the Canadian population [20], and use was reported by 21% of cases in Finland and 22% of cases in New South Wales, Australia [21]. Direct comparisons between epidemiological impact study results in these countries are not possible because of differences in how the contact-tracing apps were embedded within existing public health infrastructures, availability of data, and the degree of empirical evidence versus mathematical modelling used to estimate impact. However, overall, they showed only modest [17,18,22] to limited impact [5,21,23–25], with results being least pronounced when conclusions were predominantly based on empirical evidence.

Most impact studies cited low uptake as the main reason for the limited DCT impact, and several barriers to adoption have been identified: concerns about cybersecurity and privacy; lower levels of education, income, expectation about efficacy/perceived benefits, self-efficacy, and trust in government; higher age; and living in a community with lower levels of use [5,26–31]. Several of these barriers are amenable to intervention, such as education and promotion campaigns. These campaigns were suboptimal in the Netherlands [5]. In addition, combining DCT apps with other practical functionalities might have improved uptake. For example, the UK National Health Service (NHS) COVID-19 app included multiple functionalities and fared better than CoronaMelder: it was still regularly being used by around 28% of the British population by the end of December 2021 [18]. These functionalities included a symptom checker connected to test requesting, a QR-scanner for checking into certain venues, and current government SARS-CoV-2 prevention guidelines in the local area. The Dutch government considered combining the functionalities of the CoronaMelder and CoronaCheck apps (to generate vaccination, test, and/or cure certificates) into one app, as has been done by other European governments such as Germany (Corona-Warn-App), Austria (The Stopp Corona App), and France (TousAntiCovid) [32], but eventually decided against this. This may have been a missed opportunity. It has been estimated that the CoronaCheck app had been downloaded by 90% of the Dutch population by 1 March 2022 [33].

Our analyses suggest that the PHS MCT programme did not contribute much to epidemic control either. Unfortunately, we could not compare these findings to the above-mentioned impact studies because those studies did not place DCT in the wider context of other reasons for testing. In our study, most people in all three datasets tested because of having symptoms or because they themselves found out that someone in their social circle had tested positive. While MCT will likely continue to play an important role in the control of infectious diseases, its utility in the context of respiratory viruses that rapidly transmit within general populations should be re-evaluated. For example, instead of trying to maintain MCT at population level after containment has failed (which was regularly the case in the 2020–2022 Dutch SARS-CoV-2 epidemic), more focussed uses might be more appropriate and cheaper.

Unexpectedly, we found that DCT and MCT did not shorten delays between last exposure and testing compared to symptom-based testing. We hypothesise that suboptimal testing policies and CoronaMelder procedures are partly to blame for this. Between 1 December 2020 and 18 February 2022, exposed but asymptomatic individuals were advised to test five days after exposure, hence the fairly uniform distribution of 5-day intervals in the first RDT study. Furthermore, when a CoronaMelder user tested positive, they had to wait for a PHS employee to contact them to be able to notify their contacts. A Swiss study reported that non-household MCT contacts who had also received a SwissCovid app notification generally went into quarantine a day earlier than non-household MCT contacts who had not received one [34]. Nevertheless, the Swiss team also identified unnecessary delays due to several bottlenecks in the exposure-notification-action cascade, such as delayed delivery of codes for test result sharing, complex user interfaces, and misaligned incentives for subsequent mitigation behaviours [19]. The impact of longer exposure-testing intervals on SARS-CoV-2 transmissions also depends on whether individuals followed quarantine advice. We did not have any empirical data to evaluate this.

The main limitations of our study relate to data availability and quality. To safeguard privacy, the only publicly accessible CoronaMelder-generated data are anonymous data on app downloads and active use (the number of smartphones connecting with the backend server) [5]. We therefore had to use proxy data sources. First, all three datasets that we used consisted mostly of self-reported data, which can suffer from misreporting and/or recall bias. For example, the last exposure date was not always available. Individuals who booked a test online had to consistently answer questions about CoronaMelder use to be able to make a booking, but these questions were less consistently asked and recorded when individuals booked a test by phone or within the MCT process [35]. Second, test policies evolved over time. We already mentioned the day-5 test policy for individuals without symptoms. In addition, from 31 March 2021 onwards, self-testing became increasingly available and popular. The effects of this on our analyses are, however, limited because the three studies included in this report ran until mid-June 2021, before self-testing really took off. Third, we relied heavily on HPZone data for some of the PHS Amsterdam analyses, which limited the study population to individuals who had participated in the MCT programme. Furthermore, MCT capacity and guidelines fluctuated over time. Finally, we attempted to control for confounding in the multivariable models, but residual confounding cannot be excluded. Despite these limitations, we think that our empirical results are useful for the calibration of modelling studies.

We conclude that both DCT and MCT had limited impacts on the Dutch 2020–2022 SARS-CoV-2 epidemic. However, in future epidemics, the impact of DCT might be improved by concerted efforts to increase app use coverage, elimination of contact-tracers from the digital notification process to minimise delays, and ensuring that everyone who is notified can get tested right away, including by self-testing.

Supporting information

Acknowledgments

We thank our colleagues at the various Public Health Services and associated laboratories, the Julius Center for Health Sciences and Primary Care at the University Medical Center Utrecht, the National Institute for Public Health and the Environment (RIVM), and the Erasmus University Medical Center who contributed to the studies described in this paper.