https://orcid.org/0000-0002-6669-1110
Figures
Abstract
The use of a non-invasive fluorescence in situ hybridization (FISH)-based method on saliva for the detection of SARS-CoV-2 is evaluated in a proof-of-concept study and thereafter utilized in an outpatient setting with the Biotrack-MED® analyzer. For a proof-of-concept study, saliva samples were obtained from 28 persons with mild or moderate COVID-19-related symptoms who were tested RT-PCR positive or negative for SARS-CoV-2. In an outpatient setting, 972 individual saliva samples were utilized. All saliva samples were FISHed with a Cy3-labeled SARS-CoV-2-specific DNA probe and were analyzed manually by fluorescence microscopy (proof-of-concept) or with the SARS-CoV-2 application of the Biotrack-MED® analyzer, a semi-autonomous multi-sample filter cytometer. The proof-of-concept study showed a sensitivity of 96.0% and a specificity of 98.5% and is therefore comparable to the RT-PCR analysis of nasopharyngeal swabs. The outpatient setting showed a sensitivity of 90.9% and a specificity of 94.5% and seems therefore a valid assay for the detection of SARS-CoV-2 in individuals that are healthy, mild or moderate symptomatic. In conclusion, the method evaluated in this study, the FISH-based SARS-CoV-2 application of the Biotrack-MED® analyzer, is a sensitive and reliable assay for the detection of SARS-CoV-2 in the general population.
Citation: Tamminga GG, Jansen GJ, Wiersma M (2022) Evaluation of a fluorescence in situ hybridization (FISH)-based method for detection of SARS-CoV-2 in saliva. PLoS ONE 17(11): e0277367. https://doi.org/10.1371/journal.pone.0277367
Editor: Ralph A. Tripp, University of Georgia, UNITED STATES
Received: February 21, 2022; Accepted: October 25, 2022; Published: November 8, 2022
Copyright: © 2022 Tamminga et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript.
Funding: GGT, GJJ and MW are employed by the commercial company Biotrack. The funder provided support in the form of salaries for these authors but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: GGT, GJJ and MW have read the journal’s policy and these authors of this manuscript have the following competing interests: employed by the commercial company Biotrack and patents issued (WO 2010/040371 A1, EP 08874964.3) This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Coronavirus disease of 2019 (COVID-19), caused by the Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2), is a severe acute respiratory syndrome causing morbidity and mortality worldwide. In March 2020, the WHO declared COVID-19 as a pandemic [1]. By half October 2022, more than 621.797.00 confirmed infections and approximately 6.545.560 deaths are reported worldwide [2]. To stop the spread of the virus, countries have taken several control measures, including social distancing, restriction of group gatherings and lockdowns. Due to the pandemic and these pandemic-related measures, there is a major impact on health(care) and economy [3]. Early detection of infected persons will aid in stopping the COVID-19 pandemic.
At the moment, the reverse transcription polymerase chain reaction (RT-PCR), detecting the presence of intra- and extracellular viral nucleotides, is accepted by the WHO, the FDA and other national health organizations as being the ‘gold standard’ for detection of SARS-CoV-2 RNA. However, in the early stages of COVID-19, viral load may be below the detection limit of the RT-PCR, resulting in a false-negative result [4]. The RT-PCR is performed on respiratory samples of nasopharyngeal and/or oropharyngeal swabs, an invasive and laborious method that requires trained personnel, causes discomfort to the tested person and causes an infection risk for the trained personnel [5,6]. A less laborious method is the antigen test. Although this method shows a lower time to result, there appears to exist an alarming lack of sensitivity resulting in lower sensitivity and specificity scores. A person with a large load of SARS-CoV-2 will be tested positive with an antigen test, but a person with a small load of SARS-CoV-2 will not be detected and will be passed as negative, even though the person is contagious [7,8]. Therefore, a less invasive method that is also able to detect COVID-19 in the early stages is needed.
One option for a less invasive method is the use of saliva as a matrix for SARS-CoV-2 detection. Besides being less invasive, there is less aversion, the costs are lower and no trained personnel is necessary for sampling, thereby reducing the infection risk and increasing the number of individuals that can be tested [9]. Importantly, the salivary glands and saliva are both thought to be a SARS-CoV-2 infection site and transmission route in (pre-)symptomatic and asymptomatic individuals [10]. This is supported by the WHO, which states that SARS-CoV-2 spreads through aerosols from the mouth or nose of an infected individual [11]. Several studies have compared the results of RT-PCR on saliva versus nasopharyngeal swab and found that they correlate well in several settings (outpatient (healthy, asymptomatic, symptomatic) and inpatient), with saliva detecting sometimes up to 10% more positive individuals [10,12–16]. Prior to the COVID-19 pandemic, other respiratory viruses, including influenza A, influenza B and respiratory syncytial virus (RSV), have been detected in saliva with high sensitivity [17,18]. Saliva testing for the detection of SARS-CoV-2 is currently implemented in several countries, including South Korea, Japan, USA and Germany [12].
Previously, we described a pilot study to detect SARS-CoV-2 in saliva intracellularly by means of a fluorescence in situ hybridization (FISH)-based method [19]. Here, we first describe the manual evaluation of this method in mild to moderate symptomatic individuals as a proof-of-concept. Secondly, we show how this method operates automatically in an outpatient setting, utilizing the Biotrack-MED® analyzer as a sample reading platform.
Methods
Study population—Proof-of-concept
This study population consisted of 28 persons (88% men and 12% women) in the age of 18–70 year. Inclusion criteria were: ≥ 18 years of age, up to 2 days mild to moderate symptomatic (Table 1) at the time of saliva sampling and samples were taken between 1-10-2020 and 1-12-2020. In addition, all included persons have been tested for SARS-CoV-2 by a local Dutch Municipal Health Services by RT-PCR of a nasopharyngeal swab ± 1 day before or after the saliva sample collection. Ten persons (P1-P10) were SARS-CoV-2 RT-PCR positive, of whom 80% presented with mild symptoms and 20% with moderate symptoms. Eighteen persons (N1-18) were SARS-CoV-2 RT-PCR negative and all of these persons presented with mild symptoms.
Study population—Outpatient setting
This study population consisted of 972 persons (83% men and 17% women) in the age of 18–70 year, whom used the commercial services of Biotrack, Leeuwarden to detect the presence of SARS-CoV-2 in saliva between December 2020 and April 2021. These persons were either healthy, mild symptomatic or moderate symptomatic. Of these persons, no data was available whether they were RT-PCR tested by a local Dutch Municipal Health Services.
Ethics
All persons included used the commercial services of Biotrack, Leeuwarden to detect the presence of SARS-CoV-2 in their saliva and signed and approved for the use of their archived and fully anonymized saliva samples for research purposes. Since the saliva samples were anonymized diagnostic surplus material, no approval from the Medical Research Ethics Committee was needed, according to the Code of conduct for responsible use (Foundation Federation of Dutch Medical Scientific Societies).
Saliva sample collection
Saliva was obtained with a Biotrack saliva sample kit, containing instructions and a sample collection tube filled with 5 ml Biotrack Preservation Fluid. First, the person rinsed his/her mouth with mineral water and waited for 5 minutes. Next, the person added 1:1 saliva (5 ml) to the sample collection tube. After 10 minutes at room temperature, the saliva samples were fixed.
Pooled saliva samples–proof-of-concept
As the proof-of-concept study population consisted only of 28 persons (P1-P10 and N1-N18), the saliva samples of this study population were pooled to increase the dataset in order to reduce the risk of overfitting in the machine learning [20,21]. of the SARS-CoV-2 application of the Biotrack-MED® analyzer (see data acquisition and interpretation). P1-P10 were pooled 1:1 or 1:2 (v/v) according to the scheme of Table 2 resulting in 100 saliva samples. N1-N18 were pooled 1:1 or 1:2 (v/v) according to the scheme of Table 3 resulting in 324 saliva samples. In addition, the original samples (0 in Tables 2 and 3) were also measured.
Fluorescence in situ hybridization (FISH)
For the FISH procedure of the proof-of-concept, the Biotrack COV19 Probe directed at the antisense genomic RNA of the nucleocapsid gene of SARS-CoV-2 (+; sense probe, Cy3 labeled, 24 nucleotides, GC: 41.7%, Tm: 60.2°C, in silico specificity of 100%, LoD of 1.4 copies of viral RNA per 1 μl sample) [19], the Biotrack Wash Buffer (both part of the BTMED COV19 test kit) and microscopic slides with 10 reaction wells (VWR, The Netherlands) were used. The saliva samples were mixed 1:2 (v:v) with the Biotrack COV19 Probe, whereafter the mixtures were hybridized at 50°C for 1.5 hour. Next, 1 ml Biotrack Wash Buffer was added to the suspension and subsequently incubated during 30 min. at 60°C. Finally, the suspension was centrifuged (10 min. at 800 x g), the supernatant removed and the pellet resuspended in 50 μl Milli-Q. An aliquot of 10 μl from every saliva sample was added to a well on the microscopic slide, dried for 10 min. at 50°C and mounted with Fluoroshield (VWR, The Netherlands).
For the FISH procedure for the outpatient setting, a Biotrack proprietary test kit (BTMED COV19 test kit) was used, consisting of the Biotrack COV19 Probe, Biotrack Wash Buffer and filter cups. The same procedure as above was followed, only an aliquot of 30 μl from every saliva sample was added into a plastic sample container (a filter cup), which was fitted into an analysis disk, dried for 25 min. at 50°C and placed into the Biotrack-MED® analyzer for autonomous microscopic reading. The time-to-result for the whole procedure (FISH and Biotrack-MED® analysis) was approximately 3 hours for 18 samples, in which the FISH analysis is approximately 2,5 hours and the machine analysis (Biotrack-MED®) approximately 30 minutes.
Data acquisition and interpretation
The fluorescently labeled saliva samples of the proof-of-concept setting were manually analyzed by utilizing a Leica DM2500 fluorescence microscope equipped with a Leica EL6000 mercury lamp and a Leica DFC450C camera. Images with a magnification of 400x were taken using the software Leica application suite V4.20. The images were blinded and manually analyzed independently by three experienced researchers in a binary fashion, rendering the result as positive as at least one white blood cell contained 2 or more fluorescent spots. Thereafter, the images were used to train the multi-layer neural network of the SARS-CoV-2 application of the Biotrack-MED® analyzer, for interpretation of the samples of the outpatient setting.
The fluorescently labeled saliva samples of the outpatient setting were read and documented utilizing the Biotrack-MED® analyzer (patent EP 08874964.3, CE-IVD: NL-CA002-2020-51055, www.biotrack.nl), a semi-autonomous, multi-sample filter cytometer (Biotrack, Leeuwarden, The Netherlands). The Biotrack-MED® analyzer contains a multi-layer neural network that is trained for the interpretation of FISH-based images utilizing the SARS-CoV-2-specific probe, called the SARS-CoV-2 application, which is able to analyze approximately 36 samples per hour. This application produced approximately 150 microscopic fluorescence images (in Z-direction) at a magnification of 200x from which visual recognizable objects were isolated by a computer vision algorithm. Morphometric parameters (n = 20, including shape, area, roundness, brightness and number of spots) are calculated per object [22]. The morphometric parameter scores were subsequently entered in the neural network of the SARS-CoV-2 application, resulting in an autonomous generated test result for SARS-CoV-2 (positive or negative). For a positive result, at least one cell contained 2 or more fluorescent spots. The images generated by the Biotrack-MED® analyzer were blinded and manually analyzed independently by three experienced researchers.
Statistics
Sensitivity and specificity with 95% confidence intervals were calculated utilizing MedCalc (https://www.medcalc.org/calc/diagnostic_test.php).
Results
Proof-of-concept sensitivity and specificity
The fluorescently labeled saliva samples of the proof-of-concept study population were manually analyzed independently by three experienced researchers by means of visual inspection utilizing a fluorescence microscope. A sample was determined positive when multiple highly fluorescent spheres were present in white blood cells (Fig 1) and negative when no fluorescent signal was present. The dataset consisted of 424 samples, of which 100 were RT-PCR positive and 324 were RT-PCR negative for SARS-CoV-2. When analyzing these samples by FISH, 96 were positive, 319 were negative, 5 were false-positive and 4 were false-negative for SARS-CoV-2, compared with RT-PCR outcomes (Tables 4 and 5). This resulted in a sensitivity-score of 96.0% (95% CI 90.0%-98.9%) and a specificity-score of 98.5% (95% CI 96.4%-99.5%). Thus, this FISH-based test and the RT-PCR on nasopharyngeal swab yield comparable results regarding the presence of SARS-CoV-2 in a mild to moderate symptomatic population.
Fluorescently-labeled white blood cells (approximately 12–15 μm in size), positive for SARS-CoV-2, are shown at 400x magnification (a). The insert (b) shows a 630x magnification of a white blood cell positive for SARS-CoV-2, showing the highly fluorescent spheres that are typical for SARS-CoV-2 when applying this method.
Outpatient sensitivity and specificity
The fluorescently labeled saliva samples of the outpatient study population were analyzed by means of the SARS-CoV-2 application of the Biotrack-MED® analyzer (www.biotrack.nl), an analyzer specifically designed for FISH-based analyses. As visual inspection of microscopic images yielded reliable results comparable to the RT-PCR results in the proof-of-concept dataset, the results of the SARS-CoV-2 application were compared with the visual inspection of the microscopic images created during the analysis for all 972 samples of the outpatient study population.
The Biotrack-MED® analyzer generated a microscopic fluorescence image for each sample, from which morphometric parameters of the identified microscopic objects were calculated. By means of this information, the SARS-CoV-2 application of the Biotrack-MED® analyzer is able to interpret the microscopic image as being positive or negative for SARS-CoV-2. The saliva samples contained both epithelial cells and white blood cells and the SARS-CoV-2 positive samples showed a variance in the amount of virus-infected white blood cells (Fig 2).
Examples of a negative (a), weak positive (b), medium positive (c) and highly positive (d) test result. The arrows indicate white blood cells positive for SARS-CoV-2 in (b), (c) and (d). In the case of (d), almost all white blood cells appear to show SARS-CoV-2 particles. The images from the Biotrack-MED analyzer are 1392 x 1040 pixels, which converts to 448.9 x 335.4 μm. The insert (b, c and d) shows a positive white blood cell that is zoomed in from the original image. A white blood cell is approximately 12–15 μm in diameter.
The dataset of the outpatient setting consisted of 972 samples, of which 1 was excluded due to an undisclosed error on the Biotrack-MED® analyzer. When analyzing the remaining 971 samples on the Biotrack-MED® analyzer, applying the SARS-CoV-2 application, 140 were positive, 772 were negative, 45 were false-positive and 14 were false-negative compared to visual inspection (Table 6). This resulted in a sensitivity-score of 90.9% (95% CI 85.2%-94.9%) and a specificity-score of 94.5% (95% CI 92.7%-96.0%). Thus, the SARS-CoV-2 application of the Biotrack-MED® analyzer shows to be valid in the detection of SARS-CoV-2 in an outpatient setting with healthy, mild symptomatic and moderate symptomatic individuals.
Discussion
In this study, a FISH-based method to detect SARS-CoV-2 in saliva was first evaluated in a proof-of-concept setting. It was shown that this test has a high sensitivity (96.0%) and specificity (98.5%) when analyzing 100 RT-PCR positive and 324 RT-PCR negative saliva samples, which is according to the WHO guidelines (at least 100 RT-PCR positive and at least 300 RT-PCR negative samples for the method to be tested) [23]. Thereafter, the applicability of the method was tested in an outpatient setting utilizing the Biotrack-MED® analyzer, a reading platform specific for FISH-bases analyses. This resulted in a sensitivity of 90.9% and a specificity of 94.5%. Therefore we conclude that the FISH-based SARS-CoV-2 application of the Biotrack-MED® analyzer is a sensitive and reliable assay for the detection of SARS-CoV-2 in the general population.
FISH is an easy to use analytical method with a high sensitivity to detect specific RNA or DNA sequences and a relatively short time to result. It is a reliable technique that has been optimized in its 20+ years of existence [24,25]. Moreover, FISH has proven to be a valid an practical strategy in the direct in situ detection of SARS-CoV-2, utilizing SARS-CoV-2-specific and fluorescently-labeled oligonucleotide probes [26]. In the current evaluation study, a SARS-CoV-2-specific probe, directed at the nucleocapsid (N) gene, is utilized. This specific probe shows in silico no cross-reactivity with other viruses that may be present in saliva, such as influenza virus or rhinovirus. Using this N-gene-specific probe, SARS-CoV-2 infection in different disease states (mild to moderate) and asymptomatic persons could be detected. As also small amounts of virus could be detected (LoD of 1.4 copies of viral RNA per 1 μl sample), this FISH-based method shows a similar detection level in the early infections phases as compared with the RT-PCR. The current study did not examine whether a correlation existed between the viral load of the RT-PCR and the FISH-based method. This query is in this study not relevant, as the main aim was to evaluate the FISH-based method and whether it could detect an infected person.
The SARS-CoV-2 genomic RNA is single-stranded and has a sense orientation. During the replication of SARS-CoV-2, intermediate RNAs with an antisense orientation are formed [27]. The SARS-CoV-2 probe (COV19 Probe) utilized in this study is only binding to the SARS-CoV-2 intermediate RNA with the antisense orientation of the nucleocapsid gene. Therefore, the SARS-CoV-2 application described in this study detects SARS-CoV-2 when it is replicating and, thus, ‘active’ [19]. The RT-PCR is able to detect both intra- and extracellular sense genomic and antisense intermediate RNAs, although both techniques are able to detect SARS-CoV-2 presence, the SARS-CoV-2 application described here detects only the ‘active’ virus, the intermediate RNAs, while the RT-PCR also may only detect the genomic RNA, which gives information about the presence of the virus, but not of its replicative stage. This is confirmed by studies showing that SARS-CoV-2 could be detected by RT-PCR of nasopharyngeal swabs more than 21 days after disease onset [12]. In addition, the SARS-CoV-2 application detects only the intracellular virus (in white blood cells) and not the extracellular virus (free virions in the saliva). As the cells are the replication site for SARS-CoV-2, the use of the FISH-based SARS-CoV-2 application may yield (diagnostic) information about the viral activity.
The FISH-based method described in this study resulted in approximately 5% false positive and approximately 9% false negative results. The false positive results were most often due to contamination during sampling. For example, small food remnants may be mistaken by the SARS-CoV-2 application as positive white blood cells when the morphometrics are similar. However, a percentage false positive and false negative results are present in all diagnostic tests. In the case of the RT-PCR, percentages up to 6.9% and 58% were reported for false positive and false negative results, respectively [28–30]. The resulting sensitivity and specificity of the outpatient setting in this study are 90.9% and 94.5%, respectively. When comparing with the current ‘gold standard’ for SARS-CoV-2 detection, the RT-PCR, these percentages are comparable or higher than the RT-PCR, where the sensitivity ranges between 32–93% [30–33]. Thus, this FISH-based method in combination with the SARS-CoV-2 application of the Biotrack-MED® analyzer show comparable false positives and false negatives to the RT-PCR, which suggest it to be a reliable assay for the detection of SARS-CoV-2.
The matrix of choice in this study is saliva. For SARS-CoV-2 to enter and infect a cell, it must bind to ACE2 receptors. These receptors are found in the oral cavity, with (high) expression in the epithelial cells of the tongue and epithelial cells, T cells, B cells and fibroblasts of the oral mucosa [13]. In an early stage of the infection, the virus may only be present in the salivary glands, and from this reservoir the infection may spread throughout the body. This may result in the presence of SARS-CoV-2 in saliva in (very) early stages, while the virus is not yet present in the nose and/or throat, rendering the RT-PCR negative [10,34]. The SARS-CoV-2 presence in saliva is the highest around the time of symptom onset [13]. However, SARS-CoV-2 can also be detected in saliva in asymptomatic individuals [15]. As saliva is a matrix that is easily dispersed (in the form of droplets or aerosols, which are able to survive as long as 90 minutes) by a SARS-CoV-2 patient, persons in the vicinity may get infected by inhaling the droplets and/or aerosols or by bringing the droplets to the mouth, nose or eyes via the hands. This may occur also when the patients has no (asymptomatic) or (very) mild symptoms [13,14,34]. As described above, the probe utilized in this study only detects the ‘active’ virus, i.e. the virus in its replicative state. Therefore, this method may also be indicative of the contagiousness of the infected person. Further research should verify this latter statement. However, studies showed that RT-PCR on nasopharyngeal swabs were 20–30% more positive more than 21 days after disease onset compared to RT-PCR on saliva, suggesting that nasopharyngeal swabs detected historical infections, while saliva is more prone to detect an active infection [12]. In addition, other studies reported that there was less variation in SARS-CoV-2 RNA levels in saliva compared to a nasopharyngeal swab during the course of the infection [14,15]. This information is beneficial for both the economy and mental health as it may prevent unnecessarily long isolation periods of infected individuals.
An advantage of utilizing saliva instead of a nasopharyngeal swab is the easy and non-invasive means of sample collection. This also makes the evaluated method better applicable for children. However, there are some discrepancies between different studies comparing RT-PCR on nasopharyngeal swab and saliva. These discrepancies may be explained by differences in utilized protocols, type of saliva, variation in nasopharyngeal sampling, processing of samples, sensitivity of the utilized tests [14,15,35]. The saliva samples used in the current study are collected in sample collection tubes containing Biotrack Preservation Fluid, which fixates the saliva sample within 15 minutes after production, thereby rendering the possible present virus particles harmless for the technician handling the sample afterwards. In addition, due to the immediate fixation procedure upon collection, variables such as temperature, biological degradation and storage time are minimized. Moreover, as the person rinses his/her mouth before the collection of saliva, no influences of food and/or drinks are present.
A limitation of this study is the small amount of included persons in the proof-of-concept setting. For most of the persons that used the commercial services of Biotrack, Leeuwarden to detect the presence of SARS-CoV-2 in their saliva between 1-10-2020 and 1-12-2020, there was no information about whether the person did also a RT-PCR and/or the result of the RT-PCR. Of 28 persons, the result of the RT-PCR was known. Unfortunately, as the RT-PCR was performed by local Dutch Municipal Health Services, the result given was only negative or positive, without Ct-values. As 28 samples are too few to use for machine learning, this increases greatly the chance of overfitting. Therefore, we pooled the samples, to increase the dataset and reduce the change of overfitting in the machine learning of the SARS-CoV-2 application of the Biotrack-MED® analyzer.
In silico research showed no cross-reactivity of the COV19 Prove with other viruses that may be present in saliva, such as influenza virus or rhinovirus. However, this should also be tested in vitro. Nevertheless, in the winter 2020/2021 hardly any other viruses than SARS-CoV-2 were measured in the Dutch population [36], thus the positive results in this study are due to a SARS-CoV-2 infection.
Another limitation of this study is the inability of the COV19 Probe to distinguish between the different mutant strains of SARS-CoV-2, including the alpha-, beta- and omicron-variant. Although the utilized probe is not able to make a distinction between the strains, it is able to detect all the strains, both in silico and in vitro.
In conclusion, the evaluation of saliva samples demonstrated that the FISH-based SARS-CoV-2 application of the Biotrack-MED® analyzer is a reliable assay for detection of SARS-CoV-2 infection in an outpatient setting with healthy, mild-symptomatic and moderate-symptomatic individuals.
Acknowledgments
The authors wish to thank Noëlle ten Boer, Lucinda Asraf and Nikky LeZwijn for assistance with the experimental procedures and Willem van Wamel for critical reading of the manuscript.
References
- 1. WHO. WHO Director-General’s opening remarks at the media briefing on COVID-19–11 March 2020. In: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020.
- 2.
WHO. WHO Coronavirus (COVID-19) Dashboard. In: https://covid19.who.int/.
- 3. Sarkodie SA, Owusu PA. Global assessment of environment, health and economic impact of the novel coronavirus (COVID-19). Environ Dev Sustain. 2021;23. pmid:32837273
- 4. Xie X, Zhong Z, Zhao W, Zheng C, Wang F, Liu J. Chest CT for Typical Coronavirus Disease 2019 (COVID-19) Pneumonia: Relationship to Negative RT-PCR Testing. Radiology. 2020;296. pmid:32049601
- 5. Karlsson U, Fraenkel C-J. Covid-19: risks to healthcare workers and their families. BMJ. 2020. pmid:33115772
- 6. Kim Y, Yun SG, Kim MY, Park K, Cho CH, Yoon SY, et al. Comparison between Saliva and Nasopharyngeal Swab Specimens for Detection of Respiratory Viruses by Multiplex Reverse Transcription-PCR. J Clin Microbiol. 2017;55. pmid:27807150
- 7. Yamayoshi S, Sakai-Tagawa Y, Koga M, Akasaka O, Nakachi I, Koh H, et al. Comparison of Rapid Antigen Tests for COVID-19. Viruses. 2020;12. pmid:33322035
- 8. Scohy A, Anantharajah A, Bodéus M, Kabamba-Mukadi B, Verroken A, Rodriguez-Villalobos H. Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis. Journal of Clinical Virology. 2020;129. pmid:32485618
- 9. To KK-W, Tsang OT-Y, Yip CC-Y, Chan K-H, Wu T-C, Chan JM-C, et al. Consistent Detection of 2019 Novel Coronavirus in Saliva. Clinical Infectious Diseases. 2020;71. pmid:32047895
- 10. Huang N, Pérez P, Kato T, Mikami Y, Okuda K, Gilmore RC, et al. SARS-CoV-2 infection of the oral cavity and saliva. Nat Med. 2021;27. pmid:33767405
- 11.
WHO. Coronavirus disease (COVID-19). In: https://www.who.int/health-topics/coronavirus#tab=tab_1.
- 12. Tan SH, Allicock O, Armstrong-Hough M, Wyllie AL. Saliva as a gold-standard sample for SARS-CoV-2 detection. Lancet Respir Med. 2021;9. pmid:33887248
- 13. Xu R, Cui B, Duan X, Zhang P, Zhou X, Yuan Q. Saliva: potential diagnostic value and transmission of 2019-nCoV. Int J Oral Sci. 2020;12. pmid:32300101
- 14. Abasiyanik MF, Flood B, Lin J, Ozcan S, Rouhani SJ, Pyzer A, et al. Sensitive detection and quantification of SARS-CoV-2 in saliva. Sci Rep. 2021;11. pmid:34127708
- 15. Wyllie AL, Fournier J, Casanovas-Massana A, Campbell M, Tokuyama M, Vijayakumar P, et al. Saliva or Nasopharyngeal Swab Specimens for Detection of SARS-CoV-2. New England Journal of Medicine. 2020;383. pmid:32857487
- 16. Callahan C, Ditelberg S, Dutta S, Littlehale N, Cheng A, Kupczewski K, et al. Saliva is Comparable to Nasopharyngeal Swabs for Molecular Detection of SARS-CoV-2. Microbiol Spectr. 2021;9. pmid:34406838
- 17. To KK, Lu L, Yip CC, Poon RW, Fung AM, Cheng A, et al. Additional molecular testing of saliva specimens improves the detection of respiratory viruses. Emerg Microbes Infect. 2017;6. pmid:28588283
- 18. To KKW, Yip CCY, Lai CYW, Wong CKH, Ho DTY, Pang PKP, et al. Saliva as a diagnostic specimen for testing respiratory virus by a point-of-care molecular assay: a diagnostic validity study. Clinical Microbiology and Infection. 2019;25. pmid:29906597
- 19. Jansen GJ, Wiersma M, van Wamel WJB, Wijnberg ID. Direct detection of SARS-CoV-2 antisense and sense genomic RNA in human saliva by semi-autonomous fluorescence in situ hybridization: A proxy for contagiousness? PLoS One. 2021;16. pmid:34403446
- 20. van der Schaaf A, Xu C-J, van Luijk P, van’t Veld AA, Langendijk JA, Schilstra C. Multivariate modeling of complications with data driven variable selection: Guarding against overfitting and effects of data set size. Radiotherapy and Oncology. 2012;105: 115–121. pmid:22264894
- 21. Ying X. An Overview of Overfitting and its Solutions. J Phys Conf Ser. 2019;1168: 022022.
- 22. Tamminga GG, Paulitsch-Fuchs AH, Jansen GJ, Euverink G-JW. Development and validation of an alternative parameter for quantification of signals emitted by fluorescently labelled bacteria in microscopic images. J Microbiol Methods. 2019;166. pmid:31505168
- 23. WHO. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays–Interim guidance– 11 September 2020. Available: https://apps.who.int/iris/bitstream/handle/10665/334253/WHO-2019-nCoV-Antigen_Detection-2020.1-eng.pdf?sequence=1&isAllowed=y.
- 24. Frickmann H, Zautner AE, Moter A, Kikhney J, Hagen RM, Stender H, et al. Fluorescence in situ hybridization (FISH) in the microbiological diagnostic routine laboratory: a review. Crit Rev Microbiol. 2017;43. pmid:28129707
- 25. Levsky JM, Singer RH. Fluorescence in situ hybridization: past, present and future. J Cell Sci. 2003;116. pmid:12808017
- 26. Kusmartseva I, Wu W, Syed F, van der Heide V, Jorgensen M, Joseph P, et al. Expression of SARS-CoV-2 Entry Factors in the Pancreas of Normal Organ Donors and Individuals with COVID-19. Cell Metab. 2020;32. pmid:33207244
- 27. Ambike S, Cheng C-C, Feuerherd M, Velkov S, Baldassi D, Afridi SQ, et al. Targeting genomic SARS-CoV-2 RNA with siRNAs allows efficient inhibition of viral replication and spread. Nucleic Acids Res. 2022;50: 333–349. pmid:34928377
- 28. Braunstein GD, Schwartz L, Hymel P, Fielding J. False Positive Results With SARS-CoV-2 RT-PCR Tests and How to Evaluate a RT-PCR-Positive Test for the Possibility of a False Positive Result. J Occup Environ Med. 2021;63: e159–e162. pmid:33405498
- 29. Layfield LJ, Camp S, Bowers K, Miller DC. SARS-CoV-2 detection by reverse transcriptase polymerase chain reaction testing: Analysis of false positive results and recommendations for quality control measures. Pathol Res Pract. 2021;225: 153579. pmid:34385110
- 30. Sullivan A, Alfego D, Poirier B, Williams J, Adcock D, Letovsky S. Follow-Up SARS-CoV-2 PCR Testing Outcomes From a Large Reference Lab in the US. Front Public Health. 2021;9. pmid:34136460
- 31. Böger B, Fachi MM, Vilhena RO, Cobre AF, Tonin FS, Pontarolo R. Systematic review with meta-analysis of the accuracy of diagnostic tests for COVID-19. Am J Infect Control. 2021;49: 21–29. pmid:32659413
- 32. Watson J, Whiting PF, Brush JE. Interpreting a covid-19 test result. BMJ. 2020; m1808. pmid:32398230
- 33. Kortela E, Kirjavainen V, Ahava MJ, Jokiranta ST, But A, Lindahl A, et al. Real-life clinical sensitivity of SARS-CoV-2 RT-PCR test in symptomatic patients. PLoS One. 2021;16: e0251661. pmid:34019562
- 34. Smither SJ, Eastaugh LS, Findlay JS, Lever MS. Experimental aerosol survival of SARS-CoV-2 in artificial saliva and tissue culture media at medium and high humidity. Emerg Microbes Infect. 2020;9. pmid:32496967
- 35. Yee R, Truong TT, Pannaraj PS, Eubanks N, Gai E, Jumarang J, et al. Saliva Is a Promising Alternative Specimen for the Detection of SARS-CoV-2 in Children and Adults. J Clin Microbiol. 2021;59. pmid:33239380
- 36. National Institute for Public Health and the Environment. Annual report Surveillance of COVID-19, influenza and other respiratory infections in the Netherlands: winter 2020/2021. [cited 18 Oct 2022]. Available: https://www.nivel.nl/nl/publicatie/annual-report-surveillance-covid-19-influenza-and-other-respiratory-infections.