SARS-CoV-2 antibody tests are available in various formats, detecting different viral target proteins and antibody subclasses. The specificity and sensitivity of SARS-CoV-2 antibody tests are known to vary and very few studies have addressed the performance of these tests in COVID-19 patient groups at different time points. We here compared the sensitivity and specificity of seven commercial (SNIBE, Epitope, Euroimmun, Roche, Abbott, DiaSorin, Biosensor) and two in-house LIPS assays (LIPS N and LIPS S-RBD) IgG/total Ab tests in serum samples from 97 COVID-19 patients and 100 controls, and correlated the results with the patients’ clinical data and the time-point the test was performed. We found a remarkable variation in the sensitivity of antibody tests with the following performance: LIPS N (91.8%), Epitope (85.6%), Abbott and in-house LIPS S-RBD (both 84.5%), Roche (83.5%), Euroimmun (82.5%), DiaSorin (81.4%), SNIBE (70.1%), and Biosensor (64.9%). The overall agreement between the tests was between 71–95%, whereas the specificity of all tests was within 98–100%. The correlation with patients’ clinical symptoms score ranged from strongest in LIPS N (ρ = 0.41; p<0.001) to nonsignificant in LIPS S-RBD. Furthermore, the time of testing since symptom onset had an impact on the sensitivity of some tests. Our study highlights the importance to consider clinical symptoms, time of testing, and using more than one viral antigen in SARS-CoV-2 antibody testing. Our results suggest that some antibody tests are more sensitive for the detection of antibodies in early stage and asymptomatic patients, which may explain the contradictory results of previous studies and should be taken into consideration in clinical practice and epidemiological studies.
Citation: Naaber P, Hunt K, Pesukova J, Haljasmägi L, Rumm P, Peterson P, et al. (2020) Evaluation of SARS-CoV-2 IgG antibody response in PCR positive patients: Comparison of nine tests in relation to clinical data. PLoS ONE 15(10): e0237548. https://doi.org/10.1371/journal.pone.0237548
Editor: Gheyath K. Nasrallah, Qatar University, QATAR
Received: July 31, 2020; Accepted: October 12, 2020; Published: October 27, 2020
Copyright: © 2020 Naaber et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The study was supported by Estonian Research Council grants PRG377 (LH, PR, PP) and IUT34-19 (PN, ES). SYNLAB Estonia provided support in the form of salaries for authors (PN, KH, JH, IE) and research materials, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: The authors have read the journal's policy and have the following competing interests: PN, KH, JH, IE are employees of SYNLAB Estonia. There are no patents, products in development or marketed products associated with this research to declare. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Currently, more than 300 tests are available for SARS-CoV-2 antibody detection . These tests are produced in several formats and they detect different types of antibodies including IgG, IgM or IgA subtypes or total immunoglobulin. In addition, the target proteins used to detect antibodies vary between the tests. Commercial tests are usually designed to detect antibodies against SARS-CoV-2 nucleocapsid (N), spike1 (S1), spike2 (S2), or receptor-binding domain of the spike (S-RBD) protein, or their combinations, though not all commercial providers specify the viral proteins used. Given the large variability in antibody tests, discrepancies between test results are expected. Concordantly, at the moment no agreement exists upon which viral protein should be used as a gold standard in serodiagnosis of COVID-19 patients.
Although the producers have usually reported high sensitivity and specificity for their tests, variable clinical sensitivity has been reported by independent studies [2–5]. However, minimal data is available on their sensitivity and specificity for their differences in target proteins.
The majority of clinical studies and validations of commercial tests have been performed in patient groups with severe disease and thus reported sensitivity data may not be the same for COVID-19 patients with mild symptoms. Only a few studies have investigated the antibody responses in pauci-symptomatic or asymptomatic persons [6, 7]. Several studies have shown stronger antibody response in patients with severe disease as compared with mildly symptomatic ones. Also higher rate of absence of seroconversion in asymptomatic patients has been described. However, other studies have failed to find any correlation between clinical course and immune response [3, 6]. Since the majority of COVID-19 cases are asymptomatic, the performance of the tests in this group is important to evaluate the reliability of antibody tests in seroepidemiological studies and clinical diagnostics .
It is known that the sensitivity of the antibody test depends on sampling time. Different studies have reported variable time of appearance of antiviral IgG antibodies but in most publications the median seroconversion time has been between 6 and 14 days from symptoms onset. Although several studies have shown high IgG at least for seven weeks, rapid decline of IgG in convalescence phase has been reported in asymptomatic COVID-19 patients [2, 3, 6]. It seems, that the optimal time for IgG detection (with the highest sensitivity rate) may also depend on clinical course of COVID-19 and is not clearly defined yet.
There is variation in antibody tests design (different target viral proteins has been used) on the one hand, and the conflicting results of clinical studies (tests sensitivities, antibody response dependence on clinical cause, optimal testing time) on the other hand. Thus, there is lacking information on how different SARS-CoV-2 antibody tests perform in subgroups of COVID-19 patients and at variable time points.
Our study aimed to compare the performance characteristics of seven commercial and two in-house IgG/total Ab tests, which analyze the reactivity to several target proteins, and to correlate the results with the patients’ clinical data (with different symptoms score and age), and time from disease onset.
Material and methods
The Study has been approved by Research Ethics Committee of the University of Tartu on April 23, 2020 (nr 311/T-1). Patients signed informed consent before recrutement into the study.
Patients’ recruitment and sample collection
Serum samples from 97 persons with COVID-19 were collected between April 28 and May 07 2020 from Kuressaare hospital located on the island of Saaremaa in Estonia. Persons who had been diagnosed COVID-19 by positive SARS-CoV-2 RT-PCR regardless of clinical symptoms were invited to participate. These persons included hospitalized and ambulatory patients as well as healthy contacts of confirmed COVID-19 cases selected randomly. The time from symptoms onset or positive PCR to serum collection had to be at least one week. Included persons signed informed consent agreeing with sampling and usage of clinical data. Medical personnel in Kuressaare hospital performed blood sampling (10ml from each patient), recorded time of clinical onset and patients’ symptoms related to COVID-19 using modified WHO questionnaire. Samples and patients’ data were sent to SYNLAB Estonia central laboratory in coded manner.
Patients were scored based on a number of different symptoms present during COVID-19 episode. On that basis we classified patients as asymptomatic (no symptoms before or after positive SARS-CoV-2 RT-PCR test), patients with 1–6 symptoms and patients with ≥7 different symptoms.
Serum was separated and aliquoted before storage. All aliquots were stored at– 30° C and analyzed within one month applying one freezing/thawing cycle before testing.
For testing the specificity of applied tests, 100 anonymous serum samples collected before COVID-19 pandemic and stored in SYNLAB Estonia were used. These samples were taken from healthy persons for various health control laboratory tests and have not been screened for virus related antibodies.
Five laboratory tests for IgG (SNIBE, Euroimmun, Abbott, Epitope, DiaSorin), one laboratory total Ab (Roche) test, one rapid IgG test (SD Biosensor) and 2 in-house IgG tests (LIPS N and LIPS S-RDB) were compared. Different protein markers have been used in these tests (S1, S2, S-RBD, N or their combinations). Detailed information about the tests is summarized in S1 Table. Commercial tests were performed and interpreted according to manufacturer instruction, in-house LIPS as described previously .
The following analysis was made: correlation by Spearman's test, comparison of qualitative data (positivity rate) by Fisher exact test, groups’ comparison by Kruskal-Wallis test, pairwise comparison by Conover-Iman test with Holm-Bonferron correction using Past 4.03 and Stata 14.2 software.
According to medical history and patients’ anamnesis, 20 patients (20.6%) had no symptoms before or after the positive PCR test. In 43 patients (44%), one to six different symptoms has been recorded, and 34 patients (35%) had seven or more symptoms related to COVID-19 episode. Detailed clinical data is shown in Table 1.
Comparison of specificity by different tests
We found no false-positive results in testing 100 pre-COVID-19 sera by Roche, Abbott and Biosensor tests. Other tests gave one to two false positive results. Additionally, Euroimmun test gave 2% (2/100) and Epitope 4% (4/100) borderline results with pre-COVID-19 sera. The specificity and sensitivity of applied tests is shown in Table 2. In total, 15% (15/100) of control samples gave false positive results by any test. In most cases, only one test was positive per sample but 2% (2/100) of samples showed positive/borderline results by two (Epitope and Euroimmun) or three (SNIBE, Epitope, LIPS S-RBD) tests. The distribution of quantitative values of controls is presented in S1 Fig.
Comparison of sensitivity by different tests in COVID-19 patient
Out of 97 COVID-19 patients’ samples, 53 (55%) were positive and two (2%) negative by all nine tests used. Results varied by tests in the case of 42 (43%) samples. Sensitivity by different tests is shown in Table 2. The highest positivity rate was found in in-house LIPS N test (91.8% cases) and lowest in Biosensor rapid test (64.9%). The distribution of quantitative test values of COVID-19 cases in comparison with control samples is presented in S1 Fig.
The agreement between the tests (using qualitative interpretations) ranged between 95% (Euroimmun and DiaSorin) and 71% (LIPS N and Biosensor). The correlation between quantitative results was significant in all test combinations (p<0.001) except between in-house LIPS N and LIPS S-RBD that gave non-significant results. The strongest correlation was found between Euroimmun and DiaSorin tests (ρ = 0.95). The agreement between qualitative results and the correlation between quantitative values is shown in Table 3 and S2 Fig.
Relations between antibody detection and COVID-19 patients’ symptoms
Comparing patients’ symptom scores and the tests’ quantitative values, we found a significant positive correlation in all cases except with LIPS S-RBD. The strongest correlation was found in LIPS N (ρ = 0.41; p<0.001), followed by Roche (ρ = 0.39; p<0.001), Abbott and SNIBE (both ρ = 0.32; p = 0.001), DiaSorin (ρ = 0.31; p = 0.002), Epitope (ρ = 0.30; p = 0.003) and Euroimmun (ρ = 0.29; p = 0.004). The differences in quantitative test values and qualitative results among patient groups according to the number of symptoms are presented in Table 4.
Relations between antibody detection and time to test (TTT)
We found a significant correlation between TTT and the quantitative results of the test in case of Roche (ρ = 0.38; p<0.001), Euroimmun (ρ = 0.21; p = 0.04) and LIPS N (ρ = 0.28; p = 0.009) test. We also found significant differences between TTT groups in quantitative test data as well as in positivity rate in case of some but not all tests (Table 5).
This is the first study evaluating the performance characteristics of several commercial and in-house SARS-CoV-2 antibody tests in different patient groups and different time to test points.
We found that different tests gave diverse antibody results if applied to heterogeneous COVID-19 patient group. In less than 60% of COVID-19 cases, all tests gave identical positive or negative result. Considering that all COVID-19 patients (confirmed by positive SARS-CoV-2 PCR test) should develop IgG antibodies, the sensitivity of tests varied from 65 to 92%, which is much less than reported by the manufacturers . In agreement, similar low sensitivity rates have been reported in a recent meta-analysis on antibody tests . A combination of two tests with different protein markers may increase sensitivity. For example according to our results re-testing Abbott IgG (N protein) negatives with index value 0.3–1.39 by DiaSorin IgG (S1 and S2 proteins), as practiced in SYNLAB Estonia, increases sensitivity by ca 7%. Although this combination did not affect the specificity in our control group, comparisons in larger study groups should be done. However, even if several different tests are combined, some confirmed COVID-19 cases remained negative for antibodies.
Correlation and agreements between the tests studied here varied highly in our COVID-19 group. The best correlation was found between Epitope and DiaSorin, which could be explained by detecting the antibodies to the same viral antigen (spike protein). One of the weakest agreements in positive/negative results and absence of correlation in quantitative test values was found when comparing IgG antibodies to nucleocapsid and to RBD of spike protein detected by LIPS tests. It is thus plausible that the patients develop diverse IgG antibody reactivities to SARS-CoV-2 proteins and their epitopes.
While analyzing the results from different tests in patient groups with different symptoms scores we found that patients with more different symptoms usually had a higher positivity rate and higher levels of antibodies. This is in accordance with some previous studies . However, this relationship was not uniform in all tests. For example, LIPS detected significant differences in anti-N IgG levels between asymptomatic, pauci-symptomatic and poly-symptomatic COVID-19 patients. At the same time anti-RBD IgG variation within the patients groups was higher and no significant differences between groups were found. According to our data production of IgG antibodies against some virus proteins are more symptom-dependent than others.
Thus, according to our study, patients with more symptoms develop a higher immune response against virus proteins than asymptomatic ones. This makes the diagnosing of previous COVID-19 patients by antibody test more difficult in asymptomatic and pauci-symptomatic patients since the sensitivity of several tests is much lower in this subgroup than in highly symptomatic group. Since the majority of COVID-19 patients are asymptomatic or have only a few mild symptoms the sensitivity of antibody tests to detect disease in the general population could be lower . This may affect the reliability of antibody-based epidemiological studies.
However, for some tests (such as Abbott), the absence of clinical severity seems not to affect positivity rate so much than for others (such as Biosensor rapid test or SNIBE) where the positivity rate in asymptomatic COVID-19 cases was about two times lower than in polysymptomatic ones. More studies are needed to confirm the finding that some antibody tests (that use specific antigens) are more suitable to diagnose asymptomatic COVID-19 cases than others.
We also found test-dependent differences while comparing antibody detection among different TTT groups. For example, anti-N IgG was detected by Abbott in high level already at 7–14 days TTT group and no differences were found with later TTT groups. In some other test (such as Euroimmun) lower detection rate and antibody levels were associated with shorter TTT. The reason could be in an analytical sensitivity of the test and also in the usage of different viral proteins–antibodies to some virus proteins may appear earlier and at increased levels than others.
Our study has several limitations. Firstly, although the onset of disease could be dated relatively precisely in symptomatic cases, in asymptomatic cases the disease detection depends on random PCR screening of risk groups. Thus, the first positive SARS-CoV-2 PCR is not equivalent to disease onset and one should be careful drawing conclusions based of TTT in such heterogeneous group of patients that includes symptomatic and asymptomatic patients. Secondly, we only analyzed IgG and total antibody values. The main reasons were the absence of IgM tests from several manufacturers (Abbott, DiaSorin) in the time of testing and questionable reliability of available ones. SNIBE and Epitope IgM tests gave significantly lower positive results than IgG tests and added no additional positive cases (all IgM positive cases were also IgG positive). Euroimmun IgA had a high positivity rate (ca 90% in COVID-19 patients) but also a high proportion of nonspecific reactions (21% of positive and borderline cases) in pre COVID-19 control group. Thus, until reliable commercial IgM and IgA tests are available, we can’t evaluate the role of these antibodies in infection and applicability of these tests in clinical practice or surveillance studies.
In conclusion, our study gave new theoretical insight to COVID-19 diagnostic testing and has practical implications. We confirmed that SARS-CoV-2 antibody response depends on clinical symptoms and time of testing, but we also found that this relation is dependent on test type and viral antigens used in the tests. This means that not all antibody tests work uniformly well in symptomatic and asymptomatic cases and in different time periods from disease onset. This explains some contradictory results of previous studies and should be taken into consideration in clinical practice and epidemiological studies.
S1 Table. Detailed information about applied SARS-CoV-2 antibody tests.
S1 Fig. Distribution of quantitative results of SARS-CoV-2 antibody tests: COVID-19 patients (n = 97) vs controls (n = 100).
- 1. SARS-CoV-2 Diagnostic Pipeline. https://www.finddx.org/covid-19/pipeline/. Accessed on 10.09.2020.
- 2. Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui LP, Johnston JC, et al. Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis BMJ 2020; 370:m2516.| pmid:32611558
- 3. Health Information and Quality Authority. Evidence summary of the immune response following infection with SARS-CoV-2 or other human coronaviruses. Updated 6 August 2020. https://www.hiqa.ie/reports-and-publications/health-technology-assessment/evidence-summary-immunity-response-following.
- 4. Lassaunière R, Frische A, Harboe ZB, Nielsen ACY, Fomsgaard A1, Krogfelt KA, et al. Evaluation of nine commercial SARS-CoV-2 immunoassays. medRxiv preprint https://doi.org/10.1101/2020.04.09.20056325
- 5. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Test performance evaluation of SARS-CoV-2 serological assays. medRxiv preprint https://doi.org/10.1101/2020.04.25.20074856.
- 6. Quan-Xin Long QX, Tang XJ, Shi QL, Li Q, Deng H, Yuan J, et al. Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections. Nature Medicine. https://doi.org/10.1038/s41591-020-0965-6
- 7. Grzelak L, Temmam S, Planchais C, Demeret C, Huon C, Guivel-Benhassine F, et al. SARS-CoV-2 serological analysis of COVID-19 hospitalized patients, pauci-symptomatic individuals and blood donors. medRxiv preprint https://doi.org/10.1101/2020.04.21.2006885
- 8. Al- Sadeq DW, Nasrallah GK. The incidence of novel coronavirus SARS-CoV-2 among asymptomatic patients: A systematic review. Int J Infect Dis. 2020; 98: 372–380. https://doi.org/10.1016/j.ijid.2020.06.098 pmid:32623083
- 9. Haljasmägi L, Remm A, Rumm AP, Krassohhina E, Sein H, Tamm A, et al. LIPS Method for the Detection of SARS-CoV-2 Antibodies to Spike and Nucleocapsid Proteins. Eur. J. Immunol. 2020. 50: 1234–1236. https://doi.org/10.1002/eji.202048715