Performance of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients: A retrospective study in outbreak on a cruise ship

Objectives A few studies on antibody testing have focused on asymptomatic or mild coronavirus disease 2019 (COVID-19) patients with low initial anti-severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody responses. Anti-SARS-CoV-2 antibody-testing performance was evaluated using blood samples from asymptomatic or mild COVID-19 patients. Methods Blood samples were collected from 143 COVID-19 patients during an outbreak on a cruise ship 3 weeks after diagnosis. Simultaneously, a follow-up SARS-CoV-2 genetic test was performed. Samples stored before the COVID-19 pandemic were also used to evaluate the lateral flow immunochromatographic assay (LFA) and electrochemiluminescence immunoassay (ECLIA). Titers of anti-SARS-CoV-2 IgM and IgG antibodies against the nucleocapsid and spike proteins were measured using the enzyme-linked immunosorbent assay to confirm which antibodies were influenced on LFA- and ECLIA- false-negative result in crew-member samples. Results Sensitivity, specificity, positive-predictive, and negative-predictive values of LFA-detected IgM antibodies were 0.231, 1.000, 1.000, and 0.613, respectively; those of LFA-detected IgG antibodies were 0.483, 0.989, 0.972, and 0.601, respectively; and those of ECLIA-detected total antibodies were 0.783, 1.000, 1.000, and 0.848, respectively. All antibody titers measured using ELISA were significantly lower in blood samples with negative results than in those with positive results in both LFA and ECLIA. In the patients with negative results from the follow-up genetic testing, IgM-, IgG-, and total-antibody positivity rates were 22.9%, 47.6%, and 72.4%, respectively. Conclusions These findings suggest that anti-SARS-CoV-2 antibody testing has lower performance in asymptomatic or mild COVID-19 patients than required in the guidelines.


Introduction
The clinical indications for anti-severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody testing are limited, although many methods have been developed. The detection of anti-SARS-CoV-2 IgG or total antibodies at 3-4 weeks after symptom onset may be useful in determining past infection in selected clinical situations; however, data in this context are limited [1]. On the other hand, anti-SARS-CoV-2 antibody testing may play an essential role in the public health response to coronavirus disease 2019  and in understanding the outbreak dynamics of the COVID-19 pandemic [2,3]. However, a previous study reported that initial anti-SARS-CoV-2 antibody responses were lower in asymptomatic or mild COVID-19 patients than in severe COVID-19 patients [4]. Since the percentage of asymptomatic COVID-19 patients in large-sample studies was 43.0%-76.5% [5], it is necessary to elucidate the performance of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients.
We preserved blood samples of asymptomatic or mild COVID-19 patients collected from a large cruise ship that had experienced a COVID-19 outbreak. The cruise ship anchored at Nagasaki Port from the end of January 2020, and one of the 623 crew members complained of pneumonia symptoms and was diagnosed with COVID-19 in mid-April 2020. Thereafter, SARS-CoV-2 genetic testing was performed on all cruise ship crew members, and SARS-CoV-2 was detected in 144 individuals [6]. However, most patients received care on the ship because they were asymptomatic or did not need inpatient treatment, such as oxygen administration.
In this study, we evaluated the performance of anti-SARS-CoV-2 antibody testing using the lateral flow immunochromatographic assay (LFA) and electrochemiluminescence immunoassay (ECLIA) in blood samples from crew members and those collected from patients between November 2014 and August 2019. In addition, we measured the titers of several anti-SARS--CoV-2 antibodies and compared them between false-negative-and positive-result samples in anti-SARS-CoV-2 antibody testing using the enzyme-linked immunosorbent assay (ELISA) to confirm which antibodies were influenced on LFA-and ECLIA-false-negative results in crewmember samples.

PLOS ONE
Performance of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-

Study design
In this study, asymptomatic or mild COVID-19 was defined as a cruise ship crew member who was not hospitalized when blood samples were collected. Blood samples were collected from 178 crew members in mid-May 2020, 3 weeks after the first SARS-CoV-2 genetic testing. Thirty-four blood samples were excluded from the final analysis due to negative results in the first SARS-CoV-2 genetic testing, and one blood sample was excluded because the patient was hospitalized at the time of sample collection (Fig 1). Finally, 143 blood samples collected from crew members who had positive results in the first SARS-CoV-2 genetic test were analyzed in this study. A total of 269 stored blood samples were collected from patients at Nagasaki University Hospital between November 2014 and August 2019. After deduplication, 174 blood samples from patients were used as negative controls in this study (Fig 1). The blood samples were stored at − 80˚C until further antibody testing. Initially, anti-SARS-CoV-2 antibodies were detected using LFA and ECLIA by clinical technologists at Nagasaki University Hospital according to the manufacturer's instruction. Subsequently, to confirm which antibodies were influenced on LFA-and ECLIA-false-negative results in crewmember samples, IgM and IgG antibody titers against anti-SARS-CoV-2 spike-protein (SP) and nucleocapsid protein (NP) were measured using ELISA at Nagasaki University Hospital by Cellspect Co., Ltd, researchers to confirm which antibodies were influenced on LFA-and ECLIA-false-negative results. The definition of false-negative in this study is that the antibody testing was negative in the patients with negative results in the first SARS-CoV-2 genetic testing. Anti-SARS-CoV-2 antibodies were detected without notification of each result. Results from the follow-up SARS-CoV-2 genetic testing using real-time reverse transcription polymerase chain reaction RT-PCR [7] testing were obtained from the database. The follow-up genetic testing was performed on nasopharyngeal samples collected from cruise ship crew members at approximately the same time as blood samples were collected, 3 weeks after the first SARS--CoV-2 genetic testing.

Detection of anti-SARS-CoV-2 antibodies
SARS-CoV-2 IgM and IgG were detected using the SARS-CoV-2 Antibody IgM/IgG LFA testing kit (RF-NC001 and RF-NC002, Kurabo Industries, Ltd., Osaka, Japan) according to the manufacturer's instructions. The results from tests using these kits were assessed by two clinical technologists. Anti-SARS-CoV-2 total antibodies were detected by Elecsys Anti-SARS--CoV-2 (Roche Diagnostics K.K., Tokyo, Japan) using a Cobas e801 analytical unit for immunoassay tests (Roche Diagnostics K.K., Tokyo, Japan) according to the manufacturer's instructions. The judgment criteria for ECLIA were as follows: positive, cutoff index (COI) � 1.0; negative, COI < 1.0. To measure anti-SARS-CoV-2 IgM and IgG antibodies against SP (QuaResearch COVID-19 human IgM/IgG ELISA kit, RCOEL961N, Cellspect Co., Ltd., Iwate, Japan) and those against NP (QuaResearch COVID-19 human IgM/IgG ELISA kit, RCOEL961, Cellspect Co., Ltd., Iwate, Japan), ELISA was performed by Cellspect Co., Ltd. The antigen proteins immobilized in the ELISA kit were recombinant SP (16-1213 AA) of SARS-CoV-2 expressed in mammalian cells and recombinant NP (1-419 AA) of SARS-CoV-2 expressed in Escherichia coli. To measure IgM and IgG antibodies against SP, serum or plasma samples were diluted 1:1000 in 1% bovine serum albumin/phosphate-buffered saline with Tween detergent (PBST); to measure IgM and IgG antibodies against NP, serum or plasma samples were diluted 1:1000 in 2% non-fat milk/PBST. The absorbance was read at 450 nm using an automatic ELISA system (QRC5LB925, Cellspect Co., Ltd., Iwate, Japan) according to the manufacturer's instructions. The cutoff value determined in a previous study was used to interpret the ELISA results [8]. The cutoff values for anti-SARS-CoV-2 IgM and IgG antibody titers against SP were set at 0.25 and 0.26 and those against NP at 0.4 and 0.7, respectively [8].

Statistical analysis
All statistical analyses were performed using EZR version 1.53 (Saitama Medical Center, Jichi Medical University, Saitama, Japan), which is a graphical user interface for R (version 4.0.3; R Foundation for Statistical Computing, Vienna, Austria). Fisher's exact test or the chi-square test was used to compare categorical variables and the Mann-Whitney U test to compare continuous variables. The statistical significance level was set at P <0.05. The sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) of the antibody testing were calculated with 95% confidence intervals (95% CI). Spearman's rank-order correlation was used to evaluate the correlation between anti-SARS-CoV-2 antibody titers in ELISA and COIs in ECLIA.

Ethics
This retrospective study was approved by the Ethics Committee of Nagasaki University Hospital (20052101) and was registered in the UMIN Clinical Trials Registry (UMIN000040402). Nasopharyngeal samples and blood samples were anonymized and individually numbered when they were collected from the cruise ship crew members. They were sent to Nagasaki University for SARS-CoV-2 RT-PCR and antibody testing as routine care. Blood samples for the negative controls were stored anonymously at the Department of Laboratory Medicine, Nagasaki University Hospital. All data and samples were fully anonymized before we accessed them and the Ethics Committee of Nagasaki University Hospital waived the requirement for informed consent.
Anti-SARS-CoV-2 total antibodies were detected using ECLIA in 112 (78.3%) blood samples collected from the crew (Fig 2C). There was no positive result for ECLIA-detected anti-SARS-CoV-2 total antibodies in the negative controls. Anti-SARS-CoV-2 total antibodies were detected using ECLIA in all blood samples, with positive IgG antibody results detected by LFA. The blood-sample positivity rate for ECLIA-detected anti-SARS-CoV-2 total antibodies was significantly higher than that for LFA-detected IgM and IgG antibodies (p < 0.001). The Se, Sp, PPV, and NPV of ECLIA-detected anti-SARS-CoV-2 total antibodies were 0.783, 1.000, 1.000, and 0.848, respectively (Fig 2C).

Comparison of anti-SARS-CoV-2 antibody titers between samples with false-negative and those with positive LFA results
Anti-SARS-CoV-2 SP IgM and IgG and anti-SARS-CoV-2 NP IgM and IgG antibody titers in blood samples collected from crew members were measured using ELISA (S1 Table). The median anti-SARS-CoV-2 IgM and IgG antibody titers against SP were 0.10 (0.07-0.47) and 0.27 (0.07-0.71), and those against NP were 0.18 (0.10-1.08) and 0.56 (0.18-2.82), respectively. Antibody titers were compared between blood samples with false-negative results and those with positive LFA results (Fig 3). Antibody titers for all antibodies were significantly higher in blood samples with positive results than in those with false-negative results (Fig 3A-3D). COI in ECLIA was also significantly higher in blood samples with positive results than in those with false-negative results (Fig 3E). The positivity rates for anti-SARS-CoV-2 IgG antibodies against SP and NP were significantly lower in samples with false-negative results than in those with positive results (both p<0.001) (S3 Table).

Comparison of anti-SARS-CoV-2 antibody titers between samples with false-negative and those with positive ECLIA results
Antibody titers were compared between blood samples with false-negative results and those with positive ECLIA results (Fig 4). Antibody titers for all antibodies were significantly higher in blood samples with positive results than in those with false-negative results (Fig 4A-4D). The positivity rates for ELISA-detected anti-SARS-CoV-2 IgM and IgG antibodies against SP in samples with false-negative results were 0% (0/31) and 9.7% (3/31), and those against NP were 3.2% (1/31) and 0% (0/31), respectively. The positivity rates for ELISA-detected anti-SARS-CoV-2 IgG antibodies

PLOS ONE
Performance of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients against SP and NP were significantly lower in samples with false-negative results than in those with positive results (p = 0.004 and p<0.001, respectively) (S4 Table).
The correlation between anti-SARS-CoV-2 antibody titers in ELISA and COI in ECLIA among blood samples collected from crew members was also evaluated (Fig 5). The correlation coefficients between COI in ECLIA and anti-SARS-CoV-2 IgM and IgG antibodies against SP were 0.324 and 0.398, and those against NP were 0.332 and 0.812, respectively.

Comparison of LFA-and ECLIA-detected anti-SARS-CoV-2 antibody and result of follow-up genetic testing
Among 143 COVID-19 patients, 105 (73.4%) tested negative in the follow-up SARS-CoV-2 genetic testing using nasopharyngeal samples collected at approximately the same time as blood samples were collected. The positivity rates in crew members with negative results from the follow-up SARS-CoV-2 genetic testing were 22.9% (24/105), 47.6% (50/105), and 72.4% (76/105), respectively. The positive rates of LFA-detected anti-SARS-CoV-2 IgM and IgG antibodies and ECLIA-detected total antibodies in the crew with positive results from the followup SARS-CoV-2 genetic testing were 23.7% (9/38), 50.0% (19/38), and 94.7% (36/38), respectively. The positivity rate for ECLIA-detected anti-SARS-CoV-2 total antibodies was significantly higher in crew members with positive results from the follow-up SARS-CoV-2 genetic testing than that in crew members with negative results (p = 0.003).

Discussion
This study revealed the performance of LFA and ECLIA in detecting anti-SARS-CoV-2 antibodies in asymptomatic or mild COVID-19 patients. Although all COVID-19 patients analyzed in this study were either asymptomatic or mild, we did not know their distribution. The previous study used the same ELISA kits revealed that titers of ELISA-detected anti-SARS-CoV-2 IgG antibodies against SP and NP were significantly lower in mild COVID-19 patients than those in severe and critical COVID-19 patients [8]. The titer of anti-SARS-CoV-2 IgG antibodies against SP and NP in mild COVID-19 patients in the previous study after 22 days from onset were approximately 0.5 and more than 2.0, respectively. Because they are higher than those in this study, 0.27and 0.56, respectively, it is expected that the percentage of asymptomatic COVID-19 patients was higher than that of mild COVID-19 patients in this study.
The sensitivities of LFA for both anti-SARS-CoV-2 IgM and IgG antibodies were low (0.231 and 0.483, respectively) in this study. LFA's sensitivity in this study was lower than that reported in two previous studies. One previous study reported IgM-and IgG-antibody positivity in all 24 COVID-19 patients [9]. In another study using blood samples collected from 12 COVID-19 patients, including two asymptomatic (16.7%) and seven mild patients (58.3%), the kit's sensitivity for IgM and IgG antibodies was 0.750 and 0.727, respectively [10]. ECLIA's sensitivity for anti-SARS-CoV-2 total antibodies was 0.783 in this study, which was higher than that of LFA. However, ECLIA's sensitivity in this study was lower than that reported in previous studies using the same ECLIA kit; the sensitivity in previous studies was 0.920-0.995 [11][12][13]. One study reported that ECLIA's sensitivity in all patients was 0.920-0.927; however, ECLIA's sensitivity in 6 (4.0%) asymptomatic patients and 37 (24.8%) mild patients was reported to be approximately 0.800 [11]. The results of this study demonstrated that ECLIA's sensitivity in detecting anti-SARS-CoV-2 antibodies was relatively low in asymptomatic or mild COVID-19 patients. Considering the results of this study, the sensitivities of LFA and ECLIA in asymptomatic or mild COVID-19 patients were lower than the required clinical sensitivity in the guidelines [1].
In contrast to sensitivity, the specificities of LFA and ECLIA were very high in this study. These results were consistent with those of previous studies using the same kits [11][12][13][14][15]. Although there were no samples with false-positive results for LFA-detected anti-SARS-CoV-2 IgM antibodies and ECLIA-detected total antibodies, there were two samples with false-positive results for LFA-detected SARS-CoV-2 IgG antibodies. A previous study using the same LFA kit reported SARS-CoV-2 IgG antibody positivity in 57% (4/7) of patients with human common cold coronavirus pneumonia [9]. The results from the previous study indicated that the LFA kit used in this study had cross-reactivity with antibodies against human common cold coronavirus. Although the false-positivity rate was very low in this study, the LFA kit may not be suitable for screening COVID-19 patients or for COVID-19 serological surveillance in a population with a low prevalence of COVID-19 because specificity is especially important in such situations [1,16].
Anti-SARS-CoV-2 IgM and IgG antibody titers against SP and NP were measured using ELISA in this study. All antibody titers were significantly lower in samples with false-negative results than in those with positive results in both LFA and ECLIA. Among the antibodies, anti-SARS-CoV-2 IgG antibodies against SP and NP were noticeably divided between blood samples with positive and those with false-negative results in both methods. Additionally, the anti-SARS-CoV-2 IgG antibody titer against NP in ELISA was strongly correlated with COI in ECLIA. The ECLIA kit used in this study detects total antibodies (IgG, IgA, and IgM) against NP [17,18]; however, our data indicated that a low titer of anti-SARS-CoV-2 antibody against NP contributed to the false-negative results in asymptomatic and mild COVID-19 patients. A previous study reported that a combination of anti-SARS-CoV-2 antibodies against NP and SP increased the percentage of positive results [19]. In fact, the anti-SARS-CoV-2 antibody titer against SP in ELISA had a weaker correlation with COI in ECLIA in this study. However, anti-SARS-CoV-2 IgM and IgG positivity rates were also very low in COVID-19 patients with false-negative ECLIA results. Since all anti-SARS-CoV-2 antibody testing that was granted emergency use authorization by the United States Food and Drug Administration targeted SP and/or NP, it is currently difficult to improve the sensitivity of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients.
In this study, 73.4% of COVID-19 patients tested negative in the follow-up SARS-CoV-2 genetic testing using nasopharyngeal samples collected at approximately the same time as blood samples were collected. On the other hand, LFA and ECLIA detected anti-SARS-CoV-2 antibodies in 47.6% and 72.4% of patients with negative results in the follow-up SARS-CoV-2 genetic testing, respectively. These data support the recommendation in the guidelines, [1] although it is necessary to consider the possibility of false-negatives in anti-SARS-CoV-2 antibody testing.
This study has several limitations. First, we couldn't obtain the data regarding factors that affect antibody responses such as age, comorbid medical conditions, and immunocompromised state [1]. Since we collected the blood samples from crew members, the most patients were healthy adult. Second, only one kit each of LFA, ECLIA, and ELISA was used in this study. Thus, other kits could have possibly had higher sensitivity in asymptomatic and mild COVID-19 patients than those used in this study. However, since the ECLIA used in this study demonstrated good sensitivity compared to other ECLIA kits [11][12][13], it may be difficult to improve sensitivity by using others. Third, the time-course of the anti-SARS-CoV2 antibody testing was not evaluated in this study. This study focused on the time point that was reported as one of the best times for anti-SARS-CoV2 antibody testing [8,10,11]; however, it is possible that the optimal time for anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients varies. Finally, the neutralizing antibodies were not evaluated. A previous study reported that the results of the ECLIA kit used in this study were correlated with SARS--CoV-2 neutralization; nevertheless, it also reported that the kit had poor negative-percent agreement for SARS-CoV-2 neutralization [20]. Therefore, further investigation is required to confirm whether COVID-19 patients with false-negative results have neutralizing antibodies.

Conclusions
The findings of this study corroborate the recommendations of the guidelines [1]. They suggest that the sensitivity of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients is lower than the required clinical sensitivity. In addition, it may be difficult to improve anti-SARS-CoV-2 antibody testing sensitivity at present because anti-SARS--CoV-2 IgM and IgG antibody titers against SP and NP were very low in COVID-19 patients with false-negative results. On the other hand, anti-SARS-CoV-2 antibody testing may be useful in limited contexts, such as in patients at 3-4 weeks after symptom onset but with negative SARS-CoV-2 genetic test results.
Supporting information S1 Table. Results