Target specific serologic analysis of COVID-19 convalescent plasma

This study compared the performance of four serology assays for Coronavirus Disease 2019 (COVID-19) and investigated whether COVID-19 disease history correlates with assay performance. Samples were tested at Northshore using the Elecsys Anti-SARS-CoV-2 (Roche Diagnostics), Access SARS-CoV-2 IgG anti-RBD (Beckman Coulter), and LIAISON SARS-CoV-2 S1/S2 IgG (DiaSorin) as well as at Genalyte using Maverick Multi-Antigen Serology Panel. The study included one hundred clinical samples collected before December 2019 and ninety-seven samples collected from convalescent plasma donors originally diagnosed with COVID-19 by PCR. COVID-19 disease history was self-reported by the plasma donors. There was no difference in specificity between the assays tested. Clinical sensitivity of these four tests was 98% (Genalyte), 96% (Roche), 92% (DiaSorin), and 87% (Beckman). The only statistically significant differences in clinical sensitivity was between the Beckman assay and both Genalyte and Roche assays. Convalescent plasma donor characteristics and disease symptoms did not correlate with false negative results from the Beckman and DiaSorin assays. All four tests showed high specificity (100%) and varying sensitivities (89–98%). No correlations between disease history and serology results were observed. The Genalyte Multiplex assay showed as good or better sensitivity to three other previously validated assays with FDA Emergency Use Authorizations.


Introduction
The Coronavirus Disease 2019 (COVID-19) pandemic created an urgent need for serology tests against a novel viral pathogen. Multiple companies rapidly developed assays to detect antibodies against the causative agent, SARS-CoV-2. These in vitro diagnostic tests were then submitted to the United States Food and Drug Administration (FDA) for Emergency Use Authorization (EUA), which permitted their marketing. Alternatively, in-house developed assays were validated as laboratory developed tests (LDTs).

PLOS ONE
PLOS ONE | https://doi.org/10.1371/journal.pone.0249938 April 28, 2021 1 / 10 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is known to bind to the angiotensin-converting enzyme II (ACE2) receptor in order to gain entry to target cells. Similar to the first severe acute respiratory syndrome coronavirus (SARS-CoV), it appears that a region of the coronavirus spike protein known as the receptor binding domain (RBD) is primarily responsible for the SARS-CoV-2-ACE2 interaction. Interestingly, the spike protein is "primed" for ACE2 binding via an interaction with another human protein called TMPRSS2 [1]. Therefore, antibodies that bind regions of the spike protein other than the RBD may also inhibit viral infection. Finally, the nucleocapsid protein of SARS-CoV-2 has also been identified as highly immunogenic. Although the antiviral activity of anti-nucleocapsid antibodies are unclear, their presence may be sensitive and specific for prior exposure to SARS-CoV-2 [2].
SARS-CoV-2 are composed of 16 nonstructural proteins and 4 structural proteins; spike (S), envelope (E), membrane (M), and nucleocapsid (N). The DiaSorin LIAISON1 SARS--CoV-2 [3] is an indirect chemiluminescent immunoassay (CLIA) to detect IgG antibodies against the S1 and S2 spike proteins of SARS-CoV-2 virus. The Beckman Coulter Access SARS--CoV-2 [4] is a two-step enzyme immunoassay to detect IgG antibodies specific for the receptor binding domain (RBD) of the S1 protein of SARS-CoV-2. The Roche Elecsys1 Anti-SARS--CoV-2 [5] is a chemiluminescent double-antigen sandwich immunoassay to detect antibodies that are able to bind nucleocapsid of SARS-CoV-2. All three assays have received Emergency Use Authorization (EUA) from the U.S. Food & Drug Administration (FDA) [6]. Unlike those assays, the Genalyte Maverick Diagnostic System simultaneously detects antibodies against multiple SARS-CoV-2 targets.
Primarily due to the novel nature of COVID-19, the correlation between antibody reactivity and immunity to relapse or re-infection is unknown [7]. Nevertheless, serology tests may be useful to identify which individuals have been exposed to COVID-19 in the past and identify who could donate COVID-19 convalescent plasma [8]. However, COVID-19 serology testing may be complicated by the highly variable clinical presentation of the disease [9]. It is possible that individuals with mild disease, certain demographics, or with certain symptoms may show variable serologic responses that affect assay sensitivity.

Ethics statement
The NorthShore COVID-19 Convalescent Plasma Collection Study was approved by the NorthShore Institutional Review Board (EH20-170). Written informed consent was obtained from convalescent plasma donors. Control samples collected prior to December 2019 were residual anonymized specimens derived from clinical samples, for which Western IRB granted an exception to informed consent for research use (WIRB#20161322).

Participant recruitment and inclusion/exclusion criteria
Convalescent plasma samples used in this study were collected from donors who had been recruited in April and May 2020. Recruitment efforts included messages sent to physicians within the NorthShore University HealthSystem, messages posted on internal websites, and listing on a national website for convalescent plasma collection centers. Participants were required to be greater than 18 years old, have recovered from COVID-19 greater than two weeks prior to consent, provide documentation of a positive SARS-CoV-2 PCR result confirming their diagnosis, and test negative by repeat nasopharyngeal SARS-CoV-2 PCR prior to sample collection. Potential donors were directed to call or email the NorthShore Blood Bank.

Samples
One hundred frozen clinical samples collected prior to December 2019 by Genalyte were used as negative controls. These serum samples were collected in serum separator tubes, centrifuged for 15 minutes at 4,000 RPM, then stored in 1 ml aliquots at -80 C. All transported samples were shipped on dry ice. Samples from individuals who were confirmed to have COVID-19 were taken from convalescent plasma donors who volunteered for the NorthShore University HealthSystem COVID-19 Convalescent plasma collection study. NorthShore plasma donor samples were collected in serum separator tubes (BD vacutainer SST 367986), spun at 4,500 RPM for 7 minutes (EBA20 Hettich, Westphalia, Germany), and aliquots were frozen at -80 C.

Transport and storage
Samples were collected from convalescent plasma donors in BD serum separator tubes (Becton, Dickinson and Company, Franklin Lakes, NJ). The tubes were spun,~1ml aliquots were made, and the aliquots were frozen at -80˚C. An aliquot from each donor was shipped overnight on dry ice to Genalyte for testing; another aliquot was thawed and tested on the three other assays at NorthShore.

Serology testing
Four serology assays were used to analyze the specimens. Three of these are assays with current Emergency Use Authorizations (EUA) from the Food and Drug Administration (FDA) and adapted for use on automated high-throughput chemistry analyzers. Each of these assays targets a single antigen of the SARS-CoV-2 virus and testing was performed in a CLIA accredited laboratory at NorthShore University Health System (Evanston, IL). The final of these is a novel assay manufactured by Genalyte (San Diego, CA) which probes for antibodies against multiple SARS-CoV-2 antigens and other viral antigens. Specimens are then determined to be positive or negative for SARS-CoV-2 antibodies through the use of a multivariate machine learning model. This testing was performed in San Diego at Genalyte's headquarters. Aliquots tested at NorthShore were run on three automated assays: Elecsys Anti-SARS--CoV-2 (Roche Diagnostics), Access SARS-CoV-2 IgG (Beckman Coulter), and LIAISON SARS-CoV-2 S1/S2 IgG (DiaSorin). Each sample was tested on the same day using all three platforms. Each of these assays targets a different epitope of the SARS-CoV-2 virus and employs a different immunoassay format. The DiaSorin assay detects IgG antibodies against the S1 and S2 spike proteins of SARS-CoV-2 virus in an indirect chemiluminescent format. The Beckman assay detects IgG antibodies against the Receptor Binding Domain (RBD) of the S1 protein in a two-step immunoassay format. The Roche assay detects all antibody subtypes that are able to bind nucleocapsid viral antigens in a chemiluminescent sandwich immunoassay format. The Roche, Beckman, and DiaSorin assays are run on automated instruments with high specimen throughput capability, but relatively large physical footprint and startup costs. Conversely, the Genalyte assay is run on a non-automated device with relatively lower throughput, but also lower up-front acquisition costs and smaller physical footprint.

Genalyte assay technology
The Genalyte assay employs multiplex detection technology based on silicon photonics that uses ring resonance to measure binding of macromolecules to sensors on a miniature silicon chip. The Maverick Diagnostic System detects changes in resonance wavelength as macromolecules, such as antibodies, bind to their respective antigens that are bound to the chip. Maver-ick™ SARS-CoV-2 Multi-Antigen Serology Panel (Genalyte Inc) is designed to detect IgG and IgM class of antibodies to five SARS-CoV-2 antigens within a multiplex format based on photonic ring resonance technology [10,11]. A multi-antigen analysis algorithm analyzes individual qualitative results in order to generate an overall positive or negative result.
Genalyte's Multi-Antigen Analysis Algorithm (MAAA) employs the well-known Random Forest Ensemble method with 3000 decision "trees". Random forest is a classifier consisting of a collection of decision trees {h(X, Θ k )}, k = 1,. . .n where the {Θ k } are independent identically distributed random vectors and each tree casts a unit vote for the most popular class at input X. In particular, each new training set is drawn, with replacement, from the original training set. Then a tree is grown on the new training set using random feature selection. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. In the random forest model, the number of features to consider when looking for the best split at each node is set to d ffi ffi ffi d p e where d denotes the total number of features in the data. The Gini Impurity function is used as a measurement for the quality of splits [12].

Donor demographics and history
Convalescent plasma donor COVID-19 disease history was obtained at the time of participant consent via a donor history screening form and blood donor screening questions (see S1 Fig).
Free-text responses were converted into discrete data elements for the purpose of analysis. Whenever an abnormal finding was not reported by a donor, it was assumed to be absent / normal.

Target-specific analysis
Genalyte assay data provided target-specific quantitative response broken down into IgG and IgM-based reactivity. The target-specific data from the 5 SARS-Cov2 targets was compared with the qualitative results of both the Beckman and DiaSorin assays to determine whether the results of the Genalyte targets correlated with the likelihood of the Beckman and DiaSorin assays being positive. P values were calculated with two-tailed Fisher's exact test on a contingency table of true positives versus false negatives. The Haldane-Anscombe correction was used to calculate real-valued odds ratios for instances when a zero-valued denominator would have otherwise been present [13].

Statistical analysis
Due to a lack of a priori knowledge of the expected range of serology test results for antibodies against a SARS-CoV-2, a power calculation was not performed prior to initiation of this study. The significance of pair-wise differences in sensitivity between the four assays tested was determined using a two-tailed Fischer's exact test on a 2x2 table of true positives and false negatives. Target-specific analysis of the samples producing a false-negative result on the Beckman and DiaSorin assays was performed using the Fischer's exact test on the qualitative results of both anti-spike protein assays along with the target-specific results of the Genalyte assay.
The Maverick TM SARS-CoV-2 Multi-Antigen Serology Panel v2 utilizes a multi-analyte analysisalgorithm (MAAA) to make the determination on patient samples of positive or negative orindeterminate for antibodies to the SARS-CoV-2 virus. The algorithm employed is a well-knownensemble method called Random Forests Classification. Random Forests contain a number ofdecision trees constructed from randomly chosen features that each make predictions on the data set, the aggregation of which gives the final result. These models are capable of fitting complex datasets and are resistant to overfitting.
Our implementation of the method uses 3000 such decision trees sampled randomly from training data and validated against test data. The model was also cross validated using five-fold cross validation. Three models were trained, and the combined IgG and IgM model proved to be the most robust, and is the model carried forward to call samples positive or negative or indeterminate for antibodies to SARS-CoV-2.

Convalescent plasma donor characteristics
There was no statistically significant difference in donor gender, age, symptom duration, time from symptom resolution to sample collection, or difference in the presence or absence of any of the symptoms measured (see Table 1) between donors who tested positive by Beckman or DiaSorin and donors who tested negative with those assays (p > 0.05). Overall, the population of donors in this study reflected middle aged individuals who had symptoms but were not hospitalized for COVID-19.

Clinical sensitivity and specificity
The total antibody assays (Roche and Genalyte) showed higher sensitivity compared with the IgG assays (Beckman and DiaSorin), see Table 2. The lowest sensitivity assay was the Beckman IgG (87%) and the highest sensitivity was with the Genalyte assay (98%). The only sensitivity differences that were statistically significant were between Beckman versus Roche (p = 0.0395) and Beckman versus Genalyte (p = 0.0054). All four assays demonstrated 100% specificity, without any false positive results among the 100 negative samples tested.

Discussion
When SARS-CoV-2 serology assays were initially developed, there was concern that false-positive rates in these assays would be high due to cross reactivity with antibodies formed against non-SARS-CoV-2 coronaviruses [14]. While the results presented here cannot exclude the possibility of false positive results when larger sample sizes are tested, it appears that multiple assay manufacturers have taken steps to minimize the chance of a false positive. Perhaps due to these efforts to prevent false positive results, these assays then could have substantial rate of false negative results [15].  The false negative results detected here were most common with the assay that uses the fewest number of SARS-CoV-2 epitopes as its target, in particular the RBD domain of the spike protein. Although this domain is clearly important for viral infectivity, it is likely that antibodies against other viral epitopes may also diminish viral infectivity. This could occur by inducing or preventing conformational changes in viral proteins that are necessary for fusion with the cell membrane as well as by promoting viral clearance through opsonization. However, it is important to note that binding antibody strength does not necessarily correlate with neutralizing antibody titer.
Finally, this study tested the hypothesis that false negative results on serology assays may correlate with certain patient characteristics (e.g. age, gender) or COVID-19 disease symptoms. Such a correlation was not found for false negative results from the two anti-spike assays. This study included insufficient numbers of false negative results from the other two assays tested to draw a meaningful conclusion.
Given the lack of COVID-19 PCR testing in the United States during the initial weeks of the SARS-CoV-2 pandemic, the high rate of very mild or asymptomatic infections, and the real possibility of false negative PCR assays, serology testing will likely remain a critical component of COVID-19 clinical diagnostics in addition to epidemiologic study. Although it remains unclear whether antibodies against certain SARS-CoV-2 epitopes can confer immunity to reinfection, it is apparent that the serologic response to COVID-19 is heterogeneous [16].