Evaluation of eight commercial Zika virus IgM and IgG serology assays for diagnostics and research

Several commercial Zika virus (ZIKV) serology assays have been developed since the recognition of ZIKV outbreaks as a Public Health Emergency of International Concern in 2016. However, test interpretation for ZIKV serology can be challenging due to antibody cross-reactivity with other flaviviruses like dengue virus (DENV). Therefore, we sought to evaluate the performance of eight commercially available ZIKV IgM and IgG assays across three testing platforms, namely, immunochromatographic tests (ICT), ELISAs and immunofluorescence tests (IIFT). The test panel comprised of 278 samples, including acute and convalescent sera or plasma from ZIKV-confirmed, DENV-confirmed, non-ZIKV and non-DENV patients, and residual sera from healthy blood donors. The ZIKV IgM and IgG serology assays yielded higher test sensitivities of 23.5% - 97.1% among ZIKV convalescent samples as compared to 5.6% - 27.8% among ZIKV acute samples; the test specificities were 63.3% - 100% among acute and convalescent DENV, non-DENV samples. Among the ELISAs and IIFTs, the Diapro ZIKV IgM ELISA demonstrated high test sensitivity (96%) and specificity (80%) when tested on early convalescent samples, while the Euroimmun ZIKV IgG ELISA yielded the highest test specificity of 97% - 100% on samples from non-ZIKV patients and healthy blood donors. For rapid ICTs, the LumiQuick IgM rapid ICT yielded low test sensitivity, suggesting its limited utility. We showed that commercial ZIKV IgM and IgG serology assays have differing test performances, with some having moderate to high test sensitivities and specificities when used in a dengue endemic setting, although there were limitations in IgG serology.


Introduction
Zika virus (ZIKV) is a mosquito-borne virus that belongs to the family Flaviviridae, genus Flavivirus. The virus was first isolated from a rhesus monkey in the Zika forest of Uganda [1], with only few and sporadic cases previously reported in Africa and Asia [2]. In  onset) collected from patients suspected of having ZIKV or DENV infections in 2016. The samples were collected at primary healthcare clinics in Singapore and sent to the Environmental Health Institute (EHI) of the National Environment Agency (NEA) for diagnostic testing. The ZIKV and DENV infections were confirmed by real-time ZIKV RT-PCR analysis of urine [25] and the rapid Standard Diagnostics (SD) BIOLINE Dengue Duo Non-structural (NS) 1 Ag/IgM/IgG combo test (Standard Diagnostics, Gyeonggi-do, Republic of Korea), respectively. Set B consisted of 121 convalescent plasma samples from patients suspected of having DENV or ZIKV infection in two separate studies conducted at Tan Tock Seng Hospital (TTSH), Singapore [32,33]. Plasma samples were collected from 2010 to 2012, and from August 2016 to September 2016, for the DENV and ZIKV studies, respectively. ZIKV infections were confirmed by real-time ZIKV RT-PCR analysis of urine [25] while DENV infections were confirmed by real-time DENV RT-PCR analysis or DENV NS-1 Ag ELISA (Bio-Rad Platelia NS1 Ag ELISA, Bio-Rad Laboratories, Marnes-la-Coquette, France). The convalescent plasma samples were further grouped into two categories, samples that were collected between 7 to 14 days' (Set B1, n = 57) and those that were collected between 23 to 34 days' (Set B2, n = 64) post onset of fever. Among the dengue confirmed patients (n = 15), six and nine patients were identified as primary and secondary DENV infections respectively, using the previously described laboratory criteria [34]. In addition to the dengue patients, samples from an equal number of non-dengue patients (n = 15) were included for specificity testing.
Lastly, to evaluate the performance of the ZIKV IgG ELISAs for use in serosurveys, 79 residual serum samples obtained from healthy blood donors (Set C) were included. Among these,

Serologic analyses
Acute and convalescent dengue patient samples, convalescent non-dengue patient samples, and dengue negative healthy blood donor samples were tested for ZIKV NAb using the previously described PRNT method [30], all other samples were tested for both DENV and ZIKV NAb. Briefly, for the virus-antibody binding reaction, equal volumes (100 μL) of diluted sera (heat-inactivated and diluted 10-fold from 1:10 to 1:1000) and ZIKV or DENV (800 plaque forming units/mL) were added into 96-well plates and incubated at 37˚C for an hour. Fifty microliters of the virus-antibody mixture was added into 24-well plates containing Baby Hamster Kidney (BHK) cells (seeded at 1 × 10 5 cells/ mL) in triplicates. The plates were incubated overnight at 37˚C to allow virus adsorption onto BHK cells and were overlaid with 1% carboxymethyl cellulose (CMC) medium the following day. After 4 days of incubation at 37˚C, viral plaques were fixed with 20% formalin and stained using 0.1% naphthol blue solution. The plaques were counted and the 50% end-point plaque reduction neutralization titers (PRNT 50 ) were computed using the log probit method as described by Russell et al. [35]. Antibody titers were expressed as the reciprocal of the end point dilution. The DENV and ZIKV strains used for the PRNT are listed in S1 Table. Samples with ZIKV PRNT 50 �1:30 to both ZIKV strains and DENV PRNT 50 �1:30 to any of the four serotypes indicated the presence of neutralizing activity against the two viruses respectively. The DENV and ZIKV PRNT results of the sample panels are described in S1 Text.
The commercial kits evaluated in this study are listed in Table 2. The tests were performed and results were interpreted according to the manufacturers' recommendations. The result

Data analysis
Diagnostic accuracy indices were calculated for the ZIKV serology assays [36]. The sensitivity, specificity with 95% Wald confidence intervals (CIs) and the Area under Receiver Operating Characteristics Curve (AUROCC) values with 95% CI were determined. The AUROCC is a numerical value outcome (between 0 and 1) that assesses both the test sensitivity and specificity and could be used to determine and compare the diagnostic accuracy of tests. A perfect diagnostic test has an AUROCC value of 1.0 whereas a non-discriminating test has an area of �0.5 [37,38]. The Chi-square test was used for testing significant differences in ZIKV seropositivity obtained by the ZIKV serology assays in primary and secondary DENV samples. Statistical significance was set at p < 0.05. All analyses were performed using SPSS (Version 23, IBM) and R software [39].

Performance of ZIKV IgM and IgG tests on acute and convalescent samples from ZIKV-confirmed, DENV-confirmed, and non-ZIKV and non-DENV patients
When acute samples were tested (Set A, n = 78), ZIKV IgM test sensitivities for rapid ICT and ELISA/IIFT tests ranged from 5.6% to 27.8% (Fig 1A), with the InBios ZIKV IgM ELISA yielding the highest ZIKV IgM test sensitivity of 27.8% (95% CI: 9.69%-53.48%) and highest AUR-OCC value of 0.631 (Table 3). Test specificities for the ZIKV IgM assays ranged from 86.7% to 100% (Fig 1A), with NovaLisa 1 ZIKV IgM ELISA yielding the highest specificity. The low test sensitivities observed using ZIKV acute samples could be due to the time of collection, whereby antibodies have not been produced <5 days after fever onset. This was further explained by the absence of ZIKV NAb in almost 30% of ZIKV acute samples (S1 Text). For acute DENV and non-DENV, non-ZIKV samples, the test specificities of the ZIKV IgM assays were 73.3% -100% and 96.7% -100%, respectively (Fig 2A). The presence of crossreactive DENV antibodies in the DENV-confirmed samples could have resulted in positive ZIKV results, leading to the lower test specificities. The AUROCC analysis revealed overall low values (AUROCC values of 0.461 to 0.631) for ZIKV IgM tests in acute samples ( Table 3).
The test specificities of the ZIKV IgM and IgG tests ranged from 63.3% to 100% in Set B1 (Fig 1B) and 76.7% to 100% in Set B2 (Fig 1C) with NovaLisa 1 ZIKV IgM ELISA, achieving the highest specificity of 100%. Using the DENV convalescent samples in Set B, ZIKV IgM tests generally achieved higher test specificities as compared to ZIKV IgG tests (IgM: 40% -100%; IgG: 63.3% -90%) (Fig 2B). For non-DENV convalescent samples, the test specificities of the ZIKV IgM and IgG tests were 90% -100% and 100%, respectively (Fig 2B). The AUR-OCC values of the ZIKV IgM and IgG tests ranged from 0.582 to 0.952 ( Table 3). The 2 X 2 contingency table of the ZIKV IgM and IgG tests' results are available as S2 Table.

Performance of ZIKV IgG ELISAs on ZIKV and non-ZIKV sera samples from healthy blood donors
The utility of ZIKV IgG ELISAs in ZIKV seroprevalence studies (serosurveys) was further evaluated using residual sera from healthy blood donors that have been tested by PRNT (Set C, n = 79). The test sensitivities of the ZIKV IgG ELISAs ranged from 52.6% to 84.2%, with Diapro ZIKV IgG ELISA yielding the highest sensitivity ( Table 4). The test specificities ranged from 90.0% to 100%, with Euroimmun ZIKV IgG ELISA yielding the highest specificity (100%). Among 60 samples that comprised of an equal number of DENV NAb-only samples and DENV/ZIKV NAb-absent samples, Diapro ZIKV IgG ELISA incorrectly identified six of these samples as ZIKV IgG positive, resulting in reduced test specificity (90%). The Euroimmun ZIKV IgG ELISA did not detect any of these samples as ZIKV IgG positive and had 100% specificity.

Comparison of ZIKV seropositivity in primary and secondary DENV convalescent samples
Studies have shown that DENV-positive sera, especially those from secondary DENV infections, may cross-react with ZIKV [27,28,40,41]. In this study, the ZIKV IgM and IgG assays tested 0% -41.7% of primary DENV samples (n = 12) and 0% -72.2% of secondary DENV samples (n = 18) to be positive for ZIKV antibodies. Similar to previous reports, a higher proportion of samples were observed to be positive for ZIKV antibodies in secondary DENV convalescent samples (n = 12; χ 2 = 5.03-7.74, P<0.05) as compared to primary DENV samples (n = 18) (Table 5), using the Diapro ZIKV IgM and IgG ELISA, and Euroimmun ZIKV IgM  IIFT assays. Conversely, NovaLisa 1 ZIKV IgM ELISA did not detect any of the primary and secondary DENV samples as positives. The serological differentiation between ZIKV and DENV in regions with a high proportion of secondary DENV infections remains challenging and a larger sample size may be required to validate the above observations.

Discussion
We compared the test performance of one ZIKV IgM/IgG rapid ICT, four ZIKV IgM ELISAs, one ZIKV IgM IIFT and two ZIKV IgG ELISAs using characterized sample panels from a dengue endemic setting. Moreover, the gold standard PRNT was used as a reference in this study to determine the seroconversion in ZIKV convalescent samples collected from ZIKV RT-PCR-confirmed patients, the presence of ZIKV NAb in healthy blood donor samples, and the presence of probable cross-reactive DENV NAb. It is advantageous that PRNT is able to establish the presence of ZIKV antibodies. The performance of the assays for use in routine testing, high throughput screening, and serosurveys was assessed. For rapid ICTs, the LumiQuick ZIKV IgM rapid ICT obtained low test sensitivities (5.6% -55.6%), suggesting its limited utility; there are also limited commercial ZIKV ICTs available. This contrasts with the availability of many robust DENV rapid ICTs that have high sensitivities [42]. Regarding test reliability, the LumiQuick ZIKV rapid ICT initially gave nine invalid test results but was resolved upon retesting.
For high throughput screening, the Diapro ZIKV IgM ELISA yielded test sensitivities of 94.1%-96.3% and specificities of 80% -86.7% among ZIKV, DENV and non-DENV convalescent samples. The sensitivity values met the proposed WHO Zika target product profile's acceptable sensitivity of >95% but the specificity values fell short of the acceptable specificity of >98% for the diagnosis of patients with active infection [43]. The test sensitivity results were consistent with other studies that reported sensitivities of 69% -100% [44,45] but test specificities were lower than the reported specificity of 96% [45]. The evaluation data of the ZIKV IgM ELISA/IIFT tests obtained from this study supplemented the limited reports on test specificities in dengue endemic settings and supported the test performance results in prior studies. For instance, the Euroimmun ZIKV IgM ELISA yielded high test specificities of 93.3% -96.7% but lower test sensitivities of 23.5% -51.9%, this corroborating with the results from  [49] and for this study (100%, n = 120) as compared to results reported by Safronetz et al. [50]. These authors reported reduced test specificities of 66% in negative samples (n = 25) and 70% in DENV-positive samples (n = 10); the difference in performance could be due to the fewer number of samples used. The use of a specific ZIKV IgG assay for serosurveys can greatly reduce the false positivity rate in a dengue endemic setting. Although this study demonstrated that the ZIKV IgG tests generally achieved lower test specificities compared to ZIKV IgM tests (IgG: 76.7% -96.7%; IgM: 63.3% -100% when DENV and non-DENV convalescent samples were used), the Euroimmun ZIKV IgG ELISA, however, exhibited high test specificities in this and other studies (90.0% -96.7%, and 100%, respectively) [46,51]. Furthermore, our results showed that its test specificity was 87% when DENV convalescent patient samples were used, and 100% when both the primary DENV patient samples and DENV NAb-positive healthy blood donor samples were used. Notably, the Euroimmun ZIKV IgG ELISA has been used in ZIKV serosurveys conducted in Laos, Martinique, Bolivia and Taiwan [52][53][54][55].
A key focus of the study was the assessment of test specificities for the ZIKV IgM and IgG assays using DENV-confirmed patient samples. Reduced specificities of the ZIKV IgM and IgG tests were observed among DENV convalescent samples, likely due to the increase in cross-reactive DENV antibodies in the DENV convalescent samples. Notably, the basal force of infection in Singapore is approximately 1% and dengue infections (and in turn, our samples) could be primary infections [31]. Although other studies and our results suggest higher cross-reactivity among samples from secondary infections [27,28,40,41], the observation of cross-reactivity among DENV samples from primary infections as well emphasizes the need for specific, yet sensitive assays. In this regard, our study distinguished different test specificities among the commercial ZIKV IgM and IgG assays and provided further information on the assays' performance.
This study highlighted two key challenges of ZIKV serology testing. Firstly, seropositivity of the ZIKV serologic assays was observed in some DENV convalescent samples, due to the presence of cross-reactive DENV antibodies. This finding corroborated with other studies [41,56] where an in-house ZIKV NS1 Blockade of Binding assay found 17 of 86 secondary DENV samples obtained from Nicaraguan positive for ZIKV [41]. In another study, the analyses of DENV-immune samples collected less than 12 weeks post infection revealed IgG cross-reactivity with the ZIKV-Envelope Domain I and III antigens, although the level of cross-reactivity decreased in samples collected after more than 12 weeks post infection [56]. The use of PRNT could offer a partial solution to the serological differentiation of DENV and ZIKV infections. Similar to other studies [3,25,57,58], specific ZIKV NAb responses were detected in primary ZIKV infections, but extensive cross-reactive ZIKV and DENV NAb responses were detected in secondary ZIKV infections.
In this study, test specificities of the recombinant ZIKV NS1-based Euroimmun ZIKV ELI-SAs, Diapro ZIKV ELISAs, and LumiQuick ZIKV IgM/IgG rapid ICT were higher than that of the InBios ZIKV IgM ELISA, which utilizes a recombinant ZIKV envelope glycoprotein as the antigen. The difference in performance of the ZIKV serology tests could be attributed to the ZIKV antigen used in detection. Antibodies to NS1 antigens appear to be more virus-specific when compared to envelope glycoprotein antigens, likely due to the electrostatic differences in the NS1 surface loop among flaviviruses such as DENV, ZIKV and West Nile virus [29,60,61]. Tests that utilizes ZIKV-infected cells such as the Euroimmun ZIKV IgM IIFT also had lower test specificities. This observation corresponds with previous reports that antigen preparations from whole viruses yielded high level of antibody cross-reactivity between ZIKV and DENV [3,25,62]. Of the remaining tests, the NovaLisa 1 ELISA did not specify the ZIKV antigen which is used. In general, if ZIKV serology tests are used in an area where there is no known DENV transmission, tests with high sensitivities would be preferred, and follow-up tests using a more specific method could be done to confirm the results. On the contrary, ZIKV serology tests with high specificities would be preferred if they are performed in areas with known DENV transmission.
This study evaluated a considerable number of ZIKV serologic assays and provided invaluable information of their test performance in a dengue endemic setting.
Supporting information S1