Advertisement
  • Loading metrics

Evaluation of Commercially Available Diagnostic Tests for the Detection of Dengue Virus NS1 Antigen and Anti-Dengue Virus IgM Antibody

Evaluation of Commercially Available Diagnostic Tests for the Detection of Dengue Virus NS1 Antigen and Anti-Dengue Virus IgM Antibody

  • Elizabeth A. Hunsperger, 
  • Sutee Yoksan, 
  • Philippe Buchy, 
  • Vinh Chau Nguyen, 
  • Shamala Devi Sekaran, 
  • Delia A. Enria, 
  • Susana Vazquez, 
  • Elizabeth Cartozian, 
  • Jose L. Pelegrino, 
  • Harvey Artsob
PLOS
x

Abstract

Commercially available diagnostic test kits for detection of dengue virus (DENV) non-structural protein 1 (NS1) and anti-DENV IgM were evaluated for their sensitivity and specificity and other performance characteristics by a diagnostic laboratory network developed by World Health Organization (WHO), the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR) and the Pediatric Dengue Vaccine Initiative (PDVI). Each network laboratory contributed characterized serum specimens for the panels used in the evaluation. Microplate enzyme-linked immunosorbent assay (ELISA) and rapid diagnostic test (RDT formats) were represented by the kits. Each ELISA was evaluated by 2 laboratories and RDTs were evaluated by at least 3 laboratories. The reference tests for IgM anti-DENV were laboratory developed assays produced by the Armed Forces Research Institute for Medical Science (AFRIMS) and the Centers for Disease Control and Prevention (CDC), and the NS1 reference test was reverse transcriptase polymerase chain reaction (RT-PCR). Results were analyzed to determine sensitivity, specificity, inter-laboratory and inter-reader agreement, lot-to-lot variation and ease-of-use. NS1 ELISA sensitivity was 60–75% and specificity 71–80%; NS1 RDT sensitivity was 38–71% and specificity 76–80%; the IgM anti-DENV RDTs sensitivity was 30–96%, with a specificity of 86–92%, and IgM anti-DENV ELISA sensitivity was 96–98% and specificity 78–91%. NS1 tests were generally more sensitive in specimens from the acute phase of dengue and in primary DENV infection, whereas IgM anti-DENV tests were less sensitive in secondary DENV infections. The reproducibility of the NS1 RDTs ranged from 92-99% and the IgM anti-DENV RDTs from 88–94%.

Author Summary

Dengue virus (DENV) infection occurs throughout tropical and sub-tropical regions of the world where dengue is a major public health problem. Laboratory diagnosis of dengue with a single serum specimen obtained during the acute phase of the illness requires tests to detect IgM antibodies to DENV or the virus genome. A previous evaluation of available tests for IgM anti-DENV showed wide variability. The present study examined newly available commercial tests that detect the virus protein NS1, as well as new tests for IgM anti-DENV in microplate or rapid diagnostic test formats. This analytic study used specimens from laboratory confirmed dengue patients worldwide, which makes the results widely generalizable. The study found variability among the microplate ELISAs for both analytes but some tests performed with sensitivity and specificity acceptable for routine dengue diagnostics. The RDT's for both analytes had variable sensitivity that could be considered acceptable for routine clinical diagnostics. There is the need to maintain a network of dengue reference laboratories to conduct similar evaluations as additional dengue diagnostic tests become commercially available in order to guide the use for surveillance, clinical diagnosis and research.

Introduction

Dengue is a major public health problem with more than 2.5 billion people at risk for DENV infection and an estimated 96 million cases occur annually in over 100 tropical and sub-tropical countries [1][3]. Infection with each of the four DENV (DENV serotypes 1–4) is capable of causing dengue fever as well as severe dengue. Currently there are no vaccines or drugs available to prevent or treat dengue. However, early laboratory diagnosis can ensure timely initiation of appropriate clinical management or anticipatory guidance in the outpatient setting. Accurate diagnosis of dengue is an important component of public health surveillance since clinical diagnosis does not differentiate dengue from other diseases that present with dengue-like signs and symptoms (e.g., malaria, leptospirosis, measles, influenza, Japanese encephalitis (JEV), West Nile fever (WNV), yellow fever virus (YFV)). Hence, there is the global need for accurate dengue diagnostics.

Timely and accurate laboratory diagnosis of dengue performed on a single serum specimen must rely on detection of DENV RNA or NS1 antigen during the period from fever onset until 5–6 days later, or detection of anti-DENV IgM beginning 3–5 days after fever onset until 6 weeks later [4][6]. DENV can be detected by virus isolation, molecular amplification of DENV RNA by RT-PCR and immunoassay to detect DENV NS1 antigen. As a diagnostic technique, virus isolation is not practical since it requires cell culture facilities, has a long turn-around time and has lower sensitivity compared to molecular or immunoassay methods [7]. In low resource settings, use of molecular tests is generally not feasible hence NS1 antigen detection may be the best option for DENV detection. The NS1 test appears to have adequate sensitivity and specificity when compared to RT-PCR and virus isolation across DENV serotypes; however, there are differences in NS1 sensitivity related to patient infection status (i.e., primary versus secondary DENV infection) [8][14]. Presently, the most widely used dengue diagnostic test remains the IgM capture anti-DENV (MAC) ELISA, which lacks adequate sensitivity and specificity in the acute phase of the illness, a time when most patients seek medical attention [15], [16].

Rapid diagnostic tests (RDTs) for dengue have become increasing available over the last 5 years because of the need for point-of-care diagnosis in resource limited settings. However, previously available RDTs for IgM anti-DENV did not have acceptable sensitivity compared to anti-DENV IgM microplate ELISAs which considerably limited their utility [17]. Although individual NS1 RDTs have been evaluated in a number of studies, robust estimates of their analytic performance have not been determined in head-to-head comparisons using specimen panels that replicate the diagnostic landscape of most dengue endemic areas.

To determine the analytic performance and reproducibility of commercially available NS1 antigen tests and newly available IgM anti-DENV RDTs and microplate ELISAs, a network of laboratories established by WHO/TDR and the PDVI developed specimen panels from dengue patients infected with all DENV serotypes from both the Asian and American continents, and with primary and secondary DENV infections. In addition, the evaluation included specimens from patients with other dengue-like illnesses and other conditions that can produce false-positive test results.

Methods

The Laboratory Network

The present study was conducted by the seven laboratories of the WHO/TDR/PDVI dengue diagnostic network that previously evaluated commercially available IgM anti-DENV diagnostic tests in 2007 [17]. Laboratories in Asia included: Centre for Vaccine Development, Mahidol University, Thailand; Hospital for Tropical Medicine, Cho Quan Hospital, Viet Nam; Institut Pasteur in Cambodia; and University of Malaya, Malaysia; and in the Americas: Dengue Branch, CDC, Puerto Rico; Instituto Medicina Tropical “Pedro Kouri”, Cuba; and Instituto Nacional Enfermedades Virales Humanas “Dr. Julio I. Maiztegui”, Argentina. The laboratories at CDC, Puerto Rico and Mahidol University, Thailand served as the reference laboratories and provided specimens for proficiency testing among the laboratories, assembled and validated the evaluation panels and distributed the test kits.

Ethics Statement

This study was reviewed by the WHO Ethics Review Committee and considered exempt. All patient samples used for this study were de-linked from personal identifiers and renumbered with a study identifier.

Study Initiatives

The mission of the Accessible Quality Assured Diagnostics program within the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (WHO/TDR) is to promote and facilitate development, evaluation and deployment of diagnostic tools for the control of the tropical diseases. PDVI was a product development partnership established to accelerate development, evaluation and introduction of dengue vaccines for children in endemic countries.

The previous study by the WHO/TDR/PDVI dengue diagnostic laboratory network resulted in recommendations to included three commercially available IgM anti-DENV ELISA tests in the WHO bulk procurement scheme (WHO report; [17]). Since this study, a number of tests for detection of NS1 antigen had become commercially available in addition to a number of new anti-DENV IgM tests. On 10–12 February 2009 the steering group and directors of the network laboratories met to plan the present study to evaluate the analytic performance and operational characteristics of newly available anti-DENV IgM and NS1 antigen detection tests in microplate ELISA and RDT formats for the diagnosis of dengue.

Clinical Specimens

Specimen panels were created for anti-DENV IgM and NS1 testing (Table 1 and 2) based on the study design developed by the steering group with statistical consultation. Panels contained a sufficient number of true positive and negative specimens to give a confidence interval (CI) of +5% around the point estimates for test sensitivity and specificity [18]. Specimens were selected in a way that achieved geographic representation of the four DENV serotypes and were obtained from archived clinical samples at the participating laboratories and were submitted to the reference laboratories for assembly of the final panels. Case-specimens came from dengue patients that met the 1997 WHO dengue case classification and were laboratory confirmed by the presence of DENV detected by RT-PCR and/or virus isolation. All dengue cases included in the panels had paired, acute (≤5 days post onset of illness) and convalescent (6–14 days post onset of illness) specimens. However, the study design was not such that specimens for the acute and convalescent periods were allocated by specific days post onset of illness (see Supplemental Table S1), and analysis had to be performed for results from the aggregate acute and convalescent time periods. For certain categories in the challenge panels, serum specimens were purchased from SeraCare Diagnostics (West Bridgewater, MA).

thumbnail
Table 1. Characteristics of specimens from dengue patients in the panel to evaluate dengue virus (DENV) non-structural protein 1 (NS1) detection kits.

http://dx.doi.org/10.1371/journal.pntd.0003171.t001

thumbnail
Table 2. Characteristics of specimens from dengue patients included in the panel to evaluate anti-DENV IgM detection kits.

http://dx.doi.org/10.1371/journal.pntd.0003171.t002

The following epidemiological data were available for all case-specimens: date of onset of symptoms, date of sample collection, patient age, sex, summary of clinical history and diagnosis, travel history 12 months prior to the date of specimen collection, and vaccination history (i.e. YFV, JEV or other flavivirus vaccines). Case-patient specimens were classified as dengue fever (DF), dengue hemorrhagic fever (DHF), or dengue shock syndrome (DSS) based on data from clinical records using the 1997 WHO case definition.

Lipemic, hemolyzed or bacterially contaminated specimens were not included in the panels. Only specimens with ≤2 freeze-thaw cycles were included, all were heat-inactivated at 56°C for 30 minutes, lyophilized in aliquots, coded by the reference laboratories, assembled into the respective panels which were shipped to each evaluation laboratory. Written standard operating procedures were developed and used by participating laboratories to assure uniformity in sample handling, storage and testing. Because some commercial tests can be highly sensitive to storage and shipping conditions, data on transit time was recorded for later analysis.

Testing Protocol

Letters requesting study participation were sent to 20 dengue diagnostic kit manufacturers; seven companies agreed to participate and provided the tests shown in Table 3 and supplemental Table S2 A–D. All kits were shipped by the manufacturers at ambient temperatures, checked for damage upon arrival and stored in the specified environment. RDT kits were shipped to only one of the reference laboratories where they were subsequently shipped to the evaluation laboratories under conditions specified by the manufacturer. Microplate ELISAs were only evaluated by the reference laboratories since the previous evaluation did not show significant inter-laboratory differences in results [17]. Delays in shipping or damage were noted and reported by the evaluation laboratories.

thumbnail
Table 3. Dengue virus NS1 antigen detection and IgM anti-dengue virus tests submitted for evaluation.

http://dx.doi.org/10.1371/journal.pntd.0003171.t003

Ten specimens known to be NS1 positive using the Platelia ELISA (BioRad, Lyon, France) were tested following lyophylization, heat inactivation and one freeze-thaw cycle and showed no change in sensitivity by any of these treatments (data not shown).

Specimens from each panel were tested in duplicate in the same analytical run to measure within-run precision; inter-run precision was evaluated by comparing results across evaluation laboratories. RDTs were tested in duplicates, with two readers evaluating each result independently. Inter-reader agreement was considered an important variable since RDTs required subjective judgment. All testing was conducted with strict adherence to the manufacturer's protocol described in the package insert. Each evaluation laboratory recorded testing data on standardized data sheets, which were submitted to WHO/TDR for analysis. Information was also collected from each performing technician regarding the ease-of-use of these products. This study was initiated in 2011 and completed within the same year.

The Panels

DENV NS1 panel.

This panel consisted of a total of 390 dengue patient serum specimens in 3 sub-panels: 192 dengue patients who were DENV positive by virus isolation and/or RT-PCR for DENV RNA, 146 DENV RNA or culture negatives and 52 challenge specimens from other illnesses or conditions that may result in false positive results to determine the specificity of these tests (Tables 1 and 4). DENV RNA or culture positive serum specimens were collected 0–5 days post onset of fever (DPO). All dengue cases had paired acute (DPO 0–5) and convalescent (DPO 6–14) specimens.

thumbnail
Table 4. Characteristics of DENV negative specimens and challenge specimens for the evaluation of NS1 and anti-DENV IgM tests.*

http://dx.doi.org/10.1371/journal.pntd.0003171.t004

Network laboratories tested a limit of detection panel to ensure comparable sensitivity of their RT-PCR assay as a reference standard. The methods of RNA extraction and RT-PCR used in the network laboratories included: nested RT-PCR by Lanciotti et al., 1992 [18], real-time RT-PCR methods developed by Laue et al., 1999 [19] Kong et al., 2006 [20]and Chien et al., 2006 [21]. Network laboratories tested a panel of serially diluted cell culture-derived DENV RNA to determine the limit of detection of their RT-PCR assays and ensure comparable testing sensitivity. For virus isolation, cell culture isolation was performed using an accepted reference method [22].

DENV positive specimens included patients with primary and secondary DENV infections and represented all 4 DENV serotypes. Primary and secondary infections were distinguished according to criteria established by WHO using hemagglutination-inhibition assay described by Clark and Casals, 1958 adapted to 96-well microtiter plate [23], [24]. This approach was adapted to IgG ELISA where a primary DENV infection was defined as evidence of first exposure to DENV based on anti-DENV IgG seroconversion between and acute and convalescent specimen, and a secondary DENV infection was defined as evidence of more than one DENV infection as determined by anti-DENV IgG positive titer during the acute phase of disease [25].

This panel also included paired, acute and convalescent specimens from patients who were DENV RNA and/or virus isolation positive and anti-DENV IgM negative in the acute phase specimens but DENV RNA and/or virus isolation negative and anti-DENV IgM positive in the convalescent specimens. These specimens were used to determine the duration of NS1 antigen detection over the course of the illness. The NS1 panel was used to test microtiter ELISAs and RDTs. All tests were performed in duplicates including the RDTs. Since each RDT represents a single test the number of specimens in the table are including the duplicate and is exactly twice as many specimens number as the ELISA tests.

Negative NS1 panel.

The subpanels of DENV negatives and challenge specimens were obtained from normal, healthy persons living in dengue non-endemic areas and were negative for DENV by RT-PCR and/or virus isolation, and IgM and IgG anti-DENV negative. The challenge specimens included individuals with non-DENV flavivirus illnesses, febrile illnesses of other etiologies, or systemic conditions known to give rise to false positive results in immunoassays (Table 4) and some were purchased from a commercial source (SeraCare, West Bridgewater, MA).

Anti-DENV IgM panel.

This panel consisted of 3 subpanels with 527 patient serum specimens: 228 anti-DENV IgM positive paired specimens from patients with dengue (Table 2) and 155 DENV IgM negative and 144 challenge specimens (Table 4). All specimens were tested by the solid-phase anti-DENV IgM MAC-ELISA used by CDC or AFRIMS [26], [27]; a positive result by either test was considered the reference value. All samples were from paired specimens as previously defined for the NS1 panel. Positive specimens were selected based on optical density (OD) in the respective tests, and weighted towards low and medium ODs to increase the panel's power to evaluate test sensitivity.

Anti-DENV IgM negative panel.

DENV IgM negative specimens were obtained from healthy individuals living in dengue non-endemic areas and were negative for DENV RNA and/or virus isolation, and IgM and IgG anti-DENV. Challenge specimens were obtained from patients with febrile illness of other etiologies, non-DENV flavivirus illnesses or systemic conditions known to give rise to false positive results in immunoassays (Table 4). Challenge specimens were solid-phase anti-DENV IgM negative and some were purchased from a commercial source (SeraCare, West Bridgewater, MA).

Statistical Analysis

Sensitivity and specificity were assessed against reference test by analyte evaluated (NS1 or IgM). Kappa is a measure of the degree of non-random agreement between observers or measurements of the same categorical variable. Agreement is considered as good if kappa is between 0.60 and 0.80, and very good if greater than 0.80.

The McNemar paired test was used to compare the difference of agreement between operators. Proportions were compared using the chi-square test or the Fischer exact test, whichever was appropriate. Confidence intervals were set at 95% (CI95). P-values lower than 0.05 were considered as statistically significant. The statistical program used was STATA (version 11, Stata Corp.).

Results

DENV NS1 Microplate ELISAs

The sensitivities of the NS1 ELISAs in specimens from dengue patients during the acute phase of illness ranged from 60–75% (Table 5). Based on the overall results, test performance in order of most sensitive to least sensitive was: Panbio E, Standard Diagnostics and BioRad Platelia.

thumbnail
Table 5. Sensitivity of DENV NS1 antigen compared to RT-PCR and/or virus isolation in acute and positive rate of the convalescent specimens.

http://dx.doi.org/10.1371/journal.pntd.0003171.t005

When specimens collected at DPO 6–14 were tested, the test with the highest positive rate was Standard Diagnostics 31% (CI95 21–40 p = <.001) and the lowest positive rate was Panbio E 19% (CI95 11–27; p<.001), (Table 5). All of these rates were statistically different from those observed in the acute phase specimens (DPO 0–5) and were statistically different between tests.

Analysis by primary versus secondary DENV infection showed that sensitivity was significantly affected for all tests. For persons with a primary DENV infection, the test with the highest sensitivity was Standard Diagnostics 75% (CI95 62–88), and the least sensitive was Bio-Rad Platelia 60% (CI95 46–75); these results were significantly higher than observed among dengue cases with secondary infections: Standard Diagnostics 46% (CI95 38–54; p = .001), BioRad Platelia 42% (CI95 34–50; p = .03). This analysis included both acute and convalescent specimens (Table 5 and 6).

thumbnail
Table 6. Sensitivity of anti-DENV IgM compared to IgM reference test in acute and convalescent samples.

http://dx.doi.org/10.1371/journal.pntd.0003171.t006

The specificity of these kits, determined with the DENV negative and challenge specimen panels, is shown in Figure 1 and ranged from 71–80%. Few specimens gave a false positive result in the challenge panel; rheumatoid factor (RF) caused a false positive result in 78% and 11% of the Panbio E and BioRad Platelia NS1 ELISA, respectively and positive specimens for YFV and non-DENV flavivirus infections were reactive in the Panbio NS1 ELISA (Figure 1).

thumbnail
Figure 1. False positive rate of DENV NS1 ELISAs in DENV negatives and challenge panel specimens.

http://dx.doi.org/10.1371/journal.pntd.0003171.g001

DENV NS1 RDTs

The sensitivities of NS1 RDTs in the acute phase specimens (DPO 0–5) ranged from 40 to 59%: Panbio 60% (CI95 54–67) was most sensitive and CTK Biotech 40% (CI95 31–49) the least (Table 5). Among specimens collected in the convalescent phase of dengue (DPO 6–14), the highest test positive rate was for the SD BIOLINE Dengue Duo 59% (CI95 51–66); and the lowest test positive was the Panbio 12% (CI95 7–17); With the exception of SD BIOLINE Dengue Duo RDT, all of these were lower than the rates observed in the acute phase specimens.

Analysis by primary versus secondary DENV infection showed that most of the tests performed best among specimens from patients with primary infections: SD BIOLINE Dengue Duo 71%, (CI95 60–81) vs. 55% (CI95 50–61), (p = .016); Bio–Rad RDT 59% (CI95 48–70) vs. 31% (CI95 25–36), (p<.001); CTK Biotech 47% (CI95 36–59) vs. 21% (CI95 16–27), (p<.001), and Panbio 38% (CI95 28–49) vs. 38% (CI95 32–43) (p = .915) (Table 5).

All four RDTs had similar specificities in the DENV negative specimens (76–80%) (Figure 2). Against the challenge specimens, the CTK Biotech test had reactivity against RF and the SD BIOLINE Dengue Duo and Panbio tests had reactivity against Hantavirus. All the NS1 RDTs had very good agreement between operators with kappa values ranging from 0.84–0.99 (Table 7).

thumbnail
Figure 2. False positive rate DENV NS1 rapid diagnostic tests in DENV negatives and challenge panel specimens.

http://dx.doi.org/10.1371/journal.pntd.0003171.g002

Anti-DENV IgM ELISA

Only Venture E submitted a test kit for evaluation which showed an overall sensitivity of 96% (CI95 94–99). Comparison of acute versus convalescent specimens demonstrated 98% (CI95 95–100) sensitivity in acute phase specimens and 97% (CI95 94–100) sensitivity in convalescent specimens. When analyzed according to DENV infection status, the sensitivity was 97% (CI95 92–100) in primary infections compared to 96% (CI95 94–99) in secondary DENV infections (Table 6). The overall specificity was 84% (CI95 80–89) against the DENV negative panel. False positives reactions were observed against other flavivirus infections - St Louis Encephalitis virus (SLEV), Japanese Encephalitis virus (JEV) and West Nile virus (WNV), as well as Chikungunya virus (CHIKV) and Hantavirus. False positives rates were also observed for malaria, leptospirosis and scrub typhus (Figure 3).

thumbnail
Figure 3. False positive rate of anti-DENV IgM ELISA (Venture Technologies Sdn Bhd) in DENV negatives and challenge panel specimens.

http://dx.doi.org/10.1371/journal.pntd.0003171.g003

Anti-DENV IgM RDTs

The overall sensitivities of the IgM RDTs ranged from 52–95%. In the acute phase panel, the most sensitive test was Orgenics 95% (CI95 90–99), and the least sensitive was CTK Biotech 46% (CI95 36–55) (Table 6). In the convalescent panel sensitivities were higher than those observed in the acute panel for the following tests: CTK Biotech  = 53% (CI95 48–59; p = .155) and SD BIOLINE Dengue Duo IgM = 98% (CI95 96–100; p = .002). The following tests had lower sensitivities in the convalescent panel when compared to the acute panel: Orgenics 82% (CI95 78–87; p<.001) and Abon 56% (CI95 51–61; p = .227).

When evaluated based on DENV infection status, the SD BIOLINE Dengue Duo had a sensitivity of 96% (CI95 91–100) in primary infections compared to 84% (CI95 80–88) for secondary infections (p = .009) and the Abon test had a sensitivity of 75% (CI95 65–86) for primary infections compared to 55% (CI95 50–60) for secondary infections (p = .002). RDTs with statistically significant increase in sensitivity in secondary infections, included: Orgenics 84% (CI95 76–93) compared to 97% (CI95 96–99) (p<.001) and CTK Biotech 30% (CI95 19–41) compared to 55% (CI95 50–60) (p = <.001) for primary and secondary, respectively (Table 6).

The overall specificity of these RDTs ranged from 86–92%, with the highest false positive rate in the CTK Biotech test for RF and systemic lupus erythematosus (Figure 4). The kappa values for reader-to-reader variability ranged from 0.76–0.99 (Table 7).

thumbnail
Figure 4. False positive rate of anti-DENV IgM rapid diagnostic tests in DENV negatives and challenge panel specimens.

http://dx.doi.org/10.1371/journal.pntd.0003171.g004

Discussion

This laboratory-based, analytic evaluation of commercially available dengue diagnostic tests for NS1antigen and anti-DENV IgM used a sufficient number of specimens to yield confidence intervals of +5% around the point estimates of test sensitivity and specificity [28]. Test sensitivity was determined in the acute and convalescent phases of dengue illness where a reference test was available (i.e., IgM anti-DENV) and in patients with primary and secondary DENV infections. The goal of this study was to provide reliable information on product performance to those performing dengue diagnostics, as well as to the test manufacturers.

DENV NS1 antigen is present in the acute phase of dengue with peak detection at DPO 3 followed by a rapid decline through the convalescent phase (DPO>5) [29]. The 3 microplate ELISAs evaluated in our study showed similar findings. During the convalescent phase of illness, previous studies have shown test positivity ranging from 22% to 40% among dengue cases defined by a positive IgM anti-DENV seroconversion between acute and convalescent specimens [12], [30], [31], a finding similar to ours.

Since NS1 is a viral antigen, the only available reference standard has been the presence of DENV as determined by RT-PCR. However, viral antigens may persist in serum longer than viral nucleic acid detected by molecular amplification or virus detected by cell culture. Thus, the interpretation of NS1 ELISA and RDT results obtained from specimens in the convalescent panel is not straightforward. All specimens in our panel were anti-DENV IgM positive but negative for DENV by RT-PCR or virus isolation. By strict definition, we were not able to determine the sensitivity of these tests in convalescent phase specimens because there was no reference standard for comparison. One interpretation is these are false-positive results since the specimens were DENV RNA negative. However, it has been shown in longitudinal studies of dengue patients that NS1 antigen remains positive after DENV RNA amplicons disappear [12], [30], [31] probably because of longer half-life of NS1 protein. The 19–30% test positivity in the convalescent panel could represent detection of residual NS1 antigen although some of these could also be false positive results. Until a protein based reference standard for NS1 is developed or a confirmatory test for NS1 test-reactivity is developed, study designs that follow patients sequentially during their illness are required to determine the diagnostic utility of this test in the early convalescent phase of dengue.

Our study showed higher sensitivity in primary than secondary DENV infections for both NS1 ELISAs and RDTs, with the exception of the Panbio RDT which had equal sensitivity (Table 5). The analysis included specimens from both acute and convalescent phase. This difference in NS1 detection has been observed previously [12], [30], [31] and may be explained by presence of anti-NS1 IgG, which occurs most frequently in secondary DENV infections, and may mask antigen detection by immune complex formation. Higher NS1 ELISA sensitivity has been reported for primary DENV infections not containing NS1 immune complexes compared to secondary DENV infections with complexes.

Although generally the NS1 RDTs had reduced sensitivities compared to the NS1 ELISAs, they had similar specificities, and the RDT's good agreement (>.80) between operators. However, there were still differences noted between laboratories as observed previously in the evaluation of the IgM anti-DENV RDTs [17].

As might be expected, there were significant differences in sensitivity between acute and convalescent specimens for the IgM anti-DENV RDTs because anti-DENV IgM peaks at DPO 10, except for the Abon RDT which had essentially similar sensitivity in the both periods. Anti-DENV IgM titers have been shown to be higher in primary compared to secondary infections, with as many as 20% of dengue cases with secondary infection having undetectable levels of anti-DENV IgM in convalescent specimens [16]. When examined by DENV infection status, the present evaluation showed that two of the tests, CTK Biotech and Orgenics, had significantly higher sensitivity in secondary compared to primary infections.

The utility of using both IgM anti-DENV and NS1 antigen for dengue diagnostic testing has been evaluated in recent studies, suggesting an added diagnostic benefit to this testing combination. [14], [32] Measuring both analytes during the early course of the illness (DPO 0–8) expands the diagnostic window of opportunity since NS1 detection is best detected during DPO 0–5 and anti-DENV IgM is best detected during DPO 5–14.

This study had several limitations as well as strengths. Test performance was compared to the evaluation panels, which were only characterized by the two reference laboratories using the reference methods. Additionally, the NS1 and IgM anti-DENV test sensitivity could not be evaluated by DENV serotype because of an insufficient number of specimens to allow for this sub-analysis. In addition, the small number of samples in some of the challenge panel categories did not provide strong point and confidence interval estimates similar to those derived for the DENV positive panels, and only identified potential issues. Lastly, the infectious agents represented in the challenge panels may not be of epidemiologic importance in all populations where dengue occurs. The convalescent specimens used for the NS1 evaluation were not characterized with the same reference tests as the acute specimens and no further testing was performed to determine if a positive result was a true or false positive. The strength of the study was that a high proportion of specimens in the panels were from persons with secondary DENV infections which, reflects the situation in most dengue endemic countries. In addition, most tests tended to perform better for primary DENV infections, which would mitigate any concerns about test performance in patients with this infection status (e.g., travelers from non-endemic countries).

Results from this evaluation have been provided to the manufacturers and WHO member states and provide benchmarks for the evaluation, procurement and use of dengue diagnostic tests at the country or regional level. This study indicates that most of the NS1 microplate ELISAs performed adequately for routine, clinical diagnostic use in dengue endemic countries. However, the clinical and epidemiologic consequences of the lower sensitivity of NS1 tests compared to DENV RNA detection should be considered when using these tests for routine use.

The microplate anti-DENV IgM ELISA continues to show good performance and this evaluation indicates that the one test evaluated had good sensitivity in the early phase of the illness, an area that has been problematic in the past. Whether these tests should be combined with NS1 microplate ELISAs for routine testing or in a testing algorithm based on when a patient presents for clinical care, needs to be evaluated in large-scale clinical studies.

Unfortunately the RDTs for both NS1 and anti-DENV IgM continue to have relatively poor performance profiles; a problem for resource poor countries. Since the combination NS1/anti-DENV IgM RDTs was only evaluated for each analyte individually, no statement can be made regarding its performance as a combination test. This study did not address the issue of how RDTs, with their lower sensitivity, might be used in resource poor areas. Even with their lower sensitivity but comparable specificities compared the ELISAs, they could be used to identify dengue outbreaks. However, large studies are needed to determine the most effective way to use them for case management, disease surveillance or whether their present shortcomings can be overcome by a combination testing algorithm.

Supporting Information

Checklist S1.

STARD checklist and STARD flowchart.

doi:10.1371/journal.pntd.0003171.s001

(DOC)

Table S1.

Number of dengue virus (DENV) positive specimens in the non-structural protein 1 (NS1) panel (Table 1) by infection status (primary vs. secondary DENV infection) and days post onset of fever (DPO).

doi:10.1371/journal.pntd.0003171.s002

(DOCX)

Table S2.

Commercial test characteristics of: A) NS1 ELISAs B) NS1 rapid diagnostic tests (RDTs) C) anti-DENV IgM ELISA and D) anti-DENV IgM RDTs.

doi:10.1371/journal.pntd.0003171.s003

(DOCX)

Acknowledgments

Special thanks to WHO/TDR Staff including Ms. Isabela Suder Dayao and Ms. Laurie Ingels for their administrative support. We would also like to acknowledge the WHO/TDR Study Steering Committee: Duane Gubler, Harvey Artsob, Elizabeth Hunsperger, Maria G. Guzman, Rosanna Peeling, Michael Drebot and Sutee Yoksan.

Disclaimer

PO is a staff member of the WHO and JP, MG and RP were staff members of the WHO at the time of this study; the authors alone are responsible for the views expressed in this publication and they do not necessarily represent the decisions, policy or views of the WHO.

Author Contributions

Conceived and designed the experiments: EAH SY HSM RWP SV PB SDS VCN DAE HA MGG SK SH. Performed the experiments: EC EAH VCN DAE PB SDS SV SY. Analyzed the data: JZ MG PO RWP EAH. Contributed reagents/materials/analysis tools: EAH SY HSM SV PB SDS VCN DAE MGG. Wrote the paper: EAH HSM RWP. Contributed to the logistics of kits: JLP.

References

  1. 1. Calisher CH (2005) A very brief history of arbovirology, focusing on contributions by workers of the Rockefeller Foundation. Vector-Borne and Zoonotic Diseases 5: 202–211. doi: 10.1089/vbz.2005.5.202
  2. 2. Gubler DJ (2002) Epidemic dengue/dengue hemorrhagic fever as a public health, social and economic problem in the 21st century. Trends Microbiol 10: 100–103. doi: 10.1016/s0966-842x(01)02288-0
  3. 3. Bhatt S, Gething PW, Brady OJ, Messina JP, Farlow AW, et al. (2013) The global distribution and burden of dengue. Nature 496: 504–507. doi: 10.1038/nature12060
  4. 4. Libraty DH, Young PR, Pickering D, Endy TP, Kalayanarooj S, et al. (2002) High circulating levels of the dengue virus nonstructural protein NS1 early in dengue illness correlate with the development of dengue hemorrhagic fever. Journal of Infectious Diseases 186: 1165–1168. doi: 10.1086/343813
  5. 5. Young PR, Hilditch PA, Bletchly C, Halloran W (2000) An antigen capture enzyme-linked immunosorbent assay reveals high levels of the dengue virus protein NS1 in the sera of infected patients. Journal of Clinical Microbiology 38: 1053–1057.
  6. 6. Xu H, Di B, Pan Y-x, Qiu L-w, Wang Y-d, et al. (2006) Serotype 1-specific monoclonal antibody-based antigen capture immunoassay for detection of circulating nonstructural protein NS1: Implications for early diagnosis and serotyping of dengue virus infections. Journal of Clinical Microbiology 44: 2872–2878. doi: 10.1128/jcm.00777-06
  7. 7. Lolekha R, Chokephaibulkit K, Yoksan S, Vanprapar N, Phongsamart W, et al. (2004) Diagnosis of dengue infection using various diagnostic tests in the early stage of illness. Southeast Asian J Trop Med Public Health 35: 391–395.
  8. 8. Bessoff K, Delorey M, Sun W, Hunsperger E (2008) Comparison of two commercially available dengue virus (DENV) NS1 capture enzyme-linked immunosorbent assays using a single clinical sample for diagnosis of acute DENV infection. Clinical and Vaccine Immunology 15: 1513–1518. doi: 10.1128/cvi.00140-08
  9. 9. Blacksell SD, Mammen Jr MP, Thongpaseuth S, Gibbons RV, Jarman RG, et al. (2008) Evaluation of the Panbio dengue virus nonstructural 1 antigen detection and immunoglobulin M antibody enzyme-linked immunosorbent assays for the diagnosis of acute dengue infections in Laos. Diagnostic Microbiology and Infectious Disease 60: 43–49. doi: 10.1016/j.diagmicrobio.2007.07.011
  10. 10. Dussart P, Petit L, Labeau B, Bremand L, Leduc A, et al. (2008) Evaluation of two new commercial tests for the diagnosis of acute dengue virus infection using NS1 antigen detection in human serum. PLoS neglected tropical diseases 2: e280. doi: 10.1371/journal.pntd.0000280
  11. 11. Tricou V, Vu HT, Quynh NV, Nguyen CV, Tran HT, et al. (2010) Comparison of two dengue NS1 rapid tests for sensitivity, specificity and relationship to viraemia and antibody responses. BMC Infect Dis 10: 142. doi: 10.1186/1471-2334-10-142
  12. 12. Pok KY, Lai YL, Sng J, Ng LC (2010) Evaluation of nonstructural 1 antigen assays for the diagnosis and surveillance of dengue in Singapore. Vector Borne Zoonotic Dis 10: 1009–1016. doi: 10.1089/vbz.2008.0176
  13. 13. Wang SM, Sekaran SD (2010) Early diagnosis of Dengue infection using a commercial Dengue Duo rapid test kit for the detection of NS1, IGM, and IGG. Am J Trop Med Hyg 83: 690–695. doi: 10.4269/ajtmh.2010.10-0117
  14. 14. Wang SM, Sekaran SD (2010) Evaluation of a commercial SD dengue virus NS1 antigen capture enzyme-linked immunosorbent assay kit for early diagnosis of dengue virus infection. J Clin Microbiol 48: 2793–2797. doi: 10.1128/jcm.02142-09
  15. 15. Vaughn DW, Green S, Kalayanarooj S, Innis BL, Nimmannitya S, et al. (2000) Dengue viremia titer, antibody response pattern, and virus serotype correlate with disease severity. Journal of Infectious Diseases 181: 2–9. doi: 10.1086/315215
  16. 16. Innis BL, Nisalak A, Nimmannitya S, Kusalerdchariya S, Chongswasdi V, et al. (1989) An enzyme-linked immunosorbent assay to characterize dengue infections where dengue and Japanese encephalitis co-circulate. American Journal of Tropical Medicine & Hygiene 40: 418–427.
  17. 17. Hunsperger EA, Yoksan S, Buchy P, Nguyen VC, Sekaran SD, et al. (2009) Evaluation of commercially available anti-dengue virus immunoglobulin M tests. Emerging Infectious Diseases 15: 436–440. doi: 10.3201/eid1503.080923
  18. 18. Lanciotti RS, Calisher CH, Gubler DJ, Chang GJ, Vorndam AV (1992) Rapid detection and typing of dengue viruses from clinical samples by using reverse transcriptase-polymerase chain reaction. Journal of Clinical Microbiology 30: 545–551.
  19. 19. Laue T, Emmerich P, Schmitz H (1999) Detection of dengue virus RNA in patients after primary or secondary dengue infection by using the TaqMan automated amplification system. Journal of Clinical Microbiology 37: 2543–2547.
  20. 20. Kong YY, Thay CH, Tin TC, Devi S (2006) Rapid detection, serotyping and quantitation of dengue viruses by TaqMan real-time one-step RT-PCR. Journal of Virological Methods 138: 123–130. doi: 10.1016/j.jviromet.2006.08.003
  21. 21. Chien LJ, Liao TL, Shu PY, Huang JH, Gubler DJ, et al. (2006) Development of real-time reverse transcriptase PCR assays to detect and serotype dengue viruses. Journal of Clinical Microbiology 44: 1295–1304. doi: 10.1128/jcm.44.4.1295-1304.2006
  22. 22. Gubler DJ, Kuno G, Sather GE, Velez M, Oliver A (1984) Mosquito cell cultures and specific monoclonal antibodies in surveillance for dengue viruses. American Journal of Tropical Medicine & Hygiene 33: 158–165.
  23. 23. Clarke DH, Casals J (1958) Techniques for hemagglutination and hemagglutination-inhibition with arthropod-borne viruses. Am J Trop Med Hyg 7: 561–573.
  24. 24. WHO (1997) Dengue and dengue hemorrhagic fever in the Americas:guidelines for prevention and control.World Health Organization: .
  25. 25. Miagostovich MP, Nogueira RM, dos Santos FB, Schatzmayr HG, Araujo ES, et al. (1999) Evaluation of an IgG enzyme-linked immunosorbent assay for dengue diagnosis. Journal of Clinical Virology 14: 183–189. doi: 10.1016/s1386-6532(99)00059-1
  26. 26. Innis BL, Nisalak A, Nimmannitya S, Kusalerdchariya S, Chongswasdi V, et al. (1989) An enzyme-linked immunosorbent assay to characterize dengue infections where dengue and Japanese encephalitis co-circulate. American Journal of Tropical Medicine and Hygiene 40: 418–427.
  27. 27. Martin DA, Muth DA, Brown T, Johnson AJ, Karabatsos N, et al. (2000) Standardization of immunoglobulin M capture enzyme-linked immunosorbent assays for routine diagnosis of arboviral infections. Journal of Clinical Microbiology 38: 1823–1826.
  28. 28. Panel TDRDEE, Banoo S, Bell D, Bossuyt P, Herring A, et al. (2010) Evaluation of diagnostic tests for infectious diseases: general principles. Nat Rev Microbiol 8: S17–29. doi: 10.1038/nrmicro1523
  29. 29. Libraty DH, Young PR, Pickering D, Endy TP, Kalayanarooj S, et al. (2002) High circulating levels of the dengue virus nonstructural protein NS1 early in dengue illness correlate with the development of dengue hemorrhagic fever. Journal of Infectious Diseases 186: 1165–1168. doi: 10.1086/343813
  30. 30. Bessoff K, Phoutrides E, Delorey M, Acosta LN, Hunsperger E (2010) Utility of a commercial nonstructural protein 1 antigen capture kit as a dengue virus diagnostic tool. Clin Vaccine Immunol 17: 949–953. doi: 10.1128/cvi.00041-10
  31. 31. Dussart P, Petit L, Labeau B, Bremand L, Leduc A, et al. (2008) Evaluation of two new commercial tests for the diagnosis of acute dengue virus infection using NS1 antigen detection in human serum. PLoS Neglected Tropical Diseases [electronic resource] 2: e280.
  32. 32. Blacksell SD, Jarman RG, Gibbons RV, Tanganuchitcharnchai A, Mammen MP Jr, et al. (2012) Comparison of seven commercial antigen and antibody enzyme-linked immunosorbent assays for detection of acute dengue infection. Clin Vaccine Immunol 19: 804–810. doi: 10.1128/cvi.05717-11