Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Enhanced Sensitivity for Detection of HIV-1 p24 Antigen by a Novel Nuclease-Linked Fluorescence Oligonucleotide Assay

  • Peihu Fan,

    Affiliations School of Life Sciences, Jilin University, Changchun, Jilin, China, National Engineering Laboratory for Acquired Immune Deficiency Syndrome Vaccine, Jilin University, Changchun, Jilin, China

  • Xiaojun Li,

    Affiliations School of Life Sciences, Jilin University, Changchun, Jilin, China, National Engineering Laboratory for Acquired Immune Deficiency Syndrome Vaccine, Jilin University, Changchun, Jilin, China

  • Weiheng Su,

    Affiliations School of Life Sciences, Jilin University, Changchun, Jilin, China, National Engineering Laboratory for Acquired Immune Deficiency Syndrome Vaccine, Jilin University, Changchun, Jilin, China

  • Wei Kong,

    Affiliations School of Life Sciences, Jilin University, Changchun, Jilin, China, National Engineering Laboratory for Acquired Immune Deficiency Syndrome Vaccine, Jilin University, Changchun, Jilin, China, Key Laboratory for Molecular Enzymology & Engineering, Jilin University, Changchun, Jilin, China

  • Xianggui Kong,

    Affiliation State Key Laboratory of Luminescence and Applications, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin, China

  • Zhenxin Wang,

    Affiliation State Key Laboratory of Electroanalytical Chemistry, Changchun Institute of Applied Chemistry, Chinese Academy of Sciences, Changchun, Jilin, China

  • Youchun Wang,

    Affiliation Division of Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome and Sex-transmitted Virus Vaccines, National Institutes for Food and Drug Control, Beijing, China

  • Chunlai Jiang ,

    feng0215@gmail.com (FG); jiangcl@jlu.edu.cn (CJ)

    Affiliations School of Life Sciences, Jilin University, Changchun, Jilin, China, National Engineering Laboratory for Acquired Immune Deficiency Syndrome Vaccine, Jilin University, Changchun, Jilin, China, Key Laboratory for Molecular Enzymology & Engineering, Jilin University, Changchun, Jilin, China

  • Feng Gao

    feng0215@gmail.com (FG); jiangcl@jlu.edu.cn (CJ)

    Affiliations School of Life Sciences, Jilin University, Changchun, Jilin, China, National Engineering Laboratory for Acquired Immune Deficiency Syndrome Vaccine, Jilin University, Changchun, Jilin, China

Enhanced Sensitivity for Detection of HIV-1 p24 Antigen by a Novel Nuclease-Linked Fluorescence Oligonucleotide Assay

  • Peihu Fan, 
  • Xiaojun Li, 
  • Weiheng Su, 
  • Wei Kong, 
  • Xianggui Kong, 
  • Zhenxin Wang, 
  • Youchun Wang, 
  • Chunlai Jiang, 
  • Feng Gao
PLOS
x

Abstract

The relatively high detection limit of the Enzyme-linked immunosorbent assay (ELISA) prevents its application for detection of low concentrations of antigens. To increase the sensitivity for detection of HIV-1 p24 antigen, we developed a highly sensitive nuclease-linked fluorescence oligonucleotide assay (NLFOA). Two major improvements were incorporated in NLFOA to amplify antibody-antigen interaction signals and reduce the signal/noise ratio; a large number of nuclease molecules coupled to the gold nanoparticle/streptavidin complex and fluorescent signals generated from fluorescent-labeled oligonucleotides by the nuclease. The detection limit of p24 by NLFOA was 1 pg/mL, which was 10-fold more sensitive than the conventional ELISA (10 pg/mL). The specificity was 100% and the coefficient of variation (CV) was 7.8% at low p24 concentration (1.5 pg/mL) with various concentrations of spiked p24 in HIV-1 negative sera. Thus, NLFOA is highly sensitive, specific, reproducible and user-friendly. The more sensitive detection of low p24 concentrations in HIV-1-infected individuals by NLFOA could allow detection of HIV-1 infections that are missed by the conventional ELISA at the window period during acute infection to further reduce the risk for HIV-1 infection due to the undetected HIV-1 in the blood products. Moreover, NLFOA can be easily applied to more sensitive detection of other antigens.

Introduction

Enzyme-linked immunosorbent assay (ELISA) is a widely used method for detection of proteins such as hormones, immunoglobulins and infectious agent antigens because of its sensitivity and easiness to use [13]. A typical sandwich ELISA method has been widely used for antigen detection to date. Fluorescent substrate (4-methylumbelliferyl phosphate, 4-MUP) was thought to be more sensitive than colorigenic substrate and has been explored to increase the sensitivity of ELISA [4]. Although a 2 or 100-fold increase in sensitivity was reported [4,5], the improvement was not consistently reproduced in other studies [68]. Recently, the unique properties of nanoparticles (large ratio of surface area-to-volume, stability and high binding capability) were explored to improve ELISA sensitivity [912]. Because of their high sensitivity and specificity, many nanoparticles have been generated for diagnosis of diseases [13,14]. In all previous ELISA systems, the enzymes conjugated to the detection antibodies are horseradish peroxidase (HRP), alkaline phosphatase (ALP), β-D-galactosidase, glucose oxidase or malate dehydrogenase [1518]. They share some common features: stable, highly active, and capable of forming cross-links with other proteins (antibody, streptavidin and others). To date, no other enzymes have been used to further improve the ELISA sensitivity.

The nucleocapsid protein p24 of HIV-1 Gag has been widely used as a surrogate marker for HIV-1 infection [1922]. A large number of viral particles are produced after infection during acute infection stage [2325]. Thus, the detection of p24 and HIV-1 specific antibodies has been used for diagnosis of HIV-1 infection. This has prevented the vast majority of new HIV-1 infections due to the transfusion of HIV-1 positive blood [19,26]. Generally, the p24 concentration is too low between infection and the first time when it is detectable (window period) by the conventional ELISA [27,28]. Thus, individuals who are infected with HIV-1 during this window cannot be diagnosed by ELISA. The third and fourth-generation ELISA, which use denatured plasma samples, have improved the detection of early HIV-1 infection [2931]. However, this window period was not significantly shortened while the complexity of testing was increased [32]. The ultrasensitive immunoassay (immune complex transfer enzyme immunoassay) was more sensitive for detection of p24 (> 630-fold), but this procedure was tedious and the serious serum interference limited its application [33]. Recently, nanoparticle based assays were reported to be able to significantly lower the detection limit of p24 than the conventional ELISA [3436]. However, the complex procedures, high cost, the need for rare metal and the inherently inaccurate quantification of target molecules limited their wide use. Therefore, the sophisticated and expensive reverse transcription-polymerase chain reaction (RT-PCR) method was still the only assay capable of efficiently detecting early HIV-1 infection at the window period [26].

To develop a sensitive method for detection of low concentrations of the HIV-1 p24 antigen, we established a novel nuclease-linked fluorescence oligonucleotide assay (NLFOA) that coupled the advantages of nanoparticle technology, the nuclease and the fluorescent labeled oligonucleotides (FLOS) as the substrate. The new NLFOA method is 10-fold more sensitive than the conventional ELISA. NLFOA is also highly specific, reproducible and user-friendly. Since NLFOA can be easily applied to detection of any antigens, it can be used as a powerful tool to detect low concentration antigens, shorten the window period of infectious diseases, and diagnose other diseases.

Materials And Methods

Design of fluorescent labeled oligonucleotide substrate (FLOS)

The oligo BAM (5’-Iowa Black FQ/GCGGATCCCGCCTCTTGAGGCGGGATCCGC/Joe -3’) and the oligo ECO (5’-Iowa Black FQ/GCGAATTCCGCCTCTTGAGGCGGAATTCGC/Joe -3’) were synthesized by Integrated DNA Technology (Coralville, IA, U.S.A). The sites recognized by the restriction enzymes BamH I and EcoR I were indicated by underline. Both oligos were free of secondary structures and not homologous to sequences in the GenBank as determined by BLAST searching. Since they were labeled with a quencher Iowa Black FQ at the 5’ end and a fluorophore Joe at the 3’ end, no fluorescence could be detected from the oligo itself. The oligo could form a perfect dimer after the oligos were heated at 94°C for 5 minutes and annealed at 25°C for 2 minutes. After the oligodimer was digested by restriction enzymes BamH I or EcoR I, or after single- or double-stranded oligos were digested by exonuclease, the fluorophore Joe was released for detection.

Determination of enzyme relative activity

To identify a nuclease with the high relative activity, we tested nine nucleases: TurboNuclease (Accelagen, San Diego, CA, U.S.A.); Mung Bean Nuclease, S1 Nuclease, BAL 31 Nuclease, Exonuclease III, Exonuclease I as well as BamH I and EcoR I restrictive endonucleases (Takara, Dalian, Liaoning, China); and DNase I (Thermo Fisher Scientific, Waltham, MA, U.S.A.). The purity and concentration of each enzyme was determined on a sodium dodecylsulfate polyacrylamide gel (SDS-PAGE). Each commercial nuclease was mixed with the equal volume of 2× sample buffer (containing 0.5 M Tris-HCl pH 6.8, 25%; 0.27 M 2-Mercaptoethanol; SDS, 0.14 M; bromophenol blue, 0.003 M; glycerol, 20%; H2O, 50%), and the mixtures were then boiled for 2 minutes. Each sample (10 μL) was loaded on the gel and electrophoresed for 1.5 hours at 120 V on a Mini Protean Tetra Cell (BIO-RAD, Hercules, CA, U.S.A.). The gel was stained with Coomassie blue and scanned on an odyssey CLx infrared imaging system (LI-COR, Lincoln, NE, U.S.A.) at the 700 nm. The signal intension of each band on SDS-PAGE was quantified by the software of Image Studio Ver. 2.1 (LI-COR, Lincoln, NE, U.S.A.).

For determination of enzyme relative activity, each enzyme was diluted at 10-fold serial dilutions in the enzyme specific buffer then incubated with the FLOS substrate at 37°C for 100 minutes. The fluorescent signals were measured on a Chromo 4 System (Bio-Rad, Hercules, CA, U.S.A.) to determine the enzyme activity kinetics by reading the plate every 5 minutes. The data was analyzed by the software Opticon Monitor 3 (Bio-Rad, Hercules, CA, U.S.A.). The enzyme relative activity was calculated by dividing MFI with absolute weight of the enzyme (MFI/ng) for each enzyme. The experiment was repeated three times.

Nuclease-linked fluorescence oligonucleotide assay (NLFOA)

The NLFOA procedure was similar to that in the classical HIV-1 p24 ELISA (PerkinElmer, Waltham, MA, U.S.A.), except that Streptavidin (SA)-HRP and OPD (o-phenylenediamine dihydrochloride) were replaced with nanoparticle-streptavidin (Innova Biosciences, Cambridge, Cambs, U.K.), biotinylated TurboNuclease (bio-TurboNuclease) and the FLOS substrate (Fig 1). Bio-TurboNuclease was prepared using the ImmunoProbe Biotinylation Kit (Sigma Aldrich, Saint Louis, MO, U.S.A.). The 96-cell polystyrene microplate was coated with the p24 monoclonal antibody (from PerkinElmer HIV-1 p24 ELISA kit) and blocked with blocking buffer (PBS containing 5.0 g/L defatted milk powder). After p24 was added and the plate was incubated at 37°C for two hours, the biotinylated anti-p24 antibody (from PerkinElmer HIV-1 p24 ELISA kit) was added and incubated at 37°C for one hour. Nanoparticle-streptavidin was then added and incubated at 37°C for 40 minutes. After bio-TurboNuclease was added and incubated at 37°C for 40 minutes, FLOS in TurboNuclease reaction buffer (50 mM Tris-HCl, pH 8.0 and 1 mM MgCl2) was added and incubated at 37°C for 40 minutes. The fluorescence intensity was measured on a VICTOR X2 Series Multilabel Plate Reader (PerkinElmer, Waltham, MA, U.S.A.). A washing step with PBS containing 0.2% Tween 20 was performed between each step.

thumbnail
Fig 1. Schematic presentation of the nuclease-linked fluorescence oligonucleotide assay (NLFOA).

(A) In the conventional ELISA, the capture antibody-antigen complex binds to one primary antibody and subsequently to one enzyme-conjugated second antibody. The enzyme acts on the substrate to generate colored substrate products. The reaction is detected by measuring absorbance at a certain wavelength of the chemical reaction. (B) In NLFOA, the capture antibody-antigen complex binds to one biotinylated primary antibody, then to the nanoparticle-streptavidin complex containing multiple copies of streptavidins, and finally to multiple copies of biotinylated TurboNuclease. The fluorescent labeled oliogonucleotide substrate (FLOS), which does not emit fluorescence due to the Iowa Black FQ (quencher) at the other end of the oligo, can generate fluorescence after digestion by TurboNuclease.

https://doi.org/10.1371/journal.pone.0125701.g001

Determination of the sensitivity, specificity and reproducibility of NLFOA

The10-fold serial dilutions of p24 in the dilution buffer (PBS containing 1 g/L bovine serum albumin) and spiked in a serum from an HIV-1 negative individual were detected by NLFOA and the conventional ELISA (HIV-1 p24 ELISA). One hundred sera from HIV-1 negative individuals were included as negative controls. The written consent was obtained from all subjects who participated in the study, and the study was approved by the ethics committee of Jilin University. HIV-1 p24 was diluted at low (1 pg/mL), intermediate (10 pg/mL) and high (100 pg/mL) concentrations for determination of the reproducibility and the coefficient of variation (CV). To compare the sensitivity and reproducibility between NLFOA and ELISA, the intra-assay variation was determined with p24 spiked samples (N = 5) analyzed on the same day, and the inter-assay variation was determined with the same samples analyzed on different days. All comparisons were carried out in five independent repeats.

To demonstrate the applicability of NLFOA for detection of other antigens, the capsid protein VP1 of enterovirus 71 (EV71) was analyzed. The procedure for detection of VP1 was similar to that used for p24 detection, with the following changes: mAb 2H2 targeting VP1 was used to coat 96-cell polystyrene microplate, recombinant VP1 expressed in E. coli was spiked in sera, and the biotinylated detection antibody was prepared from VP1-immunized rabbits.

Comparison of the signal/noise (S/N) ratio between ELISA and NLFOA

As the indication of the assay precision (the variation coefficient in signal intensity value), S/N ratios of ELISA and NLFOA were determined with results from three independent replicates at various p24 concentrations spiked in sera. The S/N ratio was calculated using the total standard deviation (here denoted σtotal, and the Eq is that in which the σs is the SD of the sample and σc is the SD of the negative control), the average signal of the sample (μs) and the average signal of the negative control (μc). S/N values were determined according to the Eq S/N = (μs - μc)/σtotal. [4]

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics 21 (IBM, Armonk, NY, U.S.A.) and Microsoft Excel software (Microsoft Corp., Redmond, WA, U.S.A.). Differences in mean values were analyzed by student’s t test. P values less than 0.05 were considered significant.

Results

Identification of nuclease with the highest relative activity

To identify a nuclease with high relative activity as the coupling enzyme, we tested a total of nine enzymes: seven nucleases (TurboNuclease, Mung Bean, S1, BAL 31, Exonuclease III, Exonuclease I and DNase I) and two restrictive endonuclease (EcoR І and BamH I). Each enzyme was diluted at 1:10 serial dilutions and mixed with the fluorescent labeled oligonucleotide substrate (FLOS) (10–6 M) to release the fluorophore that was quenched by Iowa Black FQ. The kinetic of the mean fluorescence intensity (MFI) was measured every five minutes. The positive fluorescent signal could be detected for both TurboNclease and Exonuclease III at the 1:10,000 dilution, BAL 31 at the 1:1,000 dilution, DNase I and S1 at the 1:100 dilution, and BamH I, EcoR I and Mung Bean at the 1:10 dilution (Fig 2A). Only very weak signal was detected for Exonuclease I when it was not diluted. These results showed that TurboNuclease and exonuclease III had the highest activity.

thumbnail
Fig 2. Determination of relative activity of different nucleases.

(A) The nuclease activity was determined by mixing FLOS with serially diluted enzymes (1:10). The mean fluorescence intensity (MFI) was measured every 5 minutes. (B) Analysis of the enzyme contents in the commercial enzyme stocks by SDS-PAGE. Equal volume (10 μl) of each commercial nuclease was loaded onto a SDS-PAGE gel for electrophoresis. Exonuclease III and TurboNuclese were diluted at 1:100 and 1:10, respectively, due to the high protein concentration in the samples. (C) Determination of enzyme relative activity. The relative activity of each enzyme was defined by the positive MFI (2.1 folds above the background) at the lowest concentration of the enzyme (MFI/ng). All experiments were independently performed three times and results from one representative experiment are shown. Negative control (NC) contained all components except the enzyme. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g002

To determine the relative activity of each enzyme, we first determined the proportion of the enzyme protein in the stock by SDS-PAGE. One predominant enzyme protein was detected for TurboNuclease, Bal 31, Exonuclease III, DNase I, Mung Bean and Exonuclease I, while other proteins beside the enzymes were also present for S1 nuclease, EcoR I and BamH I (Fig 2B). When only the enzyme proteins were compared, similar concentrations (40–80 μg/mL) were observed for Bal 31, DNase I, S1 nuclease, EcoR I, BamH I, Mung Bean and Exonuclease I. The concentrations of TurboNuclease and Exonuclease III were about 10- and 100-fold higher than those of other enzymes, respectively. We then determined the relative activity for each enzyme by dividing MFI with absolute weight of the enzyme (MFI/ng) in the stock. The relative activity was the highest for TurboNuclease (48 MFI/ng) and the second highest for BAL 31 (26 MFI/ng) (Fig 2C). All other enzymes had much lower relative activity. Two restrictive endonucleases EcoR I and BamH I were able to recognize the specific nucleotides in the double stranded FLOS substrates. However, the signals generated by either enzyme were too weak to be applicable. These results demonstrated that TurboNuclease had the highest relative activity and was used for all subsequent experiments.

Determination of the minimal concentration of TurboNuclease required for NLFOA

We next determined whether the enzyme concentration required for generation of detectable signals was the lowest in the TurboNuclease-FLOS system by comparing it to three other commonly used enzyme-substrate system (HRP-OPD, ALP-pNPP and ALP-4-MUP). In each system, the enzyme was diluted at 1:10 serial dilutions, starting at 10–7 M and reacted with the optimal concentration of corresponding substrates. A positive signal was detected at 10–11 M for TurboNuclease (Fig 3A), while it was at 10–10 M for the enzyme-colorigenic substrate (pNPP or OPD) system (Fig 3B and 3C) and ALP-4MUP fluorogenic substrate system (Fig 3D). Thus, 10-fold less of TurboNuclease than other enzymes was needed to generate detectable signals. Some studies showed that fluorescent substrate 4-MUP was able to improve the ELISA sensitivity [5,7,37], but no improvement was reported in other studies [68]. Our results also did not show any improvement in sensitivity with fluorescent substrate 4-MUP (Fig 3D). These data demonstrated that the novel NLFOA method that used both the TurboNuclease and FLOS could be developed as a highly sensitive assay.

thumbnail
Fig 3. Comparison of minimal required enzyme concentrations among different detection systems.

The lowest concentration of the enzyme required for detection was determined for four different enzyme-substrate systems: (A) TurboNuclease and FLOS system; (B) alkaline phosphatase (ALP) and p-nitrophenylphosphate (pNPP) system; (C) horseradish peroxidase (HRP) and o-Phenylenediamine (OPD) system; and (D) ALP and 4-methylumbelliferyl phosphate (4-MUP, a classical fluorogenic substrate) system. Each enzyme was prepared at 1:10 serial dilutions (starting at 10–7 M) and reacted with its corresponding substrate. Negative control (NC) contained all components except the enzyme. The cutoff (2.1 fold higher than the NC signal) is indicated by the dotted line. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g003

Since the Turbonuclease needed to be biotinylated in NLFOA, we next determined whether biotinylation could affect the enzyme activity. When compared to the unmodified TurboNuclease, bio-TurboNuclease had similar activity (p = 0.344, Student’s t test), suggesting that biotinylation of TurboNuclease had no or minimal impact on enzyme activity (Fig 4).

thumbnail
Fig 4. Similar enzyme activity between TurboNuclease and bio-TurboNuclease.

The mean fluorescence intensity (MFI) was compared between TurboNuclease and bio-TurboNuclease at 1:10 serial dilutions (p>0.2, Student’s t test). Each assay was performed in five independent experiments. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g004

Optimization of NLFOA

To optimize the conditions for NLFOA, we tested various concentrations of nanoparticle-streptavidin, bio-TurboNuclease and FLOS as well as the diameters of nanoparticles. When one component was optimized for NLFOA, all other components were kept at excessive concentrations. MFI for nanoparticle-streptavidin was similar at 10–8 M and 10–9 M, reduced at 10–10 M, but significantly decreased at lower concentrations (10-11-10-13 M) (Fig 5A). MFI was similar at concentrations 10–8, 10–9, and 10–10 M for bio-TurboNuclease, but significantly decreased at lower concentrations (10-11-10-13 M) (Fig 5B). To determine how the FLOS concentrations affected assay system, we made 1:2 serial dilutions from the 4 × 10–6 M stock. The MFI was the highest at the 400 × 10–8 M of FLOS. It was continuously reduced as the FLOS concentrations decreased. The signal at 100 × 10–8 M was less than two-fold lower than that for the highest stock (400 × 10–8 M), but it was still significantly higher than the background (Fig 5C). However, the FLOS concentration at 100 × 10–8 M was more cost effective. Finally, we determined the influence of the size of the nanoparticle spheres on the assay signals. Among four different sizes tested, the nanoparticles with diameter of 40 nm yielded the strongest signal (Fig 5D). Taken together, these results showed that nanoparticles with a diameter of 40 nm, nanoparticle-streptavidin at 10–9 M, bio-TurboNuclease at 10–10 M, and FLOS at 10–6 M were optimal for NLFOA. These conditions were used for all the rest of the experiments.

thumbnail
Fig 5. Optimization of the components in NLFOA.

The NLFOA conditions were optimized for (A) nanoparticle-streptavidin, (B) bio-TurboNuclease, (C) FLOS and (D) diameters of the nanoparticles. Both nanoparticle-streptavidin and bio-TurboNuclease were prepared at 1:10 serial dilutions between 10–8 -10-13 M. The FLOS oligos were prepared at 1:2 serial dilutions between 400–3.1 ×10–8 M. Four different diameter sizes (13, 30, 40 and 50 nm) were analyzed for nanoparticles. Negative control (NC) contained all components except what was to be optimized. Each of the component was analyzed in five repeats and were performed in two independent experiments. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g005

The enhanced signal by nanoparticle-streptavidin and FLOS

To investigate whether the nanoparticle and FLOS could generate a more enhanced signal in NLFOA than other enzyme-substrate systems, we replaced streptavidin with nanoparticle-streptavidin in NLFOA and three other ELISA systems by testing 1:10 serial antibody dilutions. The nanoparticle-streptavidin increased the sensitivity by 10-fold, compared to the streptavidin alone (Fig 6A). Moreover, the FLOS substrate detected the antibody dilution at 1:10–5, while the OPD and pNPP colorigenic substrates (Fig 6B and 6C) and the 4-MUP fluorogenic substrate (Fig 6D) only detected the antibody dilution at 1:10–4. These results showed that the novel NLFOA method, which incorporated the FLOS substrate, nuclease and nanoparticle, was more sensitive than three conventional ELISA systems.

thumbnail
Fig 6. Increased sensitivity with nanoparticle-streptavidin and FLOS.

The signals generated by the nanoparticle-streptavidin and streptavidin systems were compared among four different enzyme-substrate systems: (A) TurboNuclease and FLOS; (B) alkaline phosphatase (ALP) and p-nitrophenylphosphate (pNPP); (C) horseradish peroxidase (HRP) and o-Phenylenediamine (OPD); and (D) alkaline phosphatase (ALP) and 4-methylumbelliferyl phosphate (4-MUP). Each enzyme was prepared at 1:10 serial dilutions (10–4–10–9 M). Negative control (NC) contained all components except the enzyme. The cutoff (2.1 fold higher than the NC signal) is indicated by the dotted line. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g006

Determination of the detection limit of NLFOA

To investigate whether NLFOA could be used as a more sensitive method for detection of p24, we next determined its detection limit in samples. First, p24 was prepared at 1:2 serial dilutions in PBS and analyzed in parallel by NLFOA and the commonly-used p24 detection ELISA kit (PerkinElmer, Waltham, MA, U.S.A.). NLFOA detected p24 at a concentration of 1.5 pg/mL (Fig 7A), while ELISA could only detect p24 at a concentration of 12.5 pg/mL (Fig 7B). Thus, NLFOA was about 10-fold more sensitive than ELISA. To investigate whether NLFOA could detect p24 in human sera at the similar sensitivity level, we spiked the p24 in a serum from an HIV-1 negative individual at concentrations from 0.25–25 pg/mL. Since both NLFOA and ELISA could detect p24 at concentrations >25 pg/ml, higher p24 concentrations were not analyzed. The concentration of p24 as low as 1 pg/mL could be consistently detected by NLFOA (Fig 7C), while only the p24 concentrations at 11 pg/mL or higher could be consistently detected by ELISA (Fig 7D). Further analysis of intra- and inter-assay variation showed that the detection limit of NLFOA (0.9 pg/mL) was significantly lower than that of ELISA (10.8 pg/mL) (p<0.0001; Table 1). These results demonstrated that NLFOA was about 10-fold more sensitive than ELISA for detection of HIV-1 p24 present in blood samples.

thumbnail
Fig 7. Comparison of the detection limit between NLFOA and the conventional ELISA.

The detection limit for HIV-1 p24 antigen diluted in PBS by NLFOA (A) and conventional ELISA (B). The detection limit of HIV-1 p24 antigen spiked in sera from a healthy human by NLFOA (C) and conventional ELISA (D). The dotted line indicates the cutoff value that is 2.1-fold of that of the negative control (NC). Experiments with p24 in PBS (A and B) were done in five repeats and experiments with p24 in sera (C and D) were done in eight repeats. Each assay was performed in three independent experiments. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g007

thumbnail
Table 1. Comparison of sensitivity and reproducibility between NLFOA and ELISA.

https://doi.org/10.1371/journal.pone.0125701.t001

To investigate if NLFOA could be adapted to detection of other antigens, we used NLFOA to detect the capsid protein VP1 of enterovirus 71 (EV71). The recombinant VP1 was prepared at 1:2 serial dilutions in sera and then analyzed in parallel by NLFOA and conventional ELISA. VP1 at 15.6 ng/mL could be consistently detected by NLFOA, while VP1 at 7.8 ng/mL was occasionally detected (Fig 8A). In contrast, VP1 at 125 ng/mL was consistently detected by the conventional ELISA method, while VP1 at the next dilution (62.5 ng/mL) could not be detected (Fig 8B). These results demonstrated that NLFOA was also ~10-fold more sensitive for detection of the VP1 antigen than ELISA.

thumbnail
Fig 8. Detection of the EV71 VP1 antigen by NLFOA.

The EV71 VP1 antigen was spiked in sera and anzlysed in parallel by NLFOA (A) and the conventional ELISA (B). The dotted line indicates the cutoff value that is 2.1-fold of that of the negative control (NC). Each assay was performed in three independent experiments. Means ± SD are shown.

https://doi.org/10.1371/journal.pone.0125701.g008

Specificity and reproducibility of NLFOA

We next determined the specificity of NLFOA using two panels of samples: 100 sera from HIV-1 negative individuals and 170 serum samples spiked with p24 at 0.5–60 pg/mL (Table 2). None of the 100 HIV-1 negative samples were positive by NLFOA or ELISA. NLFOA could detect p24 in all spiked serum samples at ≥ 1 pg/mL and 50% of the samples at 0.5 pg/mL. In contrast, ELISA could only detect p24 in the spiked serum samples at ≥ 15 pg/mL and 92% of the samples at 10 pg/mL, but could not detect p24 in samples ≤ 6 pg/mL. These results demonstrated that NLFOA was as specific as the conventional ELISA, but more sensitive.

The reproducibility of NLFOA was evaluated with three panels of the serum samples spiked with p24: low concentration (1 pg/mL), intermediate concentration (10 pg/mL), and high concentration (100 pg/mL). Twenty repeats were used for the low concentration samples, while 15 repeats were used for intermediate and high concentration samples. The coefficient of variation (CV) was determined to evaluate the assay reproducibility for each panel. The CVs were 7.8%, 9.1% and 8.4% for the low, intermediate and high concentration panels, respectively; all were lower than the accepted level (10%). To compare the reproducibility between NLFOA and ELISA, low p24 concentration samples were analyzed by both methods. The CV of NLFOA was significantly lower than that of ELISA: 7.3 vs. 8.6 for intra-assay (p = 0.03) and 7.1 vs. 8.7 for inter-assay (p = 0.01) (Table 1). Overall, the reproducibility of NLFOA was significantly better than that of ELISA (p<0.001). These results showed that NLFOA was a more reproducible than ELISA.

The signal/noise (S/N) ratio of the NLFOA

The S/N ratio of NLFOA was determined with various p24 concentrations and compared to that of ELISA. The S/N ratio values of NLFOA had a steeper upward trend than those of ELISA as the analyte concentration increased. The NLFOA S/N ratios were significantly (p = 0.014) higher (2.4 to 6.2-fold) than those of ELISA at concentrations at which both assays yielded the positive results (Fig 9).

thumbnail
Fig 9. Enhanced precision with NLFOA.

The signal/noise (S/N) ratios were determined for both NLFOA and ELISA assays at different p24 concentrations.

https://doi.org/10.1371/journal.pone.0125701.g009

Discussion

ELISA is one of the most commonly used methods to detect antigens using the enzymes as surrogates because of its specificity, sensitivity, consistency, easiness of use and low cost. However, the need to detect low concentrations of antigens in samples in clinical diagnosis and laboratory research demands development of more sensitive assays. In the present study, we developed a novel nuclease-linked fluorescence oligonucleotide assay (NLFOA) by taking the advantages of the nanoparticles coupled with multiple streptavidin copies, high affinity and specificity of streptavidin-biotin interaction, biotinylated TurboNuclease and the novel fluorescent labeled oligonucleotide substrate (FLOS). NLFOA was 10-fold more sensitive than the conventional ELISA assay. Moreover, NLFOA is highly specific, reproducible, low cost and user-friendly.

The increased sensitivity of NLFOA could be contributed by two improvements: the signal amplification by nanoparticles coupled with the nuclease and a novel fluorescent labeled oligonucleotides (FLOS) substrate. A nanoparticle could be coupled with many streptavidin molecules that could bind to additional multiple bio-TurboNucleases (Fig 1B). This would result in more substrate becoming catalyzed for amplification of detection signals as previously reported [10,12,38,39]. Previously, fluorescent substrate 4-MUP was found to be able to improve the ELISA sensitivity [4,5], but no improvement was reported in other studies [68]. We also tested the 4-MUP substrate system but did not observe any improvement in assay sensitivity (Figs 3D and 6D), similar to the results reported by others [68]. Instead, we developed a novel fluorescent labeled oligonucleotide substrate (FLOS). The fluorophore Joe at the 3’ end was fully quenched the by the quencher Iowa Black FQ at 5’ end in the same FLOS oligo. The fluorescent could only be released for detection after the FLOS was digested by nuclease. Thus, the fluorescent signal was intense and had a high signal/noise ratio [40,41], which was critical for high sensitivity, specificity and reproducibility of NLFOA. Because S/N ratio is an important factor of the assay precision including sensitivity, reproducibility and specificity, the better S/N ratio of NLFOA suggested that NLFOA could more reliably detected antigens present at very low concentrations than the conventional ELISA. By analyzing a large number of p24 negative and positive samples, we also demonstrated that NLFOA was as specific and reproducible as the previously well-established ELISA assays [42,43].

Since detection of p24 by a highly sensitive ELISA-based method can play a critical role in diagnosis of new HIV-1 infections, many sensitive p24 detection assays have been developed [3336]. However, the complex procedures, high cost, and the inherently inaccurate quantification of target molecules become major hurdles for their wide use. Recently, two highly sensitive nanoparticle-based bio-barcode amplification assays were developed for detection of HIV-1 p24 antigen present as low as 0.1 pg/mL [36]. The new assays combined both the immunoassay and the PCR method together. The p24 concentration was determined by PCR amplifying the oligos attached to the nanoparticles that were captured though anti-p24 antibodies using gel electrophoresis. The assays were based on an ELISA system that could only detected p24 at 1,000 pg/mL. If higher affinity p24 mAbs as in the commercial p24 ELISA kit were used in both assays, the p24 detection limit could be even lower. However, since p24 was tested only in PBS with BSA for both assays in the report, it is not clear how plasma samples will affect the assays’ sensitivity and specificity. In addition, PCR was required in both systems. It is not clear if both assays will be as sensitive as the single PCR assay that detect viral RNA copies in virions since the head-on comparison was not reported. The PCR cost, cumbersome gel electrophoresis and extra steps make both nanoparticle-based bio-barcode amplification assays unlike to be widely used for analyzing large numbers of samples. In contrast, NLFOA increased the sensitivity of ELISA by using nuclease, FLOS oligo and nanoparticles, but without the additional PCR step. Although NLFOA did not increase the sensitivity for p24 detection as high as these two nanoparticle-based bio-barcode amplification assays, the combination of the sensitivity, specificity, reproducibility, low cost and user-friendly procedure makes it easy to be adapted for detection of p24 or any other antigens.

A number of nucleases were tested to establish the NLFOA method. Two restrictive endonucleases EcoR І and BamH І were tested for their ability to recognize the specific nucleotides in the double stranded FLOS oligos. However, signals generated by either enzyme were too low to be useful. Among seven tested nucleases, TurboNuclease yielded the highest relative activity. TurboNuclease is an extracellular endonuclease from Serratia macescens. It nonspecifically hydrolyzes both single- and double-stranded DNA and RNA to small 5’-phosphorylated oligonucleotides (1–4 bases). We for the first time demonstrated that TurboNuclease together with novel fluorescent labeled oligonucleotide substrates could be used improve the sensitivity of the ELISA-based assays.

There were three stages that could amplify the initial antigen-antibody interaction signal (Fig 1). First, the single biotinylated antibody bounded to one nanoparticle could be coupled with many streptavidin molecules (a maximum of ~ 256 streptavidin molecules) [44]. Second, one streptavidin could bind to up to three bio-TurboNucleases. Third, a number of FLOS molecules could be digested by bio-TurboNuclease. After three stages of amplification of the antigen-antibody interaction signal, a much higher sensitivity was expected in NLFOA. However, only about a 10-fold increase in sensitivity in NLFOA compared to the conventional ELISA was observed in this study. This suggested that a better understanding of the limiting steps could lead to further improvement in sensitivity of NLFOA.

The improved sensitivity of NLFOA can have a significant impact on the clinical diagnosis of HIV-1 and other pathogens. For example, the p24 antigen, which is highly cross-reactive among HIV-1 group M subtypes [45], was the first HIV-1 antigen that can be detected [24,26]. However, the window period from infection to the first time when p24 can be detected is about 14 days (Fig 10) [19,24,25,46,47]. If donated blood is collected during this period, the HIV-1 infection cannot be diagnosed by the conventional ELISA even though the blood at this time can be highly contagious. The viral RNA can be detected at this time by the RT-PCR assay, but it can not be routinely employed in resource limited regions due to high costs and sophistication of procedure [48]. Since NLFOA is about 10 fold more sensitive than conventional ELISA, the use of NLFOA to screen donated bloods can likely shorten this window period and reduce the incidence of HIV-1 infection due to the transfusion of HIV-1 positive blood (Fig 10). NLFOA, like ELISA, is more cost effective and user-friendly than RT-PCR when individual samples are analyzed. It costs 20 times less than RT-PCR. When the pooled nucleic acid amplification testing (PNAAT) was used with pooled 10–100 samples, the cost will be reduced by ~10–100 folds, but with a reduced sensitivity for samples with low viral loads. However, positive pools need to be retested with subpools or individual samples. These will require additional time and tests to identify individual HIV-1 positive samples. Recent studies showed that PNAAT may be cost effective only in very high incidence settings, and more frequently testing with third- or fourth-generation immunoassay can be more economically efficient and may obviate the benefits of PNAAT for screening acute HIV-1 infections [4951]. More sensitive and user-friendly immunoassays like NLFOA that can identify more acute HIV-1 infection cases in the eclipse window period may further increase the cost-effectiveness of immunoassays. More importantly, NLFOA can be easily adapted to detection of low levels of any proteins, as menstruated with the detection of EV71 VP1, and serves as a powerful tool for early diagnosis of infections or other diseases.

thumbnail
Fig 10. Shortened window period of HIV-1 infection by NLFOA.

The HIV-1 p24 antigen can only be detected after a period of time after HIV-1 infection (window period) by the conventional ELISA. The higher sensitivity of NLFOA can likely shorten this window period.

https://doi.org/10.1371/journal.pone.0125701.g010

Author Contributions

Conceived and designed the experiments: PF XL WK CJ FG. Performed the experiments: PF XL WS. Analyzed the data: PF XL WS WK XK ZW YW CJ FG. Wrote the paper: PF XL CJ FG.

References

  1. 1. Gan SD, Patel KR. Enzyme immunoassay and enzyme-linked immunosorbent assay. J Invest Dermatol. 2013;133: e12. pmid:23949770
  2. 2. Lequin RM. Enzyme immunoassay (EIA)/enzyme-linked immunosorbent assay (ELISA). Clin Chem. 2005;51: 2415–2418. pmid:16179424
  3. 3. Crowther J. Enzyme Linked Immunosorbent Assay (ELISA). In: Walker JM, Rapley R, editors. Molecular Biomethods Handbook. 2nd ed. New York: Humana Press; 2008. p. 657–682.
  4. 4. Meng Y, High K, Antonello J, Washabaugh MW, Zhao Q. Enhanced sensitivity and precision in an enzyme-linked immunosorbent assay with fluorogenic substrates compared with commonly used chromogenic substrates. Anal Biochem. 2005;345: 227–236. pmid:16137635
  5. 5. Yolken RH, Stopa PJ. Enzyme-linked fluorescence assay: Ultrasensitive solid-phase assay for detection of human rotavirus. J Clin Microbiol. 1979;10: 317–321. pmid:226564
  6. 6. Shekarchi IC, Sever JL, Nerurkar L, Fuccillo D. Comparison of enzyme-linked immunosorbent assay with enzyme-linked fluorescence assay with automated readers for detection of rubella virus antibody and herpes simplex virus. J Clin Microbiol. 1985;21: 92–96. pmid:2981902
  7. 7. Yolken RH, Leister FJ. Comparison of fluorescent and colorigenic substrates for enzyme immunoassays. J Clin Microbiol. 1982;15: 757–760. pmid:7047551
  8. 8. Crowther JR, Angarita L, Anderson J. Evaluation of the use of chromogenic and fluorogenic substrates in solid-phase enzyme linked immunosorbent assays (ELISA). Biologicals. 1990;18: 331–336. pmid:2178352
  9. 9. Zhou F, Yuan L, Wang H, Li D, Chen H. Gold Nanoparticle Layer: A Promising Platform for Ultra-Sensitive Cancer Detection. Langmuir. 2011;27: 2155–2158. pmid:21319767
  10. 10. Ambrosi A, Airo F, Merkoci A. Enhanced gold nanoparticle based ELISA for a breast cancer biomarker. Anal Chem. 2010;82: 1151–1156. pmid:20043655
  11. 11. Ambrosi A, Castaneda MT, Killard AJ, Smyth MR, Alegret S, Merkoçi A. Double-codified gold nanolabels for enhanced immunoanalysis. Anal Chem. 2007;79: 5232–5240. pmid:17579481
  12. 12. Liu M, Jia C, Huang Y, Lou X, Yao S, Jin Q, et al. Highly sensitive protein detection using enzyme-labeled gold nanoparticle probes. Analyst. 2010;135: 327–331. pmid:20098766
  13. 13. Nam JM, Thaxton CS, Mirkin CA. Nanoparticle-Based Bio-Bar Codes for the Ultrasensitive Detection of Proteins. Science. 2003;301: 1884–1886. pmid:14512622
  14. 14. Agasti SS, Rana S, Park MH, Kim CK, You CC, Rotello VM. Nanoparticles for detection and diagnosis. Adv Drug Deliv Rev. 2010;62: 316–328. pmid:19913581
  15. 15. Carthew P, Gannon J, Whisson I. Comparison of alkaline phosphatase and horseradish peroxidase conjugated antisera in the ELISA test for antibodies to reovirus 3, mouse hepatitis and Sendai viruses. Lab Anim. 1981;15: 69–73. pmid:6267371
  16. 16. Engvall E, Perlmann P. Enzyme-linked immunosorbent assay (ELISA). Quantitative assay of immunoglobulin G. Immunochemistry. 1971;8: 871–874. pmid:5135623
  17. 17. Hashida S, Hashinaka K, Nishikata I, Saito A, Takamizawa A, Shinagawa H, et al. Ultrasensitive and more specific enzyme immunoassay (immune complex transfer enzyme immunoassay) for p24 antigen of HIV-1 in serum using affinity-purified rabbit anti-p24 Fab' and monoclonal mouse anti-p24 Fab'. J Clin Lab Anal. 1996;10: 302–307. pmid:8887010
  18. 18. Scharpe S, Cooreman W, Blomme W, Laekeman G. Quantitative enzyme immunoassay: current status. Clin Chem. 1976;22: 733–738. pmid:776449
  19. 19. Fiebig EW, Wright DJ, Rawal BD, Garrett PE, Schumacher RT, Peddada L, et al. Dynamics of HIV viremia and antibody seroconversion in plasma donors: implications for diagnosis and staging of primary HIV infection. AIDS. 2003;17: 1871–1879. pmid:12960819
  20. 20. Allain JP, Paul D, Laurian Y, Senn D; Group MotA-HFS. Serological markers inearly stages of human immunodeficiency virus infection in haemophiliacs. The Lancet. 1986;328: 1233–1236.
  21. 21. Cao YZ, Valentine F, Hojvat S, Allain JP, Rubinstein P, Mirabile M, et al. Detection of HIV antigen and specific antibodies to HIV core and envelope proteins in sera of patients with HIV infection. Blood. 1987;70: 575–578. pmid:2886162
  22. 22. Dowbenko DJ, Bell JR, Benton CV, Groopman JE, Nguyen H, Vetterlein D, et al. Bacterial expression of the acquired immunodeficiency syndrome retrovirus p24 gag protein and its use as a diagnostic reagent. PNAS. 1985;82: 7748–7752. pmid:3877932
  23. 23. Clark SJ, Saag MS, Decker WD, Campbell-Hill S, Roberson JL, Veldkamp PJ, et al. High Titers of Cytopathic Virus in Plasma of Patients with Symptomatic Primary HIV-1 Infection. N Engl J Med. 1991;324: 954–960. pmid:1900576
  24. 24. Cohen MS, Shaw GM, McMichael AJ, Haynes BF. Acute HIV-1 Infection. N Engl J Med. 2011;364: 1943–1954. pmid:21591946
  25. 25. Daar ES, Moudgil T, Meyer RD, Ho DD. Transient High Levels of Viremia in Patients with Primary Human Immunodeficiency Virus Type 1 Infection. N Engl J Med. 1991;324: 961–964. pmid:1823118
  26. 26. Cohen MS, Gay CL, Busch MP, Hecht FM. The detection of acute HIV infection. J Infect Dis. 2010;202 Suppl 2: S270–277. pmid:20846033
  27. 27. Laperche S, Leballais L, Ly TD, Plantier JC. Failures in the Detection of HIV p24 Antigen With the Determine HIV-1/2 Ag/Ab Combo Rapid Test. J Infect Dis. 2012;206: 1946–1947. pmid:23045630
  28. 28. Fox J, Dunn H, O'Shea S. Low rates of p24 antigen detection using a fourth-generation point of care HIV test. Sex Transm Infect. 2011;87: 178–179. pmid:21084439
  29. 29. Zaaijer HL, v Exel-Oehlers P, Kraaijeveld T, Altena E, Lelie PN. Early detection of antibodies to HIV-1 by third-generation assays. Lancet. 1992;340: 770–772. pmid:1356186
  30. 30. Nadal D, Boni J, Kind C, Varnier OE, Steiner F, Tomasik Z, et al. Prospective evaluation of amplification-boosted ELISA for heat-denatured p24 antigen for diagnosis and monitoring of pediatric human immunodeficiency virus type 1 infection. J Infect Dis. 1999;180: 1089–1095. pmid:10479135
  31. 31. Schupbach J, Boni J, Bisset LR, Tomasik Z, Fischer M, Gunthard HF, et al. HIV-1 p24 antigen is a significant inverse correlate of CD4 T-cell change in patients with suppressed viremia under long-term antiretroviral therapy. J Acquir Immune Defic Syndr. 2003;33: 292–299. pmid:12843739
  32. 32. Sutthent R, Gaudart N, Chokpaibulkit K, Tanliang N, Kanoksinsombath C, Chaisilwatana P. p24 Antigen Detection Assay Modified with a Booster Step for Diagnosis and Monitoring of Human Immunodeficiency Virus Type 1 Infection. J Clin Microbiol. 2003;41: 1016–1022. pmid:12624024
  33. 33. Hashida S, Hashinaka K, Nishikata I, Oka S, Shimada K, Saitoh A, et al. Measurement of human immunodeficiency virus type 1 p24 in serum by an ultrasensitive enzyme immunoassay, the two-site immune complex transfer enzyme immunoassay. J Clin Microbiol. 1995;33: 298–303. pmid:7714182
  34. 34. de la Rica R, Stevens MM. Plasmonic ELISA for the ultrasensitive detection of disease biomarkers with the naked eye. Nat Nanotechnol. 2012;7: 821–824. pmid:23103935
  35. 35. Tang S, Hewlett I. Nanoparticle-based immunoassays for sensitive and early detection of HIV-1 capsid (p24) antigen. J Infect Dis. 2010;201 Suppl 1: S59–64. pmid:20225948
  36. 36. Dong H, Liu J, Zhu H, Ou CY, Xing W, Qiu M, et al. Two types of nanoparticle-based bio-barcode amplification assays to detect HIV-1 p24 antigen. Virol J. 2012;9: 180. pmid:22935171
  37. 37. Chargelegue D, O'Toole CM. Development of a sensitive ELISA for HIV-1 p24 antigen using a fluorogenic substrate for monitoring HIV-1 replication in vitro. J Virol Methods. 1992;38: 323–332. pmid:1430056
  38. 38. Li YY, Zhang C, Li BS, Zhao LF, Li XB, Yang WJ, et al. Ultrasensitive densitometry detection of cytokines with nanoparticle-modified aptamers. Clin Chem. 2007;53: 1061–1066. pmid:17446332
  39. 39. Ke R, Yang W, Xia X, Xu Y, Li Q. Tandem conjugation of enzyme and antibody on silica nanoparticle for enzyme immunoassay. Anal Biochem. 2010;406: 8–13. pmid:20599640
  40. 40. Ishikawa E, Hashida S, Kohno T, Hirota K. Ultrasensitive enzyme immunoassay. Clinica Chimica Acta. 1990;194: 51–72. pmid:2073754
  41. 41. Hashida S, Hashinaka K, Ishikawa E. Ultrasensitive enzyme immunoassay. In: El-Gewely MR, editor. Biotechnology Annual Review. Amsterdam: Elsevier; 1995. p. 403–451. pmid:9704094
  42. 42. Zhou F, Wang M, Yuan L, Cheng Z, Wu Z, Chen H. Sensitive sandwich ELISA based on a gold nanoparticle layer for cancer detection. Analyst. 2012;137: 1779–1784. pmid:22278156
  43. 43. Fransen K, Mertens G, Stynen D, Goris A, Nys P, Nkengasong J, et al. Evaluation of a newly developed HIV antigen test. J Med Virol. 1997;53: 31–35. pmid:9298729
  44. 44. Chirra HD, Sexton T, Biswal D, Hersh LB, Hilt JZ. Catalase-coupled gold nanoparticles: comparison between the carbodiimide and biotin-streptavidin methods. Acta Biomater. 2011;7: 2865–2872. pmid:21232642
  45. 45. Tang S, Zhao J, Wang A, Viswanath R, Harma H, Little RF, et al. Characterization of immune responses to capsid protein p24 of human immunodeficiency virus type 1 and implications for detection. Clin Vaccine Immunol. 2010;17: 1244–1251. pmid:20534793
  46. 46. Lindback S, Thorstensson R, Karlsson AC, von Sydow M, Flamholc L, Blaxhult A, et al. Diagnosis of primary HIV-1 infection and duration of follow-up after HIV exposure. Karolinska Institute Primary HIV Infection Study Group. AIDS. 2000;14: 2333–2339. pmid:11089621
  47. 47. Horsburgh CR Jr, Jason J, Longini IM Jr, Mayer KH, Schochetman G, Schable C, et al. Duration of human immunodeficiency virus infection before detection of antibody. Lancet. 1989;2: 637–640. pmid:2570898
  48. 48. Cohen GM. Access to diagnostics in support of HIV/AIDS and tuberculosis treatment in developing countries. AIDS. 2007;21 Suppl 4: S81–87. pmid:17620757
  49. 49. Maritz J, Preiser W. Cost-effectiveness of nucleic acid amplification tests for identifying acute HIV infections. J Clin Microbiol. 2011;49: 1704. pmid:21464274
  50. 50. Hutchinson AB, Patel P, Sansom SL, Farnham PG, Sullivan TJ, Bennett B, et al. Cost-effectiveness of pooled nucleic acid amplification testing for acute HIV infection after third-generation HIV antibody screening and rapid testing in the United States: a comparison of three public health settings. PLoS Med. 2010;7: e1000342. pmid:20927354
  51. 51. Long EF. HIV screening via fourth-generation immunoassay or nucleic acid amplification test in the United States: a cost-effectiveness analysis. PLoS One. 2011;6: e27625. pmid:22110698