Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Importance of Operational Factors in the Reproducibility of Aspergillus Galactomannan Enzyme Immune Assay

  • Nicolas Guigue,

    Affiliations Laboratoire de Parasitologie-Mycologie, Groupe hospitalier Lariboisière-Saint Louis, Assistance Publique-Hôpitaux de Paris (APHP), Paris, France, Université Paris Diderot, Sorbonne Paris Cité, Paris, France

  • Samuel Lardeux,

    Affiliation Bio-Rad Laboratories, Steenvoorde, France

  • Alexandre Alanio,

    Affiliations Laboratoire de Parasitologie-Mycologie, Groupe hospitalier Lariboisière-Saint Louis, Assistance Publique-Hôpitaux de Paris (APHP), Paris, France, Université Paris Diderot, Sorbonne Paris Cité, Paris, France, CNRS URA3012, Paris, France, Institut Pasteur, Unité de Mycologie Moléculaire, Centre National de Référence Mycoses Invasives et Antifongiques, Paris, France

  • Samia Hamane,

    Affiliation Laboratoire de Parasitologie-Mycologie, Groupe hospitalier Lariboisière-Saint Louis, Assistance Publique-Hôpitaux de Paris (APHP), Paris, France

  • Marc Tabouret,

    Affiliation Bio-Rad Laboratories, Steenvoorde, France

  • Stéphane Bretagne

    stephane.bretagne@sls.aphp.fr

    Affiliations Laboratoire de Parasitologie-Mycologie, Groupe hospitalier Lariboisière-Saint Louis, Assistance Publique-Hôpitaux de Paris (APHP), Paris, France, Université Paris Diderot, Sorbonne Paris Cité, Paris, France, CNRS URA3012, Paris, France, Institut Pasteur, Unité de Mycologie Moléculaire, Centre National de Référence Mycoses Invasives et Antifongiques, Paris, France

Importance of Operational Factors in the Reproducibility of Aspergillus Galactomannan Enzyme Immune Assay

  • Nicolas Guigue, 
  • Samuel Lardeux, 
  • Alexandre Alanio, 
  • Samia Hamane, 
  • Marc Tabouret, 
  • Stéphane Bretagne
PLOS
x

Abstract

Background

The Platelia Aspergillus Ag assay (Bio-Rad) is designed for detecting Aspergillus galactomannan (GM) and is widely used for diagnosing invasive aspergillosis but is hampered by variable occurrences of unreproducible positive results. Frequency and origin of these unreproducible results have not been formally studied.

Methods

Different technicians simultaneously performed four tests on 550 consecutive sera from adult patients (Test#1-Test#2 for extraction#1 and Test#3-Test#4 for extraction#2). The samples were classified as confirmed negative [all tests with GM optical density index (GM-ODI) <0.5], confirmed positive (all tests with GM-ODI ≥0.5), extraction unreproducible positive (Test#1 and Test#2 ODIs ≥0.5, and Test#3 and Test#4 GM-ODIs <0.5, or conversely), and ELISA unreproducible positive (only one test with GM-ODI ≥0.5). The samples with positive and negative GM-ODIs within the assay coefficient of variation values were classified as non-conclusive. Four similar additional tests were performed after ≤72h storage at 4°C and a new GM test after 8 months at -20°C.

Results

Five-hundred-twenty sera (94.5%) were confirmed negative, 15 (2.7%) confirmed positive, 4 (0.7%) extraction unreproducible positive, 6 (1.1%) ELISA unreproducible positive, and 5 (0.9%) non-conclusive. Upon retesting, the unreproducible positive results turned negative except for one which turned non-conclusive. The confirmed positive and non-conclusive had similar GM-ODIs (p>0.4) upon retesting after storage ≤72h at 4°C (n = 20) or eight months at -20°C (n = 17).

Conclusions

Operational unreproducible positives represent 33% of the GM-positive results and a second sample evaluation appears mandatory to avoid useless investigations or treatments. When operational artifacts are excluded, GM remains stable at standard storage conditions.

Introduction

Galactomannan (GM) is a fungal antigen produced by several molds including Aspergillus fumigatus. GM release in bloodstream is the rationale for its use for diagnosis [1] and management [2,3] of invasive aspergillosis. The most widely accepted test is the Platelia Aspergillus Ag assay (Bio-Rad Laboratories, Marnes la Coquette, France), an Enzyme Linked Immunosorbent Assay using a rat monoclonal antibody [4]. The test is mainly used as a screening test in serum of patients at risk of invasive aspergillosis and has been implemented in numerous hematological wards since the 90’s [57]; it is currently accepted as a microbiological criterion for the diagnosis of invasive aspergillosis [8]. Therefore, a GM positive result is often the only element [9], although concerns have been raised about the accuracy of GM testing [1,10].

Performance of the assay is variable, with metaanalysis showing pooled sensitivity of 0.71 and pooled specificity of 0.89 in proven cases of invasive aspergillosis, i.e. based on biopsy diagnosis [11]. False reactivity has always been observed from 5% in adults to as much as 83% in newborn babies [10]. This false positivity rate has led the manufacturer to recommend retesting of positive samples. This resulted in publications underlying the possibility to observe unreproducible positive results [1220]. Recent reports have also questioned the role of long sample storage in the lack of reproducibility with different conclusions [13,15,16,2022].

We therefore performed a formal comparison between the routine laboratory diagnosis and a Bio-Rad referent operator who performed additional testing blind to the routine one. Based on this intra-laboratory reproducibility study, our purpose was to characterize the analytical false positive results occurring during testing and to distinguish them from positive results related to the serum and the patient. Among the possible limitations of the GM assay, little attention has been paid to the initial preanalytical preparation steps of the assay as a cause of false GM positivity leading to imprecise or erroneous conclusions in several previous studies.

Material and Methods

Ethics Statement

Saint-Louis Hospital is a 650-bed tertiary university hospital with major clinical activities in hematology, renal transplantation and oncology. This study was a non-interventional study with no change in the usual procedures. Biological material and clinical data were obtained only for standard diagnostic following physicians’ prescriptions with no specific sampling. According to the French Health Public Law (CSP Art L1121-1.1), such protocol does not require approval of an ethics committee and is exempted from specific informed consent application.

GM assay

GM detection was performed by Platelia Aspergillus Ag Assay (Ref#62794) following the manufacturer's instructions (Bio-Rad Laboratories). Serum handling was performed under safety cabinet until the heating phase of the procedure. Heating at 100°C was performed in water bath (TW 12, Julabo, Seelbach, Germany). The R3, R4 and R5 reagents are defined as negative control serum, cut-off control serum, and positive control serum, respectively. The optical density (OD) results were translated in GM-OD index (GM-ODI) using the ratio OD-sample/OD-R4 (mean of two replicates). The positivity threshold is GM-ODI ≥0.5 as recommended by the manufacturer. An internal control (mix of different R4 reagents of the same batch since the positive control of the kit cannot be used as interlot control since it is not always calibrated at the same level) was assessed in each series to calculate the inter-assay coefficient of variation (CV). The mean of 104 internal control GM ODIs was 0.94+/-0.14, i.e. a CV of 14.5%. The 95% confidence interval (95% CI) for the 0.5 GM-ODI threshold was 0.36–0.64, after the formula 0.5 +/- 1.96 SD with SD = 0.5 CV.

We tested all the consecutive sera over 22 consecutive working days using the #62794 kit from February 17th to March 10th 2014. At day 0, each serum sample was tested four times (Fig 1). One test was performed by three different routine technicians (Operator#1 Test#1) over the study period without any change to the usual procedure. The three other tests were performed by a unique technician (Operator#2) blind of routine results; Test#2 was performed on the same GM extraction as Test#1, and Tests #2 and #3 on a different GM extraction.

thumbnail
Fig 1. Schematic representation of the reproducibility study.

Each serum was extracted twice by two different workers. Interpretation was as shown knowing that other combinations were possible but that only those obtained during the study are shown. Results close to the positivity threshold value (GM optical density index = 0.5) were classified as non-conclusive taking into account the interassay coefficient of variation of the kit.

https://doi.org/10.1371/journal.pone.0124044.g001

At day 1, all the samples with at least one of the four tests (Test#1-#4) GM-ODI>0.5 were retested similarly as Day 0. Test#5 was performed by three different routine technicians (Operator#1 Extraction#3 Test#5) following the usual procedure. Three other tests were performed by Operator#2; Test#6 was performed on Extraction#3, and Tests #7 and #8 on Extraction#4.

The samples were classified as confirmed negative (Test#1–4 with ODI <0.5), confirmed positive (Test#1–4 with ODI ≥0.5) and unreproducible positive subdivided in extraction and ELISA unreproducible positive. Extraction unreproducible positive corresponded to a positivity linked to one of the two extractions (Test#1 and Test#2 ODIs ≥0.5, and Test#3 and Test#4 ODIs <0.5, or conversely). ELISA unreproducible positive corresponded to the positivity related to the ELISA performing with only one replicate with GM-ODI ≥0.5 (Fig 1). When at least one Test#1-#4 GM-ODI value was <0.5 but with the mean of the four tests within the 95% CI (0.36–0.64), these samples were classified as non-conclusive. A new classification was performed after retesting at Day 1 using the same criteria.

The remaining serum after the reproducibility study was stored at -20°C for 8 months. After thawing, a new GM extraction was performed and Test#9 performed by a unique routine technician in a single run. Test#9 was compared to Test#1.

Statistical analyses

Statistical analyses were performed using the software online OpenEpi Software Version 2.3.1, Analyse-it SoftwareVersion 2.25, and Prism 5.0 (GraphPAD Software). We used Fisher's exact test or chi-squared test for categorical variables, and t-tests or one-way ANOVA for continuous variables with a normal distribution with p<0.05 significance.

Results

We tested 567 sera from 235 patients over 22 consecutive working days in 16 series with the lot # 3L0024 of the kit. Seventeen samples were excluded from the reproducibility analysis since their volume did not allow all the tests.

Among the 550 remaining samples, 520 (94.5%) were confirmed negative upon Test#1-#4 results and 30 sera (5.5%) had at least one test positive (Table 1). Fifteen of these 30 sera (50%) were confirmed positive with upon Test#1-#4positive results, of which two (S203 and S 277, Table 1) were from one patient with probable invasive aspergillosis according to the European Organization for Research and Treatment of Cancer/ Mycoses Study Group [8].; 5 sera (16.7%) were non-conclusive; 10 sera (33.3%) were unreproducible positive divided in 6/10 (60%) extraction and 4/10 (40%) ELISA unreproducible positive (Table 1).

thumbnail
Table 1. Galactomannan optical density index (GM-ODI) results of the reproducibility study upon the four tests (Test#1-#4) of the serum samples performed at Day 0, the four tests (Test#5-#8) performed at Day 1, and the GM-ODI after 8 month storage at -20°C (Test#9).

https://doi.org/10.1371/journal.pone.0124044.t001

When retested at Day 1 with four additional tests (Test#5–8) after storage at 4°C, the Day 0 confirmed positives and non-conclusives were similarly classified but for one non-conclusive which turned confirmed positive (S311 Table 1). For the Day 0 unreproducible positive results, all turned confirmed negative but for one which was classified as non-conclusive (S25 Table 1). Table 1 also shows Test#5 compared with Test#1 results after storage at 4°C for <72h as in our routine conditions. Test#5 results of the confirmed positives and non-conclusive were similar to Test#1 GM-ODIs (paired t-test: p = 0.49). Quantitatively, we observed two distinct populations when comparing Day 0 and Day 1 results. On one hand, the confirmed positives and non-conclusives with no GM-ODI variation (slope at 0.99). On the other hand, the ELISA and extraction unreproducible positives represented a distinct group of sera (Fig 2).

thumbnail
Fig 2. Graph plotting initial Test#1 against Test#5 (see Fig 1) performed after serum storage ≤72h at +4°C.

A linear trend line was observed for confirmed positive and unreproducible positive results as in Fig 2.

https://doi.org/10.1371/journal.pone.0124044.g002

Among the 30 sera with at least one GM-ODI>0.5 at Day 0, 27 had enough serum left to be stored at -20°C which were retested (Extraction#5 Test#9) after 8 months with the lot # 4G1027 of the kit. Table 1 shows Test#9 in comparison with Test#1 results. Once again, the confirmed positive and non-conclusive sera had similar GM-ODI (paired t-test: p = 0.63) whereas the unreproducible positives at Day 0 were all retested negative as at Day1.

Discussion

The present reproducibility study allowed us to clearly delineate on initial testing of the positive GM results between those which were confirmed positive and the unreproducible positive results. For the first time, we quantified the extraction and ELISA causes of these unreproducible positives, showing that both can intervene at a global rate of 33% (10/30), within the range of previous reports: 39% [16], 40% [14], 48% [12], 55% [18], and 64% [15] of the tested samples. Therefore, in contrast to hypotheses which favor a particular component and/or storage of the serum to explain the unreproducible positive results [13,15,16], we highlighted that most of them, if not all, were due to technical variation either during the extraction process or in the ELISA plate itself. Therefore two populations of sera can be differentiated; the first one with a positive result related to the sample for which the GM-ODI remains unchanged upon retesting, either after storage at 4°C for <72h or at—20°C for at least 8 months, and the second one with an initial positive result related to technical variation for which a second testing after storage at 4°C for <72h is negative.

These results lead us to reconsider all articles that have been recently published on GM reproducibility and stability during storage where these two populations were not identified [13,15,16,2022]. In contrast to the present study, the previous studies performed only one test on first determination and one repeat after storage [13,15,16]. Consequently they could not distinguish the initial positivity related to samples from that related to any technical problem. Interestingly, a lot of the figures from these studies look like ours with two groups of sera. One group for which there is no or low difference between tests and the other one for which the samples completely turned negative. Therefore, the conclusion that freezing reduces the risk of a false positive result [15] cannot be maintained without confirming that the initial positivity is a confirmed positive. In contrast, some studies like ours showed little impact of long-term storage at—20°C on GM-ODI values and that storage did not affect the sera from patients with invasive aspergillosis [20]. One can hypothesize that sera from patients with invasive aspergillosis were confirmed positive, hence the lack of impact of freezing on GM-ODIs. Similarly, several reports [12,14,16,17] noticed that the samples that turned negative on repeated test corresponded to patients with a low probability of invasive aspergillosis, which is also consistent with our results.

Unfortunately, we could not ascribe unreproducible positive results to a specific procedure failure or to a specific technician. The manual processing of the test is very sensitive to small variations of the heat extraction procedure for instance. An external source independent to the operator can occur. One can suspect GM contamination from the water-bath, which can be regularly checked, or from tips or tubes for treatment, which could make the use of antigen-free material compulsory. The various origins can explain the different rates of unreproducible positive results between laboratories. Whatever the source of this lack of reproducibility, it strongly supports re-analysis of a positive sample as soon as possible [1]. Of course, to perform a second test on the same serum does not completely exclude the possibility to obtain a second unreproducible positive result. Therefore, obtaining serial samples in parallel is also recommended [1], acknowledging that the risk of false positive cannot be definitively excluded. The non-conclusive category we created for the reproducibility study is questionable, since we did not classify non-conclusive sera with all GM tests positive but within the CV of the GM assay. However, in case of possible false positive results and non-conclusive results, given the risk of misidentifying an invasive aspergillosis, to provide a positive answer to the clinicians seems preferable. That is why we have not created a non-conclusive category in our routine procedure and insist on the GM-ODI kinetics upon serial samples for final correct interpretation.

The present work was focused on the analytical performances of the Platelia Aspergillus Ag kit and was not designed for deciphering between false positives due to an exogenous source of GM in serum as a consequence of blood contamination with antibiotics [23] or of other invasive fungal infections such as histoplasmosis, cryptococcosis, or penicilliosis [24]. Nor do we address the issue of false positive of digestive origin [10,25]. We think that confirmed GM positives, as a general rule, must be interpreted with care using all the information needed to make a final diagnosis of invasive aspergillosis.

In conclusion, the reasons why some sera turned totally negative on retesting are operator-dependent. We also showed the good stability of GM for at least 72h at +4°C and over 8 months at -20°C when dealing with the confirmed positive sera only. As a consequence, analytical false positive results must be eliminated in performing a second evaluation of the sample as soon as possible to avoid useless investigations or treatments and for correct interpretation when re-testing after storage. Further research on unreproducible positive results is warranted to decrease their occurrence, knowing that full automation of the assay is unlikely for technical reasons.

Supporting Information

Acknowledgments

We thank all the staff of the Saint Louis mycology laboratory for their technical involvement in the study and especially Mrs. Véronique Miquelis, Isabelle Cadic, and Marie-Claude Lambert. Disclaimer: Presented in part at 54th Interscience Conference on Antimicrobial Agents and Chemotherapy September 5–9, 2014 | Washington (e-poster#1863).

Author Contributions

Conceived and designed the experiments: NG AA MT SB. Performed the experiments: NG SL SH. Analyzed the data: NG SL AA SH MT SB. Contributed reagents/materials/analysis tools: NG SL AA SH MT SB. Wrote the paper: NG AA MT SB.

References

  1. 1. Marchetti O, Lamoth F, Mikulska M, Viscoli C, Verweij P, Bretagne S. (2011) ECIL recommendations for the use of biological markers for the diagnosis of invasive fungal diseases in leukemic patients and hematopoietic SCT recipients. Bone Marrow Transplant 47: 846–854. pmid:21927034
  2. 2. Bergeron A, Porcher R, Menotti J, Poirot JL, Chagnon K, Vekhoff A, et al. (2012) Prospective evaluation of clinical and biological markers to predict the outcome of invasive pulmonary aspergillosis in hematological patients. J Clin Microbiol 50: 823–830. pmid:22170907
  3. 3. Chai LY, Kullberg BJ, Johnson EM, Teerenstra S, Khin LW, Vonk AG, et al. (2012) Early serum galactomannan trend as a predictor of outcome of invasive aspergillosis. J Clin Microbiol 50: 2330–2336. pmid:22553232
  4. 4. Stynen D, Sarfati J, Goris A, Prevost MC, Lesourd M, Kamphuis H, et al. (1992) Rat monoclonal antibodies against Aspergillus galactomannan. Infect Immun 60: 2237–2245. pmid:1375195
  5. 5. Sulahian A, Tabouret M, Ribaud P, Sarfati J, Gluckman E, Latgé JP, et al. (1996) Comparison of an enzyme immunoassay and latex agglutination test for detection of galactomannan in the diagnosis of invasive aspergillosis. Eur J Clin Microbiol Infect Dis 15: 139–145. pmid:8801086
  6. 6. Bretagne S, Marmorat-Khuong A, Kuentz M, Latge JP, Bart-Delabesse E, Cordonnier C. (1997) Serum Aspergillus galactomannan antigen testing by sandwich ELISA: practical use in neutropenic patients. J Infect 35: 7–15. pmid:9279718
  7. 7. Maertens J, Verhaegen J, Lagrou K, Van Eldere J, Boogaerts M (2001) Screening for circulating galactomannan as a noninvasive diagnostic tool for invasive aspergillosis in prolonged neutropenic patients and stem cell transplantation recipients: a prospective validation. Blood 97: 1604–1610. pmid:11238098
  8. 8. De Pauw B, Walsh TJ, Donnelly JP, Stevens DA, Edwards JE, Calandra T, et al. (2008) Revised definitions of invasive fungal disease from the European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group (EORTC/MSG) Consensus Group. Clin Infect Dis 46: 1813–1821. pmid:18462102
  9. 9. Lortholary O, Gangneux JP, Sitbon K, Lebeau B, de Monbrison F, Le Strat Y, et al. (2011) Epidemiological trends in invasive aspergillosis in France: the SAIF network (2005–2007). Clin Microbiol Infect 17: 1882–1889. pmid:21668573
  10. 10. Mennink-Kersten MA, Donnelly JP, Verweij PE (2004) Detection of circulating galactomannan for the diagnosis and management of invasive aspergillosis. Lancet Infect Dis 4: 349–357. pmid:15172343
  11. 11. Pfeiffer CD, Fine JP, Safdar N (2006) Diagnosis of invasive aspergillosis using a galactomannan assay: a meta-analysis. Clin Infect Dis 42: 1417–1427. pmid:16619154
  12. 12. Bizzini A, Marchetti O, Meylan P (2012) Response to: lack of intra-laboratory reproducibility in using Platelia Aspergillus enzyme immunoassay test for detection of Aspergillus galactomannan antigen. Transpl Infect Dis 14: 218–219. pmid:22188592
  13. 13. Dufresne SF, Beauchemin S, Lavallee C, Laverdiere M (2014) Instability of Aspergillus galactomannan in stored clinical samples. J Clin Microbiol.
  14. 14. Furfaro E, Mikulska M, Miletich F, Viscoli C (2012) Galactomannan: testing the same sample twice? Transpl Infect Dis 14: E38–39. pmid:22487375
  15. 15. Johnson GL, Sarker SJ, Hill K, Tsitsikas DA, Morin A, Bustin SA, et al. (2013) Significant decline in galactomannan signal during storage of clinical serum samples. Int J Mol Sci 14: 12970–12977. pmid:23797658
  16. 16. Kimpton G, White PL, Barnes RA (2014) The effect of sample storage on the performance and reproducibility of the galactomannan EIA test. Med Mycol 52: 618–626. pmid:24850886
  17. 17. Oren I, Avidor I, Sprecher H (2012) Lack of intra-laboratory reproducibility in using Platelia Aspergillus enzyme immunoassay test for detection of Aspergillus galactomannan antigen. Transpl Infect Dis 14: 107–109. pmid:21733049
  18. 18. Pedroza KC, de Matos SB, de Moura DL, Oliveira MB, Araujo MA, Nascimento RJ, et al. (2013) Reproducibility of positive results for the detection of serum galactomannan by Platelia aspergillus EIA. Mycopathologia 176: 295–297. pmid:23943403
  19. 19. Upton A, Gugel A, Leisenring W, Limaye A, Alexander B, Hayden R, et al. (2005) Reproducibility of low galactomannan enzyme immunoassay index values tested in multiple laboratories. J Clin Microbiol 43: 4796–4800. pmid:16145143
  20. 20. Wheat LJ, Nguyen MH, Alexander BD, Denning D, Caliendo AM, Lyon GM, et al. (2014) Long-term stability at -20 degrees C of Aspergillus galactomannan in serum and bronchoalveolar lavage specimens. J Clin Microbiol 52: 2108–2111. pmid:24719449
  21. 21. Johnson GL, Sarker SJ, Bustin SA, Agrawal SG (2014) Insufficient demonstration of long-term stability of Aspergillus galactomannan. J Clin Microbiol 52: 4118. pmid:25324240
  22. 22. Wheat LJ, Nguyen MH, Alexander BD, Denning D, Caliendo AM, Lyon GM, et al. (2014) Reply to "insufficient demonstration of long-term stability of Aspergillus galactomannan". J Clin Microbiol 52: 4119. pmid:25324241
  23. 23. Mikulska M, Furfaro E, Del Bono V, Raiola AM, Ratto S, Bacigalupo A, et al. (2012) Piperacillin/tazobactam (Tazocin) seems to be no longer responsible for false-positive results of the galactomannan assay. J Antimicrob Chemother 67: 1746–1748. pmid:22499998
  24. 24. Barton RC (2012) Laboratory diagnosis of invasive aspergillosis: from diagnosis to prediction of outcome. Scientifica (Cairo) 2013: 459405. pmid:24278780
  25. 25. Guigue N, Menotti J, Ribaud P (2013) False positive galactomannan test after ice-pop ingestion. N Engl J Med 369: 97–98. pmid:23822795