Comparative performance evaluation of Wako β-glucan test and Fungitell assay for the diagnosis of invasive fungal diseases

The Fungitell assay (FA) and the Wako β-glucan test (GT) are employed to measure the serum/plasma 1,3-β-D-glucan (BDG), a well-known invasive fungal disease biomarker. Data to convincingly and/or sufficiently support the GT as a valuable alternative to the FA are yet limited. In this study, we evaluated the FA and the GT to diagnose invasive aspergillosis (IA), invasive candidiasis (IC), and Pneumocystis jirovecii pneumonia (PJP). The FA and GT performances were compared in sera of patients with IA (n = 40), IC (n = 78), and PJP (n = 17) with respect to sera of control patients (n = 187). Using the manufacturer’s cutoff values of 80 pg/mL and 11 pg/mL, the sensitivity and specificity for IA diagnosis were 92.5% and 99.5% for the FA and 60.0% and 99.5% for the GT, respectively; for IC diagnosis were 100.0% and 97.3% for the FA and 91.0% and 99.5% for the GT, respectively; for PJP diagnosis were 100.0% and 97.3% for the FA and 88.2% and 99.5% for the GT, respectively. When an optimized cutoff value of 7.0 pg/mL for the GT was used, the sensitivity and specificity were 80.0% and 97.3% for IA diagnosis, 98.7% and 97.3% for IC diagnosis, and 94.1% and 97.3% for PJP diagnosis, respectively. At the 7.0-pg/mL GT cutoff, the agreement between the assays remained and/or became excellent for IA (95.1%), IC (97.3%), and PJP (96.5%), respectively. In conclusion, we show that the GT performed as well as the FA only with a lowered cutoff value for positivity. Further studies are expected to establish the equivalence of the two BDG assays.


Introduction
Invasive aspergillosis (IA), invasive candidiasis (IC), and Pneumocystis pneumonia (PJP, previously known as PCP) represent the most prevalent invasive fungal diseases (IFDs) worldwide a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 [1]. These diseases mainly affect immunocompromised (or immunosuppressed) hosts, causing estimated over 1.6 million deaths annually [2]. Causative agents of IC include different Candida species [3], whereas the main cause of IA remains Aspergillus fumigatus [4] and PJP is uniquely caused by Pneumocystis jirovecii, formerly Pneumocystis carinii [5]. As IFD symptoms can be subtle and/or nonspecific, it is difficult to identify and treat the cause of disease, especially in patients with hematological malignancies [6]. Furthermore, microbiological confirmation of IFD with conventional, culture-dependent methods may yield false-negative results [7], hence, molecular, culture-independent methods to enhance the diagnostic sensitivity need to be developed [8]. Thus, pending the microbiological diagnosis, an empirical treatment targeting infectious and non-infectious causes may be necessary [6].
As broad fungal biomarker for IFD (the only notable exception are mucormycosis, cryptococcosis, and blastomycosis) [9][10][11][12], serum 1,3-β-D-glucan (BDG) has shown wide utility in specific clinical settings [13], including IA, IC, and PJP [14]. In a meta-analysis, He et al. focused on the accuracy of cutoff values to diagnose IFD obtained with BDG detection assays [15]. One was the Fungitell assay (FA; Associates of Cape Cod, East Falmouth, MA), FDA cleared and Conformité Européenne (CE) marked, which has been most used in the Western Hemisphere (Europe and United States), and another was the Wako β-glucan test (GT; FUJI-FILM Wako Pure Chemical Corporation, Osaka, Japan), CE marked, which has recently been introduced in Europe.
Both assays rely on the BDG ability to activate factor G, a serine protease zymogen, in the Limulus (horseshoe crab) coagulation cascade. Activated factor G converts the inactive proclotting enzyme to the active form, which in turn cleaves an artificial substrate used for colorimetric (FA) or turbidimetric (GT) detection. Although BDG concentrations are measured through spectrophotometric readings, dissimilarity of cutoff values between the assays may be related to differences in the standards and/or affinity/reactivity of reagents in each assay [16]. Using the proposed 80 pg/mL (FA) and 11 pg/mL (GT) cutoff values [15], a recent comparison of the two assays for PJP diagnosis showed GT to be more specific and FA to be more sensitive, at a statistically significant level [17]. Interestingly, the sensitivity of GT equaled that of FA (at a cutoff of �60 pg/mL) and the specificity was significantly better than that of the FA, when the GT cutoff value lowered from 11 pg/mL to 3.616 pg/mL [17]. Consistently, previous work showed sensitivities of the FA for both IC (i.e., candidemia) and PJP diagnoses to be superior to those of the GT [18]. Again, lowering the GT cutoff value to �3.8 pg/mL resulted in sensitivities of the GT that became acceptable for candidemia (with a decreasing specificity from 98.0% to 91.0%) and excellent for PJP, respectively [18]. However, while more data are required to support the GT as a valuable alternative to the FA (especially for patients with candidemia), not enough data exist about the GT to diagnose IA. Therefore, we compared the performance of the GT with that of the FA in well-characterized groups of patients with IA, IC, and PJP, with reference to appropriate control patients. Similar to previous studies [17,18], we also tried to define the optimal GT cutoff values which could allow to reliably exclude IFD (mainly due to Aspergillus, Candida, or P. jirovecii).

Ethics statement
This study was conducted at the Fondazione Policlinico Universitario A. Gemelli (FPG) IRCCS of Rome, Italy, and was approved by the Ethics Committee of the FPG (application number 38367/19) and a waiver of informed consent was granted. Sample processing and data analysis were performed anonymously.

Study design
We conducted a retrospective performance assessment study on archived patients' serum samples, which had been collected as part of routine clinical care at the FPG IRCCS, a large tertiary care hospital in Rome, Italy. We used serum instead of plasma samples because of our previous findings showing that the two sample types may be considered equivalent [19]. Samples were from adult patients (mainly oncology/hematology patients) who were classified as having proven or probable IFD according to the 2008 European Organization for the Research and Treatment of Cancer/Mycoses Study Group (EORTC/MSG) criteria [20]. Samples were also from patients who were classified as having PJP if they had clinical signs and symptoms of respiratory infection (i.e., dyspnea, cough, or hypoxemia) supported with radiological findings (e.g., diffuse ground glass opacities on chest radiograph) and with positive Pneumocystis-specific PCR and/or immunofluorescence staining results of respiratory samples [21]. To enhance the robustness of the analysis, only patients with proven or probable IFDs were evaluated as IFD cases. Serum samples from patients who had major risk factors for IFD but who did not meet the criteria for proven or probable disease (i.e., who had no evidence of IFD) were included as controls. Furthermore, no diagnoses were based on detection of BDG (see below), which is part of the mycological criteria for the 2008 EORTC/MSG classification of patients with proven/probable/possible or no IFD [20]. At the time of writing this article-and shortly after completion the study, the EORTC/MSG education and research consortium updated definitions of IFDs [22]. However, according to the 2019 EORTC/MSG criteria, no reclassification was required for either IA or IC, while PJP cases were all classifiable as proven IFD, in our study [22].
We initially considered all the patients with a first serum BDG sample collected for routine mycological testing eligible. Then, we included only the patients for whom a classification as proven or probable IA (or IC) according to the 2008 EORTC/MSG definitions was available. All patients' samples were from the closest time (± 2 days) when evidence or no evidence of IFD was obtained for cases and controls, respectively, and were stored at -80˚C until the study time.

Serum BDG measurement
Before BDG testing, frozen serum samples were thawed at room temperature and briefly vortexed. The same investigator tested each sample's aliquots with the FA and the GT in parallel and in a blinded manner for each patient's disease classification at the time of testing. The two assays were performed in accordance with the manufacturer's instructions. In both assays, the BDG measurement relies on a modification of the Limulus amebocyte lysate (LAL), which results in colorimetric (FA) or turbidimetric (GT) reaction changes. Briefly, for the FA 5 μL of serum (in duplicate) was used and the LAL reaction was monitored at 37˚C for 40 min in an ELx808 microplate reader (BioTek Instruments, Winooski, VT). By comparing with a standard curve, the mean optical density change over time was calculated to determine the sample's BDG concentration. A positivity threshold of 80 pg/mL was used throughout the study. For the GT, 100 μL of serum was used and the LAL reaction (i.e., gelation) was monitored at 37˚C for a maximum of 90 min in a MT-6500 toxinometer (FUJIFILM Wako Pure Chemical Corporation). By comparing with a calibration curve (supplied with each lot by the manufacturer), the gelation time was calculated to determine the sample's BDG concentration. A positivity threshold of 11 pg/mL was initially used in the study (see below for positivity threshold optimization). Samples with positive results of > 500 pg/mL (FA) or > 600 pg/mL (GT) were diluted and retested.

Data collection and statistical analysis
Statistical analysis was performed using GraphPad Prism version 8.2 (GraphPad Software, La Jolla, CA) and MedCalc Statistical Software version 19.0.7 (MedCalc Software bvba, Belgium). Patients' demographics and BDG data were expressed as percentages or medians with interquartile ranges (IQR), respectively. To determine sensitivity and specificity of BDG assays, together with their respective 95% confidence intervals (CIs), we constructed 2 × 2 tables using IFD (IA, IC, or PJP) patients as true cases and non-IFD patients as controls. For both assays, receiver operating characteristic (ROC) curves were generated and cutoff values were derived to assess the diagnostic accuracy of serum BDG for case (IFD) versus control (non-IFD) patients. The highest Youden index indicated the optimal BDG cutoff. Variables were compared using chi-square test and Mann-Whitney U test, as appropriate, whereas differences in performance parameters between BDG assays were assessed using McNemar's test. Agreement between BDG assays was determined by Spearman's correlation, whereas the strength of agreement was determined by Cohen's kappa statistic. Thus, values higher than 0.80 represented excellent agreement, values between 0.80 and 0.4 represented substantial to moderate agreement, and values lesser than 0.4 represented fair to slight agreement. Statistical significance was set at a < .05 P-value. Table 1 shows demographics for the 322 patients from whom 322 serum samples were tested with two BDG assays (FA and GT), of which 78 samples were from proven IC (75 candidemia and 3 intraabdominal candidiasis) cases, 40 samples from probable IA (38 pulmonary and 2 disseminated aspergillosis) cases, and 17 samples from PJP cases. For all 135 cases, fungal disease classification (proven or probable) was obtained as previously described [20,21]. We also included 187 samples from 187 patients, for whom clinical, radiographic, and microbiological findings indicated no evidence of IFD (controls). Table 2 shows details about the mycological evidence used to classify the 135 cases as IFD.

Performances of the FA and the GT
The FA (cutoff for positivity, �80 pg/mL) and the GT (cutoff for positivity, �11 pg/mL) gave positive results in 132 (97.8%) and 110 (81.5%) of 135 IFD samples, respectively. Of 187 ) pg/mL for the FA and the GT, respectively. These levels were significantly higher than the levels in controls (0 (0-11) and 0 (0-0), respectively; P < .0001). Using the aforementioned cutoff values, the percentages of sensitivity and specificity were 92.5 and 99.5 for the FA, and 60.0 and 99.5 for the GT, respectively ( Table 3). The sensitivity of FA was significantly greater than the sensitivity of GT (92.5% versus 60.0%, P < .001).
In IC cases, the median (IQR) BDG concentrations were 516 (250-837) pg/mL and 45.57 (16.10-109.9) pg/mL for the FA and the GT, respectively. These levels were significantly higher than the levels in controls (P < .0001). Using the aforementioned cutoff values, the percentages of sensitivity and specificity were 100.0 and 97.3 for the FA, and 91.0 and 99.5 for the GT, respectively ( Table 3). The sensitivity of FA was significantly greater than the sensitivity of GT (100.0% versus 91.0%, P < .001).
In PJP cases, the median (IQR) BDG concentrations were 512 (404-639) pg/mL and 46.88 (26.43-166.3) pg/mL for the FA and the GT, respectively. These levels were significantly higher than the levels in controls (P < .0001). Using the aforementioned cutoff values, the percentages of sensitivity and specificity were 100.0 and 97.3 for the FA, and 88.2 and 99.5 for the GT, respectively ( Table 3). The sensitivity of FA was significantly greater than the sensitivity of GT (100.0% versus 88.2%, P = .01).
To improve the GT diagnostic performance, we determined the optimal positivity threshold for the GT (7.0 pg/mL) with the highest Youden index, which corresponds to the maximal sensitivity and specificity combination. As depicted in Fig 1, ROC analysis for proven/probable IFD versus no IFD at the optimized cutoff value generated an area under the curve (AUC) of 0.992 (95% CI, 0.975-0.999). Using the 7.0-pg/mL cutoff, the percentages of sensitivity and

Comparability of the FA and GT results
Assessing the quantitative agreement between the assays showed that a significant correlation between the BDG levels measured by the FA and the GT for IFD and non-IFD patient samples (Fig 2). In particular, these levels correlated significantly-although differently-for the IA (r = 0.670 and P < .0001; n = 40), IC (r = 0.703 and P < .0001; n = 78), and PJP (r = 0.667 and

Discussion
To the best of our knowledge, two retrospective case-control studies have carefully evaluated the GT-the last approved BDG assay on the European market of s-in comparison with the FA in serum samples [17,18]. Using a similar study design, we compared the GT with the FA to diagnose IFD in 322 patients categorized into IC (n = 78), IA (n = 40), PJP (n = 17), and non-IFD (n = 187) groups. Indeed, we established the overall GT and FA performances in two patient groups (135 with IFD and 187 without IFD), as well as the specific GT and FA performances in the IA, IC, and PJP groups with respect to the non-IFD group. Importantly, to compare the GT with the FA results, we followed the strategy of testing all archived patient samples in parallel, which reduces the bias due to probability of BDG degradation during sample storage, as successfully shown [17]. In contrast, Friedrich et al. performed FA measurement at the time of sampling and GT measurement on long-term stored samples [18]. By applying the manufacturer's recommended cutoffs for positivity, we showed that the sensitivity of the FA was higher than the sensitivity of the GT in IFD patients overall (97.8% versus 81.5%, respectively) and, in particular, IA (92.5% versus 60.0%, respectively), IC (100% versus 91.0%, respectively), or PJP (100% versus 88.2%, respectively) patients. While the PJP values in our study are very similar to those reported by Friedrich et al. in PJP patients (sensitivity of FA and GT were 100% and 88.9%, respectively) [18], our IC values differ substantially from those reported by Friedrich et al. in candidemia patients (sensitivity of FA and GT were 86.7% and 42.5%, respectively) [18]. However, our findings are surprising, particularly if compared to the results of several meta-analyses published over recent years [9][10][11]. The pooled sensitivity and specificity of BDG for invasive fungal infections (excluding P. jirovecii infections) varied from 76.0% and 85.0% [10], 76.8% and 85.3% [9] to 80.0% and 82.0% [11], respectively. Conversely, in one meta-analysis [11], the pooled sensitivity and specificity for PJP were 96.0% and 84.0%, respectively. In agreement with the results by meta-analyses [9][10][11] but in marked contrast with those by Friedrich et al. [18], one study [23] showed a sensitivity of 67.0%-neighbor, albeit still distant, to ours-and a specificity of 93.0% for the GT in a similar-sized cohort of patients with candidemia. Like in the Friedrich et al.'s study [18], applying a decreased cutoff of 7 pg/mL allowed Dichtl et al. to obtain a sensitivity of 73.0%, while specificities in both the studies remained as high as 91.0% [18] and 93.0% [23]. Another study by Dichtl et al. [24] conducted in the setting of PJP, showed that the specificity of GT-using an 11-pg/mL cutoff-was 100% in 25 control individuals who tested negative for P. jirovecii DNA by quantitative real-time PCR from respiratory samples. In the same study [24], the GT sensitivity increased from 86.0% to 91.0% after exclusion of the cases with slightly positive PCR results-that are usually negative in microscopy. Thus, our data unravel unexpected features of both GT and FA, which seem to subvert the paradigms of specificity as a "FA weakness" and of sensitivity as a "GA weakness" [25].
The reasons for conflicting results among studies are unknown, but we cannot exclude the influence of factors as a possible explanation for the observed differences in BDG assays' sensitivity and specificity. For example, findings from stratified (subgroup) analyses in two independent meta-analysis studies led to observe a lower EORTC/MSG diagnostic accuracy attributable to lower sensitivity compared to similar criteria (e.g., histopathological examination and/or microbiological culture from blood or sterile material) [11,15]. In the present study, we used the 2008 EORTC/MSG criteria for both IA and IC [20], and 3 (1.3%) of 78 patients had an IC diagnosis based on histology only. Unlike us, Friedrich et al. [18] exclusively used mycological culture from blood as a reference standard. Although blood culture serves to document the proven presence of invasive fungal infection [20], three patients with false-positive BDG results in the study by Friedrich et al. had negative blood cultures but were positive for the mannan antigen [18], which is a highly specific serum biomarker for IC [26]. Furthermore, fungal antigen levels in the blood are generally higher in culture-positive cases, so the inclusion of patients based on a blood culture positive may not allow a correct appraisal of the BDG performance [23]. In our study, the median (IQR) value of galactomannan OD index in 40 patients diagnosed with probable IA was 1.6 (1.3-2.1), giving a measure of moderately high fungal burdens in these patients. This could indicate later infection stages in these patientsthat in turn could result in higher BDG values-unless the exposure to mold-active antifungal prophylaxis in many patients could have masked their actual galactomannan levels [8]. Ultimately, for all analyses, we used a mixed control group of patients with different (albeit partially overlapping) risk factors for IFD [8,27], which could skew the BDG diagnostic parameters. To exclude this possibility, we performed a rough analysis using control subgroups. Interestingly, we found no one or slight differences in specificities when they were cal- Unlike IA, where the low sensitivity (60.0%) makes the GT not suitable for diagnosis using the manufacturer's recommended cutoff value of 11 pg/mL, the sensitivities of the GT in IC (91.0%) and PJP (88.2%) were already good without lowering the cutoff value. However, the GT sensitivity in IA increased to 80.0% with a positivity threshold of 7.0 pg/mL and without compromising specificity (97.3%). Applying the 7.0-pg/mL cutoff value, the GT sensitivity also increased for both IC (from 91.0% to 98.7%) and PJP (from 88.2% to 94.1%), whereas at this cutoff the specificity for both IC and PJP slightly decreased (from 99.5% to 97.3%). Interestingly, the GT cutoff value proposed in this study was almost twice the value proposed elsewhere, such as 3.8 in candidemia [18] and 3.616 in PJP [17]. Of note, our value also encompasses IA, and this is not surprising because BDG testing for yeast or yeast-like fungi such as Candida species and P. jirovecii may require a cutoff value different from that required for molds such as Aspergillus species. Accordingly, cutoffs to optimize the performance of the BDG assay may be different depending on the pathogen and host [14]. Therefore, it should be mandatory to interpret BDG results in relation to not only the specific pathogen, but also the specific patient population [28].
Our comparison of two BDG assays showed that both quantitative correlation and qualitative agreement between the FA and the GT were very good. However, one of the assays failed to categorize patients correctly. In particular, 13 IA, 7 IC, and 2 PJP samples provided false negative results only with the GT, because the BDG content in these samples did not reach the positivity threshold recommended by the manufacturer. Lowering the GT cutoff value allowed us to recover 5 IA, 6 IC, and 1 PJP samples as true positive samples, thereby indicating that the manufacturer's cutoff value might not be appropriate for patients with IA, IC, and PJP. This was in agreement with the observations reported elsewhere [17,18].
With the present study, we aimed to provide further data about the role of serum BDG in the diagnosis of IFDs by means of a comparative assessment of both assays in terms of sensitivity and specificity using a large number of patient samples. However, we acknowledge the unavoidable limitations of the study. First, we did not explore whether lower sensitivity of the GT, particularly in IA samples, could be due to the use of serum instead of plasma, which the GT manual has listed as a principal specimen for clinical investigation. Nevertheless, we previously showed that serum samples effectively equate plasma samples for BDG measured with GT in patients with probable or proven fungal diseases [19]. Second, we did not assess the predictive value of both assays because of the artificially high prevalence of IFD in our test population influencing the value. However, we well know that, at least for candidemia, high sensitivity and negative predictive value for serum BDG are both helpful to exclude disease in order to withhold treatment [29]. Third, we waived to compare both assays for important features such as their layout and workflow. However, we experimented that GT is technically less complex than the FA regarding the possibility either of testing samples individually or up to 16 samples in parallel, as well as the use of a standard curve provided by the manufacturer and, above all, more simplicity to execute the assay. Fourth, unlike previous studies [17,18], the number of PJP case tested by us was very small.
In conclusion, while confirming the good diagnostic accuracy of serum BDG assay [30], our findings support and/or extend the diagnostic value of both FA and GT into clinical settings such as IA, IC, and PJP. We show that the GT performed almost as the FA after optimizing the GT cutoff value for positivity. Further studies are yet necessary to establish the equivalence of both assays in order to provide equally reliable BDG results that can ultimately help clinicians with hard-to-diagnose fungal diseases.