Laboratory and Field Evaluation of the Partec CyFlow MiniPOC for Absolute and Relative CD4 T-Cell Enumeration

Background A new CD4 point-of-care instrument, the CyFlow miniPOC, which provides absolute and percentage CD4 T-cells, used for screening and monitoring of HIV-infected patients in resource-limited settings, was introduced recently. We assessed the performance of this novel instrument in a reference laboratory and in a field setting in Senegal. Methodology A total of 321 blood samples were obtained from 297 adults and 24 children, all HIV-patients attending university hospitals in Dakar, or health centers in Ziguinchor. Samples were analyzed in parallel on CyFlow miniPOC, FACSCount CD4 and FACSCalibur to assess CyFlow miniPOC precision and accuracy. Results At the reference lab, CyFlow miniPOC, compared to FACSCalibur, showed an absolute mean bias of -12.6 cells/mm3 and a corresponding relative mean bias of -2.3% for absolute CD4 counts. For CD4 percentages, the absolute mean bias was -0.1%. Compared to FACSCount CD4, the absolute and relative mean biases were -31.2 cells/mm3 and -4.7%, respectively, for CD4 counts, whereas the absolute mean bias for CD4 percentages was 1.3%. The CyFlow miniPOC was able to classify HIV-patients eligible for ART with a sensitivity of ≥ 95% at the different ART-initiation thresholds (200, 350 and 500 CD4 cells/mm3). In the field lab, the room temperature ranged from 30 to 35°C during the working hours. At those temperatures, the CyFlow miniPOC, compared to FACSCount CD4, had an absolute and relative mean bias of 7.6 cells/mm3 and 2.8%, respectively, for absolute CD4 counts, and an absolute mean bias of 0.4% for CD4 percentages. The CyFlow miniPOC showed sensitivity equal or greater than 94%. Conclusion The CyFlow miniPOC showed high agreement with FACSCalibur and FACSCount CD4. The CyFlow miniPOC provides both reliable absolute CD4 counts and CD4 percentages even under the field conditions, and is suitable for monitoring HIV-infected patients in resource-limited settings.


Ethics statement
We used the excess of routine blood samples from HIV-infected patients attending the clinic for routine CD4 T-cell enumeration. Blood sample records were directly anonymized and deidentified by giving them a consecutive study code prior to analysis. The study was approved by the Institutional Review Board of the Institute of Tropical Medicine, Antwerp (Belgium) and the National Ethical Committee of the Ministry of Health (Senegal) that waived the need of informed consent. For the purpose of this independent study, the CyFlow miniPOC device and its reagent kits, the FACSCount CD4 reagents, the Trucount tubes and the monoclonal antibodies were purchased.

Study design and participants
This study was conducted in two phases, the first one in the reference laboratory at the Immunology unit of Le Dantec hospital Dakar (March-May 2014), and the second in a peripheral laboratory at the Silence health center in Ziguinchor at 450 Km from Dakar (end May-June 2014). Blood samples, collected into K3-EDTA tubes, were taken consecutively from HIV-patients attending the clinic for routine CD4 counting.
The phase I consisted in evaluating the CyFlow miniPOC performances in a controlled laboratory environment where FACSCalibur and FACSCount CD4 were set as reference instruments.
In phase II, the CyFlow miniPOC was installed in a room without air conditioning, and results were compared with those from FACSCount CD4, which was used as reference instrument. During the study, working temperature in that room ranged from 30°C to 35°C. The FACSCount instrument was installed in a room with air-conditioning, as recommended.

Precision assessment
Intra-assay and inter-assay variations were assessed on 3 blood samples with clinically relevant CD4 counts (< 200, 300-500, > 500 cells/mm 3 ). The intra-assay, which assesses the tube-totube variability, including errors due to operator, was determined by repeating 10 times the entire CD4 staining procedure on the 3 blood samples. Inter-assay variation (day-to-day) was assessed by analysing each sample at three time points after blood collection (within 6h, 24h and 48h) on blood stored at room temperature. The percentage coefficient of variation (%CV) was calculated for intra-assay and inter-assay [22]. The sample carry over was measured by analyzing three sample pairs, each pair with one sample with a CD4 count greater than 600 cells/mm 3 read in duplicate (recorded as a l and a 2 ) followed by a second sample with a CD4 count lower than 300 cells/mm 3 read in duplicate too (recorded as b 1 and b 2 ), and the Broughton coefficient (k) was calculated [23,30]. The values of the internal quality control Count Check beads run daily on the CyFlow miniPOC were used to calculate the %CV of instrument precision.

Agreement between methods
CD4 counts were measured in whole blood samples within 6 hours after venipuncture on: FACS-Calibur, FACSCount CD4, and CyFlow miniPOC. All measurements were performed according to manufacturer's instructions by different operators to allow a blind reading. Before running samples, internal quality control beads were successfully tested daily on CyFlow miniPOC (Partec Count Check), on FACSCount (BD FACSCount Control), and on FACSCalibur (BD Calibrite).
The CyFlow miniPOC, flow cytometry-based CD4 POC, provides absolute and percentage CD4 T-cells. Briefly, 20mm 3 of whole blood were added into a sample tube (on top of the lyophilized antibody spot at the bottom of the tube) gently mixed and incubated for 15min (mix again after 5min) in the dark at room temperature. Then, Buffer 1 was poured and mixed, and the Buffer 2 was added directly prior to the measurement on the CyFlow miniPOC. Stained blood was completely aspirated with a new syringe until the plunger reaches the position 1 ml (avoid having air bubble in the extremity of the syringe). The syringe was attached to the device, analysis started and the results (CD4 count and CD4%) were available in less than 2min.
For FACSCalibur and FACSCount CD4 reference methods, samples were prepared, as previously described, to provide both absolute and percentage of CD4 T-cells. A CD3/CD4/CD45 panel in Trucount tubes was used on FACSCalibur, as previously described [4,12,23]. The FACSCalibur and the FACSCount CD4 are monitored by external quality control programs QASI (Quality Assessment and Standardization for Immunological measures relevant to HIV/ AIDS) and UK NEQAS (United Kingdom National External Quality Assessment Service).

Statistical analysis
The concordance correlation coefficient (ρ c = ρ x C b ) assesses the degree to which pairs of observations fall on the 45°line through the origin [31]. Passing-Bablok regression was used for method comparison [32]. Pollock and Bland Altman analyses were used to assess relative and absolute mean bias, respectively between absolute CD4 counts and CD4 percentages obtained by both methods. Mean bias (or trueness) and limits of agreement (LOA = mean bias ± 1.96 x SD) were calculated [33,34]. The percentage similarity between the results obtained from 2 different instruments was calculated for each sample. For each group, the mean similarity and the relative standard deviation (RSD) were determined [22,35]. Sensitivity and specificity to identify patients eligible for ART were calculated at commonly used CD4 thresholds (200, 350 and 500 cells/mm 3 ) to start ART. FACSCalibur or FACSCount CD4 results (CD4 count & CD4%) were set as the reference to determine ART eligible patients, and calculate misclassification, sensitivity and specificity.

Study population
A total of 188 HIV-infected patients were included in the first part of this study, of which, 35% (65/188) were male, and 64% (120/188) received ART. There were 169 adults and 19 children, including 5 under 5 years, with a global median (range) age of 39 (1-68) years. They had a median (range) CD4 count of 334 (10-1508) cells/mm 3 , and 51 had a low CD4 count 200, 87 had a CD4 count between 201 and 500, and 50 subjects had a CD4 count > 500.
At the Silence health center (second phase), 133 HIV-infected patients were enrolled, of which, 20% (26/133) were male, and 60% (80/133) received ART. There were 128 adults and 5 children, including only 1 under 5 years, with a global median (range) age of 43 (3-64) years. They had a median CD4 count (range) of 337 (2-1024) cells/mm 3 , and 35 of them had a CD4 count 200, 58 had a CD4 count between 201 and 500, and 40 had a CD4 count > 500. Table 1 summarizes the precision (%CVs) of the CyFlow miniPOC on replicates from samples with low, medium and high CD4 count. The CyFlow miniPOC showed a mean intra-assay variation (%CV) of 3.6% for CD4 counts and 4.5% for CD4%. The mean inter-assay %CV was 6.1% for CD4 counts and 3.2% for CD4%. Values at 48h did not seem inferior to values at 24h, and only the sample with a high CD4 count showed a drop from 1153 cells/mm 3 at day 0 to 906 cells/mm 3 at day 1, leading to the high CV of 13.45%. There was no significant carry over with a mean k of 0.12% for CD4 count, and 0.00% for CD4%. Based on the values obtained daily from the readings of the Count Check control beads, the CyFlow Counter showed a mean instrument precision of 2.8%.

Agreement between methods
Phase I: Evaluation of the CyFlow miniPOC performances in a controlled laboratory environment. Fig. 1 shows absolute CD4 count comparisons between CyFlow miniPOC and the conventional flow cytometers (FCM) in the reference laboratory.
The agreement between CyFlow miniPOC and each of the two reference systems (FACSCalibur and FACSCount CD4) was similar to the agreement between FACSCalibur and FACS-Count CD4. Table 3 summarizes the detailed comparisons between CyFlow miniPOC and FACSCalibur in the reference lab, within the CD4 categories. Table 4 summarizes the detailed comparisons between CyFlow miniPOC and FACSCount CD4 in the reference lab, within the CD4 categories.
Phase II: field validation of the CyFlow miniPOC at Silence health center. Fig. 3 depicts the comparison between CyFlow miniPOC and FACSCount CD4.
Similarity: mean (RSD) Comparisons between CyFlow miniPOC and FACSCount CD4 in the field within the CD4 categories are detailed in Table 5. Comparisons between CyFlow miniPOC and FACSCount CD4 in the reference lab and in the field showed a statistically different bias (p < 0.05) in overall data and within each of the CD4 categories. The CyFlow miniPOC showed better performance on the CD4 percentages in Ziguinchor compared to Dakar. However, on absolute CD4 counts, the performance was not in favor of one site (Tables 4 & 5).
The CyFlow miniPOC failed to analyze 3 samples in the reference lab, while 5 and 4 errors were reported on FACSCount CD4 respectively in the reference lab and in the field lab.

Discussion
The recent introduction of CD4 Point-of-care technologies has contributed to widen access to ART in resource-limited settings. Many POCs are much more affordable than the conventional flow cytometry based instruments and better adapted to the field as they are very mobile, easy to use and do not require a cold chain to store reagents [2,16,36]. In the present study, we independently evaluated the CyFlow miniPOC, a relatively new flow cytometry-based POC. Our study is the first to report on the performance of CyFlow miniPOC. The CyFlow miniPOC system was compared to two conventional flow cytometers, FACSCount CD4 and FACSCalibur.
The CyFlow miniPOC showed good precision with %CV < 7%, similar to precisions reported on other CD4 dedicated flow cytometers [6][7][8]10,22,23,[37][38][39][40][41][42][43]. The high %CV (13.45%) of inter assay variability for the high CD4 count specimen (Table 1) can be explained by the CD4 count drop from 1153 cells/mm 3 on fresh blood (within 6h) to 906 cells/mm 3 at 24h after venipuncture. This variability was not inherent to the CyFlow miniPOC, but to the sample itself. Indeed, when we checked on FACSCalibur for that sample at day 1, the CD4 result was similar. The CyFlow miniPOC seems to be more precise than the Pima CD4 using venous blood, which indeed showed a higher %CV, superior to 10% [21,22,44]. The CD4 counts and CD4% obtained from the CyFlow miniPOC showed high correlation and small bias as compared to the conventional flow cytometers FACSCalibur (-12.6 cells/mm 3 ) and FACSCount CD4 (-31.2 cells/mm 3 in reference lab, and 7.6 cells/mm 3 in the field). The agreement between CyFlow miniPOC and these conventional flow cytometers is similar to the agreement between FACSCount CD4 and FACSCalibur. These findings are similar to those reported in previous studies on other CyFlow instruments [38,39,45]. The CyFlow miniPOC bias is similar to those reported on other POC systems: MBio CD4 and Pima CD4 [16,18,[21][22][23][24]28] while other studies reported higher bias on Pima CD4 [21,27,44,46]. In the reference laboratory, the mean bias (absolute and relative) increased with the CD4 count. This finding is similar to the findings reported on CyFlow Counter [22] and on Pima CD4 [21][22][23]27]. In the field, there was an increase in absolute bias and a decrease in relative bias. Although the performances (biases) between the reference and the field sites were statistically different, this was not clearly in favor of one of the two sites (some parameters better in Dakar, some in Ziguinchor), and the CyFlow miniPOC showed good agreement with FACSCount CD4 in both sites.
The CyFlow miniPOC showed low rates of misclassification of patients (< 5%) at the different CD4 treatment thresholds which are similar to those reported for other dedicated flow , possibly as a result of a pipetting error. Fortunately, most of misclassified patients by the CyFlow miniPOC would benefit to an early ARTinitiation which helps to reduce morbidity and consequently HIV-transmission [48] and due to their small number would be of low importance for ART stock consumption.
With an incubation time of 15min, and a possibility to prepare samples individually or in batches, and using less than 2min per sample reading, the CyFlow miniPOC can easily run a batch of 10 samples per hour. However, the analyses cannot be automated. The CyFlow mini-POC has a turnaround time of 20min similar to other POCs. This is an important asset and limits the number of patients lost to follow up as clinicians can take adequate treatment decisions immediately.
To ensure reliability of results, the FACSCalibur and FACSCount CD4, used in this study, present good scores in the external quality assessment programs such as QASI and UK NEQAS. In addition, the operators received appropriate training on the use of the CyFlow miniPOC which took 3h (one morning), and daily calibration (internal quality control) was performed daily on FACSCalibur, FACSCount and CyFlow miniPOC before running samples.
The fact that our evaluation took place in two different settings reinforces the strength of our results, but also shows that results can be slightly different due to the type of setting (in our case, the operator was the same), and that any evaluation should be done in both settings.
In conclusion, the CyFlow miniPOC provides reliable absolute CD4 counts and CD4 percentages, which are in excellent agreement with those obtained from the conventional flow cytometers. With a batch of 10 samples per hour, a technician could easily analyze up to 60 samples/day using the CyFlow miniPOC. Thus, the CyFlow miniPOC is suitable for monitoring of HIV-infected patients and can be used in a central or peripheral laboratory. Further studies are required for a definitive performance statement.