Introduction

Flow Cytometry (FC) has been the testing platform of choice for CD4 T-lymphocyte enumeration in HIV infected patients [1–3] since the correlation between CD4 loss and HIV disease progression was first described [4–6]. South Africa has a high burden of disease with 11.2% (6.2 million) of the local populace living with HIV in 2015 (Statistics South Africa, 2015) and an average of ~ 9.7% of HIV+ patients noted to have CD4 counts less than 100cells/μl [7].

With the onset of the national HIV program in South Africa and the development of strategic HIV and AIDS Plans for South Africa [8–10], the challenges of providing CD4 enumeration as a routine test across multiple laboratories, led to the local development of a low-cost, easy-to-use, single platform, 2-color, lyse-no-wash cytometry assay, i.e. the PanLeucogate (PLG) CD4 [11–13]. This protocol incorporated CD45 FITC and CD4 PE labeled antibodies with a gating strategy based on total white cells as reference rather than using the total lymphocyte population as reference as recommended previously [14, 15]. With PLG, white cell separation is based on the differential expression of CD45 and CD4, without the need for costly additional antibodies to define T-cells (CD3) or exclude monocytes (CD14), B-lymphocytes (CD19) and natural killer cells (CD16/56) in the CD4 gating strategy. This PLG protocol was implemented onto Beckman Coulter cytometers, i.e. the XL and later the fully automated FC500 MPL/CellMek system, which through a national procurement tender, became the predicate CD4 testing platform for the National Health Laboratory Service (NHLS). By the end of 2015, 18 laboratories were using XL instruments (n = 18 systems) while 34 laboratories used MPL/CellMek (n = 66 systems) as single or multiple test platforms across 52 CD4 testing laboratories.

Ongoing development of the PLG/CD4 assay has seen the introduction of sample-by-sample quality assessment through monitoring of the flow count rate (FCR), for each sample tested [16–18]. This and other quality assessment tools developed [12, 19, 20], including an external quality assessment scheme, [21] is now an integral part of the standardized CD4 assay adhered to across the network of testing facilities (n = 49 in 2017). The PLG/CD4 assay has also been instrumental as a base for extended HIV-related biomarker assay development such as CD38 on activated CD8 T-cells [22, 23] and a Cryptococcal antigen flow assay (in development) for testing patients with a CD4 count <100cells/μl [24, 25].

Beckman Coulter (Miami, FL) developed an operator independent CD4 testing platform, the Aquios CL cytometer as a small footprint, “load and go” true volumetric platform, requiring minimal flow cytometry skills/experience. The PLG protocol for the Aquios CL cytometer was specifically developed in collaboration with the CD4 reference laboratory in Johannesburg, to comply with the specifications of the NHLS predicate PLG/CD4 protocol as well as the internal quality assessment measures developed over time; i.e. FCR monitoring with Beckman Coulter Flow Count Beads^TM for within-sample monitoring of instrument stability and CD4 enumeration accuracy (Fig 1). Additional internal quality parameters i.e. mean channel value monitoring of CD45 vs. Side Scatter expression of white cell subsets were developed specifically for the Aquios PLG version and incorporated into the quality control module to monitor instrument stability and reproducibility of cell separation on Levy-Jennings plots.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PLG/CD4 protocol on Aquios CL cytometer.

PLG/CD4 gating strategy applied for the Aquios CL cytometer platform, represented by 8 histograms (H1-8). H2 selects all CD45 positive white cells, from which all CD4 bright lymphocytes are gated (in H3) for reporting of an absolute CD4 count, while a CD4% of lymphocytes value is derived from H4. H6 and H7 indicate the bead population, currently used as an internal control measure of instrument stability.

https://doi.org/10.1371/journal.pone.0187456.g001

This paper describes the verification study conducted as part of the NHLS internal requirements for evaluation of new technology through the Health Technology Assessment Unit (HTA) in accordance with international guidelines [26]. Direct comparison of the Aquios CL instrument performance against the predicate method in the NHLS (PLG/CD4 on FC500 MPL/CellMek system) was done to assess accuracy, precision and reproducibility of this new platform.

Methods

Results

Direct comparison of samples tested on PLG/Aquios vs. MPL/CellMek PLG platform

In total, 205 samples were tested with an absolute CD4 count ranging from 2–1228 cells/μl (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of samples tested using the predicate FC500 MPL/CellMek PLG platform and comparative statistics against the Aquios PLG platform.

https://doi.org/10.1371/journal.pone.0187456.t001

Direct comparison against the predicate PLG/CD4 on FC500 MPL/CellMek platforms using the %Similarity model indicated an overall under-estimation of #CD4 by ~6% and per category of #CD4 counts. Fig 2 shows the relationship between the %Similarity and the #CD4 counts obtained with the predicate method, (i) for all samples and (ii) for samples with a CD4 count <100cells/μl. The latter shows the impact of samples with low numeric values on the overall % similarity calculation, hence the correction for comparative samples with counts <50cells/μl. These corrections did not affect the clinical outcome of patients as their CD4 counts remained <100 cells/μl.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. %Similarity and Bland-Altman analyses.

(i) The % similarity (of Aquios vs. Predicate) plotted against the predicate absolute CD4 count indicating the distribution around the acceptable limits between 90–110%. (ii) A sub-analysis of CD4<100cells/μl of %Similarity vs. predicate absolute CD4 count reveals the greatest variability seen in samples with a numeric value <10 (shaded area). One marked outlier (arrow) reveals matching comparative counts of 2 cells/μl (predicate) vs. 4 cells/μl (Aquios). This example is used to provide evidence for the need to correct counts that are less than 50 cells/ μl to 100% similar and to avoid falsely biasing %Sim analysis outcomes. Below lie the Bland-Altman analyses (n = 205) comparing CD4 test platforms for (iii) absolute CD4 counts and (iv) CD4% of lymphocytes. Differences are expressed as Aquios (test) value minus MPL (predicate) value in the Bland Altman analyses.

https://doi.org/10.1371/journal.pone.0187456.g002

An overall %Similarity for #CD4 of 94.83%±5.4 was noted with a %SimCV of 5.7%, revealing excellent precision of Aquios reporting in comparison to predicate results. Slightly higher, but acceptable variability, was noted in the #CD4 <100cells/μl category (%SimCV of 7.61%). Differences between platforms for CD4% of lymphocytes were only statistically significant in the# CD4<100cells/μl category, with overall good agreement with corresponding %SimCV values of <6% (Table 1).

Although a relatively constant difference of ~6% was evident from the %Similarity analysis, differences noted between absolute counts, as evidenced by the Bland Altman analysis (Fig 2iii and 2iv), were not clinically significant.

Misclassification rates at 100, 350 and 500 cells/μl levels revealed higher downward misclassification, confirming the under-estimation of absolute CD4 counts (calculated as number of incorrectly identified Aquios results higher than the threshold/number of reference values below the threshold x100). Upward misclassification (calculated as the number of incorrectly identified Aquios results that were lower than the threshold divided by the number of reference values above the threshold and multiplied by 100) at all thresholds were negligible at <2% (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of misclassification of absolute CD4 counts at three thresholds of importance in HIV management, indicating the number of samples misclassified and the percentage of misclassification per threshold.

https://doi.org/10.1371/journal.pone.0187456.t002

Bland-Altman analyses also confirmed the slight, though consistent under-estimation of #CD4 counts (Table 1 and Fig 2iii and 2iv) noted in the %Similarity analysis. An overall bias for #CD4 counts of -40.5cells/μl was observed, with a statistically and clinically insignificant bias for CD4% of -0.12%. Deming regression analysis of all #CD4 counts confirmed a good correlation between the testing platforms across a wide range of #CD4 counts (slope of 0.912±0.01; p,0.001 significant).

Daily quality control (DQC)

Immunotrol^TM Normal and Low were analyzed for 12 consecutive days (Table 3). Results were within package insert expected target ranges, with corresponding %CV values of <8% over the 12 days for both parameters at both Immunotrol™ levels.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary of results from manufacturer provided daily quality control material (Immunotrol^TM Normal and Low) for 12 consecutive days, where target value and range refers to package insert values.

https://doi.org/10.1371/journal.pone.0187456.t003

Reproducibility

Reproducibility testing was undertaken with internal quality assessment material (Immunotrol^TM Normal and Low) (Fig 3A and 3B) and random patient samples with low, medium and high CD4 counts (Fig 3C and 3D). Reproducibility with two Immunotrol™ Normal samples (with target values of #CD4 569±91 and 550±165 cells/μl and CD4% of 41±3.7 and 50±9% respectively) showed within-target results with corresponding %CV values of <6%. Reproducibility values with Immunotrol^TM Low were within targets of 139±33 (tested mean of 132±5.9) and 114±57 (108±7.2) for #CD4 counts and 15.3±3.3 (18.9%±0.8) and 16±7 (18.1%±1.1) for CD4% values, with corresponding CV values <8% for both parameters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Reproducibility data on Aquios.

Reproducibility data for absolute CD4 count and CD4% of lymphocytes compiled using internal quality control material, Immunotrol™ Normal (a) and Immunotrol™ Low (b). Reproducibility data of patient samples with a low, medium and high CD4 counts are represented for absolute CD4 counts (c) and CD4% of lymphocytes (d).

https://doi.org/10.1371/journal.pone.0187456.g003

Reproducibility (precision) results utilizing three patient samples per #CD4 category (low, medium and high) revealed %CV values for both parameters that were <4% at high CD4 counts (>500cells/μl) and <6% for medium counts (300–500 cells/μl), showing good reproducibility for both #CD4 counts and CD4%. The low values (<50cells/μl) had slightly elevated %CV’s (10–12%). The range of #CD4 counts with highest %CV values (of ˜11.6%) were in the range of 20–31 cells/μl (mean 27 vs. target of 38 cells/μl) and 18–26 cells/μl (mean 22 vs. target of 23 cells/μl) respectively. The corresponding CD4% values were 9.1–13.7% (with a mean of 12.1 vs. target 13.8%) and 0.6–0.8% (mean 0.7 vs. target of 0.66). The differences noted in #CD4 count and CD4% was not considered clinically significant (in that it would not change the clinical outcome or clinical intervention in patient care if detected in a clinical sample).

External quality control (EQA) performance assessment

Twenty EQA samples from 10 NHLS CD4 Proficiency Testing trials of EQA were analyzed on the Aquios instrument and an SDI calculated for each result based on the respective reported trimmed pool mean results of these trials. The EQA material included one medium/high CD4 count and one low CD4 count sample per trial. EQA panel samples tested on the Aquios platform delivered acceptable results with matched calculated SDI values for both absolute CD4 and CD4% parameters, i.e. all were shown to be within the calculated 2SD limit (Fig 4) of the reported trimmed pool means of the respective trials.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. External quality assessment results.

Radar graphs summarizing the results obtained with external quality assessment samples for absolute CD4 counts (left, including 10 trials, each with a low and a normal CD4 count on each respective trial) and CD4% of lymphocytes (right), both indicated by solid blue lines revealing the performance of the Aquios in relation to the respective 10 supplied retrospective trial pool means. The solid red lines indicate the acceptable 2 standard deviation range, while the interrupted black line represents the average overall performance (ideal) of the pooled results (at an SDI of zero).

https://doi.org/10.1371/journal.pone.0187456.g004

Discussion

At the time when HIV treatment programs were first introduced into a relatively resource-limited South Africa (2004), the provision of wide-scale CD4 services proved to be challenging [30].Tests were expensive and skilled operators required to perform flow cytometry, were not readily available [31–33]. To provide cheaper and simpler services, alternative enumeration methods were initially adopted, i.e. manual CD4 counts or use of parameters such as the total lymphocyte count or CD4 percentage of lymphocytes as predictive indicators of the impact of HIV infection on the immune system [34–40]. The unfolding HIV pandemic and growing numbers of patients requiring care however provided an impetus for the rapid evolution of new generation flow cytometric CD4 T-cell enumeration systems to meet the needs of varying service delivery/test volumes. These varied from point-of-care approaches to high volume user-independent high-end systems that are now available to laboratories previously excluded from providing flow cytometry analysis for technical or financial reasons. Affordable, simpler gating strategies developed also played an important role in meeting local skills deficiency needs and costs constraints [12, 13, 41]. Consequently, current commercially available routine CD4 flow cytometry based testing systems now cover a wide spectrum of daily throughput, including both semi- and fully automated systems for medium to high daily (>500 samples/day) test volumes [42–45]. Development of point-of-care systems provided options for very low test volume settings (<15 samples per day) typically encountered in a clinical setting where placement of flow cytometry based testing systems is not appropriate [46–51].

The Aquios instrument evaluated here is one of the new generation standardized systems designed to meet the needs of cost-constrained settings, where technical skills may be lacking and service requirements demand the use of a more cost-efficient flow cytometric approach. The verification study presented here confirmed good overall agreement (accuracy of result reporting) of the new Aquios/PLG platform to the current NHLS predicate system currently used across a network of CD4 testing laboratories at various tiers of service need. There was consistent performance of the Aquios/PLG platform over time, across a wide range of absolute CD4 counts (2-1228cells/μl and median #CD4 of 365) with negligible intra-instrument between-platform bias (BA -40.5±44.0 cells/μl bias with LOA of 26.8 to 45.8 cells/μl). Good overall agreement and tight precision to predicate were reported (%Sim 94.83±5.39cells/μl and %SimCV of 5.69%). This makes it suitable for implementation across the South African national CD4 testing service as a replacement of the XL system without impacting clinical decision making of HIV patient care. Good precision of patient sample results as well as internal quality control products, were all confirmed by %CV values of <8%. System stability and reproducibility was also confirmed. These outcomes were further consolidated in an additional workflow study (n = 1885) to assess daily workflow capacity of the Aquios/PLG platform, simultaneously testing three systems over ten days. Good outcomes were also obtained with pediatric samples and intra-instrument comparisons (S1 File). Similar results have been published with the 4-color Aquios platform, i.e. slight under-estimation of absolute lymphocyte counts [52].

Although initially introduced as a 4-color platform (Tetrachrome™) [18], the system was modified and developed specifically to accommodate PLG/CD4 testing for the South African laboratory service and other countries using the PLG/CD4 methodology and replacement of redundant BC XL cytometers currently in use in these countries. In the South African network, majority of CD4 testing is performed in clinical pathology or hematology laboratories where technicians and technologists are required to multitask and perform all CD4 testing in addition to the other routine clinical pathology or hematology services offered in these sites. In this context, the fully automated on-board sample preparation incubation and analysis of the new Aquios is anticipated to improve the workflow and consistency of quality during testing across the South African network; no manual steps are required in the preparation of samples and on-board processing enables less qualified persons to properly operate the instruments without necessarily having flow cytometry background/training or skills in flow cytometry. The barcode scanning system also ensures that the correct reagents are used within their expiry dates and data about reagent usage, linked to each test result, is collected via the laboratory interfaced management system (LIMS) for national monitoring and collation of reagent use and lot numbers. Locked-down software further provides operator-independent quality and standardized testing by preventing inadvertent or intentional deleting or changing of protocols by the operator, vital for standardized use of protocols across a networked service. Training is also easier when fixed protocols are used. On-board quality assessment monitoring of parameters provides an additional advantage; quality management parameters can be also downloaded through the LIMS interface for collation of individual instrument and laboratory performance centrally. Centralized data collation further provides for remote, coordinated proactive monitoring of the quality of testing and instrument operation at all sites, valuable for assessing downtime and other operational risks in a national service. The advantages of this system thus favor routine testing laboratories where only one or two test sets are commonly used (PLG/CD4 and/or Tetrachrome 4-color). The closed system however is restrictive for research purposes where operators need to be able to freely create/develop new test protocols.

Workflow analysis indicated a daily 8-hour throughput of around 120 samples. The autoloader can take up to eight cassettes with five samples each, with the first result ready after approximately 20 minutes and thereafter every 2 minutes. Although this capacity is slightly less than the XL cytometers platform, it is foreseen that laboratories will be able to cope with their daily testing demands by changing their workflow. This will save on hands-on time, allowing laboratory staff to multitask/rotate in the context of clinical pathology services (including microbiology, chemistry and hematology) and consolidating staff time to enable the rollout of related services in the same testing facilities. The provision of reflexed screening of early opportunistic HIV co-infections such as Cryptococcal meningitis [53, 54] with Cryptococcal antigen (CrAg) testing of all CD4 samples with counts less than 100 cells/μl is one such example. For the implementation of this national testing program no additional staff were employed in any of the CD4 laboratories, i.e. fully integrated into routine CD4 testing [55, 56]. Time saving results were recently reported in a comparative study between the 4-color Aquios and the Beckton Dickinson FACSCanto II that assessed the impact of the new platform on laboratory turn-around-time [57]. With the Aquios capacity of 120 samples per day and operator-independent features reported here, the system is ideally suited for Tier-3 and lower-end level laboratories [58] with <200 test samples per day (2 Aquios systems). Furthermore, it can provide a means to extend laboratory services to non-CD4 testing laboratories where a need for better service provision is indicated [12, 58, 59]. In the South African context, we anticipate the biggest impact of the Aquios to be on both quality and service delivery (improved turn-around time) and increased decentralized service in smaller laboratories previously utilizing existing staff. Lastly, the auto-gating feature worked well, with <2% of samples needing operator-initiated re-gating, making standardization of result reporting in a network of testing facilities easier.

Data presented here indicated an overall consistent, slight under-estimation (~6%) of the absolute CD4 count compared to the MPL/CellMek PLG predicate method/platform, and excellent agreement of CD4 percentage of lymphocyte values. Similar results of slight under-estimation of absolute lymphocyte counts were reported in the validation of the 4-color Aquios system against the Beckman Coulter FC500 [52]. The under-estimation of absolute CD4 counts generated on true volumetric systems versus counts vs. single platform bead-based systems has been previously reported [60], including systems such as the Sysmex Cyflow (previously Partec) [61]. In a national network, notice to attending clinicians concerning a precise slight downward bias can provide for a once-off adjustment [11] for a patient’s CD4 count if the difference to the predicate result is consistent and precise as demonstrated on the Aquios (if the CD4 count was used to monitor response to treatment). Misclassification around pre-determined treatment threshold cut-offs are only significant if applied to patient care. These include (i) reflexed CrAg testing at a CD4<100cells/μl, (ii) ART initiation where either the 200, 350 or 500 cells/μl treatment thresholds are still followed, or (iii) fast tracking of patients onto therapy with low CD4 baseline counts, i.e. in South Africa, a CD4<200cells/μl is applied [62, 63]. A system that under-estimates CD4 counts would qualify more patients for treatment. However, the use of misclassification analyses in the prevailing context of the international trend towards doing away with the need for CD4 counts to define eligibility for antiretroviral treatment (ART) [9, 15, 28, 29, 64–66], are irrelevant if a universal test and treat (UTT) approach is applied. The only relevant misclassification level, if the WHO guideline [62] to screen for cryptococcal disease or other opportunistic infection is applied, is thus at the CD4 ≤100cells/μl level. This study indicated that the misclassification of patients at the 100cells/μl using the Aquios/PLG platform was acceptable at very low levels i.e. a 2.1% downward (in favor of screening) and 2.5% upward misclassification where patients would miss the opportunity for CrAg screening.

Lastly, in view of the recent WHO guideline recommendations and an international growing trend to apply UTT for all HIV+ patients, it is important to re-assess the future role of CD4 enumeration in the management of HIV+ patients accessing antiretroviral treatment (ART). Although the 2016 WHO guidelines support UTT [62] and do away with CD4 counts as an indicator of eligibility to start ART, a (miss)perception has arisen that CD4 counts are no longer integral to care of HIV+ patients accessing ART [54, 63, 67]. This underestimates the role of CD4 in determining the level of immunosuppression, i.e. how sick a patient is, especially those who present late for care (with very low CD4 counts). This is especially true in the context of countries with a very high patient burden with advanced HIV disease like South Africa [7]. It is thus important to emphasize that CD4 enumeration remains an important parameter [54, 63, 67] to assess immune status of patients at baseline before ART initiation, and/or for identifying risk and/or screening for opportunistic co-infections such as Cryptococcal disease [53, 54].

Conclusion

The Aquios/PLG CD4 testing platform showed clinically acceptable result reporting, with good system stability and reproducibility. In the context of smaller clinical laboratories providing clinical pathology services in a tiered service delivery model and especially, lower-end tiers testing 120 or less CD4 samples per day, the system provides a pragmatic, practical and standardized solution for CD4 service implementation across a national network. The ease of use and accessibility of the small foot-print system enabled training of all levels of staff/ CD4 personnel with no prior flow Cytometry or CD4 testing background as the system is operator-independent, requires less hands-on time to enable multitasking in a clinical pathology setting, freeing staff to do additional laboratory tasks, as well as providing a system with on-board quality control with remote centralized monitoring to enable proactive system maintenance and large-scale continuous service delivery.

Disclaimer

The views expressed here are those of the authors and not necessarily that of their sponsor and employer, the South African National Health Laboratory Service or the University of the Witwatersrand, Faculty of Health Sciences.

Supporting information

Acknowledgments

The authors acknowledge Mrs. Moodley who performed all Aquios testing while she was still employed through the WITS Health Consortium, Johannesburg, South Africa (currently employed by Beckman Coulter SA), Dr Denise Lawrie and the staff of the NHLS Charlotte Maxeke Johannesburg Academic Hospital CD4 laboratory for assisting with collation of routine samples and the NHLS Health Technologies Unit (HTA).

The authors further acknowledge the support of the National Health Laboratory Service, the National Priority Program including Professor W. Stevens and Dr P da Silva, as well as the CD4 training staff, Mrs. S. Mojalefa and Mrs. G Mokone.

DKG acknowledges the South African National Research Foundation for funding for rated researchers.