Figures
Abstract
Introduction
In Ethiopia, CD4+ T-cell counting is still required for all patients at baseline before antiretroviral therapy (ART) and to determine eligibility and follow-up of opportunistic infection prophylaxis. However, access to CD4+ T cell count in rural health facilities remains a major challenge in Ethiopia like other resource-limited settings.
Methodology
Both capillary and venous blood was drawn from each of 325 study participant recruited in Addis Ababa and surroundings. The CD4+ T-cell count, CD4%, and hemoglobin (Hgb) were tested at one of the four study health facilities using capillary blood and BD FACSPresto™ device. These tests were also done at the national HIV reference laboratory, using venous blood with BD FACSCalibur™, Sysmex XT-1800i™, and BD FACSPresto™.
Results
BD FACSPresto™ had an absolute mean bias of -13.3 cells/ul (-2.99%) and 28.3 cells/μl (6.4%) using venous and capillary blood, respectively, compared with BD FACSCalibur™. The absolute CD4 assay on the BD FACSPresto™ had a regression coefficient (R2) of 0.87 and 0.92 using capillary blood and venous blood samples, respectively, compared with BD FACSCalibur™. The percentage similarity of the BD FACSPresto™ using capillary and venous blood was 105.2% and 99.3%, respectively. The sensitivity of the FACSPresto™ using threshold of 500 cells/μl for ART eligibility using capillary and venous blood was 87.9 and 94.3%, while the specificity was 91.4 and 83.8%, respectively. Furthermore, the BD FACSPresto™ had an absolute mean bias of -0.2 dl/μl (0.0%) (95% LOA: -1.7, 1.3) and -0.59 dl/μl (0.1%) (95% LOA: -1.49, 0.31) for Hgb using capillary and venous blood compared with the Sysmex XT-1800i™, respectively.
Citation: Gebremicael G, Belay Y, Girma F, Abreha Y, Gebreegziabxier A, Tesfaye S, et al. (2017) The performance of BD FACSPresto™ for CD4 T-cell count, CD4% and hemoglobin concentration test in Ethiopia. PLoS ONE 12(4): e0176323. https://doi.org/10.1371/journal.pone.0176323
Editor: Lindi-Marie Coetzee, National Health Laboratory Service, SOUTH AFRICA
Received: August 23, 2016; Accepted: April 7, 2017; Published: April 27, 2017
Copyright: © 2017 Gebremicael et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data is available in S1 Data.
Funding: This study was funded by UNITAID. The authors affiliated with the Clinton Health Access Initiative, Inc., an affiliated entity of the Clinton Foundation, did not fund this specific study. UNITAID had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funder provided support in the form of items, car rent, and supplies, etc. and salaries for the field staff at the site and driver staffs but not for the authors.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In Ethiopia, many HIV positive patients and those co-infected with tuberculosis, sexually transmitted infections, and other opportunistic infections currently do not have reliable access to diagnostic laboratory tests. CD4 T-cell counting is the most predictable indicator of HIV disease progression, is positively linked to long-term survival rates, and is more reliable than symptomatic staging [1,2,3]. According to the Ethiopian ART national guidelines which have been revised with a supplement after the country’s adoption of WHO’s “Test and treat”, CD4 testing is required for all patients at baseline and to determine eligibility and follow up of opportunistic infection prophylaxis as well as for monitoring immunologic failure where viral load monitoring is not accessible/possible [4].
Most of CD4+ T-cell and Hgb concentration testing are primarily available at centralized laboratories in major urban centers, which are not sufficient to provide this necessary test for all patients. Patients who do not have access to reliable CD4+ T-cell testing often devote a full day of travel for each health facility visit, including sample collection and result delivery, with the loss of travel costs and man-hours. In addition, long turn-around times for tests sent away to central laboratories can delay clinical decisions and put considerable burden on patients. Additionally, blood samples generally have a short period of sample stability, thus adding further complexity to a challenged network. Finally, instrument break-downs and limited capacity at central laboratories often put a strain on the number of tests that can be run, leading to testing backlogs [5].
To overcome problems associated with centralized testing, it is possible to build new laboratories or upgrade existing laboratories. However, the cost and time required to carry this out is substantial and could significantly delay the current expansion of treatment services to those most in need. It may also be difficult to find adequately trained technical staff to operate traditional laboratory instruments in many rural areas [5, 6, 7].
There are growing numbers of high quality point of care (POC) diagnostic technologies available and there is increasing interest in using these technologies to alleviate critical testing needs without building sophisticated laboratories [8]. Pima™ POC CD4 (Alere) is the first POC technology that has been in use in Ethiopia since 2013. BD FACSPresto™ (BD Biosciences) CD4 POC technology which has prequalification by the World Health Organization (WHO) [9] might be used an alternative choice for CD4+ T-cell, CD4 percentage, and Hgb concentration testing in Ethiopia. Providing greater access to CD4 testing using POC technologies, such as the BD FACSPresto™, may be essential to maintain and increase ART coverage, early ART initiation, and reduce pre-ART patient loss to follow up [10, 11]. The aim of this study is to report on the outcomes of the performance of the BD FACSPresto™ analyser in the health facilities setting compared to the gold standard BD FACSCalibur and Sysmex XT-1800i™.
Methods
Study design and study population
Both venous and capillary blood samples were collected from each of 325 patients enrolled at four health centers (Burayu HC, Sendafa HC, Kirkos HC and Meshualekia HC) in Addis Ababa and Finfine zuria, between June 16 and July 10, 2015. The study participants were of both sexes and all adults (18 to 65 years of age). The CD4+ T-cell count, CD4%, and hemoglobin (Hgb) were tested at one of the four study health facilities using capillary blood and BD FACSPresto™ (Becton Dickinson, San Jose California, USA) by laboratory technicians. Venous blood samples were collected and stored at room temperature and transported to the Ethiopian Public Health Institute (EPHI), the national HIV reference laboratory, within eight hours of sample collection. According to the manufacturer, 48 hours of draw is considered as the stability time of the anti-coagulated blood stored at room temperature (20°C–25°C); while the stained sample with monoclonal antibodies should be analyzed within 24 hours [12]. At EPHI, the above tests were done using venous blood with the BD FACSPresto™, BD FACSCalibur™ (Becton Dickinson, San Jose California, USA) and for Hgb with Sysmex XT-1800i™ (Sysmex Corporation, Sysmex America, Inc USA) by laboratory technicians.
Study procedure
All operators of the BD FACSPresto™ were trained on finger stick and venous blood collection and on operating BD FACSPresto™ for three days before the study was started.
After patient consent, testing was performed using capillary blood samples tested on the BD FACSPresto at the study site according to the manufacturer’s instructions [13]. An additional blood sample was taken by venipuncture and transported to EPHI reference laboratory for further testing using the BD FACSPresto™, the BD FACSCalibur™ and Sysmex XT-1800i™, according to manufacturers’ instructions [13, 14, 15]. The BD FACSCalibur™ and Sysmex XT-1800i™ were used as reference for CD4 T-cell counting both absolute and % values, and Hgb concentration respectively. Operators of the BD FACSCalibur™ and Sysmex XT-1800i™ were previously trained on operating FACSCalibur™ and full automated Sysmex XT-1800i™; and they had seven years of operating experience. CD4 T-cell counting was done using single platform BD Trucount™ tube with a calibrated number of fluorescent beads and monoclonal antibodies (MultiTest CD3/CD8/CD45/CD4 which contains FITC-labeled CD3, PE-labeled CD8, PerCP-labeled CD45 and APC-labeled CD4).
To prevent any potential bias, operators were blinded of the test results from the BD FACSPresto™ at the health center and tests performed at EPHI. Different study staff was responsible for the BD FACSPresto™, the BD FACSCalibur™, and Sysmex XT-1800i™ testing at EPHI and separate data entry forms were completed for the different testing platforms. Patient data and test results from the study sites were collected and transcribed in an Excel database manually once a week using double entry by two data clerks at the reference laboratory to minimize possible source of error.
The BD FACSPresto™ was also evaluated in terms of operational characteristics. Seven operators of the BD FACSPresto™ at the facilities and EPHI were selected randomly from 16 trained operators. The questionnaire included questions on the ease of use, additional supplies/equipment required, training need, time taken to perform the test compared to manufacturer’s claim and clarity of the instructions.
Quality assurance
Prior to the start of the study, manufacturer engineers serviced and validated the BD FACSCalibur™, the BD FACSPresto™ and Sysmex XT-1800i™ devices. Performance of the BD FACSCalibur™ reference method was monitored daily using BD calibration beads (FITC, PE, PerCP and APC bead fluorescent color) to adjust instrument settings including to set instrument fluorescence and test sensitivity. Prior to sample analysis each day, single platform Trucount™ bead controls were used to monitor the number of fluorescent beads in the BD Trucount™ tube and BD Multi-check process controls were run to control the whole process starting from sample pipetting to sample acquisition. Sysmex XT-1800i™ was also calibrated and quality control samples were run prior to sample testing each day. In addition, the BD FACSCalibur™ instrument and Sysmex XT-1800i™ at the reference laboratory were enrolled in an external quality assessment (EQA) Proficiency test (PT) program, Quality Assessment Program for Standardization and immunological Measures Relevant to HIV/AIDS (QASI). The instruments passed the PT assessment during the study and during the preceding year of EQA PT panels. The BD FACSPresto™ instrument quality check was run automatically each day when the instrument was switch on. Furthermore, study site supervision was conducted weekly during the study.
Ethical consideration
The study protocol was approved by the Scientific and Ethical Research Office (SERO) at the Ethiopian Public Health Institute, (Project number: SERO-016-5-2015) and all participants provided written informed consent before enrollment in the study. Only patients who provided informed consent were included in the study. For patients not willing to participate in the study, normal testing and clinical care were provided per national guidelines without negative consequences. The CD4+ test results obtained using the BD FACSCalibur™ and Hgb test results using Sysmex XT-1800i™ were sent back to the respective health center for routine care and treatment of the patient. BD FACSPresto™ test results were not given back to the patients, but remained confidential. All study materials, including hard copy results, consent and data collection forms, were kept in a locked cabinet.
Statistical data analysis
The data were analyzed using Stata Version 11 (StataCorp LP, College Station, TX, USA), Microsoft Excel (Microsoft Corporation, Redmond, WA, USA), and GraphPad Prism (GraphPad Software Inc, La Jolla, CA, USA). The performance of the BD FACSPresto™ compared to the BD FACSCalibur™ and/or Sysmex XT-1800i™ was determined using statistical tests including median, range, standard deviation and coefficient of variation. We calculated the coefficient of variation (CV) for instrument precision, intra-assay, and inter-instrument variations for CD4+ T-cell and CD4% counting.
The regression coefficient (r2) between the BD FACSPresto™, the BD FACSCalibur™ and/or Sysmex XT-1800i™was analyzing using a passing and Bablok regression. The Bland-Altman (difference plot) analysis was used to determine the mean bias and 95% limit of agreement (LOA = Mean bias ±1.96 SD) [16]. For this, the difference between data pairs was graphically represented on the Y-axis against their mean on the X-axis [17]. Percentage similarity was also calculated (the average of the BD FACSCalibur™ or Sysmex XT-1800i™ and the BD FACSPresto™ divided by the BD FACSCalibur™ or Sysmex XT-1800i™ multiplied by 100) [18].
Misclassification probabilities [7] were determined at CD4+ T-cell thresholds of 100 cell/μl used for cryptococcal pneumonia reflex testing, 350 cells/μl the previously recommended ART initiation threshold [19] and 500 cells/μl, the 2013 WHO recommended ART initiation threshold [20] to determine the percentage of patients correctly classified using the BD FACSPresto™ result compared with the BD FACSCalibur™ classification. The performance of the BD FACSPresto™ was also assessed using sensitivity, specificity and predictive (positive and negative) values using the clinically relevant CD4 thresholds and 2x2 tables [17].
The venous blood samples were also used to determine the precision of the BD FACSPresto™ for CD4+ T-cell, CD4%, and Hemoglobin concentration. To measure the intra-assay variability (cartridge to cartridge variation including procedural errors) of the BD FACSPresto™, 12 samples were selected randomly and tested ten times each using ten new cartridges on the same BD FACSPresto™ device. Instrument precision (run-to-run variation) was determined using 15 venous blood samples. The run-to-run variation was determined using the same cartridge and the same device run ten times for a total of 150 tests. Finally, we assessed the inter-instrument variability (instrument-to-instrument variation) using eight venous samples run ten times on two different FACSPresto™ devices with the same cartridge.
Results
Demographic characteristics of study participants
A total of 325 participants (65 from Burayu HC, 76 from Sendafa HC, 96 from Kirkos HC and 88 from Meshualekia HC) were enrolled in the study. From the 325 samples collected, ten samples were excluded from accuracy and precision analysis: due to sample quality (two); due to invalid results by the BD FACSPresto™ using capillary blood (three); due to invalid results by the BD FACSPresto™ using venous blood (six). The demographic characteristic of study participants is shown in Table 1. The median age was 37 years old and 69% were female. Data were similar if disaggregated by gender (data not shown).
Accuracy assessment
CD4+ T-cell comparison between the BD FACSPresto™ (Capillary) and BD FACSCalibur™.
The median CD4+ T-cell count using the BD FACSPresto™ with capillary (finger prick) blood and the BD FACSCalibur™ using venous blood was described in Table 2. The BD FACSPresto™ with capillary blood had an absolute mean bias of 28.3 cells/μl (6.4%) (95% LOA: -157.1, 213.7) when compared to the BD FACSCalibur™ (Table 2, Fig 1A). The BD FACSPresto™ using capillary blood and CD4 category shown in Table 1 had an absolute mean bias of 10.3, 13.6, 3.3 and 2.4 cells/μl respectively, compared to the BD FACSCalibur™ (S1 Table & S1A, S1B, S1C and S1D Fig). The coefficient of determination (R2) between the BD FACSPresto™ with capillary blood and the BD FACSCalibur™ was 0.87 (Fig 2A). The mean percentage similarity and coefficient of variance (%CV) between the BD FACSPresto™ using capillary blood and the BD FACSCalibur™ was 105.2% and 15.1% respectively.
Bland-Altman comparisons between the BD FACSPresto™ with capillary blood (a-c) and venous blood (d-f) samples with the BD FACSCalibur™ reference standard. The corresponding graphs show the absolute bias between the FACSPresto™ and FACSCalibur represented in the Bland-Altman plots for CD4 T-cell testing with capillary blood (a) and venous blood (d); Bland-Altman plots for CD4% testing with capillary blood (b), venous blood (e) samples; Bland-Altman plots for Hgb testing with capillary blood (c), venous blood (f)., The solid green lines represent the mean bias and the solid deep red lines represent the upper and lower limits of agreement (LOA = mean ± 1.96SD).
Passing and Bablok regression between the BD FACSPresto™ with capillary blood (a-c) and venous blood (d-f) compared to the BD FACSCalibur™. Comparison of the BD FACSPresto™ with BD FACSCalibur™ for CD4+ T-cell count with capillary blood (a) and with venous blood samples (d). Comparison of the BD FACSPresto™ with BD FACSCalibur™ for CD4% with capillary blood (b) and with venous blood samples (e). Comparison of the BD FACSPresto™ with Sysmex XT-1800i™with capillary blood (c) and with venous blood (f).
Using the threshold of 100 cells/μl for cryptococcal pneumonia reflex testing, the sensitivity and specificity of the BD FACSPresto™ using capillary blood were 83.3and 99.7%, respectively, compared to the BD FACSCalibur™. The total misclassification rate of the BD FACSPresto™ with capillary blood at this threshold was 0.9% (Table 3).
Using the Previous WHO 2010 ART initiation threshold of 350 cells/μl, the sensitivity and specificity of the BD FACSPresto™ with capillary blood were 82.9 and 92.9% respectively, compared to the BD FACSCalibur™. The total misclassification rate of the BD FACSPresto™ with capillary blood at this threshold was 10.7% (Table 3).
Using the WHO 2013 ART eligibility threshold of 500 cells/μl, the sensitivity and specificity of the BD FACSPresto™ with capillary blood were 87.9 and 92.9% respectively, compared to the BD FACSCalibur™. The total misclassification rate of the BD FACSPresto™ with capillary blood at this threshold was 10.9% (Table 3).
CD4% comparison between the BD FACSPresto™ (Capillary) and BD FACSCalibur™.
The BD FACSPresto™ with capillary blood had an absolute mean bias of -0.3% (-0.1%) (95% LOA: -4.5, 3.9) compared to the BD FACSCalibur™ (Table 4, Fig 1B). The BD FACSPresto™ with capillary samples had an R2 of 0.95 compared to the BD FACSCalibur™ (Fig 2B).
Hemoglobin (Hgb) comparison between the BD FACSPresto™ (Capillary) and Sysmex XT-1800i™.
The absolute mean bias of Hgb concentration using the BD FACSPresto™ with capillary blood was -0.2 dl/μl (95% LOA: -1.7, 1.3) compared to the Sysmex XT-1800i™ (Table 5 & Fig 1C). Hgb concentration testing using the BD FACSPresto™ with capillary samples had an R2 of 0.84 compared to the Sysmex XT-1800i™ (Fig 2C). The mean percentage similarity and %CV between the BD FACSPresto™ with capillary blood and the Sysmex XT-1800i™ was 94.4% and 2.7% respectively (Table 5).
CD4+ T-cell comparison between the BD FACSPresto™ (venous blood) and BD FACSCalibur™.
The BD FACSPresto™ with venous blood had an absolute mean bias of -13.3 cells/ul (-2.99%) (95% LOA: -163.2, 136.6) compared to the BD FACSCalibur™ (Table 2 & Fig 1D). The BD FACSPresto™ using capillary blood at CD4 category (Table 1) had an absolute mean bias of -17.8, 0.3, 3.6 and -5.7 cells/μl respectively, compared to the BD FACSCalibur™ (S1 Table & S1F, S1G and S1H Fig). The BD FACSPresto™ with venous blood had an R2 of 0.90 compared to the BD FACSCalibur™ (Fig 2D). The mean percentage similarity and %CV between the BD FACSPresto™ with venous blood and the BD FACSCalibur™ were 99.3% and 10.7%, respectively.
The sensitivity and specificity at the 100 cells/μl threshold of the BD FACSPresto™ with venous blood were 100 and 99.4% respectively, compared to the BD FACSCalibur™. The total misclassification rate of the BD FACSPresto™ with venous blood at this threshold was 0.63% (Table 6).
Using the previous WHO ART eligibility threshold of 350 cells/μl, the sensitivity and specificity of the BD FACSPresto™ with venous blood were 91.6 and 91.9% respectively, compared to the BD FACSCalibur™. The total, upward and downward misclassification rates of the BD FACSPresto™ with venous blood at this threshold were 8.2, 8.4 and 8.1% respectively (Table 6).
Using the 2013 WHO ART eligibility threshold of 500 cells/μl, the sensitivity and specificity of the BD FACSPresto™ with venous blood were 94.3 and 83.8% respectively, compared to the BD FACSCalibur™. The total, upward and downward misclassification rates of the BD FACSPresto™ with venous blood at this threshold were 9.2, 5.7% and 16.2%, respectively (Table 6).
CD4% comparison between the BD FACSPresto™ (venous blood) and BD FACSCalibur™.
The BD FACSPresto™ with venous blood had an absolute mean bias of 0.8 (0.2%) (95% LOA: -1.8, 3.4) (Fig 1E) compared to the BD FACSCalibur™. The BD FACSPresto™ with venous blood had an R2 of 0.98 compared to the BD FACSCalibur™ (Fig 2E). The mean percentage similarity and %CV between the BD FACSPresto™ with venous blood and the BD FACSCalibur™ were 98% and 1.6%, respectively (Table 4).
Hemoglobin (Hgb) comparison between the BD FACSPresto™ (venous blood) and Sysmex XT-1800i™.
For Hgb concentrations testing, the BD FACSPresto™ with venous blood had an absolute mean bias of -0.59 dl/μl (-0.1%) (95% LOA: -1.49, 0.31) (Fig 1F) compared to the Sysmex XT-1800i™. The BD FACSPresto™ with venous blood had an R2 of 0.94 compared to the Sysmex XT-1800i™ (Fig 2F). The mean percentage similarity and the %CV between the BD FACSPresto™ with venous blood and Sysmex XT-1800i™were 98% and 1.6%, respectively (Table 5).
CD4+ T-cell comparison between the BD FACSPresto™ (capillary) and BD FACSPresto™ (venous).
The CD4+ T-cell count of BD FACSPresto with capillary blood had an absolute mean bias of 41.7 cells/μl (9.5%) (95% LOA: -94.1, 177.5) compared to BD FACSPresto™ with venous blood (Table 2). The BD FACSPresto™ had an R2 of 0.90 with capillary blood compared to the BD FACSPresto™ with venous blood. The mean percentage similarity and %CV between the BD FACSPresto™ using capillary and venous blood were 106.2% and 11.3%, respectively (Table 2).
CD4% count comparison between the BD FACSPresto™, capillary and BD FACSPresto™, venous.
The absolute mean bias of the CD4% using the BD FACSPresto™ comparing capillary and venous blood was -1.1(0.3%) (95% LOA: -3.0, 5.3). The BD FACSPresto™ with capillary blood had an R2 of 0.95 compared to the BD FACSPresto™ with venous blood (Table 4).
Hgb concentration comparison between the BD FACSPresto™ (capillary) and BD FACSPresto™ (venous).
The mean bias of the Hgb concentration using the BD FACSPresto™ with capillary blood was -0.1dl/μl (0.1%) (95% LOA: -1.8, 1.0) compared to the BD FACSPresto™ with venous blood. The BD FACSPresto™ with capillary blood had an R2 of 0.84 compared to the BD FACSPresto™ (Table 5).
Precision assessment of the BD FACSPresto™.
Intra-assay variability was determined by repeating 12 different samples with CD4+ T-cell counts between 100–500 cells/μl (Table 7). The mean %CV of intra-assay of absolute CD4+ T-cell, CD4% and Hgb concentration was 6.2, 5.2, and 2.2% respectively. We also measured the precision of CD4+ T-cell, CD4% and Hgb concentration results using the same sample run on two different BD FACSPresto™ devices to evaluate the inter-instrument variation. The mean %CV of the CD4+ T-cell count, CD4% and Hgb concentration was 6.5, 5.7 and 1.7% respectively.
The run-to-run instrument precision using the BD FACSPresto™ for CD4+ T-cell count, CD4% count and Hgb concentration had a mean %CV of 4.2, 3.5 and 1.7% respectively.
Error rates of FACSPresto™.
A total of three (0.9%) samples out of the 325 samples produced an error when tested on the BD FACSPresto™ using capillary blood in the health facility. One of the three samples produced a valid result after repeat testing with the same cartridge. Six samples out of the total 323 (1.9%) tested on the BD FACSPresto™ using venous blood at the reference laboratory produced an error. Three of the six samples produced a valid result after repeat testing with a new cartridge using the same venous sample (Table 8).
Operational characteristics assessment.
The operational characteristics of the BD FACSPresto™ were assessed by interviewing seven laboratory technicians who operated the BD FACSPresto™ both at the health centers and the reference laboratory using a standard questionnaire. All of the operators responded that the BD FACSPresto™ was very simple to use and instrument instructions were concise and clear. Six of seven (86%) laboratory technicians responded that one day was the minimum number of days required for training to operate the BD FACSPresto™, where as one operator responded that three days of training was preferred.
Discussion
This independent technical accuracy study was conducted to assess the performance of the BD FACSPresto™ technology to accurately quantify CD4+ T cells, CD4% count, and Hgb concentration using capillary and venous blood from adult HIV patients. The performance of the BD FACSPresto™ to enumerate CD4+ T cells and CD4% was comparable to the laboratory-based BD FACSCalibur™, while the performance of the BD FACSPresto™ to quantify Hgb concentration was comparable to the laboratory-based Sysmex XT-1800i™. The BD FACSPresto™ had a sensitivity of over 90% and ≤90% to detect patients below the three thresholds measured using venous and capillary blood respectively: 100 cells/μl used for reflex Cryptococcal testing, 350 cells/μl previous ART initiation and 500 cells/μl ART failure thresholds. The precision of the BD FACSPresto™ was strong with all precision measurements having a coefficient of variation of less than 7%. Finally, the test error rates were very low, both in the health centers and in the reference laboratory. It is also easy to operate FACSPresto™ for testing.
Though the performance of the BD FACSPresto™ was comparable to the laboratory assay using venous samples, some over-quantification by the BD FACSPresto™ was observed when capillary blood samples were tested. Over 10% of patients were incorrectly classified as above each of the included CD4+ T cell thresholds, suggesting that they would not have received the required reflex cryptococcal testing. These results are consistent with those previously observed [21]. Upward misclassifying a proportion of patients would result in lower costs of testing and treatment; however, costs may not be lower in a global perspective (due to complications, not only a lack of well-being), so "direct costs" should rather be used. National programs, however, may be more willing to perform additional cryptococcal tests and initiate patients just above the ART eligibility thresholds for improved patient care and outcomes. In that setting, a technology that is more comparable to the laboratory assay or tends towards under-quantification and downward misclassification may be more appropriate for reducing patient morbidity and mortality.
While the BD FACSPresto™ performed more comparably using venous samples with reduced upward misclassification rates, some point-of-care settings require capillary blood samples when phlebotomist and pipetting skills are lacking. Being able to utilize both sample types will provide significantly more flexibility in product placement considerations and allow for further decentralization where necessary. Countries may want to consider these results as they plan strategic deployment.
Ethiopia has an extensive laboratory network, including conventional laboratory-based CD4+ T cell testing. Unfortunately, this laboratory network does not reach all patients in need of CD4+ T cell testing with many lacking on-site testing. Like PimaTM, Point-of-care technologies, such as the BD FACSPresto™, could support increased access to on-site CD4+ T cell testing, particularly in rural and hard to reach settings. Thus, alternative deployment strategies are being considered in Ethiopia and other countries. In Ethiopia, BD FACSCount technologies have been in use for many years; however, device breakdowns have rendered several inoperable. Given its high throughput, rather than procuring new BD FACSCount technologies, replacing those not functioning with BD FACSPresto™ technologies is being considered to be more beneficial. Compared to the BD FACSCount, the BD FACSPresto™ does not require laboratory or pipetting skills, refrigeration of reagents and controls, consistent electricity, or additional procurement of controls and yet has a similar test turnaround time.
The 2016 WHO ART Consolidated Guidelines recommend treating all patients with ART regardless of the CD4+ T cell count [22]. Implementing this recommendation could be impactful for countries in improving patient lives, reducing transmission, and achieving the UNAIDS’ 90-90-90 targets [23, 24]. High HIV burden countries in sub-Saharan Africa are left with difficult decisions of when and how to implement the many new recommendations in resource-limited settings. In Ethiopia, although, the guideline has been adopted, financial constraints persist that will not allow immediate uptake of treat all. In the interim, CD4+ T cell counting will remain a critical test to prioritize patients in most need of ART. Furthermore, CD4+ T cell testing is an important diagnostic in supporting clinical opportunistic infection monitoring of all HIV-positive patients.
In accordance with the 2013 WHO ART Consolidated Guidelines, Ethiopia initiates infants under five years of age on ART regardless of CD4%. While this test is no longer a barrier to ART, clinicians still find it useful in understanding the immunological status of infants prior to initiation and monitoring. Furthermore, Hgb testing remains critical for patients initiating ART using zidovudine (AZT) as the drug can cause hematological toxicity, particularly in patients with low body weight, low CD4+ T cell count, and/or advanced disease. Unfortunately, access to Hgb testing in Ethiopia is minimal; therefore, this multiplex BD FACSPresto™ technology would allow for increased access to CD4+ T cell and Hgb testing for adults as well as CD4% testing for infants.
Several limitations exist in this study. First, diagnostic testing is primarily performed by trained laboratory staff in Ethiopia. While this is consistent across the country, point-of-care technologies are often operated by non-laboratory staff in other countries. Understanding the performance of these technologies when used by non-laboratory staff would be very helpful in countries considering task-shifting. Additionally, the confidence intervals for some of the clinical comparisons (sensitivity, specificity, and misclassification) were quite wide. This was primarily due to the lower performance of the BD FACSPresto™ to correctly classify patients above or below the thresholds analyzed. Patient sample sizes were sufficiently calculated based on higher performance expectations. Though this technical evaluation took place in Addis Ababa and the surrounding area, it is not expected that the performance of the BD FACSPresto™ technology would vary if placed in a rural setting and if used for measuring CD4% in infants. Although Pima™ has been deployed for about three years in Ethiopia; this study did not compare BD FACSPresto™’s performance with this POC CD4. However, the study by Bwana et al has demonstrated their similarity [21].
In conclusion, this study highlighted comparable performance of the BD FACSPresto™ to the BD FACSCalibur™ for CD4+ T cell and CD4% enumeration and the Sysmex XT-1800i™ for Hgb counting. The BD FACSPresto™, in combination with rapid HIV diagnostic tests, provides an opportunity to rapidly diagnose and assess ART eligibility in a single visit. Furthermore, patients with more advanced disease can be prioritized for immediate follow-up and rapid ART initiation. Thus, the BD FACSPresto™ is a potential addition to the CD4 testing network to both assess ART eligibility and support opportunistic infection monitoring.
Supporting information
S1 Fig. Bland-Altman comparisons between the BD FACSPresto™ with capillary blood (a-d) and venous blood (e-h) samples with the BD FACSCalibur™ reference standard.
The corresponding graphs show the absolute bias between the FACSPresto™ and FACSCalibur at CD4≤100 cells/μl represented in the Bland-Altman plots for CD4 T-cell testing with capillary blood (a) and venous blood (e); Bland-Altman plots for CD4 absolute between 100 and 350 cells/μl category with capillary blood (b), venous blood (f) samples; Bland-Altman plots for CD4 absolute between 350 and 500 cells/μl category testing with capillary blood (c), venous blood (g); Bland-Altman plots for CD4>500 cells/μl category testing with capillary blood (d), venous blood (h). The solid green lines represent the mean bias and the solid deep red lines represent the upper and lower limits of agreement (LOA = mean ± 1.96SD).
https://doi.org/10.1371/journal.pone.0176323.s001
(TIF)
S1 Table. BD FACSPresto™ comparing with BD FACSCalibur™ based on the CD4+ T cell category.
Absolute mean bias was compared based on at CD4≤100 cells/μl, between 100 and 350 cells/μl, between 350 and 500 cells/μl and CD4>500 cells/μl category testing.
https://doi.org/10.1371/journal.pone.0176323.s002
(DOC)
S1 Data. The demographic characteristic, CD4 T-cell, CD4% and Hgb values using BD FACSPresto™ and BD FACSCalibur™ done at national laboratory and site for each study participants which coded by patient study ID.
Error result given by FACSPresto at site and national laboratory also included.
https://doi.org/10.1371/journal.pone.0176323.s003
(XLS)
Acknowledgments
The authors acknowledge the study participants and the study sites (Health Centers) for the committed contributions for this evaluation study.
Author Contributions
- Conceptualization: GG YB BB AG DK LV.
- Data curation: GG ZM ST LV.
- Formal analysis: GG LV.
- Investigation: GG YB Y. Abreha FG ST AG ZM BB Y. Assefa DK LV.
- Methodology: GG YB ZM LV DK.
- Project administration: GG YB AG DK Y. Assefa.
- Software: GG ZM ST LV.
- Supervision: LV DK Y. Assefa BB.
- Writing – original draft: GG, LV.
- Writing – review & editing: GG, BB, DK, ZM, YAssefa, LV.
References
- 1.
GIORGI J V. Characterization of T Lymphocyte Subset Alterations by Flow Cytometry in HIV Disease. Annals of the New York Academy of Sciences. 1993 Mar;677(1 Clinical Flow):126–37.
- 2. Sterling TR, Chaisson RE, Moore RD. HIV-1 RNA, CD4 T-lymphocytes, and clinical response to highly active antiretroviral therapy. AIDS. 2001 Nov;15(17):2251–7. pmid:11698698
- 3. Mellors JW, Muñoz A, Giorgi J V, Margolick JB, Tassoni CJ, Gupta P, et al. Plasma viral load and CD4+ lymphocytes as prognostic markers of HIV-1 infection. Annals of internal medicine. 1997 Jun;126(12):946–54. pmid:9182471
- 4.
Ministry of Health (MoH). Supplement to the 2014 National Comprehensive HIV Prevention, Care and Treatment Guideline of Ethiopia to Address HIV test and Start. 2016;
- 5.
FMOH. Guidelines for the implementation of POC CD4 testing technologies in Ethiopia. 2013;(June).
- 6. Assefa Y, Jerene D, Lulseged S, Ooms G, Van Damme W. Rapid scale-up of antiretroviral treatment in Ethiopia: successes and system-wide effects. PLoS medicine. 2009 Apr;6(4):e1000056. pmid:19399154
- 7.
World Health Organization (WHO). Technical Brief on CD4 Technologies. 2011;(Feb):1–37.
- 8.
UNITAID. HIV / AIDS Diagnostics Technology Landscape 5 th edition. 2015;(October).
- 9.
World Health Organization (WHO). WHO | WHO list of prequalified in vitro diagnostic products. World Health Organization; 2014;
- 10. Jani I V, Sitoe NE, Chongo PL, Alfai ER, Quevedo JI, Tobaiwa O, et al. Accurate CD4 T-cell enumeration and antiretroviral drug toxicity monitoring in primary healthcare clinics using point-of-care testing. AIDS (London, England). 2011;25(6):807–12.
- 11. Wynberg E, Cooke G, Shroufi A, Reid SD, Ford N. Impact of point-of-care CD4 testing on linkage to HIV care: A systematic review. Journal of the International AIDS Society. 2014;17:1–6.
- 12.
BD MultitestTM, CD3/CD8/CD45/CD4. 2016;
- 13.
(BD) BD. BD FACSCalibur Instructions For Use. User’s guide. 2007;(32).
- 14.
(BD) BD. BD FACSPresto TM near-patient CD4 counter instruction for use. 2016.
- 15.
Sysmex cooperation. XT-2000i TM and XT1800iTM- Advanced Technology Solutions to Meet your Lab ‘ s Needs. 2012;L.
- 16. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986 Feb;1(8476):307–10. pmid:2868172
- 17. Wade D, Daneau G, Aboud S, Vercauteren GH, Urassa WSK, Kestens L. WHO multicenter evaluation of FACSCount CD4 and Pima CD4 T-cell count systems: instrument performance and misclassification of HIV-infected patients. Journal of acquired immune deficiency syndromes (1999). 2014;66(5):e98–107.
- 18. Scott LE, Galpin JS, Glencross DK. Multiple method comparison: statistical model using percentage similarity. Cytometry Part B, Clinical cytometry. 2003 Jul;54(1):46–53. pmid:12827667
- 19.
World Health Organization (WHO). Antiretroviral therapy for HIV infection in adults and adolescents. 2010.
- 20.
World Health Organization (WHO). Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach. WHO Guidelines. 2013;(June):272.
- 21. Bwana P, Vojnov L, Adhiambo M, Akinyi C, Mwende J, Prescott M, et al. The BD FACSPresto Point of Care CD4 Test Accurately Enumerates CD4+ T Cell Counts. Plos One. 2015;10(12):e0145586. pmid:26720601
- 22.
World Health Organization (WHO). WHO | Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. WHO. World Health Organization; 2016;
- 23. TEMPRANO ANRS 12136 Study Group. A Trial of Early Antiretrovirals and Isoniazid Preventive Therapy in Africa. New England Journal of Medicine. 2015;373(9):808–22. pmid:26193126
- 24. INSIGHT START Study Group, Lundgren JD, Babiker AG, Gordin F, Emery S, Grund B, et al. Initiation of Antiretroviral Therapy in Early Asymptomatic HIV Infection. The New England journal of medicine. 2015;373(9):795–807. pmid:26192873