Figures
Abstract
Objective
To evaluate the accuracy of the PIMA point-of-care CD4 analyzer (PIMA) under field conditions in comparison to the current CD4 count system (FACSCalibur), and to evaluate the operational suitability and acceptability of health professionals (HP) and HIV-patients in using the PIMA in health clinics in the Amazon Region.
Methods
CD4 counts were measured onsite by the PIMA using fingerprick blood and in the reference laboratory by both the PIMA and FACSCalibur using venous blood. We used the Bland–Altman method to estimate the mean bias, and calculated the sensitivity and specificity at <200 and <500 cell/μL thresholds. Patients (n = 404) and HP (n = 7) were interviewed on the acceptability and operational suitability of the PIMA.
Results
Using fingerprick blood (n = 337), the PIMA showed a concordance correlation coefficient (Rc) of 0.81, mean difference of -111.9 cell/μL, 93.1%/98.5% sensitivity, and 89.2%/56.7% specificity at <200 and <500 cell/μL thresholds, respectively. Venous blood (n = 340) showed an Rc of 0.89, mean difference of -83.4 cell/μL, 98.3%/97.5% sensitivity, and 93.9%/66.0% specificity at <200 and <500 cell/μL thresholds, respectively. The capillary PIMA was well accepted and found operationally appropriate by patients and HP.
Conclusions
The agreement between both instruments was poor and the PIMA underestimated CD4 cell counts, which was more pronounced at CD4 counts ≥500 cell/μl. The PIMA’s performance with fingerprick blood was less reliable than its performance with venous blood. In Brazil, where antiretroviral treatment is initiated regardless of CD4 counts, the PIMA’s systematic bias towards CD4 underestimation may limit its role for monitoring HIV-patients.
Citation: Pinto IC, Sabidó M, Pereira AB, Mello MB, de Melo Xavier Shimizu A, Protti BL, et al. (2015) Field Evaluation of a Point-of-Care CD4 Analyzer for Monitoring HIV Patients in the Interior of the Amazon Region, Brazil. PLoS ONE 10(4): e0121400. https://doi.org/10.1371/journal.pone.0121400
Academic Editor: Yukari C. Manabe, Johns Hopkins University, UNITED STATES
Received: September 9, 2014; Accepted: January 31, 2015; Published: April 23, 2015
Copyright: © 2015 Pinto et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: The study was co-funded by Centers for Disease Control and Prevention through the Cooperative Agreement Award #1U2GPS001204, and by the National Department of STD/Aids and Viral Hepatitis from the Brazilian Ministry of Health. Dr. Sabidó has a fellowship as a visiting researcher at the Fundação de Medicina Tropical Dr. Heitor Vieira Dourado, funded by the Foundation for Research Support of the Amazonas State (FAPEAM) through the Strategic Programme in Science, Technology & Innovation in Health Foundations (PECTI/AM SAÚDE). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In Brazil, access to appropriate tests to manage patients living with HIV/AIDS remains a challenge.[1] CD4+ T-cell counts in Brazil are determined by fluorescence-activated cell sorting (FACS) using flow cytometres, which are costly, require cold chains, need routine technical maintenance, require skilled technicians, and are primarily located in urban centres far from sample collection points.[2] In isolated areas in the interior of the Amazon, logistical and operational barriers to transporting samples and a lack of laboratory infrastructure combined with scarce technical staff represent considerable obstacles to the provision of quality care to people living with HIV/AIDS in the region.
Recently, a new technology has been developed to count CD4 cells. The new PIMA point-of-care (POC) CD4 analyzer (Alere PIMA CD4, Waltham, MA, USA) has a quick turnaround time, providing both CD3 and CD4 counts in only 20 minutes. Because of its cartridge-based system, rechargeable battery, and small size, it can be used in non-laboratory settings with little infrastructure.[3] The PIMA POC CD4 analyzer has already been evaluated in several contexts, showing similar results to the conventional techniques for measuring CD4 counts (4-colour CD3/4/8/45 BD FACSCalibur flow cytometry).[4–10] However, the PIMA analyzer showed a tendency to give a lower CD4 count in some studies.[4, 5, 8, 9, 11, 12] A systematic review of the impact of POC CD4 testing on HIV care suggested that POC testing can positively increase the proportion of patients receiving CD4 measurements and results as well as reduce the time to eligibility assessments.[13] However, evidence for increasing rates of antiretroviral treatment (ART) initiation and retention into care remains unclear.
Nevertheless, these potential benefits depend on the appropriate use and performance of the PIMA POC analyzer under field conditions that can be affected by the training of health professionals using the equipment, their skills in obtaining blood samples, the local infrastructure at the clinics, and hot and humid conditions.[14] These local characteristics may affect the accuracy of the PMIA analyzer’s CD4 counts. In this sense, this study aims to evaluate the accuracy of the PIMA POC CD4 analyzer under field conditions using two methods for blood sample collection (fingerprick versus venous blood) and compare them, as well as compare each of them to the current method of counting CD4 cells (FACSCalibur) in a reference laboratory in the city of Manaus. This study also aims to assess patient and health professionals performance and acceptability of the new PIMA POC CD4 analyzer in specialized clinics that provide care and treatment of people living with HIV (SAE, standing for Serviço de Assistência Especializada em HIV/Aids in Portuguese) in the interior of the Amazon region.
Methods
Study setting
The study was conducted between July 2013 and March 2014 in two SAE located in two municipalities of the interior of the Amazonas State (Tabatinga in Alto Solimões Region, at the triple border Brazil-Peru-Colombia, and Parintins in Médio Solimões Region), and in one SAE in Manaus. The SAE were selected based on the number of HIV patients enrolled at those locations, and the distance to the reference laboratory.[15] Before the introduction of POC CD4 testing, blood samples were collected once a week and sent to Manaus. Patients were asked to return for the results after approximately two months.
The study protocol was approved by the local Ethics Committee of Fundação de Medicina Tropical Doutor Heitor Vieira Dourado (FMT-HVD) in Manaus, the Brazilian National Committee for Ethics and Research (CONEP), and received a non-research determination by the Associate Director of Science of the Center for Global Health of the U.S. Centers for Disease Control and Prevention.
Study population
Individuals were recruited consecutively among the HIV-positive patients who arrived at each SAE for routine follow-ups with blood collection for CD4 counts during a 6-month period. They were included if they were ≥ 18 years of age and agreed to participate demonstrated by a signed informed consent form.
Health professionals involved in routine examinations of SAE HIV patients were trained in the use of the PIMA POC CD4 analyzer. This included three pharmaceutical-biochemists (one per site) and four laboratory technicians (one in Tabatinga, one in Parintins, and two in Manaus).
Study procedures
Four different PIMA analyzers were used throughout the study, one in Parintins, two in Tabatinga, and one in Manaus. When sample collection in Parintins and Tabatinga was completed, the four PIMA analyzers were transferred to the laboratory in Manaus. The participants provided a 25 μL finger prick capillary sample of blood that was obtained using a Sardest Safety lancet (1.5 mm with blade, 1.6 mm depth; Sarstedt AG & CO., Nümbercht, Germany) and was immediately processed on site. Approximately half of a drop of blood was directly loaded into the cartridge.[16] The results were provided to the patients 20–30 minutes later.
Each patient also provided 5 mL of venous blood collected in BD Vacutainer tubes containing K2 EDTA (Becton Dickinson, Franklin Lakes, New Jersey, USA). These samples were kept at room temperature (20 to 25°C) and shipped by boat or plane at the same temperature to the reference laboratory at FMT-HVD Manaus. Both storage and transportation temperature and humidity were recorded. After shipment to the reference laboratory, venous samples were processed using both the PIMA POC CD4 analyzer and the FACSCalibur (Becton Dickinson Biosciences, San Jose, California, USA). All testing was performed according to manufacturer’s instructions.[17] Laboratory analyses of venous blood samples were performed within 48 hours after venipuncture, which is in agreement with the instructions for the use of PIMA POC CD4. Operators of POC and laboratory instruments only had access to the results of either one or the other instrument, i.e., they were blinded to reciprocal results of the same patient sample.
When the PIMA POC CD4 analyzer generated an error code due to rejection of the sample or aborted result, a second measurement was performed. If the second analysis did not provide a result, the error was considered repetitive, and the sample was excluded from the comparison analysis.
Two months after the start of the study, the field coordinator observed health professionals performance in using the PIMA POC CD4 analyzer and interviewed them to identify factors that facilitated or were considered barriers to the performance of the test. Trained health professionals administered a questionnaire to patients who collected demographic data, information on access to the SAE, opinions on the reference CD4 count test, knowledge of and experience with the new POC CD4 analyzer, and comparative opinions of both tests.
Quality control
Internal quality control was performed daily for both the PIMA POC CD4 analyzer and the BD FACSCalibur CD4 according to manufacturers’ instructions.[16, 18]. During the study period, only once the PIMA analyser, located in Parintins, did not produce consistent results for normal control samples.
For external quality assessment (EQA), the Ministry of Health supported an Inter-laboratory Quality Control Program. As one of the National CD4/CD8 Network laboratories, the reference laboratory in Manaus received proficiency testing panels for the EQA of its FACSCalibur. For the EQA of the PIMA CD4 analyzer, proficiency testing panels were created at the reference laboratory in Manaus and distributed to the sites. Onsite results were evaluated for their accuracy in comparison to known specimens.
Data analysis
The data were analysed using MedCalc version 14.12.0 (MedCalc Software, Mariakerke, Belgium) and Stata 10.0 software (Statacorp, College Station, TX, USA). Descriptive analyses were used to summarize patient characteristics and the acceptability of the PIMA to both patients and health professionals. Paired t-tests were performed as appropriate to compare the differences in means.
We assessed the agreement between the paired test results from the PIMA and FACSCalibur using Lin’s concordance correlation coefficient, Rc.[19] Poor agreement was defined as an Rc <0.90, moderate as an Rc of 0.90–0.95, substantial as an Rc of 0.95–0.99, and almost perfect as an Rc >0.99.[20] This coefficient includes a measurement of precision (Pearson’s correlation, ρ), and accuracy (bias correction factor, Cb).[19] Bland-Altman [21] and Pollock [22] analyses were performed to calculate the mean bias and limits of agreement (LOA), which are the mean bias ± 1.96 standard deviation (SD) of all the paired measurements. We calculated the relative bias ([PIMA CD4 testing—FACSCalibur]/mean), which is better suited than the mean absolute bias to compare absolute CD4 counts between two methods. The PIMA POC analyzer using capillary blood or venous blood and the FACSCalibur were also compared using Passing-Bablok regression analysis, which is a non-parametric linear regression and less sensitive to the presence of outliers.[23] The results were compared across SAEs and by CD4 cell count ranges. The accuracy of the PIMA POC analyzer was assessed through sensitivity, specificity, and positive and negative predictive values at clinically relevant thresholds of <200 and <500 cells/μl and was compared with reference testing on the FACSCalibur. These cut off points were selected because, according to Brazilian guidelines, they represent the initiation of chemoprophylaxis for Pneumocystis jirovecii (CD4 counts <200 cells/μl) and at the time of the study, the initiation of triple antiretroviral treatment (CD4 counts <500 cells/μl).[24]
Results
A total of 408 subjects were recruited for the study, of which five did not accept any testing and were excluded from the accuracy analysis. In total, 403 subjects had at least one test performed and were included in the accuracy analysis (174 from Manaus, 96 from Parintins, and 133 from Tabatinga). Of the 403 patients, one did not accept fingerprick, therefore 402 tests were performed on PIMA using blood collected through fingerprick (PIMA capillary blood test). Of these, 32 tests (12.6%) generated an error instead of a valid result. These samples were retested, obtaining 10 repetitive errors (2.5%). Regarding samples collected through venous blood, of the 403 samples collected, 349 were analysed using PIMA venous blood test as, 34 samples were not transported to the reference laboratory, 15 samples arrived at the reference laboratory after 48 hours and were discarded, and in five, the test was not performed at the reference laboratory. Of the 349 venous samples, nine test (2.6%) generated an error instead of a valid result in the PIMA analyzer. These samples were retested, obtaining six repetitive errors (1.7%). In 12 blood samples, FACSCalibur was not performed. Therefore, testing results were available for 392 PIMA capillary blood test, for 343 PIMA venous blood test, and for 342 FACSCalibur. In total, 337 capillary samples and 340 venous samples were analysed with both the PIMA with FACSCalibur.
PIMA performance versus FACSCalibur
The mean capillaryPIMA POC CD4 analyzer result was 363.3 (range: 16–1668), and the corresponding mean CD4 count using FACSCalibur was 478.2 (range: 12–1990) (Table 1). The mean venous PIMA CD4 result was 391.8 cells/μl (range: 18–1416), and the corresponding mean on FACSCalibur was 475.2 cells/μl (range: 12–1990). The overall mean CD4 for both capillary and venous blood samples in the PIMA CD4 analyzer was significantly lower than the mean CD4 count using the FACSCalibur (p<0.001).
PIMA POC CD4 cell count results showed poor concordance with the FACSCalibur instrument for both methods of blood sample collection (Rc 0.81 for fingerprick capillary blood samples and Rc 0.89 for venous blood samples) (Table 1). Passing-Bablok regression plots showed slopes of 0.78 (95% CI: 0.75 to 0.82) for fingerpirck capillary blood samples and 0.83 (95% CI: 0.80 to 0.85) for venous blood samples (Fig 1).
Passing-Bablok regression plots: comparison of absolute CD4 counts obtained by the PIMA POC CD4 analyzer using capillary blood samples (A) and venous blood samples (B) versus FACSCalibur using venous blood samples. The solid blue line represents the regression line and the dashed lines represent the 95% CI for the regression line. The corresponding graphs the relative bias between the PIMA POC CD4 analyzer and the FACSCalibur are represented in Pollock plots for capillary blood samples (C) and venous blood samples (D). The solid blue line represent the mean bias. The dashed lines represent the upper and lower limits of agreement (LOA = mean bias ± 1.96 SD).
PIMA POC CD4 cell counts using fingerprick capillary blood samples produced absolute CD4 counts with an absolute bias of -111.9 cells/μl (LOA: -352.9, 129.1) and with a relative bias of -26.5% (LOA: -88.0, 35.0) when compared with venous blood samples tested on a laboratory FACSCalibur instrument (Table 1). The PIMA CD4 analyzer using venous blood presented an absolute bias of -83.4 cells/μl (LOA: -269.0, 102.3) and a relative bias of 18.6% (LOA: -56.2, 19.0) when compared with venous blood samples run in the FACSCalibur instrument (Table 1). When data were stratified by CD4 cell count ranges, results for both of these comparisons showed that the absolute mean bias increased with higher CD4+ T cell counts (Table 2).
However, the mean percent bias was higher for samples with lower CD4 counts (200–499 cells/μl) (capillary: -28.9%, venous: -21.8%) than for the higher CD4 stratum at > 500 cell/μl (capillary: -17.0%, venous: -14.6%) (Table 2). Table 1 shows that concordance correlations between the PIMA CD4 analyzer and FACSCalibur results were lower with blood samples collected by fingerprick than with venous blood in the PIMA analyzer. Fingerprick blood samples resulted in slightly lower CD4 cell counts than venous blood samples in the PIMA.
We also compared performances between different study sites (Table 3). The mean CD4+ T-cell counts by the PIMA using capillary blood samples were similar between sites (Manaus: 362.6 cells/μl; Parintins: 372.7 cells/μl; Tabatinga: 368.5 cells/μl). However, as shown by the concordance correlation coefficients and LOA bias in Table 3, the performance of the PIMA CD4 test was better in Manaus, were the reference laboratory is located, than in Tabatinga and Parintins.
Accuracy of the PIMA versus the FACSCalibur
The diagnostic accuracy of the PIMA POC CD4 measurements was compared to that of the FACSCalibur at <200 cells/μl (Table 4). The PIMA POC CD4 capillary blood sensitivity was 93.1% (95% CI: 83.3–98.1%) and the specificity was 89.2% (95% CI: 85.0–92.6%). The PIMA POC CD4 venous blood sensitivity was 98.3% (95% CI: 91.1–100%) and the specificity was 93.9% (95% CI: 90.5–96.4%). Increasing the threshold to 500 cell/μl increased the sensitivity and the positive predictive value (PPV) of the PIMA capillary blood test (98.5% and 76.0%). For the PIMA venous blood test, sensitivity slightly decreased at a threshold of 500 cell/μl (97.5%), although the PPV increased (80.2%). However, the specificity and negative predictive value declined for both tests.
PIMA acceptability and operational suitability
We interviewed 404 patients (280 males, 69.3%) with a mean age of 35.9 years (range: 18–62). Mean time for patients to reach the health clinic was 2.4 hours (range: 0–72). Almost one-fifth (19.3%, 78/404) reported difficulties in getting to the clinic; and among them, 47.4% (36/76) patients reported lack of or delays in transportation used to reach the clinic. Almost one-half (44.6%, 179/404) reported having incurred transportation costs in coming to the clinic. Most (88.9%, 359/404) have had prior CD4 counts using conventional test (FACSCalibur), and approximately half (53.2%, 213/400) reported that the delivery of results was unsatisfactory. In contrast, 97.5% (387/397) agreed that the time to obtain results with the PIMA was short and satisfactory. Almost half (47.4%, 186/392) trusted FACSCalibur results the most, although 78.6% (313/398) of patients stated that they also trusted the PIMA results. Most (65.3%, 258/395) found venipuncture more painful than a fingerprick, although 20% (44/220) reported that the fingerprick caused discomfort. Most (67.1%, 265/395) of the respondents stated that given the choice they would choose PIMA to count their CD4+ T-cells.
All health professionals (100.0%, 7/7) found the PIMA easy to use and results easy to interpret, and valued the short amount of time to provide test results to patients (85.7%, 6/7). Most (85.7%, 6/7) found the instrument suitable for daily use in the SAE. Almost half (42.9% 3/7) thought that the instrument presented error messages often, but most (71.4%, 5/7) could easily identify them. Most thought that the PIMA could easily replace the reference test (71.4%, 5/7) and trusted the PIMA results (71.4% 5/7). In contrast, 42.9% (3/7) health professionals believed that the PIMA results were less reliable than the FACSCalibur, and only 42.9% (3/7) would recommend the use of the PIMA instead of the reference test. Regarding using fingerprick to collect blood for the PIMA, health professionals perceived the method easy to perform (85.7%, 6/7), less painful for the patient (85.7%, 6/7), and able to provide results as reliable as through venipuncture (85.7%, 6/7).
Observations of the execution of the PIMA showed inadequate fingerprick collection (insufficient sample and squeezing) and/or cartridge use during internal quality control (damage to cartridges during operation and air bubbles in samples) in Parintins. Temperature and humidity was only adequately monitored in Manaus.
Discussion
In the context of the Amazon, with hot and humid climate and remote settings, PIMA measurements using venous blood samples and particularly using fingerprick blood samples were in poor agreement with results of the FACSCalibur test. The PIMA analyzer tended to underestimate CD4 cell counts. Most of the studies comparing the PIMA CD4 analyzer (using capillary or venous specimens) to FACSCalibur have found similar underestimations by the PIMA.[4, 5, 8–12] However, we documented a larger negative bias from the PIMA placed at the two remote sites that was greater than 100 cell/μl for capillary blood. This divergence might be related with less experienced health professionals at those sites and/or conditions of sample shipment. We cannot exclude the possibility that weather conditions could have played a role in these divergent results. Temperature and humidity were monitored systematically for all shipments of samples from Tabatinga field site to the reference laboratory in Manaus, but only in one out of seven shipments from Parintins. To further explore the influence of temperature and humidity during shipment on study results we run a sensitivity analysis. The PIMA CD4 results using venous blood without the un-monitored shipment (n = 290 venous samples) showed a similar concordance correlation coefficient (Rc 0.90) but a lower bias (-77.8 cells/μl [LOA: -246.2 to 90.7]) than that obtained with the overall 340 venous samples shipped. Therefore, conditions during shipment might have played a role in the poorer results for the PIMA in remote sites of the Amazon.
The negative bias of the PIMA analyzer was more pronounced for CD4 counts ≥ 500 cell/μl than at lower CD4 ranges. Similar divergences have been observed in previous comparisons between the PIMA CD4 Analyzer and FACSCalibur, with greater underestimations by the PIMA at higher CD4 counts (>500 cell/μl).[4, 5] The PIMA performed better using venous blood than fingerprick blood samples. Although this has also been observed in previous studies,[10] the greater bias and larger LOA reported with fingerprick blood could be a matter of concern and might be related to fingerprick sampling method. In fact, although most of the health professionals considered fingerprick collection easy to perform, the study coordinator in one of the SAEs observed inadequate squeezing. This result suggests that additional training and regular monitoring are essential to maintaining the ongoing performance of the PIMA POC CD4 analyzer, as demonstrated for other POC testing.[25] The PIMA’s sensitivity improved using capillary blood samples at a higher CD4 threshold of 500 cell/μl, rather than of 200 cell/μl threshold, although the performance decreased slightly for when venous blood samples were used. This finding is consistent with two previous studies from Kenya[26] and Uganda[7] in which sensitivity of the PIMA increased when the analysis was performed using a CD4 cut-off of 500 cells/μl. Again, the PIMA results obtained from fingerprick blood samples were less precise than of those using venous blood samples.
In spite of the poor performance of the PIMA POC CD4 analyzer under the study conditions, we believe our findings to have clinical applicability. Brazil has recently adopted universal ART initiation to all HIV positive patients regardless of their CD4 cell counts. Under the Test & Treat approach, the rapid assessment of CD4 cell counts became less critical for ART initiation, but it is still an important tool for clinical decision on the adoption of prophylaxis of co-infections, particularly in remote areas such as those studied here. However, if the PIMA POC CD4 analyzer is to be used to monitor CD4 cell counts in HIV positive patients over time, this systematic underestimation may warrant further attention. On the other hand, many ART programs in resource-limited settings still rely on immunologic and/or clinical presentation to indicate ART initiation, to measure responses to therapy, and to determine when to change to a second-line regimen.[27] In the context of the use of the PIMA POC CD4 analyzer for determining eligibility for ART, underestimation of CD4 cells is not a major concern given that it would imply in earlier treatment initiation for some patients.
From an operational perspective, 32 additional PIMA capillary tests were required to provide CD4 results on 402 patients, resulting in an error rate of 12.6%. This error rate is consistent with previous evaluations of PIMA capillary testing, in which error rate ranged from 10% to 20%.[5, 8, 10, 11, 28, 29] Ten patients did not receive PIMA capillary tests result on site due to repeated machine errors. The PIMA analyzer using venous blood showed a lower rate of errors of 2.6% (9/349) than the PIMA analyzer using capillary blood. The most common errors may partially be explained by inadequate or inappropriate sample collection or cartridge use. The error rate of the PIMA capillary testing reported may influence the perception of patients and health professionals on the reliability of the tests results. Moreover, the high error rate negatively influences the time needed to deliver a test result. In an economic evaluation of a mobile program that used PIMA POC capillary testing, when an error rate of 3% was included in the analysis, the total test per cost increased slightly from $23.76 to $24.49.[30] With regards to operational aspects, the PIMA POC CD4 analyzer was well accepted by patients and health professionals, although health professionals showed low trust in the PIMA analyzer results, which could affect their buy-in if the equipment was adopted for patient management. In our study, health professionals considered fingerprick easy to perform but inadequate fingerprick collection was observed. The fact that blood samples can be collected by fingerprick using lancets is another important operational advantage in areas of limited infrastructure. Nevertheless, it is crucial to reinforce training to avoid underperformance of the equipment.
Immediate on-site testing is especially important in a region characterized by long distances and where transportation is a barrier to reaching clinics for one out of five patients. At those remote sites, at least two visits to the health clinic is required before results were available. The PIMA POC CD4 analyzer could allow patients to receive their CD4 results earlier and immediately begin treatment, thereby reducing one of the largest causes of HIV-related mortality, particularly in those areas were a high proportion of people are diagnosed very late stages of AIDS. The PIMA POC CD4 analyzer is suitable for small clinics that run up to 25 test daily, as those studied here, given that it take approximately 20 minutes to read and print one sample result. [5] This makes the PIMA POC CD4 analyzer suitable for the context of the Amazon region characterized by small and isolated villages and limited laboratory capacity.
The study presents limitations. There might be inter-observer variations in equipment handling given that several technicians were involved in the study. We did not assess the repeatability of the PIMA results. A previous study showed that duplicate POC CD4 cell counts with fingerprick blood produced results with repeatability similar to that of the FACSCalibur using venous blood.[4] Manabe et al. performed duplicate measurements on both capillary and venous samples using two different PIMA POC CD4 analyzers. Precision of the duplicate testing showed a significant difference in the PIMA capillary duplicates (mean difference 33.38 cell/mL, p = 0.005), but acceptable for the PIMA venous duplicates (mean difference -6.54 cell/mL, p = 0.09).[29] Wade et al. evaluated precision by running one venous blood sample 10 times on five different PIMA instruments.[28] The Pima POC CD4 analyzer showed an intra-assay variation with mean percent coefficient of variation equal or larger than the recommended value of 10% (or 15% in low CD4 counts).[31] Therefore, further studies on precision are required. The clinics used in this study are typical of the health centres found in the Amazon region in terms of numbers of patients and staff, infra-structures, and logistical barriers to access the clinic, and to the shipment of biological samples.
In conclusion, our data suggest poor agreement between the PIMA POC CD4 analyzer and the laboratory-based FACSCalibur, but additional studies are needed to understand the role that POC testing can play as a tool for monitoring CD4 cell counts. The implementation of the PIMA POC CD4 analyzer might empower SAEs in the management of their own patients and bring the clinical laboratory closer to the patient. Nevertheless, the performance with fingerprick samples was not as good as with venous samples, increasing the complexity of evaluating CD4 counts in areas with non-specialized health professionals. A high-quality management program with proper training needs to be implemented in parallel to guarantee the utility of the PIMA POC CD4 analyzer.
Acknowledgments
We would like to thank the Program on AIDS, Sexually Transmitted Infections, and Viral Hepatitis of Tabatinga and Parintins, the National School of Public Health of the Oswaldo Cruz Foundation, the National Department of STD, AIDS and Viral Hepatitis of the Brazilian Ministry of Health, the U.S. Centers for Disease Control and Prevention (CDC) for their support throughout the study. Authors are particularly thankful to Luciana Kohatsu and Larry E. Westerman from CDC’s International Laboratory Branch for the technical support with protocol development and for sharing support material and data collection instruments. As a disclaimer, the development of this study was partially supported by the HHS Centers for Disease Control and Prevention (CDC), Division of Global HIV/AIDS, under the Cooperative Agreement #1U2GPS001204. The content of this article is solely the responsibility of the authors and do not necessarily represent the official views of CDC or the UNAIDS.
Author Contributions
Conceived and designed the experiments: ICP MS ABP MBM AMXS BLP ASB. Performed the experiments: ICP ABP AMXS BLP. Analyzed the data: MS. Contributed reagents/materials/analysis tools: ICP MS ABP MBM AMXS BLP ASB. Wrote the paper: ICP MS ABP MBM AMXS BLP ASB. Institutional support: ASB.
References
- 1. Rocha GM, Machado CJ, Acurcio Fde A, Guimaraes MD. Monitoring adherence to antiretroviral treatment in Brazil: an urgent challenge. Cadernos de saude publica 2011; 27:S67–78. pmid:21503526
- 2.
Departamento de DST, Aids, e Hepatites Virais. Ministério da Saúde. Rede Nacional de Laboratórios de CD4+/CD8+. Ministério da Saúde, Brasília: 2014. Available: http://www.aids.gov.br/pagina/2012/51659 Accessed 2014 May 13.
- 3. Zachariah R, Reid SD, Chaillet P, Massaquoi M, Schouten EJ, Harries AD. Viewpoint: Why do we need a point-of-care CD4 test for low-income countries? Trop Med Int Health 2011; 16:37–41. pmid:21371207
- 4. Jani IV, Sitoe NE, Chongo PL, Alfai ER, Quevedo JI, Tobaiwa O, et al. Accurate CD4 T-cell enumeration and antiretroviral drug toxicity monitoring in primary healthcare clinics using point-of-care testing. AIDS 2011; 25:807–812. pmid:21378535
- 5. Diaw PA, Daneau G, Coly AA, Ndiaye BP, Wade D, Camara M, et al. Multisite evaluation of a point-of-care instrument for CD4(+) T-cell enumeration using venous and finger-prick blood: the PIMA CD4. J Acquir Immune Defic Syndr 2011; 58:e103–111. pmid:21909029
- 6. Rathunde L, Kussen G, Beltrame M, Costa LD, Raboni S. Evaluation of the Alere Pima for CD4+ T lymphocytes counts in HIV-positive outpatients in Southern Brazil. Int J STD AIDS 2014;25:956–9. pmid:24616116
- 7. Galiwango RM, Lubyayi L, Musoke R, Kalibbala S, Buwembo M, Kasule J, et al. Field evaluation of PIMA point-of-care CD4 testing in Rakai, Uganda. PLoS one 2014, 9:e88928. pmid:24614083
- 8. Sukapirom K, Onlamoon N, Thepthai C, Polsrila K, Tassaneetrithep B, Pattanapanyasat K. Performance evaluation of the Alere PIMA CD4 test for monitoring HIV-infected individuals in resource-constrained settings. J Acquir Immune Defic Syndr 2011; 58:141–147. pmid:21709568
- 9. Mtapuri-Zinyowera S, Chideme M, Mangwanya D, Mugurungi O, Gudukeya S, Hatzold K, et al. Evaluation of the PIMA point-of-care CD4 analyzer in VCT clinics in Zimbabwe. J Acquir Immune Defic Syndr 2010; 55:1–7. pmid:20622679
- 10. Glencross DK, Coetzee LM, Faal M, Masango M, Stevens WS, Venter WF, et al. Performance evaluation of the Pima point-of-care CD4 analyser using capillary blood sampling in field tests in South Africa. J Int AIDS Soc 2012; 15:3. pmid:22284546
- 11. Myer L, Daskilewicz K, McIntyre J, Bekker LG. Comparison of point-of-care versus laboratory-based CD4 cell enumeration in HIV-positive pregnant women. J Int AIDS Soc 2013; 16:18649. pmid:24044627
- 12.
Van Schaik N, Kranze rK, Mye rL, Raditlhalo E, Thebus E, Davies N, et al. Field validation of the PIMA Analyzer in a mobile clinic setting in South Africa. Poster session presented at: CROI 2011. 18th Conference on Retroviruses and Opportunistic Infections; 2011 Feb 27—Mar 2; Boston, USA
- 13. Wynberg E, Cooke G, Shroufi A, Reid SD, Ford N. Impact of point-of-care CD4 testing on linkage to HIV care: a systematic review. J Int AIDS Soc 2014; 17:18809. pmid:24447595
- 14. Shott JP, Galiwango RM, Reynolds SJ. A Quality Management Approach to Implementing Point-of-Care Technologies for HIV Diagnosis and Monitoring in Sub-Saharan Africa. J Trop Med 2012; 2012:651927. pmid:22287974
- 15.
Department of STDs, AIDS, and Viral Hepatitis. Health Surveillance Secretariat. Brazilian Ministry of Health. Boletim Epidemiológico (Versão Preliminar) 2011 [Epidemiological Bulletin (Preliminary Version) 2011]. Available at http//www.aids.gov.br Last accessed 20 January 2013.
- 16.
Alere PIMA CD4 Procedures. Available: http://alerehiv.com/hiv-monitoring/procedure/. Accessed 2013 Jan 20.
- 17.
BD Beckton Dickinson. Multitest CD3 FITC/CD8 PE/CD45 PerCP/CD4 APC Reagent. Available:: http://www.bdbiosciences.com/external_files/is/doc/tds/Package_Inserts_IVD/live/web_enabled/23-3600-02_3_8_45_4_IVD.pdf. Accessed 2014 May 13.
- 18. Mandy FF, Nicholson JK, McDougal JS, CDC . Guidelines for performing single-platform absolute CD4+ T-cell determinations with CD45 gating for persons infected with human immunodeficiency virus. Centers for Disease Control and Prevention. MMWR Recomm Reports 2003; 52:1–13.
- 19. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45:255–268. pmid:2720055
- 20.
Bride M, McBride GB. A proposal for strength-of-agreement criteria for Lin's Concordance Correlation Coefficient, 2005. NIWA Client Report: HAM2005-062. Available: http://www.medcalc.org/download/pdf/McBride2005.pdf Accessed 2014 Dec 09.
- 21. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8:135–160. pmid:10501650
- 22. Pollock MA, Jefferson SG, Kane JW, Lomax K, MacKinnon G, Winnard CB. Method comparison—a different approach. Ann Clin Biochem 1992; 29:556–560. pmid:1444169
- 23. Passing HB. A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, Part I. J Clin Chem Clin Biochem 1983; 21:709–720.
- 24.
Department of STDs, AIDS, and Viral Hepatitis. Ministry of Health. Protocolo Clínico e Diretrizes Terapêuticas para Manejo da Infecção pelo HIV em Adultos. Brasília, DF, 2013. Available: http://www.aids.gov.br/tags/publicacoes/protocolo-clinico-e-diretrizes-terapeuticas Accessed 2014 Dec 22.
- 25. Mabey DC, Sollis KA, Kelly HA, Benzaken AS, Bitarakwate E, Changalucha J, et al. Point-of-care tests to strengthen health systems and save newborn lives: the case of syphilis. PLoS Med 2012; 9:e1001233. pmid:22719229
- 26. Mwau M, Adungo F, Kadima S, Njagi E, Kirwaye C, Abubakr NS, et al. Evaluation of PIMA point of care technology for CD4 T cell enumeration in Kenya. PLoS one 2013; 8:e67612. pmid:23825674
- 27.
World Health Organization. WHO Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach June 2013. Chapter 7: Clinical guidance across the continuum of care: antiretroviral therapy. WHO, Geneva: 2013. Available: http://apps.who.int/iris/bitstream/10665/85321/1/9789241505727_eng.pdf?ua=1. Accessed 2014 Dec 22.
- 28. Wade D, Daneau G, Aboud S, Vercauteren GH, Urassa WS, Kestens L. WHO multicenter evaluation of FACSCount CD4 and Pima CD4 T-cell count systems: instrument performance and misclassification of HIV-infected patients. J Acquir Immune Defic Syndr 2014; 66:e98–107. pmid:24853304
- 29. Manabe YC, Wang Y, Elbireer A, Auerbach B, Castelnuovo B. Evaluation of portable point-of-care CD4 counter with high sensitivity for detecting patients eligible for antiretroviral therapy. PLoS One 2012; 7:e34319. pmid:22536323
- 30. Larson B, Schnippel K, Ndibongo B, Long L, Fox MP, Rosen S. How to estimate the cost of point-of-care CD4 testing in program settings: an example using the Alere Pima Analyzer in South Africa. PLoS One 2012, 7:e35444. pmid:22532854
- 31. Chesher D. Evaluating assay precision. Clin Biochem Rev 2008; 29:S23–S26. pmid:18852851