Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Hemoglobin point-of-care testing in rural Gambia: Comparing accuracy of HemoCue and Aptus with an automated hematology analyzer

  • Stefan A. Nass ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing

    stefan.a.nass@gmail.com

    Affiliations Medical Humanities, Amsterdam-UMC—VUmc Location, Vrije Universiteit, Amsterdam, The Netherlands, Medical Research Council Unit, The Gambia at the London School of Hygiene & Tropical Medicine, Banjul, The Gambia

  • Ilias Hossain ,

    Contributed equally to this work with: Ilias Hossain, Chilel Sanyang, Bakary Baldeh

    Roles Investigation, Project administration, Writing – review & editing

    Affiliation Medical Research Council Unit, The Gambia at the London School of Hygiene & Tropical Medicine, Banjul, The Gambia

  • Chilel Sanyang ,

    Contributed equally to this work with: Ilias Hossain, Chilel Sanyang, Bakary Baldeh

    Roles Investigation, Writing – review & editing

    Affiliation Medical Research Council Unit, The Gambia at the London School of Hygiene & Tropical Medicine, Banjul, The Gambia

  • Bakary Baldeh ,

    Contributed equally to this work with: Ilias Hossain, Chilel Sanyang, Bakary Baldeh

    Roles Investigation, Writing – review & editing

    Affiliation Medical Research Council Unit, The Gambia at the London School of Hygiene & Tropical Medicine, Banjul, The Gambia

  • Dora I. A. Pereira

    Roles Conceptualization, Methodology, Project administration, Supervision, Validation, Writing – review & editing

    Affiliations Medical Research Council Unit, The Gambia at the London School of Hygiene & Tropical Medicine, Banjul, The Gambia, Department of Pathology, University of Cambridge, Cambridge, United Kingdom

Hemoglobin point-of-care testing in rural Gambia: Comparing accuracy of HemoCue and Aptus with an automated hematology analyzer

  • Stefan A. Nass, 
  • Ilias Hossain, 
  • Chilel Sanyang, 
  • Bakary Baldeh, 
  • Dora I. A. Pereira
PLOS
x

Abstract

Background

Anemia is one of the most impactful nutrient deficiencies in the world and disproportionately affects children in low-resource settings. Point-of-care devices (PoCDs) measuring blood hemoglobin (Hb) are widely used in such settings to screen for anemia due to their low cost, speed, and convenience. Here we present the first iteration of Aptus, a new PoCD which measures Hb and hematocrit (HCT).

Aim

To evaluate the accuracy of Aptus and HemoCue® Hb 301 against an automated hematology analyzer (Medonic®) in Gambian children aged 6–35 months and the Aptus’ usage in the field.

Methods

Aptus, HemoCue® and Medonic® were compared using venous blood (n = 180), and Aptus and HemoCue® additionally using capillary blood (n = 506). Agreement was estimated using Bland-Altman analysis and Lin’s concordance. Usage was assessed by error occurrence and user experience.

Results

Mean Hb values in venous blood did not significantly differ between Aptus and HemoCue® (10.44±1.05 vs 10.56±0.93g/dl, p>0.05), but both measured higher Hb concentrations than Medonic® (9.75±0.99g/dl, p<0.0001). Lin’s coefficient between Aptus and Medonic® was rc = 0.548, between HemoCue® and Medonic® rc = 0.636. Mean bias between the PoCDs venous measurements was -0.11g/dl with limits of agreement (LoA) -1.63 and 1.40g/dl. The bias was larger for the comparisons between the Medonic® and both Aptus (0.69g/dl, LoA 0.92 and 2.31g/dl) and HemoCue® (0.81g/dl, LoA 0.17 and 1.78g/dl). ROC curves showed an AUC of 0.933 in HemoCue® and 0.799 in Aptus. Capillary Hb was higher with Aptus than HemoCue® (10.33±1.11g/dl vs 10.01±1.07g/dl, p<0.0001). Mean bias was 0.32g/dl with LoA of -1.91 and 2.54g/dl. Aptus‘ usage proved intuitive, yet time-to-results and cuvettes could be improved.

Conclusion

Both PoCDs showed a relatively limited bias but large LoA. Aptus and HemoCue® showed similar accuracy, while both overestimated Hb levels. Aptus showed promise, with its operation unimpaired by field conditions as well as being able to show HCT values.

Introduction

Anemia is one of the most prevalent nutrient deficiencies worldwide. The most recent data from the World Health Organization (WHO) in 2011 estimate that the prevalence of anemia in children under the age of five is 42.6% worldwide, with the highest prevalence in the African region at 62.3% [1]. The most prominent causative factor is iron deficiency anemia (IDA), which has been shown to have detrimental immediate and long-term impact on growth, organ function and cognitive development in children [14]. IDA is estimated to affect 1.24 billion people worldwide, leading to 35 million disability-adjusted life years (DALYs) and contributing the largest share to total years lived with disability (YLD) [1, 5, 6]. The standard measurement to determine anemia status occurs via blood hemoglobin (Hb) levels, which enables quantitative but not causative assessment of anemia [1, 7]. As iron deficiency, defined in young children as inflammation-adjusted ferritin concentration < 12μg/L, is more costly and technologically sophisticated to measure, IDA is commonly assumed to represent 50% of anemia in a given area when lacking data on iron status [5]. However, anemia can be caused by other factors such as infections, other micronutrient deficiencies or hemoglobinopathies [5]. As such, implementing tests to screen for the cause of anemia can prevent misdiagnosis and consequently mistreatment. One approach is to measure the ratio of Hb to hematocrit (HCT), referred to as the mean corpuscular Hb concentration (MCHC), which decreases in microcytic anemias such as IDA [8, 9].

Furthermore, HCT is often used to approximate Hb in low-resource and field settings where direct Hb measurement is not possible or feasible. The correlation of HCT with Hb is, however, not sufficiently consistent [9]. Therefore, measuring HCT and Hb together would provide a more refined and robust procedure for anemia diagnostics suitable to low-resource settings. To this end, point-of-care devices (PoCDs) that are affordable, durable and easy to use are required. Presently, the most commonly used PoCD, set as the standard device in the guidelines for Demographic and Health Surveys (DHS) [10], is the HemoCue® series, which requires a simple fingerprick and provides results seen as comparable to automatic hematology analyzers [11, 12]. However, it solely measures Hb.

Aptus device

The Aptus is a novel PoCD certified in accordance with the requirements of the EU Directive for in vitro diagnostics medical devices (directive 98/79/EC) and provides Hb and HCT measurements simultaneously via centrifugation and photometry. More portable than the HemoCue® Hb 301, Aptus is a handheld device that could improve the in-field diagnosis of anemia in low- and middle-income countries (LMICs). The Aptus model used here was one of the first available commercially. Data from the NHS Healthcare Trust at Imperial College, London, showed good correlation with the Sysmex® bioanalyzer [13]. However, evidence to support its use in rural community-based studies in LMICs or with children is not yet available.

HemoCue® device

The HemoCue® has been extensively evaluated and is seen as the standard PoCD for this type of setting [10]. However, as the results on its accuracy are mixed, its status as standard device is not undisputed [11, 1419]. For HemoCue® Hb301, only one other study has been conducted in young children in rural low-resource settings [14].

Aims

This paper presents the first comparative pilot study of these devices in LMIC field settings in rural clinics. The study aimed to: (1) compare Hb concentrations measured in venous blood by HemoCue® Hb 301 and Aptus with a gold standard automated hematology analyzer in young children with IDA in a rural and remote community in the Upper River Region of The Gambia; (2) compare the Hb measurements provided by HemoCue® Hb301 and Aptus in capillary blood; (3) determine the usability of Aptus in these settings and 4) highlight potential improvements to the Aptus model used here before a wider global launch. This was an ancillary study embedded within the IHAT-GUT clinical trial (Registration Number: NCT02941081) [20].

Materials and methods

Devices

The specifications of the analytical devices used in this study are summarized in Table 1. The HemoCue® Hb 301 and Aptus Model 10-100-01 used in this study are both PoCDs that can measure Hb in either capillary blood, collected from a finger prick, or venous blood, diluted with appropriate anticoagulants such as EDTA. The Aptus requires a smaller minimum sample volume of 4–8 μl (Table 1). The HemoCue® is able to measure also in arterial whole blood. The HemoCue® is larger and heavier than the Aptus, and both the cuvette storage and operating temperatures have a more limited range for HemoCue®. Humidity and altitude ranges are not quantified by the manufacturer of the HemoCue®. While the Aptus has an integrated rechargeable battery, the HemoCue® can be operated either by changeable batteries or with a power outlet. The HemoCue® has connectivity features allowing collected results to be read out on a computer, which this Aptus model lacked. Both devices have a built-in self-test and are calibrated to internationally recognized standards. Because Aptus measures Hb and HCT simultaneously, it requires an extra centrifugation step and has a measurement time of 60 seconds. The HemoCue® measures only Hb in 10 seconds. Both devices utilize photometry for their measurements: the Aptus measures three wavelengths and the HemoCue® two. Centrifugation requires the Aptus to be placed on a reasonably level surface that is not disturbed by significant movement during sample processing. The Aptus does not directly measure Hb, as is the case for the HemoCue®. Rather, it measures HCT and MCHC of the cellular constituents after centrifugation, and calculates Hb by multiplying MCHC with HCT. Additionally, the Aptus can display results in relation to pre-specified reference ranges. These include the WHO and the Dacie and Lewis ranges, which can be corrected for altitude, sex, pregnancy status and age via the touchscreen [2123]. The automated hematology analyzer used as the clinical laboratory gold standard was the Medonic® M-Series M16S/M20S Autoloader (hereafter Medonic®).

thumbnail
Table 1. Specifications for HemoCue® Hb 301 and Aptus device [2123].

https://doi.org/10.1371/journal.pone.0239931.t001

To assess usability of the Aptus in field settings, a log of error messages as displayed by the Aptus was kept. These included overfilling or underfilling of cuvettes, termination of centrifugation due to tilt or vibrations of the device, and incorrect cuvette placement. These errors, combined with the user experience of the single operator of the Aptus in this study, were used to extrapolate possible improvements for the device in future model versions.

Study setting and participants

This prospective study was conducted between 8 October and 22 November 2018 and was ancillary to the IHAT-GUT trial conducted in the Bassé government area, Upper River Region (URR), The Gambia [20]. The climate is Sahelian, following a pattern of rainy season from June-October and dry season from November-May [24]. During the study period, average temperature and humidity around Bassé were 31°C with 67% humidity in October, then 29°C with 58% humidity in November [25]. The Gambia ranks 174 out of 189 on the Human Development Index [26], with the rural URR representing the most socio-economically deprived region of the country, exhibiting limited access to health services or appropriate infrastructure. This was particularly the case for the study area, located in the Wuli and Sandu districts approximately 400 km inland of the capitol Banjul, on the opposite, northern side of the river Gambia to Bassé, the only major town in the district [24]. This area is predominantly composed of subsistence farming communities from which the study participants were recruited. The Bassé region consistently scores below the national average on most morbidity and mortality measures, with the latest Demographic and Health Survey from 2011 (DHS) listing under-five mortality rates at 92 per 1000 (1 in 11) and the proportion of stunted children at 32% [24]. Furthermore, 82.5% of children in the Bassé area meet the WHO criteria for anemia of Hb< 11.0 g/dl, with 56.4% and 6.5% meeting the criteria for moderate and severe anemia, respectively [24].

All subjects were evaluated and screened for IHAT-GUT study eligibility. In the present study, this was a convenience sample of all children enrolled in the IHAT-GUT study and who attended the study clinics at the pre-specified study timepoints for blood collection. In short, participants were otherwise healthy children of 6–35 months with IDA at enrollment, defined as Hb between 7–11 g/dl and serum ferritin below 30 μg/l, from 17 villages in the study area. Exclusion criteria included severe malnutrition, severe anemia, congenital anomalies and diarrhea or dysentery. For a comprehensive list, please refer to the main study protocol [20].

Ethical issues

Ethical approval for this ancillary study to the IHAT-GUT trial [20] was obtained from The Gambia Government/MRCG Joint Ethics Committee (SCC1632, 3/10/2018). No additional intervention or participant procedures were necessary for this study. As such, written informed consent of the parent was obtained for each participant as part of the IHAT-GUT main trial before the study procedures commenced. All staff participating in recruitment were trained in GCP, informed consent procedures and translation of the provided information material into local languages. In case of illiteracy, the trial was explained in understandable terms in a local language in the presence of an impartial witness in a room with appropriate privacy. In case of literacy, parents were given sufficient time to read the information material. Sufficient time was also given to address any questions parents might have. No child took part in the trial without prior informed consent, and all participants were protected according to the Sponsor Clinical Trial/Non-Negligent Harm Insurance and Medical Malpractice Insurance. For further information we refer to the IHAT-GUT protocol [20].

Hemoglobin measurements

The IHAT-GUT study protocol defines that on study day 85, venous blood is collected instead of capillary blood. These samples were used in the present study for the comparisons between the PoCDs and the automated hematology analyzer. Venipuncture was performed by the IHAT-GUT study nurses following standardized study protocols. Briefly, the participant’s arm was wiped with 70% ethanol. A 21-gauge Multifly® needle was used for venipuncture. The samples for the Medonic® analysis were collected in an EDTA Microtainer® blood collection tube, which was continuously inverted for 30 seconds immediately after collection, thus far exceeding the recommended minimum of 10 inversions. Immediately after the blood tubes were filled, the needle was removed and the blood that remained in the needle and attached tube systems was drained on a plastic surface to form a large drop, from which both HemoCue® and Aptus samples were collected (gravity method). The EDTA tubes were subsequently transported to the laboratory site at the MRC field station in Bassé in an isolated carrying box containing cooling elements, where they were analyzed with the Medonic® within four hours of collection.

For all other study timepoints, participants visited the study clinics weekly for a finger prick performed by the nurse following standardized study protocol. These capillary blood samples were used for the comparisons between the two PoCDs, HemoCue® and Aptus. We note that the way the finger prick was performed as part of IHAT-GUT differed from what the Aptus manufacturer recommends, i.e. blood was collected from the thumb and slight pressure was applied for the blood drop to form. Briefly, the child’s thumb was wiped with 70% ethanol and pricked with a Sherwood Safety blood lancet. A little pressure was applied to the finger so that a domed drop formed. Capillary blood for both HemoCue® and Aptus was collected from separate drops of blood from the same fingerprick to minimize variability due to the collection technique. The first drop was wiped clean with cotton, the second drop was used for the measurement with the HemoCue®, and the third drop was used for the Aptus measurement by one of two devices. HemoCue® measurements were conducted by one IHAT-GUT study nurse, whereas the Aptus was exclusively operated by the present study lead author. If during screening a measurement was Hb <7 g/dl (the WHO definition of severe anemia) a venous blood sample was taken and analyzed with Medonic within four hours. If this confirmed the field measurement, the child was classified as severely anemic and stopped the study supplementation to receive standard treatment with oral iron syrups, as per Gambian National Guidelines. Therefore, both PoCDs in this study were evaluated for their usefulness as screening tools, while the analyzer served as diagnostic standard. As such, the WHO standard as cut-off was valid for both PoCDs, which is in line with manufacturers’ recommendations. Indeterminate screening results, i.e. an Hb ≥7g/dl, were thus not analyzed with the reference standard as per the IHAT-GUT protocol [20]. In the present study, both indeterminate test results and reference standards were used in the comparative analysis and handled as continuous variables and not categorized based on any Hb threshold.

HemoCue®, Aptus and Medonic® Hb measurements were all performed in accordance with the manufacturers’ operating manuals. Results were recorded on the devices and then manually in a notebook. The study nurses also collected further clinical information during field studies as pertaining to the IHAT-GUT study. The nurses, lab staff and the principal investigator performing the Aptus tests were aware of Hb reference values via their biomedical training.

Quality control (QC)

Before each measurement, the Aptus self-calibrates and self-checks during the centrifugation phase to account for environmental influences, such as temperature and humidity. The device is also able to check for underfilling and overfilling, can remove air bubbles by centrifugation, has an internal QC and displays error messages if it fails.

The HemoCue® has an internal quality control “self-test”, which the system performs automatically every time the analyzer is turned on. The test is performed at regular intervals (typically every two hours) if the analyzer remains switched on. If the self-test fails, an error code will be displayed. Additionally, the Eurotrol Hb301 external QC (levels 1 and 2) was used to verify the accuracy of HemoCue® Hb301 devices daily.

For the Medonic®, the external QCs were run at the start of each analysis day. If the QC results were within range, then the study samples were tested. The laboratory supervisor approved the QC results daily. For external quality control, the laboratory participates in an external inter-laboratory proficiency scheme with OneWorld Canada for hematology full blood analysis and has good clinical laboratory practice (GCLP) accreditation by Qualogy Ltd (UK).

Data analysis

Aptus displayed error messages in 7 out of 190 venous blood measurements (4%) and 108 out of 615 capillary blood measurements (17.6%) (Fig 1). These were due to under- or over-filling of the cuvettes and are expected in such high throughput settings with inexperienced users. HemoCue® did not display error messages during this study, irrespective of how well the cuvettes were filled. Medonic®’s error measurements are repeated in the lab and, therefore, no further sample measurements had to be excluded from the analysis presented here. Of the 691 valid sample measurements, 183 were collected from venous blood and 508 from capillary blood (Fig 1).

thumbnail
Fig 1. Flowchart of study design.

Excluded samples (dashed rectangles), total of Aptus (rounded rectangles) and HemoCue® (rectangles) data points used for the statistical analyses are indicated. For venous blood, both Aptus and HemoCue® samples were taken during venipuncture and matched with the corresponding Medonic® measurement from the same sample tube. For capillary blood, every Aptus data point was matched with the corresponding HemoCue® measurement.

https://doi.org/10.1371/journal.pone.0239931.g001

As this study occurred between study day 57 and 113 of the IHAT-GUT trial, some participants had developed severe anemia since the beginning of the study (Tables 2 and 3) and were subsequently discontinued from supplementation in the IHAT-GUT trial to receive standard treatment. Since this study presents a methodological comparison of the devices, the clinical consequences of the measurements do not pertain to our results and these low Hb values were therefore not excluded from our analysis.

thumbnail
Table 3. Blood hemoglobin concentrations as determined by Aptus, HemoCue®, and Medonic® in venous blood samples.

https://doi.org/10.1371/journal.pone.0239931.t003

Sample size for IHAT-GUT was determined as per the published study protocol [20]. The present study is an exploratory method comparison pilot study. As such, as there is no available performance data of Aptus in field settings, no data is available to perform a meaningful power or sample size calculation.

SPSS Statistics version 25 (IBM corp.) was used for data analysis and visualization. Using three times IQR as cutoff, three Aptus measurements (15.4g/dl, 18.5g/dl and 20.7g/dl) from venous blood were considered outliers and removed from further analysis, leaving 180 venous blood measurements for analysis (Fig 1). Two Aptus Hb measurements (15.9g/dl and 18.6g/dl) in capillary blood were considered outliers, and 506 capillary blood measurements were used for the final analysis (Fig 1). Normality was confirmed for the resulting datasets using the Kolmogorov-Smirnov test. Paired t-tests and repeat measures ANOVA with the Greenhouse-Geisser correction and Bonferroni post-hoc analysis were used for comparisons between groups. Computation of Lin’s Concordance Correlation Coefficient (abbreviated rc) was conducted using a macro S1 Macro. Lin’s coefficient analysis was conducted for the comparisons with the automated analyzer in venous blood as it measures both association and agreement. It represents a modification of Pearson’s correlation coefficient, in that it assesses not merely the distance of data points to the line of best fit, but also how far this line deviates from the line of perfect agreement, as represented by the 45-degree line through the origin. Lin’s coefficient has a range of -1 to 1, with rc = 1 representing perfect agreement, in which all data points lie on the 45-degree line. As points either depart from this line, or as the line of best fit departs from the 45-degree line, the rc value diminishes [27, 28].

Results

Demographics

A total of 805 samples (190 venous blood and 615 capillary blood) were collected from 223 study participants enrolled in cohort 3 of the IHAT-GUT trial (Table 2). After excluding samples with erroneous measurements (see above) and data outliers, a total of 686 Aptus Hb measurements were included in the final dataset (Fig 1).

Venous blood Hb concentration by Aptus and HemoCue® compared to the automated hematology analyzer

Each measurement included in this set of comparisons (Table 3) is from an individual child. The mean Hb concentrations obtained by the Aptus (10.4±1.0 g/dl) and HemoCue® (10.6±0.9 g/dl) devices were not statistically different (Table 3, p = 0.15). The mean difference in Hb between Aptus and HemoCue® Hb measurements in venous blood was -0.11 (95% CI: -0.23, 0.00) g/dl (Fig 2).

thumbnail
Fig 2. Histograms of the differences in hemoglobin (Hb) concentration measured in venous blood samples.

Aptus-HemoCue® (panel A) and Aptus-Medonic® (panel B) comparisons, n = 180; HemoCue®-Medonic® (panel C), n = 183. The frequency of each difference is represented as a % of total measurements. For these comparisons, 151 (83.9%) of values in panel 1, 114 (63.3%) in panel 2 and 127 (69.4%) in panel 3 showed differences in Hb concentration between -1 and 1 g/dl.

https://doi.org/10.1371/journal.pone.0239931.g002

Both PoCDs measured significantly greater Hb concentrations than the Hb value obtained by the Medonic® automated hematology analyzer (Table 3, p<0.0001 for both comparisons). The mean difference between Hb concentration measured by Aptus vs Medonic® was 0.69 (95% CI: 0.57, 0.81) g/dl and that of HemoCue® vs Medonic® was 0.81 (95% CI: 0.73, 0.88) g/dl (Fig 2). In both cases more than 60% of measurements showed differences in Hb concentration between -1 and 1 g/dl (Fig 2).

Bland-Altman plots for the Hb concentration comparisons in venous blood samples are shown in Fig 3. There is very little bias between the two PoCDs for the measurement of Hb concentration with mean difference of -0.11 (95% CI: -0.228, 0.00) g/dl (Fig 3A). The Aptus measurements (Fig 3B) show slightly broader limits of agreement than the HemoCue® measurements (Fig 3C) for the comparison with the Medonic® automated hematology analyzer. The biases for Aptus of 0.69 (95% CI: 0.571, 0.813) g/dl (Fig 3B) and for HemoCue® of 0.81 (95% CI: 0.733, 0.879) g/dl (Fig 3C) compared to the Medonic® are similar.

thumbnail
Fig 3. Bland-Altman comparison of the hemoglobin measurements performed in venous blood samples.

Data represents the difference versus the average Hb results between Aptus-HemoCue® (panel A), Aptus-Medonic® (panel B) and HemoCue®-Medonic® (panel C). The solid lines represent the mean difference between the measurements (bias), while the dotted lines indicate the 95% limits of agreement between methods.

https://doi.org/10.1371/journal.pone.0239931.g003

The Desirable Biological Variation Database specifications propose a total error allowable (TEa) for Hb of ±4.19% from the true value as acceptable for a device measuring hemoglobin [29]. Taking the Medonic® measurements as the true value, the Aptus measured 122 measurements (68%) and the HemoCue® 159 measurements (88%) outside of this range.

Overall, if we consider the commonly used clinically meaningful difference in Hb concentrations of ± 1 g/dl (29), in the Aptus vs HemoCue® comparisons, 83.9% of values fell within this 1 g/dl threshold, for Aptus vs Medonic® this was 63.3% of values and for HemoCue® vs Medonic®, it was 69.4% of values.

Lin’s coefficient analysis plots are shown in Fig 4. The correlation coefficient (rc) for the comparison of Hb concentration determined by Aptus vs HemoCue® was rc = 0.688 (95% CI: 0.604, 0.756), for Aptus vs Medonic® rc = 0.548 (95% CI: 0.458, 0.627) and for HemoCue® vs Medonic® rc = 0.636 (95% CI: 0.571, 0.694). For all of these comparisons, there is a positive correlation between methods but the agreement is sub-optimal. Data for the comparison of hematocrit measurements in venous blood between Aptus and Medonic® are shown in S1 Table and S1 and S2 Figs.

thumbnail
Fig 4. Lin’s Concordance Correlation Coefficient graphs for venous Hb comparison.

The dotted line represents the 45-degree line of perfect agreement through the origin, while the solid blue line is the line of best fit. As the Aptus-HemoCue® panel 1 (n = 180) and Aptus-Medonic® panel 2 (n = 180) indicate, the Aptus data has a more pronounced spread in both graphs. Furthermore, the line of best fit deviates significantly from the 45-degree line than in the HemoCue®-Medonic® comparison, indicating weaker agreement. Conversely, the spread of data and angle of the line of best fit in the HemoCue®-Medonic® comparison in panel 2 (n = 183) illustrate that agreement between both PoCDs is stronger compared to the Aptus.

https://doi.org/10.1371/journal.pone.0239931.g004

A ROC analysis was also conducted on the venous blood comparison. As there was only one confirmed case of severe anemia and the median values across all devices were close to or above the threshold for moderate anemia of 10 g/dl (Table 3), these were deemed unusable as an outcome measure. Instead, absence of anemia (Hb ≥ 11g/dl) was taken as a binary outcome measure to compare PoCDs with the lab analyzer, using the reference standard outcome as the true measure. As can be seen in Fig 5, HemoCue® was superior, with an AUC of 0.933 (Std. Error: 0.023, 95% CI: 0.878, 0.989) compared with the Aptus’ AUC of 0.799 (Std. Error: 0.06, 95% CI: 0.681, 0.916).

thumbnail
Fig 5. ROC curve comparison of Aptus and HemoCue®.

HemoCue® (solid line) proved superior to Aptus (dashed line) in accuracy of measuring non-anemia, with an AUC of 0.933 compared to 0.799, respectively.

https://doi.org/10.1371/journal.pone.0239931.g005

Hb concentration measured by two PoCDs, Aptus and HemoCue®, in capillary blood

A total of 506 measurements from 223 children were included in this set of comparisons. Mean Hb concentrations measured in capillary blood by Aptus (10.33±1.11g/dl) and HemoCue® (10.01±1.07g/dl) were statistically different (Table 4, p<0.0001). The mean difference between Aptus and HemoCue® Hb concentration was 0.32 (95% CI 0.22–0.42) g/dl (Table 4). There were 170 out of 506 (33.6%) measurements where the difference between the two measurements was above the ± 1g/dl clinically significant threshold.

thumbnail
Table 4. Blood hemoglobin concentrations as determined by Aptus and HemoCue® in capillary blood samples.

https://doi.org/10.1371/journal.pone.0239931.t004

The Bland-Altman plot for the comparison between the two PoCDs is shown in Fig 6. There is no significant relationship between the bias and the average Hb result. The distance between the limits of agreement (-1.91g/dl, 2.54g/dl) is comparatively wide at 4.45 g/dl.

thumbnail
Fig 6. Bland-Altman comparison of the hemoglobin measurements performed in capillary blood samples by Aptus and HemoCue®.

Data represents the difference (Aptus–HemoCue®) versus the average Hb results. The solid lines represent the mean difference between the measurements (bias), while the dotted lines indicate the 95% limits of agreement between methods.

https://doi.org/10.1371/journal.pone.0239931.g006

Aptus device usage

The Aptus device proved easily portable and intuitive in use. The two Aptus devices used in this study were sufficient to cover the maximum of 85 participants per day. However, as photometric measurement with Aptus only commences after centrifugation has completed, the time-to-result is significantly longer in Aptus than HemoCue® (Table 1). This could be a drawback in very high-throughput settings when there is not much turnaround time between patients being screened. Additionally, the centrifugation process is sensitive to both significant vibrations and spatial orientation, which led to instances in which a measurement was not initiated or aborted. This did not pose a major impediment to normal usage when the manufacturer’s recommendations were followed. Moreover, errors in sampling are only detected with photometry, which in the majority of cases made it impossible to collect a second sample from the same participant, as they had already been processed by the IHAT-GUT nurses and left the clinic. Errors in capillary blood sampling most commonly occurred due to underfilling of the cuvette. The manufacturer’s instructions recommend that the capillary blood should form a dome shaped droplet of sufficient size before attempting to fill the cuvette. However, many children were distressed by the fingerprick, making it difficult to produce a droplet and appropriately fill the Aptus cuvette with the resulting blood film formed on the finger. This would probably be improved when collecting blood in adults, with more experienced users or improved cuvettes. This is in contrast to the HemoCue® cuvette, which draws blood on contact in any position and the device does not indicate errors from under- or over-filling. Contrarily, when placing a drop of venous blood from the needle on a plastic surface, the most common error with the Aptus was overfilling.

Adverse events

Besides the observation that blood drawing caused some discomfort to the children, the study reported no adverse events related to blood collection (capillary or venous). Abnormal blood results, i.e. severe anemia (Hb<7 g/dl) were identified in 0 samples with HemoCue® and 3 samples with Aptus. As the Aptus comparison was ancillary to the IHAT-GUT study, its results did not trigger a subsequent analysis for severe anemia by Medonic® as would have been the case if severe anemia was identified by HemoCue®. However, during this study there was one identified severe anemia case with Medonic® measurements collected as part of the outcome assessment timepoints. As per IHAT-GUT protocol this participant was discontinued from the IHAT-GUT supplementation and received standard treatment.

Discussion

The large impact and preventable nature of anemia warrant large scale screen-and-treat efforts, and cost-efficient and accurate PoCDs are a crucial element in this. The Aptus represents a new development by measuring Hb and HCT concentration. This study assessed the accuracy and usability of the new Aptus PoCD and the well-established HemoCue® 301 PoCD against the automated hematology analyzer Medonic® M-series, in an ancillary study to the IHAT-GUT clinical trial set in a community setting in rural The Gambia [20]. To our knowledge, this study is the first to test the accuracy of Aptus in a low-income rural setting in children.

Overall, both PoCDs showed similar levels of agreement with the laboratory analyzer, however Aptus showed slightly lower bias and wider limits of agreement than the HemoCue®. Both PoCDs were shown to overestimate Hb concentration, as has been shown in previous studies with HemoCue® in children in similar settings [14, 30].

Venous blood measurements exhibited better agreement both between the PoCDs and with the laboratory analyzer in comparison to the agreement observed between the PoCDs in capillary blood. This is in line with previous research that shows PoCD agreement to be inferior in capillary sampling [7, 14, 30], which is unsurprising given the mixed venous/arterial nature of capillary blood [31], within-subject variation between drops from the same finger prick [32] and dependency on proper technique for consistency of results [33]. This is reflected in the present study by wider limits of agreement in the Bland-Altman plots, the difference of which far exceeds the clinically acceptable inaccuracy of ± 1g/dl. This indicates that the within-subject variation of measurements is large and the methods are thus not unequivocal in a clinically relevant manner. Furthermore, SD values were larger for the capillary samples despite the much larger sample size than in the venous comparison, thereby also indicating more variability of results. The method used to obtain a PoCD sample from the venous blood by placing a drop on a surface first (gravity method), might be of influence here, as it was found to be more accurate than collection from finger prick in a study on the HemoCue® in Rwanda [30]. While this has not been previously evaluated for Aptus, it could partly explain the superior results in venous sampling for both PoCDs. However, whereas both devices are capable of measuring Hb in venous blood, their principal function is to be used as a rapid test in capillary blood collected from a fingerprick, particularly in low-resource field settings where PoCDs largely function as a primary diagnostic tool. Therefore, although direct comparison between the PoCDs and the laboratory analyzer from one venous sample clearly minimized biases, and is thus the most suitable method for comparing accuracy, this study also included the capillary blood comparison between the two PoCDs as this better represents their intended use.

The overestimation of Hb concentration by both PoCDs could translate to underestimating anemia rates when these devices are used as the only diagnosis tool. Averaging multiple capillary measurements, either by using multiple cuvettes [34] or collecting multiple drops of blood in a tube and sampling from it [32], have been proposed to eliminate variation and improve accuracy. However, such measures require both more time and materials, thereby making them less feasible in LMICs and high-throughput screenings in particular. Alternative non-invasive devices have not yet been shown to be consistently reliable [6, 35, 36], so more advanced PoCD capabilities—such as the MCHC measurement of the Aptus—would improve the value of invasive procedures. This holds for low-resource settings in particular, where full blood count analysis is often unavailable and PoCDs are the modus operandi.

Thus, usability and appropriateness to such settings are important considerations. Whereas the HemoCue® series has long been recommended as standard for field settings in low resource areas due to its portability, cost and user-friendliness [14], its performance is not without criticism as studies on it provide mixed results [7, 14, 15, 18, 30, 3739]. Reasons for this variability are thought to include, among others, climatological factors, handling and storage of the cuvettes, and sampling technique [11, 15, 40]. Furthermore, the lack of a specified working range for both altitude and humidity is suboptimal, as both have been shown to influence reliability [39, 40]. Regardless of these constraints, the usage of the HemoCue® in this study proved simple, rapid, and without noteworthy malfunctions.

To our knowledge, this is the first instance in which this Aptus model was tested in high-throughput and low-resource community settings in children. Aptus is far smaller in dimension and lighter than the HemoCue®, making it more portable, and has validated wide ranges of altitude, temperature and humidity. A major advantage of the Aptus is that it measures MCHC and HCT directly. As MCHC has intrinsic diagnostic value, it would be beneficial to have the option to display MCHC directly post-measurement. This is particularly relevant since this study revealed poorer agreement in the HCT values than for the calculated Hb values (Figs S2 and 4, respectively), which is interesting as the Aptus uses the gold standard microhematocrit method of determining hematocrit by centrifugation [41] whereas the flow-cytometry based models like the Medonic® calculate HCT. This suggests that the MCHC has a strong corrective influence in the Hb calculation, and is thus an important variable to show the device’s accuracy. Future studies should investigate this further.

The usage proved intuitive and simple, yet some limitations apply specifically to these settings, namely the longer duration of measurement and difficulty in handling the cuvettes. If the intended use is high-throughput screenings in rural field clinics, then the comparatively longer measurement time due to centrifugation may be a disadvantage in comparison to HemoCue®. A further consideration is that the battery of the Aptus model used was not replaceable and the lack of a consistent power supply for recharging batteries in this type of setting could lead to issues with continuous usage.

Based on field experience and statistical analysis, the Aptus appears comparable in measurement capabilities to the standard of the HemoCue® given the limitations cited. It is stressed that the Aptus results may have been confounded by not adhering strictly to the manufacturer’s specifications. This included the frequent inability to obtain a large dome-shaped blood droplet in the young children, use of the thumb for finger pricking instead of the second or third digit, and squeezing of the finger in order to procure a sufficient amount of blood from the child. This blood sampling procedure is discouraged since the resulting expulsion of interstitial fluid can skew the measurement [33]. However, as the HemoCue® sample was taken using the same technique, a resulting bias can be assumed to be present in both measurements. As the HemoCue® and Aptus samples were always taken from the second and third drop of blood produced, respectively, this bias would be comparable within measurements per device, while it is still possible that the composition between second and third drop may have varied due to this technique. Furthermore, the larger number of error messages observed with Aptus was related to the lower level of experience with this device than with HemoCue®, which we note as a limitation. Furthermore, the Aptus was operated by only one user, which could have introduced bias, whereas HemoCue® sampling was conducted by one of the five study nurses who all had been using the device for ten months prior to this study, through which possible biases would have been balanced and sampling technique could have been superior due to increased experience with the device.

Overall, to improve usability of Aptus in high throughput settings by a wide range of users, experienced or inexperienced, we recommend improvements to both the cuvettes and the timing of the error messages in the device, and the incorporation of additional battery charging capabilities. We note that the Aptus model used in this study is one of the first available commercially and, as a result of this study and feedback from other users, has undergone further development to improve its accuracy, software interface and cuvette handling.

Conclusion

This was the first study of the Aptus device in this type of setting and was intended to inform further development in terms of performance before wider adoption in resource-poor rural areas. Aptus and HemoCue® showed similar accuracy and both overestimated Hb levels, which could lead to an underestimation of anemia prevalence when used as a screening or diagnostic tool. Aptus showed promise with its operation unimpaired by the field clinic’s conditions, but operational limitations concerning duration of measurements may make it less suitable for high-throughput screening. Importantly, the additional diagnostic value of measuring MCHC may make Aptus a superior choice if the improvements highlighted in this study are achieved. Since MCHC analysis was beyond the scope of this paper, future studies should investigate this. By broadening the range of validated PoCDs available to users, we hope that anemia diagnosis and screening programs improve coverage for the benefit of billions of people around the world.

Supporting information

S1 Macro. SPSS macro for Lin’s Concordance Correlation Coefficient.

Macro used in the study to calculate the Lin’s coefficient in SPSS. Available from: https://gjyp.nl/marta/Lin.sps. Date accessed: 23.04.2019.

https://doi.org/10.1371/journal.pone.0239931.s002

(DOCX)

S1 Table. Comparison of hematocrit (HCT) measurements between Aptus and Medonic®.

Measurements in the Aptus were conducted in blood drops taken from a venous blood sample. Al HCT values given as percentage. The mean difference between the methods is 5.11 (95% CI: 4.76, 5.45) %, which was statistically significant (p<0.0001).

https://doi.org/10.1371/journal.pone.0239931.s003

(DOCX)

S1 Fig. Bland-Altman plot for hematocrit (HCT) measured in venous blood by Aptus and Medonic®.

Data represents the difference (Aptus–Medonic®) versus the average HCT results. The solid lines represent the mean difference between the measurements (bias) is 5.1%, while the dotted lines indicate the 95% limits of agreement between methods of 0.5(lower) and 9.71% (upper) limit.

https://doi.org/10.1371/journal.pone.0239931.s004

(TIF)

S2 Fig. Lin’s Concordance Correlation Coefficient for hematocrit (HCT) measured in venous blood by Aptus and Medonic®.

The dotted line represents the 45-degree line of perfect agreement through the origin, while the solid blue line is the line of best fit. Lin’s coefficient is rc = 0.285, and there is significant divergence from the 45-degree line and spread around the line of best fit (S2 Fig). It must be noted that the Aptus® does not provide decimals for HCT while the Medonic® does. This explains the columnar nature of the data in both plots, and negatively influences agreement due to an inherently lower accuracy of the Aptus®’s output.

https://doi.org/10.1371/journal.pone.0239931.s005

(TIF)

Acknowledgments

We would like to sincerely thank all participants and their families for their participation and the local communities for welcoming us to conduct this study. We would like to thank Entia Ltd for supplying the Aptus devices and consumables, for user training, and for reviewing the final draft of this manuscript.

References

  1. 1. World Health Organization. The Global Prevalence of Anaemia in 2011 [Internet]. WHO Report. Geneva; 2011. Available from: http://apps.who.int/iris/bitstream/10665/177094/1/9789241564960_eng.pdf?ua=1
  2. 2. Pasricha SR, Hayes E, Kalumba K, Biggs BA. Effect of daily iron supplementation on health in children aged 4–23 months: A systematic review and meta-analysis of randomised controlled trials. Lancet Glob Heal. 2013;1(2):e77–86.
  3. 3. Prentice AM, Mendoza YA, Pereira D, Cerami C, Wegmuller R, Constable A, et al. Dietary strategies for improving iron status: balancing safety and efficacy. Nutr Rev. 2017 Jan 14;75(1):49–60. pmid:27974599
  4. 4. Lozoff B. Iron Deficiency and Child Development. Food Nutr Bull. 2007 Dec 2;28(4_suppl4):S560–71.
  5. 5. Engle-Stone R, Aaron GJ, Huang J, Wirth JP, Namaste SML, Williams AM, et al. Predictors of anemia in preschool children: Biomarkers Reflecting Inflammation and Nutritional Determinants of Anemia (BRINDA) project. Am J Clin Nutr. 2017 Jul;106(Suppl 1):402S–415S. pmid:28615260
  6. 6. Vos T, Abajobir AA, Abbafati C, Abbas KM, Abate KH, Abd-Allah F, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990–2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2017;390(10100):1211–59. pmid:28919117
  7. 7. Whitehead RD, Zhang M, Sternberg MR, Schleicher RL, Drammeh B, Mapango C, et al. Effects of preanalytical factors on hemoglobin measurement: A comparison of two HemoCue® point-of-care analyzers. Clin Biochem. 2017 Jun 1;50(9):513–20. pmid:28412284
  8. 8. Hoffbrand A, Moss P. Essential Haematology. 6th Editio. Essential Haematology. Chichester: Wiley-Blackwell; 2011.
  9. 9. Quintó L, Aponte JJ, Menéndez C, Sacarlal J, Aide P, Espasa M, et al. Relationship between haemoglobin and haematocrit in the definition of anaemia. Trop Med Int Heal. 2006 Aug;11(8):1295–302.
  10. 10. ICF International. Biomarker field manual: Demographic and health surveys methodology [Internet]. Calverton; 2012. Available from: https://dhsprogram.com/pubs/pdf/DHSM7/DHS6_Biomarker_Manual_9Jan2012.pdf
  11. 11. Rappaport AI, Karakochuk CD, Whitfield KC, Kheang KM, Green TJ. A method comparison study between two hemoglobinometer models (Hemocue Hb 301 and Hb 201+) to measure hemoglobin concentrations and estimate anemia prevalence among women in Preah Vihear, Cambodia. Int J Lab Hematol. 2017 Feb;39(1):95–100. pmid:27997764
  12. 12. Shah N, Osea EA, Martinez GJ. Accuracy of noninvasive hemoglobin and invasive point-of-care hemoglobin testing compared with a laboratory analyzer. Int J Lab Hematol. 2014 Feb;36(1):56–61. pmid:23809685
  13. 13. Entia Ltd. Aptus Information Booklet. London; 2018.
  14. 14. Hinnouho G-M, Barffour MA, Wessells KR, Brown KH, Kounnavong S, Chanhthavong B, et al. Comparison of haemoglobin assessments by HemoCue and two automated haematology analysers in young Laotian children. J Clin Pathol. 2018 Jun;71(6):532–8. pmid:29197856
  15. 15. Rappaport AI, Barr SI, Green TJ, Karakochuk CD. Variation in haemoglobin measurement across different HemoCue devices and device operators in rural Cambodia. J Clin Pathol. 2017 Jul;70(7):615–8. pmid:28275044
  16. 16. Rudolf-Oliveira RCM, Gonçalves KT, Martignago ML, Mengatto V, Gaspar PC, Ferreira J dos S, et al. Comparison between two portable hemoglobinometers and a reference method to verify the reliability of screening in blood donors. Transfus Apher Sci. 2013 Dec;49(3):578–82. pmid:24119714
  17. 17. Karakochuk CD, Janmohamed A, Whitfield KC, Barr SI, Vercauteren SM, Kroeun H, et al. Evaluation of two methods to measure hemoglobin concentration among women with genetic hemoglobin disorders in Cambodia: A method-comparison study. Clin Chim Acta. 2015 Feb;441:148–55. pmid:25542983
  18. 18. Adam I, Ahmed S, Mahmoud MH, Yassin MI. Comparison of HemoCue® hemoglobin-meter and automated hematology analyzer in measurement of hemoglobin levels in pregnant women at Khartoum hospital, Sudan. Diagn Pathol. 2012;7(1):30.
  19. 19. Sanchis-Gomar F, Cortell-Ballester J, Pareja-Galeano H, Banfi G, Lippi G. Hemoglobin Point-of-Care Testing: The HemoCue System. J Lab Autom. 2013 Jun 6;18(3):198–205. pmid:22961038
  20. 20. Pereira DIA, Mohammed NI, Ofordile O, Camara F, Baldeh B, Mendy T, et al. A novel nano-iron supplement to safely combat iron deficiency and anaemia in young children: The IHAT-GUT double-blind, randomised, placebo-controlled trial protocol. Gates Open Res. 2018 Sep 21;2(0):48.
  21. 21. Morris LD, Osei-Bimpong A, McKeown D, Roper D, Lewis SM. Evaluation of the utility of the HemoCue 301 haemoglobinometer for blood donor screening. Vox Sang. 2007 Jul;93(1):64–9. pmid:17547567
  22. 22. HemoCue America. HemoCue Hb 301 System [Internet]. 2014 [cited 2019 Feb 27]. Available from: http://pdf.medicalexpo.com/pdf/hemocue/hemocue-hb-301-system/83757-139640.html
  23. 23. Entia. Aptus Operating Manual 10-100-01. 2018.
  24. 24. The Gambia Bureau of Statistics. Demographic and Health Survey 2013 [Internet]. Banjul; 2014. Available from: https://dhsprogram.com/publications/publication-FR289-DHS-Final-Reports.cfm
  25. 25. timeanddate.com. Past Weather in Basse Santa Su, Gambia [Internet]. 2020 [cited 2020 May 29]. Available from: https://www.timeanddate.com/weather/@2413818/historic?month=10&year=2018
  26. 26. UNDP. Human Development Indices and Indicators: 2018 Statistical Update [Internet]. 2018. Available from: http://hdr.undp.org/sites/default/files/2018_human_development_statistical_update.pdf
  27. 27. Watson PF, Petrie A. Method agreement analysis: A review of correct methodology. Theriogenology. 2010 Jun;73(9):1167–79. pmid:20138353
  28. 28. Lin LI. A Concordance Correlation Coefficient to Evaluate Reproducibility. Biometrics. 1989;45(1):255–68. pmid:2720055
  29. 29. Westgard QC. Desirable Biological Variation Database specifications [Internet]. 2014 [cited 2020 May 27]. Available from: https://www.westgard.com/biodatabase1.htm#1
  30. 30. Parker M, Han Z, Abu-Haydar E, Matsiko E, Iyakaremye D, Tuyisenge L, et al. An evaluation of hemoglobin measurement tools and their accuracy and reliability when screening for child anemia in Rwanda: A randomized study. Lam W, editor. PLoS One. 2018 Jan 4;13(1):e0187663. pmid:29300737
  31. 31. Kupke IR, Kather B, Zeugner S. On the composition of capillary and venous blood serum. Clin Chim Acta. 1981;112(2):177–85. pmid:7237825
  32. 32. Bond MM, Richards-Kortum RR. Drop-to-Drop Variation in the Cellular Components of Fingerprick Blood. Am J Clin Pathol. 2015;144(6):885–94. pmid:26572995
  33. 33. World Health Organization. WHO guidelines on drawing blood: best practices in phlebotomy. World Health Organization. Geneva; 2010.
  34. 34. Gehring H, Hornberger C, Dibbelt L, Roth-Isigkeit A, Gerlach K, Schumacher J, et al. Accuracy of point-of-care-testing (POCT) for determining hemoglobin concentrations. Acta Anaesthesiol Scand. 2002;46(8):980–6. pmid:12190799
  35. 35. von Schweinitz BA, De Lorenzo RA, Cuenca PJ, Anschutz RL, Allen PB. Does a non-invasive hemoglobin monitor correlate with a venous blood sample in the acutely ill? Intern Emerg Med. 2014;10(1):55–61. pmid:25322853
  36. 36. Gayat E, Aulagnier J, Matthieu E, Boisson M, Fischler M. Non-invasive measurement of hemoglobin: Assessment of two different point-of-care technologies. PLoS One. 2012;7(1):1–8.
  37. 37. Kolotiniuk N V., Manecke GR, Pinsky MR, Banks D. Measures of Blood Hemoglobin and Hematocrit During Cardiac Surgery: Comparison of Three Point-of-Care Devices. J Cardiothorac Vasc Anesth. 2018 Aug;32(4):1638–41. pmid:29276094
  38. 38. Baart AM, de Kort WLAM, van den Hurk K, Pasker-de Jong PCM. Hemoglobin assessment: precision and practicability evaluated in the Netherlands-the HAPPEN study. Transfusion. 2016 Aug;56(8):1984–93. pmid:26968697
  39. 39. Zhou X, Xing Y, Yan H, Dang S, Zhuoma B, Wang D. Evaluation of a portable hemoglobin photometer in pregnant women in a high altitude area: A pilot study. BMC Public Health. 2009;9:1–6. pmid:19121216
  40. 40. Nguyen HTH. High humidity affects HemoCue cuvette function and HemoCue haemoglobin estimation in tropical Australia [4]. J Paediatr Child Health. 2002 Aug;38(4):427–8. pmid:12174016
  41. 41. Gebretsadkan G, Tessema K, Ambachew H, Birhaneselassie M. The Comparison between Microhematocrit and Automated Methods for Hematocrit Determination. Int J Blood Res Disord. 2015 Jun 30;2(1):1–3.