http://orcid.org/0000-0003-1515-7759
Figures
Abstract
Early diagnosis of chronic hepatitis B virus (HBV) and hepatitis C virus (HCV) infections is pivotal for optimal disease management. Sensitivity and specificity of 19 rapid diagnostic test (RDT) kits by different manufacturers (ABON, CTK Biotech, Cypress Diagnostics, Green Gross, Human Diagnostic, Humasis, InTec, OraSure, SD Bioline, Wondfo) were assessed on serum samples of 270 Mongolians (90 seropositive for hepatitis B surface antigen (HBsAg), 90 seropositive for hepatitis C antibody (HCV-Ab), 90 healthy subjects). All tested RDTs for detection of HBsAg performed with average sensitivities and specificities of 100% and 99%, respectively. Albeit, overall sensitivity and specificity of RDTs for detection of HCV-Ab was somewhat lower compared to that of HBsAg RDTs (average sensitivity 98.9%, average specificity 96.7%). Specificity of RDTs for detection of HCV-Ab was dramatically lower among HBsAg positive individuals, who were 10.2 times more likely to show false positive test results. The results of our prospective study demonstrate that inexpensive, easy to handle RDTs are a promising tool in effective HBV- and HCV-screening especially in resource-limited settings.
Citation: Jargalsaikhan G, Eichner M, Boldbaatar D, Bat-Ulzii P, Lkhagva-Ochir O, Oidovsambuu O, et al. (2020) Sensitivity and specificity of commercially available rapid diagnostic tests for viral hepatitis B and C screening in serum samples. PLoS ONE 15(7): e0235036. https://doi.org/10.1371/journal.pone.0235036
Editor: Isabelle Chemin, Centre de Recherche en Cancerologie de Lyon, FRANCE
Received: September 9, 2019; Accepted: May 15, 2020; Published: July 15, 2020
Copyright: © 2020 Jargalsaikhan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The file containing the raw data is available from figshare.com (https://doi.org/10.6084/m9.figshare.9724157).
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
With about 1.4 million annual deaths, viral hepatitis is a major problem in global health [1,2]. Most deaths are attributable to chronic hepatitis B virus (HBV) and hepatitis C virus (HCV) infections and their long-term complications–liver cirrhosis and hepatocellular carcinoma. Early diagnosis and linkage to care is pivotal to prevent these complications. However, as early stages of HBV and HCV infection are often asymptomatic, few people are diagnosed early. In 2015, about 9% of 257 million individuals living with chronic HBV and 20% of the 71 million with chronic HCV infection were aware of their disease status globally [1].
For reducing the global burden of hepatitis, identifying those who are infected is crucial. This is especially true for HCV, since in recent years highly effective direct acting antiagents (DAAs) have become available as a reliable cure against the disease [3]. Similarly, for HBV, antiviral treatment using nucleoside analogues are not only effective in inhibiting the progress of the disease, but also in preventing the transmission of the disease [4,5]. However, for diagnostics, costs, local access and ease of use are important to reach a large population.
Especially in low- to middle-income countries like Mongolia with high HBV and HCV prevalence [6], reliable rapid diagnostic tests (RTD) represent a promising alternative to standard testing methods like enzyme linked immunosorbent assay (ELISA) for initial screening. RDTs are cheaper, quicker to perform and require less skill and instrumentation. Performance of RDTs for hepatitis B surface antigen (HBsAg) and hepatitis C antibody (HCV-Ab) detection by different manufacturers has been reported to vary [7–10]. Therefore, the performance of RDTs should be carefully assessed before use in clinical practice.
The aim of this study was to evaluate the diagnostic performance of commercially available RDTs for HBsAg and HCV-Ab detection.
Materials and methods
Patients
270 participants were prospectively recruited: 90 HBsAg positive, 90 with detectable HCV viral load (HCV-RNA positive), 90 healthy controls. The sample size of 90 participants per group is a compromise between costs and statistic accuracy, to which the reliability of RDTs can be determined.
HBsAg or HCV-RNA positive participants were randomly selected from screening registry of the Liver Center, Ulaanbaatar, Mongolia. Inclusion criteria: ≥ 18 years, positive tests for HBsAg or HCV-RNA within the year prior to the study. Patients with dual infection were excluded.
Healthy controls were randomly selected among blood donors at the National Center of Transfusion Medicine, Ulaanbaatar, Mongolia. Inclusion criterium: Three or more blood donations, ensuring that these participants were confirmed negative for HBsAg, HCV-Ab and other common chronic infectious diseases multiple times.
Ethical approval was obtained from the Ethics Committee of the Ministry of Health, Mongolia (approval number 14-12/1A). Each individual gave written informed consent prior to participation.
Performance of study and laboratory measurements
Participants were asked to provide a blood sample at the Liver Center or the National Center of Transfusion Medicine, respectively, between April and July 2015. From each individual, two samples of 5 ml venous blood were collected into vials containing clotting agent (Greetmed, Vacuum Tube Clot Activator). All further processing and testing of blood samples was performed at the Liver Center. Within 4 hours after blood drawn, serum was separated by centrifugation (1110 × g; 5 min; room temperature) and stored at -80° C until reference or index testing.
As reference, HBsAg and HCV-Ab status of all serum samples was determined by ELISA (DiaPro HBsAg and HCV-Ab 3rd generation ELISA) following manufacturer’s instructions. All samples were further checked by fully automated quantitative RT-PCR (Abbot, m2000) for quantitation of HBV-DNA and HCV-RNA according to manufacturer’s instructions. For cost reasons, a total of 90 seronegative samples of healthy controls was analyzed by RT-PCR as 3 pooled samples (3×30). This decreases sensitivity by a factor of 30.
We assessed 9 RDTs for HBsAg, 10 for HCV-Ab detection from different manufacturers (Table 1).
Prices are given as payed when purchased in 2015 in small quantities by local distributors in Mongolia. All manufacturers, except OraSure, offer RDTs for HBsAg and for HCV-Ab.
Most RDTs were purchased by local distribution partners, which were not aware that tests were to be used for quality assessment. Therefore, the shown prices represent end-customer prices for low quantities in early 2015 in Mongolia. OraSure and InTec were donated by manufacturers.
All RDTs–except those from InTec–were performed simultaneously according to manufacturer’s instructions. Fresh aliquots of serum were defrosted at room temperature and samples applied to test wells by pipetting. This was done for all 270 samples by end of July 2015. Final test outcome was subject to visual inspection by the researcher. Kits from InTec were only later (in February 2017) included upon request of the manufacturer. In all cases, the researcher was aware of the sample type and test he or she was assessing.
Data analysis, statistics
Sensitivity and specificity were determined for every test using ELISA (HBsAg and HCV-Ab) results as reference. 95% confidence intervals (95% CI) for sensitivity and specificity were calculated using Wilson score method without continuity correction (6). Positive- and negative likelihood (LR+ and LR-) ratios were calculated based on the values for sensitivity and specificity.
To put the results in the context of hepatitis screening activities in Mongolia, we assumed, as previously reported, a prevalence for HBsAg of 11.0% and for HCV-Ab of 8.5% among Mongolian adults (6). These values were used to calculate positive- and negative predictive values (PPV and NPV) and diagnostic accuracy (DA) [11].
Additional statistical analyses were conducted using Fisher´s Exact Test.
Results
Characteristics of study population
270 participants in 3 groups of 90 individuals were included in this study. 37% were of male gender. Mean age was 43.2 (18–75) years. None of the patients had any history of treatment with DAAs against HCV or nucleoside analogues against HBV since these drugs only gained widespread availability on the Mongolian market in late 2015.
ELISA confirmed serostatus of participants recruited as HBsAg positives and healthy controls. All recruited HCV-RNA positive individuals were found to be HCV-Ab seropositive in ELISA. For one participant from the HBsAg positive group, the respective HBV-DNA level was zero. For two participants from the HCV-Ab positive group, HCV-RNA results were positive, but below detection limit of the assay (<20 iU/ml). Further details see Table 2.
Results of HBsAg and HCV-Ab RDTs in serum testing
In total, 5400 RDTs were carried out within this study, 2700 for HBsAg and 2700 for HCV-Ab. None of the RDTs gave any invalid test result, which means, that the control line appeared on each test kit.
Among all RDTs for HBsAg detection, there were no false negative (FN) results and 6 out of 9 kits had no false positive results (FP) (Table 3). Only kits from CTK, Human and Cypress Diagnostics had some FN results (specificities were 98.3% (95.2–99.4%), 97.2% (93.7–98.8%) and 95.6% (91.5–97.7%), respectively). Average sensitivity for HBsAg kits was 100% (95.9–100%), average specificity 99% (96.4–99.6%). HCV-Ab serostatus had no effect on number of false positive test results with HBsAg kits (p>0.05) with 7 out of 16 FP tests coming from the HCV-Ab positive group, odds ratio 0.78 (0.29–2.09).
Each test was applied to 90 condition positive (P) and 180 condition negative (N) participants. Results of HBsAg ELISA served as reference standard. For the average, the 95% CI was averaged across all tests. For PPV, NPV and DA, an HBsAg prevalence of 11.0% was assumed.
For HCV-Ab detection, the OraSure test showed 100% in sensitivity and specificity (Table 4). Average sensitivity for all HCV-Ab tests was 98.9% (94.2–99.6%) and the average specificity was 96.4% (92.9–98.1%).
For PPV, NPV and DA, an HCV-Ab prevalence of 8.5% was assumed.
Among 64 FP results for HCV-Ab detection, 91% occurred among HBsAg positives, odds ratio 10.2 (4.4–23.8 95%CI). The average specificity for HCV-Ab among HBsAg negatives was 99.3% compared to a specificity among HBsAg positives of 93.6% (Table 5). Specifically RDTs for HCV-Ab detection from 3 manufacturers (ABON, CTK, Humasis) were significantly more likely to give false positive results in HBsAg positives than healthy individuals (p<0.05) and further 4 manufacturers produced FP results exclusively among HBsAg positives (Cypress, Green Gross, Intec, SD-Bioline).
The adjusted PPV is based on an HCV prevalence of 8.5% and and HBsAg prevalence of 11%.
Overall, sensitivities and specificities for HBsAg kits were better compared to HCV with with 0 FN and 16 FP among in total 2430 test for HBsAg compared to 10 FN and 64 FP among 2700 for HCV-Ab. However, the difference in specificity is not significant (p<0.05), when excluding HBsAg for detection of HCV-Ab.
Predictive values for screening among Mongolian adults
For detection of HBsAg under the assumption of an 11% prevalence, PPV range from 73% to 100% with an average of 92%. NPV all converge to 100% in absence of FN test results. For detection of HCV-Ab under the assumption of a prevalence of 8.5%, PPV range from 40% to 100% with an average of 72%. NPV range from 99.7% to 100% (99% average).
Discussion
Overall good results for most tests in our study demonstrates that inexpensive RDTs are highly valuable for initial HBV and HCV screening.
Detection of HBsAg using RDTs was very reliable. Of 9 kits tested, those of 6 manufacturers (ABON Biopharm, GreenCross, Humasis, InTec, SD Bioline, Wondfo) gave results in perfect agreement with the reference. The other kits gave some false positive results. However, all 90 seropositive HBsAg samples were correctly identified by all RDTs. Overall, due to their good results, Abon, GreenCross, Humasis, InTec, SD Bioline, and Wondfo RDTs can be recommended for HBV screening using blood serum.
For detection of HCV-Ab, the most striking result is the association of false positive results with HBsAg positivity of among 7 out of the 10 tests. While the cause for this effect is currently unclear, we can assume, that manufactures use similar formulations of active components in their kits. Several manufactures may have been using one component, which introduce a degree of sensitivity to HBsAg positive samples. However in previous studies it was demonstrated, that HIV positivity can have an effect on specificity of HCV-Ab RDTs [12].
Among RTDs for HCV-Ab detection, FDA-approved OraQuick demonstrated the highest diagnostic accuracy. However, this is by far the most expensive HCV-Ab test in this study [13]. Given financial constraints, a tradeoff between lower costs and slightly lower diagnostic accuracy can be advisable. Omitting HBsAg positive samples, kits by InTec, CTK and Humasis showed results in full agreement with the reference tests. However based on our results HCV-Ab test kits must be chosen with great caution especially in populations with a high prevalence of HBV.
For screening activities in Mongolia, the results highlight the main problems in practical use. Give random sampling among 1000 adults with and HCV-Ab prevalence of 8.5%, results in 85 positve and 915 negative samples. With the worst performing test with a NPV of 99.7%, we would expect 3 false negatives–potentially patients who would miss life-saving treatment due to false test results. In contrast for the same example, the lowest adjusted PPV (70.3%) would result in in 25 false positive tests which cause follow-up costs for confirmatory diagnostics.
If the tests were used to estimate the prevalence of HCV among Mongolian adults, tests from Orasure would deliver accurate of 8.5% HCV-Ab prevalence, tests from Abon would suggest a prevalence of 12.1%. Such results can lead to misguided public health interventions and in this way have a far reaching impact. Therefore, estimates of a prevalence of HBV or HCV in a population based solely on RDTs must be interpreted with a high degree of caution.
Our study was performed by well-trained laboratory personnel on serum samples. Some of the tested RDTs for HBsAg and most for HCV-Ab detection are also specified for use of whole blood. Using venipuncture or finger-stick whole blood instead of serum, could significantly simplify testing and therefore facilitate large scale testing. However, sensitivities and specificities might differ in that case and should be assessed separately.
Also it is important to point out, that for detection of HCV-Ab individuals, only patients with positive HCV-RNA test results were enrolled. Non-viremic HCV-Ab people, with possibly lower HCV-Ab concentrations (which could lead to lower RDT sensitivity values, were not included in this study.
All these issues can and should be addressed in future assessments under less defined conditions (untrained staff), simpler sample preparation (whole blood) and enrollment of non-viremic HCV-Ab positive participants, irrespective of their HCV-RNA status. Repeated assessments can also confirm a consistent quality of the manufacturers or demonstrate the opposite. For example, a study can be conducted at local primary care centers without any training for the care workers, other than the information sheet of each manufacturer.
In conclusion, our prospective study demonstrates that inexpensive RDTs can provide a good alternative for screening for HBV and/or HCV. This is especially true in scenarios in low-income countries, where the alternative would mean no screening at all. For population-wide screening for HBV and HCV e.g. within the Screening Campaign of the Hepatitis Prevention, Control, and Elimination Program in Mongolia, point-of-care screening using RDTs is an option that can be recommended.
References
- 1.
Global hepatitis report, 2017 [Internet]. 2017. Available from: https://www.who.int/hepatitis/publications/global-hepatitis-report2017/en/
- 2. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2095–128. pmid:23245604
- 3. Kish T, Aziz A, Sorio M. Hepatitis C in a new era: A review of current therapies. P T. 2017;42(5):316–29. pmid:28479841
- 4. Tang LSY, Covert E, Wilson E, Kottilil S. Chronic Hepatitis B infection a review. JAMA—J Am Med Assoc. 2018;319(17):1802–13.
- 5. Li W, Jia L, Zhao X, Wu X, Tang H. Efficacy and safety of tenofovir in preventing mother-to-infant transmission of hepatitis B virus: A meta-analysis based on 6 studies from China and 3 studies from other countries. BMC Gastroenterol. 2018;18(1):1–11.
- 6. Dashtseren B, Bungert A, Bat-Ulzii P, Enkhbat M, Lkhagva-Ochir O, Jargalsaikhan G, et al. Endemic prevalence of hepatitis B and C in Mongolia: A nationwide survey amongst Mongolian adults. J Viral Hepat. 2017;24(9).
- 7. Amini A, Varsaneux O, Kelly H, Tang W, Chen W, Boeras DI, et al. Diagnostic accuracy of tests to detect hepatitis B surface antigen: A systematic review of the literature and meta-analysis. BMC Infect Dis. 2017;17(Suppl 1).
- 8. Shivkumar S, Peeling R, Jafari Y, Joseph L, Pai NP. Rapid point-of-care first-line screening tests for hepatitis B infection: a meta-analysis of diagnostic accuracy (1980–2010). Am J Gastroenterol. 2012 Sep;107(9):1306–13. pmid:22641308
- 9. Shivkumar S, Peeling R, Jafari Y, Joseph L, Pai NP. Accuracy of rapid and point-of-care screening tests for hepatitis C: A systematic review and meta-analysis. Ann Intern Med. 2012;157(8):558–66. pmid:23070489
- 10. Khuroo MS, Khuroo NS, Khuroo MS. Diagnostic Accuracy of Point-of-Care Tests for Hepatitis C Virus Infection: A Systematic Review and Meta-Analysis. PLoS One [Internet]. 2015;10(3):e0121450. Available from: pmid:25816332
- 11. Akobeng AK. Understanding diagnostic tests 1: Sensitivity, specificity and predictive values. Acta Paediatr Int J Paediatr. 2007;96(3):338–41.
- 12. Smith BD, Drobeniuc J, Jewett A, Branson BM, Garfein RS, Teshale E, et al. Evaluation of three rapid screening assays for detection of antibodies to hepatitis C virus. J Infect Dis. 2011;204(6):825–31. pmid:21849279
- 13.
UNITAID. Hepatitis C. Diagnostics Technology Landscape. Am J Nurs [Internet]. 2015;(January). Available from: https://unitaid.org/assets/UNITAID-HCV_Diagnostic_Landscape-1st_edition-1.pdf