Diagnostic Accuracy of Point-of-Care Tests for Hepatitis C Virus Infection: A Systematic Review and Meta-Analysis

Background Point-of-care tests provide a plausible diagnostic strategy for hepatitis C infection in economically impoverished areas. However, their utility depends upon the overall performance of individual tests. Methods A literature search was conducted using the metasearch engine Mettā, a query interface for retrieving articles from five leading medical databases. Studies were included if they employed point-of-care tests to detect antibodies of hepatitis C virus and compared the results with reference tests. Two reviewers performed a quality assessment of the studies and extracted data for estimating test accuracy. Findings Thirty studies that had evaluated 30 tests fulfilled the inclusion criteria. The overall pooled sensitivity, specificity, positive likelihood-ratio, negative likelihood-ratio and diagnostic odds ratio for all tests were 97.4% (95% CI: 95.9–98.4), 99.5% (99.2–99.7), 80.17 (55.35–116.14), 0.03 (0.02–0.04), and 3032.85 (1595.86–5763.78), respectively. This suggested a high pooled accuracy for all studies. We found substantial heterogeneity between studies, but none of the subgroups investigated could account for the heterogeneity. Genotype diversity of HCV had no or minimal influence on test performance. Of the seven tests evaluated in the meta-regression model, OraQuick had the highest test sensitivity and specificity and showed better performance than a third generation enzyme immunoassay in seroconversion panels. The next highest test sensitivities and specificities were from TriDot and SDBioline, followed by Genedia and Chembio. The Spot and Multiplo tests produced poor test sensitivities but high test specificities. Nine of the remaining 23 tests produced poor test sensitivities and specificities and/or showed poor performances in seroconversion panels, while 14 tests had high test performances with diagnostic odds ratios ranging from 590.70 to 28822.20. Conclusions Performances varied widely among individual point-of-care tests for diagnosis of hepatitis C virus infection. Physicians should consider this while using specific tests in clinical practice.


Item
Specifications Type of meta-analysis Diagnostic Test Accuracy (DTA) Basis of reporting PRISMA (Preferred Reporting Items for Systematic Reviews and Metaanalysis) guidelines. Search Engines MEDLINE (via PUBMED), EMBASE (via OVID), BIOSIS and Web of Science (1980 to December 2013). MESH Terms employed for search "Hepatitis C" OR "Hepatitis C Antibodies" OR "Hepatitis C Virus" OR "Hepatitis C Antigens" AND "Point-of-Care Systems" OR "rapid test" OR "diagnostics" AND "Sensitivity and Specificity" OR "diagnostic accuracy" OR "validity".

Index Test
Rapid Diagnostic Tests (RDTs); Accuracy of individual tests, Analytical sensitivity of the tests, Comparative efficacy of the different tests, Heterogeneity within and between studies and their potential sources, Applicability of these tests in different scenarios of HBV evaluation.

BACKGROUND
Hepatitis C is a global health problem (1). An estimated 2 to 3% of the world population is chronically infected with hepatitis C virus (HCV). This amounts to an estimated 130-170 million infected persons worldwide (2). Chronic hepatitis C is associated with significant morbidity and mortality. HCV contributes to 27% of cirrhosis and 25% of hepatocellular carcinoma and causes more than 350 000 deaths each year (3). HCV infection prevalence varies widely throughout the world, even among neighboring countries and in geographic regions within the same country. The prevalence of HCV in the United States, Australia and most countries in Western Europe is less than 2%. HCV infection rates are higher (≥3%) in many countries in Eastern Europe, Latin America, the Middle East, Africa and South Asia. Chronic HCV infection is highly endemic in Egypt (≥10%), many regions of Pakistan and adjoining regions in Western India (≥6%) (4). Another issue which is of importance to the epidemiology of hepatitis C is its relationship with HIV infection. Worldwide up to 30% of the 33 million persons infected with HIV are also infected with HCV. HIV/HCV co-infections also have a varied geographical distribution (5-7). These co-infections are common in sub-Saharan Africa and are becoming common in developed countries where HIV is becoming an increasing problem in men who have sex with men. HIV/HCV co-infection are associated with accelerated progression of liver disease and higher mortality. Mode of spread of HCV in developed countries is mainly through injection drug use. Blood and blood products in these countries is routinely screened for HCV by sensitive methods and there are measures in place to facilitate infection control and safe injection practices. In contrast, unsafe injections in healthcare settings are leading cause of HCV transmission in developing countries. Recipients of blood and blood products also are at risk of infection as up to 20% of such products are not screened for hepatitis viruses in these countries. Also paid or coerced donors, a common occurrence in developing countries are more likely to transmit HCV transmission. Some distinctive risk factors play a part in HCV transmission in some regions of the World. Reuse of syringes during a schistosomiasis eradication program in the 1960 and 1970's attributed to high HCV prevalence in Egypt. Following this HCV spread was purported by widespread use of unsafe injection practice, poor infection control in hospitals and widespread use of unscreened blood for transfusions. In some countries like Japan and Korea, high prevalence of HCV is seen in elderly people with sharp decrease in young generation. This epidemiological pattern along with a disproportionate burden of HCC and liver cirrhosis relative to overall prevalence suggest a high prevalence of HCV infection among persons infected in the distant past. HCV infection in people born between 1945-1965 (Baby Boomers) account for three-fourth of all HCV infections in the United States and Western Europe. One in 30 baby boomers has been infected with hepatitis C, and most have no clue that they are infected. In addition to above around 1 million persons become permanent legal residents in the United States, and many more undocumented persons enter the country. Large number of these persons are from countries where HCV infection is endemic. Of the 40 million foreign-born persons living in the United States, nearly 20 million are from Latin America, where rates of HCV infection approach 3% in some countries. Most of the remainder of immigrants come from Asia, Europe, and Africa serve as countries of origin for remaining immigrants (1)(2)(3)(4).
In view of the above, screening of HCV infection in many high risk epidemiologic settings is mandatory. In addition testing of blood and blood products is essential to prevent HCV infection to recipients. Conventionally enzyme immunoassay (EIA) to detect antibody to HCV is the serologic hallmark of hepatitis C infection (8). Nucleic acid testing for HCV RNA and HCV genotype are needed after HCV infection is established on EIA. These tests require high facility cost, sophisticated equipment, trained technicians, continuous supply of electricity, and are unsuitable for use in poor resource endemic regions. Rapid Point-of-care testing offers significant advantages. These are divided in to: i) Point-of-Care tests (POCTs) which need no sample processing, are robust at room temperature and have long shelf-life (>6 months) and ii) Rapid diagnostic tests (RDTs) which require sample processing, storage at 0-4 0 C and have short shelf life (9-11).
Since 1990s, several RDTs and POCTs that primarily use serum, plasma, whole blood and oral fluid to test for anti-HCV have been developed. Manufacturers claim high clinical and analytical sensitivity of these tests. Based on the claims, these tests are widely used in developing countries in many settings including blood banks. However, several vital questions about their use remain unanswered which include: (i) accuracy of individual tests, (ii) comparative efficacy of the different tests, and (iii) applicability of these tests in different scenarios namely population surveys, screening of blood donors, diagnosis of hepatitis C etc.
A recent meta-analysis on accuracy of rapid and point-of-care diagnostic tests for hepatitis C attempted to address the above mentioned questions (12). However, this study had many limitations. Authors did not compare the subgroup estimates in the statistical meta-regression model and thus the interpretation of the meta-analysis were faulty. Analytical sensitivity of the tests based on low titer and sero-conversion panels were not assessed, which affected the conclusions made on the accuracy of the tests under consideration. Authors did not include the evaluation of heterogeneity (differences in reported estimates among studies) and its potential sources, an important component in meta-analysis studies (13). We believe recommendations on use of rapid point-ofcare tests can have far reaching effects on the healthcare in developing countries. For example recommending tests with low analytical sensitivity in blood banks can pose a serious threat to recipients as infected otherwise healthy donors often have low titer HCV viremia. Keeping the above in consideration, we conducted another systematic review and meta-analysis of studies pertaining to diagnostic accuracy and applicability of RDTs and POCTs for HCV.

METHODS
Two reviewers shall conduct literature search, quality assessment of the included studies and data extraction for estimating test accuracy (14). Any discrepancies shall be referred to third reviewer. We shall follow PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analysis) guidelines for conducting and reporting on this meta-analysis (15).

Acquisition of Data
The primary search shall be made in MEDLINE (via PUBMED), EMBASE (via OVID), BIOSIS and Web of Science (1980 to December 2013). MeSH terms used for key and text word searching shall be "Hepatitis C" OR "Hepatitis C Antibodies" OR "Hepatitis C Virus" OR "Hepatitis C Antigens" AND "Point-of-Care Systems" OR "rapid test" OR "diagnostics" AND "Sensitivity and Specificity" OR "diagnostic accuracy" OR "validity". Bibliographic for the relevant citations and reviews shall be manually searched for relevant citations and experts in the field were contacted to ensure that search strategy is complete. Titles and abstracts of all the above articles identified in the primary search shall be evaluated and a list of potential eligible studies identified. These studies shall be considered for full-text review. Studies which fulfil the criteria for selection shall be included in the systematic review and metaanalysis.

Criteria for Study Inclusion
Following studies shall be included in the meta-analysis: i. Studies which employed RDTs or POCTs for detection of Anti-HCV (Index test) and compared the results with a reference test and reported results to recreate the 2X2 diagnostic table for estimating test accuracy. ii.
Studies published both as abstracts and full-text articles. iv.
Studies using all study-designs, conducted in any study settings (laboratory or field-based) and regardless of sample size, study location, language of publication, and country of origin of test. Following studies shall be excluded: i. Studies which deal with accuracy of laboratory-based tests, ii.
Studies with data unable to recreate 2x2 diagnostic table, iii.
reports from the manufacturer and package inserts which are subjected to overt conflict of interest, iv.
Duplicate reports.

Data Extraction
Each study shall be subjected to following search: study author, year of publication, location of study, index test (one or more), reference standard, study design, source of sera, sample size, characteristics of the population employed for sera collection, cross reactive sera included in panel and analytical sera included for evaluating test sensitivity. Detailed information about the index test shall be extracted from the studies which included: name of the test, country of origin and name of the manufacturer, time taken to read results, specimen (serum, plasma, blood or oral fluid) needed for test, volume of the sample (µl) needed to test, storage conditions for maintaining test kit, special equipment if any needed to perform the test, shelf life of the test kit and scope of the test utility (RDTs or POCTs). For purposes of data synthesis we shall extract raw cell numbers namely true positives, false negatives, false negatives and true negatives for each test run.

Quality Assessment
Quality assessment of the studies using QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) tool (16) and the STARD (Standards for the Reporting of Diagnostic Accuracy Studies) checklists (17) shall be conducted. QUADAS-2 sheet shall be completed by following stepwise guidelines to judge risk of bias (4 domains) and concerns about applicability (3 domains) for each study. STARD checklist consists of 25 questions and each question shall be weighted equally (yes=1, No=0) and total score for each study calculated.

Statistical Analysis and Data Synthesis
While analyzing data, we shall address following questions (i) i. Accuracy of individual tests, ii.
Analytical sensitivity of the tests, iii.
Comparative efficacy of the different tests, iv.
Heterogeneity within and between studies and their potential sources, v. Applicability of these tests in different scenarios of HCV evaluation.
To do so following algorithm shall be followed:  (18). These measures shall be pooled using the random effects model (19). In addition summary receiver operating characteristics (SROC) curve plots shall be obtained (20).
Analytical Sensitivity: Analytical sensitivity of the tests shall be evaluated by analyzing the results of the tests against low titer sera and sero-conversion panels. We shall determine the lowest HCV concentration which shall be picked up by various tests and compare this with what is claimed by the manufacturers.
Subgroup analysis: For further analysis, we shall divide data in to subgroups based on: (i) scope of test utility (RDTs vs. POCTs), (ii) Specimen used to test (blood, plasma, serum or oral fluid, (iii) source of sera (blood banks, hospital/clinic, HIV clinics with included cross reactive sera), (vi) location where test was conducted (developed versus developing countries), (v) tests with sufficient data points will be pooled and the pooled estimates compared with pooled estimates of remaining tests. We shall compare summary estimates of diagnostic accuracy within subgroups to make relevant conclusions.
Heterogeneity: Heterogeneity (differences in reported estimates among studies) shall be evaluated by a Q test statistic (Chi square value with p values) and I^2 values (11). Three potential sources of heterogeneity (design of studies; study quality and year of publication) shall be evaluated in the meta-regression model. (19,20) We shall use software Meta-Analyst (Tufts Medical Centre, Boston, MA) for all statistical analysis. (

Sensitivity & Specificity
Sensitivity of a test is defined as the probability that the index test result will be positive in a diseased case.

Sensitivity= True positive ÷ [True positive + False negative] = (a) ÷ (a + c).
Specificity of a test is defined as the probability that the index test result will be negative in a non-diseased case.

LIKELIHOOD RATIOS
Likelihood ratio (LR) can be used to update the pre-test probability of disease using Bayes' theorem, once the test result is known. The updated probability is referred to as the post-test probability. For a test that is informative, the post-test probability should be higher than the pre-test probability if the test result is positive, whereas the post-test probability should be lower than the pre-test probability if the test result is negative.
Positive LR describes how many times more likely positive index test results were in the diseased group compared to the nondiseased group. The positive LR, which should be greater than 1 if the test informative. Negative LR describes how many times less likely negative index test results were in the diseased group compared to the nondiseased group. Negative LR should be less than 1 if the test is informative.

DIAGNOSTIC ODDS RATIOS
Diagnostic odds ratio (DOR) summarizes the diagnostic accuracy of the index test as a single number that describes how many times higher the odds are of obtaining a test positive result in a diseased rather than a non-diseased person. The fact that it summarizes test accuracy in a single number makes it easy to use this measure for meta-analysis but expressing accuracy in terms of ratios of odds means the measure has little direct clinical relevance, and it is rarely used as a summary statistic in primary studies. In fact, the clinician is usually interested in the sum of the number of false negative and false positive results whereas the DOR reflects their product. The DOR does, however, remain an important element in meta-analytic model building.
After the data from each primary study have been summarized by calculating 2 quantities (Di & Si for the ith study) analysis fit a simple linear regression model using D as dependent variable and S as the predictive variable as: D= alpha +beta S. TPP=True Positive Proportion or sensitivity, FPP=False Positive Proportion or Specificity.

IDENTIFY AND MEASURE HETEROGENEITY
Heterogeneity was identified and assessed as follows: i) Overlap in the confidence intervals of individual studies. Poor overlap depicted statistical heterogeneity, ii. Chi-squared (χ 2 , or Chi 2 ) test for heterogeneity with P value. A large χ 2 value with P <0.10 (rather than conventional 0.05) suggested heterogeneity, iii) Calculating I^2 for heterogeneity: I^2 is calculated as follows :.
Where Q=Chi square value for heterogeneity; df=degree of freedom A rough guide to interpretation is as follows:

I^2 value
Magnitude of Heterogeneity 0% to 40% Might not be important 30% to 60% May represent moderate heterogeneity 50% to 90% May represent substantial heterogeneity 75% to 100% Considerable heterogeneity The importance of the observed value of I 2 depends on (i) magnitude and direction of effects and (ii) strength of evidence for heterogeneity (e.g. P value from the chi-squared test, or a confidence interval for I 2 ).

HOW TO READ RECEIVER OPERATING CHARACTERISTIC (ROC) PLOT CURVES
The ROC curve of a test is the graph of the values of sensitivity and specificity that are obtained by varying the positivity threshold across all possible values. The graph plots sensitivity (true positive rate) against 1-specificity (false-positive rate). The curve for any test moves from the point where sensitivity and 1-specificity are both 1 (the upper right corner) which is achieved for a threshold at the lower end of its range (classifying all participants as test positive, so there are no false negatives but many false positives) to a point where sensitivity and 1-specificity are both zero (the lower left corner) which is achieved when the threshold moves to the upper end of its range (and all participants are classified as test negative, giving no false positives but many false negatives). The shape of the curve between these two fixed points depends on the discriminatory ability of the test.
ROC curve is estimated from a finite sample of test results and hence will not necessarily be a smooth curve. The horizontal axis for each ROC plot is labelled in terms of specificity decreasing from 1.0 to 0.0. This style of labelling is (1-specificity ranging from 0.0 to 1.0).
The position of the ROC curve depends on the degree of overlap of the distributions of the test measurement in diseased and nondiseased. Where a test clearly discriminates between diseased and non-diseased such that there is no or little overlap of distributions, the ROC curve will indicate that high sensitivity is achieved with a high specificity, that is the curve approaches the upper left hand corner of the graph where sensitivity is 1 and specificity is 1. If the distributions of test results in diseased and nondiseased coincide, the test would be completely uninformative and its ROC curve would be the upward diagonal of the square.
The ROC curves may be symmetrical about the sensitivity=specificity line (the downward diagonal of the square) or not symmetrical. Asymmetrical curves typically occur when the distribution of the test measurement in those with disease has more or less variability than the distribution in non-diseased people. Increased variability might occur, for example, where disease may cause a biomarker both to rise and become more erratic; reduced variability might occur where disease may lower biomarker values to a bounding level such as a lower level of detection.

COUPLED FOREST PLOTS
Forest plots for diagnostic test accuracy report the number of true positives and false negatives in diseased and true negatives and false positives in non-diseased participants in each study, and the estimated sensitivity and specificity, together with confidence intervals. The plots are known as coupled forest plots as they contain two graphical sections: one depicting sensitivity, and one specificity. The order of the studies can be sorted, often they are presented sorted by values of sensitivity, or grouped by test type or covariate values. Whilst it is possible to observe heterogeneity in sensitivity and specificity individually on such plots, it is not as easy to visualize whether there are threshold-like relationships. Summary statistics computed from meta-analyses can be added to coupled forest plots.

PRISMA CHECHLIST [Preferred Reporting Items for Systematic Reviews and Meta-Analyses:]
Section/topic # Checklist item Reported on page # TITLE Title 1 Identify the report as a systematic review, meta-analysis, or both.

Rationale 3 Describe the rationale for the review in the context of what is already known.
Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

Protocol and registration
5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale.
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).

Data collection process
10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

Risk of bias in individual studies
12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.
Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means).
Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I 2 ) for each meta-analysis.
Page 1 of 2

Risk of bias across studies
15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

RESULTS
Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.
Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, followup period) and provide the citations.

Risk of bias within studies
19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12).

Results of individual studies
20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.
Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency.

Risk of bias across studies
22 Present results of any assessment of risk of bias across studies (see Item 15).
Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]).

Summary of evidence
24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).
Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research.

FUNDING
Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.

STARD checklist [PDF-file] [WORD version]
The STARD checklist consist of 25 items. Please, click on the description of the items for the rationale of the item and an example. Describe data collection: Was data collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study)? Test methods 7 Describe the reference standard and its rationale. 8

Section and Topic
Describe technical specifications of material and methods involved including how and when measurements were taken, and/or cite references for index tests and reference standard.

Section and Topic
Item On page 9 Describe definition of and rationale for the units, cutoffs and/or categories of the results of the index tests and the reference standard. 10 Describe the number, training and expertise of the persons executing and reading the index tests and the reference standard. 11 Describe whether or not the readers of the index tests and reference standard were blind (masked) to the results of the other test and describe any other clinical information available to the readers.

Statistical methods
12 Describe methods for calculating or comparing measures of diagnostic accuracy, and the statistical methods used to quantify uncertainty (e.g. 95% confidence intervals). 13 Describe methods for calculating test reproducibility, if done.

Participants
14 Report when study was done, including beginning and ending dates of recruitment. 15 Report clinical and demographic characteristics of the study population (e.g. age, sex, spectrum of presenting symptoms, co morbidity, current treatments, recruitment centers). 16 Report the number of participants satisfying the criteria for inclusion that did or did not undergo the index tests and/or the reference standard; describe why participants failed to receive either test (a flow diagram is strongly recommended).

Test results
17 Report time interval from the index tests to the reference standard, and any treatment administered between. 18 Report distribution of severity of disease (define criteria) in those with the target condition; other diagnoses in participants without the target condition.