Comparative accuracy of CT perfusion in diagnosing acute ischemic stroke: A systematic review of 27 trials

Objective To systematically evaluate and compare the diagnostic accuracy of CT perfusion (CTP), non-enhanced computed tomography (NCCT) and computed tomography angiography (CTA) in detecting acute ischemic stroke. Methods We searched seven databases and screened the reference lists of the included studies. The risk of bias in the study quality was assessed using QUADASII. We produced paired forest plots in RevMan to show the variation of the sensitivity and specificity estimates together with their 95% CI. We used a hierarchical summary ROC model to summarize the sensitivity and specificity of CTP in detecting ischemic stroke. Results We identified 27 studies with a total of 2168 patients. The pooled sensitivity of CTP for acute ischemic stroke was 82% (95% CI 75–88%), and the specificity was 96% (95% CI 89–99%). CTP was more sensitive than NCCT and had a similar accuracy with CTA. There were no statistically significant differences in the sensitivity and specificity between patients who underwent CTP within 6 hours of symptom onset and beyond 6 hours after symptom onset. No adverse events were reported in the included studies. Conclusions CTP is more accurate than NCCT and has similar accuracy to CTA in detecting acute ischemic stroke. However, the evidence is not strong. There is potential benefit of using CTP to select stroke patients for treatment, but more high-quality evidence is needed to confirm this result.


Results
We identified 27 studies with a total of 2168 patients. The pooled sensitivity of CTP for acute ischemic stroke was 82% (95% CI 75-88%), and the specificity was 96% (95% CI 89-99%). CTP was more sensitive than NCCT and had a similar accuracy with CTA. There were no statistically significant differences in the sensitivity and specificity between patients who underwent CTP within 6 hours of symptom onset and beyond 6 hours after symptom onset. No adverse events were reported in the included studies.

Conclusions
CTP is more accurate than NCCT and has similar accuracy to CTA in detecting acute ischemic stroke. However, the evidence is not strong. There is potential benefit of using CTP to select stroke patients for treatment, but more high-quality evidence is needed to confirm this result. PLOS

Introduction
Stroke is the second leading cause of death and the third leading cause of disability in the world [1,2]. In 2010, there were 16.9 million incident stroke cases, 33.0 million prevalent stroke cases, 5.9 million deaths attributed to stroke, and 102.2 million lost DALYs [3,4]. According to a report from the American Heart Association, approximately 87% of all strokes were ischemic strokes [5]. An accurate and timely diagnosis of ischemic stroke is crucial for establishing an appropriate patient treatment. Non-enhanced computed tomography (NCCT) is widely used in acute ischemic stroke imaging due to its rapid performance, high tolerance, and high reliability [6]. However, NCCT has difficulty in detecting early infarct signs and is influenced by the size of the infarct and severity of ischemia. CT perfusion (CTP) is performed for physiological evaluation of the brain parenchyma, which allows better detection of ischemia. However, the accuracy of CTP in detecting acute ischemic stroke (AIS) was still uncertain. To the best of our knowledge, there were two systematic reviews to evaluate the diagnostic value of CTP for AIS. A systematic review which based on 15 studies showed that the sensitivity and specificity of CTP were 80% and 95% respectively; however, another systematic review which based on 11 studies reported that the sensitivity and specificity of CTP were 69.9% and 87.7% [7,8]. These conflicting results revealed the possibility of publication bias and omission of critical studies. Taking into account the limitations of the previously mentioned reviews and the appearance of new evidence, there is still a need to evaluate how well CTP detects acute ischemic stroke. The purpose of the present study is to review the accuracy of CTP in detecting ischemic stroke systematically, and at the same time, to compare accuracy of CTP with either NCCT or computed tomography angiography (CTA) in detecting ischemic stroke systematically.

Criteria for considering studies for this review
The studies were eligible that met the following inclusion criteria. 1) The studies either focused on CTP or compared CTP with other imaging methods.
2) The absolute numbers of the observations of true positives, false positives, false negatives, and true negatives were either available or derivable from the data which was reported in the primary studies. 3) Human subjects were the focus of analysis. 4) The studies were either prospective or retrospective. We excluded the following studies that addressed specific anatomical, metabolic, or microvascular aspects of stroke; focused on the specific technical parameters of CTP; location of final infarct; distinguished ischemic core from the salvageable brain tissue. If several reports were based on the same study, we selected the most recent or most complete publication available.

Literature search
We performed an initial literature search of PubMed, EMBASE, CENTRAL, the Cochrane Database of Systematic Reviews, the Cochrane Controlled Trials Register Database of Reviews of Effectiveness, and Health Technology Assessment database in June 2015 and updated search in June 2016. We also scanned the references of all articles which were selected for the review to find any potentially relevant articles. The search strategy was list in S1 Table. two reviewers also assessed the risk of bias independently and extracted information for each included study. Reviewers dealt with discrepancies through discussion.

Assessment of methodological quality
We used the quality assessment of diagnostic accuracy studies II (QUADASII) to assess the methodological quality of each included study [9]. QUADASII was updated in 2010 that consists of four domains as follows: patient selection, index test, reference standard and patient flow. Each domain was assessed in terms of the risk of bias, and the first three domains were also considered in terms of applicability. Clinical assessment with follow up imaging (MRI or CT) was defined as a reference standard to classify a stroke diagnosis correctly.

Data collection
We extracted information from each included study, and these information included the general study characteristics (i.e., author name, year of publication, inclusion or exclusion criteria, total number of participants, study design), patient characteristics (baseline NIHSS, time from symptom onset to CTP acquisition), index test (CT scanner, Coverage, Thickness), reference standard and indices of diagnostic performance. We constructed 2×2 contingency tables for true positive cases, false positive cases, false negative cases and true negative cases. If there were no absolute values, we calculated these values from the estimated sensitivities and specificities of each study.

Statistical analysis and data synthesis
The estimates of sensitivity and specificity and their 95% confidence interval were plotted in paired forest plots. We used a hierarchical summary ROC model (HSROC) which was performed in STATA (version 12) to pool the sensitivity and specificity of CTP in detecting acute ischemic stroke [10].
We included all studies in each pairwise comparison to compare the diagnostic accuracy of CTP with other imaging technologies. We used forest plots to presented the results of the studies which directly compared different index tests. We pooled sensitivities and specificities of the compared different index tests by using bivariate random effects models in SAS 9.0. The model's parameters were used to plot the ROC curve in RevMan.

Results
We identified 9904 citations, and 27 citations were included in our analysis after full-text reviews (Fig 1).
We included 27 studies with a total of 2168 sample size. The sample size in the included studies ranged from 10 to 422, and seven of them only included stroke patients (n = 291). All studies were based on a reference standard of follow-up CT or MRI. Thirteen studies stated all patients who underwent CTP within 6 hours after symptom onset. Patients in the other twelve studies underwent CTP in 6-24 hours after symptom onset. Two studies did not report patients' time of the symptom onset. Twelve studies retrospectively collected patients' data, eleven studies performed prospective trials and four studies did not describe their study design. The characteristics of included studies were showed in Table 1.

Methodological quality of included studies
The quality of the 27 included studies was varied. Figs 2 and 3 summarized the results of the quality assessment of the included studies. Eight studies were classified as high quality. In the patient selection domain, seven studies were assessed at high risk of bias due to its' poor reporting of the sampling procedure and the exclusion criteria. Five studies were assessed at unclear risk of bias because those studies were unclearly reported whether the included participants were consecutive or random. Seven studies were of high concern because they only included stroke patients. One study was classified as high concern because the patient cohort comprised only children.
In the index test domain, we considered five studies with high risk of bias, because the retrospective data, and whether the index test results collection and confusion were interpreted without knowledge of the reference standard results. Nine studies were judged with unclear risk of bias for poor reporting of whether the index test results were interpreted without knowledge of the reference standard results. Therefore, these studies were of unclear concern.  Thirteen studies were judged as having unclear risk of bias in reference standard domain, because they failed to report whether the reference standards were interpreted without knowledge of the index test results. However, we assessed these studies as having a low concern in applicability because of their objective results. In twelve studies, not all patients received the same reference standard. The limit time of follow up CT or MRI was unclear. These may have  risk of bias in reference standard. However, we assessed these studies as having a low risk of bias in reference standard domain because both CT and MRI were recommended as gold standards by guidelines. We deemed 26 of the studies at low risk of bias with regard to the flow and timing domain. The remaining study was judged as having an unclear risk of bias because not all patients received the same reference standard.

Findings
Accuracy of CTP in detecting stroke. Two studies (Sporns2016,van der Hoeven2015) were about posterior circulation, and they were analyzed in subgroup analysis. Fig 4 showed the coupled forest plot of CTP's sensitivity and specificity in the other twenty-five studies. The sensitivity of CTP in detecting AIS ranged from 50% to 100%, and the specificity ranged from 67% to 100%. Seven studies could not estimate the specificity because no non-disease patients were included, and the pooled sensitivity of these studies was 72% (65-79%). The pooled sensitivity of the remaining 18 studies was 82% (95% CI 75-88%), the specificity was 96% (95% CI 89-99%), the diagnostic odds ratio (DOR) was 118.8 (95% CI 45.6-309.8), the positive likelihood ratio (LR) was 21.8 (95% CI 7.8-60.6) and the negative LR was 0.18 (95% CI 0.13-0.26).  Comparison of CTP and NCCT in detecting stroke. As shown in Fig 6, ten studies compared the accuracy of CTP with non-contrast CT (NCCT) in detecting acute stroke, and two of these studies were not estimated in specificity. The sensitivity of NCCT ranged from 15% to 86%, while the specificity of NCCT was 100%. All studies showed that the sensitivity of CTP was higher than that of NCCT, and three studies showed a lower specificity of CTP. The pooled sensitivities and specificities of CTP were higher than those of NCCT as shown in Fig 7. Comparison of CTP and CTA in detection stroke. As shown in Fig 8, seven studies compared CTP with CTA directly. The sensitivity of CTA ranged from 56% to100%, and the specificity was 100%. Four studies showed that sensitivity of CTP was higher than that of CTA, and one study showed the specificity of CTP was lower than that of CTA. As shown in Fig 9, the pooled sensitivity of CTP was 91% (95% CI: 56-99%), the pooled specificity was 97% (95% CI: 36-100%), the pooled sensitivity of CTA was 93% (95% CI: 35-99%), and the pooled specificity was 100% (95% CI: 0-100%). There was no significant difference between CTP and CTA in the pooled sensitivities and specificities.
Comparison accuracy of CTP in detection acute posterior circulation stroke. Two studies compared the accuracy among CTP, NCCT and CTA in detecting acute posterior  Table 2, the sensitivity of CTP in detecting acute posterior circulation stroke was higher than that of CTA and NCCT. All of the specificities of CTP, CTA and NCCT were high, and there was no significant difference.

Subgroup analysis
Results of the subgroup analyses were summarized in Table 3. When we compared prospective study group with retrospective study group, there was no statistically significant difference in sensitivity (p>0.05), but there was significant difference in specificity (p<0.05). Differences in the sensitivity and specificity of CTP were also not statistically significant between the 6 hours study group and 6-24 hours study group (p>0.05).

Discussion
Accuracy of CTP in diagnosing acute ischemic stroke CTP had high sensitivity and very high specificity in detecting ischemic stroke. The pooled sensitivity and specificity of CTP in this systematic review were both higher than those of previous systematic review which included 15 studies [7]. In our study, CTP had higher sensitivity than NCCT and had similar accuracy to CTA in diagnosing AIS. However, CTA provides high diagnostic accuracy in detecting site of occlusion while CTP offers high diagnostic accuracy in distinguishing between infarcted and ischemic penumbra [8]. The study by Tan also found that CTA source images performed better than CTP in detecting the site of occlusion (sensitivity,94.6%; specificity, 100.0% vs sensitivity, 88.2%; specificity, 95.3%), while CTP performed better than CTA source images in prediction of the anatomic distribution of the final infarct Comparative accuracy of CT perfusion in diagnosing acute ischemic stroke (sensitivity, 80.4%; specificity, 96.8% vs sensitivity,72.0%; specificity, 98.4%) [38]. In the 195 specified false negative cases, 106 (54%) were cases of lacunar infarct, 56 (29%) were cases that not covered by CTP, eleven (6%) were cases of atherothrombotic infarction, nine (5%) were cases of territorial infarct, five (3%) case were motion artifacts, four (2%) case were due to technical failure, two (1%) were cases of reperfusion, and two (1%) were cases of cardioembolism. Therefore, some of the patient characteristics (especially the percentage of lacunar infarct patients and the coverage of CTP imaging) impacted the sensitivities of CTP. There were some ways to improve the accuracy of CTP, such as the toggling-table technique could double brain coverage, and the latest generations of CT scanners could provide whole brain coverage [12,27,33].

Factors which affect the accuracy of CTP in diagnosing acute ischemic stroke
The influence of study design in the diagnostic accuracy of CTP was small. The pooled CTP sensitivity of 9 prospective studies was higher than that of 8 retrospectively studies, but there was no significant difference. The accuracy of CTP was not correlated with the length of time after symptom onset. The sensitivity of group which underwent CTP scan within 6 hours from Comparative accuracy of CT perfusion in diagnosing acute ischemic stroke symptom onset was higher than that of the 6-24 hour group, but the difference was not statistically significant. The result was similar to the study by Hana [17].
There were no defined CTP processing parameters, biologically salient perfusion parameters and thresholds for decision making, thus different CTP protocols, image processing & analysis methods, and color maps (CBV, CBF, MTT, and TTP) maybe used in various studies, and all of these could vary the sensitivity of CTP [39][40][41][42][43].
The severity of the stroke and size of infarct also influenced both the sensitivity and specificity of CTP. The study by Hana found that CTP had higher sensitivity when the size of infarction was over 3 cm 2 (90% vs 29%, P<0.001) [17]. Another study by Huisa found that sensitivity and specificity of CTP were increased to 81% and 91% when they selected MCA territory strokes >5cc [44]. The study by Eckert showed that the sensitivity of CTP was higher in group which with average NIHSS = 8.3 than in group which with NIHSS = 4.4 (100% vs 42.6%) [15]. The study by Rother showed that the sensitivity of CTP in group with NIHSS>10 was higher than sensitivity of CTP in group with NIHSS<10(100% vs 33%) [28]. Another study by Campbell also found that the sensitivity of CTP was lower in patients with NIHSS 7 than that in patients with NIHSS = 10 (63% vs 80%) [11]. In twelve included studies which reported average NIHSS, only two studies had average NIHSS 4, thus the results of this review may overestimated.
In our study, the pooled sensitivity and specificity of CTP in diagnosing AIS were similar to those in systematic review by Biesbroek. Compared with Biesbroek's study, our systematic review included other ten studies. The sensitivity and specificity of CTP in the systematic review by Sabarudin were lowest, because the included criteria of Sabarudin's study were unclear and some studies were missed, but these missed studies were included in our review.
The safety of CTP in diagnosing acute ischemic stroke Renal damage and radiation exposure were two disadvantages of CTP. One included study reported that two patients had developed into subcutaneous contrast extravasation, but no Comparative accuracy of CT perfusion in diagnosing acute ischemic stroke Comparative accuracy of CT perfusion in diagnosing acute ischemic stroke clinically significant renal dysfunction was reported [11]. In the available literature, patients with normal renal function who had CTP had low risk of renal dysfunction. CTP had more additional radiation dose than NCCT. However, the newest scanners with optimized protocols can image the entire cranium without substantial increases in radiation dose. All included studies did not report the safety of CTP except two studies that reported the administered radiation doses [33,34] which were lower than the recommended dose of 2.0-3.4 mSv described in the Hoeffner study [45]. The effective dose of CTP was 1.1/1.2 to 5 mSv, and it was not higher than NCCT, but it was less than CTA [46][47][48][49].

Strengths and weaknesses of the review
This research not only review the accuracy of CTP in detecting ischemic stroke, but also review the comparative accuracy of CTP in detecting ischemic stroke, and the latter was the most difference compared to previous reviews. We included directly comparative studies that evaluated two index tests in the same group of patients, and these studies provided the best evidence in comparing the accuracy of imaging technologies [50]. In these included studies, the characteristics of the patient varied, and patients in half of the studies may not be representative. Some of the included studies had very small sample size, and it might influence the estimation accuracy. Thirteen studies did not report whether the investigators were blinded to the results of reference standard test and relevant clinical information, and it might overestimate the accuracy of CTP. Only eight of twenty-seven studies had low risk of bias. Like other systematic reviews of the diagnostic test accuracy, the heterogeneities of the sensitivity and specificity in the included studies were high that may impact the reliability of the pooled results.

Conclusion
CTP is more accurate than NCCT and has similar accuracy to CTA in detecting acute ischemic stroke. However, the evidence is not strong. There is potential benefit of using CTP to select stroke patients for treatment, but more high-quality evidence is needed to confirm this result.
Supporting information S1