http://orcid.org/0000-0001-5865-6828
Figures
Abstract
Background and purpose
MR contrast-enhanced techniques are undergoing increased scrutiny since the FDA applied a warning for gadolinium-based MR contrast agents due to gadolinium deposition within multiple organ systems. While CE-MRA provides excellent image quality, is it required in a screening carotid study? This study compares 2D TOF and 3D TOF MRA vs. CE-MRA in defining carotid stenosis in a large clinical patient population, and with multiple readers with varying experience.
Materials and methods
200 consecutive patients had their carotid bifurcations evaluated with 2D TOF, 3D TOF and CE-MRA sequences by 6 board-certified neuroradiologists. Stenosis and quality of examinations were defined for each study. Inter-rater reliability was assessed using two-way random effects intraclass correlation coefficients. Intra-reader reliability was computed via weighted Cohen’s κ. Weighted Cohen’s κ were also computed to assess agreement in stenosis ratings between enhanced images and unenhanced images.
Results
Agreement between unenhanced and enhanced ratings was substantial with a pooled weighted κ of 0.733 (0.628–0.811). For 5 of the 6 readers, the combination of unenhanced 2D TOF and 3D TOF showed better agreement with contrast-enhanced than either 2D TOF or 3D TOF alone. Intra-reader reliability was substantial.
Conclusions
The combination of 2D TOF and 3D TOF MRA showed substantial agreement with CE-MRA regarding degree of carotid stenosis in this large outpatient population across multiple readers of varying experience. Given the scrutiny that GBCA are undergoing due to concerns regarding CNS and soft tissue deposition, it seems prudent to reserve CE-MRA for cases which are not satisfactorily answered by the nonenhanced study or other noninvasive examinations.
Citation: Ross JS, Buckner Petty SA, Brinjikji W, Hoxworth JM, Lehman VT, Middlebrooks EH, et al. (2020) Multiple reader comparison of 2D TOF, 3D TOF, and CEMRA in screening of the carotid bifurcations: Time to reconsider routine contrast use? PLoS ONE 15(9): e0237856. https://doi.org/10.1371/journal.pone.0237856
Editor: Nick Todd, Brigham and Women's Faulkner Hospital, UNITED STATES
Received: February 17, 2020; Accepted: August 4, 2020; Published: September 2, 2020
Copyright: © 2020 Ross et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
2D TOF MRA, 3D TOF MRA and dynamic contrast-enhanced MRA (CE-MRA) have all been used to evaluate the carotid bifurcations for the degree of stenosis to assist with medical and surgical management decisions. Of the many MR based techniques available, CE-MRA is generally considered the gold standard [1–7]. Multiple studies have demonstrated that both 2D TOF and 3D TOF unenhanced techniques are close to CE-MRA and DSA in sensitivity and specificity for carotid stenosis [5,8–10].
All MR contrast-enhanced techniques are now undergoing increased scrutiny since the FDA applied a class warning for gadolinium-based MR contrast agents (GBCA) with the discovery of gadolinium deposition within multiple organ systems in patients’ bodies after receiving these agents, even in patients without severe renal dysfunction [11–15]. This information has caused changes in type of GBCA administered and a reevaluation of the widespread use of GBCA [16–19].
Evaluation of MRA techniques, both enhanced and unenhanced, tend to be limited by small patient numbers, use within a research setting, or limited reader comparisons. This study adds to the voluminous literature of carotid bifurcation MR imaging by comparing 2D TOF and 3D TOF MRA (alone and in combination) vs. CE-MRA in defining carotid stenosis in a large clinical patient population, and with multiple readers with varying experience.
The hypothesis of this study is that the combination of 2D and 3D TOF MRA has equal agreement to CE-MRA for determining carotid stenosis (based on NASCET criteria) [20]. Given the added expense and potential risks of GBCA, the agreement of the unenhanced to the enhanced technique being corroborated in large outpatient population with a variety of stenoses could improve long term patient safety and decrease cost of imaging.
Materials and methods
This retrospective study was approved by the Mayo Clinic Institutional Review Board, and written consent waived for chart review. Mayo Clinic Arizona radiology database was searched for contrast-enhanced neck MRA, which also includes both 2D TOF and 3D nonenhanced TOF sequences.
200 consecutive patients who had evaluation of their carotid bifurcations with 2D TOF, 3D TOF and CE sequences as part of routine imaging protocols were included in this study. Patients ranged in age from 29–101 years, with an average of 66.2 years. There were 97 males, and 103 females evaluated.
Indications for the studies varied widely and included stroke (n = 63, 31.5%), headache (n = 36, 18%), follow-up of stenosis or occlusion (n = 27, 13.5%), vertigo or dizziness (n = 16, 8%), syncope (n = 12, 6%), transient ischemic attack (n = 11, 5.5%), vascular disease NOS (n = 6, 3%), vision disturbance (n = 5, 2.5%), family history of cerebrovascular disease (n = 5, 2.5%), aneurysm or AVM follow-up (N = 5, 2.5%), weakness (n = 3, 1.5%), numbness or paresthesia (n = 3, 1.5%), aphasia (n = 3, 1.5%) and cognitive change (n = 2), and one each for seizure, head trauma and preoperative screening.
MR angiography
All patients were examined on one of two 1.5T MR scanners (GE Signa HD, GE Medical Systems, Milwaukee, Wisconsin) between September 2017 and August 2018. All patients were examined with 2D TOF, 3D TOF and CE sequences, with the TOF sequences performed first with an Head-Neck-Spine coil. 2D TOF and 3D TOF MRA studies were acquired in the transverse orientation to cover the carotid bifurcations with axial coverage of 15cm and 6cm respectively. 2D TOF studies were performed with TR 26, TE 5.1–6.3, 1 excitation, 50-degree flip angle, matrix of 128x256, 1.5mm slice thickness, 16cm FOV, 5.5-minute acquisition time, bandwidth 125 Hz/pixel, voxel size 0.625 x 1.25 x 1.5mm. 3D studies were performed with 26/6.3–6.9 (TR/TE), 2 excitations, 20-degree FA, 128x256 matrix, 1.2mm slice thickness, 20cm FOV, 5-minute acquisition time, 5 overlapping volumes, bandwidth 122 Hz/pixel, voxel size 0.78 x 1.5 x 1.2 mm.
CE studies were performed in the coronal plane with 6.6–7.4ms/2.0–2.2ms (TR/TE), 1 excitation, 45-degree FA, 300-320x256 matrix, 0.7mm slice thickness, 30cm FOV, 1.0 minute acquisition time, voxel size 1 x 1.17 x 0.7 mm. Mask subtraction was utilized. The CE study was acquired with bolus tracking after the administration of 0.1mmol/kg bodyweight of gadolinium-based contrast agent (Gadovist), with an injection rate of 2mL/s, which was followed by a 20mL saline flush.
Image evaluation
Six board-certified neuroradiologists with varying years of practice since fellowship (2, 4, 7, 12, 19, 27 years) evaluated the image sets. 200 patients were evaluated, with a total of 400 carotid bifurcations, and a total of 1200 sequences (2D, 3D, CE).
Each of the 1200 MRA sequences were curated into 2–4 representative views of the carotid bifurcations by the senior author (JR -32 years of experience) who did not participate in the stenosis grading. All MRA images from all sequences were first evaluated by the senior author, and then similar oblique sagittal MIP projections were chosen across the 2D TOF, 3D TOF and CE-MRA acquisitions. Sagittal oblique MIP projections were picked that showed the greatest narrowing of the proximal internal carotid. When flow voids were present due to stenosis/occlusion, then representative axial images were included at the skull base to help differentiate near occlusion from occlusion.
The sequences were placed into one of five reading batches, with the 5 batches graded sequentially over time by the six readers, with a delay of 1–2 weeks between batch reading to minimize learning. The patient sequences were labeled consecutively as 2D right bifurcation (A), 2D left bifurcation (B), 3D right (C), 3D left (D), CE right (E), CE left (F) (i.e., 1A, 1B, 1C….200E, 200F). Placement of the 6 MRA sequences (A-F) into 5 batches allowed for an offset where each batch had only one MRA sequence of each bifurcation per patient to minimize intra-batch correlations (Fig 1). Each reading batch (1–5) contained 240 sequences which were viewed in Adobe Acrobat pdf (Adobe Inc, San Jose, CA) which allowed for direct stenosis measurement. Readers were blinded to other imaging or clinical data. There was no attempt at blinding of the specific sequence types, since they were obvious to these experienced readers.
Six different MR sequences for each patient were moved into 5 separate reading batches. In any one batch, the same carotid birfurcation was not seen twice.
Stenosis was graded on a six-point scale using NASCET criteria:
Grade 1: 0–30%; grade 2: 31–50%; grade 3: 51–70%; grade 4: >70%; grade 5: Carotid near-occlusion; grade 6: occluded. Near-occlusion was defined by long segment origin signal loss with diminished caliber of the distal cervical internal carotid.
Quality was measured qualitatively on a 5-point scale, 1 being the best and 5 the worst (uninterpretable). Quality was judged based on ideal appearance for the specific sequence being evaluated and not related to quality of 2D TOF vs 3D TOF or CE. The target of the quality interpretation was primarily the carotid system, and not the posterior circulation.
1 = Excellent
2 = Good
3 = Fair
4 = Poor
5 = Uninterpretable
Intra-observer variability was assessed by having a second interpretation of batch 3 (240 sequences) by all six readers greater than 1 week after completing the last batch 5.
Statistics
To assess inter-rater reliability among the 6 raters, two-way random effects intraclass correlation coefficients (ICC) were computed for ratings of stenosis and quality for each image type. 2D and 3D TOF measurements of each reader were compared against their own measurement on the corresponding CE images for each subject. Thus the ICC is a summary statistic of variation between readers on each carotid, in that the greater variability between readers leads to lower ICC. Based on Cicchetti [21], when the coefficient is below 0.40 the level of significance is poor; when between 0.40 and 0.59 it is fair; when between 0.60 and 0.74 is good; and between 0.75 and 1.00 is excellent.
To assess intra-rater reliability, each rater was randomly assigned 80 of each of the three types of images to provide a second set of ratings. Agreement between first and second ratings for each rater was computed via weighted (quadratic) Cohen’s Kappa (κ) coefficients. Weighted Cohen’s Kappa coefficients were also computed to assess agreement in stenosis ratings between enhanced images and unenhanced images (2D TOF, 3D TOF, and average of 2D TOF and 3D) among the 6 raters. Pooled estimates of the kappa coefficients were computed based on the method suggested by De Vries et al [22]. For each image we computed the mean of the 2D TOF and 3D TOF ordinal ratings and rounded to the nearest whole number. Enhanced images with image quality of 4.5 (uninterpretable) or greater were excluded from this analysis since the CE-MRA was used as the gold standard. Non-parametric bootstrapping was used to compute 95% confidence intervals around all kappa values. We computed sensitivity, specificity, and positive predictive values with 95% confidence intervals for the unenhanced ratings of >70% stenosis using the enhanced ratings as the gold-standard. All analyses were conducted using R version 3.6 (https://www.r-project.org/).
Results
Table 1 shows inter-rater reliability (agreement between raters). The 2D TOF stenosis measurements showed an intraclass correlation (ICC) of 0.810 (0.783–0.835 95% CI), 3D TOF stenosis 0.808 (0.783–0.833) and CE-MRA 0.728 (0.693–0.760). Agreement was strong for stenosis ratings despite weaker agreement for quality ratings.
Table 2 shows intra-rater reliability (agreement within raters) for each of the 6 readers. While some readers showed better consistency in their ratings than others, agreement was generally strong for most stenosis ratings. Taking all MRA techniques, the intra-rater reliability for stenosis was substantial, with kappa ranging from a low of 0.765 (reader 1, 3D) to a high of 0.986 (reader 4, 3D). Five of 6 readers (except for #3) showed higher intra-rater reliability for unenhanced 2D TOF or 3D TOF relative to the CE-MRA study.
Table 3 shows agreement between unenhanced and enhanced ratings (for each rater and overall). The 4th column compares the average of 2D TOF and 3D TOF against CE-MRA. Agreement was substantial with a pooled weighted κ of 0.733 (0.628–0.811). For 5 of the 6 readers, the combination of unenhanced 2D TOF and 3D TOF showed slightly better agreement with enhanced than either 2D TOF or 3D TOF alone. Reader #1 showed the highest agreement for unenhanced 2D TOF vs CE (0.841), with the combined 2D TOF/3D TOF a close second (0.832).
Analysis was also dichotomized into two grades to evaluate the differences in techniques for clinically significant disease: grades 1,2 (stenosis of 0–50%) were evaluated against grades 3–6 (>50% stenosis, occlusion, near occlusion) (Table 4). Agreement for the combined 2D TOF/3D TOF MRA vs CE-MRA continued to be good at 93.4%. Agreement was substantial (κ = 0.607), but decreased from the non-dichotomous evaluation.
Regarding quality of the studies, the 2D TOF studies were categorized as good/excellent in 65%, with 2.9% uninterpretable. The 3D TOF studies were categorized as good/excellent in 74.6%, with 1.6% uninterpretable. The CE studies were categorized as good/excellent in 78.6%, with 3.2% uninterpretable.
Table 5 shows sensitivity, specificity and PPV for each of the readers for stenosis greater or equal to 70%. Sensitivities, specificities and PPV values varied from highs of 0.778, 0.989, 0.750 to lows of 0.5, 0.970, 0.560 respectively.
Contingency tables for CE-MRA ratings vs unenhanced 2D TOF and 3D TOF ratings for overall results and individual readers are present in S1 Material. Also included in the Supplement are bubble plots with cell counts to visually show the agreement between readers for the 2D TOF, 3D TOF and CE-MRA studies.
Representative image sets demonstrating the variability of reader interpretations are shown in Figs 2 and 3.
Variability of interpretation of 2D (left), 3D (middle) and CE-MRA (right) for patient 9. Scores for the 6 readers were: 2D- 3,3,1,1,4,3 (average 2.5); 3D-1,1,1,1,2,2 (1.3); CE- 2,3,2,1,5,3 (2.7) where 1 = 0–30% stenosis, 2 = 31–50%, 3 = 51–70, 4 = >70%, 5 = near occlusion, 6 = occluded. Average quality ratings were 2DTOF = 2.6, 3DTOF = 3.8, CEMRA = 1.8.
Variability of interpretation of 2D (left), 3D (middle) and CE-MRA (right) for patient 113. Scores for the 6 readers were: 2D- 1,2,1,1,1,1 (average 1.2); 3D-1,1,1,1,1,1 (1.0); CE- 1,3,1,5,5,4 (3.2) where 1 = 0–30% stenosis, 2 = 31–50%, 3 = 51–70, 4 = >70%, 5 = near occlusion, 6 = occluded. Average quality rating were 2DTOF = 2.5, 3DTOF = 1.5, CEMRA = 3.1.
Discussion
This study has shown that the combination of 2D TOF and 3D TOF MRA showed substantial agreement with CE-MRA regarding degree of carotid stenosis in this large outpatient population across multiple readers of varying experience. We observed good agreement between readers despite having varied experience levels. Given the increased scrutiny that GBCA are undergoing due to concerns regarding CNS, soft tissue and bone marrow deposition, it seems prudent to reserve CE-MRA for those cases with are not satisfactorily answered by the nonenhanced study or other noninvasive examination [15,23,24]. That is, recalling the patient for a contrast study in the more limited situations where more information is needed, for vertebrobasilar symptoms or specific problem solving. Patients who are being evaluated preoperatively for symptomatic anterior circulation disease and have additional noninvasive imaging of the carotids (US, CTA) might also benefit from use of a CE-MRA evaluation to better define the vessel origins and proximal common carotid arteries. The European Society for Vascular Surgery guidelines give a class I recommendation that when carotid endarterectomy is being considered, that Duplex ultrasound stenosis estimation be corroborated by computed tomographic angiography or magnetic resonance angiography, or by a repeat Duplex ultrasound performed by a second operator [25]. The “14 Society” guidelines state that for evaluation of symptomatic disease: In patients with acute, focal ischemic neurological symptoms corresponding to the territory supplied by the left or right internal carotid artery, magnetic resonance angiography (MRA) or computed tomography angiography (CTA) is indicated to detect carotid stenosis when sonography either cannot be obtained or yields equivocal or otherwise nondiagnostic results [26].
This study demonstrates that while unenhanced MRA methods may not quite achieve the level of accuracy of CE-MRA, the unenhanced methods are likely satisfactory, particularly if being used as a screening examination in a high-volume outpatient setting with heterogeneous population with variable presenting symptoms. Our study is novel and adds to the extensive body of literature by having a large number of patients (200), multiple MRA sequences (2D TOF, 3D TOF, CE) and multiple readers [6] with varied experience where both intra- and inter-reader variabilities were defined.
High sensitivities ranging from 86% to 90% for detection of severe stenosis are generally reported for unenhanced techniques [27]. The sensitivity and specificity of 3D TOF MRA relative to rotational catheter angiography has been reported as 95.5% and 87.2% [7]. Babiarz et al showed sensitivities and specificities of CE-MRA (relative to CTA) at 84% and 96% respectively and for 2D TOF MRA 80% and 95% and they considered unenhanced 2D TOF MRA is sufficiently accurate to identify patients with >70% ICA stenosis compared with CE-MRA [28]. Wardlaw et al performed a systematic review showing that doppler ultrasound, MRA, CTA, and CEMRA all have high sensitivities and specificities for diagnosing 70–99% carotid stenosis in symptomatic patients [5]. CE-MRA was seen as marginally more accurate, but they cautioned on drawing conclusions from early research and not routine clinical practice. Westwood et al evaluated 26 articles in a systematic review primarily of unenhanced MRA and concluded that MRA was accurate for selecting patients for carotid endarterectomy at the surgical decision thresholds established in the major endarterectomy trials, but the evidence is not robust because of study heterogeneity [29]. Debrey et al performed a meta-analysis involving 37 TOF papers and 21 CE-MRA papers comparing TOF MR to digital subtraction angiography and concluded that TOF MRA and CE-MRA showed high accuracy for the detection of high-grade ICA stenoses and occlusions [30]. In that analysis, CE-MRA had a slight edge over TOF MRA with the sensitivity/specificity of TOF MRA for the detection of >70% to 99% ICA stenoses being 91.2/88.3%, and for CE-MRA 94.6/91.9%.
Several limitations are present in this study. First, the number of clinically significant stenosis was small relative to the normal or mild stenosis categories (0–30% = 1910 vs >50% = 214). These numbers of higher grade stenosis are comparable to prior studies evaluating these techniques [4,7].
Second, CE-MRA is an imperfect gold standard with defined limitations. CE-MRA techniques tended to overestimate the stenoses compared with catheter angiography [31–33]. This occurs both with high spatial resolution CE technique as well as for time-resolved techniques with reduced spatial resolution [1,34,35]. In a study of 177 patients, Babiarz et al found that while CE-MRA better defined the neurovascular anatomy, it did not offer a significant advantage in distinguishing surgically treatable ICA stenosis over 2D TOF MRA [28]. Serfaty et al found that 3D gadolinium-enhanced MR angiography was not accurate enough to replace conventional angiography in the evaluation of extracranial carotid arteries, but requires additional information such as Doppler sonography correlation [34].
A third limitation is that our TOF techniques do not routinely encompass the vertebral origins. However, screening for posterior circulation disease is not typically recommended. The risk of stroke in patients with asymptomatic vertebral artery stenosis is much lower than for symptomatic vertebral artery stenosis. In a study of 3717 patients with atherosclerotic disease, 7.6% had asymptomatic vertebral stenoses by US, with an annual stroke risk of 0.2% [25,36]. The European Society of Vascular Surgery practice guidelines state that population screening for asymptomatic vertebral artery stenosis is not recommended, and asymptomatic vertebral stenoses should not be treated by open or endovascular interventions [25]. A completely different strategy is present if the patient has had a posterior circulation stroke. In that population, the 90-day risk of recurrent stroke is 16% in those with extracranial vertebral artery stenosis and intervention should be undertaken early after symptom onset [25,37]. With posterior fossa symptomatology, imaging solely with unenhanced MRA techniques would not be appropriate.
Fourth, this study does not cover all field strengths nor all potential unenhanced sequences for evaluation of the carotid bifurcations. Newer techniques, such as ungated radial quiescent interval slice-selective imaging (QISS) have shown promise for unenhanced imaging which is comparible to CEMRA, but have not found wide use and are currently vendor specific [37]. While extensive data is available in the literature on the utility and accuracy of both 1.5T and 3.0T MRA, we cannot define the generalizability of this specific data to 3T. Finally, for expediency sake, the data sets were curated so that the blinded readers in different geographic regions could easily avail themselves of the datasets, and the probability of completing the study markedly increased with the more limited imaging evaluation. All images available in the MRA data sets were not evaluated by the readers. The effect of curated views is unknown relative to the evaluation of all images associated with the three different MRA sequences. The presence of a limited data set could potentially improve inter-reader variability, or decrease accuracy for defining stenosis since not all projections of the MIP images are evaluated.
Fifthly, flow dephasing with signal loss was expected to occur with higher grades of stenosis and included in the estimate of stenosis for these experienced readers. General degrees of signal loss were considered to occur at >70% for the 2D TOF, and for >90% for the 3D TOF studies, but were assumed and could not be explicitly measured.
Conclusion
These results should reassure providers and radiologists that in an outpatient setting, initial evaluation of the carotid bifurcations can be adequately performed with unenhanced 2D TOF and 3D TOF MRA. Contrast enhanced MRA should be considered reserved for those cases not successfully answered by the unenhanced study, for preoperative evaluation, patients with evidence of clinically significant stenosis by other noninvasive modalities or vertebrobasilar symptoms. This has clinical relevance given the shift in clinical practice that is occurring regarding the benefit and risk of GBCA due to systemic gadolinium deposition following GBC administration.
References
- 1. Huston J, Fain SB, Wald JT, Luetmer PH, Rydberg CH, Covarrubias DJ, et al. Carotid artery: Elliptic centric contrast-enhanced MR angiography compared with conventional angiography. Radiology. 2001;218(1):138–43. pmid:11152792
- 2. Randoux B, Marro B, Koskas F, Duyme M, Sahel M, Zouaoui A, et al. Carotid artery stenosis: prospective comparison of CT, three-dimensional gadolinium-enhanced MR, and conventional angiography. Radiology [Internet]. 2001 Jul;220(1):179–85. Available from: http://www.ncbi.nlm.nih.gov/pubmed/11425993 pmid:11425993
- 3. Long A, Lepoutre A, Corbillon E, Branchereau A. Critical review of non- or minimally invasive methods (duplex ultrasonography, MR- and CT-angiography) for evaluating stenosis of the proximal internal carotid artery. Eur J Vasc Endovasc Surg. 2002;24(1):43–52. pmid:12127847
- 4. Alvarez-Linera J, Benito-Leon J, Escribano J, Campollo J, Gesto R. Prospective evaluation of carotid artery stenosis: elliptic centric contrast-enhanced MR angiography and spiral CT angiography compared with digital subtraction angiography. AJNR Am J Neuroradiol [Internet]. 2003;24(5):1012–9. Available from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=12748115 pmid:12748115
- 5. Wardlaw J, Chappell F, Best J, Wartolowska K, Berry E. Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a meta-analysis. Lancet [Internet]. 2006 May;367(9521):1503–12. Available from: http://linkinghub.elsevier.com/retrieve/pii/S0140673606686509 pmid:16679163
- 6. Chappell FM, Wardlaw JM, Young GR, Gillard JH, Roditi GH, Yip B, et al. Carotid artery stenosis: Accuracy of noninvasive tests—Individual patient data meta-analysis. Radiology. 2009;251(2):493–502. pmid:19276319
- 7. Anzalone N, Scomazzoni F, Castellano R, Strada L, Righi C, Politi LS, et al. Carotid Artery Stenosis: Intraindividual Correlations of 3D Time-of-Flight MR Angiography, Contrast-enhanced MR Angiography, Conventional DSA, and Rotational Angiography for Detection and Grading. Radiology [Internet]. 2005 Jul;236(1):204–13. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2361032048 pmid:15955853
- 8. Kramer H, Runge VM, Morelli JN, Williams KD, Naul LG, Nikolaou K, et al. Magnetic resonance angiography of the carotid arteries: Comparison of unenhanced and contrast enhanced techniques. Eur Radiol. 2011;21(8):1667–76. pmid:21476128
- 9. Weber J, Veith P, Jung B, Ihorst G, Moske-Eick O, Meckel S, et al. MR Angiography at 3 Tesla to Assess Proximal Internal Carotid Artery Stenoses: Contrast-Enhanced or 3D Time-of-Flight MR Angiography? Clin Neuroradiol [Internet]. 2015 Mar 3;25(1):41–8. Available from: http://link.springer.com/10.1007/s00062-013-0279-x pmid:24384680
- 10. Masaryk AM, Ross JS, DiCello MC, Modic MT, Paranandi L, Masaryk TJ. 3DFT MR angiography of the carotid bifurcation: potential and limitations as a screening examination. Radiology [Internet]. 1991 Jun;179(3):797–804. Available from: http://www.ncbi.nlm.nih.gov/pubmed/2027995 pmid:2027995
- 11. Kanda T, Fukusato T, Matsuda M, Toyoda K, Oba H, Kotoku J, et al. Gadolinium-based Contrast Agent Accumulates in the Brain Even in Subjects without Severe Renal Dysfunction: Evaluation of Autopsy Brain Specimens with Inductively Coupled Plasma Mass Spectroscopy. Radiology [Internet]. 2015;276(1):228–32. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2015142690 pmid:25942417
- 12. McDonald RJ, McDonald JS, Kallmes DF, Jentoft ME, Murray DL, Thielen KR, et al. Intracranial Gadolinium Deposition after Contrast-enhanced MR Imaging. Radiology [Internet]. 2015;000(0):150025. Available from: http://pubs.rsna.org/doi/10.1148/radiol.15150025
- 13. McDonald RJ, McDonald JS, Kallmes DF, Jentoft ME, Paolini MA, Murray DL, et al. Gadolinium Deposition in Human Brain Tissues after Contrast-enhanced MR Imaging in Adult Patients without Intracranial Abnormalities. Radiology [Internet]. 2017;285(2):161595. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2017161595
- 14. Murata N, Gonzalez-Cuyar LF, Murata K, Fligner C, Dills R, Hippe D, et al. Macrocyclic and other non-group 1 gadolinium contrast agents deposit low levels of gadolinium in brain and bone tissue: Preliminary results from 9 patients with normal renal function. Invest Radiol. 2016;51(7):447–53. pmid:26863577
- 15. Minaeva O, Hua N, Franz ES, Lupoli N, Mian AZ, Farris CW, et al. Nonhomogeneous Gadolinium Retention in the Cerebral Cortex after Intravenous Administration of Gadolinium-based Contrast Agent in Rats and Humans. Radiology [Internet]. 2019 Nov 26;190461. Available from: http://www.ncbi.nlm.nih.gov/pubmed/31769744
- 16. Gulani V, Calamante F, Shellock FG, Kanal E, Reeder SB. Gadolinium deposition in the brain: summary of evidence and recommendations. Lancet Neurol [Internet]. 2017;16(7):564–70. Available from: pmid:28653648
- 17. McDonald RJ, Levine D, Weinreb J, Kanal E, Davenport MS, Ellis JH, et al. Gadolinium Retention: A Research Roadmap from the 2018 NIH/ACR/RSNA Workshop on Gadolinium Chelates. Radiology. 2018;289(2):517–34. pmid:30204075
- 18. Welk B, McArthur E, Morrow SA, MacDonald P, Hayward J, Leung A, et al. Association between gadolinium contrast exposure and the risk of parkinsonism. JAMA—J Am Med Assoc. 2016;316(1):96–8.
- 19. Mithal LB, Patel PS, Mithal D, Palac HL, Rozenfeld MN. Use of gadolinium-based magnetic resonance imaging contrast agents and awareness of brain gadolinium deposition among pediatric providers in North America. Pediatr Radiol. 2017;47(6):657–64. pmid:28283727
- 20. North American Symptomatic Carotid Endarterectomy Trial Collaborators, Barnett HJM, Taylor DW, Haynes RB, Sackett DL, Peerless SJ, et al. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med [Internet]. 1991;325(7):445–53. Available from: http://www.ncbi.nlm.nih.gov/pubmed/1852179 pmid:1852179
- 21. Cicchetti D V. Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. Psychol Assess. 1994;6(4):284–90.
- 22. De Vries H, Elliott MN, Kanouse DE, Teleki SS. Using pooled kappa to summarize interrater agreement across many items. Field methods. 2008;20(3):272–82.
- 23. Maximova N, Gregori M, Zennaro F, Sonzogni A, Simeone R, Zanon D. Hepatic gadolinium deposition and reversibility after contrast agent-enhanced MR imaging of pediatric hematopoietic stem cell transplant recipients. Radiology. 2016;281(2):418–26. pmid:27276243
- 24. Guo BJ, Yang ZL, Zhang LJ. Gadolinium Deposition in Brain: Current Scientific Evidence and Future Perspectives. Front Mol Neurosci. 2018;11(September):1–12.
- 25. Naylor AR, Ricco JB, de Borst GJ, Debus S, de Haro J, Halliday A, et al. Management of Atherosclerotic Carotid and Vertebral Artery Disease: 2017 Clinical Practice Guidelines of the ESVS. Eur J Vasc Endovasc Surg [Internet]. 2018;55(1):3–81. Available from: pmid:28851594
- 26. Brott TG, Halperin JL, Abbara S, Bacharach JM, Barr JD, Bush RL, et al. 2011 ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS Guideline on the Management of Patients With Extracranial Carotid and Vertebral Artery Disease. J Am Coll Cardiol [Internet]. 2011 Feb;57(8):e16–94. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0735109710045717 pmid:21288679
- 27. U-King-Im JM, Young V, Gillard JH. Carotid-artery imaging in the diagnosis and management of patients at risk of stroke. Lancet Neurol [Internet]. 2009;8(6):569–80. Available from: pmid:19446276
- 28. Babiarz LS, Romero JM, Murphy EK, Brobeck B, Schaefer PW, González RG, et al. Contrast-Enhanced MR Angiography Is Not More Accurate Than Unenhanced 2D Time-of-Flight MR Angiography for Determining ≥70% Internal Carotid Artery Stenosis. Am J Neuroradiol [Internet]. 2009 Apr [cited 2012 Oct 30];30(4):761–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19164440 pmid:19164440
- 29. Westwood ME, Kelly S, Berry E, Bamford JM, Gough MJ, Airey CM, et al. Use of magnetic resonance angiography to select candidates with recently symptomatic carotid stenosis for surgery: Systematic review. Br Med J. 2002;324(7331):198–201.
- 30. Debrey SM, Yu H, Lynch JK, Lövblad K-O, Wright VL, Janket S-JD, et al. Diagnostic Accuracy of Magnetic Resonance Angiography for Internal Carotid Artery Disease. Stroke [Internet]. 2008 Aug;39(8):2237–48. Available from: https://www.ahajournals.org/doi/10.1161/STROKEAHA.107.509877 pmid:18556586
- 31. Fellner C, Lang W, Janka R, Wutke R, Bautz W, Fellner FA. Magnetic resonance angiography of the carotid arteries using three different techniques: Accuracy compared with intraarterial x-ray angiography and endarterectomy specimens. J Magn Reson Imaging. 2005;21(4):424–31. pmid:15779039
- 32. Cosottini M, Calabrese R, Puglioli M, Zampa V, Michelassi MC, Ortori S, et al. Contrast-enhanced three-dimensional MR angiography of neck vessels: Does dephasing effect alter diagnostic accuracy? Eur Radiol. 2003;13(3):571–81. pmid:12594561
- 33. Townsend TC, Saloner D, Pan XM, Rapp JH. Contrast material–enhanced MRA overestimates severity of carotid stenosis, compared with 3D time-of-flight MRA. J Vasc Surg [Internet]. 2003 Jul;38(1):36–40. Available from: http://linkinghub.elsevier.com/retrieve/pii/S074152140300332X pmid:12844086
- 34. Serfaty JM, Chirossel P, Chevallier JM, Ecochard R, Froment JC, Douek PC. Accuracy of three-dimensional gadolinium-enhanced MR angiography in the assessment of extracranial carotid artery disease. AJR Am J Roentgenol [Internet]. 2000;175(2):455–63. Available from: http://www.ncbi.nlm.nih.gov/pubmed/10915694 pmid:10915694
- 35. Remonda L, Senn P, Barth A, Arnold M, Lövblad K-O, Schroth G. Contrast-enhanced 3D MR angiography of the carotid artery: comparison with conventional digital subtraction angiography. AJNR Am J Neuroradiol [Internet]. 2002 Feb;23(2):213–9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/11847044 pmid:11847044
- 36. Compter A, van der Worp HB, Algra A, Kappelle LJ. Prevalence and Prognosis of Asymptomatic Vertebral Artery Origin Stenosis in Patients With Clinically Manifest Arterial Disease. Stroke [Internet]. 2011 Oct;42(10):2795–800. Available from: https://www.ahajournals.org/doi/10.1161/STROKEAHA.110.612903 pmid:21852605
- 37. Gulli G, Marquardt L, Rothwell PM, Markus HS. Stroke risk after posterior circulation stroke/transient ischemic attack and its relationship to site of vertebrobasilar stenosis: Pooled data analysis from prospective studies. Stroke. 2013;44(3):598–604. pmid:23386676