https://orcid.org/0000-0002-9275-0364
Figures
Abstract
Objectives
In breast diffusion weighted imaging (DWI) protocol standardization, it is recently shown that no breast tumor tissue selection (BTTS) method outperformed the others. The purpose of this study is to analyze the feasibility of three fixed-size breast tumor tissue selection (BTTS) methods based on the reproducibility, accuracy and time-measurement in comparison to the largest oval and manual delineation in breast diffusion weighted imaging data.
Methods
This study is performed with a consecutive dataset of 116 breast lesions (98 malignant) of at least 1.0 cm, scanned in accordance with the EUSOBI breast DWI working group recommendations. Reproducibility of the maximum size manual (BTTS1) and of the maximal size round/oval (BTTS2) methods were compared with three smaller fixed-size circular BTTS methods in the middle of each lesion (BTTS3, 0.12 cm3 volume) and at lowest apparent diffusion coefficient (ADC) (BTTS4, 0.12 cm3; BTTS5, 0.24 cm3). Mean ADC values, intraclass-correlation-coefficients (ICCs), area under the curve (AUC) and measurement times (sec) of the 5 BTTS methods were assessed by two observers.
Results
Excellent inter- and intra-observer agreement was found for any BTTS (with ICC 0.88–0.92 and 0.92–0.94, respectively). Significant difference in ADCmean between any pair of BTTS methods was shown (p = <0.001–0.009), except for BTTS2 vs. BTTS3 for observer 1 (p = 0.10). AUCs were comparable between BTTS methods, with highest AUC for BTTS2 (0.89–0.91) and lowest for BTTS4 (0.76–0.85). However, as an indicator of clinical feasibility, BTTS2-3 showed shortest measurement times (10–15 sec) compared to BTTS1, 4–5 (19–39 sec).
Conclusion
The performance of fixed-size BTTS methods, as a potential tool for clinical decision making, shows equal AUC but shorter ADC measurement time compared to manual or oval whole lesion measurements. The advantage of a fixed size BTTS method is the excellent reproducibility. A central fixed breast tumor tissue volume of 0.12 cm3 is the most feasible method for use in clinical practice.
Citation: Wielema M, Sijens PE, Dijkstra H, De Bock GH, van Bruggen IG, Siegersma JE, et al. (2021) Diffusion weighted imaging of the breast: Performance of standardized breast tumor tissue selection methods in clinical decision making. PLoS ONE 16(1): e0245930. https://doi.org/10.1371/journal.pone.0245930
Editor: Pascal A. T. Baltzer, Medical University of Vienna, AUSTRIA
Received: October 23, 2020; Accepted: January 8, 2021; Published: January 25, 2021
Copyright: © 2021 Wielema et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The study’s minimal underlying data set is made available on: https://doi.org/10.7910/DVN/E6QARM.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Breast Dynamic Contrast Enhanced MRI (DCE-MRI) has the highest negative predictive value of all imaging diagnostic techniques in the exclusion of breast malignancy [1, 2]. However, overlap in enhancement patterns of malignant and benign breast lesions exists. Diffusion Weighted Imaging (DWI) in addition to DCE-MRI improves the specificity of breast MRI and can prevent unnecessary biopsies in benign lesions [3, 4]. However, DWI cannot be used as a stand-alone parameter [5]. DWI measures the diffusion of hydrogen protons in a voxel due to Brownian motion and is most often quantified in a mono-exponential model, using the apparent diffusion coefficient (ADC). There are initiatives to improve and standardize DWI protocols, however, further research is needed [6–8]. In image analysis, literature is inconclusive on the influence of breast tumor tissue selection (BTTS) methods on the accuracy of DWI in the discrimination of benign from malignant lesions. Some authors state that the applied tumor tissue selection method (by definition of a region of interest) influences the ADC outcome [9–12], which thereby could affect the differentiation between malignant and benign breast lesions [13]. However, no superior BTTS method was found, due to the high heterogeneity in the available data, in a recent meta-analysis [14]. Therefore, there is a need to compare the accuracy of the five most used BTTS methods in the same data set, acquired with a robust MRI protocol. Furthermore, data is lacking on which method is most feasible to implement as measured by the amount of time needed to perform the assessment.
The purpose of this study is to evaluate the reproducibility, time measurement and accuracy of fixed size and shape breast tumor tissue selection methods compared to conventionally drawn tumor tissue delineation methods. For the radiologist, a standard fixed size BTTS method would be expected to save time and improve robustness of breast lesion ADC measurement.
Materials and methods
Patient population
A consecutive sample of 105 women (mean 48 years (range: 23–75)) with 116 enhancing breast masses (98 malignant) were included between April 2010 and June 2015. "The medical ethical committee of the University Medical Center Groningen approved the study and waived the need for informed consent due to the retrospective nature of the study (METc Nr: 2016/379). However, all participants were checked for registration in the local legal “opt-out of research system”. None of the included participants opted out". Lesion diameter was at least 1.0 cm, with an area of ≥ 0.8 cm2. Indications for breast MRI consisted of pre-operative/pre-chemotherapy evaluation, problem solving and screening of high-risk women. Non-mass enhancement lesions were excluded to reduce partial volume effect based on the known limited value of DWI in non-mass lesions [15]. Exclusion criteria were: previous breast malignancy, breast implants and simple cysts. Final diagnosis was acquired by pathology or follow-up of at least 2 years.
Data acquisition
MRI examinations were performed using a 1.5 Tesla (T) system (Magnetom Avanto, Siemens Healthineers), with a circularly polarized bilateral breast coil (Siemens Healthineers), with patients in prone position. The MRI protocol consisted of pre-contrast T1, T2, DWI and 5–7 DCE-T1-weighted series. DWI was performed with single shot—echo planar imaging (SS-EPI) with spectral attenuated inversion recovery (SPAIR) fat suppression and b-values of 0, 50, 200, 500, 800 and 1000 s/mm2 (TR/TE 9300/91 ms, FOV 170 x 340 mm2, matrix 192 x 384, bandwidth 1628 Hz/pixel, slice thickness 4 mm, inter-slice gap 2 mm). Acquisition time of DWI was 5 minutes and 15 seconds. In this study, the automatically calculated ADC maps with b = 0 and b = 1000 s/mm2 were used because of their proven high accuracy in literature [8, 13].
Image analysis
A radiologist with 8 years of experience in breast MRI (MDD) and a radiologist in training (MW) localized the slice with the highest lesion diameter on DCE-T1 images in consensus. Two observers (observer 1 (IVB), technical medicine physicist in training, observer 2 (JES) clinical physicist in training) independently positioned the 5 BTTSs in each lesion. Both observers were trained and tested in tumor delineation in an independent sample of 25 breast MRI tumor supervised cases. Observers were blinded to all clinical data. Observer 1 repeated all measurements after one month. Fig 1 shows the BTTS methods that were compared: BTTS1: Manual, whole breast tumor tissue selection volume; BTTS2: Oval shaped, whole breast tumor tissue selection, encompassing as much of the lesion as possible while staying within its borders; BTTS3: Standardized fixed circle of 0.3 cm2 (x 4 mm slice thickness = volume of 0.12 cm3) in the middle of the lesion; BTTS4: standard circular fixed area of 0.3 cm2 (volume of 0.12 cm3) and BTTS5: standard circular fixed area of 0.6 cm2 (volume of 0.24 cm3). Both BTTS4 and BTTS5 were positioned to obtain the lowest mean ADC, as an indicator of the most cellular part of the lesion, while avoiding necrotic parts. BTTS1 was positioned on the DCE-T1 series and copied to the ADC-map. BTTS2-5 were positioned on the ADC map. In several cases, DWI series and DCE-T1 series were not linked correctly. To correct for this registration mismatch, BTTS1 was manually moved up or down in the same slice, to where the lesion was clearly seen. Dedicated software was used for image analysis: Multiview (Hologic).
BTTS = Breast tumor tissue selection, ADC = Apparent diffusion coefficient.
Time measurements
Measurement times were registered using an online stopwatch tool (http://stopwatch.online-timers.com/online-stopwatch). Time measurement of the BTTS methods only consisted of BTTS placement. Slice selection was not included in the measurement, since it is similar for all methods. As planned on forehand, the first 10 consecutive cases were used to train the observers in using the online stopwatch tool and were not included in the time measurement analysis. The next 50 consecutive cases were timed for both observers separately and included in the data analysis. Time measurements were performed in the first session of the two tumor tissue delineation sessions.
Statistical analysis
For statistical analysis IBM SPSS Statistics 23 and MedCalc (version 12.5.0.0) were used. Average ADC (ADCmean) and minimal ADC (ADCmin) values of BTTS1-5 were measured for each breast lesion. The size (area, mm2) of BTTS1-2 was recorded. Average and minimal ADC’s of benign and malignant lesion groups were separately tested for normal distribution using Shapiro-Wilk test. Due to the non-normal distribution, median and interquartile ranges (IQR) were used in further statistical testing. ADC values of benign and malignant lesions were compared for each BTTS method using Mann-Whitney U tests (for unrelated samples). Wilcoxon signed rank test (for related samples) was used to compare ADC values between BTTS methods. Intra- and inter-observer agreement was calculated by using the Intraclass Correlation Coefficient (ICC) of measured ADC values for each BTTS method. In the discrimination between benign and malignant lesions of the different BTTS methods, the area under the ROC curve (AUC) ±standard error (SE) of ADC was measured for each BTTS method per observer. The method of DeLong et al. was used to compare the AUC’s (using the AUC’s ±SE) [16]. Time measurements were normally distributed and compared using repeated measurement ANOVA for both observers separately. Further post-hoc pairwise comparison was performed with a Bonferroni post hoc test. A p-value of <0.05 was considered to indicate a statistically significant difference.
Results
Lesions characteristics
Out of the 116 enhancing breast lesions 98 were malignant and 18 benign. Malignant lesions consisted of: invasive carcinoma no special type (n = 80); invasive lobular carcinoma (n = 13); ductal carcinoma in situ (n = 1); invasive papillary carcinoma (n = 1); malignant phyllodes tumor (n = 1); mucinous carcinoma (n = 1) and medullary carcinoma (n = 1). Benign lesions were: fibroadenoma (n = 8); sclerosing adenosis/columnar cell changes/apocrine metaplasia (n = 6); chronic inflammation (n = 3); and benign phyllodes (n = 1). For BTTS1, the mean area was 4.1 ± 3.9 cm2 for malignant lesions with a range between 0.8 cm2 and 23.4 cm2; for benign lesions the mean area was 3.7 ± 4.2 cm2 (range: 0.8–17.8 cm2). For BTTS2 the mean area was 2.3 ± 2.4 cm2 (range: 0.6–19.1 cm2) and 1.8 ± 2.1 cm2 (range: 0.6–12.2 cm2) for malignant and benign lesions, respectively. BTTS 3–5 were of standard size and shape. Figs 2–5 show examples of the BTTS methods in both malignant and benign breast lesions.
DCE-T1 subtracted images (inverted) (A) and DWI b0-1000 images (B).
Surrounding lobular carcinoma in situ. DCE-T1 subtracted images (inverted) (A) and DWI b0-1000 images (B).
DCE-T1 subtracted images (inverted) (A) and DWI b0-1000 images (B).
MRI guided biopsy proven and unchanged in 4 years of follow-up. DCE-T1 subtracted images (inverted) (A) and DWI b0-1000 images (B). BTTS = Breast tumor tissue selection.
ADC values
Table 1 shows the mean lesion ADC (ADCmean, mm2/s) and, for a comparison, also the minimum pixel ADC values (ADCmin, mm2/s) for the five BTTS methods. Values are given as median (± IQR) ADC, due to the non-normal distribution of the data. Both ADCmean (p<0.00) and ADCmin (p<0.00–0.038) showed significantly different values for benign vs. malignant lesions for each BTTS method. Median values of ADCmean for benign lesions ranged 1.29–1.74 mm2/s and 0.72–1.11 mm2/s for malignant lesions. Statistical analysis showed a significant difference in ADCmean between any pair of the 5 BTTS methods (p = 0.000–0.009), except for BTTS2 vs. BTTS3 for observer 1 (p = 0.10) (Table 2).
Results are presented per observer (and session) and for malignant and benign lesions separately. P-values indicate a significant difference between malignant and benign lesions for each BTTS method, per observer.
Inter- and intra-observer variability
Table 3 shows the inter- and intra- observer agreement in lesion ADC values obtained in the 118 breast lesions, per BTTS method. Excellent inter- and intra-observer agreement was found for BTTS2 and BTTS5 (ICC >0.9). Good agreement was found for the other BTTS methods. ADCmean showed higher inter-observer and intra-observer agreement than ADCmin.
Accuracy analysis
In the analysis of the influence of the BTTS methods on the accuracy, as measured by the area under the ROC curve (AUC), ADCmean showed higher AUCs compared to ADCmin (Table 4). BTTS methods measuring ADCmean showed comparable AUCs with the highest AUC of 0.889–0.911 for BTTS2 (Tables 4 and 5). Only for observer 1 BTTS1 and BTTS2 showed significantly different AUCs in the second reading session, due to a lower AUC for BTTS1 compared to the first reading session and compared to observer 2.
Results for ADCmean and ADCmin separately.
Results are presented for ADCmean.
Time measurements
As presented in Table 6, BTTS2 and BTTS3 were the fastest lesion ADC measurement methods. BTTS2 (whole lesion, oval) showed a mean measurement time of 13.4/14.9 seconds (2 observers) and BTTS3 (center 0.3cm2, round) 13.8/9.9 seconds (2 observers), compared to mean measurement times of at most 38.8 seconds for BTTS1 (manual whole lesion) by observer 2. The type of BTTS method was of significant influence on the measurement time (p<0.001). Post-hoc pairwise comparison did not show a significant difference between the measurement time of BTTS2 and BTTS3 (p = 1.00), which also applies for BTTS4 and BTTS5 (p = 0.544). The other BTTS methods significantly differed in measurement times (p<0.01).
Discussion
In this study on the reproducibility, accuracy and measurement time of the most widely used conventional BTTS methods and fixed size tumor delineation, ADC could discriminate benign from malignant lesions. ADCmean showed better overall performance than ADCmin, with good to excellent inter-observer agreement. In the AUC comparison, this study confirms the literature based hypothesis of no significant influence of the BTTS method on the discrimination between benign and malignant breast lesions.
Not a single BTTS method outperformed in lesion differentiation by ADC measurement, due to the high heterogeneity in available data in a recent meta-analysis [14]. The need for robust analysis of BTTS methods in an independent database was evident, especially because of the importance of the breast DWI protocol and image analysis standardization written in the latest consensus statement of the EUSOBI, reporting no consensus on the breast tumor tissue selection method [8].
A comparable high reproducibility of ADCmean for the fixed-size methods (BTTS3-5) with inter- and intra-observer ICCs of 0.882–0.939 is shown in the present study compared to the whole lesion methods (BTTS 1–2), with inter- and intra-observer ICCs of 0.899–0.940. The ADCmean showed higher agreement and AUC than ADCmin measurements with inter-observer ICCs of 0.882–0.940 vs. 0.742–0.875, respectively.
For ADC mean, all 5 BTTS methods showed comparable AUCs, except for BTTS1 vs. BTTS2 for observer 1 reading session 2. The concern that BTTS 1–3 might include the necrotic part of a lesion, which potentially causes false negative results based on higher mean ADC values can be neglected since the BTTS2 (oval shaped, whole lesion) and BTTS3 (standardized fixed volume of 0.12 cm3) showed comparable high AUCs of 0.89–0.91 and 0.86–0.88, respectively.
Furthermore, measurement times were shorter for the central volume (0.12 cm3) measurement, BTTS3 (13.8/9.6 sec, 2 observers) and the round/oval whole breast tumor tissue selection method, BTTS2 (13.4/14.9 sec, 2 observers) than for the other methods. Therefore, no time consuming methods of conventional manual tumor tissue delineation such as BTTS1 (19.2/38.8 sec, 2 observers) are necessary. Moreover, there is no need to spend time selecting the breast tumor area of lowest diffusion (BTTS 4 and BTTS5) as an indicator of the most cellular part. So far, only Bickel et al. included time measurements as a measure of user’s convenience [9].
This study was performed in accordance with the standardized protocol recommended in the consensus and mission statement of the EUSOBI International breast DWI working group [8]. This protocol consists of axial SS- EPI-DWI with SPAIR fatsupression, a slice thickness of 4 mm, b-values of 0–1000 s-mm2 a TR of 9300 ms (>3000), and the lowest possible TE of 91 ms. Bickel et al. showed comparable ICCs for ADCmean and ADCmin for their small and large breast tumor tissue selection methods, with highest ICC for ADCmean with a large tumor tissue selection (inter-observer ICC of 0.85 and intra-observer ICC of 0.89) in comparison to the available literature [9]. Time measurements, with shortest measurement time for a small BTTS methods of 7s (range: 3.3–23.7s) were comparable to those of for BTTS3 (9.6/13.8s) central fixed size measurement in the current study (2 observers). However, they presented higher AUCs for ADCmin (0.95/0.96) than in this study (0.66–0.81). Giannotti et al. showed comparable good inter and intra-observer agreement (0.864–0.997) for ADCmean, with fair inter-observer ICCs of 0.677 for ADCmin in 52 malignant lesions [17]. In the measurement of diffusion, fat containing voxels may show an ultralow ADC value, which could lead to false positive results in benign lesions when using ADCmin as measurement method instead of ADCmean. This partly explains the lower AUC for ADCmin compared to ADCmean, which is illustrated in Table 1 column 5–7, showing relatively low ADCmin values for benign lesions.
Nogueira et al. compared the ADCmean values of 2 observers: inter-observer agreement was excellent for a manual whole lesion selection (ICC = 0.97) and a 10mm2 lowest diffusion breast tumor tissue selection (ICC = 0.98), which is higher than in the current study, but measured in significantly fewer (n = 39) lesions [18]. Arponen et al. found a lower intra- and inter-observer agreement: ICC of 0.817 and 0.831 for whole lesion BTTS, respectively, versus 0.707 and 0.589 for lowest diffusion BTTS, respectively [11].
One of the known limitations of DWI is its low spatial resolution. Small lesions, such as small cancer foci, or scattered foci may not be identifiable on DWI. Most studies use a lesion diameter of 1.0 cm as a threshold. Smaller lesions are excluded [19]. No data are available on the minimum size of the lesion that can be detected by DWI, which is dependent on the scanning protocol (slice thickness and interslice gap). By confining our study to lesions larger or equal to 1.0 cm (0.8 cm2) and excluding non-mass enhanced lesions, we have limited the influence of partial volume effects on the reported ADC values [15].
Furthermore, it was noted that in some cases the DWI series and the DCE-T1 were visually not correctly linked. This is well known and is due to the difference in slice thickness of DWI and DCE-T1 in particular. To correct for this registration mismatch, BTTS1 was positioned to the right location on the same slice based on anatomical and lesion landmarks. This might have resulted in a slightly lower inter- and intra-observer agreement.
Moreover, the relatively small number of benign lesions (n = 18) compared to the 98 malignant lesions might have caused selection bias. To our knowledge, the use of 1.5T instead of 3.0T is not considered a limitation, because of the proven equal diagnostic accuracy [8, 20].
In this study, the presented breast tumor tissue selection methods all showed fair AUC’s. However, the importance of adding DWI to the breast MRI protocol is to prevent unnecessary biopsies. It is no option yet to replace histological biopsies in the diagnostic algorithm of breast masses with MRI (including DWI) and with MRI as a single diagnostic tool, since for now cancers will be missed. This is a general limitation of DWI in enhancing breast lesions. Histological diagnosis is still required in clinic and remains the gold standard. IVIM or machine learning techniques could be of interest in this matter, and should be addressed more in future studies, for example with the introduction of automated breast tumor tissue selection.
Conclusion
The performance of fixed-size BTTS methods as a potential tool for clinical decision making shows equal AUC but shorter ADC measurement time compared to manual or oval whole lesion measurements. A fixed size BTTS method is advantageous because of its excellent reproducibility. A central fixed breast tumor tissue volume of 0.12 cm3 is the most feasible method for use in clinical practice.
References
- 1. Dorrius MD, Pijnappel RM, Sijens PE, van der Weide MCJ, Oudkerk M. The negative predictive value of breast Magnetic Resonance Imaging in noncalcified BIRADS 3 lesions. Eur J Radiol. 2012;81: 209–213. pmid:21251784
- 2. Strobel K, Schrading S, Hansen NL, Barabasch A, Kuhl CK. Assessment of Bi-raDs category 4 lesions detected with screening mammography and screening us: Utility of MR imaging. Radiology. 2015;274: 343–351. pmid:25271857
- 3. Zhang L, Tang M, Min Z, Lu J, Lei X, Zhang X. Accuracy of combined dynamic contrast-enhanced magnetic resonance imaging and diffusion-weighted imaging for breast cancer detection: a meta-analysis. Acta radiol. 2015;57: 651–660. pmid:26275624
- 4. Daimiel Naranjo I, Lo Gullo R, Saccarelli C, Thakur SB, Bitencourt A, Morris EA, et al. Diagnostic value of diffusion-weighted imaging with synthetic b-values in breast tumors: comparison with dynamic contrast-enhanced and multiparametric MRI. Eur Radiol. 2020. pmid:32780207
- 5. Pinker K, Moy L, Sutton EJ, Mann RM, Weber M, Thakur SB, et al. Diffusion-Weighted Imaging With Apparent Diffusion Coefficient Mapping for Breast Cancer Detection as a Stand-Alone Parameter: Comparison With Dynamic Contrast-Enhanced and Multiparametric Magnetic Resonance Imaging. Invest Radiol. 2018;53: 587–595. pmid:29620604
- 6. Lourenco AP, Donegan L, Khalil H, Mainiero MB. Improving outcomes of screening breast MRI with practice evolution: initial clinical experience with 3T compared to 1.5T. J Magn Reson Imaging. 2014;39: 535–539. pmid:23720144
- 7. Clauser P, Marcon M, Maieron M, Zuiani C, Bazzocchi M, Baltzer PAT, et al. Is there a systematic bias of apparent diffusion coefficient (ADC) measurements of the breast if measured on different workstations? An inter- and intra-reader agreement study. Eur Radiol. 2016;26: 2291–2296. pmid:26443604
- 8. Baltzer AP, Mann RM, Iima M, Sigmund EE, Clauser P, Gilbert F. Diffusion-Weighted Imaging of the breast–A consensus and mission statement from the EUSOBI International Breast Diffusion- Weighted Imaging working group. Eur Radiol. 2019. pmid:31786616
- 9. Bickel H, Pinker K, Polanec S, Magometschnigg H, Wengert G, Spick C, et al. Diffusion-weighted imaging of breast lesions: Region-of-interest placement and different ADC parameters influence apparent diffusion coefficient values. Eur Radiol. 2017;27: 1883–1892. pmid:27578047
- 10. Hirano M, Satake H, Ishigaki S, Ikeda M, Kawai H, Naganawa S, et al. Diffusion-weighted imaging of breast masses: Comparison of diagnostic performance using various apparent diffusion coefficient parameters. Am J Roentgenol. 2012;198: 717–722. pmid:22358015
- 11. Arponen O, Sudah M, Masarwah A, Taina M, Rautiainen S, Kononen M, et al. Diffusion-Weighted Imaging in 3.0 Tesla Breast MRI: Diagnostic Performance and Tumor Characterization Using Small Subregions vs. Whole Tumor Regions of Interest. PLoS One. 2015;10: e0138702. pmid:26458106
- 12. Zhang W, Jin G-Q, Liu J-J, Su D-K, Luo N-B, Xie D, et al. Diagnostic performance of ADCs in different rois for breast lesions. Int J Clin Exp Med. 2015;8: 12096–12104. Available: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L606255844 pmid:26550121
- 13. Dorrius MD, Dijkstra H, Oudkerk M, Sijens PE, M.D. D, H. D, et al. Effect of b value and pre-admission of contrast on diagnostic accuracy of 1.5-T breast DWI: a systematic review and meta-analysis. Eur Radiol. 2014;24: 2835–2847. pmid:25103535
- 14. Wielema M, Dorrius MD, Pijnappel RM, de Bock GH, Baltzer PAT, Oudkerk M, et al. Diagnostic performance of breast tumor tissue selection in diffusion weighted imaging: A systematic review and meta-analysis. PLoS One. 2020;15: 1–23. pmid:32374781
- 15. Avendano D, Marino MA, Leithner D, Thakur S, Bernard-Davila B, Martinez DF, et al. Limited role of DWI with apparent diffusion coefficient mapping in breast lesions presenting as non-mass enhancement on dynamic contrast-enhanced MRI. Breast Cancer Res. 2019;21: 136. pmid:31801635
- 16. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44: 837–845. pmid:3203132
- 17. Giannotti E, Waugh S, Priba L, Davis Z, Crowe E, Vinnicombe S. Assessment and quantification of sources of variability in breast apparent diffusion coefficient (ADC) measurements at diffusion weighted imaging. Eur J Radiol. 2015;84: 1729–1736. pmid:26078100
- 18. Nogueira L, Brandao S, Matos E, Nunes RG, Ferreira HA, Loureiro J, et al. Region of interest demarcation for quantification of the apparent diffusion coefficient in breast lesions and its interobserver variability. Diagnostic Interv Radiol. 2015;21: 123–127. pmid:25698095
- 19. Pereira FPA, Martins G, Figueiredo E, Domingues MNA, Domingues RC, da Fonseca LMB, et al. Assessment of breast lesions with diffusion-weighted MRI: Comparing the use of different b values. Am J Roentgenol. 2009;193: 1030–1035. pmid:19770326
- 20. Shi R-Y, Yao Q-Y, Wu L-M, Xu J-R. Breast Lesions: Diagnosis Using Diffusion Weighted Imaging at 1.5T and 3.0T-Systematic Review and Meta-analysis. Clin Breast Cancer. 2018;18: e305–e320. pmid:28802529