https://orcid.org/0000-0001-8578-9976
Figures
Abstract
While the recent advancements of computed tomography (CT) technology have contributed in reducing radiation dose and image noise, an objective evaluation of image quality in patient scans has not yet been established. In this study, we present a patient-specific CT image quality evaluation method that includes fully automated measurements of noise level, structure sharpness, and alteration of structure. This study used the CT images of 120 patients from four different CT scanners reconstructed with three types of algorithm: filtered back projection (FBP), vendor-specific iterative reconstruction (IR), and a vendor-agnostic deep learning model (DLM, ClariCT.AI, ClariPi Inc.). The structure coherence feature (SCF) was used to divide an image into the homogeneous (RH) and structure edge (RS) regions, which in turn were used to localize the regions of interests (ROIs) for subsequent analysis of image quality indices. The noise level was calculated by averaging the standard deviations from five randomly selected ROIs on RH, and the mean SCFs on RS was used to estimate the structure sharpness. The structure alteration was defined by the standard deviation ratio between RS and RH on the subtraction image between FBP and IR or DLM, in which lower structure alterations indicate successful noise reduction without degradation of structure details. The estimated structure sharpness showed a high correlation of 0.793 with manually measured edge slopes. Compared to FBP, IR and DLM showed 34.38% and 51.30% noise reduction, 2.87% and 0.59% lower structure sharpness, and 2.20% and -12.03% structure alteration, respectively, on an average. DLM showed statistically superior performance to IR in all three image quality metrics. This study is expected to contribute to enhance the CT protocol optimization process by allowing a high throughput and quantitative image quality evaluation during the introduction or adjustment of lower-dose CT protocol into routine practice.
Citation: Chun M, Choi JH, Kim S, Ahn C, Kim JH (2022) Fully automated image quality evaluation on patient CT: Multi-vendor and multi-reconstruction study. PLoS ONE 17(7): e0271724. https://doi.org/10.1371/journal.pone.0271724
Editor: Mingwu Jin, University of Texas at Arlington, UNITED STATES
Received: February 18, 2022; Accepted: July 6, 2022; Published: July 20, 2022
Copyright: © 2022 Chun et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by the Korea Medical Device Development Fund grant funded by the Korean government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, Republic of Korea, the Ministry of Food and Drug Safety) (Project Number: 1711138600, KMDF_PR_20200901_0267).
Competing interests: One author (JHK) is a stockholder of ClariPi Inc., but did not have control over any of the data or information submitted for publication. One author (CA) is an employee of ClariPi Inc. and performed imaging data processing, but did not have control over any of statistical analysis for publication. Other authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.
Introduction
Computed tomography (CT) is a well-established diagnostic imaging modality, with the number of CT scans continuously increasing for a variety of disease examinations and screening purposes [1]. Because of the potential risks of radiation-induced cancer from the associated radiation exposure [2,3], optimizing the scan protocols has become a crucial task in routine imaging practices that consider the patient’s weight and age as well as body parts being scanned [4]. The scan protocol optimization requires balancing the radiation dose with sufficient image quality such that the necessary diagnostic information is not compromised [4]. Last decade has seen the developments of numerous techniques for radiation dose reduction including automatic exposure control [5–7], dual-energy imaging [8,9], iterative reconstruction (IR) [7,10–12], and deep learning-based reconstruction (DLR) [13]. IR techniques have been highlighted over the last decade with their significant dose reduction performance along with image quality improvements [14–17], and are widely accepted in routine imaging practices [18–21]. Recently, novel DLR techniques have also been developed and become commercially available by CT manufacturers and third party developers: an advanced intelligent Clear-IQ Engine (AiCE, Canon Medical System, Otawara, Japan), a TrueFidelity CT (GE Healthcare, Milwaukee, WI), PixelShine (AlgoMedica Inc, Sunnyvale, CA), and ClariCT.AI (ClariPi Inc, Seoul, South Korea) [22–34].
Phantom-based approaches focused on the performance evaluation of the CT scanner by quantitatively assessing noise characteristics and lesion detectability. According to the evaluation tasks, a task-specific phantom is a preferable option rather than directly scanning human objects in spite of their oversimplified nature and lack of the variability present in patient population [35]. A newly-devised task transfer function analyzes the signal transfer characteristics across scanners, reconstruction algorithms, and imaging tasks, enabling to apply not only FBP but also IR and DLR [36,37]. However, even with use of same scanner, scan protocols, and reconstruction techniques, image qualities in patients can differ across them because of inter-patient variation and intrinsic patient-specific status. Naturally, appropriate links between phantom and patient images are necessary to characterize physical parameters to clinically relevant metrics [38,39]. One of the most frequently employed metrics is a noise magnitude, which is a primary basis for the image quality assessment task [40]. Most IR and DLR-related publications reported significant noise reduction, thus enabling to effectively reduce radiation doses [41–46]. However, several studies reported that the degradation of imaging resolution and altered image textures caused by IR is inevitable and this might compromise the visibility of small-scale and diagnostically important tissue structures [47–52].
A few studies attempted to evaluate the image quality of IR techniques in terms of sharpness degradation of anatomical structures of interest in a qualitative manner. Deák et al. performed subjective image quality assessment of abdominal CT images with model-based IR (MBIR, GE Healthcare) and reported its superior resolution and contour delineation of tissue interfaces as well as noise reduction as compared to filtered back projection (FBP) and adaptive statistical iterative reconstruction (ASIR, GE Healthcare) [53]. Another abdominal CT study compared profile tendencies for FBP and model-based IR images on both high-contrast features (an air hole) and moderate contrast from the edges of the liver to the agar region [54]. However, these approaches required manual drawing of contours which needs special care of operator to avoid confounding structures such as vessels and lesions [55]. These show the necessity of an objective image quality assessment method that is directly applicable to patient CT images and measure clinically relevant image quality metrics for different reconstruction techniques.
Previously, we presented a fully automated method for noise measurement from patient CT images by employing the structure coherence feature (SCF) which calculating the likelihood of pixels belonging to anatomical structures [56]. We applied a low threshold to the calculated SCF to enable an automated segmentation of homogeneous regions suitable for placing a region of interest (ROI) in the measurement of noise level. In this study, we extend our previous study to enable a novel image quality assessment method which extracts a set of image quality metrics such as structure sharpness and preservation of tissue structures as well as noise level from patient CT images by employing the same SCF. First, the reference organ (liver) was segmented by using a deep learning segmentation model. Then, a thresholding was applied with two cut-off values to the calculated SCF within the reference organ to further split the segmented region into homogenous and structural transition regions, from which we extract a set of image quality metrics in a fully automated way. We apply our novel image quality assessment method to patient image dataset of contrast-enhanced liver CT exams with CT scanners from 4 different manufacturers reconstructed with FBP, IR, and DLR techniques. We show that our proposed method could robustly extract image quality metrics regardless of CT manufacturer and reconstruction techniques.
Materials and methods
Dataset
An institutional review board at Seoul National University Hospital approved this retrospective study (IRB No. 1905−077−1033), and the informed consent was waived due to the retrospective design. A total of 120 patients’ contrast-enhanced liver CT scans, whose personal information tags in DICOM files were anonymized, were retrospectively collected. The CT datasets were acquired using four multi-detector CT scanners: Scanner 1 (GE Discovery CT750 HD, GE Healthcare, Milwaukee, WI), Scanner 2 (Ingenuity CT, Philips Healthcare, Cleveland, OH), Scanner 3 (SOMATOM Definition Flash, Siemens Healthcare, Erlangen, Germany), and Scanner 4 (Aquilion ONE, Canon Medical Systems, Otawara, Japan). Images of 30 patients per scanner were collected, and all scans were performed using automatic exposure control (AEC). Detailed parameters are summarized in Table 1.
Three types of reconstruction techniques were utilized: conventional FBP, vendor-specific IRs, and a vendor-agnostic deep learning model (DLM) (ClariCT.AI, ClariPi Inc., South Korea). Vendor-specific IRs were as follows: ASIR (Adaptive statistical iterative reconstruction, GE Healthcare), iDose (Philips Healthcare), SAFIRE (Sinogram affirmed iterative reconstruction, Siemens Healthcare), and AIDR3D (adaptive iterative dose reduction 3D, Canon Medical Systems) [41–45]. Details about ClariCT.AI are described in below subsection.
DLM (ClariCT.AI).
The DLM (ClariPi Inc., South Korea) [57] was established using a convolutional neural network (CNN) algorithm and enabled image denoising by training with the paired set of simulated low-dose and original CT images as input and output, respectively [32–34,58]. To acquire a generalized learning and vendor-agnostic denoising capability, the dataset consisted of over 1 million CT images encompassing 2,100 different combinations of scan and reconstruction conditions, including varying kV, mAs, automatic exposure control, slice thickness, contrast enhancement, and convolution kernels with 24 scanner models from four different CT manufacturers. The clarity-weighted option was employed, which produced denoised images with sharpness enhancement to a degree tuned to compensate for the image blurring effect. The performance of DLM has previously been evaluated in several clinical studies [16,31–34,59–61].
Reference organ segmentation
This study used liver as the reference organ for image quality assessment [62]. A deep learning segmentation model was trained using a U-net architecture and 257 CT scans [63]. The ground truth liver mask was established by a radiation oncologist having two years of experience. The deep learning segmentation model was optimized using adaptive momentum estimation (Adam), and an early stopping technique was used to prevent overfitting [64,65]. The established model was validated using 110 CT scans and exhibited 94.5±16.8% of dice similarity coefficient with the ground truth [66]. Fig 1 shows sample segmentation results for the image dataset. From the entire images, those with segmented areas greater than 100 cm2 were selected for further process of image quality evaluation.
The results were displayed for multi-vendor CT images acquired through (a) Scanner 1, (b) Scanner 2, (c) Scanner 3, and (d) Scanner 4, respectively.
SCF
The SCF was calculated for each pixel within the segmented reference organ. The SCF consisted of an edginess feature and directional entropy feature. The edginess feature represented the likelihood of a pixel being located on an anatomical structure, whereas the directional entropy feature represented the randomness of the pixel orientation to signify the absence of an anatomical structure [56]. The SCF is defined as follows:
(1)
where IE, HG, and HT denote the edginess of each pixel, directional entropy for the gradient vector, and structure tensor, respectively [56]. As shown in Eq (2), the edginess was introduced to present transitional edge between distinct region, and defined by the weighted sum of the magnitude for the gradient (|∇I|) and the 1st eigenvalue of structure tensor (|λ1|),
(2)
and ω1 and ω2 are the corresponding weight. The sum of directional entropies for the gradient vector and structure tensor (HG, HT) are employed to describe the curvilinear edge caused by tubular structures. Directional entropies (HG, HT) are calculated by below formulas [56,67]:
(3)
(4)
where VT was the 1st eigenvector of structure tensor T, which is defined as,
(5)
(6)
Extraction of image quality metrics
Noise level estimation.
A thresholding was applied to the calculated SCF with the cut-off point at the 10th percentile, which resulted in the segmentation of homogeneous regions (RH). Then, five ROIs were randomly selected within RH, from which the average standard deviation of pixel values was set as the noise level.
Validation of noise level measurement. To demonstrate the reliability between automated and manual noise level measurements, a single FBP image from 120 patients’ images (total 120 images) were used. An experienced radiologist manually drawn ROIs on uniform soft tissue area, and obtained noise levels by taking standard deviation. The Pearson correlation coefficient was calculated to evaluate the agreement between the manual and automated measurements. Furthermore, Bland-Altman plots were drawn to demonstrate the mean difference and systematic bias.
Structure sharpness index (SSI).
The structure transition regions between two different tissues were extracted by applying a high SCF threshold. We empirically observed that the 70th percentile of SCF was appropriate for localizing enhanced hepatic regions, and the region was denoted as RS (Fig 2(D)). Regions with SCF from the 10th to 70th percentiles did not belong to any specified region. The sharpness surrogate was then defined as the mean of the SCF within the segmented RS. In order to establish the relationship between manual and SCF-based automated slope measurement, a triple modality 3D abdominal phantom (CIRS Inc., Norfolk, VA) was scanned using Brilliance Big Bore CT (Philips, Amsterdam, The Netherlands) to verify if the SSI measured as above appropriately represent vessel sharpness in the contrast enhanced liver CT. The scans were performed at 120 kV, 300 mAs, and reconstructed with filtered back projection and iDose3 using three different kernels (smooth, standard, sharp) each. The six images were intentionally smoothed by varying Gaussian sigma from 0.5 to 1.6, and sharpened with an unsharp mask by varying the amount parameter from 1.5 to 2.5 [68], and eventually 264 images were arranged. For each dataset, the edge line between the normal tissue and enhanced vessel was manually drawn, and their corresponding perpendicular line profiles were obtained (Fig 3B) [69] along the edge line at equal distance. The mean slope of the perpendicular line profile was used as a reference sharpness measure of the enhanced vessel. The 90% and 10% of maximum intensities and their pixel positions on profiles were utilized [69] to calculate the sharpness (ξl) of the line profiles using Eq (7),
(7)
where sl contains the distances in physical units of the Nl sample locations within the region of interest, and Il denotes the corresponding intensities along the profile line l [69]. A linear regression was performed to characterize the relationship between the manual and the SCF-based automated sharpness measurement yielding a relationship,
(8)
(a) A sample image of which the image quality being assessed with candidate region (liver parenchyma) segmented by using U-net deep-learning model, (b) Eroded mask from candidates with a structural element of 7 pixels, (c) SCF heat map calculated on the eroded mask, and (d) structural transition regions where the edge sharpness being quantified.
(a) An example showing enhanced vessel on a triple modality 3D abdominal phantom where the red line indicates the manually drawn edge component. (b) Perpendicular line profiles on given manual edge line. (c) Dashed lines show the edge rise trend signifying the vessel sharpness, whereas the bold solid line denotes the average value.
Validation of SSI measurement. To demonstrate the reliability between estimation and manual ξl measurements, a single image from 120 patients’ images (total 120 images) were used, and the enhanced vessel line was manually drawn by an experienced radiologist. The ξl were measured manually and automatically by using Eqs (7) and (8), respectively. Same as a validation on noise level measurement, the Pearson correlation coefficient and Bland-Altman plot were drawn to present the agreement, and limits of agreement, respectively.
Structure alteration index (SAI).
Subtraction between two CT images is an effective way to assess the structure alteration caused by the IR algorithm, which is difficult to recognize by visual analysis without subtraction [70]. We devised a novel structure alteration index that conveniently quantify the degree of structure alteration. By employing our ROI extraction scheme for structure and homogenous regions, we extracted the pixel standard deviation within the structure and homogenous ROIs and derived their ratio as follows:
(9)
where σ denotes the standard deviation, the RS and RH denote the structure and homogenous ROIs being evaluated, respectively with ΔI indicating the subtraction between the reference (FBP) and target (IR or DLM) images. Therefore, the SAI value will be lower when the target reconstruction algorithm reduces the noise without significant alteration of the structural components. In contrast, the SAI will increase if substantial residual structures exist in the subtraction image domain.
Results
Noise level assessment
As shown in Fig 4, automated noise level measurements showed a desirable reliability with manual measurement by showing Pearson’s correlation coefficient of 0.951, mean difference of 2.26, standard deviation of 1.30, and 95% limits of agreement of [-0.29, 2.26].
(a) A scatter plot with linear regression showing a Pearson’s correlation coefficient of 0.951, and (b) a Bland-Altman plot where the solid and dashed red lines indicate the mean difference and 95% limits of agreement.
A total of 722, 923, 873, and 682 images were used for image quality evaluation of Scanners 1, 2, 3, and 4, respectively. Several samples for localizing homogeneous ROIs using fully automated methods are presented in Fig 5. The validation of an automated measurement of noise levels by comparing those with manual ROI placement was reported in our earlier study [56], and thus skipped in this study. The automated noise levels across FBP, IR, and DLM are compared in Fig 6. Compared with conventional FBP reconstruction, mean noise levels decreased by 35.9%, 31.2%, 23.5%, and 46.7% with iterative reconstruction and 52.6%, 51.6%, 51.1%, and 50.0% with ClariCT.AI for Scanners 1, 2, 3, and 4, respectively. The magnitudes of noise levels were in the order of FBP, IR, and DLM, with statistically significant differences (p < 0.001).
The results were displayed for multi-vendor CT images acquired through (a) Scanner 1, (b) Scanner 2, (c) Scanner 3, and (d) Scanner 4, respectively.
Structure sharpness index
Fig 7 showed SSIs from a triple modality 3D abdominal phantom data, which presents dependencies on reconstruction kernels and methods. In original FBP data (no smoothing or sharpening), images with smooth and sharp kernels showed -3.44% and 7.80% SSI changes, respectively, compared to those with standard kernel. Furthermore, those with iDose showed -7.86% SSI changes compared to those with FBP in standard kernel.
The data included reconstructions with smooth, standard, and sharp kernel along with two kinds of reconstruction methods (FBP and IR). The images with artificially manufactured of smoothing and sharpening were included.
As shown in Fig 8, an application to the 120 patients’ images presented the reliabilities between estimation and manual ξl measurements with Pearson’s correlation coefficient of 0.793, mean difference of 1.31, standard deviation of 2.65, and 95% limits of agreement of [-3.88, 6.49].
(a) A scatter plot with linear regression showing a Pearson’s correlation coefficient of 0.793, and (b) a Bland-Altman plot where the solid and dashed red lines indicate the mean difference and 95% limits of agreement.
By thresholding the SCFs with 70th percentile, it was possible to discriminate the parenchyma-vessel transition region on enhanced hepatic region. Example localization results for the CT images with different manufacturers are shown in Fig 9. Measurements of SSI across the FBP, IR, and DLM datasets are shown Fig 10. Compared to conventional FBP reconstruction, mean sharpness decreased by 1.72%, 3.21%, 2.36%, and 4.21% with iterative reconstruction and 0.57%, 0.39%, 0.23%, and 1.19% with DLM for Scanners 1, 2, 3, and 4, respectively. When data from all manufacturers were pooled, the SSI of DLM was 2.57% higher than that of IR, with statistical significance (p < 0.05).
The results were displayed for multi-vendor CT images acquired with (a) Scanner 1, (b) Scanner 2, (c) Scanner 3, and (d) Scanner 4, respectively.
Structure alteration index
Sample localization results of homogeneous and structure edge regions on the subtraction image domain are shown in Fig 11, where subtraction can be either FBP and IR, or FBP and DL. Measurements of SAI across the FBP, IR, and DLM datasets are shown Fig 12. The mean SAI was 1.02 ± 0.16 and 0.88 ± 0.14 for IR and DLM, respectively. The difference of SAI between IR and DLM was statistically significant (p < 0.05).
Each image was acquired with a) Scanner 1, (b) Scanner 2, (c) Scanner 3, and (d) Scanner 4, respectively.
Discussion
In this study, we presented a novel image quality evaluation method that allows a fully automated assessment of three image quality metrics on patient CT images. We employed the structure coherence feature to enable an automated localization of homogeneous and structure edge regions on patient CT images from which the three key image quality metrics were calculated such as noise level, structure sharpness, and structure alteration [71,72]. We applied this method to the contrast enhanced liver CT images of 120 patients from four different CT scanners reconstructed with FBP, IR, and DLM.
Our study results showed the robust performance of the proposed method with successful extraction of image quality metrics for the patient image set with a variety of CT image quality. The extracted image quality metrics revealed the differences in the image quality characteristics of FBP, IR, and DLM. Overall, the measured noise level was in the order of FBP, IR, and DLM, whereas the noise reduction with IR and DLM was different depending on scanner manufacturers: DLM achieved 20–30% further noise reduction in three scanners but almost no further reduction in one scanner. Structure sharpness, which was difficult by visual assessment, was shown to degrade in IR but maintain in DLM compared to FBP. Furthermore, the newly introduced metric, SAI effectively quantified the changes in structure appearance after noise reduction, which was significantly higher in IR than in DLM across all the four CT manufacturers’ images.
To the knowledge of authors, this is the first study that reported the quantitative measurement of key image quality metrics in a fully automated way from patient CT images with objective comparison of image qualities among the FBP, IR, and DLM of different vendors. Frequently, noise levels were calculated as a standard deviation on uniform area, whose ROIs could be drawn manually or automatically [38,56,73–76]. Several studies attempted an automation by employing the global noise level approaches, which adopts the mode value in the histogram of pixel standard deviations on subtracted image domain across adjacent slices [38,73–75]. Their approaches were shown as robust by showing excellent correlation with reference values in abdominal, thoracic, and head CT scans. Another publication proposed an automated noise measurement technique by using an air-based global noise index in subtraction between adjacent images [76]. This technique is definitely beneficial due to their excellent correlation with those from patients as well as their global efficacy regardless of phantom or human images. They eliminated the needs to discriminate homogeneous regions in patient’s anatomy because air signal is inherently devoid with any anatomical structures. Our approach is not in accordance with previous ones by employing SCF, which intends to directly analyze how much ROIs contained structural transitions or not. As avoiding small scale-anatomical structure is also possible by adopting ROIs with low SCF values, noise levels obtained with automated measurements showed lower values than those with manual measurement. This infers SCF enable to select homogeneous ROIs with fully automated fashion, which is even hard for human observer (Fig 4(B)). Furthermore, ours enable to differentiate not only homogeneous area but also structural transition regions by using statistics of SCF, thus making it possible to evaluate noise level, SSI, and SAI with a fully automated approach.
Previous studies asserted that other aspects of image quality are also important and needed to be quantified in order to inspect the potential loss or degradation of diagnostically important image features such as visibility of small-scale structures. In fact, a study reported that one of the radiologists failed to detect small metastases on IR images owing to mild blurring and pixelation [77]. Therefore, we attempted to measure not only image noise, but also other important metrics that can influence the visual perception task in imaging diagnosis. We included structure sharpness and structure alteration in our triple metric image quality evaluation. Our results validated that the automated sharpness measurements presented in this study could reliably represent the manually measured structure slopes by an experienced observer with a strong Pearson correlation of 0.79. We devised the SAI in an attempt to inspect and report the subtle degradation of structure appearance caused by blurring or pixelation which was frequently reported to appear in IR. Evaluating the ability of SAI to report such structure degradation in newly introduced denoising techniques would be an interesting study, but was outside the scope of our study and thus remains a subject of further study.
In SSI measurements obtained with triple modality phantom data (Fig 7), they exhibited reliable representations of the sharpness by showing higher SSI in sharpened with greater alpha whereas lower in severely smoothed data (Gaussian sigma 1.5). Furthermore, it is noticeable that SSIs increased from smooth to sharp kernel, and decreased from FBP to iDose in same manipulation data. Although there existed no significant changes in patients’ SSI measurement across reconstruction methods (Fig 10), they are also meaningful because the sharpness of the enhanced-vessel structures well preserved while remarkably reducing the noise level.
This study had several limitations. Firstly, although we evaluated a vendor-agnostic DLM along with vendor-specific IRs in this study, other vendor-specific DLM algorithms such as AiCE (Canon Medical System) and TrueFidelity CT (GE Healthcare) were not included. Therefore, our study results for vendor-agnostic DLM cannot be generalized. We included the patient CT images of contrast enhance liver scanned with regular radiation dose settings. As the newly developed denoising techniques could find their full merits in application to low-dose CT (LDCT) examinations such as lung cancer CT screenings, future studies will need to include the LDCT data for lung cancer screening with multiple types of reconstruction [78–80]. In the LDCT lung cancer screening program, we hope the use of noise level, SSI, and SAI metrics could quantify image qualities, and provide insights to adopt best scan and reconstruction protocols. Our image quality metrics did not evaluate the image texture appearance. The changes in noise texture, including the grain size and shape of the noise and cartoon-like appearance in several IR algorithms will eventually lead to a shift in the peak frequency of the NPS [81]. Even with a similar amount of noise level, the observers’ diagnostic performance can also be influenced by noise texture alteration [82–85].
Despite these limitations, our study is expected to contribute to enhance the CT protocol optimization process by allowing a high throughput and quantitative image quality evaluation during the introduction or adjustment of lower-dose CT protocol into routine practice. Currently, the concept of diagnostic reference level in CT is moving to the noise and dose reference levels, which emphasizes the importance of not only radiation dose but also image quality in the justification of CT scanning according to the ALARA principle [86,87]. Therefore, our proposed method might contribute to integrate the image quality and radiation dose monitoring into quality assurance practice during CT scanning protocol optimization procedure.
Conclusions
We have developed a fully automated method for image quality evaluation on patient CT images using three image quality metrics such as noise level, structure sharpness index, and structure alteration index. We demonstrated the robust performance of our proposed method with the patient images of contrast-enhanced liver CT from four different scanners reconstructed using FBP, IR, and DLM. Our proposed method has a potential to enhance the CT protocol optimization process by allowing a high throughput and quantitative image quality evaluation for not only phantom tests but also patient scan datasets.
Supporting information
S1 Table. Image quality analysis results for 120 patients.
The noise levels, structure sharpness index, and structure alteration index are provided on average.
https://doi.org/10.1371/journal.pone.0271724.s001
(DOCX)
References
- 1. Smith-Bindman R, Miglioretti DL, Larson EB. Rising use of diagnostic medical imaging in a large integrated health system. Health Aff (Millwood). 2008;27(6):1491–502. pmid:18997204.
- 2. Miglioretti DL, Johnson E, Williams A, Greenlee RT, Weinmann S, Solberg LI, et al. The use of computed tomography in pediatrics and the associated radiation exposure and estimated cancer risk. JAMA Pediatr. 2013;167(8):700–7. pmid:23754213.
- 3. Guttikonda R, Herts BR, Dong F, Baker ME, Fenner KB, Pohlman B. Estimated radiation exposure and cancer risk from CT and PET/CT scans in patients with lymphoma. Eur J Radiol. 2014;83(6):1011–5. pmid:24656401.
- 4. Khong P, Ringertz H, Donoghue V, Frush D, Rehani M, Appelgate K, et al. ICRP publication 121: radiological protection in paediatric diagnostic and interventional radiology. Ann ICRP. 2013;42(2):1–63. pmid:23218172.
- 5. Singh S, Kalra MK, Thrall JH, Mahesh M. Automatic exposure control in CT: applications and limitations. J Am Coll Radiol. 2011;8(6):446–9. pmid:21636062.
- 6. Söderberg M, Gunnarsson M. Automatic exposure control in computed tomography–an evaluation of systems from different manufacturers. Acta Radiol. 2010;51(6):625–34. pmid:20429764.
- 7. Ha S, Jung S, Chang H-J, Park E-A, Shim H. Effects of iterative reconstruction algorithm, automatic exposure control on image quality, and radiation dose: phantom experiments with coronary CT angiography protocols. Prog Med Phys. 2015;26(1):28–35. https://doi.org/10.14316/pmp.2015.26.1.28.
- 8. Coursey CA, Nelson RC, Boll DT, Paulson EK, Ho LM, Neville AM, et al. Dual-energy multidetector CT: how does it work, what can it tell us, and when can we use it in abdominopelvic imaging? Radiographics. 2010;30(4):1037–55. pmid:20631367.
- 9. Mileto A, Barina A, Marin D, Stinnett SS, Roy Choudhury K, Wilson JM, et al. Virtual monochromatic images from dual-energy multidetector CT: variance in CT numbers from the same lesion between single-source projection-based and dual-source image-based implementations. Radiology. 2016;279(1):269–77. pmid:26536403.
- 10. Beister M, Kolditz D, Kalender WA. Iterative reconstruction methods in X-ray CT. Phys Med. 2012;28(2):94–108. pmid:22316498.
- 11. Singh S, Kalra MK, Hsieh J, Licato PE, Do S, Pien HH, et al. Abdominal CT: comparison of adaptive statistical iterative and filtered back projection reconstruction techniques. Radiology. 2010;257(2):373–83. pmid:20829535.
- 12. Geyer LL, Schoepf UJ, Meinel FG, Nance JW Jr, Bastarrika G, Leipsic JA, et al. State of the art: iterative CT reconstruction techniques. Radiology. 2015;276(2):339–57. pmid:26203706.
- 13. Othman AE, Brockmann C, Yang Z, Kim C, Afat S, Pjontek R, et al. Impact of image denoising on image quality, quantitative parameters and sensitivity of ultra-low-dose volume perfusion CT imaging. Eur Radiol. 2016;26(1):167–74. pmid:26024848.
- 14. Willemink MJ, de Jong PA, Leiner T, de Heer LM, Nievelstein RA, Budde RP, et al. Iterative reconstruction techniques for computed tomography Part 1: technical principles. Eur Radiol. 2013;23(6):1623–31. pmid:23314600.
- 15. Willemink MJ, Leiner T, de Jong PA, de Heer LM, Nievelstein RA, Schilham AM, et al. Iterative reconstruction techniques for computed tomography part 2: initial results in dose reduction and image quality. Eur Radiol. 2013;23(6):1632–42. pmid:23322411.
- 16. Lim WH, Choi YH, Park JE, Cho YJ, Lee S, Cheon JE, et al. Application of vendor-neutral iterative reconstruction technique to pediatric abdominal computed tomography. Korean J Radiol. 2019;20(9):1358–67. pmid:31464114.
- 17. Coakley FV, Gould R, Yeh BM, Arenson RL. CT radiation dose: what can you do right now in your practice? AJR Am J Roentgenol. 2011;196(3):619–25. pmid:21343506.
- 18. Gordic S, Desbiolles L, Sedlmair M, Manka R, Plass A, Schmidt B, et al. Optimizing radiation dose by using advanced modelled iterative reconstruction in high-pitch coronary CT angiography. Eur Radiol. 2016;26(2):459–68. pmid:26037718.
- 19. Arapakis I, Efstathopoulos E, Tsitsia V, Kordolaimi S, Economopoulos N, Argentos S, et al. Using “iDose4” iterative reconstruction algorithm in adults’ chest–abdomen–pelvis CT examinations: effect on image quality in relation to patient radiation exposure. Br J Radiol. 2014;87(1036):20130613. pmid:24646183.
- 20. Cornfeld D, Israel G, Detroy E, Bokhari J, Mojibian H. Impact of Adaptive Statistical Iterative Reconstruction (ASIR) on radiation dose and image quality in aortic dissection studies: a qualitative and quantitative analysis. AJR Am J Roentgenol. 2011;196(3):W336–40. pmid:21343485.
- 21. Ohno Y, Takenaka D, Kanda T, Yoshikawa T, Matsumoto S, Sugihara N, et al. Adaptive iterative dose reduction using 3D processing for reduced-and low-dose pulmonary CT: comparison with standard-dose CT for image noise reduction and radiological findings. AJR Am J Roentgenol. 2012;199(4):W477–85. pmid:22997397.
- 22. Akagi M, Nakamura Y, Higaki T, Narita K, Honda Y, Zhou J, et al. Deep learning reconstruction improves image quality of abdominal ultra-high-resolution CT. Eur Radiol. 2019;29(11):6163–71. pmid:30976831.
- 23. Willemink MJ, Noël PB. The evolution of image reconstruction for CT—from filtered back projection to artificial intelligence. Eur Radiol. 2019;29(5):2185–95. pmid:30377791.
- 24. Kang E, Chang W, Yoo J, Ye JC. Deep convolutional framelet denosing for low-dose CT via wavelet residual network. IEEE Trans Med Imaging. 2018;37(6):1358–69. pmid:29870365.
- 25. Chen H, Zhang Y, Zhang W, Liao P, Li K, Zhou J, et al. Low-dose CT via convolutional neural network. Biomed Opt Express. 2017;8(2):679–94. pmid:28270976.
- 26. Ahn CK, Yang Z, Heo C, Jin H, Park B, Kim JH. A deep learning-enabled iterative reconstruction of ultra-low-dose CT: use of synthetic sinogram-based noise simulation technique. Proc SPIE 10573, Medical Imaging 2018: Physics of Medical Imaging. 2018;10573:1057335. https://doi.org/10.1117/12.2294013.
- 27. Tamura A, Mukaida E, Ota Y, Kamata M, Abe S, Yoshioka K. Superior objective and subjective image quality of deep learning reconstruction for low-dose abdominal CT imaging in comparison with model-based iterative reconstruction and filtered back projection. Br J Radiol. 2021;94(1123):20201357. pmid:34142867.
- 28. Solomon J, Lyu P, Marin D, Samei E. Noise and spatial resolution properties of a commercially available deep learning‐based CT reconstruction algorithm. Med Phys. 2020;47(9):3961–71. pmid:32506661.
- 29. Racine D, Brat HG, Dufour B, Steity JM, Hussenot M, Rizk B, et al. Image texture, low contrast liver lesion detectability and impact on dose: deep learning algorithm compared to partial model-based iterative reconstruction. Eur J Radiol. 2021;141:109808. pmid:34120010.
- 30. Hasegawa A, Ishihara T, Thomas MA, Pan T. Noise reduction profile: A new method for evaluation of noise reduction techniques in CT. Med Phys. 2022;49(1):186–200. pmid:34837717.
- 31. Nam JG, Ahn C, Choi H, Hong W, Park J, Kim JH, et al. Image quality of ultralow-dose chest CT using deep learning techniques: potential superiority of vendor-agnostic post-processing over vendor-specific techniques. Eur Radiol. 2021;31(7):5139–47. pmid:33415436.
- 32. Yeoh H, Hong SH, Ahn C, Choi J-Y, Chae H-D, Yoo HJ, et al. Deep Learning Algorithm for Simultaneous Noise Reduction and Edge Sharpening in Low-Dose CT Images: A Pilot Study Using Lumbar Spine CT. Korean J Radiol. 2021;22(11):1850. pmid:34431248.
- 33. Park S, Yoon JH, Joo I, Yu MH, Kim JH, Park J, et al. Image quality in liver CT: low-dose deep learning vs standard-dose model-based iterative reconstructions. Eur Radiol. 2021:1–10. pmid:34821967.
- 34. Choi H, Chang W, Kim JH, Ahn C, Lee H, Kim HY, et al. Dose reduction potential of vendor-agnostic deep learning model in comparison with deep learning–based image reconstruction algorithm on CT: A phantom study. Eur Radiol. 2021:1–9. pmid:34390372.
- 35. Samei E, Bakalyar D, Boedeker KL, Brady S, Fan J, Leng S, et al. Performance evaluation of computed tomography systems: summary of AAPM task group 233. Med Phys. 2019;46(11):e735–e56. pmid:31408540.
- 36. Robins M, Solomon J, Richards T, Samei E. 3D task‐transfer function representation of the signal transfer properties of low‐contrast lesions in FBP‐and iterative‐reconstructed CT. Med Phys. 2018;45(11):4977–85. pmid:30231193.
- 37. Richard S, Husarik DB, Yadava G, Murphy SN, Samei E. Towards task‐based assessment of CT performance: system and object MTF across different reconstruction algorithms. Med Phys. 2012;39(7Part1):4115–22. pmid:22830744.
- 38. Tian X, Samei E. Accurate assessment and prediction of noise in clinical CT images. Med Phys. 2016;43(1):475–82. pmid:26745940.
- 39. Verdun FR, Racine D, Ott JG, Tapiovaara MJ, Toroi P, Bochud FO, et al. Image quality in CT: From physical measurements to model observers. Phys Med. 2015;31(8):823–43. pmid:26459319.
- 40. Schuhbaeck A, Schaefer M, Marwan M, Gauss S, Muschiol G, Lell M, et al. Patient-specific predictors of image noise in coronary CT angiography. J Cardiovasc Comput Tomogr. 2013;7(1):39–45. pmid:23352772.
- 41.
Grant K, Raupach R. SAFIRE: Sinogram affirmed iterative reconstruction. Whitepaper, Siemens AG. 2012.
- 42.
Laqmani A, Buhk J, Henes F, Klink T, Sehner S, Von Schultzendorff H, et al. Impact of a 4th generation iterative reconstruction technique on image quality in low-dose computed tomography of the chest in immunocompromised patients. RöFo-Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren; 2013.
- 43. Hernandez-Giron I, Veldkamp W, Geleijns J. AIDR 3D Enhanced—The latest hybrid modelbased iterative dose reduction technology from Canon. Canon Medical Systems. 2018.
- 44. Patro SN, Chakraborty S, Sheikh A. The use of adaptive statistical iterative reconstruction (ASiR) technique in evaluation of patients with cervical spine trauma: impact on radiation dose reduction and image quality. Br J Radiol. 2016;89(1060):20150082. pmid:26882825.
- 45. Shuman WP, Green DE, Busey JM, Kolokythas O, Mitsumori LM, Koprowicz KM, et al. Model-based iterative reconstruction versus adaptive statistical iterative reconstruction and filtered back projection in liver 64-MDCT: focal lesion detection, lesion conspicuity, and image noise. AJR Am J Roentgenol. 2013;200(5):1071. pmid:23617492.
- 46. Yoo R-E, Park E-A, Lee W, Shim H, Kim YK, Chung JW, et al. Image quality of adaptive iterative dose reduction 3D of coronary CT angiography of 640-slice CT: comparison with filtered back-projection. Int J Cardiovasc Imaging. 2013;29(3):669–76. pmid:22923280.
- 47. Hou Y, Liu X, Xv S, Guo W, Guo Q. Comparisons of image quality and radiation dose between iterative reconstruction and filtered back projection reconstruction algorithms in 256-MDCT coronary angiography. AJR Am J Roentgenol. 2012;199(3):588–94. pmid:22915398.
- 48. McCollough CH, Yu L, Kofler JM, Leng S, Zhang Y, Li Z, et al. Degradation of CT low-contrast spatial resolution due to the use of iterative reconstruction and reduced dose levels. Radiology. 2015;276(2):499–506. pmid:25811326.
- 49. Favazza CP, Yu L, Leng S, Kofler JM, McCollough CH. Automatic exposure control systems designed to maintain constant image noise: effects on computed tomography dose and noise relative to clinically accepted technique charts. J Comput Assist Tomogr. 2015;39(3):437–42. pmid:25938214.
- 50. Noël PB, Renger B, Fiebich M, Münzel D, Fingerle AA, Rummeny EJ, et al. Does iterative reconstruction lower CT radiation dose: evaluation of 15,000 examinations. PLoS One. 2013;8(11):e81141. pmid:24303035.
- 51. Vaishnav JY, Jung WC, Popescu LM, Zeng R, Myers KJ. Objective assessment of image quality and dose reduction in CT iterative reconstruction. Med Phys. 2014;41(7):071904. pmid:24989382.
- 52. Prakash P, Kalra MK, Kambadakone AK, Pien H, Hsieh J, Blake MA, et al. Reducing abdominal CT radiation dose with adaptive statistical iterative reconstruction technique. Invest Radiol. 2010;45(4):202–10. pmid:20177389.
- 53. Deák Z, Grimm JM, Treitl M, Geyer LL, Linsenmaier U, Körner M, et al. Filtered back projection, adaptive statistical iterative reconstruction, and a model-based iterative reconstruction in abdominal CT: an experimental clinical study. Radiology. 2013;266(1):197–206. pmid:23169793.
- 54. Funama Y, Taguchi K, Utsunomiya D, Oda S, Katahira K, Tokuyasu S, et al. Image quality assessment of an iterative reconstruction algorithm applied to abdominal CT imaging. Phys Med. 2014;30(4):527–34. pmid:24662097.
- 55. Jensen CT, Liu X, Tamm EP, Chandler AG, Sun J, Morani AC, et al. Image Quality Assessment of Abdominal CT by Use of New Deep Learning Image Reconstruction: Initial Experience. AJR Am J Roentgenol. 2020;215(1):1–8. pmid:32286872.
- 56. Chun M, Choi YH, Kim JH. Automated measurement of CT noise in patient images with a novel structure coherence feature. Phys Med Biol. 2015;60(23):9107–22. pmid:26561914.
- 57.
ClariPi Inc. ClariCT.AI. United States patent K183460. https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183460.pdf.
- 58.
Ahn C, Heo C, Kim JH. Combined low-dose simulation and deep learning for CT denoising: application in ultra-low-dose chest CT. International Forum on Medical Imaging in Asia; 2019: International Society for Optics and Photonics.
- 59. Lee S, Choi YH, Cho YJ, Lee SB, Cheon JE, Kim WS, et al. Noise reduction approach in pediatric abdominal CT combining deep learning and dual-energy technique. Eur Radiol. 2021;31(4):2218–26. pmid:33030573.
- 60. Hong JH, Park EA, Lee W, Ahn C, Kim JH. Incremental image noise reduction in coronary CT angiography using a deep learning-based technique with iterative reconstruction. Korean J Radiol. 2020;21(10):1165. pmid:32729262.
- 61. Kolb M, Storz C, Kim JH, Weiss J, Afat S, Nikolaou K, et al. Effect of a novel denoising technique on image quality and diagnostic accuracy in low-dose CT in patients with suspected appendicitis. Eur J Radiol. 2019;116:198–204. pmid:31153565.
- 62. Nakaura T, Awai K, Oda S, Funama Y, Harada K, Uemura S, et al. Low-kilovoltage, high-tube-current MDCT of liver in thin adults: pilot study evaluating radiation dose, image quality, and display settings. AJR Am J Roentgenol. 2011;196(6):1332–8. pmid:21606297.
- 63.
Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention; 2015: Springer.
- 64. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014. https://doi.org/10.48550/arXiv.1412.6980.
- 65. Morgan N, Bourlard H. Generalization and parameter estimation in feedforward nets: some experiments. Adv Neural Inf Process Syst. 1990;2:630–7.
- 66. Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945;26(3):297–302. https://doi.org/10.2307/1932409.
- 67. Baghaie A, Yu Z. Structure tensor based image interpolation method. AEU-international Journal of Electronics and Communications. 2015;69(2):515–22. https://doi.org/10.48550/arXiv.1402.5564.
- 68.
Haralock RM, Shapiro LG. Computer and robot vision: Addison-Wesley Longman Publishing Co. Inc.; 1991.
- 69.
Taubmann O, Wetzl J, Lauritsch G, Maier A, Hornegger J. Sharp as a Tack, Measuring and Comparing Edge Sharpness in Motion-Compensated Medical Image Reconstruction. Bildverarbeitung für die Medizin: Springer; 2015. p. 425–30.
- 70. Lee KB, Goo HW. Quantitative image quality and histogram-based evaluations of an iterative reconstruction algorithm at low-to-ultralow radiation dose levels: a phantom study in chest CT. Korean J Radiol. 2018;19(1):119–29. pmid:29354008.
- 71. Scharf M, Brendel S, Melzer K, Hentschke C, May M, Uder M, et al. Image quality, diagnostic accuracy, and potential for radiation dose reduction in thoracoabdominal CT, using Sinogram Affirmed Iterative Reconstruction (SAFIRE) technique in a longitudinal study. PLoS One. 2017;12(7):e0180302. pmid:28678818.
- 72. Benz DC, Benetos G, Rampidis G, Felten Ev, Bakula A, Sustar A, et al. Validation of deep-learning image reconstruction for coronary computed tomography angiography: impact on noise, image quality and diagnostic accuracy. J Cardiovasc Comput Tomogr. 2020;14(5):444–51. pmid:31974008.
- 73. Ahmad M, Tan D, Marisetty S. Assessment of the global noise algorithm for automatic noise measurement in head CT examinations. Med Phys. 2021;48(10):5702–11. pmid:34314528.
- 74. Ahmad M, Jacobsen MC, Thomas MA, Chen HS, Layman RR, Jones AK. A benchmark for automatic noise measurement in clinical computed tomography. Med Phys. 2021;48(2):640–7. pmid:33283284.
- 75. Christianson O, Winslow J, Frush DP, Samei E. Automated technique to measure noise in clinical CT examinations. AJR Am J Roentgenol. 2015;205(1):W93–9. pmid:26102424.
- 76. Malkus A, Szczykutowicz TP. A method to extract image noise level from patient images in CT. Med Phys. 2017;44(6):2173–84. pmid:28380245.
- 77. Mohammadinejad P, Mileto A, Yu L, Leng S, Guimaraes LS, Missert AD, et al. CT noise-reduction methods for lower-dose scanning: strengths and weaknesses of iterative reconstruction algorithms and new techniques. RadioGraphics. 2021;41(5):1493–508. pmid:34469209.
- 78. Goo JM. Lung cancer screening with low-dose CT: current status in other countries. J Korean Soc Radiol. 2019;80(5):849–59. pmid:34164287.
- 79. Pinsky PF. Lung cancer screening with low-dose CT: a world-wide view. Transl Lung Cancer Res. 2018;7(3):234–42. pmid:30050762.
- 80. Jonas DE, Reuland DS, Reddy SM, Nagle M, Clark SD, Weber RP, et al. Screening for lung cancer with low-dose computed tomography: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2021;325(10):971–87. pmid:33687468.
- 81. Stephenson I, Saunders A. Simulating film grain using the noise power spectrum. EG UK Theory and Practice of Computer Graphics; 2007.
- 82. Dolly S, Chen HC, Anastasio M, Mutic S, Li H. Practical considerations for noise power spectra estimation for clinical CT scanners. J Appl Clin Med Phys. 2016;17(3):392–407. pmid:27167257.
- 83.
Maidment AD, Albert M, Bunch PC, Cunningham IA, Dobbins III JT, Gagne RM, et al. Standardization of NPS measurement: interim report of AAPM TG16. Medical Imaging 2003: Physics of Medical Imaging; 2003: International Society for Optics and Photonics.
- 84. Kijewski MF, Judy PF. The noise power spectrum of CT images. Phys Med Biol. 1987;32(5):565. pmid:3588670
- 85. Ehman EC, Yu L, Manduca A, Hara AK, Shiung MM, Jondal D, et al. Methods for clinical evaluation of noise reduction techniques in abdominopelvic CT. RadioGraphics. 2014;34(4):849–62. pmid:25019428.
- 86. Ria F, Davis JT, Solomon JB, Wilson JM, Smith TB, Frush DP, et al. Expanding the concept of diagnostic reference levels to noise and dose reference levels in CT. AJR Am J Roentgenol. 2019;213(4):889–94. pmid:31180737.
- 87. Won T, Lee A-K, Choi H-d, Lee C. Radiation dose from computed tomography scans for Korean pediatric and adult patients. J Radiat Prot Res. 2021;46(3):98–105. https://doi.org/10.14407/jrpr.2021.00010.