A comprehensive assessment of physical image quality of five different scanners for head CT imaging as clinically used at a single hospital centre—A phantom study

Nowadays, given the technological advance in CT imaging and increasing heterogeneity in characteristics of CT scanners, a number of CT scanners with different manufacturers/technologies are often installed in a hospital centre and used by various departments. In this phantom study, a comprehensive assessment of image quality of 5 scanners (from 3 manufacturers and with different models) for head CT imaging, as clinically used at a single hospital centre, was hence carried out. Helical and/or sequential acquisitions of the Catphan-504 phantom were performed, using the scanning protocols (CTDIvol range: 54.7–57.5 mGy) employed by the staff of various Radiology/Neuroradiology departments of our institution for routine head examinations. CT image quality for each scanner/acquisition protocol was assessed through noise level, noise power spectrum (NPS), contrast-to-noise ratio (CNR), modulation transfer function (MTF), low contrast detectability (LCD) and non-uniformity index analyses. Noise values ranged from 3.5 HU to 5.7 HU across scanners/acquisition protocols. NPS curves differed in terms of peak position (range: 0.21–0.30 mm-1). A substantial variation of CNR values with scanner/acquisition protocol was observed for different contrast inserts. The coefficient of variation (standard deviation divided by mean value) of CNR values across scanners/acquisition protocols was 18.3%, 31.4%, 34.2%, 30.4% and 30% for teflon, delrin, LDPE, polystyrene and acrylic insert, respectively. An appreciable difference in MTF curves across scanners/acquisition protocols was revealed, with a coefficient of variation of f50%/f10% of MTF curves across scanners/acquisition protocols of 10.1%/7.4%. A relevant difference in LCD performance of different scanners/acquisition protocols was found. The range of contrast threshold for a typical object size of 3 mm was 3.7–5.8 HU. Moreover, appreciable differences in terms of NUI values (range: 4.1%-8.3%) were found. The analysis of several quality indices showed a non-negligible variability in head CT imaging capabilities across different scanners/acquisition protocols. This highlights the importance of a physical in-depth characterization of image quality for each CT scanner as clinically used, in order to optimize CT imaging procedures.


Introduction
In recent years, the extraordinary technical advances in x-ray computed tomography (CT) have largely increased its use in the clinical practice. Thus, CT has become a fundamental imaging tool in several body as well as head applications, providing useful information for diagnosis and patient care [1][2][3].
Given that CT imaging represents the largest source of population exposure to ionizing radiation in industrialized countries [4,5] and increased radiation dose may increase the risk of cancer [6], it is important to minimize radiation dose (without compromising the diagnostic potential) through an optimization and standardization of acquisition protocols [7-9]. Accordingly, various technical approaches (e.g. tube current modulation, automatic exposure control, iterative reconstruction algorithms) to optimize CT acquisitions in various applications can be used [10][11][12][13][14][15]. Furthermore, some previous studies have described phantom-based methods to optimize acquisition protocol for specific CT imaging techniques and applications, exploiting various physical image quality indices [9, [16][17][18][19][20][21]. For instance, Zhang et al [9] have developed a CT protocol optimization platform by combining task-based detectability calculations with a graphical user interface that demonstrates the trade-off between dose and image quality. Their platform can be used to improve individual dose efficiency and acquisition protocol consistency across various patient sizes and CT scanners. Berta et al [19] have described a method to objectively evaluate image quality when new clinical protocol performances must be compared with a standard reference. This quantitative approach has been applied to the images of a typical routine abdominal protocol, which were reconstructed with the standard filtered back projection (FBP) and the Iterative Reconstruction in Image Space (IRIS) algorithm. An adaptable and global approach for optimizing CT protocols, by evaluating the influence of acquisition parameters and iterative reconstruction algorithms, has been proposed and implemented in a software program by Greffier et al [20]. Moreover, Noferini et al [21] have proposed and validated a method that employs a Channelling Hotelling model Observer in a CT protocol optimization program, with the aim at assuring that scanners are working at their own best with regard to image quality and patient exposure.
Given also time efficiency and cost considerations, head CT is a first line imaging examination for assessing neurological disorders [53][54][55]. In particular, head CT imaging is highly sensitive to bleeding, and is an essential diagnostic modality to investigate osseous structures as well as to detect calcifications. Moreover, it is usually preferred to magnetic resonance imaging for its wide availability, rapid acquisition and high spatial resolution [56,57]. In this regard, 5 CT scanners, with different technical characteristics, are currently installed at our hospital centre. It should be noted that radiologists and neuroradiologists by various departments of our institution execute head CT examinations on these scanners by using different acquisition protocols, with similar radiation dose values. Therefore, toward an optimization of CT imaging procedures at our hospital centre, the aim of this phantom study was to comprehensively assess whether and how head CT physical image quality varies with different scanners as clinically used for routine examinations.

Scanners and phantom acquisitions
Images of the Catphan-504 phantom (The Phantom Laboratory, NY, USA) were acquired on 5 CT scanners (one 128-slice, two 64-slice and two 16-slice CT scanners) from different manufacturers ( Table 1). The phantom has a cylindrical shape with a diameter of 20 cm and it is composed of four modules. In particular, the CTP486 and CTP404 modules of the phantom were employed for CT acquisitions. The CTP486 module is a homogeneous water-equivalent module, while the CTP404 module includes multiple inserts of different materials in a waterequivalent background (Fig 1). Nominal CT Hounsfield units (HU) values of the inserts are reported in Table 2.
Sequential and/or helical clinical CT head protocols, with different acquisition parameters but similar radiation doses in terms of CTDI vol , were employed for acquisitions on CT scanners as indicated in Table 3. CT images were reconstructed by using the conventional filtered back-projection algorithm.

Physical image quality assessment
For each CT scanner/acquisition protocol, physical image quality was assessed through a number of indices which are strictly related to the main characteristics of images in terms of noise, spatial resolution and contrast properties [59][60][61][62][63][64]. In particular, quantitative metrics of noise level, noise power spectrum (NPS), contrast-to-noise ratio (CNR), modulation transfer function (MTF), low contrast detectability (LCD) and non-uniformity index (NUI) were estimated.
For each image quality index and CT scanner/acquisition protocol, 5 repeated acquisitions were performed. The estimated value and uncertainty of a quality index were obtained as the mean value and standard deviation (SD) across repeated measurements, respectively. Image analysis was performed by using ImageJ (Wayne Rasband, National Institute of Health, USA), Origin (OriginLab Corporation, MA, USA) and Matlab (The MathWorks, Inc., MA, USA) software packages.

Noise level
Noise level was evaluated by computing the SD of HU values within a 4.5 cm diameter circular region of interest (ROI), placed at the centre of the acquisition slab central image of the uniform CTP486 module.

Noise power spectrum (NPS)
Texture properties of CT image noise were assessed by computing the NPS (i.e. the spatial frequency distribution of noise) [65,66]: where f x /f y are the spatial frequencies along the main orthogonal directions, Δx/Δy are the voxel sizes, N x /N y are the number of voxels for each direction, FFT is the two-dimensional (2D) fast Fourier transform, ROI noise (x,y) is the local value of an "only-noise" ROI and < > indicates the ensemble average (i.e. the average across measurements performed on a number of ROIs). In particular, for each acquisition, an ensemble of 5 ROIs (64 pixels × 64 pixels) was selected from the acquisition slab central image of the uniform CTP486 phantom module. Given the radial symmetry of the 2D NPS, radial profiles along many directions were averaged

PLOS ONE
Physical image quality of different CT scanners with various clinical head scan protocols in order to obtain the one dimensional NPS curve. In particular, the selection of radial profiles was carried out every 10˚over 360˚, obtaining a total of 36 radial profiles. In order to estimate peak position, NPS curves were fitted by using a specific peak function (namely "InvsPoly") implemented in Origin: where x c is the peak position, A/A 1 /A 2 /A 3 are coefficients related to the amplitude of the peak, w is a parameter related to the width of the curve and y 0 is an offset.

Contrast-to-noise ratio (CNR)
Images of different inserts (i.e. teflon, delrin, LDPE, polystyrene, acrylic) of the CTP404 phantom module, whose nominal HU values are reported in Table 2 [60], were used to estimate CNR. In particular, CNR was estimated as follows [67,68]: where HU obj and HU bkg are the mean of HU values in a circular ROI (diameter 9 mm) in the considered insert and background region, respectively, while σ bkg is the SD of CT numbers in a background region close to the considered insert (Fig 1).

Modulation transfer function (MTF)
Spatial resolution was evaluated in the spatial frequency domain. The modulation transfer function (MTF) was computed through the circular edge method (i.e. starting from edge spread function measurements) as follows [65,69]: where f and ESF represent the spatial frequency and edge spread function, respectively. In particular, ESF was referred to the teflon insert of the CTP404 phantom module [58]. The spatial frequencies corresponding to 50% (f 50% ) and 10% (f 10% ) of each MTF curve were estimated.

Low contrast detectability (LCD)
A low-contrast detectability (LCD) analysis was performed on the homogeneous module of the phantom (CTP486), using a statistical method [64,70,71] based on the Rose model of threshold signal detectability [72][73][74][75]. Specifically, squared ROIs ranging from 2 × 2 pixels to 11 × 11 pixels (i.e. object sizes approximately ranging from 0.85 mm to 4.75 mm) were employed in this analysis. A set of 900 ROIs (placed randomly and covering all phantom image) were analysed for each ROI size and the contrast threshold (C t ) was evaluated, by assuming a Gaussian distribution of the average of HU values within each ROI [64,70,71], as follows: where Δ is the standard deviation of the Gaussian distribution, with mean value μ. Accordingly, a low contrast object of the same size as the ROIs can be revealed at a confidence level of 95% if its mean HU value differs more than 3.29 Δ from μ [76].

Non-uniformity index (NUI)
Variation of CT numbers within acquisition slab central image of the uniform CTP486 module was assessed through the non-uniformity index (NUI). In particular, NUI was estimated by adapting the method proposed by Li et al and suggested by the AAPM TG233 report for assessing spatial non-uniformity of noise maps [38,77]. Images of the entire phantom were divided into M = 249 small ROIs of 7 mm × 7 mm size. Then, NUI index was calculated as: ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi where I j and <I> are the average of CT numbers within the j-th ROI and average of all I j values, respectively.

Results
Noise results are reported in Table 4. An appreciable difference in noise values across different scanners/acquisition protocols was found. Noise values ranged from 3.5 ± 0.1 HU (Siemens-64 scanner) to 5.7 ± 0.1 HU (GE-16RT scanner).

PLOS ONE
Physical image quality of different CT scanners with various clinical head scan protocols and Siemens-64 (~0.21 mm -1 ) scanners showing lower peak position with respect to the other scanners (> 0.29 mm -1 ). CNR results are reported in detail in Table 6. A substantial variation of CNR values with scanner/acquisition protocol was observed for all inserts (teflon, delrin, LDPE, polystyrene, acrylic). The coefficient of variation (standard deviation divided by mean value) of CNR values   Table 7. MTF curves varied appreciably across scanners/acquisition protocols. Overall, GE-128, GE-64VCT and GE-16RT scanners were

PLOS ONE
Physical image quality of different CT scanners with various clinical head scan protocols characterized by MTF curves with higher values with respect to the other scanners ( Fig 3F). Specifically, GE-128 scanner with sequential acquisition protocol showed the best performance in terms of spatial resolution properties. The coefficient of variationof f 50% and f 10% across scanners/acquisition protocols was 10.1% and 7.4%, respectively. A relevant difference in LCD performance of different scanners/acquisition protocols was found (Fig 4). For all object sizes, the Siemens-64 and GE-64VCT scanners showed lower contrast threshold values with respect to the other scanners. The contrast threshold for a typical object size of 3 mm ranged from 3.7 HU (Siemens-64 scanner) to 5.8 HU (Toshiba-16 scanner, sequential scan mode).
NUI results are reported in detail in Table 8. NUI values ranged from 4.1% (GE-128 scanner, sequential scan mode) to 8.3% (Toshiba-16 scanner, sequential scan mode) across different CT scanners/acquisition protocols.

Discussion
Nowadays, the technological advance in CT imaging has brought increasing number of clinical exams/applications, as well as increasing heterogeneity in characteristics/performance of CT scanners. Therefore, a number of CT scanners with different models or by different manufacturers are often installed in a hospital centre and used by various departments. Previous phantom studies, mainly focused on body applications, have shown that CT image quality can vary substantially across scanners [45][46][47][48], even when similar acquisition protocols are employed [49][50][51][52]. For instance, in a multicentre study, Racine et al [45] have compared the image quality of 68 scanners, in terms of only low contrast detectability, using local clinical acquisition protocols for abdominal CT examinations. They have found an important difference in image quality levels, associated with a variability in CTDI vol values, which increased with growing phantom size. Kuo et al [47] have aimed at characterizing CT practices and performance of 16 CT scanners, in terms of noise and spatial resolution (i.e. MTF), in different centres for cystic fibrosis. A large variety in CT protocols, image quality and radiation dose among the centres was found. A task-based image quality assessment of 4 "older" (model released between 2003 and 2007) and 4 "newer" (model released between 2012 and 2014) CT scanners has been performed by another study [49], using similar acquisition protocols with fixed CTDI vol . The authors have revealed an appreciable difference in high contrast spatial resolution and low contrast detectability across CT scanners. In the study by Zhang et al [50], both subjective and objective methods were used to evaluate the high contrast spatial resolution capabilities of three 64-slice CT scanners, using the same scanning parameters. The CT scanners exhibited different performances, which resulted more relevant for the subjective than the objective method for spatial resolution assessment. Solomon et al [52], using comparable acquisition protocols, have carried out a quantitative comparison of noise texture properties (i.e. NPS analysis) of two CT scanners, for a number of reconstruction filters. The peak frequency values ranged from 0.39 mm -1 to 1.03 mm -1 for one scanner and from 0.43 mm -1 to 0.62 mm -1 for the other.
To the best of our knowledge, this is the first phantom study which comprehensively assessed physical image quality of 5 different scanners (from various manufacturers) for head CT imaging at a single centre, considering the clinical acquisition protocols used by the staff of various Radiology/Neuroradiology departments of our hospital centre for routine examinations. Specifically, we performed an in-depth analysis of head CT image by using a number of quality indices such as noise level, NPS, CNR, MTF, LCD and NUI. Indeed, noise is one of the main factors affecting image quality, given that it can yield fluctuations in raw data and CT numbers. Noise level is usually estimated as the standard deviation of CT numbers within a

PLOS ONE
Physical image quality of different CT scanners with various clinical head scan protocols relatively small ROI. However, this approach does not consider any spatial relationship among fluctuations of CT numbers within the image. We have hence included, in our study, the analysis of NPS, which is mathematically defined as the Fourier transform of the autocorrelation function and provides information regarding the correlation of the fluctuations that occur at different positions on the image [38, 65,67,76,78]. NPS is strictly related to the image appearance, i.e. image texture. Notably, different CT scanners/acquisition protocols can be characterized by different image textures, even at fixed noise level [66]. Moreover, noise can affect the detectability of an object, which depends on its contrast, as well as on its size. Therefore, for various contrast objects (teflon, delrin, LDPE, polystyrene, acrilic), we assessed CNR as the ratio between contrast (i.e. difference in CT numbers between the object and background) and noise [51,67,68,79]. Furthermore, we included the LCD analysis, which allows to assess the limiting detectable contrast threshold for a given object size [71,[80][81][82][83]. In addition, the spatial resolution of CT scanner systems with varying spatial frequencies was evaluated by computing MTF, defined as the Fourier transform of the point spread function of the system, which represents the modulation/loss of contrast as a function of the spatial frequency due to the limited spatial resolution of the imaging system [61]. Therefore, MTF is a descriptive metric of the CT system performance in detecting objects with decreasing sizes (related to the inverse of spatial frequency of MTF). Moreover, in order to make our analysis more complete, we performed also the NUI analysis. NUI assesses the non-uniformity degree of CT numbers within the image, which can reflect the presence of potential artifacts (e.g. beam hardening).
While the head CT imaging acquisition protocols for the 5 scanners at our hospital centre were characterized by similar CTDI vol values (range: 54.7-57.5 mGy), there are some differences in various elements which include collimation (range: 8-20 mm), slice thickness (range: 2.5-6 mm) and reconstruction kernel (see Table 3). These differences can partly explain some results of noise, CNR and spatial resolution analyses [42,52,84,85]. For instance, the relatively high slice thickness (6 mm) and soft reconstruction kernel (H31S) for acquisitions on the Siemens-64 scanner can contribute to the lower/higher noise/CNR values as well as to the lower spatial resolution properties with respect to the other scanners/acquisition protocols. Nonetheless, we cannot exclude a possible effect of scanner technology, as well as of some specific components of scanner hardware such as detectors and x-ray tube.
In the 5 CT scanners (from different manufacturers) enrolled in this study, given also the specific way of operating and needs of various Radiology/Neuroradiology departments of our hospital centre, different acquisition protocols for routine head CT examinations are employed, albeit they present similar CTDI vol values. We revealed a not negligible difference in CT image quality across scanners/acquisition protocols. As future research, our results can be useful to guide an optimization of head acquisition protocols for each CT scanner [42][43][44] or to possibly homogenize CT image quality across scanners. Nonetheless, given that a "best" or "worst" CT scanner/acquisition protocol for all image quality analyses was not found, the characterization that we performed can be potentially employed to allow a more appropriate selection of a CT scanner/acquisition protocol for a specific clinical situation. In fact, different clinical applications may require specific image quality properties. For instance, higher performance of a CT scanner in terms of lower noise level are needed to optimize the detection of small haemorrhagic lesions. On the other hand, higher spatial resolution properties can improve metastatic brain lesions detection, while higher low contrast detectability properties are essential for a better interpretation of images acquired after ischemic stroke [86].

Conclusions
In this phantom study, a comprehensive assessment of image quality of 5 scanners (from various manufacturers and with different models) for head CT imaging, as clinically used at a single hospital centre, was performed. While similar clinical acquisition protocols in terms of dose value (i.e. CTDI vol ) were employed, the analysis of several quality indices (including noise level, NPS, CNR, MTF, LCD and NUI) has shown an appreciable and non-negligible variability in head CT imaging capabilities across different scanners/acquisition protocols. This highlights the importance of a physical characterization of each CT scanner/acquisition protocol, in order to optimize CT imaging procedures.