Conceived and designed the experiments: KWF CFC. Performed the experiments: CFC LMM. Analyzed the data: CFC KWF LMM RDC JMB SQ MH. Contributed reagents/materials/analysis tools: KWF JMB RDC. Wrote the paper: CFC KWF MH. During experiments and data analysis, provided input on data analysis and experimental procedures: CFC MH SQ LMM RDC JMB KWF.
The authors have declared that no competing interests exist.
Therapeutic ultrasound (US) can be noninvasively focused to activate drugs, ablate tumors and deliver drugs beyond the blood brain barrier. However, well-controlled guidance of US therapy requires fusion with a navigational modality, such as magnetic resonance imaging (MRI) or X-ray computed tomography (CT). Here, we developed and validated tissue characterization using a fusion between US and CT. The performance of the CT/US fusion was quantified by the calibration error, target registration error and fiducial registration error. Met-1 tumors in the fat pads of 12 female FVB mice provided a model of developing breast cancer with which to evaluate CT-based tissue segmentation. Hounsfield units (HU) within the tumor and surrounding fat pad were quantified, validated with histology and segmented for parametric analysis (fat: −300 to 0 HU, protein-rich: 1 to 300 HU, and bone: HU>300). Our open source CT/US fusion system differentiated soft tissue, bone and fat with a spatial accuracy of ∼1 mm. Region of interest (ROI) analysis of the tumor and surrounding fat pad using a 1 mm2 ROI resulted in mean HU of 68±44 within the tumor and −97±52 within the fat pad adjacent to the tumor (p<0.005). The tumor area measured by CT and histology was correlated (r2 = 0.92), while the area designated as fat decreased with increasing tumor size (r2 = 0.51). Analysis of CT and histology images of the tumor and surrounding fat pad revealed an average percentage of fat of 65.3% vs. 75.2%, 36.5% vs. 48.4%, and 31.6% vs. 38.5% for tumors <75 mm3, 75–150 mm3 and >150 mm3, respectively. Further, CT mapped bone-soft tissue interfaces near the acoustic beam during real-time imaging. Combined CT/US is a feasible method for guiding interventions by tracking the acoustic focus within a pre-acquired CT image volume and characterizing tissues proximal to and surrounding the acoustic focus.
CT has long been applied for the characterization of tissues, such as fat and bone, in diagnostic imaging
Mild hyperthermia is an emerging technique for image-guided interventions since tumor oxygenation, vascular permeability and blood flow can be enhanced, potentially increasing the efficacy of radiotherapy and chemotherapeutic drugs
In the guidance of mild hyperthermia, tissue characterization is important since the reflection of sound waves by bone can create unanticipated regions of thermal damage. Also, during thermal therapy, changes in the speed of sound produce an apparent shift in the position of tissue within and distal to the acoustic focus over successive image acquisitions, providing a basis for ultrasonic thermometry
Open source software environments are emerging as an important component of multi-modality imaging; for example, 3D Slicer facilitates image segmentation and OpenIGTLink is an open-source protocol for rapid transfer of generic data between software and devices used in image-guided procedures
Image registration between CT and US was first quantified. Next, we characterized HU-based segmentation of fat and soft tissue by comparing fat content and tumor size (n = 12) in histology and comparable CT and US slices. We tested the feasibility of using CT to identify tissue within the acoustic beam by retrospectively fusing images from US and CT clinical scanners (n = 4). These data were supplemented by tissue characterization performed on living animals using a small animal CT scanner.
OpenIGTLink has been applied to interface a Siemens Sequoia US scanner with a cone-beam breast CT via electromagnetic (EM) positioning. The resulting open source software acquires US images in real-time, computes the 2D image slice location and transmits the image and location via OpenIGTLink within 0.1 seconds (
By bridging EM tracking hardware and image acquisition with 3D slicer, US images were acquired for real-time combined CT/US with mm-scale accuracy. The target registration error was indicated by the quality of the transformations between the coordinate systems described in
(a) Relevant hardware for combined CT/US with the coordinate systems used to join the tracked US transducer and CT image space. The US plane,
The propagation of these errors was indicated by the target registration error between the US image and comparable CT slice after co-registration of the images. Slices through the cylinders in the phantom were easily visualized in both modalities and accurate co-registration was evident by the consistent shape and alignment between the targets (
The transplanted tumor cells formed a mass that expands within the mammary fat pad (
H&E stained histology of MET-1 tumors. (a) H&E stained section of a single tumor (
In this study, the grayscale B-mode image amplitude was 111.0 ± 9.6, 76.8 ± 5.7, 107.8 ± 8.5, and 63.3 ± 7.9 for fat, muscle, bone and tumor, respectively. Fat and bone were differentiated from the less echogenic muscle and tumor tissue (p<0.05, multiple comparison ANOVA); however, fat and bone or muscle and tumor could not be differentiated from one another using US.
Segmentation of CT images according to the HU in
(a–d) Isosurfaces of a mouse with a single hindlimb tumor, (a), bone, (b), fat, (c) soft tissue, (d), combined bone, fat and soft tissue isosurfaces. (e–h) Transverse plane CT images and segmented image from location of black dotted line in (d), where yellow indicates fat, red indicates soft tissue and white indicates bone.
HU Range | Tissue | Color |
−300 to 0 | Fat | Yellow |
1 to 300 | Protein-Rich | Pink |
301 to 3000 | Bone | White |
The colors in the table are used throughout the paper to indicate tissue type.
H&E-stained histology slides and comparable CT slices had similar morphology. Representative histology images (
(a–c) Histology and (d–f) corresponding segmented CT slices for tumors of diameters 45 mm3, 113 mm3, and 254 mm3, respectively. (g) The tumor area in histology is correlated with the cross-sectional area of protein-rich tissue detected by CT (r2 = 0.92, slope = 1.2). (h) The average HU of 1 mm2 ROIs in the fat pad is less than within the tumor when averaged across all tumors (p<0.05). (i) The CT number is higher within the tumor compared to the surrounding fat pad for all tumors.
The area of fat in histology slices decreased with increasing tumor size (
(a) Fat, as measured from histological slices, decreases with increasing tumor area (r2 = 0.51, slope = −0.3). (b) Box plot of average CT histograms show the fat shifted HU values in small tumors. (c–e) Average histograms of tumors in three tumor size groups show the presence of a tail in the range of HU associated with fat. Based on CT images and histology, fat content decreases with increasing tumor volume.
A real-time tissue-type overlay from CT was acquired
(a) CT slice of a hindlimb tumor corresponding to the US slice in (b). The segmented CT slice is shown in (c) and overlaid on the US image in (d). The combined segmented CT/US image identifies fat proximal to and surrounding the tumor during ultrasonic imaging, with registration accuracy on the order of 1 mm. Nearby bone is obvious in the CT image (c) but not immediately visualized by US (b).
We have synthesized a CT/US fusion capability that can combine images from generic clinical and pre-clinical ultrasound and CT scanners to provide real-time ultrasound imaging, informed by CT tissue characterization. Using 3D Slicer and OpenIGTLink, real-time US images can be overlaid on preacquired CT images, facilitating guidance of ultrasound imaging and therapy that is informed by tissue types. The combined CT/US system presented here has clinically relevant accuracy, and CT/US images of a living mouse demonstrate the feasibility of the fusion of images acquired from clinical scanners.
Most importantly, we found that even with tumors on the order of 1 cm and with minimal intervening tissue, CT could accurately characterize fat and bone surrounding the tumor, while ultrasound imaging facilitated recognition of the tumor boundaries. The amplitude of ultrasound echoes is altered by the intervening tissue and the thickness of tissues, such as the cortical bone in small animals. Artifacts, such as speckle and shadowing, also change the image amplitude depending on the location within the body and the acoustic path from the transducer to the tissue. Further, ultrasound cannot assess the distribution of tissue components at the sub-resolution (sub-beam dimensions) level, whereas CT can identify such components based on the distribution of Hounsfield units within a region.
With the acquisition parameters used here, voxel Hounsfield estimates from the dedicated breast scanner have a standard deviation of approximately 30 HU which allows for differentiation between fatty tissue (HU∼−120) and protein-rich tissue (HU>0)
The high correlation between CT and histology for tumor area measurements suggests that CT can accurately define the tumor boundaries for therapeutic planning. Since reflection of ultrasound by bone can result in a local doubling of the thermal dose, accurate mapping of bony structures adjacent to or immediately distal to tumors is important and was successfully accomplished (
US estimates of temperature were determined by the product of a tissue-dependent constant and an apparent time shift detected by ultrasound. The tissue-dependent parameter
Further, due to the low sound speed of fat (∼1450 m/sec) compared to the assumed imaging sound speed of 1540 m/sec, the presence of fat in the acoustic path can de-focus the beam and displace the ultrasound focus away from the expected position
Finally, the acoustic attenuation coefficient of fatty tissue is approximately 1 dB/(cm⋅MHz) lower than tumor and muscle; thus, temperature will increase more slowly in fat than in other tissues for a given thermal dose
Registration accuracy on the millimeter scale is required to guide therapeutic US, but practical problems, such as tracking accuracy, sensor-transducer calibration, and mechanical contact of the transducer with the target decrease registration accuracy. The 6DOF EM sensors have a quoted accuracy of 1.1 mm and 0.6 degrees (95% confidence interval)
The Met-1 tumor cells used in these studies provided a model for developing breast cancer. After transplantation into the fat pads, a growing tumor displaced the surrounding fat and infiltrated the tissue, increasing the mean HU values as compared to fat pads without a tumor. The protein-rich tumor embedded in the fat pad is a heterogeneous tissue with decreasing fat content as the tumor grows and provided a model to test the capabilities of CT tissue characterization.
For the first time, tissue characterization has been performed using fused, clinically-relevant CT and US images; however, there are some limitations that should be acknowledged. Temporal resolution of the image update is currently limited by the 30 Hz maximum frame rate output from our commercial US system. While the physical limitations of US allow for higher frame rates, a 30 Hz update rate is likely to be adequate for interventional applications. An alternative approach is to acquire raw radio frequency signals from a research-based ultrasonic system which operates at a higher frame rate
In addition, optimization of the HU values used in segmentation has not been pursued. While the HU limits we applied for segmentation have a physical basis and are similar to those found elsewhere in literature, methods to leverage
The overall goal of this research was to create an imaging method that can accurately identify tissues in the acoustic beam path. Quantification of the sensitivity, specificity, and accuracy of such a system are important questions that are not fully addressed here. Future work will focus on quantifying the specificity, sensitivity, and accuracy of tissue characterization using fused CT/US.
In summary, we developed an open environment fused CT/US system with 1 mm resolution and applied this system to characterize bone, fat and tumor in a mouse model of developing cancer. Practical applications for this technology include acoustic therapy planning and enhancing US-based thermometry in heterogeneous tissue. In the future, the combined CT/US system will be integrated with ultrasound thermometry as described in
The University of California at Davis Institutional Animal Care and Use Committee approved our study (protocol # 15864). Syngeneic Met-1 tumors grown within the mammary fat pad provided the model used here for tissue characterization and validation. Female FVB mice underwent bilateral transplantation of Met-1 tumor cells into the fourth fat pad. After tumor growth to a diameter of approximately 0.5 cm, seven mice with 12 tumors (two tumors did not develop) were imaged with CT after euthanasia using a dedicated breast CT
Four female mice with Met-1 tumors within the mammary fat pad were imaged to test
The feasibility of CT segmentation using
Tumors were localized in the CT image stack by generating circular ROIs in multiple transverse images in the region of the hind limb using custom MATLAB software (MATLAB, Natwick, MA). The resultant ROI was sliced in the midline sagittal plane coincident with the histology slice, and the CT slice was cropped so only the tumor region was visible for comparison with histology, yielding a 2D CT image containing only the tumor and surrounding fat pad.
CT images were characterized as fat, protein-rich, or bone according to the values shown in
CT | Histology | ||||||
Tumor Volume (mm3) | Tumor Area (mm2) | Fat Area (mm2) | Percent Fat | Tumor Area (mm2) | Fat Area (mm2) | Percent Fat | |
9.5 | 2.4 | 41.0 | 94.4 | 7.4 | 48.4 | 86.7 | |
13.5 | 16.9 | 33.3 | 66.4 | 10.5 | 53.0 | 83.5 | |
|
45.4 | 14.9 | 29.0 | 66.1 | 16.4 | 35.5 | 68.4 |
74.9 | 33.3 | 17.4 | 34.3 | 26.3 | 43.3 | 62.2 | |
113.4 | 45.8 | 43.0 | 48.4 | 31.7 | 29.6 | 48.3 | |
114.1 | 38.7 | 20.1 | 34.2 | 30.1 | 32.0 | 51.5 | |
120.2 | 46.7 | 28.0 | 37.5 | 37.8 | 29.6 | 43.9 | |
147.6 | 39.3 | 13.7 | 25.9 | 37.8 | 37.8 | 50.0 | |
193.2 | 52.7 | 35.3 | 40.1 | 40.1 | 32.6 | 44.8 | |
225.1 | 72.8 | 23.8 | 24.6 | 52.0 | 30.4 | 36.9 | |
|
253.9 | 62.0 | 33.1 | 34.8 | 49.3 | 40.2 | 44.9 |
309.4 | 72.4 | 26.7 | 26.9 | 69.7 | 26.5 | 27.5 |
Histological images were used to validate the percentage of fat in the region surrounding the tumors and tumor volume. Two researchers blinded to tumor size selected ROIs containing either fat or tumor cells on histological images. Linear regression indicated similarity between the two measurement sets (fat area: r2 = 0.61, tumor area: r2 = 0.96). The mean of these two sets of measurements was used to minimize selection bias. The area of the fatty region was divided by the area of the combined tumor and fat regions. Tumor volume was estimated from histology by assuming an ellipsoidal shape and overlaying two radii of the tumor in the 2D slice (r1, r2), while the third radius (r3) was estimated by calculating the mean of the first two radii. Tumor volume was estimated by taking the volume of the ellipsoid with radii r1, r2, and r3 (measurements reported in
The area of tumors in the histological slice (as described in the
An EM tracking system (NDI Aurora, Ontario, Canada) was used to detect the spatial location and orientation of sensors affixed to the US probe (as shown in
The software component of our fused CT/US system is a plug-in created for 3D Slicer that imports the transducer position and corresponding US image and performs transformations for registration between CT and US, as described in the subsequent sections (plug-in source code, test data, and tutorial are available online:
Following notation used in Prager et al., we use MTN to indicate a coordinate transform from an arbitrary coordinate system,
Each transformation consists of a translation and three Euler angle rotations. All transformations assume fixed axes and column vectors with rotations acting on objects and a rotation order of
Accurate calculation of the orientation and translation between the transducer sensor and US image plane has been discussed previously, and many methods are outlined in
The tracked volume and CT imaging stack were registered by selecting pairs of locations in the CT stack and the corresponding location in the tracked volume. Measurements in the tracked volume were made with a second EM sensor that was not attached to the ultrasound transducer. Given these paired measurements, the affine transformation between the CT stack and the tracked volume (
The 3D position, orientation, and image must be stored with the correct time-stamp to achieve accurate registration and reconstruction. In order to verify the accuracy of the timestamps, images of the base of a water bath were acquired along with the sensor location during the scans. The ultrasound technician moved the transducer in an oscillatory motion during the acquisition. The distance from the center of the transducer to the base of the water bath was measured from the US images by applying a threshold to generate binary images and counting the number of pixels from the center row of the image to the point on the line along the image row. The time lag between the sensor location and the distance to the bottom of the water bath measured from the US image was determined by taking the cross-correlation of both of the signals.
(TIF)
(TIF)
(DOCX)
The authors would like to acknowledge Dr. Sandra Taylor for discussions about statistics, Jason Peters for technical assistance with CT imaging, members of Kevin Cleary's lab group for discussions about real-time visualization of fused imaging, and Julien Bec for mechanical design assistance.