A Tri-Modality Image Fusion Method for Target Delineation of Brain Tumors in Radiotherapy

Purpose To develop a tri-modality image fusion method for better target delineation in image-guided radiotherapy for patients with brain tumors. Methods A new method of tri-modality image fusion was developed, which can fuse and display all image sets in one panel and one operation. And a feasibility study in gross tumor volume (GTV) delineation using data from three patients with brain tumors was conducted, which included images of simulation CT, MRI, and 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) examinations before radiotherapy. Tri-modality image fusion was implemented after image registrations of CT+PET and CT+MRI, and the transparency weight of each modality could be adjusted and set by users. Three radiation oncologists delineated GTVs for all patients using dual-modality (MRI/CT) and tri-modality (MRI/CT/PET) image fusion respectively. Inter-observer variation was assessed by the coefficient of variation (COV), the average distance between surface and centroid (ADSC), and the local standard deviation (SDlocal). Analysis of COV was also performed to evaluate intra-observer volume variation. Results The inter-observer variation analysis showed that, the mean COV was 0.14(±0.09) and 0.07(±0.01) for dual-modality and tri-modality respectively; the standard deviation of ADSC was significantly reduced (p<0.05) with tri-modality; SDlocal averaged over median GTV surface was reduced in patient 2 (from 0.57 cm to 0.39 cm) and patient 3 (from 0.42 cm to 0.36 cm) with the new method. The intra-observer volume variation was also significantly reduced (p = 0.00) with the tri-modality method as compared with using the dual-modality method. Conclusion With the new tri-modality image fusion method smaller inter- and intra-observer variation in GTV definition for the brain tumors can be achieved, which improves the consistency and accuracy for target delineation in individualized radiotherapy.


Introduction
Accurate target definition plays a crucial role in radiotherapy planning of brain tumors, especially, in the image guided radiotherapy (IGRT) which aims at reducing treatment volume toward target volume while ensuring coverage of target volume in all dimensions [1]. To avoid missing target, the intracranial cancerous tissue involvement must be correctly defined in the gross tumor volume (GTV) delineation. Oftentimes single imaging modality cannot provide sufficient information because of its inherent limitations of discriminating different brain soft-tissues or diseased tissues, and combination of different imaging modalities has to be utilized to get more comprehensive understanding of the disease and fulfill an accurate GTV delineation. Therefore, a proper integration of multiple information resources is necessary for the definition of intracranial tumor extension.
Studies showed that utilization of functional imaging techniques, such as 18 F-fluorodeoxyglucose positron emission tomog-raphy ( 18 F-FDG PET), 18 F-fluoromisonidazole PET ( 18 F-FMISO PET), diffusion-weighted MRI (DW-MRI) and dynamic contrastenhanced MRI (DCE-MRI) in addition to CT provides important value to radiotherapy treatment planning for head and neck squamous cell carcinoma (HNSCC) and early response assessment [2]. And in routine radiotherapy practice delineation of GTV for brain tumors relies upon CT or MRI images, or both. However, there are limitations in visualizing tumor and detecting anaplastic tissue with these two image modalities, which may lead to potential inaccuracy in the lesion definition at different steps of the brain tumor management [3]. For example, the conventional MRI is inefficient for the visualization of gliomatous tissue after therapy [4]. Interpretation of anatomical and functional information from multimodal images opens up new possibilities for optimization of the brain tumor treatment. It was shown that PET/MRI coregistration can significantly improve the sensitivity and specificity of 18 F-FDG in evaluating recurrent tumor and treatment-induced changes for brain tumor [5]. And the radiosurgery guided by the combination of MRI or CT and PET stereotactic images was proved to be valuable for treatment [6]. The GTV could be outlined with greater accuracy with the utility of CT, MRI and PET image fusion technique for target delineation in resected high-grade gliomas before radiotherapy [7]. Furthermore, Grosu et al showed that the composite CT, MRI and PET volumes could improve target volume definition and might further reduce inter-observer variability in stereotactic fractionated radiotherapy treatment planning for meningiomas and gliomas [8,9].
The dual-modality image fusion techniques provided in current radiotherapy treatment planning systems (TPS) can fuse and display two image sets in one panel and in one operation. And two fused image pairs will be displayed in two separate panels when using three image sets of different modalities, which is inconvenient for target definition.
A commercial system named Advantage Workstation (GE Healthcare, Milwaukee, WI) is recently available in radiology which can provide PET/CT + MR tri-modality image fusion in one application for optimized brain and body imaging. Although three image sets have been obtained, it displays at most two sets in one panel [10]. And there are not yet validation reports about this tri-modality image fusion technique for radiotherapy treatment planning of patients with brain tumors.
We recently developed a new tri-modality image fusion method which can fuse and display all image sets in one panel and one operation. Its effectiveness for brain tumor GTV delineation was quantitatively evaluated through comparative study with using the current dual-modality image fusion techniques. The method and feasibility study are presented in this report.

Image data acquisition
Image data of simulation CT, MRI, and 18 F-FDG PET-CT prior to radiation therapy from three patients were used in this study with the patients' written consents. The study and consent procedure were specifically approved by the ethics committee of Tianjin Huanhu Hospital and the authors had explicit permission to publish the patient brain images. Among the three patients, one (male, 69 years old) had high-grade glioma and two (one female and one male, 71 and 47 years old, respectively.) had brain metastasis tumors from their non-small cell lung cancer and thyroid carcinoma, respectively. None of these lesions had previously been resected.
CT image sets for radiation treatment planning for the patients was acquired using a Brilliance Big Bore CT (Philips Medical Systems) in supine and head first position. Total number of transverse slices was 224 with slice thickness of 2 mm, and field of view of 2426242 mm 2 and 2246224 mm 2 for the patient with gliomas and brain metastasis respectively.
PET-CT data were obtained with a Biograph64 PET-CT scanner (Siemens Medical Solutions). The applied dose of 18 F-FDG was between 246 and 296 MBq for the imaging studies. The patients were positioned supine with head first and 75 transverse images were acquired from a transmission scan of 3 and 4 min duration time for the patient with gliomas and brain metastasis respectively. The slice thickness was 3 mm, and axial field of view was 4076407 mm 2 .
MRI data were obtained with a Magnetom Trio Tim scanner (Siemens Medical Solutions). With contrast medium of gadolinium-diethylenetriaminepentacetic acid [Gd-DTPA] (0.1 mmol/kg body weight), the axial, gradient echo T1-weighted sequences were acquired from the foramen magnum to the vertex (repetition time = 250 ms, echo time = 2.46 ms, imaging frequency = 123. 2 Hz, magnetic field strength = 3T, flip angle = 70u). The slice thickness was 2 mm, and field of view was 2306230 mm 2 .

Image registration
Normally, the rigid bone of skull confines the brain tissue and leads to little significant non-rigid transformation between the different image sets. Therefore, three-dimensional (3D) rigid-body registration is sufficient for aligning those head image sets from different modalities to the same coordinate system. The rigid-body registrations of CT+PET and CT+MRI were completed respectively with the automatic image registration tool of MIM 5.2 (MIM Software Inc., Cleveland, OH). Based on the mutual information measure, the software mapped the secondary image sets of PET and MRI to the primary image set of CT respectively. Subsequently, these aligned secondary image sets were imported into an in-house developed software for tri-modality image fusion.

New algorithm for tri-modality image fusion
In the study, we set CT image as the primary, background opaque surface and PET/MR images as the foreground semitransparent surfaces. Based on the transparency model in computer graphics [11,12], we proposed a new algorithm for trimodality medical image fusion which is illustrated in Figure 1 and expressed with Equation 1.
Here I mix is the intensity value of a pixel in the fused image; I PET , I MR , I CT are the intensity values of the corresponding pixels in PET, MR, and CT images respectively; t P and t M are the transparency factor of PET and MR images, respectively, which can be adjusted by the user according to the desired transparency weight of each modality. The display of the fused image would vary along with the change of transparency factor (t P , t M ). It provides a convenient tool for the user to look for target information over composite images of single modality, dualmodality, or tri-modality in one single panel ( Figure 2).

GTV delineation
The same group of three radiation oncologists (observers) performed target delineation independently on the composite images of dual-modality image fusion (MRI/CT) and of trimodality image fusion (MRI/CT/PET), respectively, using the contouring protocol published by Graf et al [13]. The GTVs were first contoured with CT and MRI image sets only and then corrected or re-delineated using the information from PET images. GTV delineation on CT, MRI, and PET was performed according to the following steps: (1) The contrast enhanced T1weighted MRI images were used to define GTV MRI . (2) CT images were adjusted to bone window and GTV MRI was expand to enclose all tumor manifestation in CT or MRI images to obtain the GTV MRI/CT , which was the tumor volume defined with dualmodality image fusion (CT/MRI). (3) Then, the GTV PET was delineated on 18 F-FDG PET image set without referencing the GTV MRI/CT . The window setting for 18 F-FDG-PET was made to optimize alignment between tumor margins on MRI/CT and 18 F-FDG-PET in area with tumor-to-normal brain tissue interface. (4) The overlapping region of GTV MRI/CT and GTV PET was defined as GTV common which encompassed the region with pathologic changes shown on all three image modalities (CT, MRI and PET image sets). (5) The volume of GTV MRI/CT located outside the region of enhanced 18F-FDG uptake was re-evaluated by the observers based on the information obtained from CT/MRI/PET image fusion. (6) Thereafter, a part of GTV MRI/CT was added to GTV common . (7) The volume of GTV PET located outside the region with pathologic changes shown on MRI/CT was reevaluated in the same way.   of GTV common and the added parts from GTV MRI/CT and GTV PET . Thus, each observer defined two GTVs (GTV MRI/CT and GTV MRI/CT/PET ) for the same tumor of each patient in the two composite image sets, respectively.

Volume variation.
To evaluate the inter-observer variation in the delineations of the GTVs with the schemes of dual-modality image fusion and tri-modality image fusion described in 2.4, GTV volumes of the tumors obtained by each observer were used and the coefficient of variation (COV) of the three volumes defined by the three observers with each scheme for each tumor were calculated and statistically analyzed [14]. COV is defined as the ratio of the standard deviation (SD) to the mean value. And SPSS Statistics 20.0.0. (SPSS Inc., Chicago, IL) was used for a paired sample two-tailed t test on COV to statistically measure the difference between the means for the two schemes. A two-side p value of less than 0.05 was considered to be significant.
5.2 Shape variation. A 3D shape was generated using the marching cubes (MC) algorithm [15] for each GTV involved in this inter-observer variation evaluation. Subsequently, the average distance between surface and centroid (ADSC) was calculated by averaging the distance of each triangle on the surface to its centroid. For each patient and each scheme, the mean and SD of ADSC of the three observers were calculated as a measurement of the inter-observer shape variation. The paired sample two-tailed t test was applied to evaluate the difference between using dual-and tri-modality image fusion methods.
Furthermore, a 3D median surface of GTVs delineated by the three observers was computed using Deurloo's method [16,17] for each patient and each scheme. The median surface was equal to the 50% coverage probability matrix of the three GTVs; in other words, all points in the median surface were determined at least by 50% observers. Afterwards, we calculated the perpendicular distance along the normal vector from each vertex point on the median surface to each GTV surface delineated by the observers. Three distances were obtained for each vertex point constructing the median surface. The standard variation of the three distances represents the local variation on the 3D surface and is called ''local standard deviation (SD local )'' which shows the level of variation in contouring details of GTVs. Subsequently, the median surface was divided into eight regions based on the eight quadrants of a new 3D coordinate system which moved the original point to the mean centroid of three individual GTVs. SD local was averaged at eight regions and all vertex points on the median surface for each patient and each scheme.

Evaluation of intra-observer volume variation
The intra-observer variation in GTV volume definition was also investigated. Each GTV was contoured three times by the three observers, respectively. The averaged time interval between these three delineations was about two months. Totally, 54 GTVs were obtained for the analysis. Mean values and SDs of the GTV volumes and the COV were calculated, and the intra-observer variation was statistically analyzed.

Inter-observer variation of GTV volumes
The volumes of GTVs for each patient defined by each observer, the mean values of the GTV volumes from the three observers, and the COV with the two schemes are summarized in Table 1. It can be seen that the volumes of GTVs defined with the first scheme varies more than the ones with the second scheme. All the mean values of Table 3. SD local (in cm) of GTV shape variation relative to the median GTV shape for each patient at eight quadrants and average over all vertex points on the median GTV. GTV volumes defined by the three observers are reduced with the second scheme. The mean COV is 0.14(60.09) and 0.07(60.01) for the first and second scheme, respectively, but there is no significant difference statistically (p = 0.30.0.05, 95% confidence interval of the difference is [-0.15, 0.31]), revealing that with the tri-modality image fusion the inter-observer volume variation is not significantly reduced.

Inter-observer variation of GTV shapes
The comparison of ADSC between the first and second scheme for each patient and observer is shown in Table 2. For all three cases, the means of ADSC with the second scheme are smaller than the ones with the first scheme. The SDs of ADSC is largely reduced with the second scheme in all cases. And the paired sample two-tailed t test shows that the reduction is statistically significant (p,0.05, 95% confidence interval of the difference is [0.01, 0.05]), demonstrating that 3D shape variation between the GTVs defined by the three observers is significantly reduced with the new tri-modality image fusion method.
The SD local was averaged at eight quadrants and all vertex points on the median surface for each patient and each scheme (Table 3). For patient 1, SD local is partly reduced with the second  The median surfaces of GTVs onto which the SD local was mapped in color for three patients are shown in Figure 3. Up to about 1 cm in SD local has been observed with both schemes (Figure 3a and 3b) for patient 1. Whereas, there are two regions on which the SD local is reduced obviously with the second scheme (the areas pointed with red arrows in Figure 3b). For patient 2, SD local in a large part of areas (quadrant No. 3,No. 4,and No. 7 as indicated by red, green and light blue arrows respectively in Figure 3c) on the median surface of GTVs from the first scheme is larger than 1 cm and the maximum is larger than 3 cm. For the median surface of GTVs from the second scheme (Figure 3d), however, the maximum SD local is 1.64 cm, much smaller than the former. For patient 3, up to 1.5 cm in SD local is observed in two quadrants of the median surface of GTVs from the first scheme (Figure 3e) while the maximum SD local of 1.15 cm over all vertex points on median surface is observed for the GTVs from the second scheme (Figure 3f).
Large reductions of inter-observer contouring variation in GTV delineation are also observed on axial slices of three patient CT images with the assistance of tri-modality image fusion as shown in Figure 4. The axial slices correspond to the same area in median surface as marked by the red arrows in Figure 3.

Intra-observer variation of GTV volumes
The mean value and SD of the three repeated GTVs determined by each observer for each patient, the COV with the two schemes are summarized in Table 4. Much smaller SD and COV values are observed with the second scheme than the ones with the first scheme in the results of all three observers' delineation for the patients. The mean of COV for all three observers is significantly reduced, from 0.05 (60.03) with the first scheme to 0.02 (60.01) with the second scheme (p = 0.00, 95% confidence interval of the difference is [0.01, 0.06]), revealing that the intra-observer volume variation is significantly reduced with the new tri-modality image fusion method.

Discussion
Our study has shown that the tri-modality image fusion method which integrates CT, MRI and 18 F-FDG-PET has positive impact on the radiotherapy treatment planning for brain tumors. The volume of GTV contoured for high-grade gliomas and brain metastasis can be reduced as compared with using dual-modality (CT/MRI) image fusion. The inter-observer variation of GTV shapes (Figure 3, Figure 4 and Table 2) and the intra-observer variation in defining GTV volumes (Table 4) can be significantly reduced.
The interpretation differences between observers, derived from clinical experience, image interpretation skills, compliance to set guidelines, and treatment philosophy [14], were the major component of inter-observer variation in this study. The volume of GTV defined by observer 2 was larger than other two observers' for each patient and scheme ( Table 1). The same trend could be found in Table 2 for the parameter of ADSC. Although variation in target definition existed among the observers even with the trimodality method, the SDs of ADSC were significantly reduced as compared with using dual-modality method. These results indicate that contours can be drawn more consistently with the new trimodality image fusion method and large reduction of the interobserver difference can be achieved in GTV definition for brain tumors.
The new method provides convenience to observers in the operation for target delineation due to the adjustable transparency values of foreground images in one single panel, which has not been reported in published studies so far to our knowledge. With our in-house developed software tool, observers are able to view composite images in single modality, dual-modality, or trimodality mode in one single panel. Observers can change the transparency factors (t P , t M ) and gain various combinations of trimodality images' information to facilitate a more comprehensive definition of final GTV MRI/CT/PET . And the lessened intraobserver variation of GTV volumes (mean COV is 0.05 and 0.02 for dual-and tri-modality image fusion, respectively) in our study manifests the high reproducibility of results with this software.
Although a relatively small amount of raw data was used in this feasibility study because of the difficulty of getting image data of three modalities for the same tumor site, the results have shown potentials for utilizing three modality image data for the cases for which two image modalities cannot provide sufficient information in target definition for the brain tumors. When the tumor locates close to a critical normal structure or tissue, this method will be helpful for reducing uncertainty in tumor boundary definition and sparing the adjacent normal tissues, especially when high fractional dose and small margins around the tumor are applied. Table 4. Intra-observer comparison of GTV volumes obtained with dual-modality image fusion (MRI/CT) and tri-modality (MRI/CT/ PET) image fusion.

Conclusion
With the tri-modality (CT/MRI/PET) image fusion method, the inter-observer variation in the delineation of GTV shapes and intra-observer variation in GTV volumes definition for the brain tumors can be reduced. This finding reveals the potential usability of this method for reducing uncertainty in brain tumor boundary delineation in radiotherapy treatment planning.