Inter-observer delineation variation has been detailed for many years in almost every tumor location. Inadequate delineation can impair the chance of cure and/or increase toxicity. The aim of our original work was to prospectively improve the homogeneity of delineation among all of the senior radiation oncologists in the Nord-Pas de Calais region, irrespective of the conditions of practice.
All 11 centers were involved. The first studied cancer was prostate cancer. Three clinical cases were studied: a low-risk prostate cancer case (case 1), a high-risk prostate cancer case (pelvic nodes, case 2) and a case of post-operative biochemical elevated PSA (case 3). All of the involved physicians delineated characteristically the clinical target volume (CTV) and organs at risk. The volumes were compared using validated indexes: the volume ratio (VR), common and additional volumes (CV and AV), volume overlap (VO) and Dice similarity coefficient (DSC). A second delineation of the same three cases was performed after discussion of the slice results and the choice of shared guidelines to evaluate homogenization. A comparative analysis of the indexes before and after discussion was conducted using the Wilcoxon test for paired samples. A p-value less than 0.05 was considered to indicate statistical significance.
The indexes were not improved in case 1, for which the inter-observer agreement was considered good after the first comparison (DSC = 0.83±0.06). In case 2, the second comparison showed homogenization of the CTV delineation with a significant improvement in CV (81.4±11.7 vs. 88.6±10.26, respectively, p = 0.048), VO (0.41±0.09 vs. 0.47±0.07, respectively; p = 0.009) and DSC (0.58±0.09 vs. 0.63±0.07, respectively; p = 0.0098). In case 3, VR and AV were significantly improved: VR: 1.71(±0.6) vs. 1.34(±0.46), respectively, p = 0.0034; AV: 46.58(±14.50) vs. 38.08(±15.10), respectively, p = 0.0024. DSC was not improved, but it was already superior to 0.6 in the first comparison.
Citation: Pasquier D, Boutaud de la Combe-Chossiere L, Carlier D, Darloy F, Degrendel-Courtecuisse AC, Dufour C, et al. (2016) Harmonization of the Volume of Interest Delineation among All Eleven Radiotherapy Centers in the North of France. PLoS ONE 11(3): e0150917. https://doi.org/10.1371/journal.pone.0150917
Editor: Olorunseun Ogunwobi, Hunter College of The City University of New York, UNITED STATES
Received: July 4, 2015; Accepted: February 21, 2016; Published: March 17, 2016
Copyright: © 2016 Pasquier et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by Ligue Contre le Cancer (http://www.ligue-cancer.net/), Grant number 04, Réseau Régional de Cancérologie; Région Nord Pas de Calais (http://www.nordpasdecalais.fr/jcms/c_5001/accueil), Grant number 1200 1826, Réseau Régional de Cancérologie; Fonds européen de développement régional FEDER (http://www.europe-en-france.gouv.fr/Configuration-Generale-Pages-secondaires/FEDER), Grant number 38420, Réseau Régional de Cancérologie; and Agence Régionale de Santé (http://www.ars.nordpasdecalais.sante.fr/Internet.nordpasdecalais.0.html), Grant number 03, Réseau Régional de Cancérologie. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The Nord-Pas de Calais region is the fourth most populated region of the 22 French metropolitan regions, with 4.052 million inhabitants. This region includes the North and Pas-de-Calais departments and represents 6.2% of the French population. It is one of the most densely populated regions, with 326 inhabitants/km2 compared with 115 inhabitants/km2 in metropolitan France. In Lille and the surrounding areas, where the seat of the Regional Council of Nord-Pas-de-Calais is located, a comparison with national data shows an increased incidence of head and neck, esophageal, lung, liver, bladder, kidney, colorectal, uterine and ovarian cancers . This region comprises 11 centers of radiotherapy.
Radiotherapy plays a key-role in the treatment of cancer. The highly conformal dose distributions produced using modern techniques require careful delineation of target volumes and organs at risk (OARs). An inadequate radiotherapy plan can diminish the chance of a cure and/or increase the risk of toxicity. The quality of radiotherapy plans affects the outcome of chemo radiotherapy in head and neck cancer . In a meta-analysis of eight studies (4 pediatric and 4 adult patients) the frequency of quality assurance deviations ranged from 8% to 71% and radiotherapy deviations were associated with a statistically significant decrease in overall survival (HR of death = 1.74, 95% confidence interval [CI] = 1.28 to 2.35; p < .001) .
Prostate cancer is the second most common cancer in men and remains the most common cancer in developed countries . Inter-observer variability in the definition of target volumes has been well established since the beginning of conformal and intensity-modulated radiotherapy for prostate cancer [5–7]. The aim of our work was to improve the delineation homogeneity among the radiation oncologists in the Nord-Pas de Calais region through collaborative discussions concerning clinical cases and the selection of shared guidelines.
Materials and Methods
All 11 centers were involved: eight private, two with mixed public-private activity and one academic department of radiation oncology. The first studied cancer was prostate cancer. Three fictitious clinical cases were sent to all of the centers. Each case included a detailed description of the clinical history, histologic or anatomopathologic data and computed tomodensitometry (CT) scan of anonymized images. Low- and high-risk (pelvic nodes) prostate cancer according to the D’Amico classification and post-operative biochemical elevated PSA cases were studied. In case 1, anonymized magnetic resonance (MR) images for image fusion were also sent. A detailed description of the three cases and the volumes to be delineated is presented in Table 1. These data were sent with the P2E (AQUILAB SAS) workstation that equips each center. All of the involved physicians delineated characteristically the clinical target volume (CTV) and OARs. After delineation, each center sent the data to Onco-npdc, where contours were compared (C. Viot); the delineation was also anonymized. The following validated indexes were used for delineation comparison: the volume ratio (VR), common and additional volumes (CV and AV), volume overlap (VO) and Dice similarity coefficient (DSC) (Table 2) [8–12]. The contours of a participant were randomly selected as the “reference” (method 1). The same participant was selected for all three cases during the two comparisons. Indeed, the aim of the present study was to increase the homogeneity of delineation, and we hypothesized that the choice of the “reference” contours did not significantly influence the results. We compared each contour with a common contour also comprising the delineation of 9/14 physicians (method 2). This method facilitates the evaluation of delineation harmonization and avoids the selection of a random or reference contour . The results were discussed slice by slice by senior and junior radiation oncologists during three meetings a year, and shared guidelines were selected for each clinical case. A second delineation of the same three cases was then performed to quantify the standardization. The first delineation was conducted during the month prior to the meeting, and the second delineation was achieved in the month following the meeting. The same methodology and indexes of the first comparison were used. Comparison of the OAR delineation was not realized. Statistical analyses were performed using JMP® (Version 10; SAS Institute Inc., SAS Campus Drive, Cary, North Carolina). A comparative analysis of the index before and after discussion was performed using the Wilcoxon test for paired samples. The VO and DSC of the three cases were also compared (Mann-Whitney test for unpaired samples). A p-value less than 0.05 was considered to indicate statistical significance.
All of the participating physicians were volunteers. Each one signed a document in which he or she agreed to collaborate on the work (S1 File). This study was financed by several institutions and participating centers (please see the Acknowledgments section) and was administered by the Regional Cancer Network Onco-npdc. According to French laws, this work did not require advice of an ethics committee. Agreement N1034071 was obtained from the "National Commission for Data-collection and Freedom” (‘‘Commission Nationale Informatique et Liberte´”) for the conduct of this work. Anonymized CT and MR images were used for the development of fictitious but realistic clinical cases. D.P. was responsible for anonymizing the data. No participant had access to the patient data prior to anonymization. D.P. was responsible for initially collecting these data. C.V. was responsible for collecting the anonymous results of delineation. None of the authors or participants were involved in the patient’s medical treatment.
Fourteen physicians involved in the treatment of urologic cancers at the 11 centers participated. In case 1 (low-risk prostate cancer), the first comparison using method 1 showed acceptable agreement with a DSC value of 0.83 (±0.06). Despite the use of MR images, some differences were observed in the apex and base delineations (Fig 1A, 1B and 1C). The chosen guideline was that by the European Organization for Research and Treatment for Cancer (EORTC) . The indexes were not improved during the second comparison but were considered as correct, with a DSC of 0.83 (±0.08) (Table 3).
a-c. First comparison of the clinical target volume delineation for case 1: apex (1a), middle prostate (1b) and base (1c).
Concerning case 2, the differences in the CTV delineation were mainly located at the inferior and medial borders of the obturator area, the inferior border of pre-sacral and external iliac areas and the superior border of the primitive iliac area (Fig 2). The chosen guidelines were those of the Radiation Therapy Oncology Group (RTOG) . Using method 1 the second comparison showed homogenization of the CTV delineation with a significant improvement in VO (0.41±0.09 vs. 0.47±0.07, p = 0.009) and DSC (0.58±0.09 vs. 0.63±0.07, p = 0.0098) (Table 3). The AV was also improved from 41.07 (±10.98) to 33.86 (±9.42), approaching borderline significance (p = 0.07).
Concerning case 3, the differences in the CTV delineation were located at the superior and inferior boundaries and at the anterior and superior border of the volume where CTV moves away from the posterior edge of the pubic symphysis (Fig 3). The chosen guidelines were those from the Radiation Therapy Oncology Group (RTOG) . During the second delineation, VR and AV were significantly improved: 1.71 (±0.6) vs. 1.34 (±0.46), p = 0.0034 and 46.58 (±14.50) vs. 38.08 (±15.10), p = 0.0024, respectively using method 1. The CV was probably significantly decreased relative to the large decrease in the volume ratio. DSC was not improved, but it was already superior to 0.6 in the first comparison (Table 3). Analysis of the images showed a standardization of the delineation of the anterior and superior borders of the CTV (Fig 3).
The first (a) and second (b) comparisons of the clinical target volume (CTV) delineation for case 3. Note the homogenization of the delineation of the anterior and superior borders of the CTV.
The results were similar using method 2. Concerning case 1, the comparison showed acceptable agreement, with DSC values of 0.84 (±0.07) and 0.85 (±0.09), p = 0.88. None of the other indexes were improved. Concerning case 2, the second comparison showed homogenization of the CTV delineation with a significant improvement in the VO (0.43±0.06 vs. 0.48±0.07, p = 0.05) and DSC (0.6±0.06 vs. 0.65±0.05, p = 0.003). The CV was also improved from 81.4 (±11.7) to 88.6 (±10.26) p = 0.048. Concerning case 3, the AV and VO were significantly improved: 39.3 (±17) vs. 33.9 (±14), p = 0.049 and 0.54 (±0.11) vs. 0.56 (±0.10), p = 0.04, respectively. The CV was not different between the two comparisons (87.13(±10.6) vs. 84.5(±18), p = 0.78). The RV was improved: 1.58 (±0.6) vs. 1.37 (±0.47). However, this difference was not statistically significant (p = 0.068). The DSC was not improved as in method 1, although this index was greater than 0.6 in the first comparison (0.69 (±0.1) vs. 0.71 (±0.08)), p = 0.34.
The VO and DSC were compared between cases 1, 2 and 3 for comparisons 1 and 2, using method 1. These indexes were significantly better in case 1 than in cases 2 and 3 (p<0.05) in comparisons 1 and 2. No significant difference was observed between cases 2 and 3.
The aim of our original work was to prospectively improve the homogeneity of the delineation among all of the senior radiation oncologists in the North of France, regardless of the conditions of practice. To the best of our knowledge, this is the only work of its kind in Europe. In this article, we did not seek to further describe accurately the inter-observer variations, which have already been thoroughly done in the literature, but rather to highlight the qualities of this collaborative work across the Nord-Pas-de-Calais region.
The goal of the present study was to evaluate the homogenization of delineation among physicians. There is no standard method in literature for this work; thus, a reference is necessary to calculate the indexes. We hypothesized that the random selection of the same physician would not significantly influence the results. Concerning the “reference” contours from one physician, the differences were slight between the first and second delineation (data not shown). As a limitation of the present study, we could not assert whether this hypothesis was completely right. To overcome this limitation, we compared each delineation with a common contour comprising the delineations of most of the physicians. This method facilitated the evaluation of the harmonization of delineation and avoided the selection of a random or reference contour. The results were similar whatever the method used, with the improvement of some indexes for cases 2 and 3.
It is important to note that the volumetric indexes used in our study to compare the CTV delineation are more sensitive than metric ones. For example, the volume overlap (VO) of two volumes overlapping at 85% is 0.74. The VO of two cubes composed of 10×10×10 voxels after the shifting of one voxel along the diagonal of the cube is 0.57 (729/1271), whereas the mean distance between the two cubes is around one voxel only . There is no standard value beyond the inter-observer variation that is considered low. It is commonly accepted that a value greater than 0.6 is correct; a value greater than 0.8 is considered good and close to the intra-observer variability. In the present study, the DSC values were superior to 0.6 in cases 1 and 3 in the first comparison and after the second comparison in case 2 using method 1. The indexes were not improved in case 1, for which the inter-observer agreement was considered good after the first comparison whatever the method used. Some indexes were improved during the second comparison (method 1: VO and DSC in case 2, VR and AV in case 3; method 2: CV, VO and DSC in case 2, VO and AV in case 3).
The inter-observer delineation variation was significantly larger in cases 2 and 3 than in case 1 for the two comparisons. Indeed, the complexity of these cases was more important, with a delineation based on the pelvic vascular anatomy for case 2 and the lack of macroscopic target for case 3.
Inter-observer delineation variation and its influence on dosimetry have been shown for many years in almost every tumor location [5–7,17–21]. Multimodality fusion can improve homogeneity [22–24]. Some studies have shown an improvement in the delineation homogeneity between radiation oncology residents after educational intervention [25,26]. Short-term improvement in head and neck delineation was shown in 11 residents after a teaching intervention; in this study, the evaluation was subjective as contours were scored in a blinded fashion by the investigators . Wide heterogeneity can be observed among the senior radiation oncologists. In the study by Lawton et al., significant disagreement existed in the definition of the CTV for pelvic nodal radiation therapy among genito-urinary radiation oncology experts , leading to the development of a consensus . Nevertheless, in some situations, guidelines may vary. Malone et al. compared four consensus guidelines concerning the CTV delineation for post-operative radiotherapy after prostatectomy in 20 patients. The mean volumes (±SD) were 60 (±17) cc and 102 (±24) cc for the smaller and larger ones, respectively, bringing about large differences in the doses delivered to OARs .
From this statement, scientific societies have implemented delineation courses worldwide; closer to our region, we can mention the online European and French tools as well as the training delivered during their annual conferences [28–31]. The originality of our additional work lies in the prospective exchange and collaboration of all physicians across our region in a formal setting. This work is ongoing with head and neck and breast delineation and a comparison of prostate cancer intensity-modulated radiotherapy optimization based on common volumes. We wish to extend our work to our neighboring region, Picardy, with which a merger is planned.
This prospective study showed that a collaborative discussion concerning clinical cases and the selection of shared guidelines within an established framework improved the homogeneity of the CTV delineation among the senior radiation oncologists in the Nord-Pas-de-Calais region.
Région Nord Pas de Calais, Fonds Européen de Développement Régional FEDER, Agence Régionale de Santé, Ligue contre le Cancer, Réseau Onco Nord Pas de Calais, and the physicians and physicists of the 11 centers
Conceived and designed the experiments: DP LBCC DC FD ACDC CD MF LG XL P. Martin P. Meyer JFM OO HR MT CV BC EL. Performed the experiments: DP LBCC DC FD ACDC CD MF LG XL P. Martin P. Meyer JFM OO HR MT CV BC EL. Analyzed the data: DP CV. Wrote the paper: DP EL.
- 1. Ligier K, Plouvier S, Danzon A, Martin P, Benoît E, Molinié F, et al. [Elements of completeness and results of the first year of registration of the "Registre général des cancers de Lille et de sa région"]. Rev Epidemiol Sante Publique. 2012; 60(2): 131–9. pmid:22424751
- 2. Peters LJ, O’Sullivan B, Giralt J, Fitzgerald TJ, Trotti A, Bernier J, et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol. 2010 Jun 20;28(18):2996–3001 pmid:20479390
- 3. Ohri N, Shen X, Dicker AP, Doyle LA, Harrison AS, Showalter TN. Radiotherapy Protocol Deviations and Clinical Outcomes: A Meta-analysis of Cooperative Group Clinical Trials. J Natl Cancer Inst. 2013; 105(6): 387–93 pmid:23468460
- 4. http://globocan.iarc.fr/Pages/fact_sheets_cancer.aspx Accessed 29 November 2015
- 5. Cazzaniga LF, Marinoni MA, Bossi A, Bianchi E, Cagna E, Cosentino D, et al. Interphysician variability in defining the planning target volume in the irradiation of prostate and seminal vesicles. Radiother Oncol. 1998; 47(3):293–6. pmid:9681893
- 6. Fiorino C, Reni M, Bolognesi A, Cattaneo GM, Calandrino R. Intra- and inter-observer variability in contouring prostate and seminal vesicles: implications for conformal treatment planning. Radiother Oncol. 1998; 47(3): 285–92. pmid:9681892
- 7. Lawton CAF, Michalski J, El-Naqa I, Kuban D, Lee WR, Rosenthal SA, et al. Variation in the definition of clinical target volumes for pelvic nodal conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2009; 74(2): 377–82. pmid:18947941
- 8. Bueno G, Fisher M, Burnham K. Automatic segmentation of clinical structures for RTP: evaluation of a morphological approach. In: Proceedings of Medical Image Understanding and Analysis (MIUA ‘01). Birmingham, UK: BMVA Press; 2001; p. 73–76.
- 9. Chalana V, Kim Y. A methodology for evaluation of boundary detection algorithms on medical images. IEEE Trans. Med. Imaging 1997; 16(5): 642–652 pmid:9368120
- 10. Dawant BM, Hartmann SL, Thirion JP, Maes F, Vandermeulen D, Demaerel P. Automatic 3D segmentation of internal structures of the head in MR images using a combination of similarity and free-form transformations: Part I, methodology and validation on normal subjects. IEEE Trans Med Imaging 1999; 18: 909–916 pmid:10628950
- 11. Kelemen A, Szekely G, Gerig G. Elastic model-based segmentation of 3-D neuroradiological data sets. IEEE Trans Med Imaging 1999; 18: 828–839 pmid:10628943
- 12. Pasquier D, Lacornerie T, Vermandel M, Rousseau J, Lartigau E, Betrouni N. Automatic segmentation of pelvic structures from magnetic resonance images for prostate cancer radiotherapy. Int J Radiat Oncol Biol Phys. 2007 Jun 1; 68(2):592–600. pmid:17498571
- 13. Allozi R, Li XA, White J, Apte A, Tai A, Michalski JM, et al. Tools for consensus analysis of experts’ contours for radiotherapy structure definitions. Radiother Oncol. 2010;97(3):572–8. pmid:20708285
- 14. Boehmer D, Maingon P, Poortmans P, Baron MH, Miralbell R, Remouchamps V, et al. Guidelines for primary radiotherapy of patients with prostate cancer. Radiother Oncol 2006; 79: 259–269 pmid:16797094
- 15. Lawton CA, Michalski J, El-Naqa I, Buyyounouski MK, Lee WR, Menard C, et al. RTOG GU Radiation oncology specialists reach consensus on pelvic lymph node volumes for high-risk prostate cancer. Int J Radiat Oncol Biol Phys. 2009; 74(2):383–7 pmid:18947938
- 16. Michalski JM, Lawton C, El Naqa I, Ritter M, O'Meara E, Seider MJ, et al. Development of RTOG consensus guidelines for the definition of the clinical target volume for postoperative conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys. 2010; 76(2): 361–8 pmid:19394158
- 17. Giraud P, Elles S, Helfre S, De Rycke Y, Servois V, Carette MF, et al. Conformal radiotherapy for lung cancer: different delineation of the gross tumor volume (GTV) by radiologists and radiation oncologists. Radiother Oncol. 2002; 62(1):27–36. pmid:11830310
- 18. Dewas S, Bibault JE, Blanchard P, Vautravers-Dewas C, Pointreau Y, Denis F, et al. Delineation in thoracic oncology: a prospective study of the effect of training on contour variability and dosimetric consequences. Radiat Oncol. 2011;19;6:118
- 19. Brouwer CL, Steenbakkers RJ, van den Heuvel E, Duppen JC, Navran A, Bijl HP, et al. 3D Variation in delineation of head and neck organs at risk. Radiat Oncol. 2012; 7: 32 pmid:22414264
- 20. Guo B, Li J, Wang W, Xu M, Shao Q, Zhang Y, et al. Interobserver variability in the delineation of the tumour bed using seroma and surgical clips based on 4DCT scan for external-beam partial breast irradiation. Radiat Oncol. 2015;10(1):66
- 21. Nyholm T, Jonsson J, Söderström K, Bergström P, Carlberg A, Frykholm G, et al. Variability in prostate and seminal vesicle delineations defined on magnetic resonance images, a multi-observer, -center and -sequence study. Radiat Oncol. 2013;8:126 pmid:23706145
- 22. Debois M, Oyen R, Maes F, Verswijvel G, Gatti G, Bosmans H, et al. The contribution of magnetic resonance imaging to the three-dimensional treatment planning of localizedprostate cancer. Int J Radiat Oncol Biol Phys. 1999;45(4):857–65. pmid:10571190
- 23. Guo L, Shen S, Harris E, Wang Z, Jiang W, Guo Y, et al. A tri-modality image fusion method for target delineation of brain tumors in radiotherapy. PLoS One. 2014;9(11):e112187 pmid:25375123
- 24. Jager E, Kasperts N, Caldas-Magalhaes J, Philippens M, Pameijer FA, Terhaard C, et al. GTV delineation in supraglottic laryngeal carcinoma: interobserver agreement of CT versus CT-MR delineation. Radiat Oncol. 2015; 10(1):26
- 25. Szumacher E, Harnett N, Warner S, Kelly V, Danjoux C, Barker R, et al. Effectiveness of educational intervention on the congruence of prostate and rectal contouring as compared with a gold standard in three-dimensional radiotherapy for prostate. Int J Radiat Oncol Biol Phys. 2010;76(2):379–85. pmid:19467804
- 26. Bekelman JE, Wolden S, Lee N. Head-and-neck target delineation among radiation oncology residents after a teaching intervention: a prospective, blinded pilot study. Int J Radiat Oncol Biol Phys. 2009;73(2):416–23 pmid:18538494
- 27. Malone S, Croke J, Roustan-Delatour N, Belanger E, Avruch L, Malone C, et al. Postoperative radiotherapy for prostate cancer: a comparison of four consensus guidelines and dosimetric evaluation of 3D-CRT versus tomotherapy IMRT. Int J Radiat Oncol Biol Phys. 2012;84(3):725–32 pmid:22444999
- 28. http://estro-education.org/elearning/Pages/default.aspx. Accessed 29 November 2015
- 29. http://www.siriade.org/. Accessed 29 November 2015
- 30. http://www.sfro.org/17-professionels.html. Accessed 29 November 2015
- 31. http://www.rtog.org/CoreLab/ContouringAtlases.aspx. Accessed 29 November 2015