Introduction

Improvements in the 5-year survival rates of cancer patients have long been cited as proof of the increasing effectiveness of anti-cancer treatment modalities both in scientific articles [1]–[4] and in media and policy documents [5]. The use of 5-year survival rates to reach this conclusion, however, was criticized in 1990 by Sondik [6], who argued that improvements in diagnostic abilities can contribute to increases in 5-year survival rates of cancer even in the absence of any progress towards more effective clinical interventions. Ten years ago, Welch, Schwartz and Woloshin [7] carefully investigated this criticism. The authors posited that improvements in cancer treatments should simultaneously increase survival rates and reduce mortality rates, thus leading to a correlation between the two, while improvements in survival rates due to artificial causes (e.g. due to earlier diagnoses) would not be related to decreased mortality. Indeed, they found no correlation between improving survival rates and decreasing mortality rates, which would seem to suggest that there have been no improvements in cancer management. On the other hand, if incidence rates were rising solely due to an increasing occurrence of cancer, then no correlation with survival rates should be observed in equilibrium. However, if the incidence was rising due to earlier diagnoses, then a correlation with rising survival rates should be detected; the latter was the case and a relationship was identified.

These results were interpreted as follows: “It has been reported that improvements in treatment results for patients with malignant disease represent spurious effects of diagnosis at an early stage” [8]. The insight that 5-year survival rates are a misleading measure of real improvement has since become canonical and is now included in standard textbooks on cancer biology [9]. Furthermore, these results also strongly call into question the efficiency of clinical interventions for cancer patients, since a lack of correlation between mortality rates and 5-year survival would seem to indicate that clinical management of cancer does not have any positive effects on patient survival.

We hypothesized that changes in the mortality rate itself does not fully capture improvements in cancer treatments and for this reason may not correlate with increasing survival rates. Importantly, even if cancer treatments were effective for an increasing number of patients, the mortality rate could still rise if the incidence of the given cancer type itself was increasing.

To illustrate this point, consider for example non-Hodgkin lymphoma. In the US, this disease had an age-adjusted incidence rate of 11 and a mortality rate of 5.6 per 100,000 people per year in 1975, and an incidence of 20 and a mortality rate of 7.9 per 100,000 per year in 2000 [10]. Although it seems that the outlook for patients diagnosed with this particular type of cancer deteriorated between 1975 and 2000, since the mortality rate rose by 2.3, clinical management of this cancer type actually improved dramatically, as the expected mortality rate in 2000 would have been 10.2. The expected mortality rate is determined by the fact that the incidence almost doubled and therefore, because the mortality rate is not reported as normalized by incidence, it should have almost doubled as well. The fact that the actual mortality rate only rose to 7.9 instead of the expected 10.2 signifies an improvement in the clinical management of the disease.

Based on these considerations, a more indicative measure of an improvement in cancer management should then be the change in mortality rate normalized by the change in the incidence rate (mortality over incidence, MOI). In the above example, rates of 5.6/11 in 1975 compared to 7.9/20 in 2000 reflect a 25% improvement in the overall clinical management of the disease. At the same time, the 5-year survival rate changed from 47% in 1975 to 70% in 2000. In this situation, the 5-year survival rate increased but does not correlate with the change in mortality; it does, however, correlate with the change in MOI. Despite the similarities between the fatality rate (the proportion of deaths within a designated population of cases) and the mortality over incidence, these measures are not identical; the fatality rate is obtained from a given case population, while mortality over incidence does not necessarily use the same population for the mortality and incidence. Nevertheless, due to the similarities between these two concepts, MOI can serve as an approximation for the fatality rate.

To investigate if the improvement in survival rates is related to improvements in the clinical management of cancer, we thus need to compare the change in survival to the change in MOI and not to the change in mortality rate alone. In cases in which the incidence remains unchanged, which is very uncommon, this analysis reduces to the currently used approach of correlating survival to mortality. Note that “improvements in clinical management” refers to any changes in the way cancer patients are dealt with clinically, including the administration of new drugs, the use of a more effective dosing strategy of existing drugs, or simply a cost reduction making previously available treatments more affordable and thus more widely used.

Methods

Results

Using the datasets and analytic approaches outlined in Methods, we first investigated the correlation between changes in 5-year survival rates and incidence and mortality rates, as was done by Welch et al. [7]. The results for all datasets are summarized in Table 1. Figure 1 displays the results for the US dataset as one particular example; all other plots are shown in Figures S1–S11. The correlation between the changes in 5-year survival rates and the changes in incidence and mortality rates are shown in Figure 1a and b. Similar to the findings by Welch et al [7], we also observed that there is no significant negative correlation between the change in survival and the change in mortality in all datasets. Meanwhile, in some datasets (see Table 1), there is a positive correlation between the change in survival and the change in incidence. These results were interpreted by Welch et al. [7] to indicate that improvements in survival rates are predominantly artificial, since a real improvement in the clinical management of cancer should simultaneously reduce mortality and enhance survival, while an artificial improvement, due to early diagnosis, increases survival rates and incidence rates without reducing the mortality. However, as shown in Figure 1c, there is a strong correlation between the change in incidence and the change in mortality. This finding indicates that much of the change in mortality is a result of the change in incidence, and therefore the correlation between the change in 5-year survival and the change in mortality due to improved treatment is obscured. In Figure 1d, we present the correlation between the change in 5-year survival rates and the change in MOI (Pearson's correlation coefficient −0.55, p-value  = 0.000075). In the SI, we show the correlation coefficients and p-values for alternative values of a. The correlation changes only slightly for these alternative intervals, and is significant for all the intervals investigated. The other datasets also display negative significant correlations, as is shown in the SI. The size of the correlation in some datasets is moderate (∼0.5), while it is strong in others. This difference may be due to different strengths of the artificial improvements in 5-year survival, but also may be due to the difference in data collection. Generally, the US datasets have weaker correlations than those from the Nordic countries, which may be due to the fact that with the US datasets, we divided cancer types into more detailed subtypes, which likely adds variation to the calculations. There is no general trend for the change in the correlation coefficient for different intervals that is shared by the different datasets that we used. See SI for a rigorous development of the negative correlation between the change in 5-year survival and MOI. We also repeated our analyses using Spearman's correlation (see SI), and found that the changes when using this metric were minor.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Correlations between different measures for the US dataset.

(a) Change in 5-year survival versus change in mortality. (b) Change in 5-year survival versus change in incidence. (c) Change in mortality versus change in incidence. (d) Change in the 5-year survival versus change in mortality over incidence (MOI). The Pearson's correlation coefficient (R) and its p-value are displayed on top of each panel. The change in 5-year survival is strongly linearly associated with the change in MOI.

https://doi.org/10.1371/journal.pone.0083100.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Pearson's correlation coefficients for different pairs of measures.

https://doi.org/10.1371/journal.pone.0083100.t001

Note that we have observed the pattern expected under real improvements in the clinical management of cancer; thus, there is no reason to assume that improvements in cancer treatment are not real, as implied by Welch et al. [7]. However, this observation in itself is not proof that the improvements in cancer treatment are real, because over-diagnosis [12] — the identification of cancer cases due to improved diagnostic tools that would not have resulted in death even without the diagnosis — can artificially create a negative correlation between the change in 5-year survival and the change in MOI. This fact arises because over-diagnosis artificially increases both survival and incidence, resulting in a negative correlation between the rates of survival and MOI even without any underlying improvements in the clinical management of cancer patients. It has recently become apparent that over-diagnosis is a problem when investigating incidence and prevalence rates of some cancer types [13]. In order to tackle the problem of over-diagnosis, we repeated our analyses of the correlation between the change in survival and the change in MOI using only those cancer types whose incidence did not increase during the chosen time period. We hypothesized that for those cancer types with stable or decreasing incidence rates, the contribution of over-diagnosis should be very low. For the US cohort, this approach led to the use of 35 cancer types.

When performing these analyses, we obtained a significant negative slope between changes in survival and MOI for the US male and combined datasets. For the US female dataset, we obtained a negative slope for all cases, but the slope was significant only when using mortality data from a time period of at least three years after the date at which the incidence was determined. These results are presented in the SI. These findings demonstrate that also for cancer types with almost no over-diagnosis, the negative correlation between changes in 5-year survival and in MOI still exist; thus, it is likely that the improvements in the 5-year survival rates are real even in cancer types where incidence rates did increase, as there is no reason to posit a meaningfully higher over-diagnosis rate in those cases.

The results for the US cohort are a typical example, and the figures for all other cohorts are presented in the SI. Table 1 displays the correlation coefficients and their p-values of the following comparisons for all available datasets: changes in 5-year survival rates and incidence, changes in 5-year survival rates and mortality, changes in mortality and incidence, and changes in 5-year survival rates and MOI. Note that in all cases, there is a negative correlation between the rates of survival and MOI, which is always statistically significant. Also, by using only cancer types whose incidence did not increase in the Nordic countries, we consistently obtained a significantly negative slope for all datasets, except for the female population in Norway for which there were only 6 cancer types, a sample set potentially too small to allow for significant results. Note that the mortality and incidence rates are age-adjusted, and the adjustments of the US and the Nordic data were potentially done differently; this might thus represent the source for some of the differences we observed between the US and Nordic countries.

For the US population, we repeated all calculations using 10-year survival data instead of 5-year survival data and obtained consistent results. The correlation between the change in 10-year survival and the change in MOI was negative and significant for all patients as well as for both genders. Also, when we used only the cancer types for which the incidence did not increase, we obtained a significantly negative correlation (see SI).

Discussion

In this paper, we have proposed an alternative approach to investigate the effectiveness of clinical interventions against cancer using information of incidence, survival, and mortality of cancer patients. We showed that by comparing survival to MOI (mortality normalized by incidence), the expected correlation patterns between the changes in 5-year survival and MOI are obtained. By doing so, we negated the previous claim by Welch et al. [7] that the expected correlation cannot be observed and therefore, that 5-year survival rates cannot be used to evaluate progress in the fight against cancer. In addition, we demonstrated that early diagnosis is not a major problem for the analysis of changes in survival rates. However, this finding by itself does not prove that the effects of artificial improvements in the 5-year survival are negligible; rather, it shows that there is no reason to assume that their effect is large, because even though MOI cannot improve artificially due to lead time bias, it can still artificially improve due to over-diagnosis.

To tackle the problem of over-diagnosis, we analyzed only those cancer types whose incidence did not increase during the time period of observation; thus there is no reason to believe that these cancer types are significantly influenced by over-diagnosis. For these cancer types, we also obtained a significantly negative correlation between the change in 5-year survival and the change in MOI. Therefore, for these cancer types, the improvement in 5-year survival rates is not due to early diagnosis or over-diagnosis. The fact that we obtained the same results for cancer types with an incidence that did not increase indicates that over-diagnosis does not play a major role in artificially enhancing the 5-year survival for any cancer type.

We found that in general, the changes in 5-year survival rates contain information that can be harvested to measure our progress in the war against cancer. While early diagnoses artificially increase the rates of 5-year survival, in general, their effect is minor relative to the effect of improvements in cancer treatments. This finding becomes apparent when the survival rates are compared to the mortality rates normalized by the incidence (MOI).

The survival measure and the MOI measure should ideally be used in tandem to assess improvements in cancer treatments, as no measure is perfect. Survival alone may include minor effects of lead-time bias. The MOI measure alone does not properly capture improved survival times in cases when mortality rates do not change. However, improvements in the survival measure alone can be used to provide a reliable indication of progress.

Note that the correlation between MOI and 5-year survival was already observed by Vostakolaei et al. [14] who showed that MOI was generally a good proxy for 5-year survival. However, our work differs from their investigation in that we have analyzed the correlation between the change in MOI and the change in 5-year survival, in order to determine whether improvements in survival rates are real. This question was not answered previously since the authors did not compare changes in those measurements over time, but rather looked at a single snapshot in time.

It is worth noting that other approaches to assessing the improvements in cancer treatment can be taken, such as analyzing the changes in the number and ratio of patients diagnosed at different stages of cancer development. However, such methods require more detailed information about the diagnostic tools used, and should be applied to specific types of cancers separately, thus differing from the wide-scale approach we applied here.

Supporting Information

Acknowledgments

We gratefully acknowledge support from the Dana-Farber Cancer Institute Physical Sciences-Oncology Centre (NCI U54CA143798), and M. Gorfine and the Michor lab for helpful discussions.