Study population

The study population comprised all patients diagnosed with and treated surgically for differentiated thyroid carcinoma by the Department of Health in Elda. This area is an industrial zone in the Valencian Community with a medium-low economic and cultural level. The number of inhabitants in this Department ranged from 164,999 in 1984 to 193,588 in 2013 [22].

Study design and participants

This retrospective cohort study covered 30 years (1984–2013) and included all patients diagnosed with and treated surgically for differentiated thyroid carcinoma. A patient was considered to have this disorder if the diagnosis was confirmed histologically. To determine this diagnosis, an examination was undertaken of all clinical records of patients who underwent surgery for any type of thyroid disorder. The records were examined in all the databases of any medical-surgical service possibly related with patients diagnosed with differentiated thyroid carcinoma (Otorhinolaryngology, General Surgery, Oncology, Endocrinology, Pathology, Radiology, Committee of Endocrine and Thyroid Tumors, Clinical Documentation, and Hospital Discharge Registry). Information was obtained about the time between diagnosis of differentiated thyroid carcinoma and date of death, or date of end of study (December 31, 2013), or date of last clinical contact (in those cases where we were unaware of whether the patient had died during the follow-up).

Variables and measurements

The main outcome variable was all-cause mortality during the follow-up period. This was assessed by analyzing the databases mentioned above and in case of doubt by telephoning the number recorded on the hospital records (land line and mobile).

The following secondary variables were also recorded at diagnosis: age, gender, number of inhabitants in the town of residence (</≥ 10,000 inhabitants) [22], known family history of thyroid cancer, type of thyroid surgery (total 1–2 stages, partial or none), type of cervical lymph node surgery (central+lateral, central, lateral, or none), type of differentiated cancer (papillary or follicular), histological subtypes (good or poor prognosis) [24], solitary papillary intraglandular microcarcinoma <1 cm (yes or no), multicentricity (solitary or multifocal), adenopathy with a histopathological diagnosis (central+lateral, central, lateral, or none), AJCC (American Joint Committee on Cancer) TNM staging system [25], diagnostic stage (I, II, III, IVa, IVb or IVc), permanent postoperative recurrent nerve paralysis (yes or no), permanent postoperative hypoparathyroidism (yes or no), local tumor persistence (persistence of diseased thyroid tissue in the surgical thyroid bed in patients who had previously received primary surgical treatment) and regional lymph node persistence (persistence of cervical and/or upper mediastinal lymph node disease in patients who had previously been treated with cervical lymph node surgery) (yes or no), distant metastasis (yes or no), and radioiodine therapy (yes or no). These variables were all obtained from the sources of information mentioned above, except for the population of the town of residence, which was obtained from official data by the National Statistics Institute [22], and the stage, which was obtained from the 2010 AJCC classification (7th edition) [25], using the TNM parameters, type of cancer and age, obtained as described above.

Sample size

The sample comprised 201 patients, of whom 26 died. Thus, using a 95% confidence interval and with an expected area under the ROC curve (AUC) of 0.8, in order to contrast an AUC different to 0.5 the power obtained was 99.85% [26].

Statistical analysis

Absolute and relative frequencies were used to describe the qualitative variables, with means and standard deviations for the quantitative variables. A Cox multivariate regression model was constructed to determine which variables were associated with all-cause mortality and the adjusted hazard ratios (HR) were obtained. As we had few patients, we selected a maximum number of explanatory variables in the model (one for each 50 non deaths). For this we used a stepwise algorithm to determine those variables that might predict mortality better. To obtain the variables in the model we analyzed all the possible combinations with a maximum of 3 variables (4,525), calculating the value of the C-statistic in all of them. The combination with the highest value was then selected. The goodness of fit of the model was assessed by the score test. From the β coefficients of the multivariate model a risk table was constructed based on the sum of the points to estimate the probability of death [27]. After calculating the points and their associated risk, the AUC was calculated for this mortality scale and the following cut points were selected on the scale constructed: 1) Optimum: that which minimized the square root of (1-Sensitivity)² + (1-Specificity)²; 2) Confirmation: the minimum score that had a positive likelihood ratio (PLR) ≥10; 3) Discard: the maximum score that had a negative likelihood ratio (NLR) <0.1. Points 2 and 3 were chosen because, according to the Evidence Based Medicine School, they are those that permit the positivity of a diagnostic test to be confirmed or discarded conclusively [28]. Using these cut points, we calculated the indicators of validity (sensitivity and specificity), yield (positive predictive value (PPV) and negative predictive value (NPV)) and usefulness (PLR and NLR). The risk groups for death were also designed using the cut points obtained previously: low risk (less than the discard point), medium risk (from the discard point to the optimal point), high risk (from the optimal point to the confirmation point), and very high risk (equal to or greater than the confirmation point). Finally, the survival of these groups was compared using the log-rank test. The survival curves were represented using the Kaplan-Meier technique. All the analyses were done with an α = 5% and the associated confidence interval (CI) was calculated for each relevant parameter. All the analyses were done with IBM SPSS Statistics 19.

Mobile application

The model was implemented on a mobile telephone application (App) (S1 Text). Using this tool the likelihood of all-cause death can be calculated without the need for mathematical calculations. This App is available for the following mobile telephone operating systems: Android and Apple.

Ethical questions

This study was approved by the Ethics Committee of the Department of Health of Elda. Given that the study was undertaken using retrospective data, no informed consent was requested from the patients. The Ethics Committee approved this procedure.

Study strengths and limitations

The main strength of this study resides in the lack of other predictive models of all-cause mortality, although Sebastián et al. constructed a similar model classifying patients in risk groups [17]. However, the choice of these groups was based on survival quartiles (25%, 50% and 75%) rather than on risk groups that could conclusively confirm or discard the death of a patient. Furthermore, this model only encompassed half the prediction time of our study for high-risk patients. Likewise, our model generated an AUC above 0.80 with a high statistical power, despite involving a smaller sample size than that of other studies [3,9]. This confers our model with a high discriminating power. In addition, the methodology described herein could also be used to predict mortality in other types of cancer.

Although an important limitation of our study was that the sample size was just 201 patients, this was nevertheless sufficient for the aims of the study, as we were evaluating the predictive model and its associated AUC. The model was very significant (p<0.001) and the AUC very high (>80%). Furthermore, the number of variables in the model was selected bearing in mind our number of deaths and total patients and testing all the possible combinations. However, as occurred in the Framingham study, we obtained significant HR with wide CI. Nevertheless, as in the Framingham study, we were examining the overall model and not each individual variable [29–31]. Nevertheless, if the study had included more patients we could have introduced a greater number of variables in the model and divided the study sample into two parts in order to perform internal validation [32].

As we used data for a 30-year period we must accept a certain degree of measurement bias. However, to minimize this degree, all the sources of medical information (both computerized and non computerized) were strictly checked in an orderly manner.

Thus, after extensive validation, our models could be implemented in clinical practice. As did Hay et al. after their study [33], the authors of the present study encourage other researchers to determine the validity of our scoring system in other populations and then compare their predictive power with the current models.