Assessment of a prognostic model, PSA metrics and toxicities in metastatic castrate resistant prostate cancer using data from Project Data Sphere (PDS)

Background Prognostic models in metastatic castrate resistant prostate cancer (mCRPC) may have clinical utility. Using data from PDS, we aimed to 1) validate a contemporary prognostic model (Templeton et al., 2014) 2) evaluate prognostic impact of concomitant medications and PSA decrease 3) evaluate factors associated with docetaxel toxicity. Methods We accessed data on 2,449 mCRPC patients in PDS. The existing model was validated with a continuous risk score, time-dependent receiver operating characteristic (ROC) curves, and corresponding time-dependent Area under the Curve (tAUC). The prognostic effects of concomitant medications and PSA response were assessed by Cox proportional hazards models. One year tAUC was calculated for multivariable prognostic model optimized to our data. Conditional logistic regression models were used to assess associations with grade 3/4 adverse events (G3/4 AE) at baseline and after cycle 1 of treatment. Results Despite limitations of the PDS data set, the existing model was validated; one year AUC, was 0.68 (95% CI 95% CI, .66 to .71) to 0.78 (95%CI, .74 to .81) depending on the subset of datasets used. A new model was constructed with an AUC of .74 (.72 to .77). Concomitant medications low molecular weight heparin and warfarin were associated with poorer survival, Metformin and Cox2 inhibitors were associated with better outcome. PSA response was associated with survival, the effect of which was greatest early in follow-up. Age was associated with baseline risk of G3/4 AE. The odds of experiencing G3/4 AE later on in treatment were significantly greater for subjects who experienced a G3/4 AE in their first cycle (OR 3.53, 95% CI 2.53–4.91, p < .0001). Conclusion Despite heterogeneous data collection protocols, PDS provides access to large datasets for novel outcomes analysis. In this paper, we demonstrate its utility for validating existing models and novel model generation including the utility of concomitant medications in outcome analyses, as well as the effect of PSA response on survival and toxicity prediction.


Introduction
It is commonly appreciated that the natural history of prostate cancer varies widely [1][2][3]. Indeed, there are an increasing array of prognostic tools available in the localized disease setting [4,5] however, in the metastatic disease state, the utility of prognostic scoring systems has had limited clinical impact in the pre-or post-chemotherapy setting, likely due to the rapid evolution of treatment paradigms (S1 Table) [6][7][8]. With these advances, the optimal choice of sequencing of treatments is unknown. Furthermore there are no validated predictive biomarkers of response or toxicity with any agent but there are a number of prognostic factors and models that have been derived from both pro-and retrospective studies. For example, previous a meta-analysis of patients treated with docetaxel showed that site of metastatic involvement was the most significant prognostic factor [9]. Other prognostic factors evaluated include lactate dehydrogenase (LDH), hemoglobin, performance status, Gleason score, age, albumin, alkaline phosphatase (ALP), pain, prostate-specific antigen (PSA) doubling time [6,7,10] and more recently, the neutrophil-lymphocyte ratio (NLR) [11,12].
PDS is an non-profit organization [13] that allows registered researchers to access and analyze de-identified patient-level data from comparator arms of phase III clinical trials in cancer. We sought to use PDS data to address outstanding issues in meta-analytic data in prostate cancer care. Our aims were to 1) validate a commonly utilized prognostic model for overall survival to assess prognostic factor importance and applicability 2) determine the effect of concomitant medication on overall survival after accounting for other prognostic variables 3) determine whether a sustained reduction in PSA by a variety of definitions significantly correlates with overall survival and 4) explore if any clinical factors at baseline were predictive of adverse events (AE) on docetaxel treatment.

Materials and methods
The PDS online database was accessed on December 1 st 2014 and data obtained for 2,449 subjects with mCRPC from the control arm of 6 phase III clinical trials (Table 1). For the purposes of analysis; metastatic sites were defined as lymph node (LN) only, bone (with or without LN) and no other metastasis locations, lung (with or without bone/LN), liver (with or without lung and bone/LN), or other. Survival was defined as the time from start of the trial until death. Survival analysis did not include the Prostat_Celgene_2009_90 trial due to missing data.
The prognostic model proposed by Templeton et al was validated. This model (Templeton et al [11]) was derived retrospectively using non-trial pre-chemotherapy patients and validated with a cohort using selected pre-chemotherapy trial patients from a single center. Templeton  LK is an employee of Astrazeneca plc. The specific roles of these authors are articulated in the 'author contributions' section. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing Interests: Dr. Joshua has received research funding from Janssen and Astellas. This funding has no relevance to this study. Leila Khoja is an employee of Astrazeneca plc (this work was done independently of AZ). There are no patents, products in development or marketed products to declare. No disclosures to declare by any of the other authors. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
contained all necessary data, CougarB_2008_101, Pfizer_2008_81. A continuous risk score was determined from the parameter estimates of the multivariable model they established. This risk score included the presence of liver metastasis, hemoglobin, alkaline phosphatase (ALP), NLR and lactate dehydrogenase (LDH). We also examined the performance of this model when not all risk factors were available for all datasets, by including all trials in the validation data set and determining the risk score based only on factors that were available across all studies: presence of liver metastasis, hemoglobin and ALP. Here the risk score was determined from the parameter estimates for only these three variables and NLR and LDH were excluded from the risk score calculation.
Performance was assessed with time-dependent receiver operating characteristic (ROC) curves, and the corresponding time-dependent Area Under the Curve (tAUC) at multiple time points (every 3 months) [14]. The 95% Confidence Intervals for the time-dependent AUC were computed using the bootstrap method. Models were compared using the 1 year time point only.
To create a novel model using OS as the endpoint, we first created univariable Cox proportional hazards models, which were fit for each of the 16 clinical covariates, including LDH, ALP, and an indicator variable for more than one organ with metastasis. Backward selection was used to construct a multivariable model. Albumin, LDH, lymphocyte count, NLR, urea and white blood cell count were not collected by some studies producing more than 10% missing values. Thus these variables were not considered in the multivariable model.
Concomitant medications were manually annotated and collated into 38 groups (S2 Table). Non-lipophilic statins and lipophilic statins were analyzed separately as well as a single group. The association of individual concomitant medication with survival was assessed using univariable and multivariable Cox proportional hazards models. The multivariable model was adjusted for all variables in the final multivariable model found in the previous step. The medications taken by fewer than 40 subjects were not considered. To determine if any medications added prognostic value, the time-dependent AUC was calculated for the multivariable model with and without the concomitant medications included. PSA response was defined in three ways: the percentage PSA decrease from baseline to 12 weeks (PSA12W), and a decrease in PSA (!30% or !50%) that was sustained for at least four weeks, occurring within the first 12 weeks of the start of the study (PSA50 and PSA30). Each PSA response covariate was assessed separately in both univariable and multivariable Cox proportional hazards models. The one-year time-dependent AUC was calculated for each multivariable model. For models including PSA50 or PSA30, subjects with less than 12 weeks of follow-up, whether censored or not, were excluded. This was done in order to avoid potential bias due to shorter follow-up times.
Adverse events of interest on treatment were defined and manually annotated according to the 23 most common adverse events occurring in the TAX327 trial [15] and were considered only for the docetaxel based studies (Table 1). To establish a relationship between the occurrence of a grade 3 or 4 (G3/4) adverse event (AE) and any baseline variables, in patients treated with docetaxel, conditional logistic regression models were fit for 10 variables. Backwards selection was used to construct a multivariable model. Only variables with less than 10% missing values were considered for the multivariable analysis. In a similar fashion, a model was constructed for the occurrence of a G3/4 AE after commencing the second cycle of treatment. Here, the occurrence of a G3/4 AE before the second cycle was also considered as a variable in the multivariable model.
In each of the Cox proportional hazards models, the assumption of proportional hazards was tested for each covariate with the examination of the Schoenfeld residuals. If a covariate was found to violate the proportional hazards assumption, the covariate's interaction with time was added to the model. To account for differences between the trials, all Cox proportional hazards models and condition logistic regression models were stratified by trial.

Results
Patient characteristics and baseline prognostic factors were collated including age Gleason score, concomitant medications, sites of metastases, PSA at baseline, albumin, hemoglobin, LDH, ALP, neutrophil count, lymphocyte count and NLR (S3 Table).

Validation of the prognostic models proposed by Templeton et al
When the entire dataset was used for validation and only a subset of the variables were available to calculate the risk score, the one-year tAUC was .68 (95% CI, .66 to .71) (S1 Fig). In this case the risk score was calculated utilizing only the available covariates.

Multivariable model
The final model (incorporating all available parameters defined by Templeton et al., in combination with the available data fields across all the studies) included, hemoglobin, ALP, neutrophil count and organ involvement, i.e. the number of organs with metastases (>1 vs. 1 or less) ( Table 2). A model that included metastasis location, instead of the number of metastases, had similar performance, with one-year tAUC equal to .75 (.73 to .78), as opposed to .74 (.72 to .77) for the previously chosen model. The model including the number of metastases was preferred as it was the more parsimonious of the two models. Metastasis location and number of metastases cannot be included in the same model as they are collinear; the number of metastases sites is determined as the sum of the metastasis locations.

Concurrent medication
Of the 38 medication types, only 22 were common enough to be considered. In the unadjusted, univariable analysis erythropoietin, fish oil, prazoles, non-lipophilic statins and warfarin were prognostic (Table 3). After adjusting for the factors determined to be significant from the combined model; ALP, hemoglobin, neutrophil count, and organ involvement, the use of LMWH, and warfarin were associated with poorer survival whilst Metformin and Cox2 inhibitors were associated with better outcome (Table 3). There was no evidence that the effect of Metformin differed with increasing BSA or weight (data not shown). However, adding medication use to the multivariable model established above did not improve the predictive ability.

PSA response
PSA12W was associated with survival (Table 4). Both before and after adjusting for adjusting for ALP, hemoglobin, neutrophil count and more than one disease site, patients who experienced greater reductions in PSA had a significantly lower risk of death. PSA50 and PSA30 were found to violate the proportional hazards assumption. In both the univariable and multivariable analysis, patients who had a sustained decrease in PSA of 30% or 50%, within 12 weeks of the start of trial, had a lower risk of death (Table 4). However, the effect of a PSA decrease on survival was greatest early in follow-up and diminished with increasing time ( Table 4). The one-year tAUCs did not differ significantly between the multivariable models with PSA12W, PSA50 and PSA30, indicating that no variable outperformed the others in terms of predictive accuracy.

Evaluation of the association of docetaxel treatment with toxicity
Of the 1442 patients treated with docetaxel, 1259 (87%) experienced at least one AE and 417 (29%) experienced at least one G3/4 AE ( Table 5). The risk for a G3/4 AE ( Table 6) was found Table 3 to be associated only with age in a multivariable analysis. Of the 1366 patients who received more than one cycle of medication, 184 (13%) experienced a G3/4 adverse event before the start of the second cycle. The odds of experiencing a severe AE later on in treatment were over 3 times greater for subjects who experienced a G3/4 AE in their first cycle of treatment (OR 3.53, 95% CI 2.53-4.91, p < .0001, Table 6).

Discussion
The trials accessed from PDS included 4 pre-chemotherapy and 2 post-chemotherapy trials. The pooled patient data was representative of a good prognostic population; the majority had bone and or nodal disease and had good performance status (ECOG 0-1) with normal range biochemical parameters. We validated a recent prognostic model using PDS data, which provides confirmation of its utility. The model had reasonably good performance even when not all covariates were available and that the patient characteristics differed from those of the original studies. Templeton et al's original prognostic score was determined by assigning one risk point for each significant variable in the multivariable regression model they established. Instead of using this approach, we calculated the prognostic score from the parameter estimates of their model as we believed it would provide better discriminatory ability. However, in an unreported analysis, we found that Templeton's simpler approach yielded very similar results. This further emphasizes the usefulness of this model, as it is applicable in varied populations and performs well with only a subset of the required information. Interestingly, this model incorporates NLR to be an additional independent prognostic marker. The importance of NLR as a reflection of the hosttumor interaction has been increasingly recognized across a number of malignancies lately [16][17][18], however in our larger dataset we were only able to robustly analyze neutrophil count (due to missing data), which was nevertheless significant. This may suggest that it is the detrimental effects of an inflammatory response that drives the poor prognostic effect of either the neutrophil count or the NLR. Other variables in our combined model are well appreciated such as hemoglobin and ALP, and are likely a reflection of the bulk of the disease, as is the number of metastases. Previously, published post-chemotherapy models have used response to previous treatments such as docetaxel as a prognostic factor [19]. Whilst the value of the response to previous treatments has been recognized in a number of prognostic models (e.g. duration of hormone use, duration of docetaxel administration) we sought to develop a model agnostic to previous treatments given the heterogeneity in the dataset and in order to increase the utility of the model. We defined survival as time from trial entry to death meaning that OS was dependent on the length of follow up on trial. There was no data uniformly available in PDS on date of diagnosis with mCRPC to enable use of a different time point.
The findings on concomitant medication (e.g. Metformin and cox-2 inhibitors were associated with better prognosis) are novel and suggest a possible therapeutic role for certain concomitant medications although this will require dedicated studies. The Stampede study (clinicaltrials.gov NCT00268476) in early de novo metastatic castrate sensitive prostate cancer did not find any therapeutic benefit from celecoxib alone although the different patient population to that studied here may account for our discrepant findings [20]. Nevertheless Cox-2 inhibitors have been associated with the broad range of anti-cancer properties such as apoptosis induction [21] and down-regulation of kinetochore/centromere proteins [22]. Metformin has been tested in a small phase 2 study in chemotherapy naïve castration-resistant prostate cancer, where 2/36 (~5%) of patients had a 50% PSA response [23]. It has also been associated with several anti-cancer mechanisms of action in prostate cancer including inhibition of complex 1 [24], lipogenesis [25], decreases of cyclin D1 [26], down-regulation of the insulin-like growth factor 1 receptor [27] and AR downregulation [28]. The role of anti-coagulants such as warfarin and LMWH in prostate cancer remain controversial in the literature. Our study found a detrimental effect of warfarin however other retrospective reviews have found inconclusive or no effects of warfarin [29][30][31]. However, a recent single institution retrospective series of LMWH treated men (for deep venous thrombosis only) with docetaxel (17/247) suggested benefit on multi-variate analysis (HR, 0.48; P = .035) [19]. We anticipate that the population in our study had cardiac or vascular comorbidities that significantly compromised their survival.
PSA metrics as parameters of response or surrogates for survival in prostate cancer have remained controversial depending on the trial context and power of the analysis [32]. We looked at the value of a variety of PSA metrics found that variables with the most accurate association with survival was with an estimate of the extent of PSA fall (PSA12W0) rather than the presence or absence of a certain percentage of PSA fall. However, despite these correlations, the AUC for survival at 1 year was not significantly different between models, suggesting that individual level discrimination between the PSA parameters was not sufficient, given the impact of other disease variables.
Appropriate cessation of chemotherapy for toxicity or lack of clinical benefit is used as a benchmark of quality of care in oncology [33]. Whilst predictive scores of toxicity and mortality have been proposed particularly in the elderly population, none have been validated in the clinic [34]. Selection of patients for treatment and optimal management based on these models remains problematic. Our modeling of toxicity confirms clinical acumen in that (i) elderly men are more likely to experience toxicities from chemotherapy, although previous clinical series suggest that docetaxel is very tolerable in the elderly and (ii) that patients that have experienced one adverse event are likely to experience further events.
The analysis herein has inherent limitations such as the quality and completeness of data across trial cohorts and heterogeneity of trial populations. Our findings suggest that PDS can provide a novel platform for large-scale analysis to address effect sizes and questions that are not feasible in single trials.