CAH has received funding from Celgene, Orion Pharma, Novartis, Oncopeptides and the Innovative Medicines Initiatives 2 project HARMONY for research unrelated to this study.
High-throughput drug screening has facilitated the discovery of drug combinations in cancer. Many existing studies adopted a full matrix design, aiming for the characterization of drug pair effects for cancer cells. However, the full matrix design may be suboptimal as it requires a drug pair to be combined at multiple concentrations in a full factorial manner. Furthermore, many of the computational tools assess only the synergy but not the sensitivity of drug combinations, which might lead to false positive discoveries. We proposed a novel cross design to enable a more cost-effective and simultaneous testing of drug combination sensitivity and synergy. We developed a drug combination sensitivity score (CSS) to determine the sensitivity of a drug pair, and showed that the CSS is highly reproducible between the replicates and thus supported its usage as a robust metric. We further showed that CSS can be predicted using machine learning approaches which determined the top pharmaco-features to cluster cancer cell lines based on their drug combination sensitivity profiles. To assess the degree of drug interactions using the cross design, we developed an S synergy score based on the difference between the drug combination and the single drug dose-response curves. We showed that the S score is able to detect true synergistic and antagonistic drug combinations at an accuracy level comparable to that using the full matrix design. Taken together, we showed that the cross design coupled with the CSS sensitivity and S synergy scoring methods may provide a robust and accurate characterization of both drug combination sensitivity and synergy levels, with minimal experimental materials required. Our experimental-computational approach could be utilized as an efficient pipeline for improving the discovery rate in high-throughput drug combination screening, particularly for primary patient samples which are difficult to obtain.
Cancer is one of the main causes of death worldwide. Although new treatment strategies have been achieved, they still have limited efficacy as cancer cells can easily develop drug resistance. To achieve more sustainable therapies to treat cancer, we need multi-targeted drug combinations that can inhibit cancer cells more effectively and synergistically. However, the increasing number of possible drug combinations makes a full matrix design unfeasible, even with automated drug screening instruments. Therefore, we proposed a novel cross design to access drug combinations more efficiently. We further developed a drug combination sensitivity score (CSS) that is tailored for the cross design to quantify the efficacy of a drug combination. Using public datasets, we showed that the CSS is a robust metric and highly predictive with an accuracy comparable to the experimental replicates. We also developed a CSS-based synergy score to assess the degree of drug interaction and showed its capability to correctly identify synergistic and antagonistic drug combinations. Taken together, we showed that the cross design and its scoring methods allow a more systematic and cost-effective evaluation of drug combinations. The proposed experimental and computational techniques are expected to be widely applicable in the field of drug combination discovery.
This is a
Despite great advances in the understanding of cancer, there remains a major challenge to develop more effective anti-cancer treatments. Next generation sequencing has revealed the intrinsic heterogeneity in cancer genomes, which partly explains why patients respond differently to the same therapy [
In order to identify sensitive and synergistic drug combinations, high-throughput drug screening has been applied on a large variety of cancer cell lines and more recently on patient-derived cancer samples [
Furthermore, many existing computational tools for drug combination analysis focus on the degree of interaction, i.e. drug synergy, but not the sensitivity of drug combinations. For example, Combenefit [
To overcome these challenges, we proposed a cost-effective experimental and computational procedure to facilitate the prioritization of drug combination synergy and sensitivity. We proposed a novel experimental design to allow either drug to span over multiple doses while the concentration of the other drug is fixed at its IC50 concentration. The resulting drug combination dose-response curves were utilized to determine a drug combination sensitivity score (CSS). Using a large-scale drug combination study, referred to as the O’Neil data [
We proposed a cross design to test the synergy and sensitivity of a drug pair by first introducing the concepts of background drug and foreground drug: background drug is the drug fixed at its IC50 concentration while foreground drug is added into the background drug with multiple concentrations. We allow either drug in the pair to be the background drug, so that two vectors of dose combinations will be intersected at the IC50 concentrations (
(A) The cross design to determine the drug combination sensitivity score. Compared to the full matrix design (left panel), only a single row and a single column from the matrix that correspond to the IC50 concentrations of the two drugs are utilized for the calculation of CSS (middle panel). One drug is utilized as the background drug fixed at its IC50 concentration while the other drug becomes the foreground drug with multiple doses being titrated. The resulting two dose-response curves will be summarized as the drug combination sensitivity score (CSS), from which the S synergy score can be determined as the deviation from a reference model which predicts the expected percentage inhibition effect from monotherapy dose responses. (B) Comparing the cross design with the full matrix design in terms of data size. Less materials are needed for the cross design when the size of the full matrix is two or above.
With the drug combination dose-response curves determined in the cross design, the CSS summarizes the area under the curve similar to the scoring approaches [
The dose-response curve (
The area under the log10-scaled dose-response curve (AUC) is determined according to
The AUC is further normalized as the proportion of its maximal possible inhibition (i.e. 100% inhibition) according to:
The CSS for the foreground drug is defined as a percentage which varies between 0 and 100:
As there are two drug combination dose-response curves depending on which drug is fixed as the background drug, we refer to the results of
Dose-response was measured as percentage of cell viability and retrieved from the supplementary material of O’Neil et al. [
In our analysis, the CSS for a drug combination was determined based on the average % inhibition of the four replicates. The robustness of the CSS scoring was assessed using the Pearson correlation across the four replicates. All the correlation analyses utilized Pearson correlations.
With the CSS being determined for each drug combination, we sought to evaluate the prediction accuracy of multiple machine learning methods. We considered a drug combination as a combination of their targets and chemical fingerprints. We collected the known targets that have been experimentally validated for the 38 drugs from Drugbank [
We compared three state-of-the-art machine learning methods for the CSS prediction: Elastic Net [
We focused on the model performance for predicting new drug combinations within the same cell line, as the set of drug combinations in the training data did not overlap with that in the test data. For each of the 39 cell lines, the number of drug combinations ranges from 290 to 688, with an average of 338. We randomly sampled 70% of the drug combinations to train multiple machine learning models using 10-fold cross-validation, which splits the training data randomly into 10 equally folds, 9 of which were used to fit the model and the remaining one was used to evaluate the prediction accuracy. The model with the lowest RMSE out of the 10-fold cross validation was then used for predicting the CSS values for the remaining 30% of the novel drug combinations as the testing data. As the sampling was done randomly for each cell line, the training and the testing data were therefore balanced. Four metrics including coefficient of determination (R2), root mean square error (RMSE), mean absolute error (MAE) and Pearson correlation (COR) were utilized for evaluating the prediction performance on the testing data. The whole procedure above was repeated 20 times for each cell line on the predefined seeds, and the final model performance was obtained as the mean values of these iterations. To benchmark the performance of the machine learning methods, we utilized one randomly selected technical replicate as the best possible prediction to obtain the upper limit of the performance. All the methods were implemented and evaluated using the R package
The advantage of CSS is that it allows a direct comparison of the sensitivity between a drug combination and its single drugs, and hence facilitates the quantification of drug synergy. The degree of synergy is often calculated as the deviation of the observed drug combination effect from the reference, which is defined as the expectation effect if the drugs are not interacting. However, how much the expected effect should be is a matter of mathematical modelling with certain assumptions. As the choice of the ‘best’ synergy model is rather heuristic, we proposed three variants of CSS-based synergy scores (termed as S scores) by assuming the reference model as the sum, the maximal and the mean of the AUCs for the monotherapy drug responses:
The AUC for a monotherapy drug response was defined according to [
To evaluate the prediction accuracy of the S synergy scores, we defined a set of true synergistic and antagonistic drug combinations as the gold standard, which were determined using the full dose-response matrix data including the combination and monotherapy responses. We utilized the R package
Consider that drug 1 at concentration
HSA:
Bliss:
Loewe:
ZIP:
For each of the four models, the synergy scores were determined first for a given dose combination and then were averaged over the full dose-response matrix. With the four synergy scores determined for each drug combination, the true synergistic and antagonistic drug combinations are those with all four synergy scores consistently higher than 5 and lower than 5, respectively. The aim was then to use the S synergy score which was determined by the cross design data to predict the ground truth determined by the full matrix design. The areas under the ROC curve and the precision-recall curve were used for evaluating how well the S synergy scores can predict the consensus drug combinations determined using the full dose-response matrix data.
We applied the CSS scoring on the O’Neil drug combination data, which consists of 22,737 drug combinations for 39 cancer cells [
(A) The Pearson correlation of CSS1 and CSS2 over all the drug combinations colored according to tissue type; (B) Density plot of the CSS1 and CSS2 distributions; (C) The Pearson correlation per cell line colored according to tissue type; and (D) The Pearson correlation per drug colored according to drug target class.
Notably, we found that drug combinations that involved bortezomib showed much lower correlation (0.26) between the CSS1 and CSS2 values compared to other drug combinations. Since the O’Neil data contains the replicates for single drug screening, we analyzed the coefficient of variation (CV) of the cell viability readout for each drug in the replicates. As expected, we found that bortezomib has the highest CV (0.26), suggesting a relative low quality of the drug combination sensitivity data involving this drug (
Given that the CSS is highly reproducible as a summary of the overall sensitivity of a drug combination, we explored whether CSS can be predicted using pharmacological and chemical information of the drugs. We considered a drug combination as a combination of its drugs’ target profiles as well as their chemical fingerprints, with which the machine learning approaches illustrated in the previous section can be optimized by exploring the feature space using the training data. We examined three major machine learning methods for predictions: Elastic Net, Random Forests and Support Vector Machines.
We found that all of these machine learning approaches worked reasonably well, where Elastic Net consistently achieved the best performance, with a mean MAE of 4.01 which is comparable to that (2.07) of a technical replicate (
RMSE: root mean square error, R2: coefficient of determination, COR: Pearson correlation, MAE: mean absolute error.
5.20±1.11 | 0.80±0.06 | 0.90±0.03 | 4.01±0.86 | |
6.30±1.18 | 0.71±0.07 | 0.85±0.04 | 4.75±0.9 | |
7.47±1.32 | 0.57±0.08 | 0.80±0.04 | 5.80±1.07 | |
2.87±0.59 | 0.93±0.04 | 0.97±0.02 | 2.07±0.45 |
All the values are mean+/-standard deviation.
Since both the drug target profiles and chemical fingerprints were considered as the drug combination features, we next evaluated their prediction performances separately using the Elastic Net method. For drug-target profiles we collected known targets that were experimentally validated as well as the additional secondary targets that were predicted with high confidence using the SEA method. For chemical fingerprints we used the MACCS fingerprint which contains 166 structural features [
RMSE: root mean square error, R2: coefficient of determination, COR: Pearson correlation, MAE: mean absolute error.
5.66±1.27 | 0.77±0.07 | 0.88±0.04 | 4.26±0.95 | |
5.70±1.22 | 0.76±0.06 | 0.87±0.04 | 4.34±0.95 | |
6.27±1.19 | 0.71±0.06 | 0.85±0.04 | 4.87±0.94 | |
5.30±1.14 | 0.79±0.06 | 0.89±0.03 | 4.07±0.88 | |
5.20±1.11 | 0.80±0.06 | 0.90±0.03 | 4.01±0.86 |
All the values are mean+/-standard deviation.
We considered the regression coefficients that were determined in the Elastic Net model as an indication of their importance to contribute to the CSS prediction. We found that certain drug target features were present with high coefficients across all the cell lines (
Cell-line independent as well as cancer subtype-specific features can be identified by evaluating the regression coefficients of the Elastic Net model. Features such as TOP1MT, TOP2A/B has shown consistently positive coefficients as compared to features such as AKT1/2/3 which showed cancer subtype specificity in breast cancer (indicated as arrows).
If the CSS profiles for two cell lines are similar, then their feature importance vectors are expected to be similar. We focused on the most important features that have their absolute coefficients greater than 3 for at least one cell line, resulting in 67 top features. We then utilized these feature importance scores to cluster the cancer cell lines, using unsupervised hierarchical clustering with the Euclidean metric. We found that cell lines of the same tissue type did not necessarily cluster together, indicating their distinctive drug combination response profiles. For example, we found that breast cancer cell lines did not form a single cluster due to the outlier MDAMB436. Indeed, MDAMB436 is the only triple negative breast cancer (TNBC) subtype, while the other cell lines are either ER positive (KPL1, ZR751 and T47D), or HER2 positive (EFM192B and OCUBM). It has been known that TNBC respond to anticancer drugs differently from ER and HER2 positive breast cancers due to distinct disease mechanisms [
Next, we defined the degree of drug synergy as the differences between the dose-response curves of a drug combination and its single drugs. We derived three variants of the S synergy score (
0.72 | 0.72 | 0.46 | 0.55 | |
0.71 | 0.65 | 0.51 | 0.49 | |
0.65 | 0.63 | 0.41 | 0.44 |
We found that the S synergy scores correlated relatively well with the HSA and Bliss scores, while the correlation started to decrease when comparing to the Loewe and ZIP scores. Since all the synergy scoring models utilized different assumptions for the reference of no synergy, we therefore did not expect a perfect correlation in such pairwise comparisons. For example, the Bliss model assumes that two non-interactive drugs act independently while the Loewe model assumes two non-interactive drugs act as one drug. Their differences in mathematical models have been discussed in our previous publications, such as [
Of all the three variations of S synergy score, we found that
Next, we evaluated the predicting accuracy of the S synergy scores for true synergistic and antagonistic drug combinations. To be able to formulate a binary classification problem, we first selected the true positive and true negative drug pairs by applying stringent criteria to determine the ground truth from the full dose response matrix data. The rationale of the ground truth was based on the assumption that full dose-response matrix can capture the truly synergistic and truly antagonistic drug combinations, as it allows the full factorial testing of all the possible doses for a given drug combination. However, as there exist different models for synergy scoring, we decided to apply the most stringent criteria to determine the ground truth, such that a truly synergistic or truly antagonistic drug combination has to fulfill the criteria of all the four existing synergy scoring models (HSA, Bliss, Loewe and ZIP). Namely, the drug combinations with all the four synergy scores (HSA, Bliss, Loewe and ZIP) higher than 5, or lower than -5, were classified as true synergistic or antagonistic drug combinations, respectively. The threshold of [–5, 5] was determined by the empirical distribution of the synergy scores in the O’Neil data, assuming that most of the drug combinations are non-interactive (
Once the ground truth had been determined using the full dose-response matrix data, we then asked the question: Can the S synergy scores that were determined using the cross design correctly identify the most significant synergistic and antagonistic drug combination hits that were confirmed using the full matrix design? We showed that the S synergy scores achieved the area under the ROC curves of 0.997 (
(A) The ROC curves for the S synergy scores to detect true synergistic and antagonistic drug combinations. (B) The S-S plot for all the drug combinations. The drug combinations with the 75th percentile and above for both the CSS and the S scores were highlighted in red to be considered as the prioritized hits for further experimental validation in a confirmatory screen using a full dose-response matrix design.
Drug combinations may potentially lead to more durable clinical responses by overcoming intra-tumoral heterogeneity and drug resistance to monotherapies. Identifying drug combinations that are tailored for personalized medicine is a challenge, as the number of possible combinations may easily grow exponentially [
We developed a novel scoring approach called CSS for drug combinations that can be efficiently determined using the cross design. We found that the CSS is highly reproducible and therefore can be considered as a robust metric to characterize drug combination sensitivity. To understand the drug combination sensitivity, we implemented a systematic evaluation of the prediction accuracy of three machine learning methods. We showed that machine learning in general worked well for the prediction of CSS, where Elastic Net showed the best performance according to our cross-validation setting. We found that the drug target information for the compounds as well as their chemical fingerprints are highly predictive of the CSS values, with an accuracy comparable to the level of experimental replicates. Therefore, the rationale of considering a drug combination as a function of their target and fingerprint profiles can be justified. This would also allow the augmentation of single-drug screening and drug combination screening data together to train a machine learning model, as many drugs are multi-targeted which are equivalent to a drug combination with the same target profile. In our study, we utilized the SEA method to predict new targets of compounds, by applying a combination of thresholds of Z-score, Tanimoto coefficient and p-value suggested by the authors of SEA [
A truly promising drug combinations shall reach sufficient therapeutic efficacy via a strong sensitivity and synergy. Therefore, both these aspects should be evaluated for the prioritization of most potential drug combination hits. While there have been multiple synergy scoring methods that can be applied to the full matrix design, they do not always produce consistent results. The truly synergistic and antagonistic drug combinations may therefore be determined by achieving the consensus across the different scoring methods [
The proposed cross design aimed for alleviating the limitation of the conventional dose-response full matrix design which usually requires a large amount of cancer cells that may not be amenable especially from patient-derived samples. Empowering the cross design with the CSS and S scoring methods, we are foreseeing a lower technical barrier to carry out large-scale drug combination studies with minimal cellular materials. Although we showed the proof-of-concept using the drug combination data involving cancer cell lines, the cross design coupled with the CSS and S scoring methods can be readily applied for ex_vivo drug screening, where the amount of patient-derived materials can be extremely limited and technically difficult to obtain due to culture constraints [
(XLSX)
(XLSX)
(XLSX)
(XLSX)
(TIF)
Drug classes are determined by their primary targets.
(TIF)
The color of the data points represents the tissue of origin.
(TIF)
The line plot of the minimal and maximal values for the CSS replicates combined with CSS values over the standard deviation of the CSS replicates.
(TIF)
The Pearson correlations between CSS1 and CSS2 for the drug combinations that involve a given drug were shown on top of each bar.
(TIF)
The threshold of 10 achieved a correlation (0.93) close to the midpoint (0.91) of the range.
(TIF)
LNCAP is the only line which has a negative variable importance for TOP1MT.
(TIF)
True synergistic combinations were determined as the minimal of the four scores higher than 5 while true negative combinations have the maximal of the four scores lower than 5, resulting in 20.7% and 1.2% of the total drug combinations respectively.
(TIF)
(TIF)
All the drug combinations shown here have an S synergy score higher than 5, while their CSS score lower than 10. The cell lines are colored based on their tissues of origin. Bar plot in the inset shows the mean % inhibition that can be achieved by these drug combinations (denoted as false positive group), as compared to the top 100 drug combinations ranked by CSS (denoted as true positive group).
(TIF)
We thank the authors of the O’Neil study for making the drug combination data fully accessible.