Table 1.
Protein targets used in this study.
These targets correspond to non-redundant protein-protein interaction sites for which a crystal structure or an NMR structure has been solved in complex with a small-molecule inhibitor. For this study, three protein targets have been removed from our previously reported set [29] (see Methods).
Fig 1.
Overview of the virtual screening benchmark.
A target protein is provided in complex with a known small-molecule inhibitor acting at this protein interaction site. A collection of 2500 diverse “decoy” ligands have also been docked to this site. The benchmark entails scoring each of the 2501 complexes, and determining the rank of the native ligand relative to the decoy compounds. This experiment carried out for each of 18 non-redundant protein targets (Table 1).
Fig 2.
Baseline performance of the Rosetta energy function prior to recent enhancements.
Each plot compares the performance of two different scoring functions for identifying the active compounds in our virtual screening benchmark. Each of the 18 protein targets corresponds to a single point (green dots); the rank of the active compound (relative to 2500 diverse “decoy” compounds) by each scoring function is indicated. The orange dotted line indicates a ranking of 25, corresponding to the top 1% of the decoy set. (A) FRED’s Chemgauss4 energy function outperforms Rosetta’s original energy function intended for protein-only systems, score12, but not at a statistically significant threshold (p = 0.085). (B) The variant of the score12 energy function that was developed specifically for modeling protein-ligand interactions, score12_ligand, offers improved performance over score12 (p = 0.001). (C) FRED’s Chemgauss4 energy function performs at a similar level as score12_ligand (p = 0.239). All p-values are calculated by applying the Wilcoxon Signed-Rank test to the logs of the ranks (see Methods).
Fig 3.
Recent enhancements to the functional form of the Rosetta energy function enable improved performance.
(A) Rosetta’s “Talaris” energy function [19, 20, 30] includes updates to the functional form of the hydrogen bond term and of the electrostatic term. These changes lead to improved performance relative to score12, at a statistically significant level (p = 0.013). (B) Replacing Rosetta’s default model of polar solvation, EEF1, with a newly developed model, pwSHO [31], leads to further improvement at a statistically significant level (p = 0.0004).
Fig 4.
New weights for identifying small-molecule inhibitors of protein-protein interactions.
Using the Talaris+pwSHO functional form of the energy function, we identified new weights to optimize Rosetta’s ability to distinguish true inhibitors from decoy compounds. The results presented here were obtained by leave-one-out cross validation of the weights over the benchmark set of 18 non-redundant protein targets. (A) Reweighting of the Talaris+pwSHO energy function leads to improved performance, though not at a statistically significant level (p = 0.078). (B) This new reweighted energy function provides comparable performance as FRED’s Chemgauss4 energy function (p = 0.221). (C) The new reweighted energy function slightly outperforms score12_ligand, though again not to a degree attaining statistical significance (p = 0.123).
Table 2.
Summary of the comparisons made in this study.
Among the 18 protein targets used as testcases, those for which the rankings by both scoring functions were within 10% of one another were considered to be “ties”. The reported p-values were calculated by applying the Wilcoxon Signed-Rank test to the difference in the log of the rankings, over all testcases. This (non-parametric) statistical test has the advantage that the degree to which a given method “wins” each testcase—and not just the number of “wins”—is taken into account.