Dispensing and dilution processes may profoundly influence estimates of biological activity of compounds. Published data show Ephrin type-B receptor 4 IC50 values obtained via tip-based serial dilution and dispensing versus acoustic dispensing with direct dilution differ by orders of magnitude with no correlation or ranking of datasets. We generated computational 3D pharmacophores based on data derived by both acoustic and tip-based transfer. The computed pharmacophores differ significantly depending upon dispensing and dilution methods. The acoustic dispensing-derived pharmacophore correctly identified active compounds in a subsequent test set where the tip-based method failed. Data from acoustic dispensing generates a pharmacophore containing two hydrophobic features, one hydrogen bond donor and one hydrogen bond acceptor. This is consistent with X-ray crystallography studies of ligand-protein interactions and automatically generated pharmacophores derived from this structural data. In contrast, the tip-based data suggest a pharmacophore with two hydrogen bond acceptors, one hydrogen bond donor and no hydrophobic features. This pharmacophore is inconsistent with the X-ray crystallographic studies and automatically generated pharmacophores. In short, traditional dispensing processes are another important source of error in high-throughput screening that impacts computational and statistical analyses. These findings have far-reaching implications in biological research.
Citation: Ekins S, Olechno J, Williams AJ (2013) Dispensing Processes Impact Apparent Biological Activity as Determined by Computational and Statistical Analyses. PLoS ONE 8(5): e62325. doi:10.1371/journal.pone.0062325
Editor: Alexandre G. de Brevern, UMR-S665, INSERM, Université Paris Diderot, INTS, France
Received: January 11, 2013; Accepted: March 20, 2013; Published: May 1, 2013
Copyright: © 2013 Ekins et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors have no support or funding to report.
Competing interests: JO is an employee of Labcyte Inc. while SE is an employee of Collaborations in Chemistry and AJW is an employee of the Royal Society of Chemistry. Neither of these latter two authors have any competing interests relevant to this manuscript and were not funded by Labcyte Inc. This does not alter the authors' adherence to all the PLOS ONE policies on sharing data and materials.
There have been many studies which have evaluated aspects of biological assays and the tools involved which could result in errors and erroneous data. Processes like tip-based and acoustic dispensing have a profound influence on estimates of compound activity. Several independent studies of high-throughput screening (HTS) show that the two techniques generate conflicting results , , , , . The difference in results may mean missing important lead compounds, following dead-ends and developing inappropriate compounds for activity optimization.
Previous research has impugned tip-based techniques because they can generate errors due to leachates from the plastic that may profoundly affect biological assays , , , , , . Broadly speaking, the IC50 values derived using tip-based serial dilution and dispensing tend to be greater (i.e., show lower potency) than IC50 values derived using acoustic dispensing. Some compounds appeared hundreds of times more active with the acoustic process , , , . We now address how these errors may affect computational models and propagate poor data in both proprietary and public databases, the result of which is likely to misdirect drug design.
While we are limited by the number of compounds available with data in tip-based and acoustic dispensing, this study suggests a significant impact on drug design, especially when coupled with other reports of poorly correlating IC50 results in which larger number of molecules are used but the molecular structures are not provided for computational analysis , , . We now show how dispensing processes impact computational and statistical results.
Materials and Methods
This paper is based on the published comparisons of IC50 values determined by AstraZeneca scientists ,  (Fig 1) for inhibition against the Ephrin type-B receptor 4 (EphB4), a membrane-bound receptor tyrosine kinase that binds to ephrin-B2 ligands bound to the surfaces of other cells to induce angiogenic events. Unique to these publications, the researchers provided structures of the inhibitors as well as IC50 values using both serial dilution facilitated by tip-based dispensing (Genesis, Tecan Ltd, Weymouth, United Kingdom) and direct dilution ,  with an acoustic dispensing system (Echo550, Labcyte Inc., Sunnyvale, CA). They found that the IC50 values obtained with acoustic transfers suggested that the compounds were 1.5 to 276.5 times more active than when tip-based techniques were used , .
Statistical and Computational Modeling
IC50 values (Fig 1, Table S1) derived by each method were initially used to correlate 9 molecular descriptors (molecular weight (MW), calculated logP (LogP), number of hydrogen bond donors (HBD), number of hydrogen bond acceptors (HBA), molar refractivity (MR), polar surface area (PSA), LogD, pH 7, charge at pH 7 and isoelectric point (pI, Table S1 and Table 1), all calculated with MarvinSketch 5.9.3, (ChemAxon, Budapest, Hungary)  using SAS JMP (v8.0.1, SAS, Cary, NC). Statistical significance was determined by ANOVA.
A 3D pharmacophore was developed with IC50 values as the indicator of biological activity. In the 3D pharmacophore modeling approach using Discovery Studio (Accelrys version 2.5.5. San Diego, CA, described previously ), ten hypotheses were generated using hydrophobic, HBA, HBD, and the positive and negative ionizable features, and the CAESAR algorithm  was applied to the molecular data set (maximum of 255 conformations per molecule and maximum energy of 20 kcal/mol) to generate conformers. The pharmcophore hypothesis with the lowest energy cost was selected for further analysis as this model possessed features representative of all the hypotheses. The quality of the structure-activity correlation between the predicted and observed activity values was estimated using the calculated correlation coefficient (r).
After the two different pharmacophores were developed based on the original 14 compounds, we found an additional patent from AstraZeneca that provided the IC50 values for an additional 12 compounds. None of these compounds were evaluated using both liquid handling techniques. Ten of the compounds had data based upon tip-based dispensing with serial dilution and two had data based upon acoustic dispensing and direct dilution (Table S1).
Pharmacophores for the tyrosine kinase EphB4 were generated from crystal structures in the protein data bank PDB. Pharmacophores were constructed using the receptor-ligand pharmacophore generation protocol in Discovery Studio version 3.5.5 (Accelrys, San Diego, CA) with minimum features (3) and maximum features (6) as are described elsewhere .
The correlation between the 14 Ephrin type-B receptor 4 log IC50 values obtained via tip-based serial dilution and dispensing versus acoustic dispensing with direct dilution is poor (R2 = 0.25, Fig S1). The red diagonal line indicates where the values would be if the two methods were equivalent. Note that the IC50 values for all 14 compounds were lower (more potent) when acoustic transfer was used. Upon statistical analysis of the 14 IC50 values for the two dispensing techniques (Fig 1, Table S1), calculated LogP showed a low but statistically significant correlation with log IC50 data for acoustic dispensing (r2 = 0.34, p<0.05, N = 14, Table 1). Acoustic dispensing IC50 data did not demonstrate a statistically significant ranking of tip-based dispensing data based on Spearman's rho analysis (data not shown). This would suggest that there is no statistically significant correlation or ranking between these two measures. That is, the data generated from the two techniques would lead researchers in completely different directions.
Computational Pharmacophore Modeling
The pharmacophores (Fig 2A, 2B) derived from data in Table 1 illustrates how the two techniques differ qualitatively. The correlation (r) between predicted and observed IC50 values for the pharmacophore derived via acoustic processes was higher than that for the pharmacophore derived from the tip-based processes (Table 2, Results S1). The pharmacophore derived from data generated via the tip-based process also failed, as discussed below, to identify hydrophobic features that were identified in X-ray crystallography 13,14,15,16,17. These hydrophobic features were only evident in the pharmacophore developed through the acoustic transfer process.
(A) EphB4 pharmacophore derived using acoustic dispensing with 14 compounds. The most active compound (5, S1) is shown mapped. (B) EphB4 pharmacophore derived using tip-based dispensing with 14 compounds. The most active compound (w3, S1) is shown mapped. The pharmacophore features are hydrophobic (cyan), hydrogen bond donor (purple) and hydrogen bond acceptor (green).
The pharmacophores specific to the tip-based and acoustic-based processes were used to predict affinity to EphB4 (as measured by IC50 values) for these additional 12 compounds. We used the data to create a 3D multiple conformer database. This database was searched by the two pharmacophores in order to test whether they could discriminate between those with high and low affinity (IC50 values). That is, the two pharmacophores developed on the comparative data of 14 compounds were used to test an additional 12 compounds to see whether either of the two pharmacophores developed could predict which compounds in the second set of 12 were most active.
Using the pharmacophore derived via acoustic processes, the two compounds analyzed were predicted to be potent inhibitors. Both of these were compounds that were transferred acoustically. The IC50 values actually determined placed these two compounds in the top 3 active compounds (Table S2) and correctly predicted their ranking. The tip-based pharmacophore failed to rank the retrieved compounds correctly (Table S3). This suggests that the pharmacophore developed with tip-based transfers is not useful in predicting the potency of subsequently developed molecules, while the pharmacophore developed with the acoustic procedure is preferred at predicting the activity of new compounds.
When the physical properties of the additional 12 compounds  were used in the statistical analysis the calculated LogP and logD showed low but statistically significant correlations with tip-based dispensing (r2 = 0.39 p<0.05 and 0.24 p<0.05, respectively, Table 1). This suggests that more data is required in order to observe the importance of hydrophobicity as correlating with tip-based dispensing IC50, as previously had been seen with just 14 compounds when using acoustic dispensing but would require significantly more compounds and analyses to be recognized with tip-based dispensing. It is also noted that the hydrophobic features predicted with data generated from acoustic transfers are localized in specific areas and not just a generic increase in hydrophobicity (as measured by LogP, which one might logically expect to lead to greater non-specific binding).
Receptor-ligand pharmacophores were created in 8 out of 10 cases and all consisted of hydrophobic and hydrogen bonding features (Fig 3).
The pharmacophores derived for the tyrosine kinase EphB4 are dramatically different based upon the process used to set up the dose-response experiments. The pharmacophore derived from the acoustic dispensing data suggests the importance of specific regions of hydrophobicity as well as hydrogen bonding features. The pharmacophore from the tip-derived data suggests only hydrogen bonding features as leading to binding, with no hydrophobic interactions.
In order to further understand the impact of these models, a series of another 4 papers published by AstraZeneca describing structure-based design of tyrosine kinase EphB4 inhibitors were reviewed , , , . These show inhibitors in which part of the molecules are buried in a hydrophobic selectivity pocket beyond Thr693, which appears important for potency. Also indicated were interactions between the inhibitor and Met696 of EphB4 via a hydrogen bond, or acceptor-donor pair. The hydrophobic binding pocket, shown as important for potency by these structure-based studies, was indicated by the acoustic dispensing method in the previous experiments. Interestingly the indazole ring (Fig 2A) has a hydrogen bond acceptor feature which was also noted in the crystal structure of similar compounds , suggesting that the acoustic dispensing derived pharmacophore is more representative of the crystal structure data. It should be noted these pharmacophores were derived solely from in vitro data of the original articles and not using the crystal structures of the latter work.
We have also used an automated receptor-ligand pharmacophore generation method  with the 10 current crystal structures, in order to compare with our initial in vitro data ligand pharmacophores further. Receptor-ligand pharmacophores could be created in 8 out of 10 cases and all consisted of hydrophobic and hydrogen bonding features (Fig 3). No pharmacophore was identical to those generated from in vitro data alone, however none consisted of solely hydrogen bonding features as in the case of the pharmacophore generated from data using tip-based dispensing (Fig 2B). It is clear that the reported EphB4 kinase inhibitor-crystal structure interactions , , ,  most closely reflects the pharmacophore derived with the acoustic dispensing data based on independent ligand-dependent pharmacophores, receptor-ligand pharmacophores and statistical approaches taken in this study.
In this study acoustically-derived IC50 values were 1.5 to 276.5-fold lower than for tip-based dispensing , . Our analyses suggest for the first time that not only are the IC50 values unequal but that the data generated by either liquid handling process neither correlates nor, indeed, ranks each other. While the dataset is small it is representative of larger comparisons between dispensing methods that show limited, if any, correlation between IC50 results obtained via acoustic transfer and those obtained by tip-based methods , , ,  (Table 3). No previous publication has analyzed or compared such data (based on tip-based and acoustic dispensing) using computational or statistical approaches. This analysis is only possible in this study because there is data for both dispensing approaches for the compounds in the patents from AstraZeneca that includes molecule structures. We have taken advantage of this small but valuable dataset to perform the analyses described. Unfortunately it is unlikely that a major pharmaceutical company will release 100's or 1000's of compounds with molecule structures and data using different dispensing methods to enable a large scale comparison, simply because it would require exposing confidential structures. To date there are only scatter plots on posters and in papers as we have referenced (Table 3), and critically, none of these groups have reported the effect of molecular properties on these differences between dispensing methods.
We believe our observations are novel for three reasons. First, no previous publication has shown how data quality can be impacted by dispensing and how this in turn affects computational models and downstream decision making. Second, there has been no comparison of pharmacophores generated from acoustic dispensing and tip-based dispensing. Third, there has been no previous comparison of pharmacophores generated from in vitro data with pharmacophores automatically generated from X-ray crystal conformations of inhibitors bound to receptors. We believe our insights to be highly novel and use different technologies to analyze data that cuts across different fields.
In the absence of structural data, pharmacophores and other computational and statistical models are used to guide medicinal chemistry . Our findings suggest acoustic dispensing methods could improve HTS results and avoid the development of misleading computational models and statistical relationships. While we recently described the errors reported across various internet databases used for biomedical research , , there has been no analysis of the influence of dispensing processes on such data. It would appear that tip-based dispensing is producing erroneous data based on our and other (Table 3) analyses which we see here reflected in the models and initial lack of correlations with molecular properties. We therefore request that public databases annotate this meta-data alongside biological data points, to create larger datasets for eventually comparing different computational methods in future . This may also assist in the generation of better computational and statistical models from published data. Scientists should be made aware of such dispensing issues and it is therefore important that such evaluations (however limited in molecule numbers) are made accessible for them to decide what technologies to use for dispensing. Such efforts should also encourage pharmaceutical companies to make their data available but we are under no illusions that this will only happen at their convenience e.g. when patents have issued.
A graph of the log IC50 values for tip-based serial dilution and dispensing versus acoustic dispensing with direct dilution shows a poor correlation between techniques (R2 = 0.246).
Showing pharmacophore model information for acoustic-based liquid handling with direct dilution and tip-based liquid handling with serial dilution.
Molecule structures, IC50 data and descriptors.
Test set data for searching with ‘acoustic dispensing’ pharmacophore.
Test set data for searching with ‘tip-based dispensing’ pharmacophore.
Conceived and designed the experiments: SE JO. Performed the experiments: SE. Analyzed the data: SE JO. Contributed reagents/materials/analysis tools: SE JO AJW. Wrote the paper: SE JO AJW.
- 1. Spicer T, Fitzgerald Y, Burford N, Matson S, Chatterjee M, et al.. (2005) Pharmacological evaluation of different compound dilution and transfer paradigms on an enzyme assay in low volume 384-well format. Drug Discovery Technology. Boston.
- 2. Wingfield J, Jones D, Clark R, Simpson P (2008) A model for improving efficiency through centralization. American Drug Discovery 3: 24–30.
- 3. Grant RJ, Roberts K, Pointon C, Hodgson C, Womersley L, et al. (2009) Achieving accurate compound concentration in cell-based screening: validation of acoustic droplet ejection technology. J Biomol Screen 14: 452–459.
- 4. Matson SL, Chatterjee M, Stock DA, Leet JE, Dumas EA, et al. (2009) Best practices in compound management for preserving compound integrity and accurately providing samples for assays. J Biomol Screen 14: 476–484.
- 5. Harris D, Olechno J, Datwani S, Ellson R (2010) Gradient, contact-free volume transfers minimize compound loss in dose-response experiments. J Biomol Screen 15: 86–94.
- 6. McDonald GR, Hudson AL, Dunn SM, You H, Baker GB, et al. (2008) Bioactive contaminants leach from disposable laboratory plasticware. Science 322: 917.
- 7. Belaiche C, Holt A, Saada A (2009) Nonylphenol ethoxylate plastic additives inhibit mitochondrial respiratory chain complex I. Clin Chem. 55: 1883–1884.
- 8. Reuhl TO, Amador M, Dani JA (1990) Tissue culture tube contaminant inhibits excitatory synaptic channels. Brain Res Bull 25: 433–435.
- 9. Papke RL, Craig AG, Heinemann SF (1994) Inhibition of nicotinic acetylcholine receptors by bis (2,2,6,6-tetramethyl-4-piperidinyl) sebacate (Tinuvin 770), an additive to medical plastics. J Pharmacol Exp Ther 268: 718–726.
- 10. Watson J, Greenough EB, Leet JE, Ford MJ, Drexler DM, et al. (2009) Extraction, identification, and functional characterization of a bioactive substance from automated compound-handling plastic tips. J Biomol Screen 14: 566–572.
- 11. Hubalek F, Binda C, Li M, Mattevi A, Edmondson DE (2003) Polystyrene microbridges used in sitting-drop crystallization release 1,4-diphenyl-2-butene, a novel inhibitor of human MAO B. Acta Crystallogr D Biol Crystallogr. 59: 1874–1876.
- 12. Wingfield J (2012) Impact of Acoustic Dispensing on Data Quality in HTS and Hit Confirmation. Drug Discovery 2012. Manchester, UK.
- 13. Bardelle C, Barlaam B, Brooks N, Coleman T, Cross D, et al. (2010) Inhibitors of the tyrosine kinase EphB4. Part 3: identification of non-benzodioxole-based kinase inhibitors. Bioorg Med Chem Lett 20: 6242–6245.
- 14. Bardelle C, Coleman T, Cross D, Davenport S, Kettle JG, et al. (2008) Inhibitors of the tyrosine kinase EphB4. Part 2: structure-based discovery and optimisation of 3,5-bis substituted anilinopyrimidines. Bioorg Med Chem Lett 18: 5717–5721.
- 15. Bardelle C, Cross D, Davenport S, Kettle JG, Ko EJ, et al. (2008) Inhibitors of the tyrosine kinase EphB4. Part 1: Structure-based design and optimization of a series of 2,4-bis-anilinopyrimidines. Bioorg Med Chem Lett 18: 2776–2780.
- 16. Barlaam B, Ducray R, Lambert-van der Brempt C, Ple P, Bardelle C, et al. (2011) Inhibitors of the tyrosine kinase EphB4. Part 4: Discovery and optimization of a benzylic alcohol series. Bioorg Med Chem Lett 21: 2207–2211.
- 17. Barlaam BC, Ducray R (2008) N'-(Phenyl)-N-(Morpholin-4-yl-pyridin-2-yl)-pyrimidine-2,4-diamine derivatives as EPHB4 kinase inhibitors for the treatment of proliferative conditions. In: WIPO, editor. Seden: AstraZeneca AB.
- 18. Meslamani J, Li J, Sutter J, Stevens A, Bertrand HO, et al.. (2012) Protein-Ligand-Based Pharmacophores: Generation and Utility Assessment in Computational Ligand Profiling. J Chem Inf Model.
- 19. Barlaam BC, Ducray R (2009) 2,4-Diamino-pyrimidine derivatives. In: WIPO, editor.
- 20. Barlaam BC, Ducray R, Kettle JG (2010) Pyrimidine derivatives for inhibiting Eph receptors. In: Patent US, editor: AstraZeneca AB, Sodertalje.
- 21. Gilchrist MA 2nd, Cacace A, Harden DG (2008) Characterization of the 5-HT2b receptor in evaluation of aequorin detection of calcium mobilization for miniaturized GPCR high-throughput screening. J Biomol Screen 13: 486–493.
- 22. Ekins S, Mestres J, Testa B (2007) In silico pharmacology for drug discovery: methods for virtual ligand screening and profiling. Br J Pharmacol 152: 9–20.
- 23. Williams AJ, Ekins S, Tkachenko V (2012) Towards a Gold Standard: Regarding Quality in Public Domain Chemistry Databases and Approaches to Improving the Situation. Drug Disc Today 17: 685–701.
- 24. Williams AJ, Ekins S (2011) A quality alert and call for improved curation of public chemistry databases. Drug Disc Today 16: 747–750.
- 25. Ekins S, Williams AJ (2010) Precompetitive Preclinical ADME/Tox Data: Set It Free on The Web to Facilitate Computational Model Building to Assist Drug Development. Lab on a Chip 10: 13–22.
- 26. Comley J (2007) Drug Discovery World Spring: 36–50.
- 27. Turmel M, Itkin Z, Liu D, Nie D (2010) J Labor Automat. 15: 297–305.
- 28. MarvinSketch Website. Available: www.chemaxon.com/marvin/sketch/index.php. Accessed 2013 Mar 26.
- 29. Ekins S, Crumb WJ, Sarazan RD, Wikel JH, Wrighton SA (2002) Three dimensional quantitative structure activity relationship for the inhibition of the hERG (human ether-a-gogo related gene) potassium channel. J Pharmacol Exp Thera 301: 427–434.
- 30. Li J, Ehlers T, Sutter J, Varma-O'brien S, Kirchmair J (2007) CAESAR: a new conformer generation algorithm based on recursive buildup and local rotational symmetry consideration. J Chem Inf Model 47: 1923–1932.