Figure 1.
Construction of gates with constrained fuzzy logic (cFL).
When node C depends only on node A, a normalized Hill function is used to calculate value of node C, ‘c’, given value of node A, ‘a’, where n is Hill coefficient and k is the sensitivity parameter specifying the EC50 for each gate. Several representative normalized Hill functions are shown for activating (a) and inhibiting (b) cFL gates. When C has more than one input (A and B, in this case), either an AND (c) or OR (d) gate must be used to model the interaction. In the case of the AND gate, the minimum possible value of c calculated from the transfer functions is used as the output node value. One possible response surface for levels of C given different levels of A and B with two transfer functions is demonstrated (c). For evaluation of an OR gate, the maximum value of c is used as the output node value, with the corresponding response surface (d).
Figure 2.
CellNOpt–cFL workflow and application to toy model.
Right side: Workflow (Boxes A through G and Steps 1–6). The methodology requires a dataset that describes some species in the prior knowledge network (PKN; Box A). Based on the experimental design of the dataset (Box B), the map is compressed to contain only nodes measured (blue nodes), perturbed (green stimulated nodes and orange inhibited nodes), or necessary to maintain logical consistency between nodes (Step 1). The resultant compressed network (Box C) is then expanded to contain multiple possible logic descriptions of gates connecting more than one input node to a single output node (Step 2). The resultant expanded network (Box D) is trained to the data values (Box E) using several independent runs of a discrete genetic algorithm to minimize MSE (Step 3). Each independent run results in an unprocessed cFL model represented with a grey triangle. This results in a family of unprocessed cFL models (Box F). The result of each independent optimization run is now represented with a different colored triangle. Each individual unprocessed model is reduced with several reduction thresholds (Step 4), resulting in several reduced models (different triangles shadings). The parameters of each reduced model are then refined (Step 5), resulting in reduced-refined models (triangles outlined in black). Finally, one model is chosen to represent each original unprocessed model using a selection threshold (Step 6), resulting in a family of filtered models (Box G). Left side: Application to a toy model (panels a to e). A PKN was hypothesized from the Ingenuity Systems database (www.ingenuity.com) (a.i.) and compared to an in silico dataset generated by a simulation of a cFL model with known topology and parameters (a.ii.). The PKN contains 15 molecular species represented as nodes that are believed to positively (arrows) or negatively (blunt arrows) affect others species. These intermediate nodes summarize the possible paths between experimentally stimulated ligands (green) and measured (blue) or inhibited (orange) species. The model was compressed (b) as described in [29] and then expanded (c) to contain all possible two-input AND gates. The expanded network was trained to the in silico dataset with twenty independent runs of the discrete genetic algorithm. The topologies of the resultant models (d) were identical except in the case of the gate describing activation of MEK, with sixteen models modeling this interaction with an activating gate (brown, dashed gate) and four models using an AND-NOT gate (green, dashed gate). The TNFα → JNK cFL gate was removed from all unprocessed models, reflecting that this interaction was inconsistent with the in silico data. The reduction process (Figure 3) showed that the AND-NOT gate could be described more simply without significantly affecting the MSE, resulting in a family of filtered models (e). We have labeled each gate with the sensitivity of the gate (defined in Materials & Methods), where sensitivity is scaled between zero and one and a higher sensitivity indicates that the output node is more active at lower input node values. All maps and the graphs of cFL models were generated by a CellNOpt routine using the graphviz visualization engine (www.graphviz.org) followed by manual annotation in Adobe Illustrator.
Figure 3.
Reduction of trained cFL models.
The unprocessed models resulting from twenty independent runs of the discrete genetic algorithm to train the expanded network to an in silico dataset were reduced using several reductions thresholds and subsequently refined. The behavior of three representative models is shown (a). To develop a criterion for our model selection, we note that each individual model exhibits a drastic increase in refined MSE when reduced at some reduction threshold. For our toy model, the MSEs of some reduced-refined models increase significantly (ΔMSE of 7.7x10−3) at a reduction threshold of greater than 5×10−3 (a., magenta line), whereas the MSEs of others only increase at a reduction threshold greater than 7×10−3 (a., green line). This increase in MSE of 7.7×10−3 is deemed significant because it corresponds to the models no longer fitting the in silico data of Akt and JNK under TGFα stimulation (the remaining data are still well fit). For each unprocessed model, we refer to the reduction threshold above which a significant increase in MSE is observed as the ‘filter point’ of the model. Each individual model has a filter point that is determined based on the amount that the reduced-refined model's MSE is allowed to increase. We term this allowable increase in MSE the ‘selection threshold’. For example, one model of our toy example (black line) could be described as having a filter point of 1×10−3 or 5×10−3, depending on the amount of increase in MSE allowed by the selection threshold. To choose a selection threshold, we compare the average increase in final MSE to the average decrease in the number of parameters in the resultant filtered family of models (b) and note that, at a selection threshold of 7.7×10−3, the average MSE increases while at a selection threshold of 5×10−4, average number of parameters decreases. Thus, a selection threshold of 5×10−4 to 7.6×10−3 results in the models at the “filter points” noted in (a).
Figure 4.
Initial analysis of cFL models trained to HepG2 dataset.
(a) Experimental design of a dataset describing the measured signaling response of the HepG2 cell line to six ligand stimulations in the presence or absence of inhibition of seven species. CellNOpt-cFL was used to train the PKNs (Figure S2) to this dataet. (b) The fraction of edges indicated were randomly removed from (solid line) or added to (dashed line) PKN1i to result in at least 90 altered PKNs, which were subsequently trained to the HepG2 data. The average MSEs of the altered PKNs indicates that removal of edges reduced the ability of the trained models to fit the data (solid line). Because CellNOpt-cFL does not add links to the model, this result is as expected. The addition of edges to the PKN did not reduce the ability of the trained models to fit the data (dashed line) since edges that were inconsistent with the data could be removed during the training process (Figure S6). (c) Results of ten-fold cross-validation in which the data was randomly divided into ten subsets and the optimization procedure performed to obtain a family of at least 57 models from training data comprising nine of the ten subsets; the remaining subset was considered a test set. We thus obtained ten families of trained models, one family from the use of each subset as a test set. The fit of these families of models to their respective training and test sets was then plotted as a function of the selection threshold. As expected, on average the ability of the trained models to fit the test sets was slightly worse than, but comparable to, the ability to fit the training sets, suggesting that the models were predictive. The difference between MSEs of the test versus training sets did not change as a function of the selection threshold, suggesting that the models were not overfit, even at very low selection thresholds. (d) A comparison of the average final MSE with the average final number of parameters was used to determine a range of selection thresholds (1×10−3 – 1×10−2) where the family of models has a slightly lower average number of parameters without greatly increasing the MSE.
Table 1.
Prior knowledge networks trained to HepG2 dataset.
Figure 5.
Structure of family of cFL models resulting from training PKN1i to HepG2 dataset.
Topologies of the family of filtered cFL models trained to the HepG2 dataset. Unprocessed cFL models can be found in Figure S6 and fit of the filtered models to the data in Figure S7. Nodes represent proteins that were either ligand stimulations (green), inhibited (orange), measured by a phospho-specific bead-based antibody assay (blue), or could not be removed without introducing potential logical inconsistency (white). The grey/black intensity scale of the gates corresponds to the proportion of individual models within the family that include that gate. Thus, links colored black were present in all models whereas links colored grey were present in a fraction of the models. Where visually feasible, cFL gates are labeled with a numerical value that corresponds to a quantitative sensitivity of the input-output relationship. Sensitivity is calculated as described in the Materials & Methods. The larger this value, the lower the level of the input nodes' activity required for generating significant output node activity (i.e. a gate with a high sensitivity indicates that the output node is sensitive to a low value of its input node). The uncertainties in these values arise from the various best-fit EC50 for each individual model. The graph of the cFL models was generated by a CellNOpt routine using the graphviz visualization engine (www.graphviz.org) followed by manual annotation in Adobe Illustrator.
Table 2.
Biological hypotheses about signaling network operation suggested by gates removed during CellNOpt-cFL analysis.
Figure 6.
Validation of cFL crosstalk predictions.
(a) Analysis of systematic error as well as the topologies of the family of trained cFL models (Figure 5) indicated that c-Jun was partially activated after TGFα stimulation. Models with crosstalk from Ras or PI3K to Map3K1 predicted that JNK was partially activated under these experimental conditions even though it was not partially activated in the dataset. We tested whether JNK was actually partially activated under these conditions by stimulating HepG2 cells with TGFα and measuring levels of phosphorylated JNK and c-Jun by a bead-based antibody assay after 30 minutes. Fold increase in measured phosphorylation over un-stimulated control for c-Jun (black) and JNK (red) is shown. Where available, biological replicates are indicated with filled circles. Solid lines indicate the averages of the replicates. This experiment indicates that JNK was partially phosphorylated under TGFα stimulation and the cFL models with crosstalk from Ras or PI3K to MAP3K1 were correct. (b) CFL analysis of the topologies and fit of the HepG2 training dataset to several PKNs suggested that IL6 activated downstream nodes through the Ras/MEK pathway (Table 3). To test this prediction, a validation dataset was examined [36]. This validation dataset showed that the activation of nodes other than STAT3 that responded robustly to IL6 stimulation was ablated by pretreatment with a small molecule MEK inhibitor but not other inhibitors, demonstrating that the Ras/Raf/MEK pathway mediates this crosstalk.
Table 3.
Results of cFL training of various prior knowledge networks for the investigation of IL6 crosstalk.
Figure 7.
Transfer functions predicted by trained cFL models.
The output value of the CREB node was predicted by computationally simulating each individual model in the family of cFL models with 441 combinations of p38 and MEK1/2. Three-dimensional plots were generated in MATLAB showing the average prediction (opaque surface) as well as the average prediction plus or minus the standard deviation of the predicted value (semi-transparent surfaces). The training data (black circles) and validation data (green diamonds) are also plotted. The 3-D plots have been rotated to highlight the influence of either (a) p38 or (b) MEK1/2. The predicted transfer functions agree with the validation data reasonably well except for the overestimation of CREB activation for conditions with TGFα stimulation as one of the ligands.
Figure 8.
Accuracy vs. precision of cross-validation experiments.
(a) Model predictions can be assessed based on both how well the family of models agree on a prediction (precision) as well as their accuracy. If a prediction is imprecise (i.e. the models do not agree), the models are not constrained to any single prediction. Thus, precision can be used to discredit predictions. Predictions can be both precise and accurate (green field), imprecise but accurate on average (yellow field), imprecise and inaccurate (blue field), or precise but inaccurate (orange field). Predictions that are precise and accurate (green field) are preferred. (b) The importance of considering the precision of a prediction amongst a family of models was demonstrated by a cross-validation study in which a signal under a single ligand stimulation condition in the presence or absence of any inhibitor was removed from the training data set. The mean coefficient of variance (CV) as a function of the error in the prediction (MSE) is plotted for all tests. One prediction was highly inaccurate. However, it was also imprecise (blue field), whereas no predictions were precise and inaccurate (orange field), demonstrating that taking the precision of a prediction into account can help to discredit inaccurate predictions. (c) The grey-boxed subset of (b) highlights the test sets that were precisely and accurately predicted by the family of cFL models.
Figure 9.
Trained cFL models linking ligand cues, phospho-protein signals, and cytokine release phenotypic responses.
A dataset describing release of five cytokines after three hours under conditions identical to those under which protein phosphorylation was measured was combined with the phospho-protein dataset. PKN2D was further extended to include links from protein signals that occupied unique principle component space (Text S1) to nodes of cytokine release after three hours. Training this network to the data indicated that the growth and survival pathways were not needed to describe cytokine release. Thus, the PKN was revised to link only Stat3, NFκB, c-Jun, and Hsp27 to the cytokine release nodes, and this PKN was trained to the experimental dataset of both cytokine release and protein phosphorylation. In contrast to the cFL models describing only signaling activation, we found that the family of 141 cFL models fit the cytokine response data with a wider distribution of MSE. The resultant sub-family of seven filtered cFL models that fit the data with a MSE less than the average plus one standard deviation of the family MSE is shown. Nodes represent proteins that were either ligand stimulations (green), inhibited (orange), phosphorylation states measured (blue), cytokine secretion measured (yellow) or could not be removed without introducing potential logical inconsistency (white). The grey/black intensity scale of the gates corresponds to the proportion of individual models within the family that include that gate. The graph of the cFL models was generated by a CellNOpt routine using the graphviz visualization engine (www.graphviz.org) followed by manual annotation in Adobe Illustrator.