Estimating single-mutant fitness values

As an initial study objective, we sought to assess the accuracy to which the single-mutant fitness vector could be estimated directly from the double-mutant fitness matrix . In the quantitative genetic interaction dataset of St Onge et al. [10], which was used in the following results, w is a 26-dimensional column vector and W is a 26×26-dimensional symmetric matrix. Under the multiplicative null model, Eq. 1 takes the form:(2)where is the tensor product (or outer-product) of the vector w with itself, and E is the n×n-matrix comprising the ε_ij values as its elements for each gene pair i, j = 1,2,…,n. In the ideal case, when there are no measurement inaccuracies, the approximation problem of Eq. 2 could easily be solved using the well-established machinery of linear algebra. More precisely, using the spectral decomposition theorem, one can represent any symmetric real matrix as , where λ is the largest eigenvalue of W and e is the corresponding eigenvector [15]. Under the unrealistic assumption that there are no genetic interactions among any of the gene pairs, the approximation would in fact be exact, that is, the residual error E equals zero. However, as the genetic interaction screens are bound to present with experimental variation, missing data, and hopefully also with significant genetic interactions, the estimation problem must in practice be solved by numerical means.

In the present work, we developed a sequential matrix approximation procedure, which uses increasingly larger subsets of mutation pairs in W to provide a series of estimates for w as solutions of the weighted least squares problem [16]. During the first rounds, the procedure solves the approximation problem of Eq. 2 using only those mutation pairs that best fit the multiplicative model, and then gradually extends the subset to also include pairs with larger residual errors (see Methods for details). Already when using all but the diagonal and missing entries of the double-mutant fitness matrix in the dataset of St Onge et al. [10], we obtained an estimate relatively close to the actual measured single-mutant fitness vector, as compared to the conventional median estimate (Figure 1A). The estimation accuracy could be markedly improved by excluding those pairs with the largest residual errors from the approximation subset. The pairs having the greatest impact on the approximation error, in fact, corresponded to the five confirmed synthetic lethal pairs (Figure 1B). When the sequential procedure omitted those pairs from the approximation process, an accurate estimate was obtained for each of the single-mutant fitness measurements (see Figure S1). As will be seen in the following subsections, however, the subset of mutation pairs which gives the most accurate estimates of the single-mutant fitness values does not necessarily lead to the best predictive power when identifying functionally related genes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Dynamic behavior of the sequential procedure as a function of mutation pairs.

(A) The mean absolute error between the measured single-mutant fitness vector and its estimate when using the selected mutation pairs. The dotted horizontal line indicates the estimation error obtained when using the median over the rows/columns of the double-mutant fitness matrix (average absolute error equals 0.0797). In case significant genetic interactions are rare, median of the colony sizes over all of the double mutants arising from the same single deletion strain provides an estimate of the effect of the particular single-mutant on the growth rate [12]. (B) The approximation error when using the selected mutations pairs to approximate the double-mutant fitness matrix (see Eq. 3 in Methods). The horizontal dotted line indicates the point of steepest increase in the approximation error, k = 317, which also gives on average the most accurate estimates of the single-mutant fitness values (average absolute error equals 0.0168). In each panel, the five spots after that line identify the synthetic lethal mutation pairs with double-mutant fitness value of zero. (C) The ranked deviations of the mutation pairs (i, j) defined according to their residual errors (see Eq. 4 in methods). (D) The measured deviations |ε_ij| of the selected mutation pairs obtained using the actual measurements of the single-mutant growth effects.

https://doi.org/10.1371/journal.pone.0003284.g001

Predicting specific functional relationships

Beyond the dynamic variability in the estimates of the single-mutant fitness values, the behavior of the approximation procedure with different subsets of mutation pairs revealed another interesting observation: the order in which the mutation pairs were added into the approximation process reflects on average the relative order of their actual measured deviations, even if the measurements of single-mutant fitness were not employed (Figures 1C and D). In fact, the measured deviations ε_ij and the ranked deviations obtained from the approximation procedure were highly correlated (Pearson correlation equals 0.964, see Figure S2). This led us to investigate whether such procedure-ranked deviations could be used instead of the measured deviations when predicting functional links between the mutations. To this end, we took the same set of gene pairs which were found to have a highly specific shared function in the previous study by Mani et al. [14] (see Methods for their definition). Interestingly, the majority of the mutation pairs selected towards the end of the approximation procedure shared a specific function (Figure 2). The rate of the functional enrichment observed among the 50 pairs with the largest values was significantly higher than expected (p<10⁻¹¹), whereas the functional enrichment was exceptionally low among the 50 pairs with the lowest values (p = 0.998). These results show that the sequential matrix approximation procedure gives as its by-product a ranking of the mutation pairs that is in good agreement with their likelihood of sharing a specific function.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Relationship between the ranking of the mutation pairs and their functional links.

The proportion of shared specific functional links among the top mutation pairs, when these pairs were selected in the increasing (red) or decreasing (blue) order according to their residual errors during the approximation process. The dotted line indicates the expected rate of the functional links when selecting mutation pairs at random (the expected proportion equals 0.108).

https://doi.org/10.1371/journal.pone.0003284.g002

To test more systematically whether the prediction of the functional relationships could be made to an accuracy similar to that obtained when using the measured single-mutant fitness values, we assessed how well the ranking based on values can discriminate between gene pairs with or without a specific functional link. Similarly as in the original study of St Onge et al. [10], the prediction capability was evaluated using the receiver operating characteristic (ROC) curves that show the relative trade-off between the sensitivity and specificity of the predictions at multiple decision thresholds (Figure 3). Surprisingly, the ranked deviations gave even a better prediction accuracy than the measured deviations according to the area under the ROC curve (AUC) values (AUC = 0.780 vs. AUC = 0.662, p = 0.003). The prediction capability was improved systematically at each specificity-level, demonstrating that the procedure could distinguish with high accuracy the functionally related gene pairs over the whole spectrum of the exceptional deviations (Figure 3). The order in which the mutation pairs were included into the approximation process further improved the relative classification power (AUC = 0.789, p = 0.002). The prediction accuracy of the double-mutant fitness values alone was similar to that of a random classifier (AUC = 0.506), demonstrating that the normalization by the measured or estimated single-mutant fitness values can, in any case, improve the prediction of the functional links. The estimates achieved using different subsets of mutation pairs can also lead to different degrees of prediction accuracy (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Predicting shared functional links using different measures of genetic interactions.

The accuracy of the prediction is evaluated using the receiver operating characteristic (ROC) curves for each measure: ranked deviations (red), measured deviations (blue), and the double-mutant fitness values (black). The true positive rate (TPR, or sensitivity) is the fraction of gene pairs correctly predicted to have functional links, and the false positive rate (FPR, or 1 - specificity) is the fraction of non-functionally linked gene pairs incorrectly predicted to have functional links. The overall prediction performance is summarized using the area under the ROC curve (AUC). For an ideal classifier, TPR = 1, FPR = 0 and AUC = 1, whereas a random classifier has on average AUC = 0.5 (the dotted diagonal line).

https://doi.org/10.1371/journal.pone.0003284.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Distribution characteristics for the measured and estimated deviations.

https://doi.org/10.1371/journal.pone.0003284.t001

Distribution of the estimated deviations

Finally, we investigated how the estimated deviations obtained from the approximation procedure at different subset sizes, k, are distributed relative to the true deviations ε_ij obtained when using the measured single-mutant fitness values. When comparing the different distributions, we used the same interpretation rule as in the earlier comparison by Mani et al. [14]: an ideal definition of genetic interaction should result in a tight distribution (indicating low variability) that is centered at zero (indicating low bias) for the bulk of the measured interactions (reflecting the background distribution of non-interacting genes). The subset of mutation pairs being used in the approximation process had a marked effect on the distribution of the estimated deviations (Figure 4). Moderate subset sizes generated distributions with a lower bias and variability than those obtained using smaller subset sizes or all of the mutations (Table 1). Surprisingly, the cut-off point, k = 317, which gave the most accurate estimates for the single-mutant fitness values resulted in a relatively weak prediction accuracy for the functional links (AUC = 0.679). Although the different measures of location and dispersion gave relatively similar results, the trimmed mean and interquartile range identified the distribution with least background bias and variability at k = 136 (Figure 4). The estimated deviations obtained using this particular subset of mutation pairs were also most successful in predicting the functional relationships (AUC = 0.810). These results indicate that the distribution characteristics of the estimated deviations could serve as a guide to choosing the optimal subset of gene pairs for defining genetic interactions in a given dataset. Both the optimal subset size and the overall performance of the method is likely to depend on the properties of the dataset being analysed, including the number of gene pairs and and the degree of their functional homogeneity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Distributions of the measured and estimated deviations over all mutation pairs.

Two distinct distributions are shown for the mutation pairs with specific functional links (red) and for the background pairs not sharing specific functional links (black). The measured deviations ε_ij generate bimodal distribution for the pairs with specific functional links (the upper panel). The estimated deviations generate background distributions with sharper peaks depending upon the size k of the subsets of mutations used in the approximation (the lower panels): k = 28 (initial subset of mutations), k = 136 (good prediction capability), k = 317 (smallest estimation error), and k = 323 (all mutation pairs). The five smallest deviation values in each distribution plot correspond to the five synthetic lethal mutation pairs. Table 1 lists the shape parameters of these distributions calculated over all of the mutation pairs. The two distributions in each individual plot are scaled according to their total number of pairs. The non-scaled versions of the same distributions are provided as Figure S3, which allows for better visual comparison between the functionally-linked and the functionally non-linked pairs.

https://doi.org/10.1371/journal.pone.0003284.g004

To investigate how the differences in the distributions are visible in the conventional heat map visualizations, we displayed the color-coded deviations on a two-dimensional grid spanned by the individual mutations. Here, a special emphasis was placed on analyzing the estimated deviations , which provided the most ideal definition of genetic interactions in terms of both distribution characteristics and predictive power, relative to the measured deviations ε_ij (Figure 5). In general, the interaction patterns were relatively similar between the measured and estimated deviations. However, the transformation of the double-mutation fitness measurements through the approximation process seemed to emphasize the mutation pairs with exceptionally large absolute deviations (putative genetic interactions), and pointed out especially those pairs having positive deviations (alleviating interactions), while it diminished certain subsets of mutation pairs with negative deviations (synergistic interactions). For instance, a considerable number of double-deletion strains involving either hpr5 or sgs1 mutations showed marked evidence for alleviating interactions in the color map of the estimated deviations (Figure 5A), while these pairs were not identifiable from the original map (Figure 5B). At the same time, the approximation algorithm blotted out certain moderate degree synergistic interactions in the hpr5 deletion strain, including the effects of additional mutations of mms4, mph1, or mus81. Although these and other changes contributed positively to the prediction of functional relationships in the dataset of St Onge et al., further evaluation how well these findings can be generalized beyond this relatively small set of functionally related genes is required on independent datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Estimated pairwise deviations vs. the actual measured deviations.

The color-coded heat map shows the estimated deviations (A) and the measured deviations (B) on a 26×26 grid. The estimated deviations were obtained at the cut-off point , which generates the most ideal distribution and provides the best discrimination between the functionally-linked and the non-functionally linked pairs (see Table 1). While the five confirmed synthetic lethal pairs (sgs1Δ mus81Δ, sgs1Δmms4Δ, sgs1Δslx4Δ, sgs1Δhpr5Δ and rad54Δhpr5Δ) are clearly visible in both of the maps, there are marked differences in the more subtle interaction scores at many places of the matrix between the estimated deviations and the measured deviations ε_ij. Red color corresponds to synergistic interaction scores and blue to alleviating interactions. The grey boxes indicate missing data points.

https://doi.org/10.1371/journal.pone.0003284.g005

Limitations and extensions of the procedure

Perhaps the biggest technical limitation of the present work concerns the heuristic way in which the subsets of mutation pairs were selected for the approximation of the double-mutant fitness matrix. The greedy subset selection scheme was motivated by a similar approach successfully being used in many feature selection problems [17]. An adaptive version of such a forward floating selection method was applied here because of its low computational complexity and because it was capable of excluding the most prominent outliers during the sequential approximation process (Fig. 1). Similarly, despite the weighted least squares matrix approximation algorithm being based on a rather straightforward decomposition method, it was able to reduce some degree of background variation in the data (Fig. 4). However, more sophisticated search and approximation schemes based on e.g. genetic algorithms or simulated annealing should lead to even better estimation and prediction results, or at least reduce the computational complexity. Additional modifications to enhance the present framework either in biological and/or computational terms include using deviations from the expected fitness as weights in the least squares approximation and using the sign of the deviations when including or excluding a mutation pair over the course of the sequential approximation process. While the present results were based on the rank-one approximation only, which enabled the partitioning of pairs of genes into two categories (interacting or non-interacting), utilizing the higher order ranks could allow us to classify the quantitative measurements into several categories, for instance, synergistic and alleviating interactions, or even more fine-grained classification of interactions that can occur between genes [18]. This could also help us also to distinguish those biological modules in which the distribution of genetic interactions does not follow the ideal tight and zero-centered distribution that has been used traditionally [9], [14].

Future applications and research directions

In spite of the above mentioned technical limitations, the present results support the feasibility of the approximation framework for systematic exploration of genetic interaction data, and warrant its applications to larger-scale datasets, such as those generated with the E-MAP or GIM screens, to confirm whether similar findings can be extrapolated to the genetic interactions data derived from high-throughput technologies. Other quantitative phenotypes or experimental techniques for defining or measuring genetic interactions could, in principle, also be used, although certain modifications will be needed to adapt the procedure to the specific characteristics of each genetic interaction screen. In the larger-scale screens, the gene pairs under analysis can be selected more randomly among a wider range of functions, thus increasing the expected proportion of non-interacting pairs. Accordingly, the more interactions there are being measured, the better the assumptions behind the multiplicative model will be justified, provided that significant genetic interactions remain relatively rare. The many missing values typically occurring in the large-scale screens should not pose a major problem for our approach either, due to its sequential nature being able to adapt to those subsets of double-mutant measurements with the best approximation power. Hence, the approximation approach is likely to yield even better results with larger and unbiased datasets. Similarly, even if the assumption of low frequency of significant interactions may become compromised in more targeted studies, such as the one of St Onge et al, this should not have a major effect on the results as the strongest genetic interaction pairs are effectively filtered out in the sequential estimation process. For such smaller-scale and more targeted genetic interaction studies, a further increase in the performance could be obtained by modifying the null model for non-interacting pairs to take into account the multitude of single-mutants affecting the particular double-mutant fitness value. This is one of the modeling challenges which we aim to tackle in the future.

Integrative analysis together with other data sources

More generally, the computational approach based on the sequential matrix approximation can provide a principled framework for exploring and classifying genetic networks and interactions using a wide spectrum of global data sources, including the localization, mRNA or protein expression, physical interaction and functional annotation of the proteins encoded by the genes [19]. It has previously been demonstrated that physical protein-protein interactions, in particular, provide useful information that is, by and large, complementary to that obtained from the functional genetic interactions [6], [20]. To reveal the modular structure of the underlying networks and functional organization the multitude of pathways reflected in such large-scale data types, various network partitioning methods have been used to detect either densely- or similarly-connected clusters as well as significantly-repeated motifs in the individual or integrated interaction networks [21]–[24]. However, many open questions still remain about the integrative analysis strategy of these datasets and the most meaningful interpretation of their results. For instance, the extent to which the genetic interaction could be explained by the other information sources, such as protein-protein, protein-DNA, metabolic network and protein structure data [25]–[28], and how these should be efficiently employed when scoring genetic interactions using measures such as the pairwise deviations, the S- and COP-scores, or the correlation and congruence between the interaction patterns [9]–[12], [27]–[29]. Finally, the success of any computational approach for constructing genetic interaction networks is likely to be driven by parallel improvements in the experimental technologies, such as enabling measurement of phenotypic effects in response to the mutation of more than two genes in combination.

Approximating the double-mutant fitness matrix

For a given subset of mutation pairs, we calculated the matrix approximation of Eq. 2 using the decomposition algorithm by De Leeuw [16]. Formally, we solved the following weighted least-squares optimization problem at each step l of the procedure(3)where both the given weight matrix C^(l) and the double-mutant fitness matrix W are symmetric of order n. After normalizing for the subset size, the square-root of the objective function obtained as the solution of the optimization of Eq. 3 is referred to as approximation error at phase l. Using this formulation, the solution ŵ^(l) minimizes the sum of the squared residual errors(4)over all of the k gene pairs (i, j) with which were involved in the approximation process at the l^th step of the procedure. Although the general formulation in Eq. 3 allows for an element of C^(l) to be any non-negative number, we used binary weights only; in particular, we set the diagonal entries for all i, and for the missing data points, at each step l. Throughout the operation of the approximation procedure, we ensured that the weight matrix remained symmetric.

In the dataset of St Onge et al. [10], there were two missing values because of genetic linkage between the gene pairs involved in the double-deletion strains rad57Δrad61Δ and rad55Δshu2Δ.

Selecting subsets of mutation pairs for estimation

The subset of mutation pairs used at a particular step of the approximation process was encoded in the binary weight matrix C^(l), that is, if and only if the mutation pair (i, j) is used at step l. To select increasingly large subsets of non-missing and non-diagonal mutation pairs from the double-mutant fitness matrix W, we adapted the floating search method of Pudil et al. [17]. The sequential subset search method is characterized by a dynamically changing number of features included or eliminated at each step. In our implementation, the residual errors were used as the criterion function for adding or deleting mutation pairs. The operation of the forward-type subset selection scheme was organized through the following steps:

1. Set l←0 and initialize the weight matrix C⁽⁰⁾ (see the next subsection)
2. Estimate ŵ^(l) using the decomposition algorithm (previous subsection)
3. While there are non-diagonal and non-missing entries with
   1. select pair (i, j) with having the smallest residual error (Eq. 4)
   2. set , make a new estimate of ŵ^(l) and re-calculate the residuals
   3. if there exists a pair (i, j) with and , then set
   4. set l←l+1 and repeat step 3

We modified the general subset search method by making the deletion of pairs an adaptive process over the evolution of the subset search. More specifically, the threshold t^(l) used in the conditional exclusion step 3c is multiplied by 1.5 each time the pair selected for deletion is the same as that added in step 3a. This modification enabled the forward-type algorithm to recover from poor starting tolerance values (we used t⁽⁰⁾ = 2×10⁻⁵ in the present work) or from a poor initialization of the weight matrix (see the next subsection for details), and it also made it possible to increase the size k of the subsets up to the maximal size K. Figure 1 shows the evolution of the subset selection algorithm from the initial subset configuration to the full set of K = 323 mutation pairs in the dataset of St Onge et al. [10].

Initialization of the weight matrix from the data

The approximation algorithm requires the weight matrix C^(l) to be non-singular at each stage of the procedure. Therefore, we cannot start the sequential approximation procedure from the empty subset of mutation pairs, but instead must adjust the initialization C⁽⁰⁾ = 0 for the dataset under analysis. After calculating the initial residuals from the whole double-mutant matrix, three types of adjustments were made in the present study: (i) for each i, we set for the pair (i, j) with the smallest residual to make all of the rows in C⁽⁰⁾ non-zero; (ii) we made an additional setting of for each identical row pair to reduce the linear dependence among the rows of C⁽⁰⁾; and finally (iii) we set pairs (i, j) with to 1 in the increasing order of their residuals until the determinant of C⁽⁰⁾ became non-zero. Every time a pair (i, j) was added in the initialization steps (i)–(iii), we also set its transpose entry to one, that is, , to keep the weight matrix symmetric. In the dataset of St Onge et al. [10], this resulted in an initial weight matrix C⁽⁰⁾ with 28 entries of ones.

Sequential estimation of the pairwise deviations

The measured deviations ε_ij = w_ij−w_iw_j were estimated in two different ways. The sequential approximation procedure gives as its by-product a surrogate for the deviations in the form of the residual errors of Eq. 4. More specifically, we defined the ranked deviation of a mutation pair (i, j) as its residual error at the last step l during which the pair was included into the approximation subset. In this way, even when there are multiple inclusions and deletions of a particular pair during the procedure, we obtained an unambiguous ranking of the pairs according to their values. This ranking and the corresponding ranked deviations are shown in Figure 1 for all gene pairs of the dataset of St Onge et al. [10], except for the 28 initial mutation pairs.

The second set of estimates was obtained by stopping the sequential approximation procedure at a given subset size, k, and by using the estimate of Eq. 3 in place of the measured single-mutant fitness vector w in the definition of the deviation in Eq. 1. For a mutation pair (i, j), this resulted in a sequence of estimated deviations for step l at which the size of the approximation subset equals k. Note that since the residual errors in Eq. 4 are updated each time that a new pair is added, the estimated deviations can vary considerably as a function of k. These estimates are not generally congruent with the ranked deviations. The distributions of the estimated deviations with different subset sizes k are illustrated in Figure 4 in the dataset of St Onge et al. [10].

Quantitative genetic interaction measurements

To evaluate the performance of the sequential matrix approximation procedure in practice, we applied it to a recent high-resolution genetic interaction study of St Onge et al. [10]. This particular study was chosen because it contains quantitative growth-rate measurements of both single- and double-mutant cell populations for a targeted set of 26 genes related to DNA repair in yeast S. cerevisiae. The detailed time course fitness measurements were performed in the presence and absence of the DNA-damaging agent methyl methaneusulfonate (MMS). The results of the present study were based on the growth measurements in the absence of MMS. The prediction of functionally related gene pairs was in fact more challenging in this case than in the data measured in the presence of MMS. The measured and estimated single-mutant fitness values and the double-deletion deviations in the dataset are shown in Figure S1 and Figure 5, respectively.

Defining gene pairs sharing a specific function

Functional links among the 26 genes were defined using the same approach as in many previous genetic interaction studies [6], [10], [14]. Briefly, a term in the Biological Process branch of the Gene Ontology was considered specific if it was associated with fewer than 30 yeast genes, and two genes were considered to have a specific functional relationship if they shared any of those specific terms [14]. This resulted in the set of 35 specific functional links in the dataset of St Onge et al. [10].

Statistical evaluation of the predictive power

Statistical enrichment of the specific functional links among a set of mutation pairs selected by the sequential approximation procedure was assessed using the standard hypergeometric test. Briefly, if t is the number of top mutation pairs selected according to their residual errors, and M is the total number of the functionally-related links, then the probability of obtaining at least m functionally-related pairs when selecting pairs at random from the set of K mutation pairs can be calculated using the cumulative distribution function:

The enrichment for the M = 35 functionally-linked pairs among the t = 50 mutation pairs selected on the basis of either small (m = 1) or large (m = 22) residual errors is shown in Figure 2. The dotted line shows the expected rate of the functional links when selecting mutation pairs at random, that is, M/K = 35/323 = 0.108.

The predictive power of the measured and estimated deviations was assessed using the receiver operating characteristic (ROC) curves that characterize the relative trade-off between the true positive rate (sensitivity) and the false positive rate (1 - specificity). The overall predictive performance was summarized using the area under the ROC curve (AUC). The statistical significance of the difference in the AUC values between two genetic interaction measures was assessed using a custom written algorithm based on the method of DeLong et al. [30]. This nonparametric method uses the theory of generalized U-statistics to calculate an estimated covariance matrix and hence it can also take into account the correlated nature of the data. The ROC curves and the corresponding AUC values for the prediction of the 35 functional links in the dataset of St Onge et al. [10] are shown in Figure 3.