A Direct Phenotypic Comparison of siRNA Pools and Multiple Individual Duplexes in a Functional Assay

Background RNAi is a prominent tool for the identification of novel regulatory elements within complex cellular pathways. In invertebrates, RNAi is a relatively straightforward process, where large double-stranded RNA molecules initiate sequence-specific transcript destruction in target cells. In contrast, RNAi in mammalian cell culture assays requires the delivery of short interfering RNA duplexes to target cells. Due to concerns over off-target phenotypes and extreme variability in duplex efficiency, investigators typically deliver and analyze multiple duplexes per target. Currently, duplexes are delivered and analyzed either individually or as a pool of several independent duplexes. A choice between experiments based on siRNA pools or multiple individual duplexes has considerable implications for throughput, reagent requirements and data analysis in genome-wide surveys, yet there are relatively few data that directly compare the efficiency of the two approaches. Methodology/Principal Findings To address this critical issue, we conducted a direct comparison of siRNA pools and multiple single siRNAs that target all human phosphatases in a robust functional assay. We determined the frequency with which both approaches uncover loss-of-function phenotypes and compared the phenotypic severity for siRNA pools and the constituent individual duplexes. Conclusions/Significance Our survey indicates that screens with siRNA pools have several significant advantages over identical screens with the corresponding individual siRNA duplexes. Of note, we frequently observed greater phenotypic penetrance for siRNA pools than for the parental individual duplexes. Thus, our data indicate that experiments with siRNA pools have a greater likelihood of generating loss-of-function phenotypes than individual siRNA duplexes.


Introduction
Reverse genetic studies are a dominant force in the functional characterization of eukaryotic gene products in many pertinent model systems. Unfortunately, such studies tend to be laborious, costly and time-consuming in complex eukaryotic models. Thus, whereas targeted loss-of-function studies are commonplace in comparatively straightforward organisms such as yeast, in vivo mutational analyses present considerable challenges in subjects of a more immediate biomedical relevance such as mice. The recent advent of the ''RNA universe'' fundamentally altered perspectives for reverse genetic studies in mammalian systems. Specifically, RNA interference (RNAi) bears the real promise of rapid and directed genome-wide loss-of-function studies in a broad range of cell culture models [1]. The principles of genome-scale RNAi screens were first established in C. elegans and Drosophila melanogaster and were later applied to mammalian systems [2,3,4]. Fortuitous parallel developments of automated systems such as liquid handling devices and high-throughput plate readers place genome-wide studies of direct biomedical relevance within immediate reach. For example, RNAi screens in mammalian cell culture assays successfully probed topics as diverse as embryonic stem cell self-renewal, West Nile virus infection and various aspects of cancer progression [5,6,7,8,9]. Looking forward, progress in assay miniaturization [10] and the establishment of high-content systems makes it clear that more sophisticated assays are yet to appear in the RNAi toolbox.
Despite these astounding developments, it is clear that RNAi is not the ultimate panacea for the contemporary cell biologist. A particular vexing issue that has not been fully addressed in siRNA screens is the question of knock-down efficiency. In this context, a major decision in every siRNA experiment is the amount of siRNA duplexes to deliver for each gene target. Current conventions suggest that a minimum of three siRNA duplexes that target nonoverlapping sections of the same gene product should be employed. There are two theoretical advantages to such an approach. First, as many siRNA duplexes fail to significantly deplete their intended target, increased numbers of duplexes should increase the likelihood of generating the true loss-offunction phenotype for a particular gene. Furthermore, the ability to probe a given target with multiple siRNA duplexes should decrease the impact of off-target effects on hit identification, if one sets a threshold that a minimum of two non-overlapping siRNA duplexes should yield an overlapping phenotype [11]. Based on these considerations, the standard practice for many siRNA experiments is to test a minimum of three non-overlapping duplexes and exclude all gene products that were not affected by a minimum of two siRNA duplexes.
There are currently two approaches to test multiple siRNA duplexes in genome-scale screens. In one case, each duplex is tested individually. Whereas this approach requires considerable commitments in terms of resources, it permits stratification of hits into confidence groups based on the number of individual siRNA duplexes that uncover a particular phenotype. The alternative approach is to test all siRNA duplexes that target a given gene product as a single pool. The obvious advantage to such an approach is that it maximizes resources and increases throughput. However, siRNA pool screens do not provide insights into the number of individual siRNAs that contribute to a given phenotype and the onus is on the investigator to test multiple single siRNA duplexes in secondary assays specifically designed to probe false positive rates. Both approaches are routinely applied in genomescale screens, yet the relative merits of siRNA pools and single siRNA duplexes have not been tested in a systematic phenotypic assay. Given the clear ramifications such considerations have for screen execution, we consider this a critical issue.
Here, we present the results of a direct comparison of the behavior of pooled and single siRNA duplexes in an unbiased functional screen of the entire set of human phosphatases. Our data indicate that the apparent hit frequency is higher in screens performed with siRNA pools and that phenotypes from siRNA pools are often more pronounced than phenotypes for the corresponding individual siRNA duplexes. These observations combined with the advantages represented by siRNA pools in terms of throughput and reagent consumption, lead us to propose that siRNA pools are a superior tool for genome-scale siRNA screens.

A Quantitative siRNA-Based Assay for Modifiers of TNF-Dependent Cell Death
We developed a quantitative plate-based assay to identify siRNA duplexes that modify cellular responses to the Tumor Necrosis Factor-a (TNF) cytokine. Engagement of the Tumor Necrosis Factor Receptor (TNFR) by homotrimeric TNF ligands drives the activation of JNK, NF-kB and caspase signal transduction cassettes. NF-kB family members promote the transcription of ''pro-survival'' gene products that contribute to inflammatory responses, induce cellular proliferation and differentiation and actively block caspase-mediated pro-apoptotic events. In the absence of NF-kB-responsive gene products, TNF signal transduction results in apoptotic cell death through the caspase cascade.
We used a defined set of siRNA duplexes to probe the entire annotated collection of human phosphatases (265 individual genes) for modifiers of TNF-dependent cell death. The layout of each siRNA master plate is shown in Figure 1A. Each plate contained 80 experimental siRNA duplexes that target defined phosphatases. In addition, each master plate contained multiple control wells, such as no siRNA duplexes, a non-silencing control siRNA duplex and positive control siRNA duplexes that target NEMO or caspase-8. We chose NEMO and caspase-8 as controls, as they are essential for the activation of NF-kB or the induction of apoptotic events, respectively. We prepared four master plates for each target gene: three plates contained unique non-overlapping siRNA duplexes and the fourth plate contained a pool of the three individual duplexes. Each master plate was prepared in such a way that all plates contained equimolar concentrations of total siRNA. We then screened each plate for modifiers of TNF-dependent cell death. The screen layout is described in Figure 1B. We treated two replicate plates with TNF and non-lethal doses of the translation inhibitor cycloheximide. The combined regime attenuates the expression of NF-kB responsive anti-apoptotic factors and is a widespread method for the induction of TNFresponsive cell death in a number of different cell lines. We quantified cell viability in each well under the respective culture conditions in a resazurin cell viability assay and normalized each viability score to the mean of the non-silencing control siRNA on the same plate. Normalization allowed us to perform plate-to-plate comparisons and develop a comparable ''death index'' to assess the level of TNF-induced death for each siRNA under scrutiny.
Analysis of replicate measurements confirmed that the overwhelming majority of the cell viability measurements was reproducible ( Figure 1C). Control caspase-8 siRNAs reliably blocked TNF/cycloheximide-dependent cell death, while NEMO siRNAs reproducibly increased TNF/cycloheximide-dependent death. Both phenotypes are consistent with the respective roles of caspase-8 and NEMO in the induction or inhibition of TNFdependent apoptosis. Statistical evaluation of screen data confirmed that the overall screen quality was high [12]. In summary, we are confident that the assay represents a valid springboard for the identification of siRNA duplexes that modify TNF-dependent cell death.

Statistical Evaluation of Screen Data
We then plotted the death index for each siRNA from the primary screen ( Figure 2). The corresponding raw viability measurements are available in Table S1. For statistical purposes, we defined a modifier siRNA as an siRNA duplex that produced a death index greater than 1.96 standard deviations above the median non-silencing control siRNA. These criteria define the 95 percent confidence interval for true modifier siRNAs and are routinely applied for ''hit'' identification in high-throughput RNAi screens. As we consider assaying a pool of three siRNA duplexes a distinct experimental approach to an assay that probes each siRNA duplex separately, we identified modifier siRNA pools and modifier single siRNA duplexes separately. Current standards for experiments with single siRNA duplexes dictate that a minimum of two non-overlapping siRNAs are required to produce overlapping phenotypes before the target gene product should be considered a valid modifier. Based on these critieria, we defined two separate categories of modifier single siRNAs: a highconfidence set, where all three siRNAs fell within the 95% confidence interval; and a medium-confidence set, where two of three siRNAs fell within the 95% confidence interval.

A Comparison of Hits from the Single and Pooled siRNA Duplexes
We then compared modifier gene products identified through pooled siRNAs and single siRNA assays. From these comparisons, we distinguished three broad classes of putative hits: ''dual'' hits, where the pool and a minimum of two single siRNA duplexes identified the target as a modifier; ''pool only'' hits, where only the pooled siRNA identified the target as a modifier; and ''single only'' hits, where a minimum of two single siRNA duplexes identify the target as a hit, but the pooled siRNA does not (Table 1). In total, we identified thirty three putative hits from the siRNA pools and nineteen putative hits from the individual siRNA duplexes, of which three were high-confidence modifiers and sixteen were medium-confidence modifiers ( Figure 3A and B). Thus, while we detected a general overlap between both screens in terms of ''hit'' identification, our survey indicates that the frequency of hit identification is considerably higher for screens performed with siRNA pools.

Phenotypic Distinctions between Screens with siRNA Pools and Screens with Single siRNA Duplexes
The greater apparent frequency of hits in screens performed with siRNA pools prompted us to examine the severity of the phenotypes described for pooled and single siRNA duplexes. To this end, we arranged each gene product in descending order of death index from the pooled screen and plotted the range of death indices for the single siRNA duplexes that target the same gene product ( Figure 4A). A cursory examination of the corresponding death indices reveals a broad range of death indices for a large number of single siRNAs that target a common gene. This is a common feature of individual siRNA molecules and likely reflects that fact that many siRNA duplexes fail to significantly deplete the target protein.
We were surprised to note that the death index attributed to siRNA pools often fell outside the range of death indices attributed to the corresponding set of individual siRNA duplexes. In total, we identified 82 of 265 cases (31%) where the death index of the siRNA pool did not overlap with the range of death indices for the corresponding individual siRNA duplexes. Close examination of the siRNA pools with death indices outside the corresponding single siRNA ranges revealed a particularly intriguing feature. We divided the siRNA pools into five equally sized groups (53 pools per group) that ranged from pools with the highest death index to pools with the lowest death index ( Figure 4A). In the group of pools with the highest death indices, 25 had death indices that were outside the range of the corresponding single siRNA  The death index for each pool and single siRNA are shown in the final four columns. Putative hits are divided into groups according to the screens in which they were identified as modifiers. Putative modifiers are further subdivided into confidence intervals based on the number of single siRNAs that give a particular phenotype. In the pool only list, low confidence hits are genes for which a single siRNA gave a significant modifier phenotype, and zero confidence hits were not significantly modified by any single siRNA. doi:10.1371/journal.pone.0008471.t001 duplexes. Strikingly, the death index for each one of the siRNA pools was greater than the maximum death index of the individual siRNA duplexes ( Figure 4B). We noticed an identical phenomenon when we examined the opposite end of the phenotypic spectrum. Of the 53 pooled siRNAs with the lowest death index, 18 had death indices that were outside the range of the corresponding single siRNA duplexes. In each case, the death index for the siRNA pool was lower than the lowest death index identified for the corresponding single siRNA duplexes. These data suggest that the phenotypic penetrance of siRNA pools often exceeds the phenotypic penetrance of any one of the corresponding single siRNA duplexes.

A Comparison of Pooled and Single siRNA Screen Data
The frequent disparity between phenotypic penetrance for single duplexes and siRNA pools led us to question the phenotypic relationship between both methods of siRNA delivery. For these comparisons, we postulated that a phenotype from an siRNA pool may represent the mean phenotype of the individual siRNA duplexes; the median phenotype of the individual duplexes; or the optimal phenotype of the individual duplexes. To distinguish these possibilities, we plotted the death indices for the siRNA pools against the mean, median and optimal death indices from the corresponding single siRNA duplexes. We considered two possible definitions of ''optimal'' phenotype: one where the siRNA duplex with the most extreme death index constitutes the optimal death index (single optimal); and one where the mean of the two greatest death indices from the single siRNA duplexes constitute the optimal death index (cumulative optimal). The former definition favors a scenario where a very limited number of highly efficient on-target siRNAs determine the loss-of-function phenotype. The latter definition is more biased to a scenario where the loss-of-function phenotype represents the cumulative effect of all on-target siRNAs.
For each scenario, we determined the correlation coefficient and used paired students t-tests to determine the significance of overlap between the two death indices. In each case, we noticed a general overlap between the corresponding death indices ( Figure 5A-D). These data indicate a broad phenotypic consensus between siRNA pools and multiple siRNA duplexes and argue that screens with siRNA pools are equally efficient at generating loss-of-function phenotypes as screens performed with multiple individual siRNAs. Strikingly, paired students t-tests revealed a significant phenotypic correlation between siRNA pools and the optimal death index (cumulative or single) for the corresponding single siRNA duplexes. Thus, it appears that siRNA pools often generate the maximal phenotype for the corresponding single duplexes.

Discussion
RNAi-based loss-of-function studies open entirely new perspectives in cell biology, particularly for the identification of biomarkers or pharmacologically relevant targets in critical disease states. As with many genetic techniques, groundbreaking studies in the model invertebrates C. elegans and Drosophila melanogaster guided subsequent developments in mammalian systems. In Drosophila and C. elegans, RNAi involves the introduction of large dsRNA molecules typically greater than 250 bp into target cells. The large duplexes are diced internally into numerous small 21mer duplexes, which feed the classical RISC/RNAi pathway. Thus, RNAi experiments in invertebrates are, by design, performed with large pools of individual siRNA duplexes.
As large dsRNA molecules induce non-specific interferon responses in mammalian cell lines [13], alternative strategies are required for targeted knock-down studies. Typically, investigators use lipid-based or viral vehicles to deliver a small number (1 to 4) of non-overlapping siRNAs to target cells. These siRNA molecules are either delivered individually or as pools. Despite the clear experimental differences between the separate delivery of multiple individual duplexes and a pool of the same duplexes, there are very little data on the phenotypic consequences of such an approach. The bulk of recommendations on the relative advantages and disadvantages of pooled or single siRNAs are given by product vendors with vested commercial interests in the outcome. For example, a recent review of siRNA technologies specifically discussed the relative merits of pooled versus single siRNAs from the perspective of various commercial vendors of siRNA libraries [14]. Unsurprisingly, the opinions were immediately aligned with the product portfolio of the individual companies -the leading vendor of siRNA pools recommended pools and vendors of individual duplexes strongly cautioned against the use of pools. In this study, we set out to prepare an unbiased comparison of the phenotypes generated by siRNA shows high and medium confidence hit groups from the single siRNA screen and indicates the degree to which each group overlaps with the siRNA pool hits. (B) Hierarchical cluster analysis of pool screen hits and medium to high confidence interval single screen hits. The death index of siRNA targets are clustered on a color scale ranging from the highest death index value, depicted as yellow, to the lowest death index value, depicted as black, and centered around the non-silencing control assigned a death index of one, depicted as red. Group I represents medium confidence single siRNA hits. Group II represents all putative hits identified in the pool and medium confidence single siRNA screen. Group III represents all putative hits identified in the pool screen only. Group IV represents all putative hits identified in the pool and highconfidence single siRNA screen. Group V represents high confidence single siRNA hits. doi:10.1371/journal.pone.0008471.g003 pools and the parental single siRNAs in a simple functional assay. Given the tremendous potential of siRNA screens for the identification of biomarkers or pharmacological targets and the explosive growth in prominent studies based on high-throughput siRNA studies, we believe our findings are of considerable value to a large community of biological researchers.
Perhaps the most significant aspect of our findings is the fact that the phenotypic penetrance for siRNAs pools is frequently more severe than for any of the parental siRNA duplexes. Evaluation of our phenotypic data suggests that the phenotype associated with a pool of siRNAs most closely resembles the cumulative functional phenotype of the parental siRNA duplexes. The most parsimonious explanation for this observation is that the effective target knockdown achieved with an siRNA pool is a combination of the knockdowns achieved by the constituent duplexes. Thus, in a case where several individual duplexes yield partial knock-downs that are  insufficient to uncover meaningful phenotypes, a combination of the individual knock-downs may translate into a more robust and detectable phenotype. Most siRNA design algorithms are based on empirical observations of the sequence composition of effective ontarget siRNAs. Advances in siRNA design algorithms allow vendors to make general guarantees about the likelihood of transcript destruction upon the purchase of a set of non-overlapping siRNAs that target a specific transcript. However, it is not possible to predict the success of each individual siRNA, as the exact cellular context in which siRNAs interact with target transcripts remains unresolved. For example, siRNA design algorithms do not address critical issues such as secondary structures within specific target transcripts or accessibility issue presented by RNA protein complexes. siRNA pools may partially compensate for these shortcomings by combining several weak knock-down phenotypes to generate a more penetrant phenotype.
As siRNA pools often generate more penetrant phenotypes than any of the corresponding single duplexes, we also expect that screens with pools will identify a greater number of putative hits in the primary screen. Consistent with this hypothesis, we note that standard analysis of our siRNA pool data identified 33 putative hits (12% hit rate), while analysis of our single siRNA data identified 16 medium-confidence hits (6% hit rate) and only three high-confidence hits (1% hit rate). Based on our own work and data from other siRNA screens, we consider it highly probable that some of the putative hits from the siRNA pools are false positives. However, we believe that the effort required to identify false positives in secondary analysis is preferable to the loss represented by false negatives in misguided primary analysis.
In summary, our data argue that screens performed with siRNA pools identify a larger set of hits than screens performed with single siRNA duplexes and that the phenotypic penetration for siRNA pools is often greater that any of the individual siRNAs. Thus, we conclude that screens with siRNA pools represent an optimal approach in terms of phenotypic strength, throughput and reagent efficiency.

siRNA Library of Human Phosphatases
To obtain the targeted knockdown of 265 phosphatases we used the Ambion SilencerH human phosphatase-specific siRNA library (Applied Biosystems). The library is composed of three sequenceindependent siRNAs per gene target and was handled according to manufacturer's protocol. To obtain the pooled siRNA library, equal volumes from all three target-redundant single siRNA libraries were combined. The single and pooled siRNA libraries were prepared to stock concentrations of 200 nM, thereby establishing the concentration of each unique siRNA in the pooled library at 66.67 nM.

Single and Pooled siRNA Screens Using the Resazurin Viability Assay
For siRNA library screening in a 96 well plate format, all steps were performed on a sterile automated liquid handling platform (JanusH PerkinElmer). HeLa cells of an identical passage number at 8.67610 4 cells/ml were reverse transfected with siRNA at a final concentration of 20 nM using DharmaFect 1 transfection reagent (Dharmacon). After transfection, cells were incubated with siRNA under standard mammalian cell culture conditions for 72 h. Following the RNAi incubation period, cells were either treated with cell culture medium alone or with a mixture of Tumor Necrosis Factor alpha (Roche) and Cyclohexamide (Sigma) in culture medium to a final concentration of 20 ng/ml and 5mg/ml respectively. The redox dye, resazurin (Sigma), was added to cells 13.5 h after treatment and incubated with cells for 1.5 h before reading fluorescence signal on a multi-label plate reader (EnVisionH PerkinElmer).

Statistical and Graphical Analyses
To analyze the screen data, confidence intervals were established from the standard deviation of the single siRNA screens from the non-silencing control siRNA on the respective plates. To establish a simplified view of gene clusters with differential involvement in modulating TNF/CHX-induced death, cluster analysis was performed using the Cluster 3.0 program. Data from 265 normalized death indices measured from single and pooled screens were organized by hierarchical clustering. Euclidean distance between normalized death indices was used as the metric. The results of this cluster analysis are represented as a heatmap generated by Java TreeView 1.1.3.