Browsed by
Tag: RNAi screening

Low hit validation rate for Dharmacon siGENOME screens

Low hit validation rate for Dharmacon siGENOME screens

Good experimental design is important when validating hits from RNAi screens.  Off-target effects from single siRNAs and low-complexity siRNA pools (e.g. Dharmacon siGENOME) result in high false-positive rates that must be sorted out in validation experiments.

Dharmacon siGENOME pools (SMARTpools) have 4 siRNAs, and the most common form of validation is to test the pool siRNAs individually (deconvolution).

Unfortunately, the results of such deconvolution screening rounds are difficult to interpret.

The pool phenotype could be due to the off-target effects of any single siRNA, or even synthetic off-target effects from pooled siRNAs.

Rather than deconvoluting the pool, a better approach is to test with independent reagents.  Should the phenotype be due to the seed effects of an siRNA in the siGENOME pool, the new designs (with presumably different seed sequences) should not show them.  (Note that because they have their own potential complicating off-targets, an even better option would be to use a reagent like siPOOLs that minimises the likelihood of off-target effects).

Independent validation reagents was the approach used by Li et al. in a screen looking for enhancers of antiviral protein ZAP activity.

They first did a genome-wide (18,200 genes) screen with siGENOME pools, looking for pools that increased viral infection rate.

The biggest effect was with the positive control, ZAP (aka ZC3HAV1).   Several other pools also stood out as giving large increases in viral infection (Fig 1B):

They identified 90 non-control genes with reproducible Z-scores above 3 in their replicate experiments (~0.5% of screened genes).

These 90 genes were then tested with 3 Ambion Silencer siRNAs.  (They also included a few genes in the validation round based on pathway information and off-target analysis– more on this below.)

Of the 90 candidate hit genes, only 11 could be confirmed (Fig 2B, note that ZC3HAV1/ZAP is the positive control and JAK1 was added to the validation round based on pathway info.  A gene was considered confirmed if 2 of 3 siRNAs had a Z-score > 3.):

 

We also see that only 1 of the 7 top hits from the first round (blue genes in the first figure) was confirmed.  This is a common observation in RNAi screens: the strongest phenotypes are mostly due to off-target effects.

Off-target effects are difficult to interpret, even using advanced analysis programs like Haystack or GESS.  The authors tested 4 genes identified by Haystack as targets for seed-based off-targeting.  None of those genes could be confirmed in the validation round.

Follow us or share this post:
Pooling only 4 siRNAs increases off-target effects

Pooling only 4 siRNAs increases off-target effects

In a previous post, we showed how siRNA pools with small numbers of siRNAs can exacerbate off-target effects.

Low-complexity pools (with 4 siRNAs per gene) should thus lead to overall stronger off-target effects than single siRNAs.

This phenomenon was addressed in a bioinformatics paper a few years back.  The authors created a model to predict gene phenotypes based on the combined on-target and off-target effects of siRNAs.

The siRNAs were screened either individually (Ambion and Qiagen), or in pools of four (Dharmacon siGENOME), in 3 different batcterial-infection assays (B. abortus, B. henselae, and S. typhimurium).

The model assumed that each siRNA silenced its on-target gene to the same level.  For off-target silencing, they used the predictions from TargetScan, a program for calculating seed-based knockdown by miRNAs or siRNAs.

In order to assess model quality, they checked how similar the gene phenotype predictions were when using different reagents types in the same pathogen-infection screen.

The following figure shows the rank-biased overlap (a measure of how similar lists are with regards to top- and bottom-ranked items), when estimating siGENOME off-target knockdown in one of 2 ways:

A) using the maximum TargetScan score for any of the 4 siRNAs in the siGENOME pool

B) using the mean TargetScan score for the 4 siRNAs

If low-complexity pooling increases the degree of off-target effects, we would expect the maximum TargetScan score to produce better model concordance.

And that is what the authors found.  (the two plots show the rank-biased overlap for the top and bottom of the phenotype ranked lists, respectively)

The off-target effect of a 4-siRNA, low-complexity pool is best described by the strongest off-target effect of any of the individual siRNAs.

As discussed in our NAR paper, pooling a minimum of 15 siRNAs is required to reliably prevent off-target effects.

Follow us or share this post:
Citations of our Nucleic Acids Research Paper

Citations of our Nucleic Acids Research Paper

Our 2014 Nucleic Acids Research paper provides an excellent overview of the siPOOL technology.  Google Scholar shows that our paper has been cited 64 times.

To put this into perspective, the 2012 PLoS One paper on C911 controls by Buehler et al. has 72 citations.  C911 controls are probably the most effective way to determine whether a single-siRNA phenotype is due to an off-target effect.

These citation numbers show that siPOOLs have good mind share when researchers consider the issue of RNAi off-target effects.

We have noticed, however, that in some cases our NAR paper is cited to justify approaches that we do not endorse.

For example, two recent papers (1, 2) cite our paper as support for the use of Dharmacon ON-TARGETplus 4-siRNA pools to reduce the potential for off-target effects.

Our paper shows, however, that high-complexity siRNA pools (> 15 siRNAs) are needed to reliably reduce off-target effects.

We have also discussed how low-complexity siRNA pools can in fact exacerbate off-target effects.

There’s an old saying that any publicity is good publicity, and we are certainly thankful that these authors have referenced our paper, even if we don’t agree with the interpretations.

And we are especially grateful to all the researchers who have purchased siPOOLs and referred to our products in their publications.

Follow us or share this post:
5 factors to consider in multi-gene targeting RNAi screens

5 factors to consider in multi-gene targeting RNAi screens

Summary: Effective functional genomic screening depends on a variety of factors that need to be simultaneously addressed to obtain meaningful results. A recent Cell Reports paper demonstrates this by taking a holistic approach to siRNA screening with the use of multi-isoform/multi-gene targeting to address redundant paralogs and pathways in cancer cells.

The case for multi-gene targeting

Many RNAi screens use arrayed single gene knockdowns to find genes that play an important role in a biological process. The idea is that a single bullet is enough to take down its target leaving a gaping hole that one cannot fail to notice. In some cases, this is true, and is certainly relied upon by drug developers seeking to create specific mono-target drugs.  However, in complex diseases like cancer, cells have evolved fail-safe mechanisms to make them more resistant to external assaults. A single bullet is simply not enough.

Take for example oncogenic protein RAF or Rapidly Accelerated Fibrosarcoma, a tyrosine kinase effector that is a component of the MAPK signalling pathway (Ras-Raf-MEK-ERK). RAF has three isoforms – ARAF, BRAF and RAF1 (also called CRAF). Studies in mouse embryonic development show they all share some form of functional redundancy as knocking out two isoforms produces more severe effects than knocking out each isoform alone.

Screens that target single genes/isoforms therefore tends to bias results towards genes that have no paralogs or only have single isoforms. This was indeed the reason why classical Ras effectors were not identified in previous screens.

Factors to consider in a multi-gene targeting RNAi screen

Determining gene combinations that make sense

The authors of the study did a focussed siRNA screen on 41 RAS effector nodes represented by 84 genes. Out of the 41 nodes, 25 of them had 2-4 functional paralogs where combinatorial gene silencing was carried out with multiple siRNAs. 5 nodes knocked down multiple members of a protein complex. 5 nodes had siRNAs targeting multiple steps within a pathway. Only 6 nodes silenced single genes (highlighted).

Multi-gene targeting screen design

The only caveat with designing such a screen is the requirement for prior knowledge to perform meaningful gene silencing combinations. In this instance, many of the Ras effector pathways are characterized sufficiently to do this well however in other less studied fields, this could be a challenge. Useful tools that would help in designing gene knockdown combinations would include pathway or phenotype databases such as KEGG, REACTOME or Wikipathways. The Phenovault which siTOOLs Biotech is developing, is yet another potentially useful tool.. more details to come!

Number and types of phenotypes

The authors also highlight how a screen that reads only one phenotype might miss other important gene functions. Many RNAi screens sadly still stick to measuring cell proliferation as their only read-out which is greatly influenced by siRNA off-target effects. Here, 5 different phenotypes were measured (cell size, proliferation, apoptosis, reactive oxygen species [ROS], and viability). It was noted that silencing of Cdc42 had little effect on cell viability yet a prominent effect on ROS levels.

To take this up a notch, analysis was also performed at the single-cell level in cells expressing uniform levels of GFP and co-transfected with GFP siRNA. This allowed authors to correlate phenotypes with levels of gene knockdown, generating dose-response curves. How clever!

A lot more work, but adds to data robustness especially when using single siRNAs that are known to be rather variable.

Heterogeneity of cell lines

Many reports and our own observations attest to the heterogenous response of different cell lines to the same treatment. In cancer especially, the large heterogeneity necessitates the use of multiple cell lines. Not doing so would be failing to account for the large genetic diversity observed in the clinic. The authors screened 92 cell lines derived from lung, pancreas and colorectal tissue.

Despite seeing heterogenous responses to node knockdowns, phenotypic responses could be distinguished into  several groups based on effector engagement.  A major group dependended on RAF through direct binding with KRAS, a second major group worked via RSK p90 S6 kinases to drive RSK-mTOR signalling. And a third minor group was dependent on RalGDS. They went on to focus on the first two major groups, naming them KRAS-type and RSK-type respectively.

Reagents – choosing siRNAs and siRNA concentrations

The authors used previously characterized siRNAs to select for more potent siRNAs. This involved an RNAi sensor reporter-based assay that required the generation of 20,000 clones. The reporter was also shRNA-based. Due to heterogeneity in Dicer-mediated cleavage of shRNA, its uncertain if knockdown potency is accurately reflected when translated to siRNAs (read about the difference between shRNAs and siRNAs).

siRNA off-target effects are concentration-dependent

In any case, its a lot of work to characterize all siRNAs to be used in a screen. Furthermore, off-target effects are not addressed.

The authors stuck to a maximal concentration of 12 nM where 2 nM of siRNA was applied per gene. At 2 nM per siRNA, one still risks deregulating other genes. One of the first papers by Aimee Jackson et al., demonstrated an siRNA targeting MAPK14 deregulated many other genes even at concentrations of 1-4 nM.

An important consideration is to ensure total siRNA concentrations are kept constant. In which case, a negative control siRNA has to match or follow the maximal siRNA concentration used. Using different levels of siRNAs runs the risk of biasing off-target effects towards sequences present at higher concentrations.

To learn what the causes, extent and consequences of siRNA off-target effects are, read siTOOLs Technote 1)

Validating results

As with all scientific hypothesis, it helps to arrive at the same conclusion with different approaches.

The two different effector response subgroups identified also responded differently to small molecules. The KRAS-type lines being more sensitive to EGFR and ERK inhibition while the RSK-type lines more sensitive to inhibitors of PDK1, RSK, MTOR, S6K1 and DNA repair enzymes. This was attributed to the latter’s higher basal metabolic activity manifested in larger investments towards oxidative phosphorylation and mitochondrial ribosome maintenance.

By also projecting signatures obtained from cell lines into patient samples (in The Cancer Genome Atlas, TCGA), the subtypes were also effective at predicting differential sensitivity to multiple drug treatments. This highlights the importance in designing effective drug combinations in cancer.

Interestingly, the authors also performed CRISPR pooled screens in parallel. However, due to the restraints of being only able to knockout 1 gene at a time, smaller effects were seen due to gene redundancy. However, they did go on to use CRISPR as well to mutate key genes to affirm the pathway relationships established.

siPOOLs have been used successfully for multi-gene targeting for up to 4 genes, and potentially more. They also safely address off-target effects by high complexity pooling, enabling each siRNA to be applied at picomolar concentrations. For more articles on multi-gene targeting, read an older blogpost:

Understanding gene networks with combinatorial gene knockdown

 

Follow us or share this post:
Is it important to avoid microRNA binding sites during siRNA design?

Is it important to avoid microRNA binding sites during siRNA design?

Summary: To address the question of whether one should avoid microRNA binding sites during siRNA design, we examined whether removing siRNAs that share seeds with native microRNAs would reduce the dominance of seed-based off-target effects in RNAi screening.

siRNA design and native microRNA target sites

Recently, we discussed a review of genomics screening strategies.  The authors state:

RNAi screens are powerful and readily implemented discovery tools but suffer from shortcomings arising from their high levels of false negatives and false positives (OTEs) as can be seen when comparing the low concordance among the candidate genes detected in different screens using the same species of virus, e.g., HIV-1, HRV, or IAV (Booker et al., 2011; Bushman et al., 2009; Hao et al., 2013; Perreira et al., 2015; Zhu et al., 2014).

To address these concerns, improvements in the design and synthesis of next-gen RNAi library reagents have been implemented including the elimination of siRNAs with seed sequences that are complementary to microRNA binding sites.

Given that off-target effects via microRNA-like binding are the main source of RNAi screening phenotypes, avoiding native microRNA sites during siRNA design seems like a reasonable strategy.  But does it make much difference in actual RNAi screens?

Hasson et al. 2013 performed a mitophagy screen using the Silencer Select siRNA library.  About 12% of the ~65,000 screened siRNAs have a 7-mer seed shared by a miRBase microRNA.

The screen’s main phenotypic readout, % Parkin translocation (PPT), is strongly affected by seed effects.   The intra-class correlation for siRNAs with the same seed is ~.51 (versus ~.06 for siRNAs with the same target gene).  There appears to be no difference between how siRNAs with or without microRNA seeds behave:

Is it important to avoid microRNA binding sites during siRNA design?

The same thing is found if we look at a less specific phenotype like cell count (which should be more broadly susceptible to off-target effects, as more genes should affect this phenotype):

Is it important to avoid microRNA binding sites during siRNA design?

And if we look at seeds that are enriched at the top of the screening list (sorted by descending PPT), we also don’t see much difference between siRNAs with or without native microRNA seeds.  (Note that the seed p-value is calculated in a similar way to RSA, based on how over-represented a seed is towards the top of a ranked list)

Is it important to avoid microRNA binding sites during siRNA design?

We also examined a general phenotypic readout (cell viability) in a dozen large-scale RNAi screens.

For some screens, we do see a slight shift in the values for siRNAs with or without native microRNA seeds.

For example, a genome-wide screen of Panda et al. 2017 (also using the Silencer Select library) shows a slight decrease in viability for siRNAs with native microRNA seeds:

Is it important to avoid microRNA binding sites during siRNA design?

Removing those siRNAs does not change the dominance of seed-base off-targets.

The intra-class correlation (ICC) for siRNAs with the same 7-mer seed is ~.53, with or without the inclusion of siRNAs with native microRNA seeds, while ICC for siRNAs with the same target gene is only  ~.06.

Coming back to the quote from the review article on genomic screening, next-gen RNAi library reagents that avoid native microRNA seeds are not expected to be much better than siRNAs that include them.

The most effective way to avoid seed-based off-target effects is to use high-complexity siRNA pools (siPOOLs). Learn more about siPOOLs

 

Follow us or share this post:
Correcting seed-based off-target effects in RNAi screens

Correcting seed-based off-target effects in RNAi screens

Summary: Correcting for seed-based off-targets can improve the results from RNAi screening.  However, the correlation between siRNAs for the same gene is still poor and the strongest screening hits remain difficult to interpret.

Seed-based off-target correction has little effect on reagent reproducibility

Given that seed-based off-targets are the main cause of phenotypes in RNAi screening, trying to correct for those effects makes good sense.

The dominance of seed-based off-targets means that independent siRNAs for the same gene usually show poor correlation.

If one could correct for the seed effect, the correlation between siRNAs targeting the same gene may improve.

One straightforward way to do seed correction is to subtract the ‘seed median’ from each siRNA.  (The seed median is the median for all siRNAs having the given seed.)

This was the approach used by Grohar et al. in a recent genome-wide survey of EWS-FLI1 splicing (involved in Ewing sarcoma).  They used the Silencer Select library, which has 3 siRNAs per target gene.

After seed correction, there is only minor improvement in the correlation between siRNAs targeting the same gene.  The intra-class correlation (ICC) improves from 0.031 to 0.037.  The ICC for siRNAs with the same 7-mer seed decreases from 0.576 to 0.261.

Although we have reduced the seed-based signal, it has not resulted in a correspondingly large improvement in the gene-based signal.

More sophisticated seed correction can improve reagent correlation

Grohar et al. used a simple seed-median subtraction method to correct their screening results.

A more sophisticated method (scsR) was developed by Franceshini et al. for seed-based correction of screening data.  It corrects using the mean value for siRNAs with the same seed, and weighs the correction using the standard deviation the values.  This allows seeds with a more consistent effect to contribute more to the data normalisation.

Applying the scsR method to the Grohar data, ICC for siRNAs targeting the same gene increases from 0.031 to 0.041.  It is better than the increase with seed-median subtraction (0.037), but is still only a fairly minor improvement (plot created using random selection of 10,000 pairs of siRNAs that target the same gene):

 

Off-target correction increases double-hit rate in top siRNAs of RNAi screen

The following plot shows the count for single-hit and double-hit genes as we go through the top 1000 siRNAs (of ~60K screened in total).  Double-hit means that the gene is covered by 2 (or more) hit siRNAs.

Despite the small improvement in reagent correlation, the double-hit rate is essentially the same using simple seed-median subtraction or the more advanced scsR method.

Furthermore, the number of double-hits is higher than what we’d expect by chance.

This shows that, despite the noise from off-target effects, there is some on-target signal that can be detected.

siRNAs with the strongest phenotypes remain difficult to interpret

Despite the fact that the double-hit count is higher than expected by change, most of the genes targeted by the strongest siRNAs are single-hits.  siRNAs with the strongest phenotypes remain difficult to interpret.

Seed correction is best suited for single-siRNA libraries.  Low-complexity pools, like siGENOME or ON-TARGETplus, are less amenable to effective seed correction since there are (usually) 4 different seeds per pool.  This reduces the effectiveness of seed-based correction, even though seed-based off-target effects remain the primary determinant of observed phenotypes (as discussed here, here , and here).

The best way to correct for seed-based off-targets is to avoid them in the first place.  Using more specific reagents, like high-complexity siPOOLs, is the key to generating interpretable RNAi screening results.

For help with seed correction or other RNAi screening data analysis with the Phenovault, contact us at info@sitools.de

Follow us or share this post:
Little correlation between Dharmacon siGENOME and ON-TARGETplus reagents

Little correlation between Dharmacon siGENOME and ON-TARGETplus reagents

The most common way to validate hits from Dharmacon siGENOME screens is to test the individual siRNAs from candidate pool hits (siGENOME reagents are low-complexity pools of 4 siRNAs).  In this deconvolution round, we normally see that the individual siRNAs for genes behave very differently and seed effects dominate (discussed here and here).

One could argue that deconvolution is not the correct way to validate candidate hits (even though it’s the method recommended by Dharmacon),  as testing the siRNAs individually will result in seed effects that are suppressed when the siRNAs are pooled.  One problem with this argument is that low-complexity pooling does not get rid of off-target effects (e.g. Fig 5 in this paper), something that is better done via high-complexity pooling.  But assuming it were true, validating with a second Dharmacon pool would be better.

Tejedor et al. (2015) performed a genome-wide Dharmacon siGENOME screen for regulators of Fas/CD95 alternative splicing.  ~1500 genes were identified by a deep-sequencing approach.  ~400 of those were confirmed by high-throughput capillary electrophoresis (HTCE, LabChip).  They then retested those ~400 genes (again by HTCE) using Dharmacon ON-TARGETplus pools.

The following plot shows the values for the siGENOME and ON-TARGETplus pools for the same genes (i.e. each point corresponds to 1 gene).

What’s measured is the percent of splice variants that include exon 6 following siRNA treatment.  That was compared to the values for a plate negative control (untransfected wells) and converted to a robust Z-score.  This is the main readout from the paper.

 

The Pearson correlation improves if the strong outlier at -150 for siGENOME is removed (R = 0.25), while the Spearman correlation is unchanged.

 

We see that a fairly small number of genes are giving reproducibly strong phenotypes (e.g. 13 of 400 have robust Z-scores less than -15 for both siGENOME and ON-TARGETplus reagents).

If we remove those 13 strong hit genes, the correlation approaches zero:

 

Even if the strong outlier for siGENOME is removed, the correlation is still near zero:

 

Although using a second Dharmacon pool removes some of the arbitrariness of defining validated hits (e.g. saying that 3 of 4 siRNAs must exceed a Z-score cut-off of X, or 2 of 4 siRNAs must exceed a Z-score cut-off of Y), the end result is similar:  A few strong  genes show reproducible phenotypes, while many of the strongest screening hits show inconsistent results.  The main problem, off-target effects in the main screen, is not fixed.

postscript

Tejedor et al. say that 200 genes were confirmed by ON-TARGETplus validation.  They consider a gene confirmed if the absolute value of the robust Z-score is greater than 2.  The Z-score is calculated using the median for untransfected plate controls.  I suspect that a significant proportion of randomly selected genes would also have passed this cut-off.

In table S3 (which has the ON-TARGETplus validation results), there are actually only 177 genes (including 2 controls) that meet this cutoff.  The supplementary methods state: Genes for which Z was >2 or <-2 were considered as positive, and a total number of 200 genes were finally selected as high confidence hits.

Which suggests that genes outside the cut-off were chosen to bring the number up to 200.

But if we look at the Excel sheet with the ‘200 hit genes’, it has 200 rows, but only 199 genes.  The header was included in the count.

This type of off-by-one error is probably not that uncommon.  In a case like this, it does not matter so much.

One case where it did matter was in the Duke/Potti scandal.  The forensic bioinformatics work of the heroes of the Duke scandal found that, when trying to reproduce the results from published software, one of the input files caused problems because of an off-by-one error created by a column header.  That was one of many difficulties in reproducing the Potti paper’s results which eventually led to its exposure.

Follow us or share this post:
Orthogonal design in software and RNAi screening

Orthogonal design in software and RNAi screening

The software engineering classic The Pragmatic Progammer popularised the benefits of orthogonality in software design.  They introduce the concept by describing a decidedly non-orthogonal system:

You’re on a helicopter tour of the Grand Canyon when the pilot, who made the obvious mistake of eating fish for lunch , suddenly groans and faints. Fortunately, he left you hovering 100 feet above the ground. You rationalize that the collective pitch lever [2] controls overall lift, so lowering it slightly will start a gentle descent to the ground. However, when you try it, you discover that life isn’t that simple. The helicopter’s nose drops , and you start to spiral down to the left. Suddenly you discover that you’re flying a system where every control input has secondary effects. Lower the left-hand lever and you need to add compensating backward movement to the right-hand stick and push the right pedal. But then each of these changes affects all of the other controls again. Suddenly you’re juggling an unbelievably complex system, where every change impacts all the other inputs. Your workload is phenomenal: your hands and feet are constantly moving, trying to balance all the interacting forces.

[2] Helicopters have four basic controls. The cyclic is the stick you hold in your right hand. Move it, and the helicopter moves in the corresponding direction. Your left hand holds the collective pitch lever. Pull up on this and you increase the pitch on all the blades, generating lift. At the end of the pitch lever is the throttle . Finally you have two foot pedals, which vary the amount of tail rotor thrust and so help turn the helicopter.

As the authors explain:

The basic idea of orthogonality is that things that are not related conceptually should not be related in the system. Parts of the architecture that really have nothing to do with the other, such as the database and the UI [user interface], should not need to be changed together. A change to one should not cause a change to the other.

This applies to many types of design, not just for computer systems.  The plumber should not have to depend on the electrician to fix a broken pipe.

The principle has also been used in RNAi screening, notably by Perreira et al. who introduce the MORR (Multiple Orthologous RNAi Reagent) method to increase confidence in screening hits.  Comparing the results of siRNAs from different manufacturers  is important, but because they operate by the same mechanism (including the off-target effect), they are not really orthologous.  More orthologous would be the comparison between RNAi and CRISPR experiments, which sometimes show discrepancies that point to interesting biology.

To confirm RNAi screening hits, ‘partial orthogonality’ may be preferable.  If screening hits are due to either on-target or off-target effects, confirmation with RNAi reagents that only have one or the other would be better than using CRISPR, where it is difficult to interpret the reason for discrepancies (e.g. is there no phenotype  because of genetic compensation?).

One could use C911s to create a version of the siRNA that, in theory, maintains off-target effects but eliminates on-target effects.  We have observed, however, that C911s often give substantial knockdown of the original target gene (in some ways, C911s are like very good microRNAs).  To be sure that a positive effect with C911s is not due to partial knockdown, one would also need to test that via qPCR.  C911s can create a lot of work.

Far better would be to confirm screening results with siPOOLs, which provide robust knockdown and minimal off-target effects.

One place RNAi practitioners would hope not to find orthogonality is the relationship between on-target knockdown and phenotypic strength.

Since the early days of RNAi, positive correlation between knockdown and phenotypic strength has been suggested as a means to confirms screening results.  Reagents with a better knockdown should give a stronger phenotype.

To test this, we obtained qPCR data for over 2000 siRNAs (Neumann et al.) and checked the performance of those siRNAs against the designated hit genes from an endocytosis screen (Collinet et al.).

If the siRNAs work as expected, those siRNAs with better knockdown should give stronger phenotypes than those with weaker knockdown.

There were 100 genes from the Collinet hits for which there were 3 siRNAs with qPCR data.

For those 100 siRNAs triplets, we compared the phenotypic ranks with the knockdown ranks.  (We were agnostic about the direction of phenotypic strength, and checked whether knockdown and phenotype were consistent when phenotype scores were ranked in either ascending or descending order).  For example, if siRNAs A, B, and C have phenotypic scores of 100, 90, 70 and knockdown of 15%, 20%, 30% remaining mRNA, we would say that phenotypic strength is consistent with knockdown (and because we were agnostic about phenotypic direction, we would also say it was consistent if siRNAs A, B, and C had scores of 70, 90, 100).

The observed number of cases where knockdown rank was consistent with phenotypic rank was then compared to an empirical null distribution, obtained by first randomising the knockdown data for the siRNA triplets before comparison to phenotypic strength.  This randomisation was performed 300 times.  This provides an estimate of what level of agreement between knockdown and phenotype would be expected by chance.  The standard deviation (SD) from this null distribution was then used to convert the difference between observed and expected counts into SD units.

The Collinet dataset provides data for 40 different features.  The above procedure was carried out for each of the 40 features.

To take one feature (Number vesicles EGF) as an example, we observed 34 cases where knockdown was consistent with phenotypic strength.  By chance, we would expect 33.4 (with a standard deviation of 4.9).  The difference in SD units is (34-33.4)/4.9 = 0.1.

As can be seen in the following box plot, the number of SD units between observed and expected counts of knockdown/phenotype agreement for the 40 features is centered near zero (median is 0.1 SD units):

This suggests that there is very little, if any, enrichment in cases where siRNA knockdown strength is correlated with phenotypic strength.  The orthogonality between knockdown and phenotype, given the poor correlation between siRNAs with the same on-target gene, is unfortunately not unexpected.

Follow us or share this post:
“Phenoville” – RNAi & CRISPR Screening Strategies

“Phenoville” – RNAi & CRISPR Screening Strategies

Pleasantville is a movie based on an interesting idea: two teenagers are magically transported through their TV to a town called Pleasantville set in the 1950s where everything is perfect (and also black-and-white).  As they discover the complex, imperfect emotions hidden below the idyllic surface, the black-and-white characters and objects start to gain colour.

In loss-of-function genetic screening, some reagents and screening formats may also give rise to a narrow, black-and-white view of a biological process.  A sort of “Phenoville”.  This was illustrated nicely in a recent review of screening strategies for human-virus interactions by Perreira et al. (2016).

The authors performed screens for human rhinovirus (HRV) infection using arrayed RNAi reagents (siRNAs) and pooled CRISPR reagents (sgRNAs), and then compared the resulting hit lists.

The arrayed RNAi screen produced over 160 high-confidence candidate genes, whereas the CRISPR screen only found 2.  The authors comment:

“The comparison of these two screening approaches side-by-side, using the same cells and virus, raises an interesting point. The number of host factors found for HRV14 was far greater using the MORR/RIGER approach [i.e. RNAi performed with multiple orthologous RNAi reagents and analysed by RNAi gene enrichment ranking method] and is approaching a systems level understanding based on bioinformatic analyses and the near saturation of, or enrichment for, multiple complexes and pathways (Fig. 4) (Perreira et al., 2015). By comparison our matched pooled CRISPR/Cas9 screen for HRV-HFs yielded two high-confidence candidates based on reagent redundancy, ICAM1, the known receptor for HRV14, and EXOC4, a gene involved in exocyst targeting and vesicular transport (He & Guo, 2009). Given the known role of ICAM1 as the host receptor for most HRVs, these results point to entry as the major viral lifecycle stage interrogated by a pooled functional genomic screening approach using a population of randomly biallelic null cells infected by a cytopathic virus.”

In simple terms, RNAi screening produced a richer data set that revealed system level interactions whereas CRISPR screening yielded a small number of specific hits that only affected an early-stage pathway. The ‘systems level understanding’ is nicely shown in the following diagram of the RNAi hits.  The red box at the top left is the only gene (ICAM1) that was common to the RNAi and CRISPR screens.

Perreira et al. conclude that arrayed siRNA screens permit the detection of a larger number of viral dependency factors, albeit with a significant tradeoff in a greater number of false positive hits (mainly due to off-target effects).  In contrast, pooled screens with CRISPR sgRNAs using cell survival as a readout, as also seen with most haploid cell screens, display limited sensitivity but excellent specificity in finding host genes that act early on in viral replication (e.g. ICAM1).

In Perreira et al.‘s words:

“… given the currently available functional genomic strategies if the goal is to find viral entry factors (e.g., host receptors) with high specificity its best to use a pooled survival screen, but alternatively if the aim is to obtain with relative ease a more comprehensive set of host factors, albeit with more prevalent false positives, than an arrayed siRNA screen would be the preferred method.”

Summarizing two options for genetic screeners:

  1. Arrayed RNAi screens
    • provide a richer view of the underlying biology
    • produce more false positives from OTEs
    • produce false negatives from OTEs
  2. Pooled CRISPR screens
    • provide a narrower view of the underlying biology
    • produce fewer false positives
    • produce false negatives because of genetic compensation

Off-target effects (OTEs) are the primary cause of false positives, and the resultant higher assay noise also increases the number of false negatives in arrayed RNAi screens. Reagents like siPOOLs minimize the risk of off-target effects and reduce assay noise.

One key factor not mentioned by Perreira et al. is the presence of genetic compensation in gene knockout approaches.

Putting genetic compensation in terms of human actors, imagine that you are investigating the function of bus drivers in Pleasantville.  To induce loss-of-function, assume that aliens will be abducting the bus drivers.  If the bus drivers are abducted in their sleep (equivalent to a CRISPR knock-out), you may not get a good idea of their function when you film the next day.  People may be compensating by driving, biking or staying home.  Alternatively, the bus company may have found emergency replacement drivers.

Now suppose the bus drivers are abducted in the middle of the day while driving their routes (equivalent to an RNAi knock-down).  The film will show buses crashing (hopefully without any serious injuries, since this is just a TV show!) and the public transportation system will suddenly come to a halt.

RNAi gene knockdown screens with siPOOLs  can provide a significant advantage over CRISPR gene knockout screens in obtaining a system level understanding in biological models.

Want to receive regular blog updates? Sign up for our siTOOLs Newsletter:

Follow us or share this post:
siRNA vs shRNA – applications and off-targeting

siRNA vs shRNA – applications and off-targeting

Short interfering RNA (siRNA) and short hairpin RNA (shRNA) are both used in RNAi-mediated gene silencing. In this blogpost, we explore the differences in applications of siRNA and shRNA and compare their capacity for off-targeting.

For a summary of their properties, please refer to Table 1 at the end of  the post.

In what situations should we use siRNA or shRNA?

In terms of application, siRNAs are commonly applied for rapid and transient knockdown of gene expression.

It is performed in cell lines amenable to transfection by liposomes/electroporation and effects typically last from 3-7 days though retransfection can be performed to extend the effect.

The amount of siRNA introduced can be highly controlled and efficiency of gene knockdown is dependent on the levels of siRNA in the cell which is influenced by transfection efficiency and siRNA stability. Knockdown is also influenced by characteristics of the gene. A gene that is highly transcribed for example, may experience less siRNA-mediated downregulation compared to a gene where lesser copies of RNA are produced over time. In addition, a gene which expresses a protein with a very long half-life, may require extended periods of siRNA application to see a knockdown effect.

Due to the transient effect of siRNAs, shRNAs were developed to be used for prolonged knockdown of genes.

As they are introduced by viral vectors, cells that are more difficult to transfect are better targeted with shRNA. Furthermore, promoter-driven expression allows for inducible expression of the shRNA. Depending on the viral vector used – refer to Labome’s post that covers siRNA/shRNA delivery in greater detail – the shRNA may be integrated into the host genome, allowing it to be propagated into daughter cells. This maintains a consistent gene knockdown over several generations. However, knockdown efficiency can decline over time. This is mainly due to varying levels of uptake of the shRNA among cells, with a cell population having lower shRNA expression being over-represented with time.

 

What about RNAi screening?

siRNAs and shRNAs are both used in RNAi screening to identify genes of interest in a studied phenotype. These are performed with siRNA/shRNA libraries that target a large variety of genes. There are two RNAi screening formats commonly used – arrayed and pooled.

siRNAs and shRNAs can both be used in an arrayed screening format. This means that the siRNA(s)/shRNA(s) against each gene is tested in distinct cell populations. Arrayed screens have the advantage of being compatible with various phenotypic readouts and do not suffer from possible reagent cross-talk or challenges associated with deconvoluting data. However, they are more energy and resource-intensive to perform. (See Fig. 2)

The pooled screening format in contrast, applies only with shRNAs. Here, all shRNAs (e.g. a whole-genome shRNA library) are introduced to a single cell population. As low titers of viral vectors are used, each cell in the population is expected to take up one shRNA vector.

With pooled screening, only readouts linked to cell number can be assessed. These include measurements for cell viability or altered expression of a cell surface marker assessed by fluorescence activated-cell sorting. shRNAs targeting genes which impact these readouts are expected to skew the cell population, such that only cells affected by the relevant shRNAs can be identified. This is either through negative selection, where lost cell populations are noted, or positive selection, where cells with certain shRNAs become over-represented.

The resulting cell population is then assessed by PCR, microarray hybridization or next generation sequencing to measure which shRNAs are highly or lowly-represented. The shRNAs are identified usually by means of a DNA barcode present in the vector sequence. Of note, pooled screens take up less resources to perform but require longer assay times to allow for significant changes in the overall cell population to occur.

Fig. 2 Simplified workflow for arrayed and pooled RNAi screening formats

 

Off-target effects with shRNAs?

The use of siRNAs are known to produce several off-target effects but what about shRNAs? Given they are processed the same way as siRNAs, shRNAs are also subject to microRNA-like off-target effects. In addition, because they are expressed from DNA and rely on endogenous machinery to be processed into siRNA, several variations may be introduced not found with introducing siRNA directly. Some potential sources of off-target effects for shRNAs include:

1. Promoter-driven expression. shRNAs are typically controlled with a U6 promoter which drives high levels of transcription via RNA polymerase III. The high shRNA expression levels may saturate endogenous RNAi machinery, contributing to off-target effects. To counter this, shRNAs can be expressed in a context mimicking miRNAs, utilizing RNA polymerase II for transcription instead. This has been found by several groups to reduce the incidence of off-target effects (Grimm et al., 2006, Kampman et al., 2015)

2. Dicer-mediated hairpin processing. shRNAs undergo Dicer-mediated cleavage in the cytosol to remove its hairpin loop. Gu et al., 2012 reported that Dicer cleaves with sufficient heterogeneity to generate multiple sequences. This factor was reported to generate the higher noise levels unique to shRNA screens (Bhinder and Djaballah, 2013). As specificity of Dicer cleavage is influenced by neighbouring loop and bulge structures, care should be taken in shRNA design.

3. Multiple shRNA uptake. During viral transduction, the viral titer is minimized to increase the probability that cells take up a single shRNA vector. However, this does not guarantee that multiple shRNA uptake will not occur. In this event, a combinatorial gene knockdown ensues resulting in a mixed phenotype that may generate false hits.

4. Differences in genomic integration between shRNAs. Varying efficiencies in transfection and genome integration between shRNAs may skew results to over-represent certain shRNAs over others, especially in pooled screens. Furthermore, integration into the host genome may disrupt the function of certain genes, producing more off-targets.

Studies comparing results from siRNA and shRNA screens have found extremely poor overlap, both between and within the reagent-specific screens. Bhinder and Djaballah’s (2013) analysis of results from 30 published RNAi screens (16 siRNA, 14 shRNA) searching for genes that impact cell viability saw no common genes identified across the board. Furthermore, different genes were identified depending on whether the screen used siRNA or shRNA. PLK1 for example, was a prominent hit for siRNA screens but was only marginally represented in shRNA screens. In contrast, KRAS was a top hit among shRNA screens.

Fig. 3 Reagent format of RNAi screens analysed in Bhinder and Djaballah, 2013 Screens were performed either with genome-wide (GW) or focused (FD) siRNA/shRNA libraries. For siRNA screens, Pooled refers to pools of 3 siRNAs applied together compared to Singles where a single siRNA duplex was applied. For shRNA screens, Pooled refers to a pooled format screen (Fig. 2) where ~50, 000 shRNAs were applied to a single cell population. Arrayed refers to arrayed format screen where shRNAs were applied individually (Fig. 2).

Fig. 4 Overlap of hits among genome-wide (left) and focused (right) siRNA screens (Bhinder and Djaballah, 2013) Only 4 common hits detected across the 2 lethal gene lists from genome-wide siRNA screens. In focused siRNA screens, a greater overlap was detected but still limited across the 22 lethal gene lists. PLK1 detected in 9 out of 22 gene lists.

Fig. 5 Overlap of hits among genome-wide (left) and focused (right) shRNA screens (Bhinder and Djaballah, 2013) KRAS was a top hit in shRNA GW screens, appearing in 5 out of 9 lists. In focused shRNA screens, KRAS was present in 15 out of 31 lists. 

Worryingly, an enrichment of gene candidates exclusive to pooled shRNA screens was observed as opposed to arrayed shRNA or siRNA screens. Most of the overlap seen in gene lists (80% global overlaps, 60% after stringent filtering) were specific to pooled shRNA screens. Exclusion of data from pooled shRNA screens would have reduced overlap to a mere 27%. This indicates gene targets obtained from shRNA pooled screens is specific to the technique as opposed to specific gene downregulation.

Furthermore, a greater number of hits were obtained from shRNA screens – 6664 candidates from 40 shRNA gene lists – as opposed to 1525 candidates from 24 siRNA gene lists. This indicates a generally noisier dataset associated with shRNA screens.

Bhinder and Djaballah later performed a head-to-head comparison of an arrayed siRNA and shRNA screen and reported similarly dismal results. Despite using a gain-of-function assay, which tends to yield clearer results, only a 29 hit overlap was seen between siRNA and shRNA libraries which shared 15,068 common genes. Based on a known set of positive controls, siRNAs identified 8 known regulators as opposed to shRNA which only identified 3. Furthermore, predicted siRNA sequences obtained after Dicer-processing of shRNA which corresponded to exactly the same siRNA sequence from the siRNA library yielded different phenotypes. The authors highlight that differential intracellular processing of the shRNA contributes significantly to the discrepancies observed.

It is evident that shRNAs are at risk to greater number of off-target effects than siRNAs. Much care should be taken towards the interpretation of pooled shRNA screens in particular. Secondary validation of gene hits plays an increasingly important role. It is recommended to validate gene hits with siPOOLs (high-complexity, defined siRNA pools) which have a lower off-target profile than single siRNAs or low complexity siRNA pools of 3-4. siPOOL-resistant rescue constructs enable further affirmation that the loss-of-function phenotype is attributed to the target gene. Alternative tools such as compounds, antibodies or gene knockout technologies are also highly recommended.

Want to receive regular blog updates? Sign up for our siTOOLs Newsletter:

Table. 1 Comparison of properties between siRNA and shRNA

siRNA shRNA
Structure 20-25 nucleotide long double-stranded RNA (dsRNA) with 2 nucleotide overhangs at the 3’ end

~57-58 nucleotide long RNA sequence with a dsRNA region linked by non-pairing nucleotides to form a stem-loop structure

Delivery RNA itself with liposome/electroporation-mediated delivery into cells Usually delivered to cells via viral vectors. DNA may be incorporated into host genome depending on viral vector used.
Processing In the cytosol, guide or antisense strand* (shown in blue in Fig. 1) is incorporated into RNA induced silencing complex (RISC). RISC is guided towards RNA transcripts with the complementary sequence to mediate cleavage and subsequent degradation of the transcript.

 

*Note that the sense strand may also load into RISC and mediate off-targeting but incidence of this is reduced by designing siRNA with  appropriate thermodynamic properties (refer to previous blogpost on siRNA design)

In the nucleus, shRNA is transcribed from DNA by either RNA polymerase I or III, depending on the promoter.

Drosha, a member of the ribonuclease III family, processes the RNA transcript of its long flanking single-stranded RNA sequences and the resultant shRNA is exported out of the nucleus by Exportin-5.

 

In the cytosol, the enzyme Dicer cuts off the hairpin loop of the shRNA and releases the functional active siRNA which follows the same downstream processing as siRNAs.

 

Length of expression Varies from 3-7 days. Affected by degradation of siRNA within cell and dilution of effect upon cell division. Expression can be reinstated by re-transfecting the siRNA. If the DNA is stably integrated in the host genome, knock-down is theoretically permanent.
Control of knockdown Easily controlled by varying amount of siRNA introduced. Magnitude of knockdown harder to control as determined by promoter-driven efficiency and shRNA vector uptake. Expression however can be made inducible with Tet-on/off systems.

 

Follow us or share this post:

Like what you see? Mouse over icons to Follow / Share