Upregulation of the proto-oncogene Twist1 is highly correlated with acquired drug resistance and poor prognosis in human cancers. Altered expression of this multifunctional transcription factor is also associated with inherited skeletal malformations. The mammalian Twist1 3′UTRs are highly conserved and contain a number of potential regulatory elements including miRNA target sites. We analyzed the translational regulation of TWIST1 using luciferase reporter assays in a variety of cell lines. Among several miRNAs tested, miR-145a-5p, miR-151-5p and a combination of miR-145a-5p + miR-151-5p and miR-151-5p + miR-337-3p were able to significantly repress Twist1 translation. This phenomena was confirmed with both exogenous and endogenous miRNAs and was dependent on the presence of the predicted target sites in the 3′UTR. Furthermore, the repression was sensitive to LNA-modified miRNA antagonists and resulted in decreased migratory potential of murine embryonic fibroblast cells. Understanding the in vivo mechanisms of this oncogene's regulation might open up a possibility for therapeutic interference by gene specific cancer therapies.
Citation: Nairismägi M-L, Füchtbauer A, Labouriau R, Bramsen JB, Füchtbauer E-M (2013) The Proto-Oncogene TWIST1 Is Regulated by MicroRNAs. PLoS ONE 8(5): e66070. doi:10.1371/journal.pone.0066070
Editor: Thomas Preiss, The John Curtin School of Medical Research, Australia
Received: May 11, 2012; Accepted: May 6, 2013; Published: May 31, 2013
Copyright: © 2013 Nairismägi et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by The Danish Cancer Society grant DP08036 (www.cancer.dk). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The evolutionary conserved basic helix-loop-helix (bHLH) transcription factor TWIST1 is a multifunctional proto-oncogene with a strong correlation to poor prognosis. TWIST1 is able to inhibit c-MYC induced apoptosis  and directly regulates the expression of several other oncogenes such as GLI1 , miR-10b  and AKT2 . Overexpression of TWIST1 has been observed in various types of cancer such as breast, prostate, gastric, pancreatic and bladder cancer, hepatocarcinoma, rhabdomyosarcoma, and glioma and is often associated with more aggressive phenotypes, and acquired drug resistance (reviewed in ).
Twist1 expression is regulated by a complex network of signals and has been described as an integrator of SHH, FGF and BMP signaling . In mice, a genomic fragment containing 6 kb upstream and 1.5 kb downstream of the Twist1 gene is not sufficient to recapitulate Twist1 expression during embryogenesis (unpublished data) which is consistent with the observation that a translocation breakpoint 3 kb downstream of the Twist1 gene creates a comparable haploid insufficiency as the null allele . However, a number of transcription factor binding sites have been identified in the Twist1 upstream region and direct binding of transcriptional activators like NF-κB  and repressors like Prospero-related homeobox 1 (PROX1)  has been shown.
TWIST1 is directly upregulated by hypoxia-inducible factor-1α (HIF-1α) and HIF-2α , . Intratumoral hypoxia is correlated with radiation therapy resistance and enhanced metastatic potential . TWIST1 promotes tumor metastasis  by epithelial-mesenchymal transition (EMT)  and formation of invadopodia, the specialized membrane protrusions for extracellular matrix degradation . TWIST1 is linked to the transcription factor SNAIL, which also induces EMT, in a positive feedback loop . In addition, co-expression of TWIST1, HIF-1α and SNAIL has been correlated with metastasis and poor prognosis in primary tumors of head and neck squamous cell carcinoma (HNSCC) patients .
Originally, the Twist gene was discovered as a mutation disturbing cellular motility and EMT during gastrulation in Drosophila melanogaster , . A comparable phenotype is observed in murine homozygous Twist1 null mutant embryos in which the cephalic neural crest cells are unable to form a functional mesenchyme . Haploid insufficiency causes Saethre-Chotzen syndrome (polysyndactyly and craniosynostosis) in humans and comparable symptoms in mice , .
During mammalian embryogenesis, Twist1 mRNA precedes TWIST1 protein expression, indicating a translational control of Twist1 , . The same phenomena was recently observed in MCF-10ANeoT cells that had undergone EMT . It is therefore interesting that many of the biological processes in which TWIST1 is involved, are associated with regulation by miRNAs, a hallmark of translational control. MiRNAs are endogenous small non-coding RNAs that regulate gene expression post-transcriptionally by modulating mRNA translation or stability. They influence cellular processes such as EMT, apoptosis, and differentiation, which all are essential for development and cancer , . MiRNA-mediated regulation of gene expression is complex as one mRNA is usually targeted by several miRNAs and one miRNA can target several mRNAs. Furthermore, miRNAs can act synergistically leading to higher levels of repression .
Here we show that the 3′UTRs of mammalian Twist1 genes contain conserved miRNA target sites, which make them sensitive to regulation by several miRNAs, individually and in cooperative combination. Understanding the exact mechanism of Twist1 regulation is important as it may allow to utilize this physiological process to be utilized therapeutically.
Analysis of Twist1 3′UTR
Translational regulation of mRNAs is typically mediated through evolutionary conserved regulatory regions within the UTRs. To test if this is the case for Twist1, we compared the conservation of coding sequence (CDS) and 3′UTR of Twist1 mRNA among selected amniotes. 5′UTRs were not included since the full-length sequences are not available for all species. Limiting the investigation to amniotes gave us the possibility to also compare the conservation of Twist1 with that of Twist2 (Table 1), a highly related gene that is differently expressed but functionally largely equivalent to Twist1 –.
The two genes are highly similar in their coding regions, genomic structure and length of the 3′UTR. However, the 3′UTR of Twist2 is not related to that of Twist1 (percent of identity 30%) and is, between species, remarkably less conserved compared to the unusually highly conserved 3′UTR of Twist1 indicating a functional selection of this sequence that is almost as stringent as for the CDS. Furthermore, the 3′UTR of Twist1 contains considerably more potential regulatory sequences, namely four nuclear polyadenylation signals (pA1-4 where pA3 and pA4 overlap), two cytoplasmic polyadenylation elements (CPEs), one AU-rich sequence and a number of putative miRNA target sites predicted by several algorithms (TargetScan, miRBase and PicTar). All but one of the identified potential regulatory sequences (miR-15b-3p) are 100% conserved between mouse and human Twist1 3′UTR sequences (Fig. 1) and a large number are conserved in a wide range of mammalian species (Table 2). In addition, out of 18 miRNAs predicted to target Twist1, only one (miR-145a-5p) putatively targets Twist2 as well (Table 2).
Yellow represents the conserved nucleotides in the 3′UTR of mouse (upper), human (lower), cow, pig and dog Twist1. Polyadenylation signals are indicated in red, CPEs in green, AU-rich sequence in brown and potential miRNA target sites in blue. In overlapping sequences, potential miRNA target sites are individualized by underlining. AU: AU-rich sequence, CPE: cytoplasmic polyadenylation element, H.: Homo sapiens, M.: Mus musculus, pA: nuclear polyadenylation signal.
MicroRNAs are targeting the Twist1 3′UTR
The presence of multiple potential miRNA target sites in the 3′UTR of Twist1 led us to investigate whether Twist1 expression could be regulated by miRNAs. The following miRNAs were tested for their potential to repress Twist1 translation in the human lung carcinoma cell line H1299: miR-33, miR-145a, miR-151, miR-326, miR-337, miR-361, miR-378a, miR-381, miR-409 and miR-543 (Fig. 1).
Murine pre-miRNA sequences were cloned into pdCMV2-EGFP expression vector and their correct processing was confirmed by Northern Blot (not shown). To determine whether any of the selected miRNAs are able to repress the expression of TWIST1, the miRNAs were tested individually or in pairwise combinations in a luciferase reporter assay using a construct in which the Firefly luciferase CDS was followed by the murine Twist1 3′UTR sequence. MiR-485 and miR-609 have no target site in the 3′UTR of Twist1 and were used as negative (mock) controls to calculate the unrepressed expression level of the reporter in these cells (Fig. 2). Expression of miR-145a resulted in a significant downregulation of the reporter by 44% (p<0.006). In addition, the co-expression of two pairs of miRNAs also led to a significantly increased repression: miR-151 + miR-337 resulted in a synergistic inhibition of 78% and miR-145a + miR-151 repressed TWIST1 expression by 61% (p<0.006). Notably, miR-337 alone had no effect while miR-151 only had a weak effect which in this assay was not statistically significant (p<0.15; see discussion).
H1299 cells were transfected with the indicated miRNA, pRluc-N2 and pGL3-Twist1-3′UTR Firefly luciferase reporter or the empty pGL3-control vector and analyzed after 48 h. Firefly luciferase activities were normalized to the Renilla luciferase activities which served as internal standards, averages of triplicates were calculated and results were normalized to empty pGL3-control vector. The dashed line indicates the unrepressed expression level of the reporter (0,176; calculated from the average of two negative controls (*), miR-485 and miR-609). Statistical significance of miRNA effects was calculated by comparison with this average using Student's t-test.
In order to confirm the location and functionality of the predicted miRNA target sites within the 3′UTR of Twist1, we mutated some of them to restriction enzyme recognition sites. Transfection of the wild type (wt) or mutant reporter with the corresponding miRNA confirmed that the repressive effects depended on the presence of the miRNA target sites. As shown in Fig. 3, mutating the target site of miR-145a-5p or miR-151-5p resulted in a loss of repression whereas lack of a binding site for miR-337-3p had no effect. When both miR-145a and miR-151 were present, mutating either of the target sites caused a comparable loss of repression. Interestingly, miR-337 was only able to repress the 3′UTR reporter when miR-151 and its target site both were present. This confirms that miRNAs and their target sites both are necessary to repress the expression of TWIST1 protein.
MiRNAs of interest were co-transfected with either wt or the corresponding mutant of psiCheck2-Twist1-3′UTR reporter into H1299 cells. Firefly luciferase activities were measured after 48 h, normalized to Renilla luciferase activities and subsequently to the respective wt+miRNA. Statistical significance was calculated by using Student's t-test and by comparing the mutant 3′UTR values with the wt 3′UTR reporter.
MicroRNAs can act by translational inhibition
MicroRNAs primarily act by destabilizing target mRNA through decapping and/or deadenylation. However, a smaller subset of targets is rather translationally repressed leaving the mRNA levels unaltered (reviewed in ). As the uneven ratio between Twist1 mRNA and TWIST1 protein in the mouse embryo ,  indicated translational inhibition, we compared the mRNA and protein ratios in cells treated with a combination of synthetic miR-151-5p and miR-337-3p precursors. Using high concentrations of pre-miR-151-5p and pre-miR-337-3p (above 10 nM) we consistently found comparable decrease in both the mRNA and protein level of the Twist1 3′UTR reporter (Fig. 4A). To more closely mimic the physiological conditions, we established stable pools of cells harboring the Twist1 3′UTR reporter and transfected them using low concentrations (5 nM) of synthetic miR-151-5p and miR-337-3p precursors. Under such conditions, we observed a 50% reduction of luciferase reporter activity and hardly any reduction in the corresponding mRNA level upon miRNA co-transfection (Fig. 4B). This indicates that the synergistic effect of miR-151-5p and miR-337-3p is mainly due to translational inhibition and not RNA degradation. A corresponding analysis using the endogenous TWIST1 protein was technically not possible as we found all available TWIST1 specific antibodies to also react with unidentified proteins in murine cells not expressing TWIST1, a specificity issue not seen in human cells .
(A) H1299 cells were co-transfected with wt psiCheck2-Twist1-3′UTR reporter and 20 nM synthetic precursor miRNAs. (B) H1299 cells stably expressing the psiCheck2-Twist1-3′UTR reporter were transfected with 5 nM synthetic precursor miRNAs. Renilla luciferase activity was measured after 48 h and normalized to the Firefly luciferase activity. Renilla luciferase mRNA levels were measured by qPCR, normalized to Firefly luciferase mRNA levels and subsequently to cells transfected with the negative control.
Twist1 reporter is downregulated by endogenous microRNAs
To test whether endogenous miRNAs are able to target the murine Twist1 3′UTR reporter, we analyzed the expression pattern of miRNAs of interest in some commonly used mouse cell lines: NIH-3T3, C3H/10T1/2 and C2C12 (Fig. 5). Since miR-145a-5p, miR-151-5p and miR-337-3p all were expressed in NIH-3T3 and C3H/10T1/2 but not in C2C12 cells, we selected the first two lines for further analysis.
Endogenous miRNA expression levels in NIH-3T3, C3H/10T1/2 and C2C12 cells were measured by qPCR using TaqMan probes, and normalized to U6 snRNA. Due to the high expression of miR-145a-5p, the relative values are presented on a logarithmic scale.
We then analyzed the effect of endogenous miRNAs on the wt and mutant Twist1 3′UTR reporters. As shown in Fig. 6A and 6B, endogenous miR-145a-5p, miR-151-5p and miR-337-3p are targeting Twist1 3′UTR in both cell lines, as mutating their target sites led to a statistically significant increase of reporter activities. This was confirmed by using locked nucleic acid (LNA)-modified miRNA antagonists to block each of these endogenous miRNAs. Inhibition of miR-145a-5p, miR-151-5p and miR-337-3p resulted in a decreased repression, i.e. an elevated expression of the reporter protein in both cell lines (Fig. 6C and 6D), confirming that the endogenous murine miRNAs are able to target the Twist1 3′UTR.
NIH-3T3 (A) and C3H/10T1/2 (B) cells were transfected with wt or mutant psiCheck2-Twist1-3′UTR reporter only. Endogenous miRNAs in NIH-3T3 (C) and C3H/10T1/2 (D) cells were inhibited by co-transfection of 50 nM miRCURY LNA microRNA inhibitors and wt psiCheck2-Twist1-3′UTR reporter. Firefly luciferase activities were measured after 48 h, normalized to Renilla luciferase activities and subsequently to the wt 3′UTR (A-B) or wt+anti-Scrambled (C-D). Statistical significance was calculated by using Student's t-test and by comparing the mutant Twist1 3′UTR reporters or miRNA specific antagonists with the wt 3′UTR reporter or a scrambled control (*), respectively.
MiR-151-5p and miR-337-3p reduce the mobility of murine embryonic fibroblast cells
There is no known cellular function which depends exclusively on TWIST1 and thus would give an all-or-none phenotype upon removal of TWIST1. We therefore decided to test whether the combination of miR-151-5p and miR-337-3p also effects the biological function of endogenous TWIST1, by investigating the ability of murine embryonic fibroblasts to migrate through an 8 µm pore filter. This type of assay correlates well with the EMT promotion by TWIST1 and has been used in similar studies . Compared to cells treated with a scrambled control miRNA, cells treated with a combination of miR-151-5p and miR-337-3p migrated significantly less (Fig.7). Despite the small absolute difference, which probably indicates the limited contribution of TWIST1 to the mobility of these cells, the effect was highly significant. The probabilities of cell migration were estimated as 0.898 (95% bootstrap CI 0.896–0.913) for the miR-151-5p + miR-337-3p treated cells and 0.918 (95% bootstrap CI 0.914–0.922) for the control treated cells, and a bootstrap permutation test (p-value 0.000135) showed that the difference between the probabilities is statistically significant.
Dot-plot superposed with the box-plot of the predicted probabilities of cell migration for observations from miR-151-5p + miR-337-3p (red) and scrambled control (blue) treated cells (p-value 0.000135). Note the presence of miR-151-5p + miR-337-3p treated cells with extreme low estimates of probability of cell migration.
In this study, we report a new mechanism for the regulation of TWIST1 expression. TWIST1 is a key player in tumorigenesis and metastasis and its overexpression has mostly been correlated with progressed stages of cancer and drug resistance . Furthermore, suppression of TWIST1 expression in tumor cells can lead to inhibition of metastatic potential .
We analyzed the presence of conserved miRNA target sites in the 3′UTR of Twist1 in selected amniotes (Table 1 and 2) and identified miR-145a-5p, miR-151-5p and the combinations of miR-145a-5p + miR-151-5p and miR-151-5p + miR-337-3p as the strongest regulators of murine Twist1 (Fig. 2 and 3). For the initial screen, we used human H1299 cells, which are easy to transfect. It should be noted, however, that a strong expression of endogenous miRNAs in these cells might cover the effect of exogenous miRNAs tested and thus give false negative results. In fact, we assume that the relatively high expression of endogenous miR-151-5p in H1299 cells (publically available miRNA microarray dataset; GEO accession number GSE30075) explains the significant difference between the miR-151-5p-mediated repressive effects shown in figures 2 and 3.
The distance between miR-145a-5p and miR-151-5p target sites is 394 nt, too long to expect a synergistic interaction . Indeed, the effect of their combination was only slightly stronger than the additive effect of their independent action. In contrast, the combinatorial effect of miR-151-5p and miR-337-3p clearly is synergistic, as miR-337-3p alone was not able to downregulate the Twist1 3′UTR reporter but only did so when miR-151-5p was present (Fig. 3). Importantly, this synergistic effect was not observed if the miR-151-5p target site was mutated. The 6 nt distance between these two miRNA target sites is within the optimal range for two cooperating miRNAs (between 6 to 40 nt) . In addition, miR-337-3p has no complementary sequence to Twist1 5′ of its seed region leaving this sequence free for miR-151-5p to bind. Interestingly, the target sites for miR-151-5p and miR-337-3p will still be present, even if the previously reported shortening of 3′UTRs in cancer cells  also occurs with TWIST1 and either the first or the second pA signal should be used.
By mutating miRNA target sites and inhibiting miRNA binding using specific LNA-modified oligonucleotides, we confirmed the effect of the above described miRNAs on the Twist1 3′UTR (Fig. 6) and demonstrated that the physiological expression levels of endogenous miRNAs are sufficient for a significant repression of Twist1 which most likely is due to translational inhibition rather than degradation of Twist1 mRNA (Fig. 4B). MiR-337 always required miR-151 to be present in order to have an effect. While this work was under revision, miR-214 , miR-300, miR-539 and miR-543  have also been reported to target the TWIST1 3′UTR. While two of them, miR-539 and miR-300 have no target site in the murine Twist1 3′UTR, a third, miR-543 did not show a significant effect in our screen. This might be due to quantitative differences between the transfection of a miRNA expression vector and application of miRNA-mimics. It would be interesting to see whether these miRNAs also target TWIST1 by inhibiting translation or by RNA destabilization.
All of the miRNAs shown to repress TWIST1 expression have been demonstrated to be involved both in embryonic development and cancer.
During development, miR-145a-5p functions as a critical switch in promoting smooth muscle differentiation and its downregulation is associated with proliferation . MiR-145a-5p is a known tumor suppressor with a strong inhibitory effect on cancer cell proliferation that is downregulated in a variety of tumors including bladder, breast, colon, colorectal, gastric, lung, oral, ovarian, and prostate cancers, hepatocellular and nasopharyngeal carcinoma and pituitary tumors , all of which are associated with increased levels of TWIST1. In cervical cancer cells, miR-145a-5p was shown to inhibit growth, invasion and therapy resistance , problems often observed in tumors in which TWIST1 is upregulated, and ectopic delivery into mouse pancreatic xenograft models resulted in the inhibition of tumor growth , .
MiR-151 targets the thrombopoietin receptor MPL , a gene known to be downregulated in myeloproliferative neoplasms . In hepatocellular carcinoma (HCC), miR-151-5p has been correlated with cell migration and intrahepatic metastasis . However, miR-151 has been shown to be both upregulated (HCC, nasopharyngeal carcinoma) and downregulated (pappillary carcinoma, acute myeloid leukemia) in different types of cancer . This dual function probably reflects the differences in gene regulation that leads to the various types of cancer. It would thus be interesting to investigate whether the expression of miR-151 in tumors correlates with the expression level of TWIST1.
MiR-337 is upregulated during bone formation  whereas TWIST1 is downregulated  and overexpression of Twist1 is known to inhibit osteoblast differentiation . This suggests miR-337 as a potential inhibitory factor regulating Twist1 during bone formation. In cancer, miR-337 has been shown to be overexpressed in older melanoma patients compared to younger ones  and is negatively associated with survival in advanced ovarian cancer . So far, nothing is known about the co-expression of miR-151 and miR-337. In light of our results, it would be interesting to see whether miR-337 is able to acquire a tumor suppressor function when co-expressed with miR-151 in TWIST1 positive tumors.
TWIST1 itself is also known to regulate the expression of several miRNAs, many of which are cancer-related. E.g. the oncogenic miR-10b is directly activated by TWIST1 and promotes the invasion and metastasis through repressing HOXD10 expression . Also, the miR-200 family and miR-205 were shown to be repressed by TWIST1 . These miRNAs are frequently silenced in advanced cancer , . They target ZEB1 and ZEB2, transcriptional repressors of E-cadherin, and thereby inhibit EMT and tumor invasion . Recently, miR-223, overexpressed in metastatic gastric cancer cells, was also found to be induced by TWIST1 . Considering its repression by miRNAs as demonstrated in this study, TWIST1 appears as a central player in the regulating network of miRNAs and transcription factors necessary for cellular homeostasis.
Recently, RNAi-mediated silencing of TWIST1 was shown to suppress the proliferation of human cervical cancer cells and to improve the chemosensitivity to cisplatin treatment, indicating a novel therapeutic strategy to overcome drug resistance . Likewise, Burns et al. demonstrated a strong therapeutic effect in oncogene driven non-small cell lung cancer after silencing TWIST1 expression . Our results about the miRNA regulation of TWIST1 provide an alternative approach to suppress this potent oncogene utilizing an endogenous mechanism.
The reduced mobility of miR-151-5p + miR-337-3p treated embryonic fibroblasts further shows the therapeutic potential in the combination of these two miRNAs (Fig. 7). The relatively small absolute difference in the migration probability is most likely explained by the fact that cell motility is a property influenced by a number of different factors. However, the high statistical significance of the difference indicates that the treatment with miR-151-5p and miR-337-3p has a reliable effect.
In summary, we have shown that TWIST1 is regulated by miR-145a-5p, miR-151-5p and miR-337-3p. The additive and synergistic effects of these miRNAs could reduce unwanted ‘off target’ effects and might open up new possibilities to specifically interfere with TWIST1 translation in therapeutic approaches.
Materials and Methods
Sequence comparison was done with CLC Main Workbench Version 5.7 (CLC bio). Parameters for alignments were: gap open cost = 10 and gap extension cost = 1. The following sequences were pairwise compared: Twist1: human (GeneBank: NM_00047); cow (GeneBank: XM_002686684); mouse (GeneBank: NM_011658); chicken (GeneBank: NM_204739; 3′UTR without nt 845-873 and 899-971 which correspond to 2 stretches of sequence not found in any other species). Twist2: human (GeneBank: NM_057179); cow (GeneBank: NM_001083748); mouse (GeneBank: NM_007855); chicken (GeneBank: NM_204679). In addition, Twist1 sequences used for miRNA target site analysis were: dog (GeneBank: XM_857736); frog (GeneBank: NM_001085883).
Cell lines and constructs
All cell lines were obtained from American Type Culture Collection and cultured at 37°C in 5% CO2. H1299 cells were maintained in RPMI 1640 (Gibco) and NIH-3T3, C3H/10T1/2 and C2C12 cells in DMEM (Gibco), all supplemented with 10% FCS, 100 µg/ml streptomycin and 100 U/ml penicillin (Gibco).
Murine pre-miRNA sequences with about 200 nt 5′ and 3′ flanking regions were amplified from C57Bl/6J mouse genomic DNA and cloned into NotI/SalI sites in the pdCMV2-EGFP expression vector. The miRBase accession numbers for miRNAs are: mmu-miR-33 (MI0000707), mmu-miR-145a (MI0000169), mmu-miR-151 (MI0000173), mmu-miR-326 (MI0000598), mmu-miR-337 (MI0000615), mmu-miR-361 (MI0000761), mmu-miR-378a (MI0000795), mmu-miR-381 (MI0000798), mmu-miR-409 (MI0001160) and mmu-miR-543 (MI0003519). Accession numbers for two miRNAs used as negative controls are: hsa-miR-485 (MI0002469) and hsa-miR-609 (MI0003622). Primers used for cloning were: mmu-miR-33-upper (5′ctgtcagcggccggaagcctactcgggcaatgtgcc 3′), mmu-miR-33-lower (5′ gctgcgatcgtcgacggaagcagctagttctaacctcc 3′), mmu-miR-145a-upper (5′ agctatgcggccggactctacgcacatgaaatgcttcttcc 3′), mmu-miR-145a-lower (5′ gctgcgatcgtcgacggattggatgtaatataaatgaagcaaacc 3′), mmu-miR-151-upper (5′ ctgtcagcggccgctttagggcctgagaatatcttga 3′), mmu-miR-151-lower (5′ gggaccgcgtcgacgaagcaaacctcaataacagaactc 3′), mmu-miR-326-upper (5′ cctgtagcggccgctactccgcatagccactgaga 3′), mmu-miR-326-lower (5′ gcggatccgtcgacctagcccagggccatatacatg 3′), mmu-miR-337-upper (5′ ctgtcagcggccgccatgagagtttataagaagtgagg 3′), mmu-miR-337-lower (5′ gggaccgcgtcgacgccaaggaatgatctcaggtatgg 3′), mmu-miR-361-upper (5′ agctatgcggccgcttaacatgccttggtttgcaga3′), mmu-miR-361-lower (5′ acctagcggtcgacctttgctgctttctttgcttcc 3′), mmu-miR-378a-upper (5′ cctgtagcggccggtaattgataccaggagctttgcagcc 3′), mmu-miR-378a-lower (5′ acctagcggtcgacccagacagctcacatgcaaacacagg 3′), mmu-miR-381-upper (5′ catgtagcggccgccaatctgttgtaacatctgccagt 3′), mmu-miR-381-lower (5′ gctgcgatcgtcgacagaacatgcacacttgggtac 3′), mmu-miR-409-upper (5′ cctgtagcggccggggaaggctttagatattcttggaagg 3′), mmu-miR-409-lower (5′ acctagcggtcgaccacggtcgatctcccttcaagtaccagc 3′), mmu-miR-543-upper (5′ agctatgcggccgggagactccaaagacctccccaaagg 3′) and mmu-miR-543-lower (5′ gggaccgcgtcgacgtggaggagggaggagggagcaggagcc 3′).
Mouse Twist1 3′UTR sequence (GeneBank: NC_000078; genomic DNA position 34,643,544 to 34,645,282) was cloned into the XbaI/NotI sites of pGL3-control vector (Promega). This sequence includes the unique intron of the Twist1 gene and thus represents a construct which will report all aspects of Twist1 mRNA processing including splicing. PRluc-N2 (PerkinElmer) encoding Renilla luciferase was used for normalization.
Due to high fluctuations in the pGL3-control transfection efficiencies, mouse Twist1 3′UTR sequence (GeneBank: NM_011658; nucleotide position 926-1634) was cloned into XhoI/NotI sites in psiCheck2 vector (Promega) which expresses both Firefly and Renilla luciferases. MiRNA target sites were subsequently mutated to restriction enzyme recognition sites: miR-145a-5p to SacI; miR-151-5p to AgeI; miR-337-3p to SalI.
Stable pools of cells expressing psiCheck2-Twist1-3′UTR were established as described in .
Transfections and luciferase assays
All transfections were carried out in triplicates on 48-well plates in three independent experiments and assayed after 48 h using the Dual-Luciferase Reporter Assay System Kit (Promega). 48 h was chosen to allow the cells to recover from the transfection and grow up to ∼80–100% confluence at the time of measurement.
In the miRNA screen, 104/cm2 H1299 cells were transfected with 250 ng pdCMV2-EGFP-miR-X (or 125 ng each if two miRNAs were combined), 50 ng pRluc-N2 and 50 ng pGL3-Twist1-3′UTR or pGL3-control using TransIT-LT1 transfection reagent (Mirus) according to manufacturer's guidelines. For normalization, the Firefly luciferase activity was divided by the Renilla luciferase activity. Triplicates were averaged and normalized to the pGL3-control.
In the target site mutation assay, 104/cm2 H1299 cells were transfected with 250 ng (or 125 ng each) miRNA construct and 50 ng wt or mutant psiCheck2-Twist1-3′UTR using TransIT-LT1 transfection reagent (Mirus) according to manufacturer's instructions. For normalization, Renilla luciferase activity was divided by Firefly luciferase activity and the mutant-transfected cells were subsequently normalized to the respective wt 3′UTR reporter treated with miRNA.
The transfection efficiency of miRNA-encoding vectors was monitored by the expression of EGFP with the Leica DM fluorescence microscope. The efficiency was always above 60%.
104/cm2 NIH-3T3 and C3H/10T1/2 cells were transfected with 50 ng psiCheck2-Twist1-3′UTR using Fugene HD transfection reagent (Roche) according to manufacturer's guidelines. 50 nM miRCURY LNA microRNA inhibitors (Exiqon) were added in the inhibition assay: mmu-miR-145a-5p (139465-00), hsa-miR-151-5p (414654-00), mmu-miR-337-3p (139530-00) and a Scrambled negative control (199004-00). For normalization, Renilla luciferase activity was divided by Firefly luciferase activity.
10,000 stable psiCheck2-Twist1-3′UTR-expressing cells per 96-well plate well were transfected while seeding in 100 µl serum-free RPMI 1640 (Gibco) using 0.2 µl Lipofectamine 2000 (Invitrogen) and 20 or 5 nM Pre-miR miRNA Precursor (Applied Biosystems) per well: pre-mmu-miR-151 (PM11537), pre-mmu-miR-337 (PM12817) and Scrambled Negative Control #1 (AM17110). Simultaneously, 30,000 cells/well were transfected on a 24-well plate for RNA extraction and analysis using more reagents, accordingly. After 4 h of transfection, 10% FCS (Gibco) of total volume was added to each well. Renilla luciferase mRNA and protein levels were measured after 48 h.
Total RNA was purified using TRIzol (Invitrogen) according to the manufacturer's instructions. 500 ng of total RNA was reverse transcribed with M-MLV reverse transcriptase (Invitrogen) using 200 ng random hexamer primer (Roche).
Renilla luciferase expression was quantified using Platinum SYBR Green qPCR SuperMix-UDG (Invitrogen) and 25 ng cDNA as template. The primers used were: R-luc_fwd (5′ gttccctaacaccgagttcgtg 3′), R-luc_rev (5′ ctccacgaagctcttgatgtac 3′), F-luc_fwd (5′ ctgatcgacaaggacggctgg 3′), F-luc_rev (5′ ggtagcccttgtacttgatcag 3′). Renilla luciferase mRNA levels were normalized to Firefly luciferase mRNA levels and subsequently to cells transfected with the negative control. Relative quantification of mRNA levels was performed using the ΔΔCt-method.
For miRNA analysis, 25 ng of total RNA was reverse transcribed using TaqMan MicroRNA Reverse Transcription Kit (Applied Biosystems). QPCR was carried out in triplicates using 1.33 µl cDNA and TaqMan Universal PCR Master Mix (Applied Biosystems). The TaqMan MicroRNA Assays used were: hsa-miR-145a-5p (002278), hsa-miR-151-5p (002642), mmu-miR-337-3p (002532) and U6 snRNA (001973). Relative quantities of miRNA were calculated by the ΔCt method.
Cell migration assay
3T3 immortalized murine embryonic fibroblasts  were plated on a 12-well plate at a density of 104/cm2 in 1 ml complete growth medium. While seeding, cells were transiently transfected with a total of 20 nM pre-mmu-miR-151-5p (PM11537) and pre-mmu-miR-337-3p (PM12817) or with Scrambled Negative Control #1 (AM17110; all Applied Biosystems) using Lipofectamine 2000 (Invitrogen) according to manufacturer's instructions. After 24 h, cells were trypsinized, resuspended in 400 µl medium, distributed equally on 2 cell culture inserts 200 µl each (24-well, 8.0 µm pore size, Falcon, 35 3097) and then placed in a 24-well plate (Falcon, 35 3504) which contained 600 µl medium per well. The 2 inserts were designated as ‘A,over’ and ‘B,under’ to study the analysis of non-migrated and migrated cells, respectively. Cells were allowed to migrate for 21 h, washed with PBS and fixed in 4% freshly dissolved paraformaldehyde for 30 min at room temperature. Following 2× washing with PBS, the cells on the lower surface of insert A and the cells on the upper surface of insert B were wiped off with a cotton swap. The insert membranes were cut off and embedded in ProLong Gold Antifade Reagent with DAPI (Invitrogen). Migrated (under) and non-migrated (over) cells were counted in 10 (primary magnification 10×) and 5 (primary magnification 5×) random areas, respectively, on micrographs taken with a Leica DM fluorescence microscope.
Statistical significance for the transfection experiments was calculated using Student's t-test and 2-sided p-values.
The effect of the treatment on the probability of cell migration was studied in the following way: Denote the counts of the migrated cells and the non-migrated cells for the mth observed microscopic field for the rth replication subject to the tth treatment by Mmrt and Smrt, respectively. The expected number of migrated and non-migrated cells are given by E(Mmrt) = pt Urt and E(Smrt) = (1-pt) Urt/40, respectively. Here pt is the probability of migration common to all the observations subject to the tth treatment and Urt is the expected number of cells transferred from the suspension associated to the rtth replicate. The factor 1/40 arises from the fact that different macroscopic field areas were observed for the independent determinations of the migrated and non-migrated cells. Using a first order Taylor expansion yields that the expectation of Pt = Mmrt/(Mmrt + Smrt) is the probability pt, therefore Pt is an unbiased estimate of pt. The effect of the treatment on the probability of cell migration was then formally tested using the Monte Carlo permutation test  with 1,000,000 bootstrap permutations. Moreover, 95% bootstrap confidence intervals for the probabilities of cell migration were constructed with non-parametric bootstrap , with 10,000 parametric bootstrap each.
We thank Agata Magdziarz and Urzula Kania for help in cloning some of the miRNAs and Twist1 3′UTR reporters, Claus Bus and Tine Birch for technical support and Karen Colbjørn Søgaard for the critical reading of the manuscript.
Conceived and designed the experiments: MLN AF JBB EMF. Performed the experiments: MLN AF JBB. Analyzed the data: MLN AF RL JBB EMF. Contributed reagents/materials/analysis tools: MLN AF RL JBB EMF. Wrote the paper: MLN EMF.
- 1. Maestro R, Dei Tos AP, Hamamori Y, Krasnokutsky S, Sartorelli V, et al. (1999) Twist is a potential oncogene that inhibits apoptosis. Genes Dev 13: 2207–2217. doi: 10.1101/gad.13.17.2207
- 2. Villavicencio EH, Yoon JW, Frank DJ, Füchtbauer EM, Walterhouse DO, et al. (2002) Cooperative E-box regulation of human GLI1 by TWIST and USF. Genesis 32: 247–258. doi: 10.1002/gene.10078
- 3. Ma L, Teruya-Feldstein J, Weinberg RA (2007) Tumour invasion and metastasis initiated by microRNA-10b in breast cancer. Nature 449: 682–688. doi: 10.1038/nature06174
- 4. Cheng GZ, Zhang W, Wang LH (2008) Regulation of cancer cell survival, migration, and invasion by Twist: AKT2 comes to interplay. Cancer Res 68: 957–960. doi: 10.1158/0008-5472.can-07-5067
- 5. Qin Q, Xu Y, He T, Qin C, Xu J (2012) Normal and disease-related biological functions of Twist1 and underlying molecular mechanisms. Cell Res 22: 90–106. doi: 10.1038/cr.2011.144
- 6. Hornik C, Brand-Saberi B, Rudloff S, Christ B, Füchtbauer EM (2004) Twist is an integrator of SHH, FGF, and BMP signaling. Anat Embryol (Berl) 209: 31–39. doi: 10.1007/s00429-004-0412-3
- 7. Krebs I, Weis I, Hudler M, Rommens JM, Roth H, et al. (1997) Translocation breakpoint maps 5 kb 3′ from TWIST in a patient affected with Saethre-Chotzen syndrome. Hum Mol Genet 6: 1079–1086. doi: 10.1093/hmg/6.7.1079
- 8. Li CW, Xia W, Huo L, Lim SO, Wu Y, et al. (2012) Epithelial-mesenchymal transition induced by TNF-alpha requires NF-kappaB-mediated transcriptional upregulation of Twist1. Cancer Res 72: 1290–1300. doi: 10.1158/0008-5472.can-11-3123
- 9. Chang TM, Hung WC (2012) Transcriptional repression of TWIST1 gene by Prospero-related homeobox 1 inhibits invasiveness of hepatocellular carcinoma cells. FEBS Lett 586: 3746–3752. doi: 10.1016/j.febslet.2012.08.034
- 10. Yang MH, Wu MZ, Chiou SH, Chen PM, Chang SY, et al. (2008) Direct regulation of TWIST by HIF-1alpha promotes metastasis. Nat Cell Biol 10: 295–305. doi: 10.1038/ncb1691
- 11. Gort EH, van Haaften G, Verlaan I, Groot AJ, Plasterk RH, et al. (2008) The TWIST1 oncogene is a direct target of hypoxia-inducible factor-2alpha. Oncogene 27: 1501–1510. doi: 10.1038/sj.onc.1210795
- 12. Harris AL (2002) Hypoxia--a key regulatory factor in tumour growth. Nat Rev Cancer 2: 38–47. doi: 10.1038/nrc704
- 13. Yang J, Mani SA, Donaher JL, Ramaswamy S, Itzykson RA, et al. (2004) Twist, a master regulator of morphogenesis, plays an essential role in tumor metastasis. Cell 117: 927–939. doi: 10.1016/j.cell.2004.06.006
- 14. Eckert MA, Lwin TM, Chang AT, Kim J, Danis E, et al. (2011) Twist1-induced invadopodia formation promotes tumor metastasis. Cancer Cell 19: 372–386. doi: 10.1016/j.ccr.2011.01.036
- 15. Dave N, Guaita-Esteruelas S, Gutarra S, Frias A, Beltran M, et al. (2011) Functional cooperation between Snail1 and twist in the regulation of ZEB1 expression during epithelial to mesenchymal transition. J Biol Chem 286: 12024–12032. doi: 10.1074/jbc.m110.168625
- 16. Laursen KB, Mielke E, Iannaccone P, Füchtbauer EM (2007) Mechanism of transcriptional activation by the proto-oncogene Twist1. J Biol Chem 282: 34623–34633. doi: 10.1074/jbc.m707085200
- 17. Hebrok M, Füchtbauer A, Füchtbauer EM (1997) Repression of muscle-specific gene activation by the murine Twist protein. Exp Cell Res 232: 295–303. doi: 10.1006/excr.1997.3541
- 18. Spicer DB, Rhee J, Cheung WL, Lassar AB (1996) Inhibition of myogenic bHLH and MEF2 transcription factors by the bHLH protein Twist. Science 272: 1476–1480. doi: 10.1126/science.272.5267.1476
- 19. Thisse B, Stoetzel C, Gorostiza-Thisse C, Perrin-Schmitt F (1988) Sequence of the twist gene and nuclear localization of its protein in endomesodermal cells of early Drosophila embryos. EMBO J 7: 2175–2183.
- 20. Simpson P (1983) Maternal-Zygotic Gene Interactions during Formation of the Dorsoventral Pattern in Drosophila Embryos. Genetics 105: 615–632.
- 21. Chen ZF, Behringer RR (1995) twist is required in head mesenchyme for cranial neural tube morphogenesis. Genes & Development 9: 686–699. doi: 10.1101/gad.9.6.686
- 22. el Ghouzzi V, Le Merrer M, Perrin-Schmitt F, Lajeunie E, Benit P, et al. (1997) Mutations of the TWIST gene in the Saethre-Chotzen syndrome. Nat Genet 15: 42–46. doi: 10.1038/ng0197-42
- 23. Füchtbauer EM (1995) Expression of M-twist during postimplantation development of the mouse. Dev Dyn 204: 316–322. doi: 10.1002/aja.1002040309
- 24. Gitelman I (1997) Twist protein in mouse embryogenesis. Dev Biol 189: 205–214. doi: 10.1006/dbio.1997.8614
- 25. Nairismägi ML, Vislovukh A, Meng Q, Kratassiouk G, Beldiman C, et al. (2012) Translational control of TWIST1 expression in MCF-10A cell lines recapitulating breast cancer progression. Oncogene 31: 4960–4966. doi: 10.1038/onc.2011.650
- 26. Erson AE, Petty EM (2008) MicroRNAs in development and disease. Clin Genet 74: 296–306. doi: 10.1111/j.1399-0004.2008.01076.x
- 27. Cano A, Nieto MA (2008) Non-coding RNAs take centre stage in epithelial-to-mesenchymal transition. Trends Cell Biol 18: 357–359. doi: 10.1016/j.tcb.2008.05.005
- 28. Doench JG, Sharp PA (2004) Specificity of microRNA target selection in translational repression. Genes Dev 18: 504–511. doi: 10.1101/gad.1184404
- 29. Scaal M, Füchtbauer EM, Brand-Saberi B (2001) cDermo-1 expression indicates a role in avian skin development. Anat Embryol (Berl) 203: 1–7. doi: 10.1007/pl00008244
- 30. Li L, Cserjesi P, Olson EN (1995) Dermo-1: a novel twist-related bHLH protein expressed in the developing dermis. Dev Biol 172: 280–292. doi: 10.1006/dbio.1995.0023
- 31. Franco HL, Casasnovas J, Rodriguez-Medina JR, Cadilla CL (2011) Redundant or separate entities?--roles of Twist1 and Twist2 as molecular switches during gene transcription. Nucleic Acids Res 39: 1177–1186. doi: 10.1093/nar/gkq890
- 32. Kapinas K, Delany AM (2011) MicroRNA biogenesis and regulation of bone remodeling. Arthritis Res Ther 13: 220. doi: 10.1186/ar3325
- 33. Peinado H, Olmeda D, Cano A (2007) Snail, Zeb and bHLH factors in tumour progression: an alliance against the epithelial phenotype? Nat Rev Cancer 7: 415–428. doi: 10.1038/nrc2131
- 34. Grimson A, Farh KK, Johnston WK, Garrett-Engele P, Lim LP, et al. (2007) MicroRNA targeting specificity in mammals: determinants beyond seed pairing. Mol Cell 27: 91–105. doi: 10.1016/j.molcel.2007.06.017
- 35. Mayr C, Bartel DP (2009) Widespread shortening of 3′UTRs by alternative cleavage and polyadenylation activates oncogenes in cancer cells. Cell 138: 673–684. doi: 10.1016/j.cell.2009.06.016
- 36. Li B, Han Q, Zhu Y, Yu Y, Wang J, et al. (2012) Down-regulation of miR-214 contributes to intrahepatic cholangiocarcinoma metastasis by targeting Twist. FEBS J 279: 2393–2398. doi: 10.1111/j.1742-4658.2012.08618.x
- 37. Haga CL, Phinney DG (2012) MicroRNAs in the imprinted DLK1-DIO3 region repress the epithelial-to-mesenchymal transition by targeting the TWIST1 protein signaling network. J Biol Chem 287: 42695–42707. doi: 10.1074/jbc.m112.387761
- 38. Cordes KR, Sheehy NT, White MP, Berry EC, Morton SU, et al. (2009) miR-145 and miR-143 regulate smooth muscle cell fate and plasticity. Nature 460: 705–710. doi: 10.1038/nature08195
- 39. Zhang J, Guo H, Qian G, Ge S, Ji H, et al. (2010) MiR-145, a new regulator of the DNA fragmentation factor-45 (DFF45)-mediated apoptotic network. Mol Cancer 9: 211. doi: 10.1186/1476-4598-9-211
- 40. Shi M, Du L, Liu D, Qian L, Hu M, et al. (2012) Glucocorticoid regulation of a novel HPV-E6-p53-miR-145 pathway modulates invasion and therapy resistance of cervical cancer cells. J Pathol 228: 148–157. doi: 10.1002/path.3997
- 41. Pramanik D, Campbell NR, Karikari C, Chivukula R, Kent OA, et al. (2011) Restitution of tumor suppressor microRNAs using a systemic nanovector inhibits pancreatic cancer growth in mice. Mol Cancer Ther 10: 1470–1480. doi: 10.1158/1535-7163.mct-11-0152
- 42. Kent OA, Chivukula RR, Mullendore M, Wentzel EA, Feldmann G, et al. (2010) Repression of the miR-143/145 cluster by oncogenic Ras initiates a tumor-promoting feed-forward pathway. Genes Dev 24: 2754–2759. doi: 10.1101/gad.1950610
- 43. Girardot M, Pecquet C, Boukour S, Knoops L, Ferrant A, et al. (2010) miR-28 is a thrombopoietin receptor targeting microRNA detected in a fraction of myeloproliferative neoplasm patient platelets. Blood 116: 437–445. doi: 10.1182/blood-2008-06-165985
- 44. Moliterno AR, Hankins WD, Spivak JL (1998) Impaired expression of the thrombopoietin receptor by platelets from patients with polycythemia vera. N Engl J Med 338: 572–580. doi: 10.1056/nejm199802263380903
- 45. Ding J, Huang S, Wu S, Zhao Y, Liang L, et al. (2010) Gain of miR-151 on chromosome 8q24.3 facilitates tumour cell migration and spreading through downregulating RhoGDIA. Nat Cell Biol 12: 390–399. doi: 10.1038/ncb2039
- 46. Wang Y, Lee CG (2009) MicroRNA and cancer--focus on apoptosis. J Cell Mol Med 13: 12–23. doi: 10.1111/j.1582-4934.2008.00510.x
- 47. Annalisa P, Furio P, Ilaria Z, Anna A, Luca S, et al. (2008) Anorganic bovine bone and a silicate-based synthetic bone activate different microRNAs. J Oral Sci 50: 301–307. doi: 10.2334/josnusd.50.301
- 48. Murray SS, Glackin CA, Winters KA, Gazit D, Kahn AJ, et al. (1992) Expression of helix-loop-helix regulatory genes during differentiation of mouse osteoblastic cells. J Bone Miner Res 7: 1131–1138. doi: 10.1002/jbmr.5650071004
- 49. Bialek P, Kern B, Yang X, Schrock M, Sosic D, et al. (2004) A twist code determines the onset of osteoblast differentiation. Dev Cell 6: 423–435. doi: 10.1016/s1534-5807(04)00058-9
- 50. Jukic DM, Rao UN, Kelly L, Skaf JS, Drogowski LM, et al. (2010) Microrna profiling analysis of differences between the melanoma of young adults and older adults. J Transl Med 8: 27. doi: 10.1186/1479-5876-8-27
- 51. Shih KK, Qin LX, Tanner EJ, Zhou Q, Bisogna M, et al. (2011) A microRNA survival signature (MiSS) for advanced ovarian cancer. Gynecol Oncol 121: 444–450. doi: 10.1016/j.ygyno.2011.01.025
- 52. Wiklund ED, Bramsen JB, Hulf T, Dyrskjot L, Ramanathan R, et al. (2011) Coordinated epigenetic repression of the miR-200 family and miR-205 in invasive bladder cancer. Int J Cancer 128: 1327–1334. doi: 10.1002/ijc.25461
- 53. Gottardo F, Liu CG, Ferracin M, Calin GA, Fassan M, et al. (2007) Micro-RNA profiling in kidney and bladder cancers. Urol Oncol 25: 387–392. doi: 10.1016/j.urolonc.2007.01.019
- 54. Dyrskjot L, Ostenfeld MS, Bramsen JB, Silahtaroglu AN, Lamy P, et al. (2009) Genomic profiling of microRNAs in bladder cancer: miR-129 is associated with poor outcome and promotes cell death in vitro. Cancer Res 69: 4851–4860. doi: 10.1158/0008-5472.can-08-4043
- 55. Gregory PA, Bert AG, Paterson EL, Barry SC, Tsykin A, et al. (2008) The miR-200 family and miR-205 regulate epithelial to mesenchymal transition by targeting ZEB1 and SIP1. Nat Cell Biol 10: 593–601. doi: 10.1038/ncb1722
- 56. Li X, Zhang Y, Zhang H, Liu X, Gong T, et al. (2011) miRNA-223 promotes gastric cancer invasion and metastasis by targeting tumor suppressor EPB41L3. Mol Cancer Res 9: 824–833. doi: 10.1158/1541-7786.mcr-10-0529
- 57. Zhu K, Chen L, Han X, Wang J (2012) Short hairpin RNA targeting Twist1 suppresses cell proliferation and improves chemosensitivity to cisplatin in HeLa human cervical cancer cells. Oncol Rep 27: 1027–1034. doi: 10.3892/or.2012.1633
- 58. Burns TF, Dobromilskaya I, Murphy SC, Gajula RP, Thiyagarajan S, et al.. (2013) Inhibition of TWIST1 Leads to Activation of Oncogene-Induced Senescence in Oncogene Driven Non-Small Cell Lung Cancer.Mol Cancer Res: In press .
- 59. Bramsen JB, Pakula MM, Hansen TB, Bus C, Langkjaer N, et al. (2010) A screen of chemical modifications identifies position-specific modification by UNA to most potently reduce siRNA off-target effects. Nucleic Acids Res 38: 5761–5773. doi: 10.1093/nar/gkq341
- 60. Füchtbauer A, Lassen LB, Jensen AB, Howard J, Quiroga Ade S, et al. (2011) Septin9 is involved in septin filament formation and cellular stability. Biol Chem 392: 769–777. doi: 10.1515/bc.2011.088
- 61. Davison AC, Hinkley DV (1997) Bootstrap Methods and Their Application. NY USA : Cambridge University Press.161–175, 193 p.