Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A systematic review of the asymmetric inheritance of cellular organelles in eukaryotes: A critique of basic science validity and imprecision

A systematic review of the asymmetric inheritance of cellular organelles in eukaryotes: A critique of basic science validity and imprecision

  • Anne Collins, 
  • Janine Ross, 
  • Shona H. Lang
PLOS
x

Abstract

We performed a systematic review to identify all original publications describing the asymmetric inheritance of cellular organelles in normal animal eukaryotic cells and to critique the validity and imprecision of the evidence. Searches were performed in Embase, MEDLINE and Pubmed up to November 2015. Screening of titles, abstracts and full papers was performed by two independent reviewers. Data extraction and validity were performed by one reviewer and checked by a second reviewer. Study quality was assessed using the SYRCLE risk of bias tool, for animal studies and by developing validity tools for the experimental model, organelle markers and imprecision. A narrative data synthesis was performed. We identified 31 studies (34 publications) of the asymmetric inheritance of organelles after mitotic or meiotic division. Studies for the asymmetric inheritance of centrosomes (n = 9); endosomes (n = 6), P granules (n = 4), the midbody (n = 3), mitochondria (n = 3), proteosomes (n = 2), spectrosomes (n = 2), cilia (n = 2) and endoplasmic reticulum (n = 2) were identified. Asymmetry was defined and quantified by variable methods. Assessment of the statistical reliability of the results indicated only two studies (7%) were judged to have low concern, the majority of studies (77%) were 'unclear' and five (16%) were judged to have 'high concerns'; the main reasons were low technical repeats (<10). Assessment of model validity indicated that the majority of studies (61%) were judged to be valid, ten studies (32%) were unclear and two studies (7%) were judged to have 'high concerns'; both described 'stem cells' without providing experimental evidence to confirm this (pluripotency and self-renewal). Assessment of marker validity indicated that no studies had low concern, most studies were unclear (96.5%), indicating there were insufficient details to judge if the markers were appropriate. One study had high concern for marker validity due to the contradictory results of two markers for the same organelle. For most studies the validity and imprecision of results could not be confirmed. In particular, data were limited due to a lack of reporting of interassay variability, sample size calculations, controls and functional validation of organelle markers. An evaluation of 16 systematic reviews containing cell assays found that only 50% reported adherence to PRISMA or ARRIVE reporting guidelines and 38% reported a formal risk of bias assessment. 44% of the reviews did not consider how relevant or valid the models were to the research question. 75% reviews did not consider how valid the markers were. 69% of reviews did not consider the impact of the statistical reliability of the results. Future systematic reviews in basic or preclinical research should ensure the rigorous reporting of the statistical reliability of the results in addition to the validity of the methods. Increased awareness of the importance of reporting guidelines and validation tools is needed for the scientific community.

Introduction

A systematic review uses transparent and systematic methods to minimise bias and random error. It seeks to combine the results from multiple studies to generate a more powerful, overall finding about the effectiveness of a technology or evidence under investigation. Systematic reviews have been used predominantly for clinical research, but it is possible to conduct a review in any field, or in any topic, by synthesising different types of evidence. Systematic review techniques are rigorous, to minimise bias, and include a critical appraisal of the quality of the evidence. Traditional reviews often include a range of sources of potential bias including: failure to clearly state the review question, no attempt to identify all relevant literature, poor explanation of the inclusion and exclusion of evidence, lack of consideration of study methods or quality, and typically are conducted by a single reviewer with a professional interest in the field [1]. Bias is a systematic error, usually due to an issue with study design, such that repetition of the same experiment would still give the wrong answer. Imprecision is the random error, repetition of the result will reduce sampling error, therefore larger studies are more precise.

The use of systematic reviews within basic research is not commonly performed. In recent years there have been systematic reviews of preclinical, animal-based studies. These reviews have led to the establishment of CAMARADES (Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies; http://www.dcn.ed.ac.uk/camarades/default.htm), the development of the ARRIVE reporting guidelines for in vivo studies [2], and the SYRCLE risk of bias tool [3]. Systematic reviews for basic science offer the same advantages as for those carried out for preclinical animal studies: to statistically combine the results of a number of similar studies, to provide more reliable results upon which to base decisions (scientific rather than subjective), to identify evidence gaps, to allow an evidence based translation of basic science to the clinic, and to improve the validation of basic research by identifying results within multiple model systems [4].

The authors of this review have published on the fate and differentiation of human prostate stem cells [58], as well as performing clinical systematic reviews [912]. Stem cell research is of great biomedical interest and stem cell therapies are a growing area of research, although the basic biology is not well understood. Whilst performing cell tracking experiments, we observed the asymmetric inheritance of a fluorescent lipid soluble dye known as PKH26 [5,6]. Many groups have observed that PKH26 becomes internalised into the endocytic pathway [13], which led us to postulate that we had witnessed early asymmetric inheritance of lipid membrane organelles. We decided to perform a systematic review to establish what evidence exists for the asymmetric inheritance of any eukaryotic organelle after cell division, and specifically to identify if this can occur in stem cells or human cells. We discovered that existing risk of bias tools were not appropriate for cell based studies. Therefore, we produced validity tools to assess: a) the model, b) the marker and c) the imprecision of the result. This review concentrates on the assessment of the scientific quality of the included studies (rather than the findings). We aimed to demonstrate the validity and reliability of the included studies and we aimed to evaluate whether other basic science systematic reviews assessed quality or experimental validity and imprecision.

Methods

The methods for the literature searches and systematic review adhered to the Cochrane Collaboration guidance [14], to reduce the risk of bias and error. This study was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [15]. The study protocol was registered with the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES), http://www.dcn.ed.ac.uk/camarades/default.htm.

Definitions

Asymmetric inheritance was defined as the division of an organelle and its inheritance to either the mother or daughter cell but not to both, or the unequal inheritance between two daughter cells.

Literature searches

Attempts were made to identify studies on asymmetric cell division. Searches in bibliographic databases were not limited by publication date, language or publication status (published or unpublished). Search strategies are presented in S1 File. The following databases were searched on 18 November 2015: Embase (OvidSP): 1974–2015/11/18, Medline (OvidSP): 1946–2015/11/WK2, Medline In-Process Citations & Daily Update (OvidSP): up to 2015/11/18, PubMed (NLM) (Internet) (http://www.ncbi.nlm.nih.gov/pubmed): up to 2015/11/19. The reference lists of all included articles and relevant reviews were also searched to identify studies for inclusion. A second search was performed to identify systematic reviews of basic research which included eukaryotic and cell based assays; the methods used are described above.

Inclusion and exclusion criteria

We included original publications of the asymmetric division (mitotic or meiotic) or inheritance of cellular organelles in normal animal eukaryotic cells. We specifically included the following organelles: golgi, endoplasmic reticulum, sarcoplastic reticulum, mitochondria, vacuole, proteosome, lysosome, centrosome, microtubule organising centre, centriole, spindle pole body, autophagosome, exosome, peroxisome. The inclusion criteria are summarised in S1 Table. Other organelles were considered if identified from the searches.

We excluded prokaryotes, plants and models derived from diseased eukaryotic animals (e.g. cancer cell lines). Non-English language articles, conference proceedings, abstracts, commentaries and reviews were not included. Studies which examined intracellular polar distribution or asymmetric localisation in the absence of cell division were not included. Proteins, genes, transcription factors, cell size, chromosomes, chromatids were not included.

The second review included any systematic reviews of basic research which included cell based assays. We excluded all reviews which included clinical, diagnostic or prognostics outcomes (S1 Table).

Study selection and data extraction

Publications were loaded onto the systematic review web app, Rayyan, for title and abstract screening [16]. Titles and abstracts were independently screened by two reviewers. Articles meeting the inclusion criteria were obtained as full paper copies. These were independently examined in detail by two reviewers to determine whether the full paper met the inclusion criteria of the review. All papers excluded at this second stage of the screening process were documented along with the reasons for exclusion. Any discrepancies between reviewers were resolved through consensus.

Data extraction was performed by one reviewer and checked by a second reviewer. Any discrepancies were resolved through discussion. Studies were identified by the surname of the first author and by the publication year. To avoid the duplication of data, multiple publications from the same research group were not extracted (if reporting the same methodology for the same marker and model, and reporting the same result for asymmetry but the authors were progressing the research), instead the paper providing the most robust evidence (imprecision or marker validity) was chosen as the primary evidence source. When a study identified more than one marker for a given organelle, all results for all markers were extracted (even if one marker did not identify asymmetric inheritance).

Quality assessment

Study quality was initially assessed using SYRCLE's risk of bias tool for animal studies [3]. Further validity assessments were created based on the requirements of the authors and by considering the ARRIVE Guidelines Checklist [2], the ROBINS-I tools [17], Downs and Black checklist [18], and QUADAS-2 [19]. These new validity tools were based on model validity, statistical imprecision and marker/experimental validity (S2S4 Tables). Two reviewers independently assessed study quality and any discrepancies were resolved through discussion.

Data synthesis

A narrative summary of all the included studies was compiled. The data were sorted according to organelle and according to phylogenic order. The organisms most direct to humans were ordered to the top of the table and organisms most indirect to the bottom. Results were presented alongside overall judgements for concerns regarding the validity and imprecision of the result.

Results and discussion

Literature searches and inclusion assessment

A summary of the identification and selection of studies for inclusion in this review is presented in Fig 1, in accordance with the PRISMA statement [15]. Literature searches of electronic databases retrieved 6,496 articles and hand searching identified four additional articles. After de-duplication 4,356 titles/abstracts were screened and 3,877 papers were excluded as having no relevance to the review. Full papers of 479 potentially relevant references were selected for further examination. Of these, 445 papers were excluded after reading the full paper, the reasons for exclusion are provided in Fig 1. Thirty one studies (34 publications) met the inclusion criteria.

Asymmetric inheritance of organelles

Using systematic review methodology, we identified 31 studies describing the asymmetric inheritance of organelles after mitotic or meiotic division, in animal eukaryotic cells. Nine studies reported asymmetric inheritance for centrosomes, two for cilia, two for endoplasmic reticulum, six for endosomes, three for the midbody, three for mitochondria, four for P granules, two for proteosomes and two for spectrosomes (Table 1). Nine different organisms were identified for the model systems. Four studies used human models [2023], the majority used Drosophila and mouse. Most studies used embryonic cells but three were in adult stem cells and five were in adult non-stem cells. Two studies of meiotic division were in oocytes; the remaining studies were of mitotic division. Three studies had two publications each [2426], three studies had data for more than one organelle [22,27,28] and one study [23] had models for more than one organism. Publication dates ranged from 1991 to 2015. Only six (19%) studies reported that the result was repeated in another model [22,23,2932]. The aim of the current review was to focus on the imprecision and quality of the included studies rather than presentation of the results in further detail.

thumbnail
Table 1. Characteristics and validity of studies showing asymmetric inheritance of organelles.

https://doi.org/10.1371/journal.pone.0178645.t001

Validity

All studies were assessed for risk of bias using the SYRCLE checklist [3], this is the only risk of bias tool for non-clinical studies. Using SYRCLE we judged most domains to be unclear or not reported (S5 Table). SYRCLE is based on the risk of bias tools created for randomised controlled clinical trials [14], we found that this was not applicable to the design of basic scientific studies. Although the signalling questions of SYRCLE are not irrelevant to basic science studies they do not use a language that is meaningful to laboratory scientists nor do they critique all issues relevant to the biases of basic research. SYRCLE is relevant for preclinical mouse studies which are designed on the same basis as clinical trials. Laboratory based research does not formally randomise experiments, nor use allocation concealment or blinding. Good practice should ensure whoever sets up the experiment does not measure the outcome. In addition, all efforts are made to ensure that the experiment and controls are treated alike during all stages of the experiment. Based on our experience of designing and implementing laboratory experiments, we created three new tools to judge the quality of the research question under review. We believe that the fundamental biases in basic research derive from the choice of model (and its functional validation if necessary) and from demonstrating the experimental outcome is valid for the effect you want to measure. The reliability of a result is as important as the quality, therefore we also interrogated the statistical reporting. Therefore, we created tools to judge the choice and validation of the model system(s) and the organelle marker(s), and in addition, the reliability or imprecision of the reported result. The summary judgements are provided in Table 1 and the tools and detailed judgements are given in S2S4 Tables.

Imprecision

An imprecision tool was created, to judge how well the authors reported statistical variability, sample size and statistical methodology (S2 Table). We determined the minimum requirement for low risk was that the authors reported technical repeats, interassay repeats and variability. Only two studies (7%) had an overall judgement of low risk [46,48], the majority of studies (77%; 24/31) were 'unclear'. Five (16%) studies had an overall rating of 'high concern'; the main reasons for which were low technical repeats (<10).

Analysis of the individual signalling questions indicated that 84% of studies reported a technical repeat (Table 1), 19% of studies reported interassay repeats and 39% reported a measure of variability. None of the studies considered whether the pooling of data from interassay repeats was appropriate (heterogeneity) and none of the studies calculated a sample size. Only one study described how indeterminate results were handled and only 29% clearly described the statistical methods (summarised in Fig 2, details in S2 Table).

thumbnail
Fig 2. Assessments of imprecision, model validity and marker validity.

Black bars = number of studies for judgements of 'no' or 'not reported'. White bars = number of studies for judgements of 'yes'. Light grey bars = number of studies 'not applicable' for question (unclear for imprecision). Dark grey bars = number of studies where the question was justified with a reference. *based on our judgements, see relevant tool for explanation. Judgements were based on the author's reported organelle marker, primary model and the result with the highest technical repeat.

https://doi.org/10.1371/journal.pone.0178645.g002

The lack of sample size calculations, technical repeats, interassay repeats and variability measures is of great concern. These are the basic determinants for demonstrating the reliability of a result and are essential for clinical translation or for deriving firm conclusions. For the purposes of a working tool we decided that an appropriate technical repeat should be greater than 10 and that there should be at least three interassay repeats. This number was determined by Charan and Kantharia 2013 [55], who suggested that in the absence of a power calculation a law of diminishing returns can be applied for a crude estimation of sample size in animal studies. We would like to emphasise that this is not a replacement for sample size calculations and authors should strive to perform pilot studies to identify the best sample size for their study. We determined that only 19% of studies had a sufficient sample size (Fig 2). We also determined whether the statistical methods employed were appropriate or not. For this review the judgement was based on whether the authors used appropriate t- tests (paired or unpaired). Although 29% of studies reported statistical methods, only 23% of studies were judged to have an appropriate statistical test (Fig 2). For two of the nine studies, which reported statistical methods [23,36] it was unclear whether the result was derived from pooling interassay repeats or not and therefore we could not judge if the test was appropriate. It should be noted that other tests besides the t–test could be appropriate for analysis but these were not reported by the included studies and therefore the guidance here concentrated on the t-test.

Model validity

A model validity tool was created (S3 Table), to judge how well the authors reported the details of the model and whether the model was valid for the research question (e.g. if a stem cell was investigated, did the authors prove stem cell function?). A variety of two and three dimensional models using whole organisms, primary cultures, tissue and cell lines were employed in the studies. We determined the minimum requirement for low risk was that all domains were clearly reported and that there were no additional concerns. Most studies (61%; 19/31) were judged to be valid, ten studies (32%) were judged to be unclear for model validity. Two studies (7%) were judged to have high concern for model validity [22,47]; both models were described as 'stem cell-like' without providing experimental evidence to confirm this (e.g. pluripotency and self-renewal).

Analysis of the individual signalling questions (Fig 2, details in S3 Table) indicated that 83% (5/6) of relevant studies reported an ethical statement (whole mouse models or human tissue). 74% reported clear descriptions of the model, 71% described routine maintenance and 100% provided clear details of how the model was manipulated for experimentation. Such reporting details are important to allow a comparison of results between similar models and to clearly understand how the experiments were performed and whether sufficient details were available to repeat the experiment. Closer validation of the studies indicated that only one of three stem cell models provided evidence of stem cell function. None of the six studies which employed cell lines reported that they routinely checked for the absence of contaminants in the cultures. This is an important consideration because cell lines can be many years old and may have deviated from the original clone or become contaminated by other cells or microorganisms if not strictly maintained. Both studies using primary cultures provided data to confirm the tissue origin of the cells.

With regards to the model, other points to consider when establishing which studies provide the most valid data include: the cellular complexity of the model and the proximity of the phylogenic order to the organism stated in the research question. The best models are those of whole organisms or tissue, where the physiology, architecture and cellular niches are present and recapitulate tissue. In contrast, cell cultures grown in two dimensions do not recapitulate tissue architecture. This can be improved with the use of three dimensional cultures or multiple cell types grown in a co-culture system. Therefore, the best human evidence for the 'asymmetric inheritance of organelles’ in human models was derived from endosomes (in adult haematopoietic stem cells), the mid body (embryonic stem cells, adult epithelial cell lines) and proteosomes (adult embryonic cell line) [20,23,30]. This evidence largely came from two-dimensional cell culture models which illustrates that, for clinical translation, there is often a trade-off between how direct the evidence is and the complexity of the model. Before basic research is translated to the clinic there should always be evidence in the most valid human model available (cultures or tissue).

Marker validity

A marker validation tool was created to judge the ability of the selected marker to correctly identify the organelle (S4 Table). This tool was also used to evaluate whether a marker can correctly identify a given cell (if relevant). We determined the minimum requirement for low risk was that all domains were clearly reported and there were no additional concerns.

One study was judged to have clear validity [49], the majority of studies were judged to be unclear (96.5%; 30/31), indicating that for most studies there were insufficient details to judge if the markers were appropriate. One study (3.5%) was judged to have high concern for marker validity [45]; in this study the asymmetry of SARA-marked early endosomes was reported, however the localisation of SARA was validated using co-localisation with Rab5c, but Rab5c did not exhibit asymmetry and therefore a conflict exists between these results.

Analysis of the individual signalling questions (Fig 2, details in S4 Table) indicated that only 43% of studies reported the choice of marker ‘a priori’. In basic research this is not unusual, authors tend not to state whether the marker used in a study was the only marker employed or whether several markers were tried and only one worked (and was reported). In a clinical review this would be referred to as 'selective outcome reporting’. This is still an important consideration and bias for basic research. Authors should be encouraged to report negative results for additional markers, as this has relevance for the validity or reliability of the measured result. 55% of studies reported a functional validation for the marker. Markers for the centrosome, mid body and spectrosome were rated as valid because their cellular location (using microscopy) confirmed their function. Mitochondrial and proteosomal markers were generally confirmed by co-localisation with a second marker or by functional validation. However, the absence of marker validation for many studies led the review authors to obtain further evidence from GeneCards and Flybase databases which confirmed the function of 88% of relevant markers. Only six studies used colocalization with a second marker to confirm the cellular location.

In studies which used genetic manipulation to mark the organelle only two studies (6%) provided data to confirm that the manipulation did not influence the result [29,44]. Most studies reported clear methods for imaging techniques, but 9 of 25 (36%) confocal studies did not consider the focal plane.

Very few studies (17%) reported positive controls and none reported negative controls. The reporting of controls should be fundamental, but their reporting is often overlooked often due to the space restriction applied by scientific journals, compounded by the complexity and number of experiments reported therein. We believe that authors should still report these results in appendices, especially when imaging techniques are used or threshold values are applied to decide between a negative or positive result. The importance of threshold values is illustrated by the different methods used to quantitate asymmetry (of the marker). Asymmetric inheritance was quantified by subjective assessment in 65% (20/31) of included studies (Fig 2; S4 Table); whereby the authors used visual inspection of the marker to decide whether asymmetry had occurred. Subjective assessment is not considered reliable unless verified by an independent researcher, but verification was not reported by any study. Table 2 summarises the studies which used quantitative methods to measure asymmetry; based on the pixel intensity of fluorescent microscopic images.

All the studies using quantitative methods used a threshold to determine whether asymmetry had occurred. The threshold was based on the ratio between pixels measured in one daughter cell compared to the other. A deviation away from 1 (or 1:1) was sufficient for one study to indicate asymmetry, but the fold increase reported ranged from 1 to 15 overall. The thresholds were not predefined and illustrate that, what one authors considers as asymmetry may not be sufficient for another researcher. For example, if we select a ratio of 3 as the threshold only 5/11 studies in Table 2 would meet this criterion. Two of the studies based a positive result of the proportion of cells in a given population which displayed asymmetric distribution of the marker between two daughter cells. One study [45] reported that asymmetry had not occurred because it was only observed in 7–9% of the cell population, whilst in another study this result would have been considered as asymmetry (0.8 to 11.5% of the population)[48]. Therefore, systematic reviewers need to carefully interrogate how the outcome is defined and should investigate closely the potential heterogeneity in the modes of measurement and any thresholds applied. Consideration of the outcome definition is a consideration for the reviewer but is not a limitation of the study; how well the results can be combined is influenced by the heterogeneity inherent in these descriptions.

Risk of bias assessment and validity in other cell based systematic reviews

To establish how well other systematic reviews of based research, including cell assays, have assessed the quality of the included studies we performed a second review. We identified 16 systematic reviews (flow diagram in S1 File). We aimed to identify the use of formal or informal reporting guidelines, risk of bias tools or consideration of marker and model validity. We gave special focus to imprecision. The results are summarised in Fig 3, and further details are presented in S6 Table.

thumbnail
Fig 3. Assessments of guideline adherence, risk of bias, imprecision and validity in similar systematic reviews.

Black bars = number of reviews not providing information for validity reporting/ assessment. White bars = number of reviews which did provide information for validity reporting/ assessment.

https://doi.org/10.1371/journal.pone.0178645.g003

Fifty six percent of the reviews reported adherence to PRISMA or ARRIVE reporting guidelines. Only 44% reported a formal risk of bias assessment (Cochrane, GRADE, SYRCLE, Tool for Studies with Diverse Designs), one reported their own grading system [56]. Of the four reviews reporting Cochrane or SYRCLE tools it was notable that the majority of domains were rated as unclear, reiterating that the study designs are not based on randomisation, allocation concealment or blinding [5760]. This finding agrees with our own assessments and indicates that basic research requires its own risk of bias tools. 56% of the reviews did not consider how relevant or valid the models were to the research question. Five reviews discussed the limitations of cell lines or culture methods [6165], one review considered whether the origin of the primary culture was tracked [66]. 75% reviews did not consider how valid the markers were. One review did consider the marker validity [66], three discussed outcome validity or quantification methods [57,62,63]. 69% of reviews did not consider the impact of imprecision; those that did consider any aspect of imprecision reported minimal details. If we consider the minimum requirements for our imprecision tool, only 19% of reviews reported technical repeats, 13% reported interassay repeats and 19% reported the statistical variability of the result. These results clearly demonstrate that current systematic reviews carried out in basic research are not assessing the quality of the research nor the imprecision of the results. We found that the reviews which do assess quality are not conforming to any useful standards. Encouragingly some of the signalling questions we have devised for our model and marker validity tools where identified as important considerations in other reviews.

Conclusions

Using a systematic review of basic science studies, we have identified the asymmetric inheritance of several organelles occurs across many eukaryotic animal species. However, we also found that current risk of bias tools are not adequate and are not widely used in basic science systematic reviews. Improvements are needed in the reporting of basic science and to improve the tools to judge the quality of basic scientific studies.

To overcome issues surrounding quality we have developed new tools to assess the imprecision of outcomes, marker validity and model validity. In the review presented here, these tools demonstrate that authors do not present sufficient information to judge the statistical reliability of their results. Most models were valid, except when specific cell types were employed (e.g. adult stem cells). Half of the included studies did not validate the function of their marker and only a small minority reported controls. These tools represent a first step towards the improvement of basic science appraisals using systematic review. We believe that the tools created here will be broadly applicable to many other fields and we encourage other researchers to use them as a starter for other assays e.g. cell division or microarray experiments.

Our results indicate that the requirements for publishing basic research are not rigorous, especially with regards to the statistical reliability of the results. The studies in this review were published between 1991 to 2015 in a range of journals. Although attempts are being made to improve basic science reporting (see checklists for Nature) this guidance is going unnoticed or is not a strict requirement for publication. As ‘sloppy science’ becomes increasingly recognised [6769] it is likely that scientists will give more credence to the evaluation and reporting of their work.

This systematic review also highlights the need for basic science reviewers to identify the best quality evidence and the most reliable, rather than just a summary of results. Our evaluation of other basic science systematic reviews indicated that none successfully considered the statistical reliability of the results and few judged the quality. We firmly believe that the ‘best’ results are dependent on both the quality of the research and the statistical reliability. In addition, the reviewer must closely interrogate the outcome definitions, to determine whether they can be combined or compared across studies. The best systematic reviews are not just a process but indicate the best available evidence for a given question. This is especially important for research that will be used for clinical translation. A systematic review evaluating statistical reliability and validity will provide better evidence to ensure greater clinical success, reduce early investment costs and reduce unnecessary animal experimentation.

Supporting information

S1 Table. Inclusion and exclusion criteria for screening.

https://doi.org/10.1371/journal.pone.0178645.s003

(DOCX)

S2 Table. Imprecision tool and assessments.

https://doi.org/10.1371/journal.pone.0178645.s004

(DOCX)

S3 Table. Model validity tool and assessments.

https://doi.org/10.1371/journal.pone.0178645.s005

(DOCX)

S4 Table. Marker validity tool and assessments.

https://doi.org/10.1371/journal.pone.0178645.s006

(DOCX)

S6 Table. Validity assessments and adherence to reporting guidelines in systematic reviews of cellular research.

https://doi.org/10.1371/journal.pone.0178645.s008

(DOC)

Acknowledgments

The authors wish to express their thanks to Gill Worthy for her helpful statistical discussions and comments on the manuscript.

Author Contributions

  1. Conceptualization: SL.
  2. Data curation: JR SL.
  3. Formal analysis: SL.
  4. Investigation: SL AC JR.
  5. Methodology: SL JR.
  6. Project administration: SL.
  7. Supervision: SL.
  8. Validation: SL AC.
  9. Visualization: SL.
  10. Writing – original draft: SL JR AC.
  11. Writing – review & editing: SL JR AC.

References

  1. 1. Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001;322: 98–101. pmid:11154628
  2. 2. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biology. 2010;8: e1000412. pmid:20613859
  3. 3. Hooijmans CR, Rovers MM, de Vries RB, Leenaars M, Ritskes-Hoitinga M, Langendam MW. SYRCLE’s risk of bias tool for animal studies. BMC Medical Research Methodology. 2014;14: 43. pmid:24667063
  4. 4. de Vries RBM, Wever KE, Avey MT, Stephens ML, Sena ES, Leenaars M. The Usefulness of Systematic Reviews of Animal Experiments for the Design of Preclinical and Clinical Studies. ILAR Journal. 2014;55: 427–437. pmid:25541545
  5. 5. Lang S, Frame F, Collins A. Prostate cancer stem cells. The Journal of Pathology. 2009;217: 299–306. pmid:19040209
  6. 6. Lang SH, Anderson E, Fordham R, Collins AT. Modeling the prostate stem cell niche: an evaluation of stem cell survival and expansion in vitro. Stem cells and development. 2010;19: 537–546. pmid:20102283
  7. 7. Birnie R, Bryce SD, Roome C, Dussupt V, Droop A, Lang SH, et al. Gene expression profiling of human prostate cancer stem cells reveals a pro-inflammatory phenotype and the importance of extracellular matrix interactions. Genome Biol. 2008;9: R83. pmid:18492237
  8. 8. Richardson GD, Robson CN, Lang SH, Neal DE, Maitland NJ, Collins AT. CD133, a novel marker for human prostatic epithelial stem cells. J Cell Sci. 2004;117: 3539–3545. pmid:15226377
  9. 9. Whiting J, Wolff R, Deshpande S, Di Nisio M, Duffy S, Hernandez A, et al. Cannabinoids for medical use: A systematic review and meta-analysis. JAMA. 20115;313: 2456–2473.
  10. 10. Korobelnik J- F, Kleijnen J, Lang SH, Birnie R, Leadley RM, Misso K, et al. Systematic review and mixed treatment comparison of intravitreal aflibercept with other therapies for diabetic macular edema (DME). BMC ophthalmology. 2015;15: 52. pmid:25975823
  11. 11. Lang S, Armstrong N, Noake C1, Kleijnen J. A systematic review to evaluate the effectiveness of enzyme replacement therapy for lysosomal storage disorders in comparison to the treatment of similar diseases with higher prevalence. RARE DISEASES AND ORPHAN DRUGS An International Journal of Public Health. 2014;1.
  12. 12. Leadley RM, Lang S, Misso K, Bekkering T, Ross J, Akiyama T, et al. A systematic review of the prevalence of Morquio A syndrome: challenges for study reporting in rare diseases. pmid:25404155
  13. 13. Grecchi S, Malatesta M. Visualizing endocytotic pathways at transmission electron microscopy via diaminobenzidine photo-oxidation by a fluorescent cell-membrane dye. European Journal of Histochemistry. 2014;58.
  14. 14. Higgins JPT, Altman DG. Higgins 2011 Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Cochrane Handbook for Systematic Reviews of Interventions. 2011.
  15. 15. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Medicine. 2009;6: e1000097. pmid:19621072
  16. 16. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic Reviews. 2016;5: 210. pmid:27919275
  17. 17. Sterne JAC, Higgins JPT, Reeves BC on behalf of the development group for ROBINS-I: a tool for assessing Risk Of Bias In Non-randomized Studies of Interventions, Version 7 March 2016. Available from http://www.riskofbias.info [accessed 22.7.16].
  18. 18. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52: 377–384. pmid:9764259
  19. 19. Whiting PF. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Annals of Internal Medicine. 2011;155: 529. pmid:22007046
  20. 20. Beckmann J, Scheitza S, Wernet P, Fischer JC, Giebel B. Asymmetric cell division within the human hematopoietic stem and progenitor cell compartment: identification of asymmetrically segregating proteins. Cell. 2007;109: 5494–5501.
  21. 21. Kuo T- C, Chen C- T, Baron D, Onder TT, Loewer S, Almeida S, et al. Midbody accumulation through evasion of autophagy contributes to cellular reprogramming and tumorigenicity. Nature Cell Biology. 2011;13: 1214–1223. pmid:21909099
  22. 22. Katajisto P, Dohla J, Chaffer CL, Pentinmikko N, Marjanovic N, Iqbal S, et al. Asymmetric apportioning of aged mitochondria between daughter cells is required for stemness. Science. 2015;348: 340–343. pmid:25837514
  23. 23. Ogrodnik M, Salmonowicz H, Brown R, Turkowska J, Średniawa W, Pattabiraman S, et al. Dynamic JUNQ inclusion bodies are asymmetrically inherited in mammalian cell lines through the asymmetric partitioning of vimentin. Proceedings of the National Academy of Sciences of the United States of America. 2014;111: 8049–54. pmid:24843142
  24. 24. Januschke J, Llamazares S, Reina J, Gonzalez C. Drosophila neuroblasts retain the daughter centrosome. Nature communications. Nature Publishing Group; 2011;2: 243.
  25. 25. Rose LS, Kemphues K. The let-99 gene is required for proper spindle orientation during cleavage of the C. elegans embryo. Development (Cambridge, England). 1998;125: 1337–1346.
  26. 26. Lin H, Spradling AC. Fusome asymmetry and oocyte determination inDrosophila. Developmental Genetics. 1995;16: 6–12. pmid:7758245
  27. 27. Salzmann V, Chen C, Chiang C-YA, Tiyaboonchai A, Mayer M, Yamashita YM. Centrosome-dependent asymmetric inheritance of the midbody ring in Drosophila germline stem cell division. Molecular biology of the cell. 2014;25: 267–75. pmid:24227883
  28. 28. Dalton CM, Carroll J. Biased inheritance of mitochondria during asymmetric cell division in the mouse oocyte. Journal of Cell Science. 2013;126: 2955–2964. pmid:23659999
  29. 29. Anderson CT, Stearns T. Centriole Age Underlies Asynchronous Primary Cilium Growth in Mammalian Cells. Current Biology. Elsevier Ltd; 2009;19: 1498–1502. pmid:19682908
  30. 30. Kuo T- C, Chen C- T, Baron D, Onder TT, Loewer S, Almeida S, et al. Midbody accumulation through evasion of autophagy contributes to cellular reprogramming and tumorigenicity. Nature cell biology. 2011;13: 1214–23. pmid:21909099
  31. 31. Goss JW, Toomre DK. Both daughter cells traffic and exocytose membrane at the cleavage furrow during mammalian cytokinesis. Journal of Cell Biology. 2008;181: 1047–1054. pmid:18573914
  32. 32. Chang JT, Ciocca ML, Kinjyo I, Palanivel VR, McClurkin CE, DeJong CS, et al. Asymmetric Proteasome Segregation as a Mechanism for Unequal Partitioning of the Transcription Factor T-bet during T Lymphocyte Division. Immunity. Elsevier Inc.; 2011;34: 492–504. pmid:21497118
  33. 33. Wang X, Tsai J, Imai JH, Lian W, Vallee RB, Shi S. Asymmetric centrosome inheritance maintains neural progenitors in the neocortex. Nature. 2009;461: 947–955. pmid:19829375
  34. 34. Tamura M, Nemoto S. Reproductive maternal centrosomes are cast off into polar bodies during maturation division in starfish oocytes. Experimental cell research. 2001;269: 130–9. pmid:11525646
  35. 35. Holy J, Schatten G. Differential behavior of centrosomes in unequally dividing blastomeres during fourth cleavage of sea urchin embryos. J Cell Sci. 1991;98: 423–431. Available: http://jcs.biologists.org/content/98/3/423.long pmid:2055969
  36. 36. Conduit PT, Raff JW. Cnn Dynamics Drive Centrosome Size Asymmetry to Ensure Daughter Centriole Retention in Drosophila Neuroblasts. Current Biology. Elsevier Ltd; 2010;20: 2187–2192. pmid:21145745
  37. 37. Rebollo E, Sampaio P, Januschke J, Llamazares S, Varmark H, González C. Functionally Unequal Centrosomes Drive Spindle Orientation in Asymmetrically Dividing Drosophila Neural Stem Cells. Developmental Cell. 2007;12: 467–474. pmid:17336911
  38. 38. Yamashita YM, Mahowald AP, Perlin JR, Fuller MT. Asymmetric inheritance of mother versus daughter centrosome in stem cell division. Science (New York, NY). 2007;315: 518–21.
  39. 39. Rusan NM, Peifer M. A role for a novel centrosome cycle in asymmetric cell division. The Journal of Cell Biology. 2007;177: 13–20. pmid:17403931
  40. 40. Shimizu T. Behaviour of centrosomes in early Tubifex embryos: Asymmetric segregation and mitotic cycle dependent duplication. Roux’s Archives of Developmental Biology. 1996;205: 290–299. pmid:28306032
  41. 41. Piotrowska-nitsche K, Caspary T. Live imaging of individual cell divisions in mouse neuroepithelium shows asymmetry in cilium formation and Sonic hedgehog response. 2012;1: 1–10.
  42. 42. Smyth JT, Schoborg TA, Bergman ZJ, Riggs B, Rusan NM. Proper symmetric and asymmetric endoplasmic reticulum partitioning requires astral microtubules. Open biology. 2015;5.
  43. 43. Coumailleau F, Fürthauer M, Knoblich JA, González-Gaitán M. Directional Delta and Notch trafficking in Sara endosomes during asymmetric cell division. Nature. Macmillan Publishers Limited. All rights reserved; 2009;458: 1051–5.
  44. 44. Emery G, Hutterer A, Berdnik D, Mayer B, Wirtz-Peitz F, Gaitan MG, et al. Asymmetric Rab11 Endosomes Regulate Delta Recycling and Specify Cell Fate in the Drosophila Nervous System. Cell. 2005;122: 763–773. pmid:16137758
  45. 45. Kressmann S, Campos C, Castanon I, Fürthauer M, González-Gaitán M. Directional Notch trafficking in Sara endosomes during asymmetric cell division in the spinal cord. Nature cell biology. Nature Publishing Group; 2015;17: 333–9. pmid:25706234
  46. 46. Loubéry S, Seum C, Moraleda A, Daeden A, Fürthauer M, Gonzalez-Gaitan M. Uninflatable and Notch Control the Targeting of Sara Endosomes during Asymmetric Division. Current Biology. 2014;24: 2142–2148. pmid:25155514
  47. 47. Montagne C, Gonzalez-Gaitan M. Sara endosomes and the asymmetric division of intestinal stem cells. Development (Cambridge, England). 2014;141: 2014–23.
  48. 48. Rivolta MN, Holley MC. Asymmetric segregation of mitochondria and mortalin correlates with the multi-lineage potential of inner ear sensory cell progenitors in vitro. Developmental Brain Research. 2002;133: 49–56. pmid:11850063
  49. 49. Gallo CM, Wang JT, Motegi F, Seydoux G. Cytoplasmic partitioning of P granule components is not required to specify the germline in C. elegans. Science (New York, NY). 2010;330: 1685–9.
  50. 50. Basham SE, Rose LS. Mutations in ooc-5 and ooc-3 disrupt oocyte formation and the reestablishment of asymmetric PAR protein localization in two-cell Caenorhabditis elegans embryos. Developmental biology. 1999;215: 253–263. pmid:10545235
  51. 51. Boyd L, Guo S, Levitan D, Stinchcomb DT, Kemphues KJ. PAR-2 is asymmetrically distributed and promotes association of P granules and PAR-1 with the cortex in C. elegans embryos. Development (Cambridge, England). 1996;122: 3075–3084.
  52. 52. Pang Ishidate, Nakamura Shirayama, Trzepacz Schubert, et al. The minibrain kinase homolog, mbk-2, is required for spindle positioning and asymmetric cell division in early C. elegans embryos. Developmental biology. 2004;265: 127–139. pmid:14697358
  53. 53. de Cuevas M, Spradling a C. Morphogenesis of the Drosophila fusome and its implications for oocyte specification. Development (Cambridge, England). 1998;125: 2781–2789.
  54. 54. Deng W, Lin H. Spectrosomes and Fusomes Anchor Mitotic Spindles during Asymmetric Germ Cell Divisions and Facilitate the Formation of a Polarized Microtubule Array for Oocyte Specification inDrosophila. Dev Biol. 1997;189: 79–94. pmid:9281339
  55. 55. Charan J, Kantharia N. How to calculate sample size in animal studies? Journal of Pharmacology and Pharmacotherapeutics. 2013;4: 303. pmid:24250214
  56. 56. Xiao Z, Li C, Shan J, Luo L, Feng L, Lu J, et al. Mechanisms of renal cell apoptosis induced by cyclosporine A: A systematic review of in vitro studies. American Journal of Nephrology. 2011;33: 558–566. pmid:21613783
  57. 57. Batista Napotnik T, Reberšek M, Vernier PT, Mali B, Miklavčič D. Effects of high voltage nanosecond electric pulses on eucaryotic cells (in vitro): A systematic review. Bioelectrochemistry. 2016;110: 1–12. pmid:26946156
  58. 58. Shanbhag S, Pandis N, Mustafa K, Nyengaard JR, Stavropoulos A. Cell Cotransplantation Strategies for Vascularized Craniofacial Bone Tissue Engineering: A Systematic Review and Meta-Analysis of Preclinical In Vivo Studies. Tissue Engineering Part B: Reviews. 2016;23: ten.teb.2016.0283.
  59. 59. Ramamoorthi M, Bakkar M, Jordan J, Tran SD. Osteogenic Potential of Dental Mesenchymal Stem Cells in Preclinical Studies: A Systematic Review Using Modified ARRIVE and CONSORT Guidelines. Stem cells international. 2015;2015: 378368. pmid:26106427
  60. 60. Hynes K, Bright R, Proudman S, Haynes D, Gronthos S, Bartold M. Immunomodulatory properties of mesenchymal stem cell in experimental arthritis in rat and mouse models: A systematic review. Seminars in Arthritis and Rheumatism. Elsevier; 2016;46: 1–19. pmid:27105756
  61. 61. Pavan LM, Rego DF, Elias ST, De Luca Canto G, Guerra EN. In vitro Anti-Tumor Effects of Statins on Head and Neck Squamous Cell Carcinoma: A Systematic Review. PLoS One. 2015;10: e0130476. pmid:26098683
  62. 62. Bus P, Siersema PD, Van Baal JWPM. Cell culture models for studying the development of Barrett’s esophagus: A systematic review. Cellular Oncology. 2012;35: 149–161.
  63. 63. Gudbergsson JM, Johnsen KB, Skov MN, Duroux M. Systematic review of factors influencing extracellular vesicle yield from cell cultures. Cytotechnology. Springer Netherlands; 2016;68: 579–592. pmid:26433593
  64. 64. Korpershoek J V., de Windt TS, Hagmeijer MH, Vonk LA, Saris DBF. Cell-Based Meniscus Repair and Regeneration: At the Brink of Clinical Translation? Orthopaedic Journal of Sports Medicine. 2017;5: 232596711769013.
  65. 65. Rahman NA, Rasil ANHM, Meyding-Lamade U, Craemer EM, Diah S, Tuah AA, et al. Immortalized endothelial cell lines for in vitro blood-brain barrier models: A systematic review. Brain research. 2016;1642: 532–545. pmid:27086967
  66. 66. Harkin DG, Foyn L, Bray LJ, Sutherland AJ, Li FJ, Cronin BG. Concise reviews: can mesenchymal stromal cells differentiate into corneal cells? A systematic review of published data. Stem cells (Dayton, Ohio). 2015;33: 785–791.
  67. 67. Moylan EC, Kowalczuk MK. Why articles are retracted: a retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open. 2016;6: e012047. pmid:27881524
  68. 68. Bouter LM, Tijdink J, Axelsen N, Martinson BC, ter Riet G. Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Research Integrity and Peer Review. 2016;1: 17.
  69. 69. Vogt L, Reichlin TS, Nathues C, Würbel H. Authorization of Animal Experiments Is Based on Confidence Rather than Evidence of Scientific Rigor. Dirnagl U, editor. PLOS Biology. 2016;14: e2000598. pmid:27911892