Introduction

The search for new cancer therapies is exacerbated by the fact that cancer is a highly heterogeneous disease. Cancer cells do not only vary phenotypically between patients and affected organs, but already within single tumours. This phenomenon was first explained by a clonal or stochastic model of cancer, where the heterogeneity of the cells is due to continuing mutagenesis [1,2,3]. Here, all cells have tumourigenic potential and, under the right external conditions, can re-establish a tumour. Early on it was challenged by the observation that in different kinds of cancers only a very small fraction of tumour cells proliferated when seeded in vitro or in vivo in mice [4,5] and that only certain primary tumour cells are capable to form metastasis in distant sites [6].

When Bonnet & Dick showed that in Acute Myeloid Leukaemia (AML), this subset of cells had an exclusive phenotype (CD34+/CD38–) [7], a new model to explain cancer cell heterogeneity was proposed: cancer stem cells (CSC) [8,9]. Here, a tumour is seen as an “abnormal organ” to which “the principles of normal stem cell biology can be applied” [8]. Analogous to normal tissues, the heterogeneous cell population of a tumour is exclusively replenished by multipotent CSCs that are able to self-renew and give rise to phenotypically different cells through asymmetric cell divisions. This concept is supported by the observation that many of the pathways that regulate self-renewal in stem cells were found to be active in cancer cells, such as the Wnt, the Sonic hedgehog (Shh) and the Notch pathway [8,10]. It also offers an explanation to the riddle why some cancer patients stay, seemingly cancer free, in remission for prolonged episodes of time before they relapse. Quiescence, a property of stem cells, would give CSCs the ability to survive treatment with chemotherapy in a dormant state to later re-enter the cell cycle and repopulate a tumour cell population [11].

While similar evidence was found for many cancer types, the CSC model has since been challenged as well. For each of the unique characteristics that were used to define CSCs as a separate population of cells (rarity, specific surface markers, tumourigenicity, differentiation potential, unlimited capacity for self-renewal, resistance against chemotherapy), contradictory evidence was found [12]. Additionally, further research has indicated that xenotransplantation, the “gold standard for in vivo testing” of the CSC model [11], might not be sufficient proof for an exclusive tumourigenicity of cells identified as CSCs. Quitana et al. found a surprising 25% of melanoma cells to be capable of tumourigenesis when they were injected into even more immunocompromised NOD/SCID IL2Rγnull mice (NGS) [13,14]. These tumourigenic cells could not be defined by CD133 or any other previously identified CSC marker and were phenotypically highly heterogeneous. It is thus arguable whether previous studies simply selected for cells with a unique capability to proliferate in mice rather than tumourigenic potential in humans. Another explanation for the conflicting results could be that some forms of cancer are indeed hierarchically organized with CSCs at the top, while others, like melanoma, are not.

Furthermore, a combination of both models is possible where CSCs do exist and are clonally selected, explaining the co-existence of different phenotypes of CSCs in studies on AML and glioblastoma [15,16]. Or maybe CSCs do exist but rather describe a transient state of cancer cells, which are able to switch their phenotype [17,18].

The two theoretical models are highly relevant because they result in different therapeutic strategies [19]. In the stochastic model, drugs must have a cytotoxic effect on all the heterogeneous cancer cell types of a tumour. In contrast, the existence of CSCs would have the attractive implication that it is only necessary to target a small fraction of cells in order to eradicate cancer from a patient. Additionally, stem cells have mechanisms of chemotherapy resistance such as a high expression of ABC transporters that if shared with CSCs need to be addressed in therapy development [9]. Accordingly, several new therapeutic strategies have been developed in the last years based on the assumption that CSCs are the drivers of cancer pathogenesis and progression. A focal area of research on these therapies has been the development of monoclonal antibodies or vaccines against the identified CSC markers such as CD133 or CD44, and small molecule inhibitors of receptors expressed predominantly by CSCs [20,21,22], although some researchers have criticized this strategy [12,19,23,24].

Materials & methods

To generate a dataset with all available empirical studies on cancer heterogeneity, we conducted a systematic literature review following the PRISMA-P 2015 guidelines for systematic reviews and meta-analyses (see PRISMA diagram in S1 Fig; PRISMA checklist in S1 Table) [32]. First, we systematically searched the Web of Science (WoS) that tested the broad definition of the CSC hypothesis. Using the terms “cancer stem cells” or “tumor-initiating cells” or “tumour-initiating cells” or “tumorigenic cells” or “tumourigenic cells” or “tumor heterogeneity” or “tumour heterogeneity" on 6 September 2016. As this work is a first test whether the HoH approach is a useful method for CSC research synthesis in principle, we limited the scope of the study to the first 1000 publications (sorted by publication date) out of a total of 14,291 publications returned by the search. The truly relevant publications were identified by their titles and abstracts (returning 115 possibly relevant publications) and then thoroughly reading the publications in question. In this way, we identified a sample of 51 publications that tested whether a certain cell sub-population had CSC properties–independently of whether they were called as such or not. We thereby relied on the consensus definition of stem cells from the AARC cancer stem cell workshop [32], which defines a CSC as “a cell within a tumour that possess the capacity to self-renew and to cause the heterogeneous lineages of cancer cells that comprise the tumour.” Additional characteristics ascribed to CSCs when the concept was conceived and still implemented by current studies are rarity and the expression of stem cell markers [8,33]. We thus selected all studies that tested whether a subset of cancer cells complied with these characteristics compared to bulk tumour cells. Studies that only used cells the authors defined as CSCs and did not compare them to the rest of the cancer cell population were rejected. We also excluded theoretical (modelling) studies and reviews. In other words, the HoH analysis only included primary empirical studies.

From each of these 51 publications, we extracted the separate empirical tests the authors had performed, and recorded for each of them in a database: cancer type, model cancer cells (primary tumour cells, cell line, patient-derived xenograft …), animal model, CSC isolation method, indicator, assay, number of patient samples or cell lines tested, and result of the respective test.

To build an HoH of the CSC model, we defined different sub-hypotheses by classifying the evidence on three levels: (1) indicators assessed to test the existence of CSCs, (2) experimental assay and (3) model cancer cells used. On the 1st level of sub-hypotheses, the different indicators that are used to define CSCs in contrast to the bulk of cancer cells are located. As outlined above, these are the indicators ubiquitously used as defining characteristics of CSCs [8,33,34]. Further down, the 2nd level of sub-hypotheses contains the type of assays used to measure the respective indicators. For instance, differences in tumourigenicity between certain cancer cell subsets can be assessed by different empirical tests: by comparing the overall tumourigenic capacity of cell subsets (as measured by respective latency, number of tumours induced or overall tumour volume) after subcutaneous injections or orthotopic transplantations of cancer cells in vivo; or by comparing the frequency of tumour-inducing cells determined by limiting dilution injections in vivo. On the 3rd level of sub-hypotheses, the different origins of cells used in the assays are discriminated: human or other species, cultured or primary cancer tissue samples.

Finally, each empirical test recorded in the systematic review above was assigned to the sub-hypotheses at the lowest level, and was classified as either supporting, being undecided or questioning the sub-hypothesis in question (if the evidence was conflicting because e.g. the results from different cell lines tested were contradictory, the test was classified as “undecided”). For this scoring approach, we considered primarily effect sizes and the results of statistical significance tests given by the respective publications. The following specifications were made: the characteristic “rarity” is a major aspect of the CSC concept, but is not statistically evaluated in most studies. Here, a percentage of cells under 1% of the whole cell population was defined as “rare”. Another specification was necessary for assessing the support from expression analysis of RNA or protein markers. In many papers, several markers are assessed and therefore we set a cut-off at more than 50% of measured genes or proteins that had to be significantly differentially expressed to be counted as “supporting” evidence. If no measured stem-cell marker was differentially expressed, the evidence was counted as “questioning”, in-between as “undecided”. Many tumourigenicity assays only included a very small number of mice, and study authors often did not calculate statistical significance. But the evidence was still counted as “supporting” under the condition that tumours did form when the cell set defined as CSC was injected, while no tumours formed in mice when bulk tumour cells were injected. Please note that the HoH approach can also be combined with a fully quantitative meta-analytic framework if effect sizes are comparable, but this was not the case here, thus we applied the semi-quantitative scoring approach outlined in this paragraph.

To evaluate the support for the CSC model, the percentage of supporting, undecided and questioning evidence was calculated for each sub-hypothesis on all levels of the HoH. To account for the fact that most studies included several empirical tests, which are thus not independent of each other, each publication was only included once for each sub-hypothesis or the main hypothesis. Additionally, the amount of support was calculated in the same manner for different types of cancer, different model organisms and types of cancer cells used. We used a Mann-Whitney U-test to test for significant differences between the empirical support for sub-hypotheses and between different kinds of cancer etc. where the number of studies was ≥5. Statistical tests were performed using the software PAST version 3.23 (March 2019) [35].

Results

The 51 relevant publications that we identified studied putative CSCs in very different kinds of cancer using different model systems and different laboratory methods. As the authors tested several components of the CSC hypothesis in each study, the resulting number of empirical tests was 174 (on average 3–4 tests per study). A table with all empirical tests analysed is included in the S2 Table. We then assigned these tests to the respective sub-hypotheses and evaluated the empirical support for each of them and, due to the HoH approach, also for broader, higher-level sub-hypotheses. Fig 1 outlines the HoH structure for the CSC hypothesis, including the number of empirical tests summarized on each level and the amount of support for each sub-hypothesis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Hierarchy-of-hypotheses for the Cancer Stem Cell (CSC) hypothesis.

The sub-hypotheses were defined by classifying the evidence on three levels: (1) indicators assessed to test the existence of CSCs, (2) experimental assay and (3) model cancer cells used. The empirical tests of analysed 51 studies were sorted into the HoH. Green indicates that more than 50% of the studies supported the sub-hypothesis; red indicates that more than 50% questioned the sub-hypothesis; yellow indicates sub-hypotheses for which neither supporting nor questioning evidence reached 50%. The number of studies with supporting / undecided / questioning evidence is indicated for each hypothesis in parentheses.

https://doi.org/10.1371/journal.pone.0225898.g001

The detailed results of the HoH analysis are illustrated in Fig 2. Within the 51 studies, the overall empirical support was 49.0% and highly variable for each second-level sub-hypothesis (Fig 2A). Most noticeable, the conception that putative CSCs are a rare subset of cells (for this analysis defined as 1% of tumour cells or less) could not be confirmed by most studies (13.5% support). The amount of supporting evidence was significantly higher in all other sub-hypotheses on the second level of the HoH.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Empirical evidence regarding sub-hypotheses and further aspects of the Cancer Stem Cell hypothesis.

The empirical evidence of each evaluated test was categorized as either supporting (S), being undecided (U) or questioning (Q), based on the data reported in the respective studies. N is the numbers of studies that tested the sub-hypotheses. Where the level of empirical support was significantly different (p<0.05, Mann Whitney U-tests), the respective p-value is specified.

https://doi.org/10.1371/journal.pone.0225898.g002

The analysis also indicates that the support for the CSC concept varies between what cells were used–while most experiments using primary tumour cells did not support the CSC model, those that used cultured cells mostly confirmed it (Fig 2B). While more than half of the studies that used cells that grew selectively in stem-cell medium or expressed CD90 as putative CSCs found supporting evidence, studies that used other surface markers or side population staining did not. The level of support for the CSC model also varied greatly by which cancer type was investigated. While all studies from research on glioblastoma multiforme and ovarian cancer found evidence for the existence of CSCs, most studies in our dataset that investigated breast cancer did not (Fig 2D). Likewise, the support varied with the mouse model used in the studies. While most tumourigenicity tests using the NOD/SCID mouse model could confirm a CSC subset (75.0%), the level of support from tests using the more immunodeficient NGS mouse model was low (25.0%, Fig 2E).

Discussion

Applying the HoH approach to studies in CSC research revealed highly interesting results, but of course a drawback of this pilot study is that only a fraction of the available literature was analysed. Extrapolating from this analysis, another 700–800 relevant publications are available in the literature.

Our results demonstrate that a comprehensive evaluation of all CSC studies available would be very valuable for the research community. For example, the notion that CSCs are a rare subset of cells within a tumour could face challenge if most tests reject this hypothesis also in a larger dataset. This would have implications for the idea that it is possible to develop drugs targeting a small number of CSCs to eradicate cancer from a patient. Also, several of the markers like CD133, which was thought by some to be a universal CSC marker [22], failed to define CSCs in a significant portion of empirical tests. Possibly the CSC phenotype is defined by different markers in different cancer types after all. A future lager dataset should also record cancer sub-types, as it is possible that even sub-types could feature different CSC markers.

In our analysis, the majority of tumourigenicity assays did show that the ability to form tumours was indeed limited to the subset of putative CSCs. But as others have observed [14], outcomes varied greatly by the used mouse model. This finding was not associated with different experimental contexts, e.g. different cancer types. Thus, the mouse model itself potentially influences the experimental outcome. The often used NOD-SCID mice are highly immunocompromised, but still tumour tissues often fail to engraft as some immune function remains. NGS mice additionally bear a mutation in the interleukin-2 gamma chain receptor that renders them highly receptive to engraftment of human cells [36]. Our results indicate that tumourigenicity assays used in the evaluated studies could simply select for cells that are good at evading the immune system of immunocompromised mice, not cells uniquely able to generate new tumours. Furthermore, experiments that used primary human cells from patients more often failed to prove the existence of CSCs in tumourigenicity tests. In contrast, the majority of experiments using cultured cells supported the CSC model. Replicated in a larger dataset, this could challenge the notion that cultured cells can be used as a simple substitute for gaining knowledge on tumours in patients. This is of course not a new observation–a recent study showed that the breast cancer cell line MCF7, which was also used in five studies in our dataset, is very different from the original patient sample and has become highly unreliable due to its many genetic and epigenetic alterations gained in culture [37].

If these results can be confirmed in a larger analysis, the implications for CSC research would be immense. One explanation for our findings could be that CSC is an unstable tumour cell state. Although when the CSC model was conceived it was (and still is) unclear how CSC emerge, the original model does not suggest that CSC is a highly plastic phenotype. Since then, some authors have proposed a redefinition where tumour cells can shift back and forth between CSC and non-CSC phenotype in response to environmental conditions as their findings did not match the model [17,18]. In our HoH, we applied the original hypothesis to first synthesise all available evidence and settle whether this original CSC model holds true–or which sub-hypotheses of the model are supported by evidence, and under which experimental conditions the CSC model can be corroborated. In a second step, a new model could then be proposed and tested. This approach would provide the urgently needed clarity of terminology, as the term CSC at this point has different meanings to different scientists.

In the field of invasion ecology, the HoH approach has helped to demonstrate that several intuitively appealing and widely used hypotheses are actually not supported by the majority of available empirical studies [30]. Similarly, this form of analysis could provide a useful tool to let go of disproven, but appealing hypotheses in the biomedical field–or provide a substantial empirical basis for their support. Additionally, the HoH analysis proved to be a useful tool to structure and categorize the seemingly heterogeneous research results surrounding the concept of CSCs. Up close, the 51 studies analysed all included very similar experiments to test the same characteristics. While it would have been difficult to summarize the unstructured results, the HoH analysis made it possible to compare different contexts and evaluate the support for the overarching hypotheses of the CSC model.

Basic research has recently been under much discussion for a possible “reproducibility crisis” with one culprit being the lack of published “negative” data, meaning results that do not show statistical differences or pointing in the “wrong” direction [38,39,40,41]. There has been a special concern for published basic cancer research after two pharmaceutical companies revealed that they could only reproduce a fraction of landmark studies [42]. In our preliminary analysis, we noticed that a major part of publications used different cell lines for different assays without giving an explanation. A possible reason for this phenomenon is that they performed the assays with all cell lines, but only presented the data that demonstrated a significant difference. Of course, we do not know if that was really the case. Such a practice would have a highly distorting effect on the available evidence that would be mirrored even in the most comprehensive HoH analysis. Thus, an ideal HoH would also include unpublished results. A solution could be an interactive open-access online platform where researchers can enter their data and in turn receive detailed information on overall support for the different sub-hypotheses in the exact context they are investigating or plan to investigate.

A further challenge that is not unique to the HoH approach, but shared by all quantitative and semi-quantitative meta-analyses sensu lato is the question of interdependence. Some publications include multiple tests of different sub-hypotheses, which are obviously not independent of each other. Other authors publish such tests of different sub-hypotheses in several smaller publications, which are again not independent of each other. Frequently, the same data are re-used in different publications, sometimes by different author teams. Studies based on different data but done by the same people are also not independent of each other. Even studies from different research groups are not necessarily independent of each other, as scientists influence each other through educational, institutional and specialization ties [43]. Co-author network analyses from different scientific disciplines revealed that relationships between researchers are highly clustered in “small worlds” where knowledge and resources are shared [44]. While knowledge flows between scientists have long been a study subject within social and data sciences by means of network analysis [43,45,46], to our knowledge these insights have not yet been integrated into meta-analyses on biomedical (or any other natural science) research. A first effort in this direction is a recent study from the field of biogeography [47] in which there was indeed a statistical association between the group of authors conducting a study and the outcome of the study. In further developing the HoH approach, these and other factors could be included in an “independence weight” of study outcomes, e.g. effect sizes reported in a study.

Supporting information

Acknowledgments

We appreciate helpful comments and suggestions by Claudia Baldus.