Phenotypic Heterogeneity and the Evolution of Bacterial Life Cycles

Most bacteria live in colonies, where they often express different cell types. The ecological significance of these cell types and their evolutionary origin are often unknown. Here, we study the evolution of cell differentiation in the context of surface colonization. We particularly focus on the evolution of a ‘sticky’ cell type that is required for surface attachment, but is costly to express. The sticky cells not only facilitate their own attachment, but also that of non-sticky cells. Using individual-based simulations, we show that surface colonization rapidly evolves and in most cases leads to phenotypic heterogeneity, in which sticky and non-sticky cells occur side by side on the surface. In the presence of regulation, cell differentiation leads to a remarkable set of bacterial life cycles, in which cells alternate between living in the liquid and living on the surface. The dominant life stage is formed by the surface-attached colony that shows many complex features: colonies reproduce via fission and by producing migratory propagules; cells inside the colony divide labour; and colonies can produce filaments to facilitate expansion. Overall, our model illustrates how the evolution of an adhesive cell type goes hand in hand with the evolution of complex bacterial life cycles.


Introduction
In nature, most bacteria live in surface-attached colonies [1,2]. Inside these colonies, cells typically express a remarkable diversity of phenotypes [3,4]. This phenotypic heterogeneity can be induced by genetic mutations, inherent stochasticity or the environment [3][4][5][6][7]. For example, during colony growth in Pseudomonas aeruginosa, genetic changes result in phenotypic heterogeneity [8]. Some cells express accelerated colony development, others have higher abilities to disseminate and yet others increase the resistance of the colony to environmental stressors. In Pseudomonas putida inherent stochasticity in the expression of a quorum-sensing signal leads to phenotypic heterogeneity. Some cells express the quorum-sensing signal and consequently disperse away from the colony, while others do not and remain tightly attached [9]. Probabilistic cell differentiation also influences the onset of colony formation. In Bacillus subtilis, motile cells can stochastically differentiate into matrix-producing cell chains, which can adhere to the surface [10][11][12][13]. Even inside a B. subtilis colony, matrix production can be heterogeneously expressed, in which only a fraction of cells expresses matrix [11,[13][14][15][16][17]. Since matrix can be shared between cells, it is often hypothesized that cells divide labour [15,18,19]: some cells produce matrix, while others specialize on complementary tasks (for an example of heterogeneous matrix expression in B. subtilis see S1 Text and S1 Fig).
Adhesive cells, like the matrix-producing cells in B. subtilis, are critical to surface colonization, because they allow for cell-to-surface and cell-to-cell adhesion [20][21][22][23]. In the lab, surface colonization readily evolves de novo. For example, when Pseudomonas fluorescens is grown in static liquid culture, cells evolve matrix production in order to colonize the air-liquid interface [24][25][26], where oxygen is available for aerobic respiration. The adhesive molecules that allow for colony formation can also trap cells inside the colony and, hence, prevent them from dispersing. Nadell and Bassler [27] demonstrated this in Vibrio cholera by growing matrix-producing and matrix-deficient cells together in a flow chamber. Whereas matrix-producing cells are more effective in colonizing the surface than matrix-deficient cells, they are strongly outnumbered by the latter in terms of propagule production. The same trade-off between surface colonization and dispersal was also apparent in an experiment of Poltak and colleagues [28,29]. They evolved Burkholderia cenocepacia cells for consecutive rounds of surface colonization and dispersal. Cells were grown in test tubes, were they could colonize a submerged plastic bead. Every day, the bead was transferred to a new test tube that contained a yet uncolonized bead, which was the next to be transferred. Thus, every day, cells had to disperse from their original bead and colonize the new one. Over evolutionary time, colony variants evolved that differed in their capacity to colonize and disperse: the variants that could easily colonize the surface were bad in dispersing and vice versa. In a recent experiment of Hammerschmidt and colleagues [30], a population of Pseudomonas fluorescens was forced to go through consecutive rounds of surface attachment at the air-liquid interface and surface detachment. This resulted in the evolution of a genetic switch that through slipped-strand mispairing produced alternating phenotypes that could colonize the surface by over-producing an adhesive molecule or detach to the liquid. In other words, evolution resulted in a bacterial life cycle in which cells alternated between living on the surface and living in the liquid. Altogether, the above experiments show that adhesive cell types control surface colonization and dispersal.
Even though numerous models have examined how bacterial cells, including matrix-producing cells, can interact on a surface [19,[31][32][33][34][35][36][37][38][39], few models have examined the evolution of cells in a dynamical environment where cells can alternative between living on a surface and living in the liquid [40]. In this study, we examine the evolution of adhesive cells in the context of surface colonization. Inspired by the above experiments, we constructed an individual-based model in which cells can evolve a 'sticky' phenotype. The model does not enforce a particular life cycle, but instead cells can 'choose' to become sticky and adhere to the surface or remain non-sticky and stay in the liquid. Sticky cells not only facilitate their own attachment, but also that of other cells. We implement three model variants to examine alternative induction mechanisms of cell differentiation. For all model variants, surface colonization readily evolves, which in most cases is accompanied by phenotypic heterogeneity, where sticky and non-sticky cells occur side by side on the surface. In the presence of regulation, phenotypic heterogeneity orchestrates the bacterium´s life cycle by affecting a colony´s survival rate, expansion rate and propagule production. Our model therefore illustrates that the evolution of cell differentiation goes hand in hand with that of a bacterium's life cycle.

Model structure
As illustrated by the examples above, cells can colonize a wide range of surfaces: including the air-liquid interface [26,41,42], air-solid interface and liquid-solid interface-e.g. plant roots [43][44][45][46], soil particles [47,48], fungi [49]. At these surfaces, attachment is governed by distinct biophysical mechanisms, although generally speaking adhesive cells are acquired. For simplicity, our model ignores the biophysical details of attachment and simply assumes that adhesive cells can adhere to the surface, whereas non-adhesive cells cannot. As such, the model does not resemble any specific type of surface colonization, instead we aim to make a first step in exploring how adhesive cell types evolve in a dynamical environment where cells can attach and detach from a surface at any moment in time.
We assume that the model consists of two environments: the liquid and the surface (Fig 1). At the onset of evolution, cells only occur in the liquid. Cells can express two cell types: sticky and non-sticky cells. Sticky cells have a reduced cell division rate (R) and are required for surface attachment. They can attach to any unoccupied position on the surface. Non-sticky cells can also attach to the surface, but only when immediately neighbouring a sticky cell. In other words, non-sticky cells can hitchhike with sticky cells, like observed in the lab (e.g. S1 Text, S1C and S1D Fig). The surface consists of a two-dimensional hexagonal grid, so each sticky cell can have at most six non-sticky neighbours (Fig 1). Surface attachment is beneficial, because it allows cells to escape from competition in the liquid. This benefit is present as long as there is space available on the surface. At the same time, surface attachment requires sticky cells that carry the cost of a lower cell division rate. Non-sticky cells that attach to the surface by hitchhiking with sticky cells escape from the costs of being sticky, but still have the benefits of surface attachment.
We examine the evolution of sticky cells for three model variants. These variants differ with respect to the differentiation strategy that evolves (Fig 1): cells either have a (1) pure strategy, (2) probabilistic strategy, (3) decision-making strategy. In the pure strategy, cells can only switch between being sticky and non-sticky by mutations. As a result, each genotype expresses one phenotype. In the probabilistic strategy, cells differentiate with a certain probability (P). This probability can change over evolutionary time, by the accumulation of mutations (for details see Material and Methods). In the decision-making strategy, cells can differentiate in response to the environment. Cells sense two environmental cues: the niche in which they occur (N = 0 in the liquid and N = 1 on the surface) and the fraction of sticky cells (i.e. stickiness, S). On the surface, cells only sense the fraction of sticky cells in the neighbouring positions on the grid. In the liquid, the fraction of sticky cells is determined with respect to the entire population. The sensory input to a cell is weighted by so-called connection weights (W 1 and W 2 ). When the sum of regulatory input exceeds a given threshold (θ) a cell differentiates to a sticky cell. Over evolutionary time the connection weights and activation threshold can evolve (see Fig 1 and Material and Methods).
For each model variant, we start evolution with a population of non-sticky cells in the liquid. All genotypic variables are set to zero (model variant 2: P = 0, and model variant 3: W 1 = W 2 = θ = 0). Cells can evolve for 400.000 time step. At each time step, one of the following events can occur (see Material and Methods): (i) migration to the surface, (ii) migration to the liquid, (iii) cell differentiation, (iv) cell death, (v) cell division. The event that occurs is chosen randomly. We explore the outcome of evolution by varying two modelling parameters: R and P m . R is the relative cell division rate of sticky cells. When the costs of being sticky are high, sticky cells cannot divide (R = 0) and, when the costs are low, sticky and non-sticky cells are equally likely to Fig 1. Model structure. The model structure consists of two parts: ecology and development. Ecology: cells can express two phenotypes, non-sticky cells or sticky cells. The sticky and non-sticky cells occur in one of two niches: the liquid or the surface. Cells in liquid are well mixed. Cells on the surface are placed on hexagonal grid and can only stay on the surface when being sticky or directly surrounded by sticky cell. Cells can migrate from the liquid to the surface and vice versa. Migration to the surface is only possible when being sticky or when a cell is directly surrounded by sticky cell on the surface. Development: three differentiation strategies are examined in the model: (1) pure strategy, (2) probabilistic strategy, (3) decision-making strategy. In the pure strategy, cells only express one phenotype and can switch via mutations. In the probabilistic strategy, cells have a probability P to differentiate. In the decision-making strategy, cells can differentiate in response to the environment. Cells sense two environmental cues: the niche in which they occur (N) and the fraction of surrounding sticky cells (i.e. stickiness, S). Cells differentiate when the sum of regulatory input, weighted by connection weights (W 1 and W 2 ), exceeds the activation threshold (θ). divide (R = 1). P m is the probability to migrate to the surface. As default setting P m = 0.1, which means that cells have a 10% probability to migrate to a random location on the surface. This does not mean that they necessarily attach to this particular location. A cell can only attach when the randomly chosen position on the surface is vacant and the cell is sticky or surrounded by a sticky cell.

Surface colonization of three differentiation strategies
We first examined the evolution of surface colonization at various relative growth rates of the sticky cells (R = 0, 0.4, 0.8, 1). Fig 2 shows some representative surfaces at the end of evolution, for the pure, probabilistic and decision-making strategy. Sticky cells are shown in red and the non-sticky cells in blue. The three differentiation strategies differ in their capacity to colonize the surface (see also Fig 3A). The pure strategy shows some surface colonization at all cost levels, but the number of cells on the surface is very low at high costs of being sticky, i.e. low cell division rates of sticky cells (R). The few sticky cells that occupy the surface at R = 0 express a maladaptive phenotype, because these cells cannot reproduce, nor can they switch phenotype (in the pure strategy, cells can only switch phenotype through mutations that occur during cell division). The probabilistic strategy evolved a nearly full surface colonization at most costs, but cannot colonize the surface at the highest costs (R = 0). The decision-making strategy can colonize the surface at all costs; even when sticky cells cannot divide (R = 0).
The fraction of sticky cells on the surface decreases for higher costs of being sticky. Moreover, the fraction of sticky cells is lower in colonies from the decision-making strategy than in colonies from either the probabilistic or pure strategy. In the decision-making strategy, cells furthermore show spatial pattern formation (Fig 2). At R = 0 and R = 0.4, sticky cells only have non-sticky neighbours. In other words, the sticky cells-together with their non-sticky neighbours-form separated islands on the surface. At R = 0.8, sticky cells also form filaments. Filaments are short concatenations of sticky cells (2-8 sticky cells), which are surrounded by nonsticky neighbours. Only in a few cases do sticky cells also clump together. When there are no costs of being sticky (R = 1), all cells express the sticky phenotype and there is no spatial pattern formation. There are no regular spatial patterns for the probabilistic and pure strategy, because these strategies cannot account for the number of sticky neighbours.

The fraction of sticky cells on the surface and in the liquid
Next, we examined the fraction of sticky cells on the surface and in the liquid (Fig 3B). On the surface, as shown by Fig 2, the decision-making strategy produces the lowest fraction of sticky cells. In the pure and probabilistic strategies, the fraction of sticky cells gradually increases with higher R values. In other words, at lower costs of being sticky, a larger fraction of cells expresses the sticky phenotype. In the decision making strategy, the fraction of sticky cells does not change gradually with R. Instead, the fraction of sticky cells is around 20% (R = 0-0.72), 30% (R = 0.8-0.96) or 100% (R = 1). These levels correspond to distinct spatial patterns observed in Fig 2: the 20% sticky cells correspond to isolated islands of sticky cells; the 30% sticky cells correspond to short filaments; and the 100% sticky cells correspond to clumps of sticky cells.
The fraction of sticky cells in the liquid differs from that on the surface (Fig 3B). For the pure strategy the difference is small. The fraction of sticky cells in the liquid is lower than that on the surface, due to the high cell division rate of non-sticky cells. Sticky cells that migrate to  the surface remain attached until they die. Despite this permanent attachment, there is still a relatively large fraction of sticky cells in the liquid (especially for high R values), because surface-attached sticky cells can dislodge their daughter cells to the liquid after cell division (we assume that the colony is flat, so any cell division in the z-direction would result in the migration of a cell to the liquid; see Material and Methods). For the probabilistic strategy, the fractions of sticky cells in the liquid and on the surface are nearly the same. Even though there is a selective advantage for non-sticky cells in the liquid (i.e. higher cell division rate), this does not affect the frequency of sticky cells too much, because all cells have a given probability to become sticky. For the decision-making strategy, we observed a surprisingly large difference between the fraction of sticky cells in the liquid and on the surface. At very low and very high costs of sticky cells (R % 1 or R % 0), the fraction of sticky cells in the decision-making strategy is more or less the same as that for the pure and probabilistic strategies, but at intermediate costs (R = 0.2-0.8) almost 90% of the cells in the liquid are sticky.
How can there be so many sticky cells in the liquid, while these cells have a lower cell division rate than the non-sticky cells? In order to answer this question, we have to examine the population dynamics. There is a migratory asymmetry between the liquid and surface, while cells in the liquid can only migrate to the surface when finding a vacant position, cells from the surface can always migrate to the liquid and furthermore dislodge cells to the liquid during cell division. The migration rate of cells to the liquid is therefore much higher than that of cells to the surface. The migratory asymmetry is even bigger when the fraction of sticky cells on the surface is low, because this offers fewer possibilities for non-sticky cells to adhere to the surface. The surplus of migrants to the liquid results in a much higher competitive pressure in the liquid than on the surface. Cells therefore profit if they can increase the probability of surface attachment. Sticky cells are more effective migrants than non-sticky cells. As a result, cells in the decision-making strategy evolved such that they express the sticky phenotype in the liquid. Not all the replicate simulations evolved a high fraction of sticky cells in the liquid, because mutations that trigger cell differentiation in the liquid are often harmful on the surface (S3 Fig). The pure and probabilistic strategies do not have a high fraction of sticky cells in the liquid, because cells cannot adjust their behaviour with respect to the environment in which they occur.
Another surprising result in the decision-making strategy occurs at high costs of being sticky. At R = 0, there are almost no sticky cells in the liquid, while approximately 20% of the cells on the surface are sticky (see black arrows in Fig 3). The lack of sticky cells in the liquid is surprising for two reasons. First, non-sticky cells cannot colonize the surface by themselves. If we would initiate our simulations with this evolved genotype, it would not be able to colonize the surface. Second, the surface contained around 2000 sticky cells. How can the number of sticky cells be so high, while sticky cells cannot divide and there are no sticky cells that migrate from the liquid to the surface? In the next section, we address the above questions by examining the decision-making strategy in more detail.

Colony expansion and division of labour
As shown by Fig 2, in the decision-making strategy, at R = 0, sticky cells are only surrounded by non-sticky neighbours. That means that a cell only differentiates when it has no sticky neighbours. If one of the neighbours is already sticky, a cell would remain non-sticky. Given this differentiation program, a colony would only be able to expand when the following sequence of fortunate events occurs (see scheme in Fig 4). First, the sticky cell should die. Second, in response to its death, some of the non-sticky neighbours should become sticky. At least two cells need to differentiate for the colony to split in two. Moreover, this should happen before the cells are dislodge from the surface, while in the absence of a sticky cell, non-sticky cells cannot stay on the surface. Finally, the remaining non-sticky neighbours have to divide in order to fill up the vacant positions next to the two novel sticky cells.
Since colony fission starts with cell death, higher death rates of the sticky cells should increase the probability of colony fission. We examine this by competing three genotypes that have the same decision-making strategy, but differ with respect to the death rate of the sticky cells: the sticky cells have a 0%, 5% or 10% probability to die, respectively (in the evolutionary simulations we assumed a 10% death probability). Hundred cells of each genotype were randomly placed on the surface. These genotypes were competed for 10.000 time steps under a reduced migration rate (P m = 0.01), in order to focus on colony expansion. Fig 4 shows the frequency of each genotype over time. The genotype with the highest death rate indeed expanded faster (green line in Fig 4). This genotype had a competitive advantage over the other two genotypes at the onset of colony growth. However, when the population size went through its inflection point, the competitive advantage disappeared (see black line in Fig 4). At the inflection point, population growth is curtailed by the high cell density. There is less space to expand. As a consequence, colony longevity becomes more important for competition than colony expansion. Since lower death rates increase the longevity of a colony, the genotype with the lowest death rate slowly takes over the population of sticky cells (see S2 Fig). Once all sticky cells belong to this genotype, there is no selective difference between the genotypes anymore, because the non-sticky cells of all genotypes are identical (i.e. same fitness; see S2 Fig).
In summary, Fig 4 illustrates that a single sticky cell, together with its non-sticky neighbours, can colonize the entire surface. The sticky cell and its neighbours often have the same genotype (see time step 2000 in Fig 4), because the non-sticky cells are produced by the sticky cell before cell differentiation or vice versa. Thus, cells inside the colony divide labour: sticky cells sacrifice their fitness, thereby increasing the fitness of their non-sticky clonal neighbours. . Although the first colony type can expand over the surface, as shown in the previous section (Fig 4), there is still a substantial risk that all non-sticky cells are dislodged after the sticky cell dies. This risk is not present for the second colony type, because there are multiple concatenated sticky cells. Therefore we expected filamentous genotypes to be better colonizers.

Colony expansion and ecological succession
As in the previous section, we performed a competition experiment, placing hundred cells from each genotype-the isolated sticky cells and sticky filaments-on the surface. In order to focus on colony expansion, we reduced the migration rate from the liquid to the surface (P m = 0.01). In addition, we assumed that sticky cells could not divide (R = 0). Fig 5 shows that filamentous sticky cells indeed function as a colonizing genotype, while the isolated sticky cells function as a climax genotype. The colonizing genotype shows a higher expansion rate than the climax genotype (S4 Fig). However, when the population size goes through its inflection point, the fitness of the colonizing genotype drops and the climax genotype takes over. The climax genotype is less efficient in colonization, but-due to the lower fraction of sticky cells-has a higher cell division rate and therefore produces more propagules that can migrate to the liquid and initiate new colonies. In the evolutionary simulations, the climax genotype dominates the surface when the costs of being sticky are high (i.e. low R values). Only when the costs of being sticky are fairly low (R = 0.8 in Fig 2 and Fig 3), the colonizing genotype (i.e. filamentous genotype) can outcompete the climax genotype, because the benefit of colony expansion outweigh the loss of propagule production. Since the colonizing genotype is more effective in colonizing the surface, the cell density at the surface is higher at low costs (i.e. there is a sudden increase in the cell density around R = 0.7 in Fig 3A, which corresponds to the dominance of the colonizing genotype). Fig 5 illustrates that there is a trade-off between colony expansion and propagule production in our model: isolated sticky cells produce many propagules and expand slowly, while sticky filaments produce few propagules and expand rapidly. As a consequence, genotypes can specialize to grow at different stages of ecological succession. The colony's expansion rate, longevity and propagule production depend on spatial pattern formation and, hence, the cell differentiation program that underlies phenotypic heterogeneity.

Evolution of life cycles
In the previous sections, we investigated how the relative growth rate of sticky cells (R) affects the evolution of phenotypic heterogeneity. One would expect that ecological parameters play an important role as well. In this section, we vary both the relative cell division rate of sticky cells (R) and the migration rate towards the surface (P m ). Note that migration towards the surface does not guarantee attachment, because cells first have to find an available spot, before they can actually attach. For each parameter combination we examine the evolved populations of all three differentiation strategies.
We first examined the population size and fraction of sticky cells for each parameter combination of R and P m (R = 0-1 and P m = 0-0.5). This was done in both the liquid and on the surface. In the pure and probabilistic strategy, the population size and fraction of sticky cells were more or less independent of the migration rate in both the liquid and on the surface (Fig 6). As shown by Fig 3, both the population size and the fraction of sticky cells decreased with the Phenotypic Heterogeneity and the Evolution of Bacterial Life Cycles costs of being sticky (Fig 6). In the decision-making strategy, there was an effect of the migration rate, but only at high costs of being sticky. At R < 0.2, higher migration rates towards the surface paradoxically lead to lower population densities on the surface (Fig 6). At the same time, there is also a change in the fraction of sticky cells in the liquid. At low migration rates the fraction of sticky cells in the liquid is nearly zero, while at high migration rates it is almost one. Sticky cells are more likely to colonize the surface than non-sticky cells, which makes it even more surprising that high migration rates result in a drop of the population density on the surface. How can we explain these paradoxical results? One possible explanation for the low population density at high migration rates is the occurrence of exploitation. At high migration rates, non-sticky cells are more likely to migrate to the surface and exploit sticky cells. Such exploitation can lead to the collapse of colonies. To investigate whether or not such exploitation indeed occurs, we examined the evolutionary outcome of the decision-making strategy in more detail.
For each parameter combination (R = 0-1 and P m = 0-0.5), we examined the 25 most abundant genotypes that were present at the end of evolution. The decision-making strategy was   production. Propagules get dislodged from the surface and migrate to establish new colonies in other regions of the surface. In life cycle 2, at lower R values, colonies lose their filamentous property (Figs 7 and S6). The costs of being sticky outweigh the advantage of being filamentous. At this stage, the colonies consist of isolated sticky cells. Even though these colonies can still expand, via cell death (Fig 4), they expand slower than the filamentous colonies. The unoccupied parts of the surface leave more space for migrating propagules to establish new colonies (Figs 6 and S6). In life cycle 3, at very low R values, there is a change in the propagule stage of the life cycle. Colonies still reproduce through fission, but propagules do not differentiate to sticky cells. Instead, they remain non-sticky and colonize the surface by exploiting the sticky cells. The non-sticky cells migrate to vacant positions that are available in existing colonies, such position are more common at low R values. Thus, the non-sticky migrants act as parasites that, in some cases, even take over established colonies (see S3B Fig that shows how parasitizing propagules invade in the population). In life cycle 4, at higher migration rates, colony formation becomes less common (S6 Fig). At high migration rates, sticky cells on the surface are more likely to be exploited by non-sticky migrants from the liquid. This exploitation breaks down the benefits of colony formation. As a consequence, a new unicellular life cycle evolves. Cells differentiate to sticky cells in the liquid, which allows for surface colonization. Once at the surface, these sticky cells de-differentiate to non-sticky cells, which can divide before being dislodged to the liquid. In this life cycle, sticky-cells can still be exploited by non-sticky migrants, but it is less likely to occur, because cells are only sticky for a transient life stage. In the absence of multicellular colony formation, there is no colony expansion and, hence, a lower cell density on the surface. This explains why higher migration rates result in lower cell densities (Fig 6).
In summary, at low costs of being sticky, colonies form filaments and reproduce by colony fission and propagule production (Fig 7). At intermediate costs, filamentous growth disappears and more space becomes available for surface colonization of propagules. At low costs, these propagules can only colonize the surface by parasitizing already existing colonies. At low costs and high migration rates, colony reproduction fails, because colonies succumb under the parasite pressure. In this case, a unicellular life cycle evolves in which surface attachment forms a transient life stage. Interestingly, the same life cycles also evolve for alternative surface geometries (e.g. triangular and square grid, instead of hexagonal grid) and for a three dimensional implementation of the surface (see S2 Text and S7-S12 Figs). Only, when a cell's neighbourhood is discontinuous-meaning that a cell's neighbours do not neighbour each other-surface colonization becomes impossible at low R values (see S2 Text).

Discussion
Inside colonies, bacterial cells often express many different phenotypes. Using individualbased simulations, we studied the evolution of a sticky cell type in the context of surface colonization. We show that under the majority of parameter conditions surface colonization evolves. In many cases, colonization is associated with phenotypic heterogeneity, in which sticky and non-sticky cells co-occur on the surface. Phenotypic heterogeneity results from the trade-off between cell division and surface attachment (see also [50][51][52][53][54]): sticky cells have a reduced cell division rate, but can colonize the surface, while non-sticky cells have a high cell division rate, but cannot colonize the surface by themselves. In our model, we compared three alternative differentiation strategies: pure strategy, probabilistic strategy and decision-making strategy. In the pure strategy, cells consistently express the same phenotype and can only switch via mutations. In the probabilistic strategy, cells differentiate with a certain probability. In the decisionmaking strategy, cells differentiate in response to the environment. Both the probabilistic and decision-making strategy evolve surface colonization for relatively high costs of being sticky, but only the latter can colonize the surface for extreme costs-i.e. when sticky cells hardly divide (R << P d ). In the decision-making strategy, cells cooperate by dividing labour: the sticky cell sacrifices its fitness for the benefit of the colony (i.e. non-sticky cells that surround the sticky cell). Cells in the probabilistic strategy can reciprocate benefits, but are not capable of coordinating their behaviour.
One striking outcome of the model is that-under the decision-making strategy-different life cycles evolved [55]. The evolution of life cycles follows from the dynamical environment in which bacterial cells live, in which they can alternate between growing on the surface and growing in the liquid. In most life cycles, the surface-attached colony forms the dominant life stage. The surface provides a scaffold on which cells can organize themselves [56,57]. The propagules in the liquid have only a marginal chance to colonize the surface. Yet, once a propagule attaches to the surface, it can form a colony that is relatively long lived and thereby produces many new propagules. Cells can affect the colony properties by coordinating their behaviour. The fitness of a colony is determined by three key properties [55]: (1) colony expansion, (2) colony longevity and (3) propagule production. Cells increase the longevity of colonies by regulating cell differentiation: if the sticky cell dies one or more neighbouring cells will differentiate, thereby guaranteeing the survival of the colony. This simple type of coordination allows for surface colonization in the toughest conditions (R = 0). Life cycles can only evolve in the decision-making strategy, because spatial and temporal organization can only come about when cells can respond to the environment [58][59][60][61]. For simplicity, we assumed that cells could only sense two environmental cues, yet, in reality cells can sense many more cues [62], which presumably allow for many alternative forms of coordination. It would be interesting to explore how environmental information facilitates or constrains spatial and temporal organization.
The surface-attached colony is vulnerable to exploitation. Cells inside the colony cooperate. As explained above, the sticky cell sacrifices its fitness for the benefit of the colony. As long as the sticky cell is surrounded by its non-sticky siblings, it cannot be exploited. However, when one of the neighbours dies, there is a risk that the cell from the liquid migrates next to the sticky cells and reaps the benefits of surface attachment without paying the costs. In our model, exploitation is more likely to occur when migration rate is high, when a cell's neighbourhood is large and when a cell's neighbourhood is discontinuous (see S2 Text). More generally, one could say that adhesive cells are less likely to be exploited when the diffusion of adhesive molecules is limited, such that only sibling cells profit from adhesion and invasion from outside is minimized [34,35,38,39,[63][64][65][66][67]. A recent study of Nadell and colleagues [68] also illustrates that cells can actively prevent exploitation. They show that in Vibrio cholerae colonies, cells secrete a protein that facilitates a closer association between cells and the extracellular matrix. This prevents open spaces in the colony, which subsequently prevents cells from invading the interior of the colony and hence exploitation. Alternatively, non-adhesive cells might simply be less likely to join a colony than adhesive cells [69,70]. This results in the segregation of adhesive and non-adhesive cells, which in turn prevents exploitation.
Bacterial life cycles are often hard to study empirically. First, it is nearly impossible to trace bacterial individuals in nature. Second, in many cases, the ecological relevant unit of a bacterial life cycle is not the individual cell, but the colony [58,71]. Cells are relatively short-lived an only survive a part of colony formation, yet colonies often go through coordinated life stages [1,2,72]. Thus, instead of tracing the fate of a single cell, one should trace the fate of all cells in the colony. Despite these difficulties, life cycles are well characterized for a number of bacterial species [73]. For many of them, the life cycle consists of a surface-attached life stage and a unicellular dispersal stage [61,74]. In the surface-attached life stage, cells often organize into colony structures that facilitate colony expansion or dispersal. For example, many bacteria develop filamentous structures to facilitate colony expansion [75][76][77][78], while other bacteria develop fruiting bodies to facilitate dispersal [79][80][81][82][83][84]. Our study indicates that the ecological significance of the observed colony structures-and the associated adhesive cell types-can only be fully appreciated when considering the entire life cycle of a bacterium, including the dynamical environment in which bacteria make their living.

Experiments
Cells were grown in 2.5mL static liquid MSgg [85] using twelve-well plates. Plates were incubated for 50 hours at 30°C. The inoculum was prepared by growing strains overnight on 1.5% agar LB plates at 37°C. Overnight colonies were scraped from the plates and diluted in phosphate buffered saline (PBS) to an optical density of 0.2 (OD 600 = 0.2). The wells were inoculated with 2μL of this sample. All strains were derived from a non-domesticated wild type B. subtilis strain called NCIB 3610 [86]. The regulatory mutant strain could not express two operons, eps and tapA, which are essential for matrix production (strain DS91, see [85,87]). The fluorescent strain expresses cyan fluorescent protein (CFP, artificially coloured red in S1 Fig) in all matrix-producing cells (strain DL823, see [16]). For microscopy, cells were isolated from top of the pellicle (i.e. colony at air-liquid interface), placed on an object glass with solidified 200μL of 1.5% agarose PBS and examined using an inverted microscope. The inverted microscope was a Nikon Eclipse TE2000-U microscope equipped with a 20× Plan Apo objective and a 60× Plan Apo oil objective. Images were taken using CFP filter. Image analysis was performed with ImageJ (1.48v).

Model
In the model we assume there are two niches: the surface and the liquid. At the onset of evolution cells only occur in the liquid, where cells are assumed to freely float around and there is no spatial structure. The surface consists of a hexagonal grid (Fig 1). In order to colonize the surface, cells have to become 'sticky'. When cells are sticky they can attach to the surface. Sticky cells not only facilitate their own attachment to the surface, they can also mediate non-sticky cells to adhere to the surface, but only if these non-sticky cells are located immediately adjacent to the sticky cells. Since the surface consists of a hexagonal grid, one-sticky cell can be surrounded by maximally six non-sticky cells. Stickiness typically results from the production of costly substances, such as extracellular polysaccharides, we therefore assume that being sticky reduces the rate of cell division (R). Yet, despite these costs, becoming sticky can be beneficial, while cells can avoid competition in the liquid by adhering to the surface. Once adhered to the surface, a cell can also migrate back to the liquid, by de-differentiating to a non-sticky cell.
At each time step, one out of five events can occur: (1) cell migration from the liquid to the surface, (2) cell migration from the surface to the liquid, (3) cell differentiation, (4) cell death and (5) cell division. For each cell, the event that occurs is selected randomly, to randomize the order in which cell events occur. We tested three different version of the model in which the strategy underlying cell differentiation is different (Fig 1). These strategies will be discussed in detail below.
1. Cell migration from the liquid to the surface. Cell migration from the liquid to the surface occurs with a certain chance (P m ). A cell migrates to a random location on the surface, where it can only attach when two criteria are fulfilled. First, the randomly selected location on the grid should still be available (i.e. there is no cell yet that is attached to this location). Second, the cell that migrates should either be sticky or should be surrounded by a sticky cell on the surface. Only by immediately neighbouring a sticky cell, a non-sticky cell can attach to the surface. When the criteria are not fulfilled, the cell immediately returns to the liquid as if nothing happened.
2. Cell migration from the surface to the liquid. Cell migration from the surface to the liquid occurs when a cell is not sticky and does not have a sticky neighbour. Cell migration therefore always occurs in the local absence of sticky cells. Due to migration, the population size in the liquid becomes bigger and can even exceed its carrying capacity (K). To correct for this migration flux, the population size gets normalized at the end of every time step. That is, if the population size in the liquid is bigger than the carrying capacity, we simply assume that there is a dilution process (i.e. randomly selected cells from the liquid are removed) that normalizes the population size to that of the carrying capacity. Migration towards the liquid environment can also occur during cell division on the surface, in which the daughter cell is dislodged from the surface and migrates to the liquid (see details below).
3. Cell differentiation. We explore the evolution of three possible strategies that could underlie differentiation (Fig 1): (i) pure strategy, (ii) probabilistic strategy, (iii) cell-decision strategy. In the first strategy, cells can either be sticky or not. They can only switch between both states by mutations; therefore the sticky and non-sticky cells correspond to different genotypes. In the second strategy, cells are sticky with a certain chance (P). Thus, every time a cell gets the opportunity to differentiate it has a chance, P, to become/remain a sticky cell and a chance, 1-P, to become/remain a non-sticky cell. Sticky and non-sticky cells can therefore have the same genotype. P can evolve (see below for details) and therefore the relative fraction of sticky and non-sticky cells can change over evolutionary time. In the last and third strategy, cells can differentiate in response to the environment. We assume that cells can sense two environmental cues: the niche (N = 0 in the liquid and N = 1 on the surface) in which they occur and the fraction of sticky cells (S). The fraction of sticky cells is determined differently on the surface and in the liquid. On the surface, cells only sense their local environment, S is therefore determined by the fraction neighbouring locations around a cell that are occupied by sticky cells (the total number of neighbouring locations is equal to six). In the liquid, S is determined by the fraction of sticky cells in the entire population. The environmental inputs are weighted by so-called connection weights (W 1 and W 2 ). A cell differentiates when the sum of regulatory input exceeds the activation threshold: W 1 ÁN + W 2 ÁS > θ. W 1 , W 2 and θ form the genotype of an individual and can change over evolutionary time, thereby evolving the response of cells towards the environmental conditions to which they are exposed.
4. Cell death. All cells have a fixed chance of dying (P d ) that is equal to 10%. There is no difference between the death rate on the surface and in liquid and there is also no difference between the death rate of sticky and non-sticky cells (with the exception of the competition experiment in Fig 4).
5. Cell division. Sticky cells have a cell division rate that is lower than or equal to that of non-sticky cells (0 R 1). When R = 1, the cell division rate of sticky and non-sticky cells is equal. When R = 0, sticky cells do not divide, while non-sticky cells do. Cells in the liquid can only divide when the population size in the liquid is lower than the carrying capacity (K). On the surface, cell division depends on the local neighbourhood of a cell. A cell can be surrounded by at most six vacant neighbouring locations. In case a cell divides, the daughter cell either goes to one of these neighbouring locations or towards the liquid. A daughter cell could only remain on the surface when surrounded by a sticky cell or when being sticky itself. In the absence of sticky cells, a daughter cell could only go to the liquid environment. The daughter cell is equally likely to go to each of the neighbouring locations on the surface as it is likely to go to the liquid. For example, if a cell has 6 available vacant neighbouring locations, then a daughter cell could end up with a chance 1/7 at each of these neighbouring locations (i.e. chance of 6/7 to remain on the surface) and with a chance 1/7 it would migrate to the liquid. The migration of a daughter cell to the liquid resembles a cell division event in the z-direction. Since we assume colonies are flat two dimensional structures, every cell division in the z-direction results in the dislodgment of a cell from the surface to the liquid. When none of the six neighbouring locations are available and the population size in the liquid is at its carrying capacity, cells on the surface cannot divide. Even though cells on the surface and in the liquid get the same number of opportunities to divide, cells on the surface have a higher chance to divide. This is because daughter cells from surface-attached cells can end up in both niches, while daughter cells from cells in the liquid cannot. Cells in the liquid are therefore more strongly constraint by the carrying capacity than cells on the surface.
In case a cell divides, the daughter cell has a certain chance to mutate (μ r ) each of the evolvable variables. The number of evolvable variables differs between the three strategies. In case, the mutating variable is a continuous variable (e.g. P, W 1 , W 2 and θ), a value is added to the variable that is taken from a normal distribution with mean 0 and standard deviation μ s . All simulations ran for 400.000 time steps (T) unless reported differently. The carrying capacity (K) and grid size (G) are kept constant for all simulations, as well as the mutation rate (μ r ) and size (μ s ). See S1 Table for  For each parameter combination of P m and R a simulation was performed. At the end of evolution, the 25 most abundant genotypes of each simulation were examined. For each genotype the phenotypic strategies on the surface and in the liquid were determined. On the surface, we determined if a genotype would always differentiate, never differentiate or differentiate when there are less than n neighbouring sticky cells (n = 1, 2, 3, 4, 5 or 6). In the liquid, the genotype is examined in the same way, however instead of determining if the genotype would differentiate when there are less than n sticky neighbours, we determine if the genotype would differentiate when the fraction of sticky cells in the population is less than n/6 (since cells sense the average fraction of cells). Each phenotypic strategy corresponds to a colour as shown in the legend on the right. Since we examine each genotype in both environments, every genotype is associated with two colours, one for the strategy on the surface and one for the strategy in the liquid. shown on the surface (R = the relative cell division rate of sticky cells and P m is the migration rate from the liquid to the surface). Note that for life cycle 4 the migration rate is increased relative to that of life cycle 3, but the population density on the surface decreases. This is because multicellular colonies, which are good in colonizing the surface, are disfavoured by selection. To facilitate comparison, we therefore determined if a genotype would differentiate on the surface when the fraction of sticky neighbours was less than n/6 (n = 1, 2, 3, 4, 5 or 6) using the same colour combination as in S5 Fig. Since we examine each genotype in both environments, every genotype is associated with two colours, one for the strategy on the surface and one for the strategy in the liquid. The strategies of the 25 most abundant genotypes are shown by the 25 colour pixels in each quadrant (i.e. every quadrant corresponds to a parameter combination). The pixels are sorted from the most abundant genotype (upper left corner of each quadrant) to the least abundant genotype (lower right corner of each quadrant). The quadrants that are entirely grey correspond to the parameter combinations in which there was no surface colonization. (TIF) S1