Causal models on the neuronal level

To construct causal models on the neuronal level, we first need to decide on a framework for causal modeling. A variety of causal modeling frameworks have been developed for and evaluated on neuronal data [24]. To be applicable in the MCM framework, we require causal models that can make empirically testable predictions on the effects of experimental interventions. The framework of Causal Bayesian Networks (CBNs) [18, 25] fulfills this requirement and hence serves as the causal modeling framework in the present setting.

Causal relations in CBNs are modeled by structural causal models (SCMs):

Definition 1 (Structural Causal Model—SCM). A SCM is a triple with X a set of N random variables endogenous to the model, E a set of N exogenous noise variables, and F a set of N functions defining each endogenous variable as a function of its direct causes (i.e., parents—pa()) and its corresponding exogenous noise variable, so that for each i ∈ {1, …, N} we have X_i ≔ F_i(pa(X_i), E_i) where the F_i are chosen such that no variable is a (direct or indirect) cause of itself.

The joint probability distribution P(E) over the exogenous noise variables induces a joint probability distribution P(X) over the endogenous variables (via the pushforward measure). If the noise variables are mutually independent, the SCM is causally sufficient, i.e., no unobserved confounders that influence multiple endogenous variables exist. In the present setting, each endogenous variable X_i represents the activity of one neuron. We note that in general, X and E may represent continuous- as well as discrete variables, implying that P(X) and P(E) denote probability densities or distributions, respectively. In the following, we assume all variables to be discrete. We consider an extension of the MCM framework to continuous-valued random variables feasible but beyond the scope of the present work. We denote random variables by upper- and their realizations by lower-case letters, e.g., we write P(X_i = x) to indicate the probability that the neuron represented by the random variable X_i takes on the value .

Causal relations in SCMs are commonly depicted by directed acyclic graphs (DAGs), with an arrow drawn from node A into node B if the endogenous variable represented by node A is a parent of the endogenous variable represented by node B. An arrow from A into B indicates a direct causal influence of A on B, and a directed path from node A to node B indicates an indirect causal influence of A and B. We note that direct- and indirect causal relations are relative to the set of variables endogenous to the model and may change when dropping or adding nodes.

Knowledge of the DAG, in combination with the joint probability distribution over all endogenous variables (represented by the nodes of the DAG), enables us to reason about the probabilistic effects of experimental interventions on variable subsets [18]. Experimental interventions are represented mathematically by the do()-operator, e.g., do(X_i = x) represents the experimental intervention of setting variable X_i to the value x. In empirical settings, the DAG and the joint probability distribution are usually not known and have to be inferred from a combination of experimental and observational data. We show in the section on learning MCMs that knowledge of the causal model on the neuronal level is not a prerequisite for constructing a causally consistent mapping between the neuronal- and the cognitive level.

In their original form, SCMs model independent and identically distributed (i.i.d.) data, i.e., data without temporal structure. It is straightforward, however, to extend SCMs to model dynamical neuronal systems by unrolling the endogenous variables across time, as exemplified in the middle row of Fig 1. Here, the set of random variables X[t₀] represents the global state of all N neurons at time t₀ and the arrow from X[t₀] into X[t₁] indicates that the global neuronal state at time t₀ is a cause of the global neuronal state at time t₁, in the sense that the state of each neuron at time t₁ is a function of a subset of the neuronal states at time t₀ and their respective exogenous noise term. We remark that this representation also allows feedback loops, e.g., as in X_i[t] → X_j[t + 1] → X_i[t + 2]. More formally, we define the extension of SCMs to dynamic settings as follows:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Relations between cognitive-, neuronal-, and behavioral states in MCM.

Solid and dotted arrows denote causal and constitutive relations, respectively.

https://doi.org/10.1371/journal.pcbi.1012674.g001

Definition 2 (Dynamic Structural Causal Model—dSCM) A dynamic SCM (dSCM) is family of T triples , indexed by t ∈ {1, …, T}, with X[t] a set of N random variables endogenous to the model, E[t] a set of N exogenous noise variables, and F[t] a set of N functions defining each endogenous variable as a function of its direct causes (i.e., parents—pa()) and its corresponding exogenous noise variable, so that for each i ∈ {1, …, N} and t ∈ {1, …, T} we have X_i[t] ≔ F_i[t](pa(X_i[t]), E_i[t]), where pa(X_i[t]) may contain any endogenous variables prior to time t.

We note that we do not allow instantaneous causal relations in Definition 2, i.e., situations where pa(X_i[t₁]) may include other neurons at time t₁, because our current understanding of physical processes posits that causes must precede effects. This constraint may have to be reconsidered when modeling neuronal dynamics that are observed at sampling rates slower than the system’s dynamics.

If the exogenous noise terms are mutually independent and pa(X[t]) only includes variables at the previous time step for all t, as in the example in Fig 1, the dSCM represents a first-order Markov process with a transition probability distribution (1)

Markov processes are of particular relevance in the MCM framework, because they define causally sufficient state spaces, in the sense that the global state at any given point in time is causally sufficient for the probability distribution over the global state at the next time point. More formally, Markovianity implies (under the faithfulness assumption) that the exogenous noise terms at different time points are mutually statistically independent and hence (via the backdoor criterion, see [18]) P(X[t]|X[t − 1]) = P(X[t]|do(X[t − 1])) for all t, i.e., the observational- and interventional conditional distributions are identical. A counter-example would be the case of an exogenous variable, e.g., a stimulus, affecting both X[t − 2] and X[t]. In this case, the Markov property would be violated (X[t ∓ 2] ⫫̸ X[t]|X[t − 1]) and hence P(X[t]|X[t − 1]) ≠ P(X[t]|do(X[t − 1])) (we use the symbols ⫫̸ and ⫫ to denote statistical dependence and independence, respectively). As we discuss in the section on how to learn an MCM, empirical tests for Markovianity are essential to ensure that cognitive variables are causally consistent abstractions.

In the following, we assume that the functional mappings between states, as well as the probability distribution over their exogenous variables, do not change over time, i.e., we assume the process to be time-homogeneous. We further assume that any process we consider has converged to a stationary probability distribution. These assumptions are not required from a theoretical perspective but greatly simplify empirical inference.

To model the causal effect of neuronal activity on behavior, we extend the DAG in Fig 1 by a behavioral state vector B[t] and let X[t] → B[t], i.e., we model the behavioral states at time t to be caused by (a subset of) the neuronal states at time t. We note that it is reasonable to consider relations between neuronal activity and behavior as causal because we can, in principle, manipulate behavior independently of neuronal activity, e.g., we can fixate an animal and thereby prevent neuronal activity from causing an actual movement. We further note that we represent the behavioral states across time by measurement nodes, i.e., by nodes that have no causal effect on any other variables [26]. As such, the behavioral states do not form a Markov process, and B[t] ⫫̸ B[t − 2]|B[t − 1] due to the common effect of X[t − 2] on both B[t − 2] and B[t]. We denote the extension of the dSCM by the behavioral state vector as .

To model feedback loops of the neuronal system with its environment, the DAG could be further extended by a state vector S[t] that represents stimuli, which could be influenced by the system’s past behavior, e.g., S[t₀] → X[t₀] → B[t₀] → S[t₁]. We leave such an extension for future work.

Bridging the neuronal- and the cognitive level

Before discussing how to bridge the neuronal- and the cognitive level, we must first consider the nature of cognitive states. Somewhat surprisingly, there currently exists no generally agreed-upon definition of what a cognitive state is (cf. [27], pp. 41 ff.). The broader term cognition is commonly used to denote any kind of mental operation or structure that can be studied in precise terms [28]. To arrive at a working definition that allows us to operationalize cognitive states for the MCM framework, we consider three refinements of this concept of cognition. First, we propose that a cognitive state must be mathematically quantified to be studied in precise terms. Second, we consider an operation to represent a mechanism that translates a system from one state to another. And third, we consider the addendum or structure to indicate that a physical substrate supports any mental operation. These considerations lead us to the following working definition, which we will render mathematically precise towards the end of this section:

Definition 3 (Cognitive state) A cognitive state is a causally meaningful abstraction of a neuronal state.

To elucidate this definition, we need to explain what the terms causally meaningful and abstraction mean in this context.

We consider a concept as causally meaningful if it is conceptually (but not necessarily technically) feasible to fix its state to a specific value by an external intervention. For instance, the concept of age, defined as the time since birth, is not causally meaningful because there is no intervention conceivable that would allow an experimenter to alter the age of a subject. In contrast, the concept of membrane potential is causally meaningful because it is conceivable (and in this case also technically feasible) to set the membrane potential of a neuron to a desired voltage.

We consider a state to be an abstraction if it is a macroscopic description of the state of a system that is consistent with (potentially infinitely) many microscopic configurations of the system. Revisiting the example from the introduction, the temperature of a gas is a macroscopic abstraction of the microscopic states of the gas molecules because there are infinitely many configurations of kinetic energies of the molecules in the gas that give rise to the same temperature. This relation, however, is asymmetric. It is impossible to change the temperature of a gas without altering the kinetic energies of its molecules. Accordingly, the temperature of a gas and its molecular configuration do not stand in a causal but in a constitutive relationship.

Combining these two concepts, we call an abstraction causally meaningful if the microscopic configurations that determine the macroscopic state are causally meaningful. For instance, consider the statement the gas will ignite if we raise its temperature above a certain threshold. Even though there exists no direct intervention on the macroscopic concept of temperature, the experimental intervention of raising the temperature is meaningful because there exist microscopic interventions (increasing kinetic energies) that are identical to raising the temperature. These causally meaningful microscopic interventions endow the macroscopic concept of temperature with causal meaning.

Definition 3 expresses the notion that cognitive- and neuronal states stand in a similar constitutive relationship. While cognitive states are not intrinsically causally meaningful, e.g., it is unclear how one could set the cognitive state of anxiety to a desired level by an experimental intervention that directly acts on anxiety, the neuronal states that are consistent with a certain level of anxiety are intrinsically causally meaningful, in the sense that experimental interventions are conceivable (though not necessarily technically feasible) that realize one out of the consistent neuronal configurations. Furthermore, if intervening on neuronal states has a causal influence on future neuronal dynamics and on behavior, which we consider uncontroversial, this causal efficacy is inherited by cognitive states.

To summarize, Definition 3 expresses our understanding that we use cognitive concepts to reason about mental processes because we consider them to be causally effective in terms of the future evolution and behavior of the system under study due to their realization through intrinsically causally meaningful neuronal dynamics.

We now formalize the relations discussed above. In analogy to causal models on the neuronal level, we represent the cognitive state of a model system or organism by a cognitive dSCM with C[t] the cognitive state vector (note that E[t] and F[t] are specific to each model and not shared across neuronal- and cognitive dSCMs). To link the the neuronal- and the cognitive dSCM in a causally consistent manner, we require the two models to be behaviorally and dynamically causally consistent:

Definition 4 (Behavioral Causal Consistency—BCC) Let a neuronal dSCM in a behavioral context and a cognitive dSCM. Denote the state space of X[t] and C[t] by and , respectively. We call the triple behaviorally causally consistent if is a surjective mapping such that for every pair x₁, x₂ ∈ τ⁻¹(c) and for every we have (2) where τ⁻¹(c) is the pre-image of c.

In this case, we define do(C[t] = c)≔ do(X[t] = x|x ∈ τ⁻¹(c)) and have (3) for all .

If a neuronal- and a cognitive SCM are behaviorally causally consistent, every experimental intervention on the neuronal level has a matching intervention on the cognitive level and every cognitive-level intervention has at least one neuronal implementation, in the sense that both interventions lead to the same probability distribution over the behaviors. The surjectivity of τ ensures, first, that the cognitive level does not include any states that do not represent at least one neuronal state, and, second, that the cognitive level can form an abstraction of the neuronal level, i.e., that many neuronal states may map to the same cognitive state yet distinct cognitive states can only be linked to distinct neuronal states. Intuitively, the cognitive SCM constitutes a lossless compression of all information in the neuronal SCM that is causally relevant for a given set of behaviors. As such, behavioral causal consistency allows us to reason interchangeably about the causes of behaviors on the neuronal- and on the cognitive level. Importantly, behavioral causal consistency endows cognitive states with a causal meaning, in the sense that cognitive interventions, for which it may be unclear how the intervention can be experimentally implemented, e.g., increase happiness, can be translated into equivalent interventions on the neuronal level, for which a well-defined experimental procedure is available, e.g., stimulate a set of neurons.

The definition of dynamic causal consistency is analogous to behavioral causal consistency except that it considers the probability distribution over the next cognitive state.

Definition 5 (Dynamic Causal Consistency—DCC) We call a triple dynamically causally consistent if for all pairs of cognitive states (4) for all x₁, x₂ ∈ τ⁻¹(c) where τ⁻¹(c) is the pre-image of c.

In this case, we define do(C[t] = c)≔ do(X[t] = x|x ∈ τ⁻¹(c)) and have (5) for all and x ∈ τ⁻¹(c).

While behavioral causal consistency defines the consistency of neuronal- and cognitive states with respect to behavior, dynamic causal consistency defines consistency with respect to the dynamics of the neuronal system. Intuitively, a cognitive SCM that is DCC constitutes a lossless compression of all information in the neuronal SCM that is causally relevant to the dynamics of the neuronal system. In analogy to behavioral causal consistency, DCC grounds causal relations between cognitive states in the neuronal level, e.g., statements such as reducing boredom is likely to lead to increased happiness can be translated into the neuronal-level statement inducing a neuronal activity pattern x′ that represents low boredom, e.g., by electrical stimulation, is likely to lead to a neuronal state that corresponds to increased happiness.

We are now in a position to formally introduce the primary contribution of this work:

Definition 6 (Neuro-Cognitive Multilevel Causal Model—NC-MCM) We call a triple a neuro-cognitive multilevel causal model (NC-MCM) if it is behaviorally and dynamically causally consistent.

We remark that we have chosen τ not to depend on time because we consider a constant mapping between levels more desirable. It would be straightforward to extend the definition to time-varying mappings, but this generalization would substantially complicate learning the mapping from empirical data. We further note that in contrast to Rubenstein et al. [21] we only consider experimental interventions on the full neuronal- and cognitive state vectors at each time step and leave the extension of NC-MCMs to interventions on subsets of state variables for future work.

We illustrate the concept of a NC-MCM in the following toy example.

Example 1 Consider a neuronal dSCM with one endogenous variable (N = |X| = 1) and discrete state space , where each state represents one out of a total of four neuronal activity patterns. Further, assume there are three distinct behaviors with conditional probabilities given the neuronal state

Note that the conditional probability distribution over the three behaviors is identical for {x₁, x₂} and for {x₃, x₄}. As such, one may interpret the system as having two distinct macro-states, {x₁, x₂} and {x₃, x₄}, with each macro-state subserving a particular behavioral pattern that is expressed by a constant probability distribution over the three behaviors. We may thus define a mapping (6)

These two macro-states form the cognitive state space , where we may choose to name the two cognitive states foraging and recuperating, respectively. Next, assume the neuronal dSCM has the steady-state transition probability distribution T_X shown in the left-hand side of Fig 2. The transition matrix T_X and the mapping τ induce a cognitive dSCM with the transition probabilities shown in the right-hand side of Fig 2. The triple is behaviorally causally consistent, because the cognitive macro-states condense all information in the neuronal micro-states that are causally relevant for the set of behaviors, i.e., for every intervention on the neuronal micro-level there is an equivalent intervention on the cognitive macro-level that entails the same probabilities over the behaviors (cf. Definition 4). The triple is further dynamically causally consistent because the probabilities of remaining in the current or progressing to another cognitive state are identical for each neuronal state that maps to the same cognitive state. Because the triple is behaviorally and dynamically causally consistent, it fulfills the criteria for a NC-MCM. As such, the cognitive dSCM is a causally meaningful abstraction of the underlying neuronal dSCM because causal statements on the effects of current cognitive states on behavior and on future cognitive states are grounded in the neuronal level, e.g., the cognitive-level statement ‘the animal is moving around and feeding because it is foraging’ can be translated into the neuronal-level statement ‘the animal is moving around and feeding because it is in neuronal state x₁ or x₂’. We note that behavioral- does not imply dynamic causal consistency, e.g., introducing an asymmetry in the neuronal-level transition probabilities in Fig 2 would break dynamic- but not behavioral causal consistency.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. State-space diagram with transition probabilities for Example 1.

https://doi.org/10.1371/journal.pcbi.1012674.g002

We remark that the toy example above is constructed to illustrate the idea that in the NC-MCM framework redundancies in the neuronal dynamics and their relation to behaviors are exploited to build a cognitive-level model that preserves all causally relevant information. In practice, the challenge is to learn which (potentially widely-distributed and complex) neuronal activity patterns are redundant with respect to a behavioral context and with respect to the neuronal dynamics, a problem that we turn to in the next section.

Learning neuro-cognitive multilevel causal models

A NC-MCM model is specified by the triple , a neuronal-level dSCM in a behavioral context, a cognitive-level dSCM, and a causally consistent mapping between the two. A learning problem arises when any subset of these three components is not fully specified and needs to be inferred from data. In this section, we consider the case where we have access to a set of samples , generated by some unknown neuronal dSCM in a behavioral context, and our goal is to learn a mapping τ that induces a (behaviorally and dynamically) causally consistent cognitive dSCM. We consider this setting of particular interest because it amounts to learning the cognitive states that a neuronal system has developed in a specific behavioral context from empirical data. Other learning problems are briefly discussed in the final discussion section.

Learning a mapping τ that gives rise to a causally consistent cognitive dSCM proceeds in two steps. The first step is to learn a causal model between neuronal states and behaviors. This model then forms the basis for learning the mapping τ that induces a behaviorally and dynamically consistent cognitive dSCM. In this section, we discuss these steps from a conceptual perspective. A particular instantiation of a pipeline that learns a causally consistent cognitive dSCM from empirical data is presented in a later section.

Learning a causal model that relates neuronal activity to behavior amounts to learning the interventional distribution P(B[t]|do(X[t])). The gold standard to identify interventional distributions is by experimentation, i.e., by repeatedly setting X[t] to random values via an intervention and observing the induced behaviors. However, large-scale neuronal stimulation with concurrent recordings of neuronal activity and behavior remains a challenge [29]. Alternatively, we may attempt to learn the interventional distribution from observational data only. A variety of causal inference algorithms have been developed for this purpose [18, 25, 30] and applied to neuronal data [15, 17, 24]. However, causal structure learning from observational data is also a hard problem, the computational complexity of which typically grows exponentially in the number of variables. We thus follow a third approach in which we merely learn an observational prediction model P(B[t]|X[t]) from the set of samples . We discuss the implications of substituting an interventional model for an observational one at the end of this section.

After learning the interventional or observational distribution between neuronal activity and behavior, the second step is to learn the actual mapping τ. Learning this mapping can be broken down into two further steps. The first step is to construct a behaviorally consistent mapping between neuronal activity and cognitive states. The second step is to empirically test whether the induced cognitive dSCM is also dynamically causally consistent. We note that designing a one-step algorithm that directly constructs a mapping that is guaranteed to be behaviorally and dynamically consistent would be preferable but is beyond the scope of the present work.

To construct a behaviorally consistent mapping, we need to find subsets of the neuronal feature space, i.e., partitions, for which P(B[t]|X[t]) is constant. One way to solve this problem is to first learn a model that predicts the probability of each behavior as a function of the neuronal state, e.g., using a logistic regression model, a neural network model, or any other suitable modeling approach. We remark that the sampling rate of the neuronal dynamics must be faster than the transitions of the behavioral labels to allow for good decoding performance. In the second step, the predicted probabilities can then be clustered, e.g., using k-means or any other preferred clustering algorithm. The inferred clusters define a partition of the neuronal feature space with (approximately) constant conditional probabilities over all behaviors, i.e., the mapping τ. This mapping then induces a cognitive dSCM that is by construction behaviorally consistent, with the number of cognitive states being equal to the number of clusters.

Because behavioral does not imply dynamic causal consistency, we additionally need to test whether the learned mapping τ also induces a dynamically causally consistent cognitive dSCM. To do so, we first note that our definition of dynamic causal consistency (Def. 5) is identical to condition (3) in Theorem 1 of [31], with the exception that the former and the latter are based on interventional- and observational transition probabilities, respectively. Accordingly, we term condition (3) in [31] dynamic observational consistency. We can then invoke Theorem 4 of [31] to conclude that Markovianity of the cognitive dSCM is sufficient for dynamic observational consistency. Finally, we recall that Markovianity of a dSCM implies that the observational- and interventional transition probabilities coincide. As such, Markovianity of the cognitive dSCM is also sufficient for dynamic causal consistency.

To empirically test a cognitive dSCM for Markovianity, we test the null-hypothesis H₀: C[t − 1] ⫫ C[t + 1]|C[t]. If we find sufficient evidence against Markovianity, we reject the null hypothesis and conclude that the cognitive dSCM is not dynamically causally consistent. Otherwise, we accept the null hypothesis and conclude that the cognitive dSCM induced by τ is behaviorally and dynamically causally consistent with the data-generating neuronal dSCM. We note that if we find a cognitive dSCM not to be dynamically causally consistent, we can vary the number of clusters to tune the granularity of the cognitive dSCM and repeat the test for Markovianity.

If we base the procedure described above on the observational distribution P(B[t]|X[t]) (instead of on the interventional distribution P(B[t]|do{X[t]})), the cognitive dSCM is behaviorally observationally consistent (BOC) but not (in general) behaviorally causally consistent (BCC). However, the causal coarsening theorem states that a causal partitioning is almost always a coarsening of an observational partitioning [32, 33]. As such, a cognitive dSCM that is BCC can be obtained from a cognitive dSCM that is BOC by fusing subsets of cognitive states. While experimental interventions may be required to identify which cognitive states cause behaviors with identical probabilities and thus should be fused, the number of required interventions is on the order of the number of cognitive states and not on the order of the (potentially orders of magnitude larger) number of neuronal states. As such, learning a cognitive dSCM on observational data first and reducing the number of cognitive states by experimental interventions afterward is experimentally more tractable than directly constructing the interventional distribution P(B[t]|do(X[t])) on the neuronal level. If even a reduced number of experimental interventions is not feasible, we may have to accept that a cognitive dSCM is merely BOC.

The data

The data set we use has been recorded by Kato et al. [37] and is available at https://osf.io/2395t/. We use the data subset that has been recorded without externally applied chemosensory stimulation. It consists of data from five immobilized worms with 107—131 neurons recorded in each individual worm for a period of 18 minutes at a sampling rate of approximately 2.85 Hz. We subsequently refer to a sample of the calcium traces as the neuronal state vector X[t], which represents the state of all neurons recorded in one animal at time t.

Even though all five worms were immobilized during the recordings, established equivalences between the activity of individual neurons and the worms’ behavior were used to label each neuronal state vector with the behavior that would have been concurrently observed in a non-immobilized worm; these encompass motor commands for forward crawling, forward slowing, backward crawling, turning dorsally and turning ventrally [37]. Based on distinct activity patterns, the neuronal state corresponding to backward crawling was further subdivided into reversal 1, reversal 2, and sustained reversal [37]. To date, the behavioral correlates of these sub-divisions are not known. The behavioral labels assigned to the neuronal states by [37] are . Neuronal states that appeared ambiguous, i.e., for which the behavioral label could not be clearly identified, are labeled as nostate, corresponding to a small fraction (∼1%) of the data frames.

Learning a cognitive model of C. elegans

Learning a cognitive model proceeds in three steps: Predicting the probabilities of all behaviors for every neuronal state vector, clustering the predicted probabilities to obtain the cognitive states, and testing the induced cognitive state transitions for Markovianity.

To learn an observational model that relates neuronal activity patterns to behavior in each individual animal, i.e., to estimate , we trained a set of logistic regression models to discriminate between all pairs of the eight behaviors, resulting in models per animal. We remark that while we chose to implement multi-class classification through a set of 28 pairwise classifiers, eight one vs. all classifiers or other intrinsically multi-class classifiers could also be used, as long as they provide estimates of the probabilities of the behavioral labels. We chose logistic regression models for this step because they directly estimate the probability of a behavior as a function of the neuronal state. Each classifier was trained on all recorded neurons of each individual animal, excluding only those neurons that were used to identify the behavioral labels (AVAL, AVAR, SMDDR, SMDDL, SMDVR, SMDVL, RIBR, and RIBL) [37]. Classification accuracy was evaluated via majority voting in combination with 10-fold cross-validation using random data splits. This evaluation resulted in a group-average decoding accuracy of 92.2%, with decoding results between 89.4% and 95.6% across individual worms, indicating that the linear logistic regression models provide a good characterization of the observational relationships between neuronal activities and behaviors. For subsequent modeling steps, the regression models were trained on all data without cross-validation. We then used these models to compute the conditional probabilities each pairwise classifier assigned to their respective behaviors as a function of the neuronal state, resulting in a 28-dimensional vector of behavioral probabilities for each neuronal state, which we subsequently term the behavioral probability trajectories. We note that the logistic regression models estimate observational- and not interventional probabilities.

To learn the mapping τ, we employed k-means clustering in the 28-dimensional space of probability estimates to assign neuronal states to clusters with (approximately) constant conditional behavioral probabilities (cf. Definition 4). We varied the number of clusters for k-means between two and 20 and re-ran k-means 100 times for each k with random initial seeds. For each k and run, we thereby obtained an assignment of every neuronal state to one out of k cognitive states, which we subsequently refer to as a cognitive state trajectory C[t].

We then tested each of the 5 (worms) × 20 (range of the number of cognitive states) × 100 (clustering runs) cognitive state trajectories for Markovianity by, first, estimating the first- and second-order cognitive state transition probability matrices P(C[t]|C[t − 1]) and P(C[t]|C[t − 1], C[t − 2]), second, computing the total variance of P(C[t]|C[t−1]) across all states of C[t−2] (whose expected value for a Markov process is zero), and third, computing the same total variance for one thousand simulated Markovian cognitive state trajectories with state transition probability matrix P(C[t]|C[t − 1]). This enabled us to estimate the p-value under the null hypothesis of a Markovian cognitive state trajectory as the frequency at which the simulated total variance exceeded the observed one (this test for Markovianity is implemented as the function markovian() in the nilab toolbox). Because higher p-values signify a better clustering result, in the sense that the learned cognitive state trajectory exhibits no evidence against Markovianity, we then picked, for each worm and number of cognitive states, the cognitive state trajectory with the highest p-value. We remark that we could also split the data into a training- and a test set, learn and select the best clustering on the training set, and then test for Markovianity on the held-out test set. Because the number of samples available in this setting is limited, and splitting the data would further reduce the probability of finding evidence against Markovianity on the test set, using all data for clustering is the more conservative approach. This issue is relevant when invoking the DCC property, which we revisit in the section on decision making.

Fig 3 shows the p-values of the selected cognitive state trajectories as a function of the number of cognitive states for each worm (dots) as well as averaged across all worms (line). When choosing α = 0.05 as a lower threshold for accepting the null hypothesis, we obtain Markovian cognitive state trajectories for any number of cognitive states in the range from three to 19, with a maximum average p-value for seven states. As such, we infer that all cognitive state trajectories with three to 19 states are dynamically causally consistent. As discussed below, we can then control the degree of granularity at which we study the dynamics of C. elegans by varying the number of cognitive states in this range.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. p-values for rejecting the null-hypothesis of Markovianity for each worm (and averaged across worms) as a function of the number of cognitive states.

https://doi.org/10.1371/journal.pcbi.1012674.g003

To build a better intuition for the procedure of learning a cognitive state trajectory, Fig 4 illustrates the relevant steps on the data of the first worm with five cognitive states (all trajectories are projected to their first two principal components for visualization): The neuronal state trajectories together with the behavioral labels (A) are used to estimate the behavioral probability trajectories (B). These trajectories are then clustered to assign each element of the behavioral probability trajectories to a cognitive state (C). We remark that Fig 4B and 4C are projections from a 28- to a two-dimensional space via PCA, which does not adequately represent the spatial separation of neuronal states that belong to distinct cognitive clusters. Re-projecting the cognitive labels onto the neuronal state trajectories gives the neuro-cognitive state trajectories (D), from which we can compute the cognitive state transition model P(C[t]|C[t − 1]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Illustration of the procedure of learning a cognitive state trajectory (see main text for details).

https://doi.org/10.1371/journal.pcbi.1012674.g004

We remark that the cognitive dSCM of each animal is behaviorally observationally consistent by construction and dynamically causally consistent if the cognitive dSCM is Markovian (we emphasize again that Markovianity can not be verified; we can only fail to find evidence against Markovianity and thus not reject the null hypothesis). Because we built the cognitive dSCM on the observational distribution P(B|X) rather than on the interventional distribution P(B|do(X)), which is not available in the present setting, behavioral causal consistency is not guaranteed. In its present form, the cognitive dSCM thus only supports causal statements about the dynamics of the cognitive dSCM but not about its relation to behavior. However, its observational behavioral consistency can be used to derive causal hypotheses that guide the design of experimental studies to establish causal relations between neuronal states and behaviors via interventions, a topic we discuss below in more detail.

Interpreting the cognitive model of C. elegans

In this section, we demonstrate the ability of the learned cognitive models to reveal behavioral motifs of C. elegans, show that from a data-visualization perspective the NC-MCM framework can be understood as a neuronal manifold learning technique that abstracts the essential features of a manifold into a directed graph, and discuss how the cognitive model can provide insights into decision-making processes.

Behavioral motifs of C. elegans.

In the following, we juxtapose the behavioral state transition diagram with the state diagram that is obtained by expanding the behavioral- by the cognitive states, i.e., we compare P(B[t]|B[t − 1]) and P(C[t], B[t]|C[t − 1], B[t − 1]). In particular, we show that the behavioral state transition diagram conflates distinct behavioral motifs that are revealed when considering the behavior commands in relation to the cognitive states in which they appear. We first illustrate the results on the third worm in the data set because this worm exhibits the most simple state transitions of all the five worms. We then show that the behavioral motifs discussed below are present with small variations in every worm.

Fig 5 shows the behavioral- (A, left column) and the cognitive-behavioral state diagram with three cognitive states (B, right column) of the third worm in the data set (we have chosen three cognitive states because this is the lowest number of cognitive states for which we do not find evidence against Markovianity; the behavioral motifs discussed below are also apparent when considering models with more cognitive states). In the left diagram, each node represents one behavioral state. The size of each node represents the probability of the worm being in the corresponding behavioral state at any point in time. The width of an outgoing edge represents the probability of the worm transitioning from a particular state to another state (to reduce spurious edges that result from jitters of the neuronal trajectories across the decision boundaries in the probability space, we only plot state transition if the length of a state exceeds two samples). Edges indicating the probabilities of staying in a particular behavioral state, i.e., edges from one node to itself, are not drawn to reduce clutter.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Behavioral- and cognitive-behavioral state diagrams of the third worm (see text for details).

Arrows that account for less than 0.1% of outgoing transitions of each node have been removed to reduce clutter.

https://doi.org/10.1371/journal.pcbi.1012674.g005

Fig 5A reveals a clear structure in the worm’s behavioral command (or states) sequences. The worm is most often found executing a sustained reversal state (revsus). A sustained reversal is either followed by a ventral- (vt) or dorsal (dt) turn state, both of which segue into a slowing state (slow). The slowing state alternates with a forward state (fwd) before returning to a sustained reversal state via either of two types of reversal states (rev1 or rev2).

We have chosen observational language to describe this behavioral state transition diagram because the behavioral dynamics are not Markovian (the statistical test for Markovianity described earlier rejects the null hypothesis of Markovianity at α = 0.05 (p = 0.015)). This non-Markovianity indicates that past behavioral states provide information about the future state that is not contained in the current state, as was suggested by [37]. This fact becomes intuitively plausible when considering the cognitive-behavioral state transition diagram in Fig 5B. This diagram expands the behavioral by the cognitive states, i.e., we represent the eight behaviors separately for each cognitive state in which they occur (note that the cognitive states are drawn counter clockwise with the eight behaviors of each cognitive state also arranged in circles). This Markovian representation reveals two distinct behavioral motifs: A revsus → vt → slow → rev1 → revsus and a revsus → dt → fwd ↔ slow → rev2 → revsus loop. These two loops that occur in cognitive states C3 and C2, respectively, are connected via cognitive state transitions between C1 and C3 during a sustained reversal (which we discuss in more detail below). The first motif represents a repeated execution of a state sequence composed of sustained reversal followed by a ventral turn, slowing and reversal 1. The second motif is composed mostly of switching between slowing and forward crawling states. In the third worm, this motif always starts with a dorsal turn state and terminates with a reversal 2. These behavioral motifs are well known in C. elegans [38, 39]. The first motif corresponds to a pirouette, where animals execute multiple reverse-turning events in close succession. The second motif corresponds to a forward run episode, where animals persistently head in a forward direction. This motif is terminated by a rev2 state, corresponding to the neuronal signature that terminates a forward run episode (cf. [37]).

Notably, the NC-MCM framework discovered these motifs without any prior knowledge of the underlying behavioral dynamics. Rather, the behavioral motifs are revealed by the NC-MCM framework because they are embedded in distinct neuronal dynamics. Importantly, this entails the ability of the NC-MCM framework to assign time frames that were manually annotated with the same behavioral label to distinct cognitive states if the executed behaviors differ in their neuronal realization. In Fig 5, this is apparent in two distinct slowing behaviors, one in each behavioral motif, that are supported by cognitive states C2 and C3, respectively. In C2, slowing could correspond to brief episodes of reducing locomotion speed while the animals remain in a forward run [37]. In C3, slowing could correspond to intermittent pausing or brief forward crawling during successive re-orientations. In the behavioral state diagram in the left column of Fig 5, these two different types of slowing movements are conflated. This conflation leads to a non-Markovian representation because the probabilities of entering a rev1 and rev2 state are higher when the slowing movement is preceded by a ventral and a dorsal turn, respectively. Conversely, the Markovian representation of the cognitive-behavioral state diagram ensures that it correctly distinguishes seemingly similar behaviors that are supported by distinct neuronal dynamics.

While the third worm exhibits the most distinct separation of the two behavioral motifs, both motifs are present in every worm (Fig 6). Because the worms slightly vary in the complexity of their state transitions, with the other worms also showing infrequent state transitions between the two motifs, the number of cognitive states at which the two behavioral motifs first emerge varies across worms (four cognitive states for the first two worms and three cognitive states for the remaining three worms).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Behavioral- and cognitive-behavioral state diagrams of all worms.

Arrows that account for less than 0.075% of outgoing transitions of each node have been removed to reduce clutter.

https://doi.org/10.1371/journal.pcbi.1012674.g006

Representing C. elegans’ neuronal manifold as a directed graph.

Individual neurons are embedded in brain networks that collectively organize their high-dimensional neuronal activity patterns into lower-dimensional neuronal manifolds [40, 41]. The goal of neuronal manifold learning techniques is to find low-dimensional representations of neuronal data that enable insights into the structure of neuronal dynamics and their relation to behavior [9]. In neuroscience, classic dimensionality reduction techniques, such as principal component analysis (PCA), Laplacian eigenmaps (LEM), and t-SNE, are complemented by modern techniques such as UMAP [10], MIND [11], CEBRA [12], and BundDLe-Net [13]. In the following, we show that the cognitive-behavioral state diagrams introduced in the previous section can be interpreted as a neuronal manifold learning technique that represents the essential aspects of the manifold as a directed graph.

Fig 7 shows a side-by-side comparison of the neuronal manifold of the third worm as inferred by BundDLe-Net [13] (left column) and the cognitive-behavioral state diagram of the NC-MCM framework (right column). Looking at the left column, it is immediately apparent that the neuronal manifold exhibits two main cycles, a revsus (gray) → vt (turquoise) → slow (red) → rev1 (green) → revsus (gray) and a revsus (gray) → dt (orange) → slow (red) → fwd (yellow) → rev2 (blue) → revsus (gray) cycle, that correspond to the behavioral motifs revealed by the cognitive-behavioral state diagram in the right column of Fig 7 and discussed in the previous section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Side-by-side comparison of the neuronal manifold of the third worm as learned by BunDLe-Net and its representation as a directed graph in the NC-MCM framework.

https://doi.org/10.1371/journal.pcbi.1012674.g007

To better understand the structure of the neuronal manifold, we now consider the branching and convergence points of the neuronal trajectories. We identify the dense cluster of neuronal states that correspond to a sustained reversal (revsus shown in gray at the bottom of the left plot in Fig 7) as the root node of the manifold that acts both as a branching and a convergence point. In particular, the neuronal state trajectories branch out from here into one of the two behavioral motifs via a ventral- (vt—turquoise) or a dorsal turn (dt—orange). Once a ventral turn is initiated, the neuronal trajectories are highly consistent; all trajectories execute a loop (accompanied by a ventral turn, slowing, and reversal 1) before converging again into the root node during a sustained reversal. If a dorsal turn is initiated, on the other hand, the neuronal trajectories enter a second branching point during the subsequent slowing movement (red), from which they either return directly during a rev2 behavior (blue) into the root node (gray) or take the longer route into the extended branch of the persistent alternation between slowing (red) and forward (yellow) behavior. Both of these branches converge again in the rev2 behavior before returning to the sustained reversal root node. To summarize, the dynamics on the neuronal manifold are determined by a small number of branching and convergence points with highly consistent trajectories between these points.

In the cognitive-behavioral state diagram in the right column of Fig 7, the branching and convergence points of the neuronal manifold are represented by nodes with an out- and in-degree greater than one, respectively. The trajectories between the branching and convergence points are represented by nodes with an out- and in-degree of one. Based on this correspondence, it is easy to verify visually that the cognitive-behavioral state diagram constitutes a representation of the structure of the neuronal manifold as a directed graph in the sense that the nodes of the graph represent bundles of trajectories on the neuronal manifold and the edges between the nodes represent the possible paths that the neuronal trajectories can take on the manifold. This correspondence also holds for the other four worms shown in Figs 8–11 (three-dimensional rotating GIF visualizations of the embeddings are available in the BunDLe-Net repository at https://github.com/akshey-kumar/BunDLe-Net).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 1.

https://doi.org/10.1371/journal.pcbi.1012674.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 2.

https://doi.org/10.1371/journal.pcbi.1012674.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 4.

https://doi.org/10.1371/journal.pcbi.1012674.g010

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 5.

https://doi.org/10.1371/journal.pcbi.1012674.g011

As such, the cognitive state transition models of the NC-MCM abstract away variability of the the neuronal dynamics to reveal the essential structure of the neuronal manifold in a directed graph. In this way, the NC-MCM reduces the study of the structure of neuronal manifolds to a graph theoretic problem, a mature scientific field for which a plethora of computationally efficient algorithms are available [42].

Towards understanding decision making in C. elegans.

As we have seen in the previous section, the neuronal trajectories alternate between segments with highly consistent dynamics, e.g., when executing the behavioral motif initiated by a ventral turn, and segments with high uncertainty about the future dynamics and behavior, e.g., when performing a sustained reversal movement. In the following, we show how the cognitive-behavioral state transition diagrams of the NC-MCM framework provide insights into the neuronal basis of future neuronal dynamics during segments with high uncertainty, i.e., we show how to use the NC-MCM framework to study decision-making in C. elegans. In particular, we study how a ventral vs. a dorsal turn is initiated during a sustained reversal and how the alternating forward and slowing behavior is terminated by a rev2-reversal.

We begin again by considering the cognitive-behavioral state transition diagram of the third worm in Fig 7. In particular, we note that the sustained reversal movement (revsus) is distributed across the two cognitive states C1 and C3, with frequent, cyclical transitions between the two states. Importantly, the probability that the worm transitions from a sustained reversal to a particular turning behavior differs between these cognitive states. When the worm is in cognitive state C3, it transitions from a sustained reversal into a ventral- and a dorsal turn at frequencies of 3.04% and 0.72%, respectively. In cognitive state C1, on the other hand, the frequencies of transitions from a sustained reversal into a ventral- and a dorsal turn are 0.17% and 0.84%, respectively. Ventral turns are thus roughly 18 times more frequently initiated in state C3 than in state C1 while dorsal turn initiations occur with similar frequencies in both cognitive states (they are 1.17 times more likely in C1 than in C3). We can thus interpret a cognitive state transition from C1 into C3 during a sustained reversal movement as a decision process that makes a ventral turn more likely than a dorsal turn and vice versa. We now turn to the question of how these cognitive state transitions are realized on the neuronal level.

To do so, we analyze which global perturbation of neuronal states during a particular type of movement the model predicts to induce a cognitive state that increases the probability of another type of behavior. To illustrate this idea, consider again the two states C1:revsus and C3:revsus of the third worm. By first computing the mean activation of every neuron in each of the two states, i.e., the mean of all neuronal states that map to C1:revsus and the mean of all neuronal states that map to C3:revsus, and then subtracting the two means, we obtain the neuronal perturbation that the cognitive-behavioral model predicts to induce a state transition from C1:revsus to C3:revsus; a state transition that increases the probability of transitioning into a ventral turn from 0.17% to 3.04%. To generalize from this example, we can pick a source behavior, e.g., a sustained reversal, and a target behavior, e.g., a ventral turn, determine all cognitive states in which the source behavior occurs, for each pair of cognitive states compute the neuronal perturbation that the NC-MCM model predicts to induce a transition from one to the other, weigh this neuronal perturbation by the difference in probability in the target behavior between the two cognitive states, and finally average the neuronal perturbations across all possible state transitions and worms. This results in a neuronal perturbation vector, which we subsequently denote by , that the NC-MCM model predicts, on average across all cognitive states and worms, to induce a state transition that increases the probability of the target behavior while maintaining the current source behavior.

This procedure necessitates that the cognitive-behavioral state diagram of each worm represents the source behavior in multiple cognitive states. As can be checked in Figs 7–11, out of the cognitive-behavioral state diagrams with three cognitive states only the state diagram of the third worm represents the uncertainty during the sustained reversal movement by more than one cognitive state. This limitation can be easily remedied by considering more than three cognitive states. In the following, we consider cognitive-behavioral state diagrams with seven cognitive states, because we found seven to be the optimal number of cognitive states with respect to (the absence of evidence against) Markovianity across all worms, cf. Fig 3 (we remark that among all Markovian cognitive models there is no correct or incorrect number of cognitive states—we can vary the number of cognitive states and thus choose the level of granularity at which we analyze the cognitive dynamics). Fig 12 displays the corresponding cognitive-behavioral state diagram for the third worm. Increasing the number of cognitive states from three to seven preserves the overall structure as well as the two primary behavioral motifs while splitting up the cognitive states into a more fine-grained representation of the neuronal dynamics and their relation to behavior. In particular, the pirouette motif that is initiated by a ventral turn and that was represented by cognitive state C3 in Fig 7 is represented in Fig 12 by the two cognitive states C1 and C2. The forward-run motif which is initiated by a dorsal turn and that was represented in Fig 7 by C2 is represented in Fig 12 by the two cognitive states C4 and C5. The uncertainty in the transitions between these two motifs, which was characterized in Fig 7 by the cyclical alterations between cognitive states C1 and C3 during a sustained reversal movement, is represented in the seven-states model by transitions between the cognitive states C2, C6, and C7. Moving to cognitive-behavioral state diagrams with seven cognitive states in each worm, we can thus compute the neuronal perturbation vector between a source and a target behavior across a broad range of cognitive states across all worms.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Cognitive-behavioral state transition diagram of the third worm with seven cognitive states.

https://doi.org/10.1371/journal.pcbi.1012674.g012

Fig 13 displays the neuronal perturbation vectors , , and for all neurons that are shared across the recordings of all five worms, i.e., the three neuronal perturbation vectors that during a sustained reversal render the maintenance of the sustained reversal, a transition to a ventral turn, and a transition to a dorsal turn more likely. Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. Statistical tests were carried out with an ANOVA based on 10.000 random permutations with Bonferroni correction for multiple comparisons according to the number of neurons. Not considering those neurons used for labeling the behaviors (AVAL, AVAR, SMDDR, SMDDL, SMDVR, SMDVL, RIBR, and RIBL), the neuronal perturbation vector exhibits highly statistically significant differences across specific sets of descending interneurons. In particular, the NC-MCM model predicts that increasing the activation of AVBL and AVBR while reducing the activity of AVEL and AVER increases the probability of a turning behavior, with weaker and stronger changes linked to dorsal- and ventral turns, respectively. Conversely, applying the opposite pattern makes it more likely that worms maintain a sustained reversal. We remark that RIVL and RIVR may play a further role in differentiating between dorsal and ventral turns. Because these two neurons show no significant differences after Bonferroni correction for multiple comparisons, we refrain from further interpreting their effects. These neuronal perturbation vectors for maintaining or terminating a sustained reversal are markedly different from those obtained for changes between cognitive states that either maintain a slowing-forward motif or interrupt this motif through a rev2-type reversal, as shown in Fig 14. Here, an increase in activation of AIBL, AIBR, ASKR, RIVL, and RIVR and a decrease in activation of RMED, RMEL, and RMER are predicted to maintain a slowing behavior, with the converse pattern inducing a cognitive state that interrupts the slowing-forward motif by a rev2-type reversal.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Neuronal perturbations that, when applied during a sustained reversal, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a sustained reversal movement (blue), initiating a ventral turn (orange), or initiating a dorsal turn (green).

Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively.

https://doi.org/10.1371/journal.pcbi.1012674.g013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Neuronal perturbations that, when applied during a slowing movement, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a slowing movement (blue), initiating a forward movement (orange), or initiating a rev2 reversal (green).

Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively.

https://doi.org/10.1371/journal.pcbi.1012674.g014

Due to the DCC property of the cognitive models discussed above, we may interpret the neuronal perturbation vectors as empirically testable predictions which perturbations of neurons via external stimulation induce cognitive states that render certain behaviors more or less likely (because the learned cognitive models are only observationally but not causally behaviorally consistent, we may not apply the same causal reasoning to the relations between cognitive state transitions and behaviors). However, it remains an open question how these perturbations come about naturally, i.e., how the worm decides which behavioral motif to initiate and when. A priori, there are two potential explanations. In the first explanation, the worm’s neuronal dynamics could be deterministic with the apparent randomness during sustained reversals and the slowing-forward motifs being due to some unobserved (latent) neurons. In this interpretation, observing all behaviorally relevant neurons would eliminate the randomness and lead to non-overlapping trajectory bundles in Figs 7–11. In the second interpretation, the indeterminacy may be due to inherent randomness in neuronal activation. Due to the DCC property, our results are consistent with the second but not with the first interpretation: A latent neuron that affects at least one observed neuron would induce a dependence between the current- and past states of the observed neuron, rendering the neuronal dynamics non-Markovian. Because we found no evidence against Markovianity in the cognitive state transitions, which constitute an abstraction of all behaviorally relevant information in the neuronal dynamics, our results are consistent with the interpretation that decision-making in C. elegans has an intrinsically random component, as was suggested for switches between forward and backward movement [43]. However, we remark that in the present setting the worms have been immobilized and received no time-varying sensory input. In more ecologically realistic settings, sensory stimuli may override inherent randomness.

Discussion of the experimental results in C. elegans

The NC-MCM framework, applied to brain wide Ca2+-imaging data of C. elegans, revealed cognitive states that correspond to neuronal representations of behavioral hierarchies that sit on top of a previously described motor hierarchy [44]. A pirouette is a long time scale behavioral motif that encompasses the repetitive execution of reversal-turn command sequences, which can be triggered spontaneously or by sensory inputs [38, 45]. In our data, a pirouette manifests as a motif of switching dynamics between neuronal activity patterns representing these actions; notably the NC-MCM framework was able to find these patterns in the absence of neuronal activities that specifically mark the initiation, duration or termination of a pirouette, i.e., which we would have interpreted as explicit upper-hierarchy neuronal representations of a pirouette. We cannot exclude that our recording technique is insensitive to such activation patterns. But alternatively, and in analogy to the pressure of a gas (see Introduction), a pirouette could be seen as a useful causally consistent description of a higher-level navigational strategy of C. elegans [38, 45] that emerges from the ‘microscopic’ dynamical switching behavior of neuronal circuits without any explicit neuronal representation. This has implication on potential control mechanisms, which, thereby could act directly at the level of synapses and neuronal excitability within these circuits, and without the need of upper-hierarchy command-circuits. In the same vein, the learned NC-MCM suggests that forward run is a dynamical motif that encompasses continuous forward crawling with intermittent slowing events. It further suggests that slowing during forward runs is neuronally distinct from slowing during pirouettes, which were, however, lumped together into one state in our previous work [37].

When allowing for a finer granularity of many cognitive states, the NC-MCM framework can be a useful discovery tool to identify circuit elements that are potentially implicated in decision making. How C. elegans decides between a dorsal or ventral turn following a reversal is not known [37]. The learned NC-MCM suggests circuit elements that terminate sustained reversals and influence the subsequent binary choice via expected motor neuron classes (RIV, SMDV or RMED) [46], but also unexpected neuronal cell types like the descending interneuron class AVE or the neuromodulatory neuron class ALA. Likewise, the NC-MCM makes concrete suggestion for the circuitry that controls slowing, like the AIB neurons that show Ca2+-transients during discrete slowing events in freely moving worms [37], or RIB neurons that have been implicated in the control of locomotion speed [37, 44, 47]. Interestingly, ASK sensory neurons, which exhibit spontaneous Ca2+-fluctuations under the experimental conditions used here [48, 49] are further suggested to control slowing. The NC-MCM framework, thereby, can effectively guide future interrogation strategies, such as optogenetics, to confirm the role of these classes as decison making neurons.