Supervenience

Our analysis considers two time points of the evolution of the system, denoted as t and t′, with t < t′. The corresponding dynamics are encoded in the transition probability . We consider features generated via a conditional probability that are supervenient on the underlying system; i.e. that does not provide any predictive power for future states at times t′ > t if the complete state of the system at time t is known with perfect precision. We formalise this in the following definition.

Definition 1. A stochastic process V_t is said to be supervenient over X_t if V_t − X_t − X_t′′ form a Markov chain for all t′′ ≠ t.

The above condition is equivalent to require V_t to be statistically independent of X_t′′ when X_t is given. The relationship between supervenient features and the underlying system is illustrated in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Diagram of causally emergent relationships.

Causally emergent features have predictive power beyond individual components. Downward causation takes place when that predictive power refers to individual elements; causal decoupling when it refers to itself or other high-order features.

https://doi.org/10.1371/journal.pcbi.1008289.g002

This formalisation of supervenience characterises features V_t that are fully determined by the state of the system at a given time t, but also allows the feature to be noisy—which is not critical for our results, but is useful for extending their domain of applicability to practical scenarios. In effect, Definition 1 includes as particular cases deterministic functions such that V_t = F(X_t), as well as features calculated under observational noise—e.g. V_t = F(X_t) + ν_t, where ν_t is independent of X_t for all t. In contrast, features that are computed using the values of X_t at multiple timepoints (e.g. the Fourier transform of X_t) generally fail to be supervenient.

Partial information decomposition

Our theory is based on the Partial Information Decomposition (PID) framework [14], which provides powerful tools to reason about information in multivariate systems. In a nutshell, PID decomposes the information that n sources X = (X¹, …, Xⁿ) provide about a target variable Y in terms of information atoms as follows: (1) with being the set of antichain collections [14]. Intuitively, for α = {α₁, …, α_L} represents the information that the collection of variables provide redundantly, but their sub-collections don’t. For example, for n = 2 source variables, α = {{1}{2}} corresponds to the information about Y that is provided by both of them, α = {{i}} to the information provided uniquely by X_i, and, most interestingly, α = {{12}} corresponds to the information provided by both sources jointly but not separately—commonly referred to as informational synergy.

One of the drawbacks of PID is that the number of atoms (i.e. the cardinality of ) grows super-exponentially with the number of sources, and hence it is useful to coarse-grain the decomposition according to specific criteria. Here we introduce the notion of k^th-order synergy between n variables, which is calculated as with . Intuitively, Syn^(k)(X;Y) corresponds to the information about the target that is provided by the whole X but is not contained in any set of k or less parts when considered separately from the rest. Accordingly, only contains collections with groups of more than k sources.

Similarly, we introduce the unique information of X^β with β ⊂ [n] with respect to sets of at most k other variables, which is calculated as

Above, , and X^−β being all the variables in X whose indices are not in β. Put simply, represents the information carried by X^β about Y that no group of k or less variables within X^−β has on its own. Note that these coarse-grained terms can be used to build a general decomposition of I(X, Y) described in S1 Appendix (Section 1), the properties of which are proven in S1 Appendix (Section 2).

One peculiarity of PID is that it postulates the structure of information atoms and the relations between them, but it does not prescribe a particular functional form to compute . In fact, only one of the information atoms must be specified to determine the whole PID—usually the redundancy between all individual elements [14]. There have been multiple proposals for specific functional forms of in the PID literature; see e.g. Refs. [15–18]. A particular method for fully computing the information atoms based on a recent PID [19] is discussed in Section Measuring emergence via synergistic channels.

Conveniently, our theory doesn’t rely on a specific functional form of PID, but only on a few basic properties that are precisely formulated in S1 Appendix (Section 2). Therefore, the theory can be instantiated using any PID—as long as those properties are satisfied. Importantly, as shown in Section Practical criteria for large systems, the theory allows the derivation of practical metrics that are valid independently of the PID chosen.

Defining causal emergence

With the tools of PID at hand, now we introduce our formal definition of causal emergence.

Definition 2. For a system described by X_t, a supervenient feature V_t is said to exhibit causal emergence of order k if (2)

Accordingly, causal emergence takes place when a supervenient feature V_t has irreducible causal power, i.e. when it exerts causal influence that is not mediated by any of the parts of the system. In other words, V_t represents some emergent collective property of the system if: 1) contains information that is dynamically relevant (in the sense that it predicts the future evolution of the system); and 2) this information is beyond what is given by the groups of k parts in the system when considered separately.

To better understand the implications of this definition, let us study some of its basic properties.

Lemma 1. Consider a feature V_t that exhibits causal emergence of order 1 over X_t. Then,

1. The dimensionality of the system satisfies n ≥ 2.
2. There exists no deterministic function g(⋅) such that for any j = 1, …, n.

Proof. See S1 Appendix, Section 3.

These two properties establish causal emergence as a fundamentally collective phenomenon. In effect, property (i) states that causal emergence is a property of multivariate systems, and property (ii) that V_t cannot have emergent behaviour if it can be perfectly predicted from a single variable.

In order to use Definition 2, one needs a candidate feature V_t to be tested. However, in some cases there are no obvious candidates for an emergent feature, for which Definition 2 might seem problematic. Our next result provides a criterion for the existence of emergent features based solely on the system’s dynamics.

Theorem 1. A system X_t has a causally emergent feature of order k if and only if (3)

Proof. See S1 Appendix, Section 2.

Corollary 1. The following bound holds for any supervenient feature V_t: .

This result shows that the capability of exhibiting emergence is closely related to how synergistic the system components are with respect to their future evolution. Importantly, this result enables us to determine whether or not the system admits any emergent features by just inspecting the synergy between its parts—without knowing what those features might be. Conversely, this result also allows us to discard the existence of causal emergence by checking a single condition: the lack of dynamical synergy. Furthermore, Corollary 1 implies that the quantity serves as a measure of the emergence capacity of the system, as it upper-bounds the unique information of all possible supervenient features.

Theorem 1 establishes a direct link between causal emergence and the system’s statistics, avoiding the need for the observer to propose a particular feature of interest. It is important to remark that the emergence capacity of a system depends on the system’s partition into microscopic elements—in fact, it is plausible that a system might have emergence capacity under one microscopic representation, but not with respect to another after a change of variables. Therefore, emergence in the context of our theory always refers to “emergence with respect to a given microscopic partition.”

A taxonomy of emergence

Our theory, so far, is able to detect whether there is emergence taking place; the next step is to be able to characterise which kind of emergence it is. For this purpose, we combine our feature-agnostic criterion of emergence presented in Theorem 1 with Integrated Information Decomposition, ΦID, a recent extension of PID to multi-target settings [20].

Using ΦID, one can decompose a PID atom as (4)

For example, if n = 2 then represents the information shared by both time series at both timesteps (for example, when are all copies of each other); and corresponds to the synergistic causes in X_t that have a unique effect on (for example, when ). More details and intuitions on ΦID can be found in Ref. [20].

With the fine-grained decomposition provided by ΦID one can discriminate between different kinds of synergies. In particular, we introduce the downward causation and causal decoupling indices of order k, denoted by and respectively, as (5) (6)

From these definitions and Eq (4), one can verify that (7)

Therefore, the emergence capacity of a system naturally decomposes in two different components: information about k-plets of future variables, and information about future collective properties beyond k-plets. The ΦID atoms that belong to these two terms are illustrated within the ΦID lattice for two time series in Fig 3. The rest of this section shows that and are natural metrics of downward causation and causal decoupling, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Integrated information decomposition (ΦID).

ΦID lattice for n = 2 time series [20], with downward () causation and causal decoupling () terms highlighted.

https://doi.org/10.1371/journal.pcbi.1008289.g003

Practical criteria for large systems

While theoretically appealing, our proposed framework suffers from the challenge of estimating joint probability distributions over many random variables, and the computation of the ΦID atoms themselves. As an alternative, we consider approximation techniques that do not require the adoption of any particular PID or ΦID function and are data-efficient, since they are based on pairwise distributions only.

As practical criteria to measure causal emergence of order k, we introduce the quantities , , and . For simplicity, we write here the special case k = 1, and provide full formulae for arbitrary k and accompanying proofs in S1 Appendix, Section 4: (10a) (10b) (10c)

Our next result links these quantities with the formal definitions in Section A formal theory of causal emergence, showing their value as practical criteria to detect causal emergence.

Proposition 1. is a sufficient condition for V_t to be causally emergent. Similarly, is a sufficient condition for V_t to exhibit downward causation. Finally, and is sufficient for causal decoupling.

Proof. See S1 Appendix, Section 4.

Although calculating whether a system has emergent features via Proposition 1 may be computationally challenging, if one has a candidate feature V one believes may be emergent, one can compute the simple quantities in Eq (10) which depend only on standard mutual information and bivariate marginals, and scales linearly with system size (for k = 1). These quantities are easy to compute and test for significance using standard information-theoretic tools [21, 22]. Moreover, the outcome of these measures is valid for any choice of PID and ΦID that is compatible with the properties specified in S1 Appendix, Section 2.

In a broader context, and belong to the same whole-minus-sum family of measures as the interaction information [14, 23], the redundancy-synergy index [24] and, more recently, the O-information Ω [25]—which cannot measure synergy by itself, but only the balance between synergy and redundancy. In practice, this means that if there is redundancy in the system it will be harder to detect emergence, since redundancy will drive and more negative. Furthermore, by summing all marginal mutual informations (e.g. in the case of ), these measures effectively double-count redundancy up to n times, further penalising the criteria. This problem of double-counting can be avoided if one is willing to commit to a particular PID or ΦID function, as we show next.

It is worth noticing that the value of k can be tuned to explore emergence with respect to different “scales.” For example, k = 1 corresponds to emergence with respect to individual microscopic elements, while k = 2 refers to emergence with respect to all couples—i.e. individual elements and their pairwise interactions. Accordingly, the criteria in Proposition 1 are, in general, harder to satisfy for larger values of k. In addition, from a practical perspective, considering large values of k requires estimating information-theoretic quantities in high-dimensional distributions, which usually requires exponentially larger amounts of data.

Measuring emergence via synergistic channels

This section leverages recent work on information decomposition reported in Ref. [19], and presents a way of directly measuring the emergence capacity and the indices of downward causation and causal decoupling. The key takeaway of this section is that if one adopts a particular ΦID, then it is possible to evaluate and directly, providing a direct route to detect emergence without double-counting redundancy, as the methods introduced in Section Practical criteria for large systems do. Moreover, additional properties may become available due to the characteristics of the particular ΦID chosen.

Let us first introduce the notion of k-synergistic channels: mappings p_V|X that convey information about X but not about any of the parts X^α for all |α| = k. The set of all k-synergistic channels is denoted by (11)

A variable V generated via a k-synergistic channel is said to be a k-synergistic observable.

With this definition, we can consider the k^th-order synergy to be the maximum information extractable from a k-synergistic channel: (12)

This idea can be naturally extended to the case of causal decoupling by requiring synergistic channels at both sides, i.e. (13)

Finally, the downward causation index can be computed from the difference (14)

Note that , which is a direct consequence of the data processing inequality applied on V − X_t − X_t′ − U, and therefore .

By exploiting the properties of this specific way of measuring synergy, one can prove the following result. For this, let us say that a feature V_t is auto-correlated if I(V_t;V_t′)>0.

Proposition 2. If X_t is stationary, all auto-correlated k-synergistic observables are k^th-order emergent.

Proof. See S1 Appendix, Section 4.

In summary, and provide data-driven tools to test—and possibly reject—hypotheses about emergence in scenarios of interest. Efficient algorithms to compute these quantities are discussed in Ref. [26]. Although current implementations allow only relatively small systems, this line of thinking shows that future advances in PID might make the computation of emergence indices more scalable, avoiding the limitations of Eq (10).

Canonical examples of putative emergence

Here we present an evaluation of our practical criteria for emergence (Proposition 1) in two well-known systems: Conway’s Game of Life (GoL) [27], and Reynolds’ flocking boids model [28]. Both are widely regarded as paradigmatic examples of emergent behaviour, and have been thoughtfully studied in the complexity and artificial life literature [29]. Accordingly, we use these models as test cases for our methods. Technical details of the simulations are provided in S1 Appendix, Section 5.

Conway’s Game of Life.

A well-known feature of GoL is the presence of particles: coherent, self-sustaining structures known to be responsible for information transfer and modification [30]. These particles have been the object of extensive study, and detailed taxonomies and classifications exist [29, 31].

To test the emergent properties of particles, we simulate the evolution of 15x15 square cell arrays, which we regard as a binary vector X_t ∈ {0, 1}ⁿ with n = 225. As initial condition, we consider configurations that correspond to a “particle collider” setting, with two particles of known type facing each other (Fig 4). In each trial, the system is randomised by changing the position, type, and relative displacement of the particles. After an intial configuration has been selected, the well-known GoL evolution rule [27] is applied 1000 times, leading to a final state X_t′. Simulations showed that this interval is enough for the system to settle in a stable state after the collision.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Causal emergence in Conway’s Game of Life.

The system is initialised in a “particle collider” setting, and run until a stable configuration is reached after the collision. Using particle type as a supervenient feature V, we find the system meets our practical criterion for causal emergence.

https://doi.org/10.1371/journal.pcbi.1008289.g004

To use the criteria from Eq 10, we need to choose a candidate emergent feature V_t. In this case, we consider a symbolic, discrete-valued vector that encodes the type of particle(s) present in the board. Specifically, we consider , where iff there is a particle of type j at time t—regardless of its position or orientation.

With these variables, we compute the quantities in Eq 10 using Bayesian estimators of mutual information [32]. The result is that, as expected, the criterion for causal emergence is met with . Furthermore, we found that , which is orders of magnitude smaller than I(V_t;V_t′) = 0.99±0.02. Errors represent the standard deviation over surrogate data, as described in S1 Appendix, Section 5. Using Proposition 1, these two results suggest that particle dynamics in GoL may not only be emergent, but causally decoupled with respect to their substrate.

Reynolds’ flocking model.

As a second test case, we consider Reynolds’ model of flocking behaviour. This model is composed by boids (bird-oid objects), with each boid represented by three numbers: its position in 2D space and its heading angle. As candidate feature for emergence, we use the 2D coordinates of the center of mass of the flock, following Seth [6].

In this model boids interact with one another following three rules, each regulated by a scalar parameter [6]:

• aggregation (a₁), as they fly towards the center of the flock;
• avoidance (a₂), as they fly away from their closest neighbour; and
• alignment (a₃), as they align their flight direction to that of their neighbours.

Following Ref. [6], we study small flocks of N = 10 boids with different parameter settings to showcase some properties of our practical criterion of emergence. Note that this study is meant as an illustration of the proposed theory, and not as a thorough exploration of the flocking model, for which a vast literature exists (see e.g. the work of Vicsek [33] and references therein).

Fig 5 shows the results of a parameter sweep over the avoidance parameter, a₂, while keeping a₁ and a₃ fixed. When there is no avoidance, boids orbit around a slowly-moving center of mass, in what could be called an ordered regime. Conversely, for high values of a₂ neighbour repulsion is too strong for lasting flocks to form, and isolated boids spread across the space avoiding one another. For intermediate values, the center of mass traces a smooth trajectory, as flocks form and disintegrate. In line with the findings of Seth [6], our criterion indicates that the flock exhibits causally emergent behaviour in this intermediate range.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Causal emergence in the flocking boids model.

As the avoidance parameter is increased, the flock transitions from an attractive regime (in which all boids orbit regularly around a stable center of mass), to a repulsive one (in which boids spread across space and no flocking is visible). a) Our criterion Ψ detects causal emergence in an intermediate range of the avoidance parameter (error bars represent the standard deviation estimated over surrogate data). b) Sample trajectories of boids (grey) and their center of mass ().

https://doi.org/10.1371/journal.pcbi.1008289.g005

By studying separately the two terms that make up Ψ we found that the criterion of emergence fails for both low and high a₂, but for different reasons (see Fig 5). In effect, for high a₂ the self-predictability of the center of mass (i.e. I(V_t;V_t′)) is low; while for low a₂ it is high, yet lower than the mutual information from individual boids (i.e. ). These results suggest that the low-avoidance scenario is dominated not by a reduction in synergy, but by an increase in redundancy, which effectively increases the synergy threshold needed to detect emergence. However, note that, due to the limitations of the criterion, the fact that is inconclusive and does not rule out the possibility of emergence. This is a common limitation of whole-minus-sum estimators like Ψ; further refinements may provide bounds that are less susceptible to these issues and perform accurately in these scenarios.

Mind from matter: Emergence, behaviour, and neural dynamics

A tantalising outcome of having a formal theory of emergence is the capability of bringing a quantitative angle to the archetype of emergence: the mind-matter relationship [35, 36]. As a first step in this direction, we conclude this section with an application of our emergence criteria to neurophysiological data.

We study simultaneous electrocorticogram (ECoG) and motion capture (MoCap) data of Japanese macaques performing a reaching task [34], obtained from the online Neurotycho database. Note that the MoCap data cannot be assumed to be a supervenient feature of the available ECoG data, since it doesn’t satisfy the conditional independence conditions required by our definition of supervenience (see Section A formal theory of causal emergence). This is likely to be the case, because the neural system is only partially observed—i.e. the ECoG does not capture every source of relevant activity in the macaque’s cortex. Note that non-supervenient features are of limited interest within our framework, as they can satisfy Proposition 1 in trivial ways (e.g. time series which are independent of the underlying system satisty Ψ > 0 if they are auto-correlated). Instead, we focus on the portion of neural activity encoded in the ECoG signal that is relevant to predict the macaque’s behaviour, and conjecture this information to be an emergent feature of the underlying neural activity (Fig 6).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Causal emergence in motor behaviour of an awake macaque monkey.

a) Position of electrocorticogram (ECoG) electrodes used in the recording (in blue) overlaid on an image of the macaque’s left hemisphere (front of the brain towards the top of the page). b) Sample time series from the 64-channel ECoG recordings used, which correspond to the system of interest . c) 3D position of the macaques’s wrist, as measured by motion capture () and as predicted by the regression model (), taken as a supervenient feature . d) Our emergence criterion yields , detecting causal emergence of the behaviour with respect to the ECoG sources. Original data and image from Ref. [34] and the Neurotycho database.

https://doi.org/10.1371/journal.pcbi.1008289.g006

To test this hypothesis, we take the neural activity (as measured by 64 ECoG channels distributed across the left hemisphere) to be the system of interest, and consider a memoryless predictor of the 3D coordinates of the macaque’s right wrist based on the ECoG signal. Therefore, in this scenario and . To build V_t, we used Partial Least Squares (PLS) and a Support Vector Machine (SVM) regressor, the details of which can be found in S1 Appendix, Section 6.

After training the decoder and evaluating on a held-out test set, results show that Ψ > 0, confirming our conjecture that the motor-related information is an emergent feature of the macaque’s cortical activity. For short timescales (t′ − t = 8 ms), we find , which is orders of magnitude smaller than , suggesting that the behaviour may have an important component decoupled from individual ECoG channels (errors are standard deviations estimated over time-shuffled data). Furthermore, the emergence criterion is met for multiple timescales t′ − t of up to ≈0.2s, beyond which the predictive power in V_t and individual electrodes decrease and become nearly identical.

As a control, we performed a surrogate data test to confirm the results in Fig 6 were not driven by the autocorrelation in the ECoG time series. To this end, we re-run the analysis (including training and testing the PLS-SVM) using the same ECoG data, but time-shuffling the wrist position—resulting in a V_t that does not extract any meaningful information from the ECoG, but has the same properties induced by autocorrelation, filtering and regularisation. As expected, the resulting surrogate is significantly lower than the one using the un-shuffled wrist position, confirming the measured is positive and higher than what would be expected from a similar, random projection of the ECoG (details in S1 Appendix, Section 6).

This analysis, while just a proof of concept, helps us quantify how and to what extent behaviour emerges from collective neural activity; and opens the door to further tests and quantitative empirical explorations of the mind-matter relationship.

Scope of the theory

Our theory focuses on synchronic [38] aspects of emergence, analysing the interactions between the elements of dynamical systems and collective properties of them as they jointly evolve over time. As such, our theory directly applies to any system with well-defined dynamics, including systems described by deterministic dynamical systems with random initial conditions [11] and stochastic systems described by Fokker-Planck equations [39]. In contrast, the application of our theory to systems in thermodynamic equilibrium may not be straightforward, as their dynamics are often not uniquely specified by the corresponding Gibbs distributions (for an explicit example, when considering the Ising model, Kawasaki and Glauber dynamics are known to behave differently even when the system is in equilibrium [40]; and thus may provide quite different values of the measures described in Section Measuring emergence). Finding principled approaches to guide the application of our theory to those cases is an interesting challenge for future studies.

In addition, given the breadth of the concept of “emergence,” there are a number of other theories leaning more towards philosophy that are orthogonal to our framework. This includes, for example, theories of emergence as radical novelty (in the sense of features not previously observed in the system) [41], most prominently encapsulated in the aphorism “more is different” by Anderson—see Refs. [42, 43], particularly his approach to emergence in biology (note that some of Anderson’s views, particularly the ones related to rigidity, are nevertheless closely related to the approach developed by our framework), and also articulated in the work of Kauffman [44, 45]. Also, contextual emergence emphasises a role for macro-level contexts that cannot be described at the micro-level, but which impose constraints on the micro-level for the emergence of the macro [46, 47]. These are valuable philosophical positions, which have been studied from a statistical mechanics perspective in Ref. [46, 48]. Future work shall attempt to unify these other approaches with our proposed framework.

Causality

The de facto way to assess the causal structure of a system is to analyse its response to controlled interventions or to build intervention models (causal graphs) based on expert knowledge, which leads to the well-known do-calculus spearheaded by Judea Pearl [10]. This approach is, unfortunately, not applicable in many scenarios of interest, as interventions may incur prohibitive costs or even be impossible, and expert knowledge may not be available. These scenarios can still be assessed via the Wiener-Granger theory of statistical causation, which studies the blueprint of predictive power across the system of interest by accounting non-mediated correlations between past and future events [12]. Both frameworks provide similar results when all the relevant variables have been measured, but can neverthelss differ radically when there are unobserved interacting variables [10]. The debate between the Wiener-Granger and the Pearl schools has been discussed in other related contexts—see e.g. Refs. [49, 50] for a discussion regarding Integrated Information Theory (IIT), and Ref. [51] for a discussion about effective and functional connectivity in the context of neuroimaging time series analysis (in a nutshell, effective connectivity aims to uncover the minimal physical causal mechanism underlying the observed data, while functional connectivity describes directed or undirected statistical dependences [51]).

In our theory, the main object of analysis is Shannon’s mutual information, I(X_t;X_t′), which depends on the joint probability distribution . The origin of this distribution (whether it was obtained by passive observation or by active intervention) will change the interpretation of the quantities presented above, and will speak differently to the Pearl and the Wiener-Granger schools of thought; some of the implications of these differences are addressed when discussing Ref. [7] below. Nonetheless, since both methods of obtaining allow synergy to take place, our results are in principle applicable in both frameworks—which allows us to formulate our theory of causal emergence without taking a rigid stance on a theory of causality itself.

Relationship with other quantitative theories of emergence

This work is part of a broader movement towards formalising theories of complexity through information theory. In particular, our framework is most directly inspired by the work of Seth [6] and Hoel et al. [7], and also related to recent work by Chang et al. [52]. This section gives a brief account of these theories, and discusses how they differ from our proposal.

Seth [6] proposes that a process V_t is Granger-emergent (or G-emergent) with respect to X_t if two conditions are met: (i) V_t is autonomous with respect to X_t (i.e. I(V_t;V_t′|X_t)>0), and (ii) V_t is G-caused by X_t (i.e. I(X_t;V_t′|V_t)>0). The latter condition is employed to guarantee a relationship between X_t and V_t; in our framework an equivalent role is taken by the requirement of supervenience. The condition of autonomy is certainly related with our notion of causal decoupling. However, as shown in Ref. [14], the conditional mutual information conflates unique and synergistic information, which can give rise to undesirable situations: for example, it could be that I(V_t;V_t′|X_t)>0 while, at the same time, I(V;V_t′) = 0, meaning that the dynamics of the feature V_t are only visible when considering it together with the full system X_t, but not on its own. Our framework avoids this problem by refining this criterion via PID, and uses only the unique information for the definition of emergence.

Our work is also strongly influenced by the framework put forward by Hoel and colleagues in Ref. [7]. Their approach is based on a coarse-graining function F(⋅) relating a feature of interest to the system, V_t = F(X_t), which is a particular case of our more general definition of supervenience. Emergence is declared when the dependency between V_t and V_t′ is “stronger” than the one between X_t and X_t′. Note that V_t − X_t − X_t′ − V_t′ is a Markov chain, and hence I(V_t;V_t′)≤I(X_t;X_t′) due to the data processing inequality; therefore, a direct usage of Shannon’s mutual information would make the above criterion impossible to fulfill. Instead, this framework focuses on the transition probabilities and , and hence the mutual information terms are computed using maximum entropy distributions instead of the stationary marginals. By doing this, Hoel et al. account not for what the system actually does, but for all the potential transitions the system could do. However, in our view this approach is not well-suited to assess dynamical systems, as it might account for transitions that are never actually explored. The difference between stationary and maximum entropy distributions can be particularly dramatic in non-ergodic systems with multiple attractors—for a related discussion in the context of Integrated Information Theory, see Ref. [50]. Additionally, this framework relies on having exact knowledge about the microscopic transitions as encoded by , which is not possible to obtain in most applications.

Finally, Chang et al. [4] consider supervenient variables that are “non-trivially informationally closed” (NTIC) to their corresponding microscopic substrate. NTIC is based on a division of X_t into a subsystem of interest, , and its “environment” given by . Interestingly, a system being NTIC requires V_t to be supervenient only with respect to (i.e. ), as well as information flow from the environment to the feature (i.e. ) mediated by the feature itself, so that X_t − V_t − V_t′ is a Markov chain. Hence, NTIC requires features that are sufficient statistics for their own dynamics, which is akin to our concept of causal decoupling but focused on the interaction between a macroscopic feature, an agent, and its environment. Extending our framework to agent-environment systems involved in active inference is part of our future work.

Limitations and open problems

The framework presented in this paper focuses on features from fully observable systems with Markovian dynamics. These assumptions, however, often do not hold when dealing with experimental data—especially in biological and social systems. As an important extension, future work should investigate the effect of unobserved variables on our measures. This could be done, for example, leveraging Takens’ embedding theorem [53–55] or other methods [56].

An interesting feature of our framework is that, although it depends on the choice of PID and ΦID, its practical application via the criteria discussed in Section Practical criteria for large systems is agnostic to those choices. However, they incur the cost of a limited sensitivity to detect emergence due to an overestimation of the microscopic redundancy; so they can detect emergence when it is substantial, but might miss it in more subtle cases. Additionally, these criteria are unable to rule out emergence, as they are sufficient but not necessary conditions. Therefore, another avenue of future work should search for improved practical criteria for detecting emergence from data. One interesting line of research is providing scalable approximations for and as introduced in Section Measuring emergence via synergistic channels, which could be computed in large systems.

Another open question is how the emergence capacity is affected by changes in the microscopic partition of the system (c.f. Section Defining causal emergence). Interesting applications of this includes scenarios where elements of interest have been subject to a mixing process, such as the case of electroencephalography where each electrode detects a mixture of brain sources. Other interesting questions include studying systems with non-zero emergence capacity for all reasonable microscopic partitions, which may correspond to a stronger type of emergence.