Interpreting evolution as a Markov process

Consider two random vectors x₀ and x₁ whose joint probability distribution is p(x₀, x₁). We will interpret them as the state of a time-dependent system x(t) at two separate times t₀ and t₁. For example, if these are two vectors of 5 bits each, then p is a table of 2¹⁰ numbers giving the probability of each possible bit string, while if these are two vectors in 3D space, then p is a function of 6 real continuous variables. We obtain the marginal distribution p⁽ⁿ⁾(x_n) for the n^th vector, where n = 0 or n = 1, by summing/integrating p over the other vector.

Below we will often find it convenient to denote these vectors as single indices i = x₀ and j = x₁. For example, this allows us to write the marginal distribution p₀(x₀) as ∑_j p_ij, where the sum over j is to be interpreted as summation/integration over all allowed values of x₁. We also adopt the notation where replacing an index by a dot means that this index is to be summed/integrated over. This lets us write the marginal distributions p⁽⁰⁾(x₀) and p⁽¹⁾(x₁) as (1)

As illustrated in Fig 1, it is always possible to model this relation between x₀ and x₁ as resulting from a Markov process, where x₁ is causally determined by a combination of x₀ and random effects. If we write the marginal distributions from eq (1) as vectors p⁽⁰⁾ and p⁽¹⁾, this Markov process is defined by (2) where the Markov matrix M_ji specifies the probability that a state i transitions to a state j, and satisfies the conditions M_ji ≥ 0 (non-negative transition probabilities) and M_⋅i = 1 (unit column sums, guaranteeing probability conservation). The standard rule for conditional probabilities gives (3) which uniquely determines the Markov matrix as (4) which is seen to satisfy the Markov requirements M_ji ≥ 0 and M_⋅i = 1.

Note that any system obeying the laws of classical physics can be accurately modeled as a Markov process as long as the time step Δt ≡ t₁ − t₀ is sufficiently short (defining x(t) as the position in phase space). If the process has “memory” such that the next state depends not only on the current state but also on some finite number of past states, it can reformulated as a standard memoryless Markov process by simply expanding the definition of the state x to include elements of the past.

Also note that although full knowledge of the Markov matrix M completely specifies the dynamics of the system, a person wishing to compute its integration may not know M exactly. If M is not known from having built the system or having examined its inner workings, then passively observing it in action (without active interventions) may not provide enough information to fully reconstruct M [21]. The n → ∞ limit of continuous variables describes a convenient class of systems where M is relatively easy to determine in practice.

A taxonomy of integration measures

We will now see that this Markov process interpretation allows us create a simple taxonomy of integration measures ϕ that quantify the interaction between two subsystems. The idea is to approximate the Markov process by a separable Markov process that does not mix information between subsystems, and to define the integration as a measure of how bad the best such approximation is. Consider the system x as being composed of two subsystems x^A and x^B, so that the elements of the vector x are simply the union of the elements of x^A and x^B, and let us define the probability distribution (5) (For brevity, we will sometimes refer to this distribution p_{ii′ jj′} as simply p below, suppressing the indices, and we will sometimes write x without indices to refer to the full state at both times.) The Markov matrix of eq (4) then takes the form (6) The Markov process of eq (2) is separable if the Markov matrix M is a tensor product M^A ⊗ M^B, i.e., if (7) for Markov matrices M^A and M^B that determine the evolution of x^A and x^B.

If our system is integrated so that M cannot be factored as in eq (7), we can nonetheless choose to approximate M by a matrix of the factorizable form M^A ⊗ M^B. If we retain the initial probability distribution p_ii′⋅⋅ for x₀ but replace the correct Markov matrix M by the separable approximation M^A ⊗ M^B, then eq (6) shows that the probability distribution (8) gets replaced by the probability distribution q_{ii′ jj′} given by (9) which is an approximation of p_{ii′ jj′}. If M is factorizable (meaning that there is no integration), we can factor M such that the two probability distributions q_{ii′ jj′} and p_{ii′ jj′} are equal and, conversely, if the two probability distributions are different, we can use how different they are as an integration measure ϕ.

To define an integration measure ϕ in this spirit, we thus need to make four different choices, which collectively specify it fully and determine where the ϕ-measure belongs in our taxonomy:

1. Choose a recipe defining an approximate factorization M ≈ M^A ⊗ M^B.
2. Choose which probability distributions p and q to compare for exact and approximate M (the distribution for x, x₁ or , say).
3. Choose what to treat as known about p_ii′⋅⋅ when computing these probability distributions.
4. Choose a metric for how different the two probability distributions p and q are.

These four options are described in Tables 3, 4 and 5, and we will now explore them in detail.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Different options for approximate factorizations M ≈ M^A ⊗ M^B and .

These options correspond to the first superscript in ϕ-measures such as ϕ^ofuk. The optimal factorizations maximize the accuracy of the approximate probability distribution that they predict, while the “noising” factorizations are instead defined by treating the input from the other subsystem as random noise, either with uniform distribution (option “n”) or with the observed marginal distribution (option “m”).

https://doi.org/10.1371/journal.pcbi.1005123.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Different options for which probability distributions p and q to compare, corresponding to the second and third superscripts in ϕ-measures such as ϕ^ofuk.

The last three columns specify the formula for q for the three conditioning options we consider: when the state x₀ is unknown (u), has a separable probability distribution (s) and is known (k), respectively.

https://doi.org/10.1371/journal.pcbi.1005123.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Different options for measuring the difference d between two probability distributions p and q: Kullback-Leibler divergence d_KL, L₁-norm d₁, L₂-norm d₂, Hilbert-space distance d_H, Shannon-Jensen distance d_SJ, Earth-Movers distance d_EM and Mismatched Decoding distance d_MD.

These options correspond to the fourth superscript in ϕ-measures such as ϕ^ofuk. In the text, we considered options where p and q had one, two or four indices, but in this table, we have for simplicity combined all indices into a single Greek index α.

https://doi.org/10.1371/journal.pcbi.1005123.t005

Options for approximately factoring M

Table 3 lists five factoring options which all have attractive features, and we will now describe each in turn.

Options for which probability distributions to compare

Table 4 lists four options for which probability distributions p and q to compare. Arguably the most natural option is to simply compare the full distributions p_ii′jj′ and q_ii′jj′ that describe our knowledge of the system at both times (the present state and the future state). Another obvious option is to merely compare the predictions, i.e., the probability distributions p_⋅⋅jj′ and q_⋅⋅jj′ for the future state. A third interesting option is to compare merely the predictions for one of the two subsystems (which we without loss of generality can take to be subsystem A), thus comparing p_⋅⋅j⋅ and q_⋅⋅j⋅.

Generally, the less we compare, the easier it is to get a low ϕ-value. To see this, consider a system where A affects B but B has no effect on A. We could, for example, consider A to be photoreceptor cells in your retina and B to be the rest of your brain. Then the second comparison option (“f”) in Table 4 would give ϕ > 0 because we predict the future of your brain worse if we ignore the information flow from your retina, while the third comparison option (“a”) in the table would give ϕ = 0 because the rest of your brain does not help predict the future of your retina. In other words, comparison option “a” makes ϕ vanish for afferent pathways, where information flows only inward toward the rest of the system.

IIT argues that any good ϕ-measure indeed should vanish for afferent pathways, because a system can only be conscious if it can have effects on itself—other systems that it is affected by without affecting will act merely as parts of its unconscious outside world [10]. Analogously, IIT argues that any good ϕ-measure should vanish also for efferent pathways, where information flows only outward away from the rest of the system. The argument is that other systems that the conscious system affects without being affected by will again be unconscious, acting merely as unconscious parts of the outside world as far as the conscious system is concerned.

Option “p” in Table 4 has this property of ϕ vanishing for efferent pathways. It is simply the time-reverse of option “a”, quantifying the ability of to determine its past cause instead of quantifying the ability of to determine its future effect .

To formalize this, consider that there is nothing in the probability distribution p_ii′jj′ that breaks time-reversal symmetry and says that we must interpret causation as going from t₀ to t₁ rather than vice versa. In complete analogy with our formalism above, we can therefore define a time-reversed Markov process whereby the future determines the past according to the time-reverse of eq (2): (15) where eqs (6) and (9) get replaced by (16) and (17) This time reversal symmetry doubles the number of q-options we could list in Table 4 to six in total, augmenting q_ii′jj′, q_⋅⋅jj′ and q_⋅⋅j⋅ by , and . In the interest of brevity, we have chosen to only list , because of its ability to kill ϕ for efferent pathways—the formulas for the two omitted options are trivially analogous to those listed.

Options for what to treat as known about the current state

Above we listed options for which probabilities p and q to compare to compute ϕ. To complete our specification of these probabilities, we need to choose between various options for our knowledge of the present state; the three rightmost columns of Table 4 correspond to three interesting choices.

The first option is where the state is unknown, described simply by the probability distribution we have used above: (18) This corresponds to us knowing M, the mechanism by which the state evolves, but not knowing its current state x₀. Note that a generic Markov process eventually converges to a unique stationary state p = p⁽⁰⁾ = p⁽¹⁾ which, since it satisfies Mp = p, can be computed directly from M as the unique eigenvector whose eigenvalue is unity (the only Markov processes that do not converge to a unique steady state are ones where M has more than one eigenvalue equal to unity; these form a set of measure zero on the set of all Markov processes). This means that if we consider a system that has been evolving for a significantly long time, its full two-time distribution p_ii′jj′ is determined by M alone; conversely, p_ii′jj′ determines M through eq (6). Alternatively, if p_ii′jj′ is measured empirically from a time-series x_t which is then used to compute M, we can use eq (18) to describe our knowledge of the state at a random time.

A second option is to assume that we know the initial probability distributions for and , but know nothing about any correlations between them. This corresponds to replacing eq (18) by the separable distribution (19) and can be advantageous for ϕ-measures that would conflate integration with initial correlations between the subsystems.

A third option, advocated by IIT [10], is to treat the current state as known: (20) i.e., we know with certainty that the current state x₀ = kk′ for some constants k and k′. IIT argues that this is the correct option from the vantage point of a conscious system which, by definition, knows its own state.

A natural fourth option is a more extreme version of the first: treating the state not merely as unknown, with p⁽⁰⁾ given by its ensemble distribution, but completely unknown, with a uniform distribution: (21) Although straightforward enough to use in our formulas, we have chosen not to include this option in Table 4 because it is rather inappropriate for most physical systems. For continuous variables such as voltages, it becomes undefined. For brains, such maximum-entropy states never occur: they would have typical neurons firing about half the time, corresponding to much more extreme “on” behavior than during an epileptic seizure. The related option of consistently treating as known but as unknown when predicting (and vice versa when predicting ) corresponds to the noising factorization options described above. For further discussion of this, including so-called “noising at the connection”, see [11, 14, 22].

Finally, please note that if we choose to determine the past rather than the future (the “p”-option from the previous section and Table 4), then all the choices we have described should be applied to rather than .

Options for comparing probability distributions

The options in the past three sections uniquely specify two probability distributions p and q, and we want the integration ϕ to quantify how different they are from one another: (22) for some distance measure d that is larger the worse q approximates p. There are a number of properties that we may consider desirable for d to quantify integration:

1. Positivity: d(p, q) ≥ 0, with equality if and only if p = q.
2. Monotonicity: The more different q is from p in some intuitive sense, the larger d(p, q) gets.
3. Interpretability: d(p, q) can be intuitively interpreted, for example in terms of information theory.
4. Tractability: d(p, q) is easy to compute numerically. Ideally, the optimal factorizations can be found analytically rather than through time-consuming numerical minimization.
5. Symmetry: d(p, q) = d(q, p).

Any distance measure d meets the mathematical requirements of being a metric on the space of probability distributions if it obeys positivity, symmetry and the triangle inequality d(p, q) ≤ d(p, r) + d(r, q).

Table 5 lists seven interesting probability distribution distance measures d (p, q) from the literature together with their definitions and properties. All these measures are seen to have the positivity and monotonicity, and all except the first are also symmetric and true metrics. We will now discuss them one by one in greater detail.

The distance d_KL is the Kullback-Leibler divergence, and measures how many bits of information are lost when q is used to approximate p, in the sense that if you developed an optimal data compression algorithm to compress data drawn from a probability distribution q, it would on average require d_KL(p, q) more bits to compress data drawn from a probability distribution p than if the algorithm had been optimized for p [23]. This has been argued to be the be the best measure because of its desirable properties related to information geometry [24, 25].

d₁ and d₂ measure the distance between the vectors p and q using the L₁-norm and L₂-norm, respectively. The former is particularly natural for probability distributions p, since they all have L₁ norm of unity: d₁ (0, p) = p. = 1. It is easy to see that 0 ≤ d₁(p, q) ≤ 2 and .

The measure d_H is the Hilbert-space distance: if, for each probability distribution, we define a corresponding wavefunction , then all wavefunctions lie on a unit hypersphere since they all have unit length: 〈ψ|ψ〉 = p. = 1. The distance d_H is simply the angle between two wavefunctions, i.e., the distance along the great circle on the hypersphere that connects the two, so d_H(p, q) ≤ π/2. It is also the geodesic distance of the Fisher metric, hence a natural “coordinate free” distance measure on the manifold of all probability distributions.

The measure d_SJ is the Shannon-Jensen distance, whose square is defined as the average of the KL-divergences of the two distributions to their average: (23) It is bounded by 0 ≤ d_SJ ≤ 1, satisfies the triangle inequality and is information-theoretically motivated [26].

The measure d_EM is the Earth-Movers distance [27]. If we imagine piles of earth scattered across the space x, with p(x) specifying the fraction of the earth that is in each location, then d_EM is the average distance that you need to move earth to turn the distribution p(x) into q(x). The quantity d_ij in the definition in Table 5 specifies the distance between points i and j in this space. For example, if x is a 3D Euclidean space, this may be chosen to be simply the Euclidean metric, while if x is a bit string, d_ij may be chosen to be the L₁ “Manhattan distance”, i.e., the number of bit flips required to transform one bit string into another. IIT 3.0 argues that the earth mover’s distance d_EM is the most appropriate measure d on conceptual grounds (whereas IIT 2.0 was still implemented using d_KL). Unfortunately, d_EM rates poorly on the tractability criterion. It’s definition involves a linear programming problem which needs to be solved numerically, and even with the fastest algorithms currently available, the computation grows faster than quadratically with the number of system states—which in turn grows exponentially with the number of bits. For continuous variables x, the number of states and hence the computational time is formally infinite.

The measure d_MD is based on “mismatched decoding” as advocated by [15]. The distance measure d_MD is defined not for all probability distributions, but for all distributions over two variables, which we can write with two indices as p_ij: (24) where (25) and the conditional distribution q_j|i ≡ q_ij/q_i⋅. Here I(p) is simply the mutual information between the two variables, since combining eq (34) with the conditional entropy definition from eq (37) gives the well-known equivalent expression for mutual information (26) I*(p, q, β) can be interpreted as the amount of information that one variable predicts about the other if the correct conditional distribution p_j|i is replaced by a possibly incorrect one (renormalized to sum to unity) when making the prediction [28]. This renormalization is strictly speaking unnecessary, because it cancels out between the two terms in I*(p, q, β). Raising probabilities to positive powers β has the effect of concentrating them (decreasing entropy) if β > 1 and spreading them more evenly (increasing entropy) if β < 1. It can be shown that I*(p, q, β) ≤ I(p) with equality for q = p and β = 1, and that I*(p, q, β) ≥ 0, so one always has 0 ≤ d_MD(p, q) ≤ I(p) [28]. Mismatched decoding can presumably be further generalized by replacing the maximization over powers p^β by maximization over arbitrary monotonically increasing functions f(p) that map the unit interval onto itself.

The integration measures of IIT3.0 have a more complex probability comparison that cannot be fully cast in the form of a simple function of d(p, q): it makes the metric choice d(p, q) = d_EM(p, q), but considers not only probability distributions for the whole system and a bipartition, but also for all possible subsets, providing an elaborate interpretation of the results in terms of “conceptual structures” [11].

Optimal factorization with d_KL

Our taxonomy of integration measures is determined by four choices: of factorization, variable selection, conditioning and distance measure. Although we have now explored these four choices one at a time, there are important interplays between them that we must examine. First of all, the three optimal factorization options in Table 3 depend on what is being optimized, so let us now explore which of these optimizations are feasible and interesting to perform in practice and let us find out what the corresponding factorizations and ϕ-measures are.

The mathematics problem we wish to solve is (27) i.e., minimizing d(p, q) over M^A and M^B given the constraints that M^A and M^B are markov Matrices: , , and . Table 4 specifies the options for how p and q are computed and how q depends on M^A and M^B, while Table 5 specifies the options for computing the distance measure d. We enforce the column sum constraints using Lagrange multipliers, minimizing (28) and need to check afterwards that all elements of M^A and M^B come out to be non-negative (we will see that this is indeed the case).

As mentioned, numerical tractability is a key issue for integration measures. This means that it is valuable if the Lagrange minimization can be rapidly solved analytically rather than slowly by numerical means, since this needs to be done separately for large numbers of possible system partitions. There is only one d-option out of the above-mentioned five for which I have been able to solve the optimization over M-factorizations analytically: the KL-divergence d_KL. The runner-up for tractability is d₂, for which everything can be easily solved analytically except for a final column normalization step, but the resulting formulas are cumbersome and unilluminating, falling foul of the interpretability criterion. Although d_KL lacks the symmetry property, it has the above-mentioned positivity, monotonicity and interpretability properties, and we will now show that it also has the tractability property.

Let us begin with the q-options in the upper left corner of Table 4, i.e., comparing the two-time distributions treating the present state as unknown. Substituting eq (9) into the definition of d_KL from Table 5 gives (29) where the entropy for a random variable x with probability distribution p is given by Shannon’s formula [29] (30) To avoid a profusion of notation, we will often write as the argument of S a random variable rather than its probability distribution. For convenience, we will take all logarithms to be in base 2 for discrete distributions (so that entropies are measured in units of bits) and in base e for continuous Gaussian distributions (so that equations get simpler). In the latter case, where the entropy is based on the natural logarithm, entropy is measured in “nits” or “nats” which equal 1/ln2 ≈ 1.44 bits.

Substituting eq (29) into eq (28) and requiring vanishing derivatives with respect to , , λ_j and μ_j′ shows that the solution to our minimization problem is (31) We recognize these equations as simply the Markov matrix estimator from eq (4) applied separately to subsystems A and B after marginalizing over the other system. Substituting this back into eq (9) gives (32) Although the full probability distributions q and p typically differ, eq (32) implies that three marginal distributions are identical: q_i⋅j⋅ = p_i⋅j⋅, q_{⋅i′⋅j′} = p_{⋅i′⋅j′} and q_ij′⋅⋅ = p_ij′⋅⋅.

Substituting eq (32) back into the definition of d_KL gives the extremely simple result that the integration is (33) where the mutual information between two random variables is given in terms of entropies by the standard definition (34)

Since we will be deriving a large number of different ϕ-measures that we do not wish to conflate with one another, we superscript each one with four code letters denoting the four taxonomical choices that define it. These letter codes are

1. factorization: n/m/o/x/a
2. comparison: t/f/a/p
3. conditioning: u/s/k
4. measure: k/1/2/h/s/e/m

and are defined in Tables 3, 4 and 5. For example, the integration measure ϕ^otuk from eq (33) denotes optimized (o) factorization comparing the two-time (t) probability distributions with the current state unknown (u) and KL-divergence (k). Almost all measures discussed below will involve the k-measure (KL-divergence), so when this is the case we will typically drop this last index k to avoid a confusing profusion of indices, for example writing ϕ^otuk = ϕ^otu. For brevity, we will also define ϕ^M ≡ ϕ^otu, since we will be referring to this “Markov measure” ϕ^otu many times below.

Although we derived this optimal factorization by comparing the two-time distribution (option t) for an unknown state (option u), an analogous calculation leads to the exact same optimal factorization for the options a+u, s+f and a+s. The option t+s is undefined and the option f+u gives messy equations I have been unable to solve analytically. It is therefore reasonable to view eq (31) as the optimal factorization when the state is unknown (option o), and for the remainder of this paper, we will simply define the o-option as using the factorization given by eq (31).

Note that our result in eq (33) involves a time-asymmetry, singling out t₀ rather than t₁ in the second term. This is because we chose to interpret our Markov process as operating forward in time, determining the state at t₁ from the state at t₀. As we discussed in the previous section, we could equally well have done the opposite, using the Markov process operating backward in time, which would have yielded the alternative integration measure (35) In practice, one usually estimates all statistical properties from a time-series that is assumed to be stationary. This means that , so that the these two integration measures become identical.

Comparison with the Ay/Barrett/Seth integration measures

In the paper [13] where Barrett & Seth proposed their easier-to-compute integration measure ϕ^B (see below), they also mentioned an alternative measure that they termed , defined by (36) where the conditional entropy of two variables A and B is defined by (37) This measure had been introduced earlier by Ay [30, 31] in a context unrelated to IIT, under the name “stochastic interaction”, and was further discussed in [15, 32]. Applying eqs (37) and (34) to eq (36) shows that (38) i.e., that is identical to the time-reversed Markov measure . This equivalence provides another convenient interpretation of : as the average KL-divergence between (i) the probability distribution of the past state x₀ given the present state x₁ and (ii) the product of these conditional distributions for the two subsystems.

It is also interesting to compare our result in eq (33) with the popular integration measure (39) proposed by Barrett & Seth [13]. The intuition behind this definition is to take the amount of information that a system predicts about its future and subtract of the information predicted by both of its subsystems. Unfortunately, the result can sometimes go negative [14, 15], violating the desirable positivity property and making the ϕ_B difficult to interpret. Consider the simple example of two independent bits that never change. If they start out perfectly correlated, then they will remain perfectly correlated, giving and integrated information ϕ^B(p) = −1.

By substituting eq (34) into eqs (33) and (39), we find that (40) In other words, we can make the Barrett-Seth measure non-negative by adding back any final mutual information between the two subsystems. When this is done, it becomes the integration measure we derived, therefore having a simple information-theoretic interpretation: it is the KL-divergence between the actual probability distribution p and the best separable approximation, which is guaranteed to be non-negative.

Comparison with the mismatched decoding integration measure

The measure ϕ^M is also closely related to the mismatched decoding measure ϕ^MD introduced in [15]. ϕ^MD makes the same taxonomical choices “otu” as ϕ^M for the first three options: optimal factorization (o), comparing full two-time distributions (t), and treating the past state as unknown (u). However, it uses probability distance measure “m” (mismatched decoding d_MD) instead of KL-divergence. We can therefore write this measure in our notation as ϕ^otum = d_MD(p, q), where q is the optimal factorization given by eq (32). Whether this factorization is also optimal in the sense of minimizing d_MD(p, q) is not obvious.

The measure ϕ^M (or more specifically its time-reverse ) has been criticized in [15, 25] for being able to exceed the mutual information I(x₀, x₁) between the past and present: for example, if a two-bit system evolves from “00” to either “00” or “11” with equal probability, then bit, even though I(x₀, x₁) = 0. This means that ϕ^M counts as a contribution to integration also correlated random noise added to both subsystems. It is debatable whether this should count as integration: the “con” argument is that no information flows between the subsystems, while the “pro” argument is that the two subsystems get linked by shared information flowing into both of them.

Both ϕ^M and ϕ^MD have intuitive bounds: 0 ≤ ϕ^M ≤ I(x^A, x^B) and 0 ≤ ϕ^MD ≤ I(x₀, x₁); these upper bounds correspond to the total mutual information across space and time, respectively.

Optimal state-dependent factorization

Let us now turn to factorization option “x”, optimized knowing the current state. Consider some conscious observer (perhaps the system itself) who knows nothing about the system except its dynamics (encoded in M) and its state at the present instant, encoded in x₀ = kk′. What can this observer say about the system state at earlier and later times? How integrated will this observer feel that the system is? To answer this question, we simply want to find the best approximate factorization of the conditional future state M_jj′kk′ (or the past state M_kk′ii′), where k and k′ are known constants.

To gain intuition for this, let us temporarily write this conditional distribution as p_ii′, suppressing the known parameters kk′ for simplicity. Given an arbitrary bivariate probability distribution p_ii′, what is best separarable approximation q_ii′ ≡ a_i b_i′ in the sense that it minimizes d_KL (p, q)? By minimizing d_KL (p, q) using Lagrange multipliers, one easily obtains the long-known result that a_i = p_i., b_i′ = p_.i′ and d_KL (p, q) = I, the mutual information of p. In other words, even if we had never heard of marginal distributions or mutual information, we could derive them all from d_KL: the best factorization simply uses the marginal distributions, and the mutual information of a bivariate distribution is simply the KL-measure of how non-separable it is.

This means that the optimal factorization given k and k′ is simply the one giving the marginal conditional distributions (41) and the corresponding integration is simply (42) ϕ^xtkk is identical. We can alternatively obtain this result directly from eq (33) by noting that the -term vanishes now that the state x₀ is known.

This result highlights a striking and arguably undesirable feature of measures based on the x-factorization option: they vanish for any deterministic system! If the system is deterministic and the present state x₀ is known, then the future state x₁ is also known, so all entropies in eq (42) vanish and we obtain ϕ = 0. With ϕ-measures based on x-factorization, the only source of integration is therefore correlated noise generated by the system.

Minimizing integration on average

Let us now turn to our final factorization option, “a”, where we pick the state-independent factorization that minimizes integration on average. Given the present state x₀ = kk′, let us compare the exact and approximate future probability distributions (43) by computing their KL-divergence ϕ = d_KL(p, q). The answer clearly depends on the present state kk′, and we saw in the previous section what happens when we minimize separately for each state kk′. Let us now instead average d_KL(p, q) over all current states and find the state-independent factorization that minimizes this average: (44) Substituting eq (6) shows that this expression is identical to that from eq (29), so minimizing it gives the exact same optimal factors M^A and M^B and the exact same minimum ϕ. The comparison option “t” gives the same result as well, so in conclusion, although they appear quite different from their definitions, the factorization options “o” and “a” are in fact identical.

The full taxonomy

Now that we have derived the explicit form of all our factorization options, we can complete our integration measure classification. Our taxonomy is determined by four choices: of factorization (n/m/o/x/a), variable selection (t/f/a/p), conditioning (u/s/k) and distance measure (k/1/2/h/s/e/m). Although this nominally gives 5 × 4 × 3 × 7 = 420 different integration measures, most of these options turn out to be zero, undefined or identical to other options. For noising factorizations (factorization options n and m), subsystem B is randomized, so the only well-defined options are ϕ^nas*, ϕ^nak*, ϕ^nps*, ϕ^npk*, ϕ^mas*, ϕ^mak*, ϕ^mps* and ϕ^mpk*, where * denotes any option for the distance measure. For o-factorization, we find that ϕ^oau* = ϕ^oas* = ϕ^opu* = ϕ^ops* = 0 and ϕ^otk* = ϕ^ofk*. For x-factorization, ϕ^xt** is undefined and one easily shows that ϕ^xak* = ϕ^xpk* = 0, ϕ^xau* = ϕ^xas* and ϕ^xpu* = ϕ^xps*. We interpret k-conditioning as x₀ being known for o-factorization and as being known for noising factorizations, since the reverse options vanish and are undefined, respectively.

Whereas there are strong interactions between the factorization, variable selection and conditioning, we can freely choose any of the 7 distance measures independently of the other choices without changing whether ϕ vanishes or is well-defined. We consider the option k (KL-divergence) by default below since it results in the simplest and most intuitive formulas; the formulas for the other options are straightforward to derive by combining Tables 3, 4 and 5. This leaves us with only the 21 separate options shown in Table 2 to consider. To provide intuition for these formulas, let us recapitulate key definitions in words:

• ϕ^M is the KL-divergence of the two-state probability distribution and the best separable approximation.
• ϕ^MD is a measure of how much less information the present gives about the past if factorized dynamics is assumed.
• is the KL divergence between (i) the future of the whole given the specific present state of the whole, and (ii) the product of this for the parts calculated separately.
• ϕ^oak is the KL divergence between (i) the distribution for the future state of subset A given the current state of A and (ii) the distribution for the future state of subset A given the current state of the whole system.
• ϕ^opk is ϕ^oak swapping “future” for “past”.
• The subsequent ones are versions from above with different factorizations applied.

Which integration measures are best?

Table 1 summarizes the desirable and undesirable traits for each of these integration measures, showing that merely a handful lack any major drawbacks. Let us now rate the various options in more detail.

For the choice of probability distance measure (k/1/2/h/s/e/m), option “e” (the Earth-Mover’s distance d_EM used in ϕ^3.0 [11]) remains an attractive candidate for discrete distributions with small number of bits, but is otherwise computationally unfeasible as we discussed above. All options in Table 1 except ϕ^3.0 and ϕ^MD therefore use option “k” (the KL-divergence). Note that whether it is an advantage for the probability distance measure to be symmetric (as advocated in [11]) depends on the interpretational context. For example, there is nothing asymmetric about the mutual information that ends up defining ϕ^M in Table 2.

For the choice of factorization (n/m/o/x/a), we can quickly dispense with option “a” (for being identical to “o”) and option “x” (because it has the highly undesirable property of always vanishing for deterministic systems). Which of the remaining options (n/m/o) is preferable depends on other choices. If one wishes to use a distance measure other than the KL-divergence, then the noising options “n” or “m” are computationally preferable, since the optimal factorization “o” can no longer be found analytically. Otherwise, “m” is arguably inferior to “o” because it is no simpler to evaluate and can overestimate the integration as described above. If one has a philosophical preference for the factorization depending only on the mechanism M and not on any other information about state probabilities, then “n” is the only choice. If one wishes to consider continuous systems, on the other hand, “n” is undefined. In summary, the best factorizations are therefore “o” and “n”, depending ones preferences. In practice, numerical experiments show that “n”, “m” and “o” usually give quite similar ϕ-values for a wide range of M-matrices and probability distributions, so the choice between the three is a relatively minor one.

Turning now to the choice variable selection and conditioning, Table 1 shows that many of the otherwise well-defined integration measures from Table 2 have serious flaws.

Neither ϕ^ots and ϕ^ofs are guaranteed to vanish for separable systems, which means that we cannot in good conscience interpret them as measures of integration. Numerical experiments show that ϕ^nas, ϕ^nps, ϕ^mas and ϕ^nps tend to be extremely small in practice (ϕ^mas is plotted in Fig 2). This is because they differ little from the corresponding measures using optimal factorization (ϕ^oas and ϕ^ops), which always vanish. In other words, they are not really measures of integration, merely measures of how suboptimal the factorizations “n” and “m” are. For brevity, we have included merely three of these six flawed measures in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Numerical comparison of different integration measures, averaged over 3,000 random trials.

In the bottom panel, all elements of p are independently drawn from a uniform distribution and normalized to sum to unity. In the top panel, only p⁽⁰⁾ is randomly generated, and M is defined so as to swap the two subsystems, i.e., M_jj′ii′ = δ_ij′ δ_i′j.

https://doi.org/10.1371/journal.pcbi.1005123.g002

Fig 2 shows that ϕ^ofu also tends to be much smaller than some other integration measures. We can intuitively understand this by recalling that ϕ^oau = 0, which means that optimal factorization lets us predict the future marginal distributions for A and B perfectly. Since ϕ^ofu quantifies the inability of optimal factorization to predict the full future distribution, we expect that it will at most be of the order of , the extent to which this distribution is not separable (determined by its marginal distributions). For randomly generated probability distributions generated as in Fig 2), one can show that bits in the limit where n → ∞, and numerical experiments indicate that ϕ^ofu is never much larger than this value for any p.

Dispensing with flawed/problematic ϕ-measures narrows our list of remaining top candidates to merely nine: ϕ^otu, ϕ^otum, ϕ^ofk, ϕ^oak, ϕ^opk, ϕ^nak, ϕ^npk, ϕ^mak and ϕ^mpk. Morover, the last six can be elegantly combined into merely three even better ones. As we discussed above, they have the advantage that they vanish for either afferent or efferent systems.

By following the prescription of [10] and taking the minimum of two such complementary measures, we can construct an even better one that vanishes for both afferent and efferent systems. All three of these improved measures are listed in Table 2. The first is ϕ^2.5 ≡ min{ϕ^nak, ϕ^npk}. We denote it “2.5” because it combines attractive features of both IIT2.0 and IIT3.0: it starts with the ϕ^npk, which is precisely the IIT2.0 measure, and improves it by taking the minimum of cause/effect integration in the spirit of IIT3.0 (but retaining the KL-divergence of IIT2.0 instead of the harder-to-compute Earth-mover’s distance of IIT3.0). The second is , which has the advantage of remaining defined even for continuous variables. The third is , which uses the optimal factorization.

How large can ϕ get?

In summary, our taxonomy of ϕ-measures produces merely a handful of truly attractive options: ϕ^2.5, , , ϕ^3.0, ϕ^MD, ϕ^M and . Fig 2 shows examples of what they evaluate to numerically. The lower panel shows that for randomly generated probability distributions, none of them exceed 1 − 1/2ln2 ≈ 0.28 bits on average, which as mentioned above is the mutual information in a random bivariate distribution. However, ϕ^2.5, , , ϕ^M, ϕ^MD and can get arbitrarily large for some systems, as illustrated in the top panel, growing logarithmically with the size n of the subsystems A and B. In other words, the maximum integration is of the order of the number of subsystem bits. For the example shown where the dynamics merely swaps the two subsystems, we obtain ϕ^2.5 = log₂ n, because noising gives M^A = 1/n, q = 1/n² and p is a Kronecker δ. ϕ^M, ϕ^MD and are seen to give about twice the integration for this example.

Note that although this dynamics M that merely swaps the subsystems has such a large ϕ-value only for this particular cut that separates the systems being swapped. Consider, for example, a system of four bits labeled 1, 2, 3 and 4, where the dynamics swaps 1 with 3 and 2 with 4. There is a different cut where ϕ = 0: simply define the new subsystems A’ and B’ to be the first and second halves of the A and B-systems, i.e., A′ = 1, 3 and B′ = 2, 4. The swapping is now carried out internally within A’ and B’, revealing that there is no integration and upper-case Φ = 0.

However, there are plenty of systems for which even the true integration Φ grows like the number of subsystem bits, log₂ n. A simple example accomplishing this (in the spirit of the random coding example in [12]) is when the n⁴ probabilities p_ii′jj′ are all set to zero except for a randomly selected subset of n² of them that are set to 1/n². Now ϕ^M ∼ log₂ n even when minimized over all bipartitions of the 2log₂ n bits in the system. For this example, we have S(x) = log₂ n² = 2 log₂ n. The marginal distributions for x^A, x^B, and are all rather uniform, with entropy on average less than a bit from the value for a uniform distribution, giving S(x^A) ∼ S(x^B) ∼ log₂ n², , I(x^A, x^B) = S (x^A) + S (x^B) − S(x) ∼ 2 log₂ n, and therefore .

Fig 3 shows that the measures ϕ^M and ϕ^MD can sometimes be quite similar: they give numerically similar values for the 3,000 random examples shown. Moreover, they appear to satisfy the inequality ϕ^ofum ≤ ϕ^otuk. Further examination shows that for these these random examples, the β-complication in eq (24) makes essentially no perceptible difference in practice, in the sense that the computation of ϕ^MD can be accurately accelerated by setting β = 1 rather than minimizing over it. However, [15] shows that there are real-world cases where β is far from unity and also where ϕ^MD ≪ ϕ^M, particularly when noise correlations dominate over causal correlations. To understand this, consider the extreme case of two perfectly correlated bits that are independently randomized by both time 0 and time 1, so that and , with no correlation between the two times. Then ϕ^MD = 0 whereas , which is arguably undesirable.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Numerical comparison of the two measures ϕ^otuk and ϕ^ofum for 3,000 random trials, generated the same way as in Fig 2.

The two measures are seen to be rather similar for these examples, and to satisfy the inequality ϕ^ofum ≤ ϕ^otuk.

https://doi.org/10.1371/journal.pcbi.1005123.g003

The n → ∞ limit of continuous variables

All our previous results are fully general, applying regardless of whether the variables are discrete (such as bits that equal zero or one) or continuous (such as voltages or other variables measured in fMRI, EEG, MEG or electrophysiology studies). We can view the latter as the n → ∞ limit of the former, since a single real number can be represented as an infinite string of bits. In this section, we will focus on the continuous case and see how our previous formulas can be greatly simplified by assuming Gaussianity. We therefore replace i, i′, j and j′ in all our formulas by , , and , respectively, and replace all sums by integrals.

Graph-theory approximation to make computations feasible

An approximate solution.

Being able to compute Φ approximately is clearly better than not being able to compute it at all. In this spirit, let us explore an approximation that exponentially accelerates the computation of Φ. Starting with the linear dynamics x_i+1 = Ax + n from eq (55), let us motivate our approximation by considering the case where the noise is n uncorrelated (where ∑ is diagonal) so that it introduces no correlations between the two systems, regardless of the cut. This means that the only source of integration can be the A-matrix transferring information between the two subsystems. Let us visualize this information flow as a directed graph (Fig 4, bottom), where each node represents a variable i and each edge represents a non-zero element A_ij, i.e., non-zero information flow from element j to element i. If this graph consists of two disconnected parts A and B of equal size, as in the lower right corner of Fig 4, then we clearly have Φ = 0, since there is no information flow and hence no integration between these two parts. In other words, if we permute the elements so that all elements of A precede all elements of B, the matrix A becomes block-diagonal (Fig 4, middle right), for which all integration measures in the right column of Table 2 will give ϕ = 0.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Illustration of our fast Φ-approximation for an n = 16 example.

The structure of the A-matrix can be visualized either as a grid (top four examples) where each pixel color shows the value of the corresponding element A_ij ranging from the smallest (black) to the largest (white), or as a graph (bottom examples) showing all non-zero matrix elements. Both of the matrices on the left correspond to the same graph below them, and both of the matrices on the right correspond to the same (disconnected) graph below them. Our method zeros all matrix elements |A_ij| < ϵ below the threshold ϵ that makes the largest connected graph component involve merely half of the elements, which in the matrix picture means that there is a permutation of the elements (rows and columns) rendering the matrix block-diagonal (middle right). Whereas it would take exponentially long to try all matrix permutations, graph connectivity can be determined in polynomial time, thus enabling us to rapidly find a good approximation for the “cruelest cut” bipartition.

https://doi.org/10.1371/journal.pcbi.1005123.g004

Note that before the elements were permuted (Fig 4, top right), this fact that ϕ = 0 was less obvious. Moreover, examining all n! permutations (or all symmetric bipartitions) would have been an enormously inefficient way of finding that best bipartition for which ϕ vanishes. In contrast, finding the connected components of a graph is quite simple, as is evident from staring at Fig 4, with complexity between O(n) and O(n²). This means that if we know that Φ = 0, then we can find the best bipartition (“cruelest cut”) easily, in polynomial time.

Let us now define an approximation taking advantage of this idea: replace all unimportant elements |A_ij| < ϵ by zero, and adjust ϵ so that the largest connected component has size as close as possible to n/2. Letting this largest connected component define our approximation of the best bipartition, we now compute its ϕ-value and use this as our approximation for Φ.

Note that this approximation can be trivially generalized to asymmetric bipartitions (the subtle conceptual challenges of how to weight or otherwise handle asymmetric partitions [10, 11, 13, 22] are neither ameliorated nor exacerbated by our fast approximation).

In practice, we determine ϵ by using the interval halving method. A final technical point is that we have two separate definitions of graph connectivity to choose between: weak and strong. A graph is strongly connected if you can move between any pair of elements following the directional arrows on the edges. This means that every element can (at least through intermediaries) affect and be affected by every other element, precisely capturing the integration spirit of [10]. Strong connectivity is therefore the logical choice when using our approximation to compute Φ^2.5, , , since it will reflect their property that integration vanishes for afferent and efferent pathways. A graph is weakly connected if you can move between any pair of elements ignoring edge arrows—in other words, if it simply looks connected when drawn. Using weak connectivity is arguably the better approximation for the Φ-measures that do not vanish for afferent/efferent pathways, and numerical experiments confirm this.

Fig 5 illustrates the accuracy of our approximation. For this example, we randomly generate 7,000 different 16 × 16 matrices A and compute Φ^M both exactly (as the minimum of ϕ^M over all symmetric bipartitions) and using our approximation. Here we generate A-matrices by first computing (78) where A₀, A₁ and A₂ are random matrices (whose elements are independent Gaussian random variables with zero mean), each normalized to have their largest eigenvalue equal to unity. We then renormalize A so that its largest eigenvalue equals 0.99. The parameter η controls the typical level of integration: η = 0 gives Φ = 0 since A is block-diagonal, whereas η → ∞ gives maximal integration, with no special cut put in by hand; η is randomly chosen to be 0.1, 0.3, 0.5, 0.7, 1, 2 or 10 with equal probability. Once we have generated A, we compute C as the solution to the Lyapunov equation C = ACA^t + ∑ with ∑ = I.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. How well our fast Φ-approximation works for 7,000 simulations of the n = 16 Φ^M-example described in the text.

Whereas it is seen to be excellent at finding the best bipartition when not all are comparably good, (i.e., when Φ_max/Φ_min ≫ 1), the approximation is seen to overestimate Φ by up to 15% (the median) when there is no clear winner (left side). From top to bottom, the three curves show the 95th, 50th and 5th percentiles of the overestimation factor. The shaded region delimits the largest overestimation possible, when Φ_appox = Φ_max.

https://doi.org/10.1371/journal.pcbi.1005123.g005

For comparison, we also compute the maximum over the bipartitions. The ratio Φ_max/Φ ≥ 1 (where Φ ≡ Φ_min) quantifies how relatively decomposable a system is, whereas the ratio Φ_approx/Φ ≥ 1 quantifies how well our approximation works, with a value of unity signifying that it is perfect and found the optimal bipartition. Fig 5 plots these two quantities against each other, and reveals that they are strongly related. For fairly separable systems, the approximation tends to be excellent: it gives exactly the correct answer 95% of the time when Φ_max/Φ > 2 and 99.96% of the time when Φ_max/Φ > 3. When Φ_max/Φ ≲ 2, on the other hand, so that there is less of a clear winner among the bipartitions, our approximation is seen to overestimate the true Φ-value by up to 15% on average (this is the median).

An alternative implementation, which we find works even better for some examples, is to apply the above-mentioned graph-based bipartition-finding scheme not to the evolution matrix A but to the covariance matrix C. We therefore recommend computing two approximate bipartitions, one based on A and one based on C, and selecting the one producing the smaller ϕ-value.

Which Φ-measures are best?

As described in detail in Results, six Φ-measures stand out from the taxonomy of hundreds of measures as particularly attractive: Φ^M, , Φ^3.0, Φ^2.5, and . Φ^M retains all the attractive features of the Barrett/Seth measure Φ^B and adds further improvements: it is guaranteed to vanish for separable systems and to never be negative. If state-dependence is viewed as desirable, then its cousin adds that feature too.

Φ^3.0 is the measure advocated by IIT3.0 and has the many attractive features described in [11]. It has the drawback of being the slowest of all the measures to evaluate numerically: its definition involves a linear programming problem which needs to be solved numerically, and even with the fastest algorithms currently available, the computation for a given bipartition grows faster than quadratically with the number of system states—which in turn grows exponentially with the number of bits, and is infinite for continuous variables.

The remaining three top measures, Φ^2.5, and , share with Φ^3.0 the arguably desirable feature of vanishing for afferent and efferent systems, but are much quicker to compute. Φ^2.5 combines core ideas from IIT3.0 with the computational speed of IIT2.0 [10, 11] and elegantly depends only on the system’s dynamics and present state, not on any assumptions about which states are more probable. Its drawback of being infinite for continuous variables is overcome by its cousin .

A potential philosophical objection to both Φ^2.5 and is that they are arguably not measures of integration, but measures of how suboptimal the factorizations “n” and “m” are, since they would both vanish if an optimal factorization were used—the measure eliminates this concern.

Outlook

Although the results in this paper will hopefully prove useful, there is ample worthwhile work left to do on integration measures.

One major open question is how to best handle asymmetric partitions. We deliberately sidestepped this challenge in the present paper, since it is independent of our results, which is why the subtle normalization issue raised by [10, 11, 13, 22] never entered. The crux is that if we apply any of the measures in our taxonomy with an asymmetric bipartition, the resulting ϕ-value will tend to get small when any of the two subsystems is very small, so simply defining Φ as the minimum of ϕ over all bipartitions (symmetric or not) makes no sense. IIT3.0 makes an interesting proposal [11] for how to handle asymmetric partitions, and it is worthwhile exploring whether there are other atttractive options as well.

Another foundational question is whether our taxonomy can be placed on a firmer logical footing. Although our classification based on factorization, comparison, conditioning and measure may seem sensible and exhaustive, it is interesting to consider whether one or several Φ-measures can be rigorously derived from a small set of attractive axioms alone, in the same spirit as Claude Shannon derived his famous entropy formula, eq (30).

Yet another foundational question is whether integration maximization can be placed on a firmer physical footing, as advocated by [33, 34] in the context of continuous physical fields and by [12] in the context of quantum systems. The formulas in our taxonomy take information, measured in bits, as a starting point. But when I view a brain or computer through my physicist eyes, as myriad moving particles, then what physical properties of the system should be interpreted as logical bits of information? I interpret as a “bit” both the position of certain electrons in my computer’s RAM memory (determining whether the micro-capacitor is charged) and the position of certain sodium ions in your brain (determining whether a neuron is firing), but on the basis of what principle? Surely there should be some way of identifying consciousness from the particle motions alone, or from the quantum state evolution, even without this information interpretation? If so, what aspects of the behavior of particles corresponds to conscious integrated information? In other words, how can we generalize the quest for neural correlates of consciousness to physical correlates of consciousness? IIT argues that the consciousness occurs at precisely the level of course-graining in space and time that maximizes Φ [10], which is a prediction that should be tested.

A more practical question involves exploring ways of generalizing and further improving our graph-theory-based approximation for exponential speedup. One obvious generalization would involve taking advantage of the structure of ∑ (which our method ignored) and the effect of x (for those Φ-measures that are state-dependent). Another interesting opportunity is to generalize from continuous Gaussian systems to arbitrary discrete systems. For example, if the system consists of b different bits coupled by a nonlinear network of gates, one can apply a similar graph-theory approach by defining a b × b coupling matrix A_ij that in some way quantifies how strongly flipping the j^th bit would affect the i^th bit at the next timestep. As an example, consider defining A_ij as the probability that flipping the j^th bit will flip the i^th bit at the next timestep. If we have six bits evolving according to then the coupling matrix is were p_c denotes the probability that c₀ = 1, p_de denotes the probability that d₀ = 1 and e₀ = 1, etc. This coupling matrix is block-diagonal, showing that the bits a, b are completely independent of the others. For a state-independent Φ-measure, these probabilities can be computed as time-averages, otherwise they are each zero or one depending on the state. In either case, some elements of the A-matrix can be small but non-zero (making the graph-theory approximation useful) if the system involves noisy gates or other randomness.

As regards practical challenges, it is important to note that there are many other issues besides speed that deserve further work because they have hindered the practical computation of integration Φ-measures from real brain data, including non-stationarity, statistical issues with estimating large numbers of parameters from short data windows without overfitting, possibilities of statistical bias, numerical instabilities, etc.

Last but not least, a veritable goldmine of data is becoming available in neuroscience and other fields, and it will be fascinating to measure Φ for these emerging data sets. In particular, the exponentially faster Φ-measures we have proposed will hopefully facilitate quantitative tests of theories of consciousness.