^{*}

Conceived and designed the experiments: ABB AKS. Performed the experiments: ABB. Analyzed the data: ABB AKS. Contributed reagents/materials/analysis tools: ABB. Wrote the paper: ABB AKS.

The authors have declared that no competing interests exist.

A recent measure of ‘integrated information’, Φ_{DM}, quantifies the extent to which a system generates more information than the sum of its parts as it transitions between states, possibly reflecting levels of consciousness generated by neural systems. However, Φ_{DM} is defined only for discrete Markov systems, which are unusual in biology; as a result, Φ_{DM} can rarely be measured in practice. Here, we describe two new measures, Φ_{E} and Φ_{AR}, that overcome these limitations and are easy to apply to time-series data. We use simulations to demonstrate the in-practice applicability of our measures, and to explore their properties. Our results provide new opportunities for examining information integration in real and model systems and carry implications for relations between integrated information, consciousness, and other neurocognitive processes. However, our findings pose challenges for theories that ascribe physical meaning to the measured quantities.

A key feature of the human brain is its ability to represent a vast amount of information, and to integrate this information in order to produce specific and selective behaviour, as well as a stream of unified conscious scenes. Attempts have been made to quantify so-called ‘integrated information’ by formalizing in mathematics the extent to which a system as a whole generates more information than the sum of its parts. However, so far, the resulting measures have turned out to be inapplicable to real neural systems. In this paper we introduce two new measures that can be applied to both realistic neural models and to time-series data garnered from a broad range of neuroimaging and electrophysiological methods. Our work provides new opportunities for examining the role of integrated information in cognition and consciousness, and indeed in the function of any complex biological system. However, our results also pose challenges for theories that ascribe a direct physical meaning to any version of integrated information so far described.

How can the complex dynamics exhibited by networks of interconnected elements best be measured? Answering this question promises to shed substantial new light on many complex systems, biological and non-biological. Neural systems in particular are characterized by richly interconnected elements exhibiting complex dynamics at multiple spatiotemporal scales

Several measures now exist which operationalize the above intuition under different assumptions and with varying practical applicability

A first version of

It has been shown, using simulations, that

In this paper we introduce an alternative measure of integrated information,

The difference between

The above distinction carries implications for theories, such as the IITC, that ascribe physical meaning to measures of integrated information. Under the IITC, consciousness is explicitly characterized in terms of the capacity of a system

The ‘

We use bold upper-case letters to denote multivariate random variables, and corresponding bold lower-case letters to denote actualizations of random variables. Matrices are denoted by upper-case letters. The

Let

Entropy

We write

The Kullback-Leibler (KL) divergence

We examine integrated information generated by systems of interconnected dynamical elements. We use the letter

A stationary system is one for which the probability density function for

In this section we review, following Ref.

Let

The

To specify the probability distributions in (0.14), one must use Bayes' rule. For the distribution of the whole system the formula is

The

For a state-independent alternative to

We emphasize that

In this section we introduce a new measure of integrated information,

The key modification is that rather than measuring information generated by transitions from a hypothetical maximum entropy past state,

We now define

The integrated information

For numerical computation, the required probability distributions can in principle be obtained directly from data, although in practice it may be difficult to obtain sufficient data to enable accurate estimation of all the relevant entropies. As we explain in the section ‘Computing

For analytic computation of

A few further remarks about

Our definition (0.28) for the effective information,

Network | (i) |
(iv) |
(v) |
|||

1(a) | 0.0323 | 0.0323 | 0.0323 | |||

1(b) | 0.0645 | 0.0645 | 0.0645 | |||

1(c) | 0.1283 | 0.1387 | 0.1313 | |||

1(d) | 0.0795 | 0.0894 | 0.0755 | |||

1(e) | 0.1285 | 0.1376 | 0.1303 | |||

1(f) | 0.1294 | 0.1383 | 0.1307 | |||

1(g) | 0.1266 | 0.1362 | 0.1288 | |||

2(a) | 0.2502 | 0.2652 | 0.1254 | |||

2(b) | 0.2965 | 0.3012 | 0.2647 |

In summary, we have defined a new measure of integrated information

Under Gaussian assumptions, equation (0.10) furnishes an expression for

In this section we describe analytical computation of

We present results from computing

(a)–(g) Connectivity diagrams for seven systems as specified by the corresponding connectivity matrices

For all systems, except 1(g) (which we discuss below), the analytically derived (true) value of

The values of integrated information mostly correspond with expectations. For example, a ring of reciprocal connections (1(c)) integrates approximately twice as much information as a ring of unidirectional connections (1(b)), which itself integrates approximately twice as much information as a (non-closed) chain of unidirectional connections (1(a)). Also as expected, the homogenous system 1(d) has a low

For values of

Network 1(g) exhibited instability when measuring

One may consider that this problem of instability could be avoided by using non-normalized

To examine whether network structures other than reciprocally connected rings could generate high levels of

The results of the optimizations are shown in

(a) Optimal network for 2 afferents of 0.25 to each node. This has

The observation that the fittest network found in each condition was only reached by a minority of GAs suggests that the

(a) Histogram of

The instability arising from using normalized effective information to find the MIB, (see ‘Canonical examples’), suggests that there may be discontinuities, as well as ruggedness, in the

It is instructive to compare results obtained using

The network 2(a) has a value for

We also compared results obtained using

The analyses in the previous section were concerned with integrated information measured across a single time-step for MVAR(1) processes. However,

(a) Network 1(c). (b) Network 2(b). (c) Example MVAR(3) process, see Eq. (0.44).

We have presented a measure of integrated information,

We now describe how, even for the non-Gaussian case, the recipe used to calculate

First we rehearse the concept of linear regression. Let

To demonstrate the use of

Each panel shows an empirical probability distribution as a histogram taken from 3000 data points from element 1 in (a) network 1(b), (b) network 1(d), and (c) network 2(b).

In this paper we have presented two new measures of integrated information,

As mentioned, many of the restrictions in applicability of

Our new measure

In principle, use of the empirical distribution de-emphasizes the notion of ‘capacity’ because the generation of information is measured with respect to what the system

Use of the empirical, rather than maximum entropy distribution also changes the means by which

While Gaussian dynamics are common in biology (and the assumption of Gaussianity even more so), many systems depart from this assumption. For example, the spiking activity of populations of neurons typically exhibit exponentially distributed dynamics. For the non-Gaussian case, one can still in principle calculate

All versions of

The use of normalization, as just described, leads to instabilities. Our simulations have shown that

Previous measures of integrated information (

An important feature of the IITC as previously expressed is that consciousness

The notion that

Although

Causal density, like

Neural complexity is calculated as the sum of the average mutual information across all bipartitions of a system

These relations together suggest common foundations for measures of coexisting integration and differentiation. However, further work is needed to fully establish their theoretical interdependencies and their empirical convergences and divergences.

Characterizing complexity is a diverse field, and there are other measures that capture complex properties other than conjoined differentiation and integration. For example, ‘thermodynamic depth’

Although our measures represent substantial improvements in practical applicability of measures of integrated information, several limitations remain. Most prominently, the normalization procedure leads to instabilities in the measurement process and undercuts ascription of physical meaning to

We have only considered a first-order, linear approximation for computing entropies/information from data. While this is useful for drawing comparison with Granger causality and causal density, there now exist more advanced approximation techniques that could be used in future work, for example additive regression

As well as addressing the above challenges, future work will (i) empirically examine integrated information for time-series data acquired from neuroimaging and other biological datasets, in order to test intuitions regarding consciousness and other neurocognitive processes; (ii) investigate in models how integrated information is modulated by input and output relations of a system embedded in, and interacting with, a surrounding environment, and (iii) determine theoretically the relations between integrated information and alternative measures of dynamical complexity and metastability.

To extend

Having defined a distribution for the initial state

Given

To compute the conditional probability

Here we show how to compute

The method outlined in ‘Computing

Toolbox for computing integrated information as Φ_{E} or Φ_{AR}. ‘phiemvarp.m’ computes Φ_{E} from an MVAR(_{AR} ( = Φ_{E} if Gaussian), from stationary time-series data. ‘statdata.m’ creates time-series data from an MVAR(

(0.01 MB ZIP)

We are grateful to Lionel Barnett and Chang-Sub Kim for useful discussions. Nihat Ay, David Balduzzi and one anonymous reviewer provided extremely detailed and valuable comments on a first draft of this paper.