Advertisement
  • Loading metrics

Stability of spontaneous, correlated activity in mouse auditory cortex

  • Richard F. Betzel,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Resources, Software, Visualization, Writing – original draft, Writing – review & editing

    Affiliations Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana, United States of America, Cognitive Science Program, Indiana University, Bloomington, Indiana, United States of America, Program in Neuroscience, Indiana University, Bloomington, Indiana, United States of America, Network Science Institute, Indiana University, Bloomington, Indiana, United States of America

  • Katherine C. Wood,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Writing – original draft, Writing – review & editing

    Affiliation Department of Otorhinolaryngology: HNS, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America

  • Christopher Angeloni,

    Roles Data curation, Methodology, Resources

    Affiliation Department of Otorhinolaryngology: HNS, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America

  • Maria Neimark Geffen,

    Roles Data curation, Funding acquisition, Investigation, Methodology, Resources, Supervision, Writing – original draft, Writing – review & editing

    Affiliation Department of Otorhinolaryngology: HNS, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America

  • Danielle S. Bassett

    Roles Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Resources, Supervision, Visualization, Writing – original draft, Writing – review & editing

    dsb@seas.upenn.edu

    Affiliations Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, Department of Electrical and Systems Engineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, Department of Physics & Astronomy, College of Arts & Sciences, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America, Santa Fe Institute, Santa Fa, New Mexico, United States of America

Stability of spontaneous, correlated activity in mouse auditory cortex

  • Richard F. Betzel, 
  • Katherine C. Wood, 
  • Christopher Angeloni, 
  • Maria Neimark Geffen, 
  • Danielle S. Bassett
PLOS
x

Abstract

Neural systems can be modeled as complex networks in which neural elements are represented as nodes linked to one another through structural or functional connections. The resulting network can be analyzed using mathematical tools from network science and graph theory to quantify the system’s topological organization and to better understand its function. Here, we used two-photon calcium imaging to record spontaneous activity from the same set of cells in mouse auditory cortex over the course of several weeks. We reconstruct functional networks in which cells are linked to one another by edges weighted according to the correlation of their fluorescence traces. We show that the networks exhibit modular structure across multiple topological scales and that these multi-scale modules unfold as part of a hierarchy. We also show that, on average, network architecture becomes increasingly dissimilar over time, with similarity decaying monotonically with the distance (in time) between sessions. Finally, we show that a small fraction of cells maintain strongly-correlated activity over multiple days, forming a stable temporal core surrounded by a fluctuating and variable periphery. Our work indicates a framework for studying spontaneous activity measured by two-photon calcium imaging using computational methods and graphical models from network science. The methods are flexible and easily extended to additional datasets, opening the possibility of studying cellular level network organization of neural systems and how that organization is modulated by stimuli or altered in models of disease.

Author summary

Neurons coordinate their activity with one another, forming networks that help support adaptive, flexible behavior. Still, little is known about the organization of these networks at the cellular scale and their stability over time. Here, we reconstruct networks from calcium imaging data recorded in mouse primary auditory cortex. We show that these networks exhibit spatially constrained, hierarchical modular structure, which may facilitate specialized information processing. However, we show that connection weights and modular structure are also variable over time, changing on a timescale of days and adopting novel network configurations. Despite this, a small subset of neurons maintain their connections to one another and preserve their modular organization across time, forming a stable temporal core surrounded by a flexible periphery. These findings represent a conceptual bridge linking network analyses of macroscale and cellular-level neuroimaging data. They also represent a complementary approach to existing circuits- and systems-based interrogation of nervous system function, opening the door for deeper and more targeted analysis in the future.

Introduction

Distributed and often redundant coding is a hallmark of neural systems [1], providing robustness to single-neuron variability [2] and supporting complexity in the system’s potential behavioral repertoire [3]. A key challenge in understanding this code lies in determining how the nature and strength of correlations between neurons is related to a stimulus [4]. Recent evidence suggests that so-called noise correlations have marked and diverse functions [5], from impacting information encoding and decoding [68], to tuning the amount of information present and thus the nature of ensuing cortical representations [912]. Correlations in spike trains have also been noted to contain important information about excitability, latency, and synchronization [1315]. Even apart from task-evoked activity, spontaneous activity and correlations of that activity can profoundly impact cortical responses to a sensory input, thereby playing a critical role in information processing [16, 17].

To better understand the nature of coherent multi-unit interactions both during intrinsic and stimulus-induced processing, it is necessary to have a language in which to study inter-unit interaction patterns. In related work in other species and other spatial scales, network science has proven its utility as just such a candidate language [18]. The notion of a network in its simplest form is akin to the notion of a graph in the field of mathematics known as graph theory [19]. Specifically, an undirected binary graph is composed of nodes, which represent the units of the system, and edges, which link pairs of nodes according to some physical connection, functional relation, or shared feature [20]. This simplest version of a network can also be expanded to include weights on edges, weights on nodes, dynamics on edges, dynamics on nodes, or multiple types of nodes or edges forming a multilayer or multiplex structure [21, 22]. By either the simple or expanded encoding, network models of neural systems seek to distill the most salient organizational features of the system, allowing investigations to focus on how the network topology constrains or supports the system’s function [23]. Importantly, the network modeling approach is flexible in the sense that its components can be redefined at different spatial scales, and is thus equally applicable to cellular data at the microscale as it is to regional data at the large scale [24].

Recent studies have begun to build and characterize network models of cellular activity as measured by calcium imaging [2531], and have demonstrated their biological relevance across a neural system’s development [32]. For example, one notable study provided initial evidence that immature cells in the developing brain display spontaneous correlation patterns that are characterized by small-world architecture and that critically regulate neural progenitor proliferation [33]. In a mature system (ferret visual cortex), recent evidence suggests that local connections in early cortical circuits can generate structured long-range network correlations that guide the formation of visually evoked distributed functional networks that display striking network modularity [34]. The architecture of correlations in spontaneous activity can be regulated by synaptotagmin [35], modulated by acetylcholine [27], blocked by glutamatergic antagonists [36], and mediated by a combination of intrinsic and circuit mechanisms [36]. Yet, little is known about the conservation or variation of network architecture in spontaneous correlations across different regions of the brain. Moreover, while the activity can be temporally quite precise in a given instance [36], little is known about how patterns of spontaneous activity change over the course of days and weeks after the critical period of development has passed. Understanding the principles of these dynamics is important for understanding the conserved rules that the architecture must obey, as well as the variability that can be exercised to meet the demands of the ever changing internal or external environments.

Here, we take steps to address some of these gaps in knowledge by measuring correlated spontaneous neuronal activity using two-photon calcium imaging, modeling those correlation patterns as networks, and assessing network architecture and dynamics over the course of several weeks. We focus our measurements specifically on mouse auditory cortex because of its rich organizational characteristics, with distributed representations of tone frequency [37], spatially overlapping locations for the representations of pitch and timbre [38], and the capacity for single neurons within the wider network to encode simultaneous stimuli by switching between activity patterns [39]. We choose mouse as our species of interest largely to prepare for future efforts using two-photon optogenetics [40] to perturb the network architecture, with the goal of probing network response to stimulation and validating recently posited theories of network control [4143]. We begin by testing the hypothesis that networks reconstructed from fluorescence correlations exhibit hierarchical modular structure, and that network modules fluctuate over the timescales of days or weeks. We also test the hypothesis that some units participate in these temporal fluctuations more than others such that the system is best characterized by the existence of a stable temporal core surrounded by a fluctuating and variable periphery. Each of these hypotheses is motivated by prior observations in non-invasive imaging data acquired from humans [4446], where evidence points to the importance of hierarchical modularity and temporal core-periphery structure for effective cognitive function [18, 45, 4749]. Thus, collectively our hypotheses are predicated on the notion that neural systems are constrained to display some degree of preservation in network architecture across species, from human to mouse [43, 50, 51], as well as scale invariance, from the level of large-scale areas to the level of small-scale units [5254].

Results

We recorded spontaneous activity from four awake, head-fixed mice over the course of 5, 5, 6, and 7 sessions spanning between 2 and 4 weeks. Specifically, we used two-photon microscopy to detect changes in fluorescence of GCaMP6s in transfected neurons caused by fluctuations in calcium activity. We estimated functional connectivity from the fluorescence traces using a cross-correlation of differenced activity for every pair of cells. We note that our analysis was carried out on continuous calcium traces rather than sparse spike trains, which can be obtained using deconvolution methods. We modeled the cell-to-cell correlation matrix as a network [23], and quantitatively characterized the network’s architecture using well-developed tools from network science [19]. Specifically, we assessed the modularity of the network structure using a commonly applied community detection technique known as modularity maximization [5557]. Further, we assessed temporal fluctuations in this modular structure using tools for the analysis of dynamic graphs [21, 58, 59]. For further details on our methodological approach, see Materials and methods.

Networks exhibit multi-scale modular structure

One of the most important organizational principles of biological neural networks is their organization into cohesive modules [45]. These modules are thought to support specialized information processing while conferring robustness to perturbations. Moreover, converging evidence from micro- and macro-scale network analyses suggest that network modules are also organized hierarchically, with larger modules subtending broader brain function and smaller modules playing more specialized roles [24, 60, 61]. In this section, we test the hypothesis that networks reconstructed from fluorescence correlations in mouse auditory cortex exhibit hierarchically modular structure.

To address this hypothesis, we leverage recent advances in community detection methods [62]—a collection of algorithms and heuristics that use data-driven approaches to uncover the modular structure of networks. Specifically, we use an extension of the popular modularity maximization algorithm [55]. The standard version of this algorithm defines a module as a group of network nodes whose internal density of connections is maximally greater than what would be expected under a chance model. The extension of this algorithm samples modules over multiple organization scales, ranging from coarse divisions of the network into a few large modules to finer divisions of the network into many small modules. Importantly, unlike past applications, this extension also includes built-in null statistical testing capable of rejecting modular structure at different levels of the proposed hierarchy if they were consistent with a null model.

Here, we applied this approach to investigate the hierarchically modular structure of networks derived from correlated fluorescence traces (Fig 1a). The module detection method was applied separately to networks constructed from data in each recording session, which allowed us to take full advantage of all cells recorded on a given day. The algorithm resulted in a hierarchy of communities that passed statistical testing for significance (p < 0.05; Fig 1b–1f). In general, we found that the fluorescence networks exhibited hierarchical, multi-scale modular structure. Of the 43 recordings (aggregated across all mice) we observed statistically significant hierarchies in all. Across recording sessions, the average number of scales in a hierarchy was 19 (inter-quartile range of [10.25, 27.75]; Fig 1c).

thumbnail
Fig 1. Detection of hierarchical modular structure.

(a) Mean fluorescence of pixels, averaged over the full recording session. (b) Co-classification matrix generated using all statistically significant hierarchical levels. The dendrogram to the right depicts module splits. (c) The number of hierarchical levels aggregating data from all mice and all recording sessions. Panels (d), (e), and (f) depict module assignments at different levels of the hierarchy. The network diagrams shown in these panels are identical to one another and represent binarized matrices obtained by thresholding the jitter-adjusted correlation matrix of fluorescence traces between pairs of cells. Panels (g), (h), (i), and (j) depict distributions of z-scored mean intra-module Euclidean distance for each module and for each mouse. Panels (b), (d), (e), and (f) depict representative results from mouse 1. Here, the acronyms “IQR” and “Med.” represent interquartile range and median, respectively. Note also that nodes in panels b and d-f are ordered according to their hierarchical community assignments.

https://doi.org/10.1371/journal.pcbi.1007360.g001

Additionally, we also computed spatial statistics for each module. Past studies have shown that communities tend to be spatially co-localized, so that other cells located near one another are more likely to belong to the same module compared to cells located far from one another [63]. To test whether this was also the case in our data, we computed the Euclidean distance from each cell to the nearest cell assigned to the same community. We then averaged this measure over all nodes in the same module. If cells were arranged in spatially dense, compact modules, then this measure would be small. Here, we calculated this measure for each module at every level of the hierarchy and compared these values against a null distribution generated by randomly and uniformly permuting the cell’s spatial locations but preserving their module assignments. For each module, we expressed the mean nearest-neighbor distance as a z-score with respect to this distribution. We found that the observed modules tended to be more spatially compact than expected by chance. For each mouse, the median z-score was less than zero and in all cases the inter-quartile range of z-scores excluded a value of zero (Fig 1g, 1h, 1i and 1j), indicating that the observed modules tended to be more spatially compact than expected by chance.

Lastly, we asked whether a module’s spatial features varied as a function of where it appeared in the hierarchy. To address this question, we aggregated z-scored nearest neighbor distance for each module along with modules’ hierarchical levels. A hierarchical level of 1 indicates large communities comprised of many nodes, while the deeper hierarchical levels refer to divisions of the network into smaller communities. Then, separately for each mouse, we computed the correlation of z-scored nearest neighbor distance with hierarchical level. We found negative association in three of the four mice, suggesting that deeper hierarchical levels, i.e. smaller communities, may be more spatially contiguous than larger communities. The Spearman correlation of hierarchical level and nearest neighbor distance was ρ = −0.22, −0.19, +0.01, and −0.27 for the four mice.

Network and module similarity decays over time

In the previous section we demonstrated that the correlation pattern of fluorescence traces exhibits modular structure across multiple scales, and that these multi-scale modules unfold as part of a hierarchy. In these analyses, the network’s modular structure was derived separately for each recording session. While this approach allowed us to characterize the modular structure on a given day, it tells us little about how those modules fluctuate over the timescales of days or weeks. Here, we address this question directly, taking advantage of the longitudinal tracking of cells across multiple recording sessions to assess the temporal consistency of the network’s overall organization, as reflected in the full correlation matrix, and in the network’s mesoscale organization, as reflected in its modular structure.

We begin by calculating the similarity between the correlation structure for any two recording sessions, u and v (Fig 2a). We first identified the set of cells from which fluorescence traces were recorded in both sessions. The average number of cells observed in any pair of recording sessions was 193.0±15.6 (Mouse 1), 132.7 ± 40.8 (Mouse 2), 121.7 ± 20.0 (Mouse 3), and 107.0 ± 35.6 (Mouse 4). We then extracted the correlation structure among those subsets of cells for each of the two recording sessions, resulting in two correlation matrices: Wu and Wv. Next, we vectorized the upper triangles of both matrices and computed the correlation of their elements, ruv. Finally, we expressed this correlation as a z-score, , with respect to a null distribution generated by randomly and uniformly permuting rows and columns of Wu and recomputing the correlation of with Wu (essentially the Mantel test [64]). Accordingly, large positive z-scores indicate that the correlation of ruv was much greater than expected in the non-parametric permutation-based null model. Aggregating z-scores across all pairs of recording sessions resulted in the z-scored similarity matrix, .

thumbnail
Fig 2. Reconfiguration of correlation structure over time.

(a) Analysis pipeline for comparing correlation structure. For any two correlation matrices, Wu and Wv, whose elements have been z-scored against those obtained under a “jittered” null model in which random offsets were added to timeseries (see Methods), we vectorize the upper triangular elements and compute their similarity using a Pearson correlation coefficient. We compare the observed correlation coefficient against that which we would expect under a null model in which rows and columns of Wu are permuted uniformly at random. In panels (b), (c), (d), and (e), we show the scatterplots of standardized similarity scores for pairs of correlation matrices with the number of days separating their respective recording sessions. In panels (f), (g), (h), and (i), each point represents the standardized similarity scores of module co-assignment matrices across pairs of recording sessions.

https://doi.org/10.1371/journal.pcbi.1007360.g002

To assess the degree to which the similarity in correlation structure depended upon the time interval that separated the recordings, we also computed the distance matrix D = [duv], which measures the distance (in number of days) between recording sessions u and v. We then compared the upper triangular elements of ZW with the corresponding elements of D. In general, we observed that decayed monotonically as a function of duv. Notably, this observation was consistent across all mice (mean±standard deviation Pearson correlation of ) (Fig 2b–2e). These findings indicate that the magnitude with which individual cells are correlated with one another over time varies systematically over recording sessions. Specifically, the correlation structures of recording sessions separated by a short period of time tend to be similar to one another, whereas those separated by longer periods of time tend to be dissimilar.

In addition to assessing whether cell-to-cell correlation patterns varied across recording sessions, we also aimed to assess the variability of modular structure. To address this question, we performed an analogous procedure to the one described above where we substitute the module co-assignment matrices Cu = [Ciju] and Cv = [Cijv] for the correlation matrices, Wu and Wv. Here, the element Ciju indicates the fraction of all detected community partitions in which cells i and j were co-assigned to the same module. Otherwise, this procedure for relating the modular structure of networks from different recording sessions proceeded exactly as described above. We denote the z-score matrix from the module comparison as . As before, we observed that the correlation of similarity in modular structure decays with time (mean±standard deviation Pearson correlation of ), indicating the presence of marked quotidian variation (Fig 2f–2i). We also repeated this analysis after dividing communities into terciles according to their hierarchical level to assess whether there was a relationship between hierarchy and module stability. We found evidence of decaying similarity at all levels, but no clear dependence on hierarchy (S1 Fig).

Temporal core-periphery structure

In the previous section we demonstrated that, on average, hierarchical modular structure becomes increasingly dissimilar over time. However, it may be the case that some sets of brain areas maintain their modular structure despite the passage of time, forming a stable temporal core surrounded by a fluctuating and variable periphery [44]. To test this hypothesis, we focused on sequences of recording sessions and characterized the stability of modules across those sessions.

Because the number of recording sessions varied from one mouse to another, we focused on sequences of five recording sessions (the greatest number that was available for all mice). For each mouse, we modeled the thresholded connectivity data from each of these five recording sessions as the layers in a multi-layer network object [22, 65], and we used multi-layer modularity maximization [66] to track the fluctuations in modular structure across those five sessions (Fig 3a; see Materials and methods for more details). The multi-layer modularity maximization approach extends the traditional modularity maximization approach [55] by incorporating networks from all five sessions into a singular multi-layer network object, and then detecting modules in all layers simultaneously. The main advantage of this procedure is that the same set of module assignments are preserved across all layers, making it possible to track the formation and dissolution of modules over time and to seamlessly compare modules from one recording session to the next. Given such a mapping, one can then calculate measures like the local “network flexibility” [44, 67], which indicates how frequently a given node changes its module assignment across layers. Past studies have used this flexibility measure to identify temporally stable cores and variable peripheries (clusters of nodes with low and high flexibility, respectively) [44]. The multi-layer modularity maximization procedure required that we maintain a consistent set of nodes across layers, i.e. a fixed set of cells that were observed on all recording sessions. This conjunction results in a smaller set of cells than the pairwise comparisons utilized in the previous section. The number of cells retained for the multi-layer analysis was 144 (Mouse 1), 68 (Mouse 2), 67 (Mouse 3), and 54 (Mouse 4).

thumbnail
Fig 3. Estimation of core-periphery structure and network flexibility.

(a) Thresholded correlation matrices are separately treated as: a) layers in a multi-layer network, their communities estimated, and network flexibility estimated as the frequency with which a node changes its community assignment across layers; b) the consistency matrix is submitted to a core-periphery detection algorithm and each node’s “coreness” is estimated. Here, consistency measures the fraction of layers (recording sessions) in which a connection was present. (b) Node’s flexibility scores plotted in anatomical space. (c) Nodes’ “coreness” plotted in anatomical space. The size of nodes in panels b and c is proportional to their average weight across all five recording sessions. (d) Because flexibility is a measure of variability while “coreness” is a measure of stability, we find that the two are inversely correlated with one another (red line represents the identity line). (e) Cross-subject consistency of optimal parameters for fitting the core-periphery model. For each mouse, we calculated the difference between observed core quality and that of a null model, and we retained the top 10% of those points. These points are depicted at the level of individual mice in panel f. In panel e, we aggregate those values across all mice.

https://doi.org/10.1371/journal.pcbi.1007360.g003

Obtaining estimates of network flexibility requires the detection of communities using multi-layer modularity maximization, which depends upon two parameters, γ and ω. These parameters control the resolution (size and number) of modules detected and their stability across layers, respectively. Here, we use a recently developed procedure that allows us to obtain a representative sample from the parameter space defined by these two variables [68]. For each such sample, we calculated a local (node-level) measure of flexibility, ranked the flexibility scores of all nodes, and subsequently averaged these ranked flexibility scores across all samples to generate an average flexibility profile for the population of cells. We note that each layer in this model represents the structure of a network estimated during different recording sessions. The multi-layer model makes it possible to aggregate these different networks into a single mathematical object rather than treating them as independent and disjoint estimates of the network [69]. This framework has been used widely in network neuroscience for modeling nervous systems whose network structure evolves over time [70, 71], differs across individuals [68], and spans multiple association modalities [72, 73].

In addition to the flexibility approach, we also used a second method to provide converging evidence of temporal core-periphery structure (Fig 3a) [74]. In this procedure, we calculated the session-averaged connectivity matrix (over the five recording sessions), and based on its organization we algorithmically assign cells (nodes) to a continuously defined core and periphery (see Materials and methods for more details). Intuitively, core nodes are nodes that maintain strong connections to one another and to the periphery across recording sessions, while peripheral nodes are those whose connections are variable (e.g., observed in only a few recording sessions or absent altogether). The size of the core and the smoothness of the transition from core to periphery are controlled by two free parameters, α and β. We systematically explored this parameter space and at each point, we fit the core-periphery model to the session-averaged network to calculated the core quality [74]. We compare the quality of cores fit to the observed session-averaged matrix against the qualities fit to random matrices generated by a permutation-based null model. This comparison allows us to identify points of interest in the parameter space: points where the observed core was of greater quality than that of the null model.

We note that, here, we focus on temporal core-periphery structure, where the core is comprised of nodes that maintain their connection to one another over time and the periphery is made up of nodes whose connections to one another are variable or infrequent [44]. This definition is distinct from core-periphery structure defined based on topological features alone [75, 76]. In the case of topological core-periphery structure, the core represents a strongly connected cluster of nodes that is weakly connected to a set of peripheral nodes. The peripheral nodes make few connections to one another.

In general, we found evidence that cortical activity in all mice exhibited temporally-stable cores of nodes that maintained community assignments and connectivity over many days. In general, the flexibility measure converged with the coreness measure, implicating roughly the same sets of nodes as temporally stable (i.e., manifesting high coreness, low flexibility) (Fig 3b–3d). Across mice, cores tended to be fairly exclusive (Fig 3e and 3f); core quality was maximally greater than the null model at points in parameter space corresponding to a small subset of cells. At these points in parameter space and across all mice, we found the average core included ≈ 20% of all nodes, an estimate that is obtained after thresholding the sigmoidal coreness measure at its midpoint. We note, however, that as a result of parametric variation we observed cores containing as few as 0.09% of all nodes and as many as 31.6% of nodes. The smoothness of the transition between core and periphery was more variable, suggesting that these networks may exhibit multiple cores with different degrees of smoothness separating the core from the periphery.

The detection of temporal core-periphery structure suggests that there exists a small subset of nodes whose modular organization is preserved across time. To test whether this was the case, we repeated the procedure from the previous section, wherein we calculated the similarity of cell-to-cell correlations across multiple days. Here, we calculated the similarity of connections among core nodes (the top 10% ranked by coreness) and connections among non-core nodes. If the core were indeed stable, we would expect that the connections within the core would be more stable across time compared to the connections within the non-core. Indeed, we found that this was the case. For mice 1–4, we found that the similarity of connections among core nodes were −0.26, +0.92, −0.16 and +0.06, compared to connections involving the non-core nodes, which were −0.29, −0.52, −0.33, −0.76. On average, the difference in correlation between core and non-core was 0.41±0.38. This observation suggests that the core nodes are more temporally stable than the periphery nodes, both in terms of their community structure and in terms of their connectivity patterns.

Discussion

Spontaneous fluctuations in neural activity at the cellular scale can modulate behavioral responses to incoming sensory stimuli [16, 17]. Yet the nature of that modulation is not well understood, in part due to the fact that such spontaneous activity does not appear to be random in nature, but instead displays heterogeneous dependencies or correlations among units. Little is known about the rules constraining the architecture of these correlations, or their variability over time. Here we sought to partially address this gap in knowledge by using recently developed techniques in network science to examine the network architecture of correlations in spontaneous activity in mouse auditory cortex as measured by two-photon microscopy and calcium imaging over the course of several weeks. We found that networks exhibited striking modular architecture, with smaller modules being located within larger modules in a multi-scale hierarchy. We also found significant temporal rearrangement of modular architecture, as indicated by the fact that the similarity in modules decreased monotonically as a function of the time interval between recording sessions, even when only considering those units that were present in both sessions. Finally, we found that the broadly observed temporal rearrangement of modules was complemented by the presence of a small number of cells whose modular allegiance remained stable throughout the 2–4 weeks of experimentation. We confirmed with additional testing that the co-existence of stable and unstable units was consistent with a temporal core-periphery model of system dynamics, where a stable core of units is accompanied by a flexibly periphery.

Multi-scale modular network structure

Biological systems generally and neural systems specifically, are frequently required to develop, adapt, and evolve in changing environments [77, 78]. This pervasive demand for adaptation is thought to be a partial explanation for the striking modular structure observed in biological systems [79, 80]. Each module is thought to have the capacity to change or adapt without adversely impacting the function of other modules. In neural systems, modules are thought to exist in order to segregate specific cognitive function or computations, allowing enhanced specialization of the organism [45, 81]. Such modular structure has also been observed in spontaneous recordings of intact zebrafish larvae, where topographically compact assemblies of functionally similar neurons reflect the tectal retinotopic map despite being independent of retinal drive [82, 83]. These data suggest that spontaneous activity displays modular structure that is a functional adaptation specifically tuned to support the system’s behavior. Similar observations have also recently been made in ferret visual cortex, where widespread modular correlation patterns in spontaneous activity accurately predict the local structure of visually evoked orientation columns several millimeters away [34].

Hierarchical modularity in biological systems is further thought to allow for a decomposability of the system’s temporal responses to the environment, with fast processes occurring in small modules at a low level of the hierarchy and slow processes occurring in large modules at a high level of the hierarchy [84]. Prior work at the large-scale has demonstrated the presence of hierarchically modular structure in neural systems specifically, and suggested that large modules support broad cognitive functions while small modules support specialized cognitive functions [24, 60, 61, 85]. Here we extend these prior observations by showing that over short time periods approximately equal to the duration of a recording session, neurons assemble into cohesive modules of varying size, ranging from large, spatially-distributed clusters of weakly coupled neurons to compact, highly correlated ensembles. In theoretical work, it is interesting to note that hierarchical modularity provides an efficient solution to the problem of evolving adaptable systems while minimizing the cost of connections [86]. This relation between hierarchical modularity and low cost yet efficient information processing in neural systems has also been supported by both theoretical work and analysis of neural data in both C. elegans and human [85]. When considering our results in this light, it is useful to note that the spatial compactedness of modules suggests that maintaining long-distance correlated activity may be metabolically costly and therefore uncommon. Overall, these findings are consistent with those observed in other micro- and macro-scale networks and suggest that the organizational principles of modular architecture and spatially-compact, low-cost clusters may be conserved across spatial scales [45, 87].

Daily variation in network architecture and module constituency

Accompanying the nascent use of tools from network science to understand interaction or connection patterns between neural units, there has been a marked interest in understanding the dynamics of interaction patterns as a function of time, and across a variety of different time scales [21, 58, 88, 89]. Particularly in the human imaging literature, efforts have begun to understand principles of dynamic network reconfiguration on the time scale of minutes or hours [70, 90], days [91, 92], weeks [93], months [46, 94], and years [95, 96]. Here we exercise that interest in the domain of network models of correlation matrices derived from spontaneous activity in mouse auditory cortex over 2 to 4 weeks of experimentation. Our findings suggest that quotidian variation in correlation structure is manifest at multiple scales: (i) at the level of cell-to-cell correlations, but also (ii) at the level of large-scale and module patterns in the network. This latter observation is particularly interesting to consider in light of findings at the macro-scale level of whole-brain networks derived from fMRI data. Specifically, at this large scale, much of the modular organization of spontaneous correlations in the human brain is conserved across the time scales of days and weeks, with notable flexibility largely present at module boundaries. One could speculate that gross temporal stability in macro-scale networks is underpinned by notable micro-scale variability. It would be interesting in future to more directly address the question of the functional role of this micro-scale network reconfiguration, and specifically to test the hypothesis that the correlation structure of fluorescence traces in mouse primary auditory cortex is reorganized over timescales of days to weeks to support cortical functional reorganization.

A stable network core accompanied by a flexible network periphery

In other natural dynamical systems, it has been noted that density tends to support temporal stability, while sparsity tends to support temporal instability [97]. In the context of networked systems, the notion can be expanded to describe the phenomenon in which a core of densely interconnected units tends to display weak or slow temporal fluctuations, while a periphery of sparsely interconnected units tends to display strong or fast temporal fluctuations [44]. In the context of the human brain, this temporal core-periphery structure has been raised as a model for the balanced constraints of task-general processes, implemented by the temporal core, and task-specific processes, implemented by the temporal periphery [98]. It is interesting to consider whether such a delineation into temporal core and periphery is also characteristic of cellular networks, and whether that separation is functionally meaningful in a similar sense. Our findings suggest that, while calcium fluorescence correlation structure changes markedly over time, there remains a relatively small set of cells whose interactions, both as single connections but also as communities, are spared and preserved. There is some evidence in theoretical studies that such core-like structures emerge early in development, and are strengthened through functional activation [99]. In analyses of macro-scale networks, core stability and peripheral flexibility have been associated with learning [44], leading us to speculate that the emergence of core-periphery structure in micro-scale networks may serve a similar role in preserving learned (auditory) relationships, while maintaining enough variability to learn and map novel stimuli. Thus, future work could be directed to investigate the functional roles of cores and peripheries during task conditions.

Our observations are not without precedent and are in line with other previous cellular-level studies. For instance, in a study of Tritonia motor programs, [100] found that in addition to a “core” set of neurons that consistently responded to stimulation, the motor network expanded to include a peripheral set of neurons whose membership varied across trials. The authors went on to show that the variable set of neurons included those that were strongly coupled to the network at rest, suggesting that spontaneous coupling patterns predispose cells to be recruited into the motor program. Similarly, [101] distinguished between sets of cells as “soloists” and “choristers,” which respectively comprise those whose activity was distinct from that of the rest of the population and those whose activity was consistent with that of the bulk of the population. Our finding of temporal core-periphery structure echoes both of these studies. In our study, however, we find analogous cores whose community affinity is preserved over days. Although the functional relevance of the core-periphery structure reported here remains unclear, large-scale studies of brain networks have linked reorganization of these types of structures to learning [44]. Outside of neuroscience, cores are thought to represent polyfunctional units in a network, positioned where modules overlap with one another [102], and to occupy positions of influence within the broader topology [103]. Future studies could better disambiguate the functional role of temporal core-periphery structure by linking those types of structures to behavior.

Methodological considerations and limitations

There are several methodological considerations and limitations that are pertinent to the interpretation and generalizability of our results. First, we note that the experimental methods allow us to sample only a subset of neurons within a specific “slice” of the auditory cortex. It is likely that most of the neurons that directly target the neurons that we image are not captured by the analysis. Therefore, the estimates for the network connectivity should not be taken as an approximation for the actual physical connectivity in the cortical circuit. Another important aspect of data collection is that we focus on a specific cortical layer: layer 2/3. Neurons in the cortex differ tremendously in their connectivity patterns across different layers [104, 105]. It would be important in future studies to sample the activity across cortical depth to better understand integration of information across cortex.

Second, we note that we have examined correlations in spontaneous activity fluorescence traces, and this approach has the strengths of computational simplicity and ease of interpretation [106]. However, we acknowledge that correlation-based approaches focus on pairwise functional interactions, and remaining agnostic to underlying structural connectivity as well as to higher-order (non-pairwise) relations between units. It would be interesting in future to consider maximum entropy models as an alternative method to estimate connections between units [107], both for its sensitivity to underlying structure [108], and for its ability to assess higher-order interactions [109]. Approaches that could then take advantage of the richer assessment of higher order interactions in these data include emerging tools from algebraic topology [110, 111], which have already proven relevant for understanding structure-function relationships at both large and small scales in neural systems [112, 113].

Finally, an additional limitation concerns the measures used to establish the presence or absence of connections between cells. Specifically, we constructed networks where nodes represented individual cells and where edges represented the correlation magnitude of fluorescence traces. Importantly, correlated activity is not a direct proxy for underlying structural connectivity [114], and thus a pair of neurons that may not be directly synaptically connected can exhibit correlated activity; rather than reflecting structural connections, functional connectivity provides information about the interactions between neurons due to their function [18]. Moreover, correlated activity also does not represent the coupling matrix that prescribes the temporal evolution of brain activity [115]. Rather, the correlation structure of neural activity represents the product of a dynamical system whose evolution is constrained by structural connections. Though correlated activity at the large-scale has proven useful for investigating the functional organization of brain networks [30, 116, 117], its utility for understanding and characterizing the structure and function of micro-scale networks remains unclear and largely untested [118]. Future studies should both investigate in greater detail the relative advantages of alternative, domain-specific measures of functional connectivity [119, 120] and the relationship of these measures to other connection modalities [121].

A related limitation concerns the properties of network connections or edges. Here, we used fully weighted and signed representations to calculate network similarity (see Fig 2) and sparse, binary networks to detect modules and core-periphery structure (See Figs 1 and 3). We adopted these particular edge definitions because they allowed us to compute similarity using the full distribution of edge weights and to avoid making future, somewhat arbitrary, decisions about the appropriate null model for weighted and signed networks in modularity maximization [122]. Nonetheless, there exists a spectrum of possible ways to define edges and their properties. Future work is needed to understand the tradeoffs between different definitions and to ultimately ground edge definition in biophysical properties of nervous systems.

Conclusion

Across many scientific disciplines from plant biology [123] to biogeodynamics [124] and the study of biodiversity [125], scientists are faced with the challenge of bridging two or more scales of investigation into the function of complex systems. For example, in evolutionary biology, a key challenge is to bridge physical scales from protein sequences to fitness of organisms and populations [126], while in the study of cancer progression a key challenge is to map genotype to phenotype [127]. Neuroscience is no exception. Ongoing efforts seek to bridge the gap between the connectome and the transcriptome [128], between brains and social groups [129], or between large-scale brain regions and small-scale cellular circuitry [130]. In each case, the development of a formal understanding will depend upon the capacity to build mathematical descriptions and theories across scales. One natural approach to this challenge is to use a formalism that is scale invariant, a characteristic that makes network science particularly appealing. Our work in this study is an example of considering tools and conceptual paradigms previously exercised at the large-scale of brain regions, and exercising them at the level of cellular circuitry. We look forward to future efforts explicitly measuring and examining the network architecture of neural systems across both of these scales simultaneously in the same animal, with the goal of better understanding and predicting behavior.

Methods

Ethics statement

All experimental procedures were performed in accordance with NIH guidelines and approved by the IACUC at the University of Pennsylvania.

Animals

All experiments were performed with equal numbers of adult male and female mice (supplier—Jackson Laboratories; age, 12-22 weeks; weight, 20-36 g; PV-Cre mice, strain: B6;129P2-Pvalbtm1(cre)Arbr/J).

Two-photon microscopy and calcium imaging

Four mice were implanted with cranial windows over auditory cortex. Briefly, the mice were anaesthetized with 1.5-3% isoflurane and a 3mm circular craniotomy was performed over auditory cortex (stereotaxic coordinates) using a 3mm biopsy punch. An adeno-associated virus (AAV) vector encoding the calcium indicator GCaMP6s (AAV1-SYN-GCAMP6s, UPENN vector core) was injected for expression in layer 2/3 neurons in left A1 within the window (750 nl, 1.89 × 10-12 genome copies per ml) [131]. After injection a glass circular 3mm coverslip (size 0, Warner Instruments) was placed in the craniotomy and fixed in place using a mix of Krazy glue and dental cement. A custom-made stainless steel head-plate (eMachine Shop) was fixed to the skull using C&B Metabond dental cement (Parkell). All imaging sessions were carried out inside a single-walled acoustic isolation booth (Industrial Acoustics) as previously described. Mice were placed in the imaging setup, and the headpost was secured to a custom base (eMachine Shop) serving to immobilize the head. Mice were gradually habituated to the apparatus over 3 days, 3-4 weeks after surgery.

Using two-photon microscopy (Ultima in vivo multiphoton microscope, Bruker) changes in fluorescence of GCaMP6s in transfected neurons caused by fluctuations in calcium activity were recorded in awake, head-fixed mice. We recorded from the same cells over many days in layer 2/3 of auditory cortex, using blood vessel architecture, depth from the surface, and the shape of cells to return to the same imaging site. Laser power at the brain surface was kept below 30 mW. Chronic imaging of the same field of view across days was carried out for the duration of the experiment.

Recordings were made at 512×512 pixels and 13-bit resolution at approximately 30 frames per second. Spontaneous activity was recorded for 10 minutes in each session. Publicly available toolboxes [132] were used to register the resulting images, select regions of interest, estimate neuropil contamination, and extract the changes in fluorescence from each cell. Upon conclusion of the imaging sessions, brains were extracted following perfusion in 0.01M phosphate buffer pH 7.4 (PBS) and 4% paraformaldehyde (PFA), post-fixed in PFA overnight and cryopreserved in 30% sucrose solution for 2 days prior to slicing. The location and spread of GCaMP6s was confirmed through fluorescent imaging. These methods are consistent with the recommendations of the American Veterinary Medical Association (AVMA) Guidelines on Euthanasia.

Cell tracking over days

To identify ROIs from different imaging sessions that correspond to the same cell we performed a multi-step routine: (1) The mean fluorescence images from each day were registered by transforming the coordinates of landmarks present in both images in MATLAB (2017a) using the fitgeotrans function. The resulting transformation was used to transform the ROIs from the second imaging session to match the first; all sessions were aligned to the first imaging session. (2) We calculated the distance between all pairs of centroids across the two sessions. For each ROI from session 2, we computed the percentage overlap of the 10 cells with the smallest centroid distances from session 1. Cells that had more than 1 ROI with higher than 20% overlap were manually inspected; the ROI that matched the current cell was selected from the overlapping ROIs or none were selected if it was unclear whether they were the same cell. A good match was determined by considering the percent overlap and the shape of the ROIs. All other cells were assigned the closest ROI as matching. (3) We manually inspected any cells that had duplicate matching ROIs; again considering the shape and the percent overlap of matching ROIs, we selected the ROI that matched that cell, or we decided that none was a good match. (4) To check for false positive matches, we manually inspected any matches where the centroid distances were greater than the mean + 1 std of all the matches, and any matches that had less than 30% ROI overlap; the match was deemed good or not depending on the match criteria. Neurons that were not matched to any ROI were counted as different or new and assigned a new cell number. This process was repeated for subsequent sessions, registering the imaging field to the first session and comparing the ROIs to the cumulative ROIs from previous sessions. A final manual inspection of all the unique ROIs was performed after all the imaging sessions were registered; ROIs that overlapped were excluded from the dataset since it was unclear whether they were the same or different cells. Examples of tracked cells and aligned ROIs are shown in S2 Fig.

Network reconstruction

We estimated functional connectivity from fluorescence traces. Let xi(t) indicate the intensity of fluorescence in cell i at time t. Next, we computed the cross-correlation of differenced fluorescence traces for every pair of cells: (1) where μi and σi are the mean and standard deviation of the differenced time series, respectively. Here, “differenced activity” refers to a preprocessing strategy sometimes used to reduce serial (auto) correlation in time series data and to make a time series approximately stationary (i.e. constant mean over time) [133]. Given a time series of length T defined as x = [x1, …, xT], the differenced time series has length T − 1 and is calculated as xdiff = [x2x1, …, xTxT−1]. This difference transform is particularly appropriate when networks are constructed using Pearson correlations, where time series are assumed to be stationary and to contain uncorrelated samples.

To reduce the likelihood that the observed correlations were driven by chance fluctuations, we “jittered” the time series of cells by adding or subtracting < 1 second offsets to each cell independently. We then computed jittered cross correlations, . We repeated this procedure 1000 times. More specifically, the jittering procedure generates surrogate time series by iterating cell by cell, selecting a random starting index, p, that occurs within the first 2 seconds (the first 60 samples given the 1 second offset and 30 Hz sampling rate), and retaining the next T − 2p − 1 samples. The resulting time series have length T − 2p. The jittered time series underwent the same differencing procedure as the original time series.

We estimated for every pair of cells the probability that the jittering procedure would generate a correlation as strong as that which was observed empirically, and we made binary connections between those cells with p < 0.05. This procedure resulted in a sparse matrix, with elements Aij ∈ [0, 1]. We note that the fully-weighted and signed matrices were used in the analysis of the day-to-day network similarity. The thresholded, binary, and sparse matrices were used for community and core-periphery detection due to the complexity of the algorithms.

Module detection

We used modularity maximization to detect network modules based on connectivity data [55]. This method aims to divide network nodes (cells) into modules whose internal density of connections is maximally greater than what would be expected under a null model. This intuition is formalized by the modularity quality function [134]: (2)

In this equation, ki = ∑j Aij is the degree of node i and 2m = ∑i ki is the total number of connections in the network. The term gives the expected number of connections between node i and node j given the null model in which each node’s degree is preserved but connections are formed at random. The resolution parameter, γ, scales the relative contribution of the null model. The module assignment of node i is encoded as gi and δ(gi, gj) is the Kronecker delta, whose value is equal to unity when gi = gj and is zero otherwise.

In this manuscript, we used two variants of modularity maximization. First, we studied the network community structure for each recording session independently. For this analysis, we combined modularity maximization with a newly-developed multi-resolution technique that divides the network into communities of different sizes (scales) that are related to one another hierarchically [135]. This procedure allows us to examine community structure across a range of scales, from large communities to smaller communities that might support more specialized information processing.

Additionally, we used a multi-layer variant of modularity maximization that makes it possible to track the evolution, formation, and dissolution of communities across recording sessions [66]. In this procedure, the standard modularity maximization equation is modified to read: (3) Here, the subscript s denotes network layers, u ∈ {1, …, T}. That is, each network layer corresponds to connectivity patterns estimated on a different day. So Aiju represents the presence or absence and weight of the connection between node i and node j in layer u. Similarly, kiu = ∑j Aiju is the degree of node i in layer u and giu is the community to which node i is assigned in layer u. Unique to the multi-layer variant of modularity maximization is the inter-layer coupling parameter ω, which links node i to itself across layers. From the perspective of maximizing Q, non-zero values of ω make it advantageous to group node i into the same community across layers. When ω is small, the advantage is correspondingly small, and the detected communities emphasize the unique community structure of layers. On the other hand, when ω is large, the detected communities are consistent across layers and emphasize shared features of community structure.

Here, we used a recently-developed procedure to obtain estimates of community structure with the values of {γ, ω} sampled from a restricted parameter space [68]. This procedure involved first estimating the boundaries of a restricted parameter space wherein any {γ, ω} pair would result in community structure where the number of communities is > 1 and < N × T (where T is the total number of layers; T = 6, in this case), and where community structure is neither uniform across layers (flexibility of exactly 0) nor is it maximally dissimilar (flexibility of exactly 1). See [24] for more details on how these boundaries were estimated. We then sampled 10000 {γ, ω} pairs from within this parameter space and for each sample we maximized the corresponding Q(γ, ω). All subsequent analyses were carried out on these detected communities.

The principle advantage of the multi-layer formulation is that it estimates communities for all layers simultaneously and preserves nodes’ community labels across layers. This advantage makes it possible to directly compare the community assignment of a given node in layer u and in layer vu, and to identify nodes whose community assignments are flexible (varying from one layer to another) or inflexible (remaining in the same community across layers). We can quantify this intuition using the network measure flexibility [67, 70]: (4) Intuitively, flexibility counts the fraction of times that nodes’ community assignments in layers u and u + 1 differ. Nodes that differ more frequently have flexibility values closer to 1, while nodes that differ less frequently have flexibility values closer to 0. Here, we used the flexibility measure as an index of change in network community structure across recording sessions.

Co-assignment probability

Based on the 10,000 partitions generated using single-layer modularity maximization, we were able to estimate the probability that nodes i and j were assigned to the same module. Let cih denote the community assignment of node i in partition h ∈ 1, …, 10000. The co-assignment probability of nodes i and j was calculated as: (5) The value of Cij ranges from 0 to 1, indicating that nodes i and j were never assigned to the same module or were always assigned to the same module, respectively.

Core-periphery detection

Separately, we also characterized the stability of network organization across recording sessions by computing a temporal core and periphery. In this context, a core refers to a group of nodes that are densely internally connected and to the periphery, which is weakly internally connected [103]. To identify temporal core-periphery structure, we first generated a connection consistency matrix, whose element Gij = ∑u Aiju represented the fraction of layers (recording sessions) in which a network connection was present. In this matrix, a core refers to a group of nodes whose connections are maintained across time, while the periphery is a set of nodes whose connections are more variable.

We used a variant of a common core-periphery definition in which the transition from core to periphery varies smoothly (non-binary). We begin by defining the N × 1 vector Ci of non-negative elements [74]. Given this vector, we then defined the matrix Cij = CiCj subject to the constraint that ∑ij Cij = 1. The values in the vector C are permutations of the vector: (6) The coreness of each node is the permutation of that maximizes the core quality function: (7) This method introduces two free parameters, α ∈ [0, 1] and β ∈ [0, 1]. The value of α determines the sharpness of the core-periphery boundary. With α = 1, the transition is binary while the transition with α = 0 is maximally fuzzy. Similarly, the value of β determines the size of the core; as β ranges from 0 to 1, the size of the core varies from N to 0. In our application, we performed a grid search of 31 linearly-spaced values of α and β, using a simulated annealing algorithm to maximize R (with 10 restarts).

Supporting information

S1 Fig. Module similarity at different hierarchical levels.

https://doi.org/10.1371/journal.pcbi.1007360.s001

(TIF)

S2 Fig. Examples of cell tracking over days.

https://doi.org/10.1371/journal.pcbi.1007360.s002

(TIF)

References

  1. 1. Schneidman E, Puchalla JL, Segev R, Harris RA, Bialek W, Berry MJn. Synergy from silence in a combinatorial neural code. J Neurosci. 2011;31(44):15732–15741. pmid:22049416
  2. 2. Montijn JS, Meijer GT, Lansink CS, Pennartz CM. Population-Level Neural Codes Are Robust to Single-Neuron Variability from a Multidimensional Coding Perspective. Cell Rep. 2016;16(9):2486–2498. pmid:27545876
  3. 3. Ganmor E, Segev R, Schneidman E. A thesaurus for a neural population code. Elife. 2015;4(06134). pmid:26347983
  4. 4. Schneidman E, Bialek W, Berry MJn. Synergy, redundancy, and independence in population codes. J Neurosci. 2003;23(37):11539–11553. pmid:14684857
  5. 5. Kohn A, Coen-Cagli R, Kanitscheider I, Pouget A. Correlations and Neuronal Population Information. Annu Rev Neurosci. 2016;39:237–256. pmid:27145916
  6. 6. Averbeck BB, Lee D. Effects of noise correlations on information encoding and decoding. J Neurophysiol. 2006;95(6):3633–3644. pmid:16554512
  7. 7. van Bergen RS, Jehee JFM. Modeling correlated noise is necessary to decode uncertainty. Neuroimage. 2018;180(Pt A):78–87. pmid:28801251
  8. 8. Eyherabide HG, Samengo I. When and why noise correlations are important in neural decoding. J Neurosci. 2013;33(45):17921–17936. pmid:24198380
  9. 9. Kanitscheider I, Coen-Cagli R, Pouget A. Origin of information-limiting noise correlations. Proc Natl Acad Sci U S A. 2015;112(50):E6973–E6982. pmid:26621747
  10. 10. Ferezou I, Deneux T. How do spontaneous and sensory-evoked activities interact? Neurophotonics. 2017;4(3):031221. pmid:28630882
  11. 11. Orbán G, Berkes P, Fiser J, Lengyel M. Neural variability and sampling-based probabilistic representations in the visual cortex. Neuron. 2016;92(2):530–543. pmid:27764674
  12. 12. Berkes P, Orbán G, Lengyel M, Fiser J. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science. 2011;331(6013):83–87. pmid:21212356
  13. 13. Brody CD. Disambiguating different covariation types. Neural Comput. 1999;11(7):1527–1535. pmid:10490936
  14. 14. Brody CD. Correlations without synchrony. Neural Comput. 1999;11(7):1537–1551. pmid:10490937
  15. 15. Grewe J, Kruscha A, Lindner B, Benda J. Synchronous spikes are necessary but not sufficient for a synchrony code in populations of spiking neurons. Proc Natl Acad Sci U S A. 2017;114(10):E1977–E1985. pmid:28202729
  16. 16. Arieli A, Shoham D, Hildesheim R, Grinvald A. Coherent spatiotemporal patterns of ongoing activity revealed by real-time optical imaging coupled with single-unit recording in the cat visual cortex. J Neurophysiol. 1995;73(5):2072–2093. pmid:7623099
  17. 17. Arieli A, Sterkin A, Grinvald A, Aertsen A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science. 1996;273(5283):1868–1871. pmid:8791593
  18. 18. Bassett DS, Mattar MG. A Network Neuroscience of Human Learning: Potential to Inform Quantitative Theories of Brain and Behavior. Trends Cogn Sci. 2017;21(4):250–264. pmid:28259554
  19. 19. Newman MEJ. Networks: An Introduction. Oxford University Press; 2010.
  20. 20. Butts CT. Revisiting the foundations of network analysis. Science. 2009;325 (5939). pmid:19628855
  21. 21. Khambhati AN, Sizemore AE, Betzel RF, Bassett DS. Modeling and interpreting mesoscale network dynamics. Neuroimage. 2018;180(Pt B):337–349. pmid:28645844
  22. 22. Vaiana M, Muldoon SF. Multilayer brain networks. Journal of Nonlinear Science. 2018; p. 1–23.
  23. 23. Bassett DS, Zurn P, Gold JI. On the nature and use of models in network neuroscience. Nat Rev Neurosci. 2018;19(9):566–578. pmid:30002509
  24. 24. Betzel RF, Bassett DS. Multi-scale brain networks. Neuroimage. 2017;160:73–83. pmid:27845257
  25. 25. Mann K, Gallen CL, Clandinin TR. Whole-Brain Calcium Imaging Reveals an Intrinsic Functional Network in Drosophila. Curr Biol. 2017;27(15):2389–2396.e4. pmid:28756955
  26. 26. Goltstein PM, Montijn JS, Pennartz CM. Effects of isoflurane anesthesia on ensemble patterns of Ca2+ activity in mouse v1: reduced direction selectivity independent of increased correlations in cellular activity. PLoS One. 2015;10(2):e0118277. pmid:25706867
  27. 27. Runfeldt MJ, Sadovsky AJ, MacLean JN. Acetylcholine functionally reorganizes neocortical microcircuits. J Neurophysiol. 2014;112(5):1205–1216. pmid:24872527
  28. 28. Modi MN, Dhawale AK, Bhalla US. CA1 cell activity sequences emerge after reorganization of network correlation structure during associative learning. Elife. 2014;3:e01982. pmid:24668171
  29. 29. Warp E, Agarwal G, Wyart C, Friedmann D, Oldfield CS, Conner A, et al. Emergence of patterned activity in the developing zebrafish spinal cord. Curr Biol. 2012;22(2):93–102. pmid:22197243
  30. 30. Vanni MP, Chan AW, Balbi M, Silasi G, Murphy TH. Mesoscale mapping of mouse cortex reveals frequency-dependent cycling between distinct macroscale functional modules. Journal of Neuroscience. 2017; p. 3560–16.
  31. 31. McVea DA, Murphy TH, Mohajerani MH. Large scale cortical functional networks associated with slow-wave and spindle-burst-related spontaneous activity. Frontiers in neural circuits. 2016;10:103. pmid:28066190
  32. 32. Humphries MD. Dynamical networks: finding, measuring, and tracking neural population activity using network science. Network Neuroscience. 2017;1(4):324–338. pmid:30090869
  33. 33. Malmersjo S, Rebellato P, Smedler E, Uhlen P. Small-world networks of spontaneous Ca(2+) activity. Commun Integr Biol. 2013;6(4):e24788. pmid:23986813
  34. 34. Smith GB, Hein B, Whitney DE, Fitzpatrick D, Kaschube M. Distributed network interactions and their emergence in developing neocortex. Nat Neurosci. 2018;21(11):1600–1608. pmid:30349107
  35. 35. Chiang CW, Chen YC, Lu JC, Hsiao YT, Chang CW, Huang PC, et al. Synaptotagmin I regulates patterned spontaneous activity in the developing rat retina via calcium binding to the C2AB domains. PLoS One. 2012;7(10):e47465. pmid:23091625
  36. 36. Mao BQ, Hamzei-Sichani F, Aronov D, Froemke RC, Yuste R. Dynamics of spontaneous activity in neocortical slices. Neuron. 2001;32(5):883–898. pmid:11738033
  37. 37. Funamizu A, Kanzaki R, Takahashi H. Distributed representation of tone frequency in highly decodable spatio-temporal activity in the auditory cortex. Neural Netw. 2011;24(4):321–332. pmid:21277165
  38. 38. Allen EJ, Burton PC, Olman CA, Oxenham AJ. Representations of Pitch and Timbre Variation in Human Auditory Cortex. J Neurosci. 2017;37(5):1284–1293. pmid:28025255
  39. 39. Caruso VC, Mohl JT, Glynn C, Lee J, Willett SM, Zaman A, et al. Single neurons may encode simultaneous stimuli by switching between activity patterns. Nat Commun. 2018;9(1):2715. pmid:30006598
  40. 40. Carrillo-Reid L, Yang W, Kang Miller JE, Peterka DS, Yuste R. Imaging and Optically Manipulating Neuronal Ensembles. Annu Rev Biophys. 2017;46:271–293. pmid:28301770
  41. 41. Gu S, Pasqualetti F, Cieslak M, Telesford QK, Yu AB, Kahn AE, et al. Controllability of structural brain networks. Nat Commun. 2015;6:8414. pmid:26423222
  42. 42. Yan G, Vertes PE, Towlson EK, Chew YL, Walker DS, Schafer WR, et al. Network control principles predict neuron function in the Caenorhabditis elegans connectome. Nature. 2017;550(7677):519–523. pmid:29045391
  43. 43. Kim JZ, Soffer JM, Kahn AE, Vettel JM, Pasqualetti F, Bassett DS. Role of Graph Architecture in Controlling Dynamical Networks with Applications to Neural Systems. Nat Phys. 2018;14:91–98. pmid:29422941
  44. 44. Bassett DS, Wymbs NF, Rombach MP, Porter MA, Mucha PJ, Grafton ST. Task-based core-periphery organization of human brain dynamics. PLoS Comput Biol. 2013;9(9):e1003171. pmid:24086116
  45. 45. Sporns O, Betzel RF. Modular brain networks. Annual review of psychology. 2016;67:613–640. pmid:26393868
  46. 46. Betzel RF, Satterthwaite TD, Gold JI, Bassett DS. Positive affect, surprise, and fatigue are correlates of network flexibility. Sci Rep. 2017;7(1):520. pmid:28364117
  47. 47. Bertolero MA, Yeo BT, D’Esposito M. The modular and integrative functional architecture of the human brain. Proc Natl Acad Sci U S A. 2015;112(49):E6798–807. pmid:26598686
  48. 48. Gallen CL, Turner GR, Adnan A, D’Esposito M. Reconfiguration of brain network architecture to support executive control in aging. Neurobiol Aging. 2016;44:42–52. pmid:27318132
  49. 49. Arnemann KL, Chen AJ, Novakovic-Agopian T, Gratton C, Nomura EM, D’Esposito M. Functional brain network modularity predicts response to cognitive training after brain injury. Neurology. 2015;84(15):1568–1574. pmid:25788557
  50. 50. van den Heuvel MP, Bullmore ET, Sporns O. Comparative Connectomics. Trends Cogn Sci. 2016;20(5):345–361. pmid:27026480
  51. 51. Betzel RF, Bassett DS. Specificity and robustness of long-distance connections in weighted, interareal connectomes. Proc Natl Acad Sci U S A. 2018;115(21):E4880–E4889. pmid:29739890
  52. 52. Billeh YN, Schaub MT, Anastassiou CA, Barahona M, Koch C. Revealing cell assemblies at multiple levels of granularity. Journal of neuroscience methods. 2014;236:92–106. pmid:25169050
  53. 53. Scholtens LH, Schmidt R, de Reus MA, van den Heuvel MP. Linking macroscale graph analytical organization to microscale neuroarchitectonics in the macaque connectome. J Neurosci. 2014;34(36):12192–12205. pmid:25186762
  54. 54. Scholtens LH, van den Heuvel MP. Multimodal Connectomics in Psychiatry: Bridging Scales From Micro to Macro. Biol Psychiatry Cogn Neurosci Neuroimaging. 2018;3(9):767–776. pmid:29779726
  55. 55. Newman MEJ, Girvan M. Finding and evaluating community structure in networks. Phys Rev E. 2004;69(026113).
  56. 56. Porter MA, Onnela JP, Mucha PJ. Communities in Networks. Notices of the American Mathematical Society. 2009;56(9):1082–1097, 1164–1166.
  57. 57. Fortunato S, Hric D. Community detection in networks: A user guide. Physics Reports. 2016;659:1–44.
  58. 58. Sizemore AE, Bassett DS. Dynamic graph metrics: Tutorial, toolbox, and tale. Neuroimage. 2018;180(Pt B):417–427. pmid:28698107
  59. 59. Holme P, Saramaki J. Temporal Networks. Phys Rep. 2012;519:97–125.
  60. 60. Zhou C, Zemanová L, Zamora G, Hilgetag CC, Kurths J. Hierarchical organization unveiled by functional connectivity in complex brain networks. Physical review letters. 2006;97(23):238103. pmid:17280251
  61. 61. Meunier D, Lambiotte R, Fornito A, Ersche K, Bullmore ET. Hierarchical modularity in human brain functional networks. Frontiers in neuroinformatics. 2009;3:37. pmid:19949480
  62. 62. Fortunato S. Community detection in graphs. Physics reports. 2010;486(3-5):75–174.
  63. 63. Muldoon SF, Soltesz I, Cossart R. Spatially clustered neuronal assemblies comprise the microstructure of synchrony in chronically epileptic networks. Proceedings of the National Academy of Sciences. 2013;110(9):3567–3572.
  64. 64. Mantel N. The detection of disease clustering and a generalized regression approach. Cancer research. 1967;27(2 Part 1):209–220. pmid:6018555
  65. 65. De Domenico M. Multilayer modeling and analysis of human brain networks. GigaScience. 2017;6(5):1–8. pmid:28327916
  66. 66. Mucha PJ, Richardson T, Macon K, Porter MA, Onnela JP. Community structure in time-dependent, multiscale, and multiplex networks. science. 2010;328(5980):876–878. pmid:20466926
  67. 67. Bassett DS, Porter MA, Wymbs NF, Grafton ST, Carlson JM, Mucha PJ. Robust detection of dynamic community structure in networks. Chaos: An Interdisciplinary Journal of Nonlinear Science. 2013;23(1):013142.
  68. 68. Betzel RF, Bertolero MA, Gordon EM, Gratton C, Dosenbach NU, Bassett DS. The community structure of functional brain networks exhibits scale-specific patterns of variability across individuals and time. bioRxiv. 2018; p. 413278.
  69. 69. Kivelä M, Arenas A, Barthelemy M, Gleeson JP, Moreno Y, Porter MA. Multilayer networks. Journal of complex networks. 2014;2(3):203–271.
  70. 70. Bassett DS, Wymbs NF, Porter MA, Mucha PJ, Carlson JM, Grafton ST. Dynamic reconfiguration of human brain networks during learning. Proceedings of the National Academy of Sciences. 2011;108(18):7641–7646.
  71. 71. Betzel RF, Mišić B, He Y, Rumschlag J, Zuo XN, Sporns O. Functional brain modules reconfigure at multiple scales across the human lifespan. arXiv preprint arXiv:151008045. 2015;.
  72. 72. Bentley B, Branicky R, Barnes CL, Chew YL, Yemini E, Bullmore ET, et al. The multilayer connectome of Caenorhabditis elegans. PLoS computational biology. 2016;12(12):e1005283. pmid:27984591
  73. 73. Battiston F, Nicosia V, Chavez M, Latora V. Multilayer motif analysis of brain networks. Chaos: An Interdisciplinary Journal of Nonlinear Science. 2017;27(4):047404.
  74. 74. Rombach P, Porter MA, Fowler JH, Mucha PJ. Core-periphery structure in networks (revisited). SIAM Review. 2017;59(3):619–646.
  75. 75. Betzel RF, Bertolero MA, Bassett DS. Non-assortative community structure in resting and task-evoked functional brain networks. bioRxiv. 2018; p. 355016.
  76. 76. Betzel RF, Medaglia JD, Bassett DS. Diversity of meso-scale architecture in human and non-human connectomes. Nature communications. 2018;9(1):346. pmid:29367627
  77. 77. Kirschner M, Gerhart J. Evolvability. Proc Natl Acad Sci U S A. 1998;95(15):8420–8427. pmid:9671692
  78. 78. Wagner GP, Altenberg L. Complex adaptations and the evolution of evolvability. Evolution. 1996;50(3):967–976. pmid:28565291
  79. 79. Lipson H, Pollack JB, Suh NP. On the origin of modular variation. Evolution. 2002;56(8):1549–1556. pmid:12353747
  80. 80. Lorenz DM, Jeng A, Deem MW. The emergence of modularity in biological systems. Phys Life Rev. 2011;8(2):129–160. pmid:21353651
  81. 81. Thivierge JP, Cisek P. Nonperiodic synchronization in heterogeneous networks of spiking neurons. Journal of Neuroscience. 2008;28(32):7968–7978. pmid:18685022
  82. 82. Romano SA, Pietri T, Perez-Schuster V, Jouary A, Haudrechy M, Sumbre G. Spontaneous neuronal network dynamics reveal circuit’s functional adaptations for behavior. Neuron. 2015;85(5):1070–1085. pmid:25704948
  83. 83. Pietri T, Romano SA, Perez-Schuster V, Boulanger-Weill J, Candat V, Sumbre G. Cell Rep. 2017;19(5):939–948. pmid:28467907
  84. 84. Simon HA. The Architecture of Complexity. Proceedings of the American Philosophical Society. 1962;106(6):467–482.
  85. 85. Bassett DS, Greenfield DL, Meyer-Lindenberg A, Weinberger DR, Moore SW, Bullmore ET. Efficient physical embedding of topologically complex information processing networks in brains and computer circuits. PLoS Comput Biol. 2010;6(4):e1000748. pmid:20421990
  86. 86. Mengistu H, Huizinga J, Mouret JB, Clune J. The Evolutionary Origins of Hierarchy. PLoS Comput Biol. 2016;12(6):e1004829. pmid:27280881
  87. 87. Bullmore E, Sporns O. The economy of brain network organization. Nat Rev Neurosci. 2012;13(5):336–349. pmid:22498897
  88. 88. Kopell NJ, Gritton HJ, Whittington MA, Kramer MA. Beyond the connectome: the dynome. Neuron. 2014;83(6):1319–1328. pmid:25233314
  89. 89. Calhoun VD, Miller R, Pearlson G, Adali T. The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron. 2014;84(2):262–274. pmid:25374354
  90. 90. Xie H, Calhoun VD, Gonzalez-Castillo J, Damaraju E, Miller R, Bandettini PA, et al. Whole-brain connectivity dynamics reflect both task-specific and individual-specific modulation: A multitask study. Neuroimage. 2018;180(Pt B):495–504. pmid:28549798
  91. 91. Mattar MG, Thompson-Schill SL, Bassett DS. The network architecture of value learning. Netw Neurosci. 2018;2(2):128–149. pmid:30215030
  92. 92. Heitger MH, Ronsse R, Dhollander T, Dupont P, Caeyenberghs K, Swinnen SP. Motor learning-induced changes in functional brain connectivity as revealed by means of graph-theoretical network analysis. Neuroimage. 2012;61(3):633–650. pmid:22503778
  93. 93. Bassett DS, Yang M, Wymbs NF, Grafton ST. Learning-induced autonomy of sensorimotor systems. Nat Neurosci. 2015;18(5):744–751. pmid:25849989
  94. 94. Poldrack RA, Laumann TO, Koyejo O, Gregory B, Hover A, Chen MY, et al. Long-term neural and physiological phenotyping of a single human. Nat Commun. 2015;6:8885. pmid:26648521
  95. 95. Di Martino A, Fair DA, Kelly C, Satterthwaite TD, Castellanos FX, Thomason ME, et al. Unraveling the miswired connectome: a developmental perspective. Neuron. 2014;83(6):1335–1353. pmid:25233316
  96. 96. Zuo XN, He Y, Betzel RF, Colcombe S, Sporns O, Milham MP. Human Connectomics across the Life Span. Trends Cogn Sci. 2017;21(1):32–45. pmid:27865786
  97. 97. Huntsman BM, Petty JT. Density-dependent regulation of brook trout population dynamics along a core-periphery distribution gradient in a central Appalachian watershed. PLoS One. 2014;9(3):e91673. pmid:24618602
  98. 98. Fedorenko E, Thompson-Schill SL. Reworking the language network. Trends Cogn Sci. 2014;18(3):120–126. pmid:24440115
  99. 99. Fuchs E, Ayali A, Ben-Jacob E, Boccaletti S. The formation of synchronization cliques during the development of modular neural networks. Phys Biol. 2009;6(3):036018. pmid:19648664
  100. 100. Hill ES, Vasireddi SK, Wang J, Bruno AM, Frost WN. Memory formation in Tritonia via recruitment of variably committed neurons. Current Biology. 2015;25(22):2879–2888. pmid:26549261
  101. 101. Okun M, Steinmetz NA, Cossell L, Iacaruso MF, Ko H, Barthó P, et al. Diverse coupling of neurons to populations in sensory cortex. Nature. 2015;521(7553):511. pmid:25849776
  102. 102. Yang J, Leskovec J. Overlapping communities explain core–periphery organization of networks. Proceedings of the IEEE. 2014;102(12):1892–1902.
  103. 103. Borgatti SP, Everett MG. Models of core/periphery structures. Social networks. 2000;21(4):375–395.
  104. 104. Llano DA, Sherman SM. Differences in intrinsic properties and local network connectivity of identified layer 5 and layer 6 adult mouse auditory corticothalamic neurons support a dual corticothalamic projection hypothesis. Cerebral cortex. 2009;19(12):2810–2826. pmid:19351905
  105. 105. Theyel BB, Lee CC, Sherman SM. Specific and non-specific thalamocortical connectivity in the auditory and somatosensory thalamocortical slices. Neuroreport. 2010;21(13):861. pmid:20647961
  106. 106. Zalesky A, Fornito A, Bullmore E. On the use of correlation as a measure of network connectivity. Neuroimage. 2012;60(4):2096–2106. pmid:22343126
  107. 107. Onken A, Dragoi V, Obermayer K. A maximum entropy test for evaluating higher-order correlations in spike counts. PLoS Comput Biol. 2012;8(6):e1002539. pmid:22685392
  108. 108. Watanabe T, Hirose S, Wada H, Imai Y, Machida T, Shirouzu I, et al. A pairwise maximum entropy model accurately describes resting-state human brain networks. Nat Commun. 2013;4:1370. pmid:23340410
  109. 109. Ganmor E, Segev R, Schneidman E. Sparse low-order interaction network underlies a highly correlated and learnable neural population code. Proc Natl Acad Sci U S A. 2011;108(23):9679–9684. pmid:21602497
  110. 110. Giusti C, Ghrist R, Bassett DS. Two’s company, three (or more) is a simplex: Algebraic-topological tools for understanding higher-order structure in neural data. J Comput Neurosci. 2016;41(1):1–14. pmid:27287487
  111. 111. Sizemore AE, Phillips-Cremins JE, Ghrist R, Bassett DS. The importance of the whole: Topological data analysis for the network neuroscientist. Network Neuroscience. 2018;Epub Ahead of Print.
  112. 112. Sizemore AE, Giusti C, Kahn A, Vettel JM, Betzel RF, Bassett DS. Cliques and cavities in the human connectome. J Comput Neurosci. 2018;44(1):115–145. pmid:29143250
  113. 113. Reimann MW, Nolte M, Scolamiero M, Turner K, Perin R, Chindemi G, et al. Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function. Front Comput Neurosci. 2017;11:48. pmid:28659782
  114. 114. Goñi J, van den Heuvel MP, Avena-Koenigsberger A, de Mendizabal NV, Betzel RF, Griffa A, et al. Resting-brain functional connectivity predicted by analytic measures of network communication. Proceedings of the National Academy of Sciences. 2014;111(2):833–838.
  115. 115. Cole MW, Ito T, Bassett DS, Schultz DH. Activity flow over resting-state networks shapes cognitive task activations. Nature neuroscience. 2016;19(12):1718. pmid:27723746
  116. 116. Craddock RC, Jbabdi S, Yan CG, Vogelstein JT, Castellanos FX, Di Martino A, et al. Imaging human connectomes at the macroscale. Nature methods. 2013;10(6):524. pmid:23722212
  117. 117. Horwitz B. The elusive concept of brain connectivity. Neuroimage. 2003;19(2):466–470. pmid:12814595
  118. 118. Schröter M, Paulsen O, Bullmore ET. Micro-connectomics: probing the organization of neuronal networks at the cellular scale. Nature Reviews Neuroscience. 2017;18(3):131. pmid:28148956
  119. 119. Stetter O, Battaglia D, Soriano J, Geisel T. Model-free reconstruction of excitatory neuronal connectivity from calcium imaging signals. PLoS computational biology. 2012;8(8):e1002653. pmid:22927808
  120. 120. Feldt S, Bonifazi P, Cossart R. Dissecting functional connectivity of neuronal microcircuits: experimental and theoretical insights. Trends in neurosciences. 2011;34(5):225–236. pmid:21459463
  121. 121. Scholtens LH, Feldman Barrett L, van den Heuvel MP. Cross-species evidence of interplay between neural connectivity at the micro-and macroscale of connectome organization in human, mouse and rat brain. Brain connectivity. 2018;(ja). pmid:30479137
  122. 122. Rubinov M, Sporns O. Weight-conserving characterization of complex functional brain networks. Neuroimage. 2011;56(4):2068–2079. pmid:21459148
  123. 123. Duran-Nebreda S, Bassel GW. Bridging Scales in Plant Biology Using Network Science. Trends Plant Sci. 2017;22(12):1001–1003. pmid:29029828
  124. 124. Zerkle AL. Biogeodynamics: bridging the gap between surface and deep Earth processes. Philos Trans A Math Phys Eng Sci. 2018;376 (2132). pmid:30275155
  125. 125. Hirt MR, Grimm V, Li Y, Rall BC, Rosenbaum B, Brose U. Bridging Scales: Allometric Random Walks Link Movement and Biodiversity Research. Trends Ecol Evol. 2018;33(9):701–712. pmid:30072217
  126. 126. Bershtein S, Serohijos AW, Shakhnovich EI. Bridging the physical scales in evolutionary biology: from protein sequence space to fitness of organisms and populations. Curr Opin Struct Biol. 2017;42:31–40. pmid:27810574
  127. 127. Gerlee P, Kim E, Anderson AR. Bridging scales in cancer progression: mapping genotype to phenotype using neural networks. Semin Cancer Biol. 2015;30:30–41. pmid:24830623
  128. 128. Fornito A, Arnatkeviciute A, Fulcher BD. Bridging the Gap between Connectome and Transcriptome. Trends Cogn Sci. 2018;S1364-6613(18):30253–5.
  129. 129. Falk EB, Bassett DS. Brain and Social Networks: Fundamental Building Blocks of Human Experience. Trends Cogn Sci. 2017;21(9):674–690. pmid:28735708
  130. 130. Towlson EK, De Vico Fallani F. Bridging Scales and Levels. Netw Neurosci. 2018;2(3):303–305. pmid:30294701
  131. 131. Chen TW, Wardill TJ, Sun Y, Pulver SR, Renninger SL, Baohan A, et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature. 2013;499(7458):295. pmid:23868258
  132. 132. Pachitariu M, Stringer C, Schroder S, Dipoppa M, Rossi LF, Carandini M, et al. Suite2p: beyond 10,000 neurons with standard two-photon microscopy. bioRxiv. 2017;061507.
  133. 133. Das S. Time series analysis. Princeton University Press, Princeton, NJ; 1994.
  134. 134. Reichardt J, Bornholdt S. Statistical mechanics of community detection. Physical Review E. 2006;74(1):016110.
  135. 135. Jeub LG, Sporns O, Fortunato S. Multiresolution Consensus Clustering in Networks. Scientific reports. 2018;8(1):3259. pmid:29459635