Procedural descriptions.

Most simulators provide a primitive for connecting a source neuron to a target neuron:

Typically, source and target above refer to indices in some neuron enumeration scheme. For example, both NEST [62–64], NEURON [65, 66] and Arbor [67, 68] have the concept of a global identifier or GID which associates a unique integer with each neuron. Many simulation environments offer a generic programming language where it is possible to write algorithms based on Connect to describe network connectivity. For example, the all-to-all connectivity pattern shown in Fig 7B, where each source neuron is connected to every target neuron, could be achieved by the procedural description:

A common pattern in such algorithms is to loop over all possible connections, as above, and call connect only if some condition is fulfilled.

If condition above is random() < p, where random() returns a uniformly distributed random number r, 0 ≤ r < 1, we obtain the pairwise Bernoulli pattern with probability p as shown in Fig 7E.

Procedural descriptions are the most general form of network specification: Any kind of connectivity pattern can be created by combining suitable Connect calls. Procedural descriptions of connectivity at the neuron-to-neuron level are for instance supported by the simulators NEST [62–64], NEURON [65, 66], Arbor [67, 68], Brian [69], Moose [70], and Nengo [71], as well as the description language PyNN [46].

Our example for the procedural approach already exposes two shortcomings. First, the explicit loop over all possible combinations is generic, but it is also costly if the condition is only fulfilled in a small fraction of the cases. For particular distributions an expert of statistics may know a more efficient method to create connections according to the desired distribution. Taking the example of a Bernoulli trial for each source-target pair, this knowledge can be encoded in a simulator function pairwise_bernoulli(). In this way also non-experts can create Bernoulli networks efficiently. Second, the explicit loops describe a serial process down to the primitive Connect() between two nodes. This gives simulators little chance to efficiently parallelize network construction.

Declarative population-level descriptions.

A declarative description of connectivity describes the connectivity at a conceptual level: It focuses on the connectivity pattern we want to obtain instead of the individual steps required to create it. Typically, the declarative description names a connectivity rule which is then used in setting up connectivity between two neuronal populations or from a population to itself. A common example is:

Declarative descriptions operating on populations are expressive, since they specify connectivity in terms of generic rules. Simulator software can be optimized for each rule, especially through parallelization. Rule-based specification of connectivity helps the reader of the model description to understand the structure of the network and also allows visualization and exploration of network structure using suitable tools. Usually the user is limited to a set of pre-defined rules, although some simulation software allows users to add new rules.

Declarative population-level descriptions are for instance supported by the simulators NEST, Moose, and the description languages PyNN (connectors) and NineML. Commonly provided connectivity rules are: one-to-one, all-to-all, and variants of probabilistic rules. The “transforms” in Nengo can also be regarded as declarative descriptions of connectivity. NeuroML supports lists of individual connections. Its associated language NetworkML [44] provides declarative descriptions akin to those of PyNN and NineML, while its associated lower-level declarative language LEMS [45] supports the definition of types of connectivity based on partially procedural constructs (“structure element types”) such as ForEach and If, giving greater flexibility but departing to some extent from the spirit of declarative description.

Algebraic descriptions.

Using algebraic expressions to describe connectivity rivals procedural descriptions in the sense that they are generic and expressive. Such descriptions are also declarative, with the advantage of facilitating optimization and parallelization.

The Connection Set Algebra (CSA) [58] is an algebra over sets of connections. It provides a number of elementary connection sets as well as operators on them. In CSA, connectivity can be concisely described using expressions of set algebra. Implementations demonstrate that the corresponding network connectivity can be instantiated in an efficient and parallelizable manner [72].

In CSA, a connection is represented by a pair of indices (i, j) which refer to the entities being connected, usually neurons. A source population of neurons can be enumerated by a bijection to a set of indices selected among the non-negative integers (1) and a target population can be similarly enumerated. A connection pattern can then be described as a set of pairs of indices. For example, if , and both sets of neurons are enumerated from 0 to 1, a connection pattern consisting of a connection from a to c and a connection from b to d would in CSA be represented by {(0, 0), (1, 1)}.

However, in CSA it turns out to be fruitful to work with infinite sets of indices. E.g., the elementary (predefined) connection set δ = {(0, 0), (1, 1), …} can be used to describe one-to-one connectivity in general, regardless of source and target population size. We can work with CSA operators on infinite connection sets and extract the actual, finite, connection pattern at the end. Given the finite index sets above, , we can extract the finite one-to-one connection pattern between and through the expression where ∩ is the set intersection operator and × is the Cartesian product.

Another example of an elementary connection set is the set of all connections (2)

For the case of connections within a population (i.e., ) it is now possible to create the set of all-to-all connectivity without self-connections: (3) where − is the set difference operator.

Random pairwise Bernoulli connectivity can be described by the elementary parameterized connection set ρ(p), which contains each connection in Ω with probability p. The random selection operator ρ_N(n) picks n connections without replacement from the set it operates on, while the operators ρ₀(k) and ρ₁(k) randomly pick connections to fulfill a given out-degree or in-degree k, respectively.

Multapses are treated by allowing multisets, i.e., multiple instances of the same connection are allowed in the set. The CSA expression for random connectivity with a total number of n connections, without multapses, is: (4) where the Cartesian product of the source and target index sets, , constitutes the possible neuron pairs to choose from.

By instead selecting from a multiset, we can allow up to m multapses: (5) where ⊎ is the multiset union operator.

The operator (6) replaces each connection in a set C with an infinity of the same connection such that, e.g., ρ₀(k)MC means picking connections in C to fulfill fan-out k, but now effectively with replacement. Without going into the details, multisets can also be employed to set limits on the number of multapses.

Population-level connectivity rules of languages and simulators.

Most neural network description languages and simulators provide several descriptors or routines that can be used to specify standard connectivity patterns in a concise and reproducible manner. We here give an overview over the corresponding connection rules offered by a number of prominent model description languages and simulators. This brief review supplements the literature review to identify a set of common rules to be more formally described in the next section.

We have studied connectivity rules of the following model specification languages and simulators:

1. NEST is a simulator which provides a range of pre-defined connection rules supporting network models with and without spatial structure. To create rule-based connections, the user provides source and target population, and the connection rule with applicable parameters and specifications of the synapses to be created, including rules for the parameterization of synapses. The information here pertains to NEST version 3.0.
   In addition to the built-in connectivity rules, NEST provides an interface to external libraries, for example CSA, to specify connectivity.
2. PyNN is a simulator-independent language. It provides a class of high-level connectivity primitives called Connector. The connector class represents the connectivity rule to use when setting up a Projection between two populations. The information here pertains to PyNN version 0.9.6.
3. NetPyNE is a Python package to facilitate the development, simulation, parallelization, analysis, and optimization of biological neuronal networks using the NEURON simulator. It provides connectivity rules for explicitly defined populations as well as subsets of neurons matching certain criteria. A connectivity rule is specified using a connParams dictionary containing both parameters defining the set of presynaptic and postsynaptic cells and parameters determining the connectivity pattern. The information here pertains to NetPyNE version 1.0.
4. NineML is an XML-based cross-simulator model specification language.
5. Brian is a simulator which has a unique way of setting up connectivity. Connections between a source and target group of neurons are specified using an expression that combines procedural and algebraic aspects, passed to the connect method of the synapse object S:
   Here, EXPR is an integer-valued expression specifying the targets for a given neuron i. This expression may contain the variable VAR which obtains values from RANGE. For example, to specify connections to neighboring neurons, we can say
   where skip_if_invalid tells Brian to ignore invalid values for j such as −1.

The simulators NEURON and Arbor do not support high-level connectivity rules and are therefore not included here.

The population-level connectivity rules shared—under different names—between two or more of the above simulators are the following:

1. One-to-one connects each source to one corresponding target.
2. All-to-all connects each source to all targets.
3. Explicit connections establishes the connections given in an explicit list of source-target pairs.
4. Pairwise Bernoulli performs a Bernoulli trial for each possible source-target pair. With a certain probability p, the connection is included.
5. Random, fixed total number establishes exactly N_syn connections between possible sources and targets.
6. Random, fixed in-degree connects exactly K_in sources to each target (where the same source may be counted more than once).
7. Random, fixed out-degree connects each source to exactly K_out targets (where the same target may be counted more than once).

Languages and simulators vary with regard to whether autapses or multapses are created by a connectivity rule and whether it is possible to choose if they are created or not. Table 1 details the extent to which the rules above are implemented in the languages and simulators NEST, PyNN, NETPyNE, and NineML. In addition, PyNN supports the following rules:

• Pairwise Bernoulli with probability given as a function of either source-target distance, vector, or indices.
• Small-world connectivity of the Watts-Strogatz type, with and without autapses; out-degree can be specified.
• Connectivity specified by a CSA connection set provided by a CSA library.
• Explicit Boolean connection matrix.
• Connect cells with the same connectivity as the given PyNN projection.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Connectivity rules present in a selection of languages and simulators.

X: The rule is supported, A: The rule is supported and it is possible to specify whether autapses are created or not, M: Ditto for multapses.

https://doi.org/10.1371/journal.pcbi.1010086.t001

The pairwise Bernoulli and random, fixed in- and out-degree rules in NEST support connectivity creation based on the relative position of source and target neurons.

Deterministic connectivity rules.

Deterministic connectivity rules establish precisely defined sets of connections without any variability across network realizations.

1. One-to-one
   Symbol: δ
   CSA: δ
   Definition: Each node in is uniquely connected to one node in .
   and must have identical cardinality N_s = N_t, see Fig 7A. Both sources and targets can be permuted independently even if . The in- and out-degree distributions are given by P(K) = δ_K,1, with Kronecker delta δ_i,j = 1 if i = j, and zero otherwise.
2. All-to-all
   Symbol: Ω
   CSA: Ω
   Definition: Each node in is connected to all nodes in .
   The resulting edge set is the full edge set . The in- and out-degree distributions are for , and for , respectively. An example is shown in Fig 7B.
3. Explicit connections
   Symbol: X
   CSA: Not applicable
   Definition: Connections are established according to an explicit list of source-target pairs.
   Connectivity is defined by an explicit list of sources and targets, also known as adjacency list, as for instance derived from anatomical measurements. It is, hence, not the result of any specific algorithm. An alternative way of representing a fixed connectivity is by means of the adjacency matrix A, such that A_ij = 1 if j is connected to i, and zero otherwise. We here adopt the common computational neuroscience practice to have the first index i denote the target and the second index j denote the source node.

Probabilistic connectivity rules.

Probabilistic connectivity rules establish edges according to a probabilistic rule. Consequently, the exact connectivity varies with realizations. Still, such connectivity leads to specific expectation values of network characteristics, such as degree distributions or correlation structure.

1. Pairwise Bernoulli
   Symbol: p
   CSA: ρ(p)
   Definition: Each pair of nodes, with source in and target in , is connected with probability p.
   In its standard form this rule cannot produce multapses since each possible edge is visited only once. If , this concept is similar to Erdős-Rényi-graphs of the constant probability p-ensemble G(N, p)—also called binomial ensemble [73]; the only difference being that we here consider directed graphs, whereas the Erdős-Rényi model is undirected. The distribution of both in- and out-degrees is binomial, (7) and (8) respectively. The expected total number of edges equals E[N_syn] = pN_tN_s.
2. Random, fixed total number without multapses
   Symbol:
   CSA:
   Definition: N_syn ∈ {0, …, N_s N_t} edges are randomly drawn from the edge set without replacement. For this is a directed graph generalization of Erdős-Rényi graphs of the constant number of edges N_syn-ensemble G(N, N_syn) [74]. There are possible networks for any given number N_syn ≤ N_sN_t, which all have the same probability. The resulting in- and out-degree distributions are multivariate hypergeometric distributions. (9) and analogously with K_out instead of K_in and source and target indices switched.
   The marginal distributions, i.e., the probability distribution for any specific node j to have in-degree K_j, are hypergeometric distributions (10) with sources and targets switched for P(K_out,j = K_j).
3. Random, fixed total number with multapses
   Symbol: N_syn, M
   CSA:
   Definition: N_syn ∈ {0, …, N_sN_t} edges are randomly drawn from the edge set with replacement.
   If multapses are allowed, there are possible networks for any given number N_syn ≤ N_sN_t. Because exactly N_syn connections are distributed across N_t targets with replacement, the joint in-degree distribution is multinomial, (11) with p = 1/N_t.
   The out-degrees have an analogous multinomial distribution , with p = 1/N_s and sources and targets switched. The marginal distributions are binomial distributions and , respectively.
   The M-operator of CSA should not be confused with the “M” indicating that multapses are allowed in our symbolic notation.
4. Random, fixed in-degree without multapses
   Symbol:
   CSA:
   Definition: Each target node in is connected to K_in nodes in randomly chosen without replacement.
   The in-degree distribution is by definition . To obtain the out-degree distribution, observe that after one target node has drawn its K_out sources the joint probability distribution of out-degrees K_out,j is multivariate-hypergeometric such that (12) where ∀_j K_j ∈ {0, 1}. The marginal distributions are hypergeometric distributions (13) with Ber(p) denoting the Bernoulli distribution with parameter p, because K ∈ {0, 1}. The full joint distribution is the sum of N_t independent instances of Eq 12.
5. Random, fixed out-degree without multapses
   Symbol:
   CSA:
   Definition: Each source node in is connected to K_out nodes in randomly chosen without replacement.
   The out-degree distribution is by definition , while the in-degree distribution is obtained by switching source and target indices, and replacing K_out with K_in in Eq 12.
6. Random, fixed in-degree with multapses
   Symbol: K_in, M
   CSA:
   Definition: Each target node in is connected to K_in nodes in randomly chosen with replacement.
   N_s is the number of source nodes from which exactly K_in connections are drawn with equal probability p = 1/N_s for each of the N_t target nodes . The in-degree distribution is by definition . To obtain the out-degree distribution, we observe that because multapses are allowed, drawing N_t times K_in,i = K_in from is equivalent to drawing N_tK_in times with replacement from . This procedure yields a multinomial distribution of the out-degrees K_out,j of source nodes [75], i.e., (14)
   The marginal distributions are binomial distributions (15)
7. Random, fixed out-degree with multapses
   Symbol: K_out, M
   CSA:
   Definition: Each source node in is connected to K_out nodes in randomly chosen with replacement.
   By definition, the out-degree distribution is a . The respective in-degree distribution and marginal distributions are obtained by switching source and target indices, and replacing K_out with K_in in Eqs 14 and 15 [75].

Networks embedded in metric spaces.

The previous sections analyze the connectivity between sets of nodes without any notion of space. However, real-world networks are often specified with respect to notions of proximity according to some metric. Prominent examples are spatial distance and path length in terms of number of intermediate nodes. The exact embedding into the metric space, such as the distribution of nodes in space or the boundary conditions, can have a strong impact on the resulting network structure. ρ here denotes the density, not to be confused with the CSA operator.

Given a distance-dependent connectivity, degree distributions result from this distance dependence combined with the distribution of distances between pairs of nodes [76]. If nodes are placed on a grid or uniformly at random in space, different asymptotic approximations to the degree distributions can be made [77–79]. If the node distribution is (statistically) homogeneous, and the connection probability is isotropic, the average in- or out-degree for connections to or from any node i at a given distance from the node follows 〈K(r)〉 ∼ ρ(r)p(r), which is usually easier to derive than the full joint degree distribution, and can be used to statistically test whether network realizations are correctly generated [75, 80].

Here we specify the properties of spatial networks, which are also relevant for networks with feature-specific connectivity (e.g., based on sensory response tuning). In order to fully specify networks embedded in a metric space and with distance-dependent connectivity, the following quantities need to be listed:

1. Dimension: Most often the space is one-dimensional (e.g., ring networks), two-dimensional (e.g., a layer of neurons), or three-dimensional (i.e., a volume of neurons).
2. Layout: The layout specifies how nodes are arranged, for instance on a regular grid (e.g., orthogonal, isometric, or hexagonal) or uniformly at random.
3. Metric: The metric specifies the concept of distance. On an orthogonal grid the max-norm metric (ℓ_∞) on the grid index can be the metric of choice, while for a uniformly random distribution of nodes the Euclidean metric (ℓ₂) is typically chosen.
4. Boundary conditions: If nodes are embedded into a space with boundaries, there tend to be inhomogeneities in the connectivity close to these boundaries. To avoid such potential inconsistencies, boundary conditions are often assumed to be periodic, i.e., opposite borderlines are folded back onto each other (e.g., a line into a ring, a layer into a torus, etc.).
5. Distance dependence of the connectivity profile: The connectivity profile , sometimes called spatial footprint, specifies which nodes j are connected to a node i as a function of their distance . Profiles can be deterministic (e.g., a node connects to all other nodes within a certain distance r_max, specified via a boxcar profile f(r) ∼ Θ[r_max − r]) or probabilistic (a node connects to another node at a certain distance r with probability p(r) ∈ [0, 1], e.g., boxcar: p(r)∼c Θ[r_max − r], linear: p(r)∼max(c₁ − c₂ r, 0), sigmoidal: , exponential: , Gaussian: , or more complex, e.g., non-centered multivariate Gaussian with covariance matrix Σ: , , etc.). These distance-dependent connectivity profiles may be combined with rules for the establishment of multapses and higher-order moments. In the case of feature-specific connectivity as well as other generalized spaces and cases where a metric is difficult to define, it can be useful to generalize f to be a direct function of the sets of sources and targets, like a CSA mask: f = f(i, j) where . The distance could be treated similarly: r_ij = r(i, j) corresponding to a CSA value function.

Larger-scale and multiscale networks can have more complicated, heterogeneous structures, such as layers, columns, areas, or hierarchically organized modules. Distance dependencies may then have to be specified with respect to the different levels of organization, for example specific to their horizontal (laminar) and vertical (e.g., columnar) dimension (cf. “Introduction”). One example is networks modeling axonal patches, i.e., neurons that have axonal arborization in a certain local range, as well as further axonal sprouting in several distinct long-range patches [24, 27, 81–83].

We discuss an explicit example of how to describe such connectivity rules in Section “Examples”.

Network node.

A network node in the graphical notation represents one or multiple units. These units are either neuron or neural population models, or devices providing input or output. Network connectivity is defined between these graphically represented nodes. Nodes are drawn as basic shapes. A textual label can be placed inside the node for identification. Nodes are differentiated according to a node class and a node type.

Node class.

The node class determines if a node represents an individual unit or a population of units by different frames of the shapes depicting the nodes. The distinction is a recommendation for diagrams that contain both kinds of nodes.

1. Individual unit
   A node representing an individual unit may be depicted as a shape with a thin, single frame. Note that such an individual unit may be a population (e.g., neural mass) model.
2. Population
   A node representing a population of units may be depicted as a shape with either a thick frame or a double frame. It is in principle possible to represent a group of population models this way.

Node type.

The node type refers to a defining property of a node and is expressed by a unique shape.

1. Generic node
   A generic node, represented by a square, is used if the specific node types do not apply or are not intended to be emphasized.
2. Excitatory neural node
   An excitatory neural node, depicted by a triangle, is used if the units represent neurons, and their effect on targets is excitatory.
3. Inhibitory neural node
   An inhibitory neural node, depicted by a circle, is used if the units represent neurons and their effect on targets is inhibitory.
4. Stimulating device node
   A stimulating device node, depicted by a hexagon, provides external input to other network nodes. Stimulating devices can be abstract units which for instance supply stochastic input spikes. Nodes with more refined neuron properties can also be considered as stimulating devices if they are external to the main network studied.
5. Recording device node
   A recording device node, depicted by a parallelogram, contains non-neural units that record activity data from other network nodes.

Network edge.

A network edge represents a connection or projection between two nodes. Edges are depicted as arrows. Both straight and curved lines are possible. Edges are differentiated according to the categories determinism, edge type, and directionality.

Determinism.

The notation distinguishes between deterministic and probabilistic connections via the line style of network edges. Edges between two nodes representing individual units are usually deterministic.

1. Deterministic
   Deterministic connections, depicted by a solid line edge, define exactly which units belonging to connected nodes are themselves connected.
2. Probabilistic
   Probabilistic connections, depicted by a dashed-line edge, are constructed by connecting individual neurons from source and target populations according to probabilistic rules.

Edge type.

Analogously to the node type, the edge type emphasizes a defining property of the connection by specific choices of arrowheads. The edge types given here can be used for connections between all node types.

1. Generic edge
   A generic edge, represented by a classical (or straight barb) arrowhead, is used if the specific edge types do not apply or the corresponding properties are not intended to be emphasized.
2. Excitatory edge
   An excitatory edge, depicted by a triangle arrowhead, is used if the effect on targets is excitatory.
3. Inhibitory edge
   An inhibitory edge, depicted by a filled circle tip, is used if the effect on targets is inhibitory.

Directionality.

The directionality indicates the direction of signal flow by the location of one or two arrowheads on the edge.

1. Unidirectional
   Unidirectional connections are depicted with a tip at the target node’s end of the edge.
2. Bidirectional
   Bidirectional connections are symmetric in terms of the existence of connections and their parameterization. Such connections are depicted with edges having tips on both ends. If the same units are connected but parameters for forward and backward connections are not identical, two separate unidirectional edges should be used instead.

Annotation.

Network edges can be annotated with information about the connection or projection they represent. Details on the rule specifying the existence of connections and their parameterization may be put along the arrow.

Connectivity concept.

The properties in this category further specify the presence or absence of connections between units within the connected nodes.

1. Concept
   The definitions and symbols given in Section “Connectivity concepts” are the basis for this property.
2. Constraint
   Specific constraint or exception to the connectivity concept.
   1. Autapses allowed
      Autapses are self-connections. The letter A indicates if they are allowed.
   2. Multapses allowed
      Multapses are multiple connections between the same pair of units and in the same direction. The letter M indicates if they are allowed.
   3. Prohibited
      The symbol of a constraint struck out reverses allowed to prohibited. E.g., autapses and multapses are prohibited: .

Parameterization.

Properties of the parameterization of connections, e.g., of weights w and delays d, can be expressed with mathematical notation.

1. Constant parameter
   A parameter, e.g., a weight, which takes on the same value for all individual connections is indicated by an overline: .
2. Distributed parameter
   A tilde between a parameter (e.g., the weight) and a distribution indicates that individual parameter values are sampled from the distribution . This example uses for a generic distribution, but specific distributions, such as a normal distribution denoted by , are also possible.

Further specification.

Annotations for both the connectivity concept and the parameterization of connections can be specified further.

1. Functional dependence
   Functional dependence on a parameter is expressed with parentheses, here indicated with a generic function f(·). Common use cases are the dependence on the inter-unit distance r or on time t. Connections drawn with a distance-dependent profile can be indicated with f(r). The exact function f used should be defined close to the diagram; already defined concepts such as a spatially modulated pairwise Bernoulli connection probability can also be used: p(r). Another example for a distance-dependent parameter could be a delay d(r). Plastic networks, in which the weights change with time, can be indicated with w(t).

Customization and extension.

The definitions given above are intended as a reference for illustrating network types that are in the scope of this study. Further graphical techniques may be used that go beyond these fundamental definitions, such as adding meaning to the size of network nodes (e.g., making the area proportional to the population size) or using colors (e.g., to highlight network nodes or edges sharing certain specifics). In the community, two ways of distinguishing excitatory and inhibitory neurons tend to be used: the “water tap” notation in which the excitatory neurons are shown in red and the inhibitory neurons in blue (e.g., [87]), and notations in which the inhibitory neurons are shown in red and the excitatory neurons in either blue or black, which may be thought of as “bank account” notation (blue: [5], black: [14]).

Fig 7 uses the proposed symbols for generic node and edge types to demonstrate basic connectivity patterns; in addition, we employ colors to differentiate source and target nodes and their connections. In Fig 1 we distinguish with blue and red between excitatory and inhibitory neurons, respectively, to give an example for the bank account notation.

Encoding the same feature in multiple ways is also encouraged if it supports intuition; in the proposed graphical notation, we use double encoding for node shapes and arrowheads. For complex or hierarchical networks, multiple diagrams may be created: for instance, one that provides an overview and others that bring out specific details.

The modular structure of our graphical notation framework allows for extension to features that are not yet covered. Symbols for additional network elements may be defined for example in the figure legend and applied as the researcher sees fit. The common classification of neural nodes into excitatory and inhibitory types used in the notation is one such example. On the one hand, a model-specific definition of these types can be formulated. On the other hand, further classification detail can be added to the graph (e.g., in the form of annotations) or additional node types can be introduced if necessary to represent nodes with further biophysical properties which are not covered by the above simple classification.

In the same way as our propositions for node types can be customized, adjustment of the other graphical elements is also encouraged. For example, having so far considered only networks coupled via chemical synapses, another possible extension is to define gap junctions as a novel edge type. One possibility here is to use the common symbol for electrical resistance:

1. Gap junctions
   Electrical coupling via gap junctions is represented by a zig-zag line connecting the nodes.

Two-population balanced random network.

The first example is the random, fixed in-degree variant of the balanced random network model also shown in Fig 1A (for details see Figs 12–15). Fig 9 shows different means for describing the connectivity of the model; the same options are covered in the model review in Fig 3B. The illustration (Fig 9A) uses the elements for nodes, edges, and annotations introduced in Section “Proposal for a graphical notation for network models” to depict the network composed of an excitatory (E, triangle) and an inhibitory (I, circle) neuron population, and a population of external stimulating devices (E_ext, hexagon). Recurrent connections between the neurons in the excitatory and inhibitory populations are probabilistic (dashed edges) and follow the “random, fixed in-degree” rule (K_in) with the further constraints that autapses are prohibited () and multapses are allowed (M).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Different means to describe connectivity of a balanced random network.

Example descriptions for the model used in Fig 1A with description means similar to Fig 3B. (A) Network diagram according to the graphical notation introduced in Section “Proposal for a graphical notation for network models”. Symbols in annotations refer to the concepts and not the explicit parameters. (B) Textual description of the model layout. Subscript “” labels connections from source population E to target population ; the same applies to “” with source population I. and represent the explicit values used for the in-degrees. (C) Table according to the guidelines by Nordlie et al. [84]. (D) Equations according to the Connection Set Algebra (CSA) [58] using the index sets E and I. (E) PyNEST source code [63] specifying connections from source (pre) to target (post) populations with a connection dictionary (conn_spec). The use of all-to-all instead of one-to-one connectivity here is due to the specific implementation of the external drive in NEST.

https://doi.org/10.1371/journal.pcbi.1010086.g009

Connections between different, non-intersecting populations by definition cannot have autapses and therefore it is not required to specify this along the corresponding edges. Neither does the absence of multapses between E_ext and the neuronal populations need to be specified as we here assume a one-to-one connectivity (δ). This network diagram not only indicates if connections exist but also shows that their parameters, weights (w), and delays (d) are the same for each connection. However, the diagram does not express the parameter values, just as the numbers of incoming connections are left to be defined elsewhere. In contrast, the textual description (Fig 9B) adds subscripts to the connectivity concept to indicate that the excitatory and inhibitory in-degrees may be different: and , respectively. The table (Fig 9C) follows the guidelines by Nordlie et al. [84] and structures each connection in terms of a name, the source and target populations, and the connectivity rule. The set of equations (Fig 9D) formulates the connectivity by means of the Connection Set Algebra (CSA) [58]. While panels A–D of Fig 9 are primarily concerned with the conceptual description of connectivity, Fig 9E gives an implementation example using the PyNEST [63] interface of the simulator NEST [62]. The excitatory (E) and inhibitory (I) population are here represented by NodeCollections, storing the IDs of each neuron. By default, autapses and multapses are allowed; here we set both values explicitly for clarity. E_Ext in the code stands for a poisson_generator, a stimulating device node in NEST which generates independent sequences of input spikes sampled from the same Poisson process for each of its target neurons. In other words, E_Ext refers to just one NEST node which acts like the population of external stimulating devices indicated with E_Ext in Fig 9A–9D. Due to this specific implementation of the poisson_generator, the default connection rule all_to_all as a generalization of one-to-all connectivity is here applied instead of one_to_one.

Previous studies preferentially combine different ways of describing connectivity (Fig 3B) and also the example in Fig 9 highlights that one means alone may not be sufficient to exhaustively cover all aspects of the connectivity. For a comprehensive description, we recommend using at least one network diagram and a textual description for rapidly conveying the network structure, and a table for providing details. In addition, default assumptions, e.g., the presence or absence of multapses, should be made explicit; this can be done in the text.

Cortical microcircuit with distinct interneuron types.

The second example, shown in Fig 10, is a cortical microcircuit model [88] adapted from Potjans and Diesmann [89]. Extending the two-population network in Fig 9, this model comprises four cortical layers (L2/3, L4, L5, and L6). With its cell-type and layer-specific connectivity, the Potjans-Diesmann model represents the structure and dynamics of local cortical circuitry which is similar across different areas and species. The model has been used in a number of recent validation and benchmarking studies, and implementations for different simulators exist, including NEST [90], SpiNNaker [90, 91], Brian [92], GeNN and PyGeNN [93, 94], NeuronGPU [95], NetPyNE [96], and PyNN (available as a PyNN example and via Open Source Brain, https://www.opensourcebrain.org/projects/potjansdiesmann2014). While the original model by Potjans and Diesmann has only one excitatory (E) and one inhibitory (I) neuron population per layer, the model considered here distinguishes between three different inhibitory neuron types (SOM, VIP, and PV). All neuron populations receive external Poisson input E_ext as in Fig 9 and additional input from an external thalamic population E_th. The thalamic targeting of all layers is in contrast to the Potjans-Diesmann model where only L4 and L6 receive thalamic input. Fig 10 shows three different diagrams to emphasize different aspects of the model. Here, the first two panels are used to give an intuitive overview of the network, while the third panel adheres to the proposed graphical notation to unequivocally represent the connectivity rules. Fig 10A uses a colored illustration to convey the overall components without specifying the connection rules. For the general model overview, cortical layers and subnetworks of inhibitory populations are framed by boxes. To avoid clutter, not all connections are shown and the distinction between probabilistic and deterministic connections via dashed and solid lines, as suggested in Fig 8, is not applied. Instead, only connections above a threshold connection probability are shown with solid lines, and two levels of line thickness help to distinguish between low- and high-probability connections. By taking this freedom we illustrate that customizations remain possible for overview figures, as long as the network is unequivocally described in the remainder. Arrows to or from a box represent the average connection probabilities to or from the network nodes contained in the box. The average connection probability equals the expected total number of connections divided by the maximum number of possible connections while considering all involved pairs of populations. For example, the average connection probability from an excitatory population E to the inhibitory populations is given by: (16)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Multi-layer microcircuit model with three inhibitory neuron types.

(A) Schematic overview of all neuronal populations, external inputs, and main connections. Inhibitory populations are grouped by boxes. In panels A and B, for probabilistic connections, only those with a probability of at least 4% are shown (thin lines: 4 to 8%, thick lines: ≥8%). (B) Detailed L2/3 connectivity between excitatory population and all three inhibitory populations; in panel A these connections are combined in two arrows (from and to the box). (C) Excitatory-inhibitory subnetwork with external inputs depicted with annotations according to the graphical notation in Fig 8. The connectivity is described with the rules “one-to-one” (δ) and “pairwise Bernoulli” (p), and the constraints autapses allowed (A) and multapses prohibited (). The synaptic weights (w) and delays (d) are specified as either constant (i.e., ) or sampled from lognormal distributions (i.e., ). Interneuron types: somatostatin expressing (SOM), vasoactive intestinal peptide expressing (VIP), parvalbumin expressing (PV).

https://doi.org/10.1371/journal.pcbi.1010086.g010

Fig 10B zooms into layer L2/3 to highlight the connectivity between the excitatory population and the three inhibitory populations in this single layer, resolving the arrows in and out of the box. In panel A there is only one outgoing arrow from the inhibitory neuron box in L2/3 connecting to the excitatory population, but in panel B it becomes clear that the inhibitory subpopulations SOM and PV both have strong connections to E while VIP does not.

Fig 10C follows the proposed notations, as in Fig 9A, to illustrate the general components and connection rules that apply to the whole network regardless of layer and inhibitory cell type. While the original model by Potjans and Diesmann uses connectivity of the type “Random, fixed total number with multapses”, this model uses “pairwise Bernoulli” connectivity as indicated by the symbol p.

Combining these illustrations helps to understand the structure and characteristics of this model more intuitively. However, we do encourage deviations from and extensions to the proposed notation if it helps to improve the clarity of the diagrams, but these changes should be explained with care.

Spatial network with horizontally inhomogeneous structure.

The third example is a network embedded into two-dimensional space introduced in a paper by Voges & Perrinet [97] to model the dynamics of neocortical networks with realistic horizontal connectivity. The “PB model”, as it is called by the authors, incorporates both local and non-local connections between cells as observed for instance in the laminar structure of the visual cortex of cats [97, 98]. Local connectivity (footprint ≲ 150–300 μm) is observed to be approximately isotropic, with nearby cells being more likely to be connected than cells farther apart. On longer scales (≳ 1mm) so-called patches can be observed where the axons sprout and form several connections in a confined area (see Introduction).

As mentioned in Section “Connectivity concepts”, in order to define spatially embedded networks, the dimensions of the space, layout of neurons, metric of distances, boundary conditions, and, for distance-dependent connectivity, the form of this distance dependence need to be specified (Fig 11A). Here, N_E excitatory and N_I = N − N_E inhibitory neurons are embedded into a two-dimensional Euclidean space of size [0, L) × [0, L) with periodic boundary conditions (Fig 11A). Excitatory neurons are placed randomly according to a uniform distribution, while inhibitory neurons are distributed on jittered grid positions with grid constant and jitter with maximal jitter J where denotes the uniform distribution. Both populations have local and non-local, patchy connections [97] with different parameterizations for excitatory and inhibitory neurons (Fig 11A), based on [23].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Two-dimensional spatial network with patchy long-range connections.

(A) Spatial networks need to be defined in terms of dimension, layout, metric, boundary conditions, and the spatial or distance dependence of the connectivity, where applicable. In this example, neurons have both local and structured long-range connections [97]. Θ(x) = 0 if x < 0, 1 otherwise. (B) Sketch of patchy connectivity and parameters needed to define p_patch. (C) Graphical notation of network connectivity corresponding to Fig 8.

https://doi.org/10.1371/journal.pcbi.1010086.g011

The global connection density (number of realized connections over number of possible connections) c_total splits up into respective local and non-local parts motivated by anatomy [26, 97] (cf. Introduction), such that each neuron i in a given subpopulation has local and non-local connection densities c_loc(i) and , where Npn is the number of patches per neuron (see Fig 11A). These underlying biologically motivated numbers then serve as constraints for the choices of the parameters needed in the following definitions [26, 97].

Local connections: In order to satisfy constraints with respect to both the fraction of connections assigned as local and the local spatial footprint, the out-degrees are in a first step drawn from a binomial distribution with a mean that produces the right connectivity fraction c_loc. In a second step, random elements (i, j) of the set of potential synapses are drawn, and a connection is established with probability , until the required number is achieved. Multapses and autapses are excluded. The local connectivity of each neuron i at follows a Gaussian connectivity profile with center , maximal connection probability p₀ and space constant or footprint σ, indicated by colored circles in Fig 11B.

Non-local, patchy connections: Non-local connection patterns in Fig 11B are determined for groups of neighboring neurons, such that all neurons located within a certain region (squares) project to a fixed subset of spatially distributed patches (light gray disks) allowed for this region. Again, first an out-degree is determined, and then the required number of synapses is established probabilistically according to Bernoulli trials, where the connection probability ) from a neuron at to each cell within one of its target patches centered at is constant within a certain radius. Multapses are excluded.

The basic parameters to characterize patchy projections are then:

1. Np: the number of patches per group of neurons,
2. Npn: the number of patches per single neuron (Npn≤Np),
3. rp: the radius of a patch,
4. dp: the distance between the center of a group , and patch center ,
5. ϕ: the angle which characterizes a patch position relative to , see Fig 11A and 11B.

In particular, here the respective Np’s are drawn from uniform distributions with distinct minimum Np_min and maximum Np_max values for each population (E, I) while the Npn’s (Npn≤Np) are drawn from binomial distributions with specified means and the corresponding cutoff values. The distances dp come from normal distributions, while the angles ϕ are sampled uniformly from the interval [0, 2π) (Fig 11A and 11B, [97]). The Npn patches to which a given neuron projects are chosen uniformly at random from the Np patches for the group.

The remaining connectivity specifications are shown in the graphic in Fig 11C. Each population receives an external drive modeled as Poisson-process spike input. Moreover, delays are distance-dependent [97]. The exact connectivity parameters for each population, as well as weights and delays would need to be specified for instance in a table, which is beyond the scope of this example.