Sample data

The sample data were generated by one of the authors using BORIS and contains the logged behavior of 20 min video recordings from three cichlid species: Lamprologus ocellatus, Neolamprologus multifasciatus and Telmatochromis temporalis. These species are small fish of the tribe Lamprologini, native to Lake Tanganyika (Africa) that inhabit empty snail shells on the lake floor with conspecifics located at species-typical distances to each other. Due to this circumstance, they are commonly thought to exist on a broad spectrum, ranging from rather solitary to socially-living species. Given anecdotal field observations, which suggest that the three species exhibit a shared behavioral repertoire, this makes for a fascinating case to explore the behavioral mechanisms that gave rise to such divergent strategies of social living [18]. As a reference, the sample for N. multifasciatus can be downloaded from the website.

Data collection was performed using underwater cameras and caused minimal disturbance to the animals. This work adhered to the ASAB/ABS Guidelines for the Use of Animals in Research and field work was carried out with the permission of the Fisheries Department of Zambia under a study permits issued by the government of Zambia (SP008735 and SP2268511/9–21) and a memorandum of understanding (MOU 101/14/11), and adhered to the regulations of the Zambian Prevention of Cruelty to Animals act. These species are listed “Least Concern” on the IUCN Red List of Threatened Species.

Distinct behavior chart

The distinct behavior chart visually represents the distribution of behaviors, offering options to display either the number of observations or their total duration as a bar chart. Alternatively, a relative display option presents the data as a pie chart. Behavioral categories can also be displayed instead of individual behaviors. Individuals, behaviors, or behavioral categories (if selected) can be excluded from the calculation by clicking the button representing the respective individual or behavior. The charts are constructed using the plotting library matplotlib [19], see also Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Distinct behavior charts for subject 1 (top left and right) and subject 5 (bottom left and right) from the sample data of Neolamprologus multifasciatus from the website (S1 Dataset).

The total count for distinct behaviors is shown at the left, with the relative proportion per behavior on the right.

https://doi.org/10.1371/journal.pcbi.1012425.g002

Behavioral timeline

The behavioral timeline shows the time and duration of observed behavior. For this purpose, the time is shown on the x-axis and each individual behavior is shown as a line, with a colored horizontal bar indicating when a behavior occurs. Behavioral categories can also be displayed instead of individual behaviors. Individuals, behaviors, or behavioral categories (if selected) can be excluded from the calculation by clicking the button representing the respective individual or behavior. The charts are constructed using the plotting library matplotlib [19], see also Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Behavioral timelines for subject 9 from the sample data of Neolamprologus multifasciatus from the website (S1 Dataset).

The timeline for the behaviors is shown above. They describe the same data as Fig 2. The timeline for the corresponding behavioral categories is directly below.

https://doi.org/10.1371/journal.pcbi.1012425.g003

Temporal occurrences chart

The temporal occurrence chart combines the two previous charts in the sense that it shows the temporal accumulation of behavior. For this purpose, the time is shown on the x-axis and the cumulative count of selected behaviors shown up to that time (separately for each individual) on the y axis. Alternatively, one line per behavior per individual can be shown. Behavioral categories can also be displayed instead of individual behaviors. Individuals, behaviors, or behavioral categories (if selected) can be excluded from the calculation by clicking the button representing the respective individual or behavior. The charts are constructed using the plotting library matplotlib [19], see Fig 4. An interpretation of the charts follows in the results and discussion section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Temporal occurrences charts generated from sample data from the website showing the time on the x axis and the cumulative count of selected behaviors per individual on the y axis.

Data for the two shown plots originate from two different recordings: Neolamprologus multifasciatus (S1 Dataset) on the left and Lamprologus ocellatus (S2 Dataset) on the right side, each documenting behavior over a 20-minute period. There are 7 individuals recorded on the left and 10 on the right.

https://doi.org/10.1371/journal.pcbi.1012425.g004

Interaction network

The interaction network displays the number and direction of interactions between individuals (see Fig 5). It is a directed weighted network where nodes represent individuals and edges are drawn from individual A to individual B if A is the subject of a behavior and B is the recipient (i. e. the corresponding value in the column Modifier). The number of interactions determines the weight of an edge, which is displayed as edge label and mapped to the edge width. Individuals emanating and receiving behavior may be deselected separately and a weight threshold for edges to be displayed may be set. Node attributes and centrality metrics are calculated for the network, which describe the number of incoming and outgoing edges (interactions with different individuals) for each node (individual), as well as the sum of their labels (number of interactions) and four centralities (in-degree, out-degree, closeness and betweenness centrality). Degree centrality quantifies, by summing up the number of edges, the direct connections (i.e., relationships) a focal node (= focal individual) has with other nodes (= members of the social network). The number of connections is considered indicative of an individual’s influence on group members and possibly the whole network [11, 20]. In-degree refers to the number of incoming edges and reflects from how many group members the individual receives behaviors; whereas out-degree refers to the number of outgoing edges and reflects to how many group members the individual is exhibiting behaviors. Closeness centrality quantifies the reciprocal of the average distance from a focal node (= focal individual) to all other nodes (= group members) in the network [21]. This metric thus takes into account both direct as well as indirect connections (= relationships) and is thought to better reflect the potential influence the individual has on the entire group [20]. Betweenness centrality quantifies the fraction of shortest paths (i.e., fewest possible number of consecutive edges) between pairs of nodes in the network which include the focal node (= focal individual) [22]. Betweenness therefore reflects the importance of the focal individual as a point of social connection and transfer between different sections of the network [11, 20]. Furthermore, individuals with high betweenness centrality are thought to be important for group stability as their removal may lead to a fragmentation of the network into smaller subunits [23–25].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Interaction networks generated from the sample data of Lamprologus ocellatus from the website (S2 Dataset).

On the left side, all interactions i. e. behavior emanating from one individual and directed towards another are displayed, the betweenness of nodes (individuals) is mapped to node size and color brightness. On the right side detailed views for behavior emanating from subject 1 (top) and targeting subject 9 (bottom) are depicted.

https://doi.org/10.1371/journal.pcbi.1012425.g005

The four centralities can be visually represented through a mapping to node size and color brightness, allowing users to select a color for the latter option. The node attributes and centralities can be exported and also be sorted by clicking on the respective attribute or centrality. The interaction networks are computed with networkx [26], layout and visualization of the respective graphs is achieved with GraphViz [27].

Behavior transition network

The behavior transition network displays temporal sequences of behavioral events. It is visualized by a directed, weighted network where the nodes represent either behavioral events or behavioral categories and edges represent the transition from one behavior to another (see Fig 6). That is, an edge describes a behavioral transition or sequence (consecutive display of behavior). A self-loop occurs if a behavior is displayed, halted and then displayed again without any intermediate other behavior.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Behavior transition networks generated from the sample data of Neolamprologus multifasciatus from the website (S1 Dataset).

Both are set to use behavioral categories as nodes. On the left hand side, the average time a behavioral category is shown is mapped to node label, size and color brightness, while on the right side the total time is applied as mapping. The node properties are set as linear normalized. Edge values depict the transitional frequency from one category to another, rounded to the second digit. The edge width is set to be dependent on the edge weighting.

https://doi.org/10.1371/journal.pcbi.1012425.g006

The behavior transition network is generated as follows. First, behavior is considered separately for all individuals (unique values in column Subject), allowing the temporal sequence of behavior per individual to be read from successive rows. From this, a list consisting of 3-tuples (ordered lists of three elements) is generated: the first two elements being observed behaviors and the third showing the count of behavioral successions i. e. how often the second behavior was shown by the individual directly after the first behavior. These lists are accumulated for all selected individuals to derive the edge weights and are then visualized as the behavior transition network. A threshold for the edge weights may be set such that edges with weights below are excluded from the visualization (but still included in the calculation to avoid distorting the transitional frequencies). Individual behaviors, behavioral categories or individuals may be deselected and thereby excluded from the calculation and visualization. Following further customizations are available:

1. Node values Set either unique behaviors or behavioral categories (if present) as node values.
2. Edge values Set either the absolute number of transitions or the relative transitional frequencies (i. e. the relative frequency with which a certain behavior follows another behavior) as edge weights. In the latter case, for each unique edge, the ratio of its weight and the sum of the weights of all other outgoing edges from the same node is taken to derive the frequency of the respective behavioral sequence.
3. Color setting Select one of two coloring options: either one color is set and the nodes differ in the color saturation depending on the chosen mapping or each node and its outgoing edges are mapped to colors according to their behavioral category. In the latter case, the options for color hue and color saturation are disabled.
4. Edge width Select whether the width of drawn edges is fixed or dependent on the weights. In the latter case one may choose an edge width factor to control the edge width range.
5. Normalization Select whether edge width, node size, and node saturation are normalized in either a linear or logarithmic fashion.
6. Node size/color saturation/label mapping Node appearance may be altered by mapping the total count, the average time or the total time or any of the calculated centralities of a behavioral event to the node size, the node color saturation or onto a label inside the node.

Node attributes, which describe for each node (behavior) the total and average time it is observed, the amount of occurrences, the amount of ingoing and outgoing edges (distinctive behavioral transitions), as well as four centralities (in-degree, out-degree, closeness and betweenness centralities [28]) are calculated and can be exported, sorted and mapped in the same way as in the interaction networ. The in-degree centrality quantifies the number of incoming edges (transitions) to a node (behavior). Behaviors with high in-degree centrality have a variety of possible preceding behaviors. The out-degree centrality quantifies the number of outgoing edges (transitions) from a particular node (behavior). Behaviors with high out-degree centrality have a variety of possible subsequent behaviors. The closeness centrality considers the average distance from a node (behavior) to all other nodes (behaviors). Closeness centrality in a behavior transition network assesses how quickly a behavior can be reached from other behaviors in terms of behavioral transitions. Betweenness centrality measures the frequency with which a node (behavior) lies on the shortest paths between pairs of other nodes (behaviors). Betweenness centrality in a behavior transition network identifies behaviors that act as critical connectors between other behaviors.

The behavior transition networks are computed with networkx [26], layout and visualization of the respective graphs is achieved with GraphViz [27].

Comparing behavior transition networks

To facilitate the analysis of behavioral relationships between different species (or individuals), TIBA also provides methods to compare and cluster different behavior transition networks. Multiple data sets can be uploaded, for each of which a separate behavior transition network is generated. Pairwise comparison of these networks can now be done by one of the available distance measures, which effectively is an algorithm that computes a continuous similarity score for two provided networks. After selecting a distance measure, networks are compared pairwise. The resulting distance matrix (see Fig 7) can be exported or users can apply multidimensional scaling (to embed the distances into two dimensions, see Fig 8), or hierarchical clustering (see Fig 9). The following functionalities are available:

1. Choose network distance measure Available measures are the Network Portrait Divergence [29], the Frobenius norm, the Hamming weight, the Distributional Non-backtracking Spectral Distance [30], the Ipsen-Mikhailov Distance [31] and the Degree Divergence (see also [32]). More information on their exact definition and computation can also be found in the documentation of networkx [26].
2. Output Three outputs (indexed corresponding to the upload indexing) are generated: a distance matrix, a 2D embedding using multidimensional scaling and a hierarchical clustering tree. The first two characters of the filenames are used for indexing, optionally enumerated.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Distance matrix from pairwise comparisons of behavior sequences of the two cichlid species Neolamprologus multifasciatus and Lamprologus ocellatus (S4–S17 Datasets), using the Network Portrait Divergence as distance measure.

As distances are symmetric and a network has a distance of zero to itself, the distance matrix is itself symmetric with zeros on the diagonal.

https://doi.org/10.1371/journal.pcbi.1012425.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Two-dimensional embeddings of distances between transition networks as result of multi-dimensional scaling (MDS) applied to the distance matrix (Fig 7), where distances between single data points are preserved as good as possible while reducing dimensions.

Data of two cichlid species, Neolamprologus multifasciatus and Lamprologus ocellatus (S4–S17 Datasets), are compared pairwise with the Network Portrait Divergence applied as distance measure. The indices correspond to the first letters of the species, where the appended number allows the assignment to the same record in the dendrogram in Fig 9. The four embeddings were created with the parameter n_init set to 400 (the number of times the algorithm runs) and with the starting states 0, 40, 65 and 77 from the top left to the bottom right corner.

https://doi.org/10.1371/journal.pcbi.1012425.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Dendrograms of distances across transition networks as a result of agglomerative clustering of the distance matrix (Fig 7).

The average and complete distances between clusters are set as linkage criteria, on the left and on the right, respectively. Merged clusters with distances below 0.4 are set to have distinct colors. Data of two cichlid species, Neolamprologus multifasciatus and Lamprologus ocellatus (S4–S17 Datasets), are compared pairwise and the Network Portrait Divergence is applied as distance measure. The indices correspond to the first letters of the species, where the appended number allows the assignment to the same record in the plot in Fig 8.

https://doi.org/10.1371/journal.pcbi.1012425.g009

Multidimensional Scaling (MDS) is a statistical technique used to visualize the relative similarities or dissimilarities among a set of objects based on a distance matrix. First, the distance matrix is computed with the distance measure selected by the user. MDS then reduces the dimensionality of the data to a lower-dimensional space (two dimensions), thereby aiming to preserve the original distances between elements (in our case networks) as much as possible. The reduced-dimensional representation obtained through MDS is then plotted in a scatterplot, where each point represents a behavior transition network. The distances between points in the plot reflect the dissimilarities between the corresponding networks in the original data. Networks that appear close together in the MDS plot have similar transition patterns between behaviors. Networks that are distant from others may represent unique or distinct behavioral states or patterns. The arrangement of points in the plot may suggest transitional pathways between different behavioral states or clusters. Two parameters are available: A starting state which is necessary to initialize the algorithm. The same starting state guarantees the exact same output across different runs but one must test different starting states and verify that the emerging patterns in the output stay consistent. Only a pattern that emerges consistently across different starting states should be considered. Also, the number of times the algorithm runs (n_init) may be set to find a good balance between the running time and the precision of the results.

Hierarchical (agglomerative) clustering is a method used to cluster similar objects into groups based on their pairwise similarities or dissimilarities. Similar to MDS, a distance matrix serves as the basis for the clustering. Hierarchical clustering proceeds by iteratively merging clusters until all objects belong to a single cluster. The applied agglomerative clustering algorithm uses linkage criteria to determine the distance between clusters. Three linkage criteria are selectable by the user: Average linkage, complete linkage and single linkage. Average linkage computes the average distance between all pairs of points in two clusters. It tends to produce clusters with more uniform sizes. Complete linkage measures the maximum distance between any pair of points in two clusters. It tends to create compact clusters, suitable for identifying distinct groups. Single linkage determines the distance between the closest points (minimum distance) of two clusters. It tends to form clusters with elongated or chain-like shapes. The output may also be parameterized by setting a distance threshold for different coloring of the branches to indicate close distances. The resulting dendrogram or cluster tree shows the hierarchical relationships between clusters. Distinct clusters or branches in the dendrogram may correspond to species or groups of individuals exhibiting specific behavioral characteristics. Further, the branching structure of the dendrogram can reveal hierarchical relationships between different behavioral clusters, indicating transitions between different levels of behavioral complexity.

In summary, multidimensional scaling and hierarchical clustering offer powerful tools for exploring the structure and relationships within behavior transition networks, allowing researchers to uncover patterns and associations that may not be apparent through other analytical approaches. Both visualizations are computed using scikit-learn (sklearn) [33] and visualized with matplotlib [19].

Exporting images and networks

All visualizations may be exported from the web application as Scalable Vector Graphics (SVG). The interaction and behavior transition networks are also available as Grapviz Dot (GV) [34] and Graph Modelling Language (GML) [35] files. The node attributes and centralities for the interaction and behavior transition networks may be exported as Comma-Separated Values (CSV) files.

Implementation

TIBA is hosted as a web application at the University of Konstanz and works on all current browsers. It is implemented using open source software. The web server is implemented using Python 3.8+ and django [36], using matplotlib [19] for plotting and networkx [26] for network computation while the layout and visualization of the respective graphs is achieved with GraphViz [27]. The User Interface is written in ReactJS [37]. The user data is sent to the API with each request which is answered with the respective image and network files to display and download.