^{1}

^{2}

^{3}

^{¤}

^{1}

^{2}

^{1}

^{2}

^{1}

^{2}

^{*}

Current address: LPTMC, Université Pierre et Marie Curie, Paris, France

Conceived and designed the experiments: PF. Performed the experiments: NM. Analyzed the data: NM PF. Contributed reagents/materials/analysis tools: AL JPC UM PF. Wrote the paper: NM AL JPC UM PF. Worked on the mathematical aspects underlying the methods for analyzing the sequences and their interpretation: AL.

The authors have declared that no competing interests exist.

Nicotinic acetylcholine receptors (nAChRs) are widely expressed throughout the central nervous system and modulate neuronal function in most mammalian brain structures. The contribution of defined nAChR subunits to a specific behavior is thus difficult to assess. Mice deleted for ß2-containing nAChRs (ß2−/−) have been shown to be hyperactive in an open-field paradigm, without determining the origin of this hyperactivity. We here develop a quantitative description of mouse behavior in the open field based upon first order Markov and variable length Markov chain analysis focusing on the time-organized sequence that behaviors are composed of. This description reveals that this hyperactivity is the consequence of the absence of specific inactive states or “stops”. These stops are associated with a scanning of the environment in wild-type mice (WT), and they affect the way that animals organize their sequence of behaviors when compared with stops without scanning. They characterize a specific “decision moment” that is reduced in ß2−/− mutant mice, suggesting an important role of ß2-nAChRs in the strategy used by animals to explore an environment and collect information in order to organize their behavior. This integrated analysis of the displacement of an animal in a simple environment offers new insights, specifically into the contribution of nAChRs to higher brain functions and more generally into the principles that organize sequences of behaviors in animals.

Understanding mechanisms underlying complex behaviors and the abnormalities that accompany most neuropathologies is a current challenge in biomedical research. A number of approaches is primarily based on the identification of genes and their associated molecular pathways implicated in complex motor or cognitive pathologies. However, optimal use of the large body of genetic, molecular, electro-physiological, and imaging data is hampered by the practical and theoretical limitations of currently available behavioral analysis methods. Complex behaviors consist of a finite number of actions combined in a variety of spatial and temporal patterns. In this paper we develop a sequential analysis of mouse displacement in an open-field paradigm and demonstrate that a description based on a Markov model can be used to describe quantitatively patterns of behaviors and to detect changes in the way that animals organize their displacement, especially in mice lacking nicotinic acetylcholine receptor subunits. This paper would be of broad interest not only to those concerned with this particular mice model but also generally to those interested in modeling complex behavior traits in mice.

nAChRs are well-characterized transmembrane allosteric oligomers composed of five
identical (homopentamers) or different (heteropentamers) subunits

The issue then becomes how to tackle this problem in mouse models that allow
pharmacological and genetic manipulations, but for which
“psychological” processes must be inferred from observable
behaviors. Mice deleted for ß2-subunit containing nAChR
(ß2−/−) have been the first nicotinic receptor mutant
to be characterized, and found to exhibit more rigid behavior and less behavioral
flexibility than wild-type (WT) animals

ß2−/− mice were shown to be hyperactive in an
open-field paradigm, with a reduced movement at low speed, and consequently an
increased movement at high speed. Hyperactivity in an open field is often used as a
general and non-specific term characterizing experimental conditions where animals
show either an increased amount of displacement and related locomotor behaviors, or
changes in the frequency of specific motor acts

Open-field behaviors have been used to study forced exploration of a new environment.
It has been shown that it involves both exploratory and stress/fear components

We developed further the method already successfully applied to detect modifications
of locomotor behavior caused by mutations in ß2−/−
mice

Both WT and ß2−/− mice were active in the
open-field. They exhibited movements along the wall, sequences of trajectories
in the middle of the field (

(A) Mouse in an open field (1 meter diameter), and two-dimensional trajectory of 30 minutes duration. Position of the animal is here digitized at 25 frames per second. (B) Transformation of continuous variables, velocity and position, into binary symbols. A velocity threshold was set to differentiate inactivity (I - White) and activity (A - Black) periods. Sample of trajectories with two enlarged periods corresponding to an inactivity period and to a velocity decrease following a change in direction (marked by an arrow in the velocity graph) and not identified as an inactivity period. Furthermore, the arena was divided into two concentric zones, P (periphery, shaded) and C (center), the radius of the latter being equal to 0.65. (C) Symbolic sequence analysis: Combining symbols leads to the definition of four states PI | PA | CI | CA. The trajectory is then represented by a sequence of symbols (marked by steps) and associated residence times (τ).

ß2−/− mice have been shown to be hyperactive in the
open-field (Granon et al 2003, Avale et al, 2008), with a distance traveled
during 30 min being 1.25 times longer in KO compared to WT mice (

(A) Boxplot of the total traveled distance and (B) time spent in inactive state during a 30 min session in the open-field respectively for wild-type (WT, n = 32) and mutant mice (ß2−/−, n = 33). (C) Relation between the times spent in a given state (PA and CA in red, PI and CI in black) and the distance traveled during this time. Best linear fits were indicated for active and inactive states with the respective slope (µ). Number of stars indicates the statistical level of significance (- p>0.05, * p<0.05, ** p<0.01, *** p<0.001).

A change in the time spent in inactive states does not give any insight into the
modification of the temporal structure of behaviors. Analysis of transition
frequencies and conditional probabilities between different states of the animal
were then carried out (

(A) Flow diagram: (A1) Transition matrix between the four states can be used to build a flow diagram, where conditional probabilities of transition between states are indicated by number (percentage) and by the thickness of the connecting arrows. Transitions from PI to CA and CI to PA are almost never observed (p<1/1000) and then are not represented in the flow diagram. (A2) Conditional probability of transition from PA to CA depends on previous state. Comparison of P(X|YZ) and P(X|Y) for X = PA, Y = CA (red points) and Z = PA (left) or CI (right) indicates no significant difference (NS). For X = CA, Y = PA (black points) and Z = PA (left) or PI (right) a significant difference appear. (A3) First-order Markov description of the sequence with a distinction between Pap and PAc (see text). (B) First-order Markov description of ß2−/− sequence. Red connecting arrows indicate probabilities of transition that are statistically modified when compared with WT. (C) Comparison of the distributions of time spent within PI, PAp and PAc states respectively (from left to right). Inset: Boxplot of the mean duration of the indicated state. Number of stars indicates statistical level of significance (- p>0.05, * p<0.05, ** p<0.01, *** p<0.001).

Stationarity has been tested by comparing transition probabilities obtained during the first and the second 15 minutes of the experiment. We observed (i) a slight modification of (PI → PA) probability of transition (it decreases from 97.7% to 95.5%, and from 98.0% to 96.0% in WT and ß2−/− respectively), and (ii) an increase of (CA → CI) transition with time (from 22.3% to 32.2 and from 13.9% to 25.3 in WT and ß2−/− mice respectively). This last modification indicates that animals have a higher tendency to stop at the center in the second part of the experiment. This increase is similar in WT and in ß2−/− mice.

Distributions of residence times were also modified in
ß2−/− mice (

In the state sequence, CA is preceded either by PAp, PAc or CI. In WT, there was no significant difference between time distributions of CA, depending on the preceding state (Wilcoxon test). In contrast, CA resident time was increased after a CI when compared with PI preceding a PAp or a PAc (mean = 3.09 against 2.7 and 2.8 sec, Wilcoxon test, p<0.001 in both cases). Similar dependencies on preceding state were observed for PI state duration. Mean duration varied significantly (mean = 4.01, 5.16 and 4.11 sec, Wilcoxon test, p<0.001 in pair comparison) after CI, PAc or PAp, respectively (mean = 4.01, 5.16 and 4.11 sec, Wilcoxon test, p<0.001 in all pair comparisons). Similar properties were observed in ß2−/− mice (mean = 3.08, 4.32 and 4.08 sec, Wilcoxon test, p<0.001 in all pair comparisons).

Deletion of the ß2-subunit gene affected both the residence time
distribution and the transition matrix. To identify more specifically the locus
of the behavioral sequence where the mutation effect takes place, we used a
modeling strategy (see

We first checked the validity of the simulation (see also

(A) Comparison between the total time spent in PI, CI, PAp, CA and PAc states (from left to right) during a 30 min session in the open-field, for WT (black circle) and ß2−/− (red circle) and with the simulation obtained from WT first-order transition matrix and residence time distributions (black triangles) and with the simulation obtained from ß2−/− first-order transition matrix and residence time distributions (red triangles). Note that distributions of experimental and simulated data fit perfectly meaning that the simulations reproduce the dynamics as regards the average time spent in each state. (B) Simulated time spent in PI (left) and PAc (right) obtained by combining transition matrices and distributions of state durations (see text). WT/WT, ß2/WT, WT/ß2 and ß2/ß2 indicate that sequences are simulated using WT or ß2−/− matrices of transition (before /) and WT or ß2−/− state duration distributions (after /). (e.g. WT/ß2 indicates simulation with WT matrix of transition and ß2−/− residence time distribution). “Matrix” and “Time” indicate that the discrepancy originates from the effect of changing the transition matrix and the residence time distribution, respectively.

To further dissect the respective contribution of the transition matrix and of
the residence time distributions, we modeled data based on: (i) transition
matrix of WT and residence time distribution of WT (labeled WT/WT), (ii)
transition matrix of ß2−/− and residence time
distribution of WT (ß2/WT), (iii) transition matrix of WT and
residence time distribution of ß2 (WT/ß2), and (iv)
transition matrix of ß2−/− and residence time of
ß2−/− (ß2/ß2), and we compared
the time spent in PI and in PAc (

A final question was whether a single modification of a WT sequence property
could reproduce most of the ß2−/− phenotype. The
observed behavioral changes between WT and ß2−/−
are open to a variety of interpretations. One of them is that
ß2−/− specifically reduce some stops. The main
advantage of such hypothesis is that modification of only one element (decreased
number of stop) accounts for matrix and residence time difference between WT and
ß2−/− mice. A simple simulation (see

(A) Comparison between the number of PI, CI, PAP, CA and PAc states (from
left to right) simulated using WT first-order transition matrix and
residence time distributions (black circles) and after a transformation
consisting in removing a fixed percentage of inactivity (red circles -
see

Finite-state systems deriving from the discrete analysis of a continuous movement
necessarily coarsen the fine structure of that movement. What has been, so far,
identified as inactivity in this paper, is a mode of motion close to a complete
stop of the animal. During this period of inactivity the mouse can however make
a variety of movements. The animal can progress forward slowly (with a small but
constant speed), freeze, perform a number of action patterns (i.e., grooming,
rearing, scratching, etc), or orienting movements (head scanning, sniffing,
etc). In order to be able to differentiate some of these patterns, we have
simultaneously recorded the position of the animal and digitized video images
(25 frames/second). These images have been used as the input for fine off-line
movement analysis (

(A) Ethogram quantifying activity of the mice during the inactivity
state. Comparison of the percentage of rearing, scanning, grooming, wall
rearing and sniffing in PI behaviors (see

New information obtained by the splitting of PI into five subtypes identified by the dominant behavioral acts, i.e. rearing, scanning, etc., can challenge the description of the sequences in two ways. First, the knowledge of the animal acts during a PI state can modify the probabilities of consecutive states without modifying the first-order Markov description. Second, new information about PI can modify not only the conditional probabilities but also the order of the Markov description, thus requiring a more complex description of the process.

The conditional probability of transition from PA to CA was modified by the
knowledge of the behavioral act performed during stops preceding PA (

Providing new information about the PI state modified the Markov order of the
description. We therefore switched to Variable Length Markov Chain modeling (see

If we consider two main populations of stops, i.e. scanning and no-scanning, a
tree representation of the influence of the past behavior, i.e “the
context”, on a given decision can be built. For this purpose, the
sequence of symbols was fitted using a Variable Length Markov Chain model (VLMC,
see

The WT mice context tree (

Sequences are described using 6 states CI, PAp, CA, Pac, defined as previously, and PInsc and PIsc that correspond to PI without or with scanning. Context tree is drawn in landscape mode with the root (X) placed on the left and past dependencies on the right. Probability distribution over the next symbols appears after each context in red (percentages). For example for WT, (0,0,80,0,16,4) indicates that P(X|CI) = 0; 0; 80; 0; 16 and 4% for X = CI , PAp, CA, PAr, Pinsc and PIsc respectively. Each horizontal line indicates a step in the past. {} indicates a choice between different symbols (A) Fitted context tree (Left) for concatenated sequence of n = 14 WT animals. and schematic representation (Right) of the “choice point”, to enter or not in the center after a PI (B) Fitted context tree (Left) for concatenated sequence of n = 11 ß2−/− animals and schematic representation of chaining (Right).

The context tree of β2−/− mice was made of eight
contexts, four of them (CI, Pap, PInSC, PIsc) being of first order. The
architecture of the tree was clearly modified when compared to WT. Strikingly,
dependence between movements during PI and “transition to
center” completely disappeared. In contrast, the tree highlighted
different chains in the ß2−/− sequence of
behavior, with chains of second or third order that organized movements and
relations between PAc and CA (

In this paper we have investigated the processes underlying ß2−/− mouse hyperactivity in an open field. These mice exhibit an increase in the total distance traveled in the open field by about 40% when compared to WT. Consistent with this hyperactive phenotype, ß2−/− mice spent more time in fast, and less time in slow, movements. To analyze mouse trajectories we developed a specific approach based on a dissection of mouse behavior in the open field as a sequence of motor activities organized in patterns. We have shown evidence for two main modifications of the behavior in ß2−/− mice: (i) quantitatively, mutant mice show a reduced number of stops and modification of specific transition probabilities, and (ii) structurally, the organization of the sequence of behavior was different between strains.

Streams of complex acts or movements exhibit some regularity that is the basis of the
subdivision of behaviors into units, or species-specific movements. In rodents, a
variety of complex sequences of action have been identified

Analysis of behavior in terms of sequences and Markov processes has been already
applied to different species

Hyperactivity in an open field can take different forms, including faster locomotion, longer periods of travel, fewer pauses, shorter pauses, etc. The question is then whether the reduction of the number of stops is sufficient to explain the hyperactive profile. Our experiments demonstrate that locomotion is not faster in ß2−/− mice, and that the difference lies in the patterns and organization of behaviors. Furthermore, a simulation approach suggests that hyperactivity cannot be explained only by changes in the matrix, or only by changes in the duration of the various states, but by their joint effect. Hyperactivity would then emerge from alterations of many different underlying processes. However, we here propose that in ß2−/− mice hyperactivity is mainly due to the “lack of stops”. Most characteristics of the sequences of ß2−/− mice can be explained by the fact that these mice do not observe certain “stops” and that after a stop they organize their behavior differently. The significance of such a modification and the underlying changes it reflects is, however, not trivial.

Open-field behavior, also called exploratory behavior or locomotor behavior in a
novel environment has been initially used as an indicator of anxiety/emotionality

Schematic representation of the modulation of the probability to engage a movement in the center of the arena after a stop at the periphery for WT mice (black) and ß2−/− mice (red). Baseline probability (filled circles and dashed lines) is increased (upward arrow) or decreased (downward arrow) by scanning or recent center excursion respectively. Range between the two baselines (dashed horizontal line) marked baseline difference between WT and ß2−/−. Fear and stress (downward left array) are supposed to decrease center excursion while exploration increases (up-ward left array) it.

The ability to adapt to an unfamiliar or uncertain environment is fundamental, and an
essential point in adaptation would be that animals actively look for a modification
in the environment. Displacement of an animal in a novel environment is
characterized by intermittent locomotion, scanning, and pauses that can be used to
gather information about environment but also to reduce unwanted detection by an
organism's predators

It has been proposed that the alteration of behavioral adaptation in
ß2−/− mice, coupled with unimpaired memory and
anxiety, may model cognitive impairment observed in human disorders

Exploratory activity was recorded in a 1-m diameter circular open-field. Experiments were performed out of the sight of the experimenter and a video camera, connected to a Videotrack system (View-point, Lyon, France), recorded the trajectory of the mouse for 30 minutes. To characterize stopping behavior ethologically, home-made softwares (Labview, National instrument) were used to acquire film with a higher resolution.

Initially introduced in a purely mathematical context, symbolic dynamics has also
been developed as an efficient tool for data analysis

An organism's locomotor behavior consists of an alternation
between progression and stopping. These alternations have been shown to
be ethologically meaningful

Mice in a circular arena travel in both the center and along the perimeter of the open-field. Traveling close to the wall is an important feature of the mice behavior, and it has been suggested that the wall confers security while the center is anxiogenic. However, exploratory behaviors also drive the mouse to explore all the open space. Spatial distribution of mouse position is then expected to be non homogeneous. To account for the spatial organization of the open-field behavior, the arena is then divided into two regions, a central zone C (Centre) and an annulus P (periphery).

When combined, these symbols give four codewords or states

The 2-D paths were smoothed using triangular filter. The instantaneous velocity
can be then meaningfully computed from these smoothed data, simply implementing
its definition (first time-derivative of the position).

Instantaneous velocity range was partitioned in two sub-ranges delineated by the
threshold θ_{1}. A second threshold θ_{2} has
to be involved in order to faithfully assess activity, according to the
following rule:_{v}(t). In other words, it means
that crossing the low threshold _{1} can be considered as the
starting point of a significant active phase if and only if the velocity reaches
the high threshold _{2}. This high threshold determines
qualitatively the active type of the period whereas the low threshold determines
quantitatively its duration. This dual criterion avoids spurious alternation of
active and inactive phases of arbitrary small duration. Indeed, since the
acceleration of the mouse is bounded above by some value a_{max}, the
duration of an active phase is at least
(_{2}-_{1})/_{max}

The two-threshold criterion masks the presence of weak peaks in the velocity that
do not overwhelm significantly _{1} (even if they last long)
while the explicit constraint on duration masks the narrow peaks (fast
fluctuations) even if they reach high velocity values. The combination of these
two criteria moreover ensures that the resulting binary sequence is not very
sensitive to the precise value of _{1} (this feature has also
been checked directly).

The area of the arena was divided in two regions, with a central zone C (Center)
with R_{c}<1 and an annulus P (periphery). Then, depending on the
continuous radial position
R(t) = (x^{2}+y^{2}
)^{1/2}, defined in such a way that it ranges from 0 to 1 depending
on whether the mouse was close to the border of the arena
(_{p}(t) by:

In this study Rc = 0.65.

In order to be able to differentiate patterns of inactivity, video of the animal
displacement was recorded (25 frames/second) and used to detect the position of
the animal. To classify the stops without bias, only parts of the movie
considered as PI in the behavioral sequence were watched without looking either
at the duration of the stops, or at the following sequence. We used five classes
of behavior for this classification, rearing, grooming, border rearing, sniffing
and scanning

Henceforth, we shall call “symbol” each of the 4 codewords PA, PI, CA, CI since the binary symbols will never be considered in isolation in what follows.

One way to analyze a sequence consists in analyzing the probability of transition
from one state to another. From the initial time series written with an alphabet
of x symbol, a x*x matrix T = (tij) can
be calculated, where tij is the number of times a given symbol i is followed by
another symbol j in the sequence. T is called a transition frequency matrix. A
conditional transition matrix can be obtained by dividing each row of the
transition frequency matrix by its sum. Conditional probabilities for each state
are then estimated by unbiased estimator
p(A|B) = n(BA)/n(B) where (n(BA) designates the
number of 2 symbol sub-sequences where B is followed by A. Transition frequency
matrices and conditional transition matrices are a concise way of expressing the
statistical relationship between consecutive states. They give preliminary clues
to the organization of the sequence of states. This is generally summarized in a
flow diagram, giving a simple graphical representation of these matrices. Nodes
in the diagram represent states, while arrows of variable thickness represent
the frequencies with which the different transitions occur. This representation
provides a suitable overview of the organization of the sequence of behaviors
(see

The matrix of transition describes the statistics of transitions from one state
to the other but it does not provide any information about the dynamic nature of
the relationship between successive states. Obtaining information about the
dynamics in short and long terms from the sole knowledge of the transition
matrix is possible only if the dynamics is Markovian: A process is a first-order
Markov chain if the transition probability from state A to the next state B
depends only on the present state A and not on the previous ones. A first-order
Markov model is then a mathematical model fully prescribed by the transition
matrix that describes, in probabilistic terms, the dynamic behavior of the
system, namely the probability of transitions over any duration between any two
states. In such a model, the present state contains all the information that
could influence the choice of the next state, that is captured in the transition
matrix. A classical way to demonstrate that a process is Markovian is to show
that the sequence cannot be described by a zero order process, i.e. that
P(B|A)≠ P(B) and that P(C|B) = P(C|AB),
but see

The residence times, defined as the time spent in a given state, were studied separately. We described the dynamics of transition between states using an alternate renewal process. That is the sequence is described by the convolution of a Markov chain describing the transitions between the states associating a unit time step to each transition, with the above residence-time distributions, describing the actual duration of each step. Thus, there is no repetition of states in the sequence and the transition matrix has vanishing diagonal elements

The most interesting part of the Markov formalism is that the knowledge about the transition probability, i.e. the elementary properties of the system, is sufficient to describe the whole dynamics of the system, either in the short or long term. In practice, this means that as soon as a first-order Markov process has been demonstrated, modifications induced by drugs, genetic mutation or other manipulation of the system can be localized in the transition probabilities and/or in the time distribution of state duration (provided the investigated perturbation does not affect the first-order nature of the dynamics) and the same modeling strategy can be used.

Modeling procedure is as follows. We used (i) the conditional probabilities from a given state to specify the next one, and (ii) the residence time distributions to determine durations of the successive states. This whole procedure is reiterated until the total duration reaches half an hour of experiment. These synthetic data can then be compared with those obtained experimentally. In a second time, specific modification of transition probabilities or residence time distributions are used to access impact of such a modification.

A specific model, consisting in “stop reduction” has been particularly used. In this model, sequences of symbols are generated using WT matrices and distribution. In a second step a fixed percentage of stops (35% of both PI and CI) are removed in such a way that PA-PI-PA becomes PA-PA, that is a unique PA event but with a longer duration (and similarly for CA-CI-CA). The total length of the sequences is adjusted in a way that it represents a half-an-hour experiment.

When the dynamics is not accounted for by a first-order Markov chain, but
displays larger dependence on the past states, “variable length Markov
chains” (VLMC) provide an efficient modeling

A VLMC is thus characterized by: (i) a set of finite-length context, and (ii) a family of transition probabilities associated to each context. The context defines the finite portion of the past that is relevant to predict the next symbol (whatever it is). Given a context, its associated transition probabilities define the distribution of occurrence of the next symbol.

VLMC analyses were performed on concatenated chains obtained from different animals of the same group. The R-package VLMC was used to fit data. Fittings were performed in two steps. First a large Markov chain is generated containing the context states of the time series. In our analysis only nodes that appear n = 5 times per animals (that is 70 for 14 WT and 55 for 11 β2−/−) were taken into account to generate the initial tree. The obtained results are almost insensitive to the value of this parameter n. In the second step, many states of the Markov chains were collapsed by pruning the corresponding context tree. The pruning requires definition of a cutoff value. A large cutoff yields a smaller estimated context tree. In our analysis cutoff value corresponding to 1‰ was used in order to extract strong and significant contexts.

All data were analyzed using R, a language and environment for statistical
computing. Data are plotted as mean±95% confidence
intervals. Boxplot is also used when information about distribution is important
(see

Total number (n) of observations in each group and statistics used are indicated in figure captions. Classically comparisons between two means are performed using two-sample t.test. When there is doubt about the normality of the data distribution, non-parametric Wilcoxon rank-sum test is preferred. For variable Markov chain model fitting, VLMC package is used.

Comparison of simulations using Markov, semi-Markov and non-stationnary
models (see

(1.27 MB TIF)

Simulation of the sequence. (A) Comparison between the number of PI, CI, PAP, CA and PAc states (from left to right) in WT (black circle), ß2−/− (red circles), simulation obtained from WT first-order transition matrix and residence time distributions (black triangle) and simulation obtained from ß2−/− first-order transition matrix and residence time distributions (red triangles). Note that distributions of experimental and simulated data fit perfectly. (B) Typical recurrence plot of an experimental sequence (left) and a simulated sequence, in WT (B1) and in ß2−/− mice (B2).

(7.35 MB TIF)

Supplementary material file and legends for

(0.04 MB DOC)

We are grateful to Pierre Collet for indicating to us references about VLMC and Sylvie Granon for discussion and comments on the manuscript.