There is accumulating evidence that the brain’s neural coding strategies are constrained by natural stimulus statistics. Here we investigated the statistics of the time varying envelope (i.e. a second-order stimulus attribute that is related to variance) of rotational and translational self-motion signals experienced by human subjects during everyday activities. We found that envelopes can reach large values across all six motion dimensions (~450 deg/s for rotations and ~4 G for translations). Unlike results obtained in other sensory modalities, the spectral power of envelope signals decreased slowly for low (< 2 Hz) and more sharply for high (>2 Hz) temporal frequencies and thus was not well-fit by a power law. We next compared the spectral properties of envelope signals resulting from active and passive self-motion, as well as those resulting from signals obtained when the subject is absent (i.e. external stimuli). Our data suggest that different mechanisms underlie deviation from scale invariance in rotational and translational self-motion envelopes. Specifically, active self-motion and filtering by the human body cause deviation from scale invariance primarily for translational and rotational envelope signals, respectively. Finally, we used well-established models in order to predict the responses of peripheral vestibular afferents to natural envelope stimuli. We found that irregular afferents responded more strongly to envelopes than their regular counterparts. Our findings have important consequences for understanding the coding strategies used by the vestibular system to process natural second-order self-motion signals.
Citation: Carriot J, Jamali M, Cullen KE, Chacron MJ (2017) Envelope statistics of self-motion signals experienced by human subjects during everyday activities: Implications for vestibular processing. PLoS ONE 12(6): e0178664. https://doi.org/10.1371/journal.pone.0178664
Editor: Markus Lappe, University of Muenster, GERMANY
Received: November 14, 2016; Accepted: May 17, 2017; Published: June 2, 2017
Copyright: © 2017 Carriot et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This research was supported by Canadian Institutes of Health Research (K.E.C., M.J.C.), and the Canada research chairs (M.J.C.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Understanding the set of transformations by which sensory input gives rise to behavior (i.e. the neural code) remains a central problem in systems neuroscience. Growing evidence suggests that the coding strategies used by sensory systems are adapted to the statistics of natural input [1–9], thus making knowledge of these statistics vital for understanding the neural code. The prevailing view is that natural stimuli display scale invariance (i.e., they are self-similar when observed at different temporal or spatial scales). As a result, their spectral power decays as a power law with increasing spatial or temporal frequency . Studies performed across systems have shown that the properties of sensory neurons optimize their coding of natural stimuli based on both probability of occurrence in the natural environment  as well as their spectral structure. For the latter, optimized coding can be achieved by decorrelating the sensory input: such “temporal whitening” has been observed across systems and species and requires that a neuron’s tuning curve opposes stimulus spectral power such that the neural response to natural stimulation is independent of frequency (i.e., “white”) [4–6, 8, 10, 11].
In the temporal domain, natural stimuli frequently consist of a fast time-varying waveform (a first-order attribute that is also referred to as the carrier) whose amplitude (i.e. a second-order attribute commonly referred to as the envelope) varies independently and more slowly [3, 12–18]. There is accumulating evidence that envelope waveforms carry critical information and thus must be encoded by the brain [14, 19–24]. Notably, as the envelope temporal frequency content differs from that of the carrier, recovering the envelope of a signal (also known as signal demodulation) can only be achieved by nonlinear transformations [19, 25]. This fundamental property required for the demodulation of envelopes complicates efforts to understand how these behaviorally relevant stimulus features are encoded in the brain.
To this end, we took advantage of the vestibular system which is well-defined anatomically and physiologically and benefits from easily characterized sensory stimuli (i.e., head acceleration/velocity). The vestibular system is essential for the generation of the most automatic reflexes, as well as for accurate spatial perception and motor control [26, 27]. Vestibular afferents innervate the receptor cells of the vestibular sensors and provide crucial information about head motion to target neurons in the central vestibular nuclei. In the absence of stimulation, vestibular afferents display a wide range of resting discharge variability and are characterized as regular or irregular- a classification that correlates with differences in morphological features and response dynamics [28–31]. Both afferent classes in turn project to reflex pathways as well as higher brain areas, thereby mediating perception and behavior.
To date, the responses of afferent and their central vestibular neural targets have been almost exclusively characterized using artificial (e.g. sinusoidal, noise) stimuli, leading to the conventional wisdom that early vestibular processing is inherently linear [28, 32, 33]. If this were the case, then single vestibular neurons should not respond to the time varying envelope of self-motion signals. However, recent studies have shown that vestibular neurons respond nonlinearly to naturalistic self-motion stimuli [34, 35] and thus actually respond to envelopes . Furthermore, the encoding of envelopes by the vestibular system may be important for adapting sensory processing to the current stimulus amplitude range, as has been observed behaviorally [37–39]. Recent studies have characterized the statistics of carrier self-motion signals  and shown that the tuning of peripheral afferents is adapted to optimally encode these . However, whether vestibular pathways have also adapted to optimally encode natural second-order self-motion signals based on ther statistics is unknown, in part because these statistics have not been characterized to date.
Informed written consent was obtained from all subjects before the study. All experiments and procedures including obtaining informed written consent from all subjects were approved by McGill University’s Human Ethics Committee. All experiments were furthermore performed in accordance with the guidelines of Ethical Conduct for Research Involving Humans. All data were gathered and previously analyzed for first-order self-motion signal statistics in .
Subjects and head movement recordings
Head movements were recorded in 8 healthy human subjects with no past history of visual or vestibular impairments (4 male, 4 female; age, 22–34 years) during normal everyday activities. We used a micro-electromechanical systems (MEMS) module (iNEMO platform, STEVAL-MKI062V2, STMicroelectronics) that combined three linear accelerometers (linear accelerations along the Fore-Aft, Inter-Aural, and Vertical axes) and was augmented by a STEVAL-MKI107V2 three axis gyroscope (angular velocity about pitch, roll, and yaw). Data from all six sensors were sampled at 100 Hz and recorded on a microSD card. All equipment (MEMS module, battery, microSD card) were positioned on a small light enclosure that could be comfortably attached to the subject’s head or fixed to the environment (e.g., a seat in a vehicle). The Fore-Aft and Inter-Aural axes were set parallel to the subject’s Frankfurt plane (i.e., the plane passing through the inferior margin of the orbit and the upper margin of the external auditory meatus), as done previously . The noise level in the MEMS module was determined by recording signals for 15 minutes while not moving.
Each subject was asked to perform everyday activities each typically lasting 2 minutes in a random order. Activities consisted either of voluntary (i.e. active) (walking, going up and down the stairs, running, running through the woods, sprinting, jumping forward, jumping up and down, hopping on one foot, playing soccer, biking on a city street, biking on a grassy field) or passively applied (riding the city subway seated, riding the city subway standing up, riding a city bus seated and, riding a city bus standing up) self-motion.
Recorded angular velocity signals were projected onto the semicircular canal planes (left anterior–right posterior [LARP], right anterior–left posterior [RALP], and YAW) as done previously . The signals obtained for different activities (i.e., passive, active, or both) were then concatenated for each subject as done previously . We note that this approach is similar to that used in other systems [41–43]. The time varying amplitude or envelope E(t) was then extracted from each resulting signal S(t) (i.e. either angular velocity along the YAW, LARP, and RALP axes; or linear acceleration along the Fore-Aft, Inter-Aural, and Vertical axes) using the Hilbert transform [21, 36, 44, 45]: where C[…] is the Cauchy principal value. Probability distributions were obtained using binwidths of 0.01 G and 10 deg/s for linear acceleration and angular velocity, respectively. Since the envelope can only be positive by definition, we fitted a half-Gaussian to the probability distribution. The excess kurtosis was then computed as: where μ and σ are the mean and standard deviation of the distribution, respectively.
Power spectral densities of the signals recorded during self-motion were computed using Welch’s average periodogram with 512 points and a Bartlett window (512 ms duration).
Power spectral densities were fit using a single power law model (model 1): where A is a constant, f is frequency, P is power, and α is the power law exponent. In practice, parameter values were obtained by performing a linear least squares fit on the logarithms of the power spectral density and frequency over the range 0–50 Hz.
We also used a double power law model (model 2): Here A1, A2 are constants, α1, α2 are the power law exponents, and ft is the transition frequency. All parameter values were obtained by performing a linear least squares fit on the logarithms of the power spectral density and frequency over the ranges 0-ft and ft-50 Hz.
The cutoff frequency ft was determined in the following way. The goodness of fit of each model was assessed by computing the variance-accounted-for given by: where VAR(…) is the variance, y is the data, is the fit to the data, and is the mean of the data. The sampling interval of the data increased exponentially, such that the datapoints were evenly spaced when taking the logarithms of power spectral density and frequency. This was done in order to give equivalent weighting for low and high values of the logarithm of frequency. The transition frequency was chosen as that for which the VAF was maximized.
We determined which model was the best fit to the data by testing whether the low frequency power law exponent was significantly different than the high frequency power law exponent when using model 2. If this difference was not significantly different from zero, we used model 1 to fit the data. Otherwise, if the difference was significantly different from zero, then we used model 2 to fit the data.
Values are reported as mean ± STD throughout unless otherwise noted. The shaded gray bands in the figures show 1 STD.
We first used previously established linear models to predict afferent responses to the experimentally recorded natural stimuli. Specifically, we assumed that the output firing rate r(t) in response to stimulus S(t) is given by the following: r(t) = r0 + (H * s)(t), where the asterisk denotes a convolution with a filter H(t) and r0 is the baseline (i.e., in the absence of stimulation) firing rate. We used r0 = 100 Hz which corresponds to the average baseline firing rate observed experimentally . We used standard expressions for the Fourier transform of H(t) (i.e., the transfer function) [34, 35]: where f is frequency and . For regular afferents, we used k = 2.83 (spk/sec)/(deg/sec), T1 = 0.0175 s, T2 = 0.0027 s, and Tc = 5.7 s. For irregular afferents, we used k = 27.09 (spk/sec)/(deg/sec), T1 = 0.03 s, T2 = 0.0006 s, and Tc = 5.7 s. The output firing rate was then passed through a clipping nonlinearity: rectification was implemented by setting negative values of r(t) to zero while saturation was implemented by setting values of r(t) greater than 400 spk/sec to 400 spk/sec. These values were taken based on experimental observations [34, 46] and the value used for saturation did not qualitatively affect the nature of our results (not shown).
Finally, the sensitivity to the envelope was computed as: where Per(f) is the cross-spectrum between the output firing rate after implementing the clipping nonlinearity and the envelope, and Pee(f) is the power spectrum of the envelope. We note that the sensitivity is the ratio of the output to the input amplitude at a given frequency and has been previously used to quantify tuning to envelopes in the electrosensory system [8, 10, 21, 47] as well as in the auditory system [48, 49] (see  for review).
Envelope statistics of vestibular signals during natural self-motion
As mentioned above, it is important to note that the envelope signal can only be extracted by a nonlinear transformation. For example, consider the waveform shown in Fig 1A consisting of a sinusoidal carrier whose amplitude (i.e. envelope) is also varying sinusoidally at a lower frequency. Fourier analysis (which is a linear transformation) performed on the complete waveform reveals that power is present only at the high frequency content of the carrier but not at the low frequency content of the envelope (Fig 1A, top). Thus, to extract the envelope, nonlinear transformations (e.g., half or full-wave rectification) are necessary (Fig 1A, bottom).
A: Schematic showing a sinusoidal trace whose amplitude also varies sinusoidally (blue trace, top left). The power spectrum is non-zero only at the carrier frequency Fc (top right). The envelope of the signal (red trace, bottom left) oscillates with a different frequency Fe than that of the full signal as confirmed by taking its power spectrum (bottom right). B: A MEMS module consisting of three gyroscopes and three linear accelerometers was mounted on the subject’s head and measured linear accelerations along the Fore-Aft, Inter-Aural and Vertical axis as well as rotations about the Pitch, Yaw, and Roll axes. C: Example signal (gray) recorded from the MEMS module and its time varying envelope (red). D,E,F,G,H,I: Example angular velocity or linear acceleration envelope signals recorded during everyday activities for Inter-Aural (D), Fore-Aft (E), Vertical (F), LARP (G), RALP (H), and YAW (I). In each case, shown are an example time series (left), the probability distributions plotted using logarithmic (middle left) and linear (middle right) scales, together with a Gaussian fit (dashed black), and the population-averaged excess Kurtosis (right). Gray bands show 1 STD.
In a previous analysis we characterized the first-order statistics of self-motion stimuli (i.e., also referred to as the carrier) experienced by human subjects during natural everyday behaviors (e.g. running, jumping, riding in a vehicle) . Stimuli along six axes of translational and rotational motion were measured using a portable MEMS module that was attached to the subject’s head (Fig 1B). Angular velocity signals were projected onto each subject’s semicircular canal planes (LARP, RALP, and YAW) prior to analysis. Here we instead characterized the second-order statistics of self-motion stimuli (also referred to as the envelope) by applying a nonlinear transformation (see Methods) as done previously for other sensory modalities [44, 45, 50] (Fig 1C).
The envelope statistics of vestibular signals for all activities are summarized in Table 1 (passive) and Table 2 (active). Overall, these signals could reach high values (~450deg/s and ~4G) that varied greatly across activities. Envelope signals were characterized by probability distributions with long tails that decreased more slowly than a half-Gaussian distribution as quantified by large excess kurtosis values (Fig 1D, 1E, 1F, 1G, 1H and 1I). Previous studies performed in other systems have shown that envelope signals are scale invariant (i.e., look similar at different spatial and temporal timescales) [18, 20, 51]. A characteristic of scale invariance is that spectral power will decrease as a power law as a function of frequency. Thus, to test whether the envelopes of natural self-motion signals were scale invariant, we computed their power spectra as a function of temporal frequency. We found that these spectra decreased more sharply for high (> 2 Hz) than for low (<2 Hz) frequencies (Fig 2A) and were thus not well fit by a single power law (blue lines), indicating deviation from scale invariance. The spectra were however better fit by two power laws with exponents near -1 and -3 over the low and high frequency ranges (black lines), respectively (Fig 2A and 2B). The population-averaged best-fit exponents over the low and high frequency ranges were significantly different from one another, and were furthermore significantly different from the best-fit exponent of the single power law model for all six motion dimensions (p<0.01 in all cases, one-way ANOVAs, Fig 2B). The frequency at which the transition from a slow to a fast decrease occurred ranged between 4 and 15 Hz across motion dimensions (Fig 2C). Thus we conclude that the envelopes of vestibular signals encountered across everyday activities that include both active and passive self-motion are not scale invariant prior to reaching the sensory organs in the subject’s head. This result has important consequences for neural coding as further discussed below.
The maximum and mean values are expressed in mG for the Lateral, Fore-Aft and Vertical linear acceleration while they are expressed in deg/s for the LARP, RALP and Yaw angular velocity.
The maximum and mean values are expressed in mG for the Lateral, Fore-Aft and Vertical linear acceleration while they are expressed in deg/s for the LARP, RALP and Yaw angular velocity.
A: Subject-averaged power spectra (red lines) with best-fit power laws over the low and high frequency ranges (black lines) as well as best-fit single power law over the entire frequency range (blue lines). Also shown are the best-fit power law exponents with confidence interval as well as the transition frequency. The dashed gray lines show the “noise floor”, which is the spectrum of the noise in the measurement obtained when the sensor was not moving (see Methods). Gray bands show 1 STD. B: Subject-averaged best-fit power law exponents over the low (gray) and high (black) frequency ranges for all six motion dimensions. Also shown for comparison are the subject-averaged best-fit power law exponents for a single power law over the entire frequency range (blue). “*” indicates statistical significance at the p = 0.01 level using a one-way ANOVA. C: Subject-averaged frequency at which the power spectrum starts decreasing more sharply for all six motion dimensions.
Voluntary self-motion causes deviation from scale invariance primarily for translational envelope signals
What causes deviation from scale invariance in the envelopes of natural self-motion signals? Previous studies of other sensory modalities have shown that active movement can alter the statistics of natural visual input impinging upon sensors . If this is the case in the vestibular system, then the deviation from scale invariance seen in the envelope of vestibular signals should be due to voluntary movements made during everyday activities (e.g., walking).
To test whether active movements contribute to causing deviation from scale invariance, we segregated self-motion signals resulting primarily from active activities from those resulting primarily from passive activities (Fig 3A) and compared the power spectra of their respective envelopes. We found that the envelope power spectra for signals resulting from active motion were qualitatively similar to those obtained across our entire dataset (compare Figs 3B, 3C, 3D, 3E, 3F and 3G to 2A). Indeed, power spectra for signals resulting from active motion decayed more slowly over low frequencies and more sharply over high frequencies (Fig 3B, 3C, 3D, 3E, 3F and 3G, left panels). Consequently, these were well fit by two power laws with different exponents over the low and high frequency ranges (black lines) rather than a single power law over the entire frequency range (blue lines) (Fig 3B, 3C, 3D, 3E, 3F and 3G, left panels). Indeed, the population-averaged best-fit low and high frequency power law exponents were almost always significantly different from one another as well as from the best-fit single power law exponent (Fig 4A).
A: Schematic showing a subject engaged in active self-motion (left) and in passive self-motion (right). B,C,D,E, F, G: Subject-averaged envelope power spectra for active (left panels) and passive (right panels) activities for inter aural (B), Fore-Aft (C), Vertical (D), LARP (E), RALP (F), and YAW (G). In each case, the power spectra were fitted using two power laws over the low and high frequency ranges (black lines) as well as by a single power law over the entire frequency range (blue lines). Also shown are the best-fit power law exponents with confidence interval as well as the transition frequency.
A: Subject-averaged best-fit power law exponents over the low (gray) and high (black) frequency ranges for all six motion dimensions for active self-motion. Also shown for comparison are the subject-averaged best-fit power law exponents for a single power law over the entire frequency range (blue). B: Subject-averaged best-fit power law exponents over the low (gray) and high (black) frequency ranges for all six motion dimensions for passive self-motion. Also shown for comparison are the subject-averaged best-fit power law exponents for a single power law over the entire frequency range (blue). “*” indicates statistical significance at the p = 0.01 level using a one-way ANOVA.
We next compared the power spectra of envelope signals resulting from passive motion (Fig 3B, 3C, 3D, 3E, 3F and 3G, right panels) to those of envelope signals resulting from active motion (Fig 3B, 3C, 3D, 3E, 3F and 3G, left panels). For Inter-Aural and Vertical translations, we found that the power spectra resulting from passive activities tended to decay more uniformly as a function of increasing frequency than those from active activities (Fig 3B and 3D, compare left and right panels). Although these spectra could also be well fit by two power laws over the low and high frequency ranges, the low and high frequency best-fit power law exponents obtained by using a two power law model were similar to one another in value. Indeed, further analysis revealed that the two best-fit power law exponents were not significantly different from one another, or from the exponent obtained by fitting a single power law over the entire frequency range (Fig 4B). These results suggest that active movements strongly contribute to causing deviation from scale invariance for translational envelope signals along these axes.
For rotations (i.e., LARP, RALP, and YAW) as well as Fore-Aft translations, we found that the envelope power spectra of signals resulting from active and passive activities were more similar in structure as they both decayed more slowly for low frequencies and more sharply for high frequencies (Fig 3C, 3E, 3F and 3G, compare left and right panels). Consequently, the envelope spectra of rotational signals resulting from passive activities were better fit by two power laws with different exponents over the low and high frequency ranges than by a single power law over the entire frequency range. Further analysis revealed that the two best-fit power law exponents were for the most part significantly different from one another as well as from the exponent obtained by fitting a single power law over the entire frequency range (Fig 4B). Our results thus suggest that active self-motion at best contributes minimally to causing deviation from scale invariance for rotational self-motion envelopes as well as Fore-Aft translations.
Filtering by the human body gives rise to deviation from scale invariance primarily for rotational self-motion
We next investigated whether filtering by the human body could contribute to causing deviation from scale invariance in envelope self-motion signals. This is because previous studies have shown that such filtering causes deviation from scale invariance for carrier self-motion signals . Indeed, vestibular signals experienced during typical everyday activities are transmitted through the body before reaching the vestibular sensors in the head. For example, when a person is riding in a vehicle, vibrations from the ground travel through the subject’s body prior to reaching the head. Similarly, filtering by the human body will also be present during active self-motion (e.g., vibrations caused by the foot striking the ground during walking travel through the subject’s body prior to reaching the head).
To test whether filtering by the human body contributes to causing deviation from scale invariance for natural self-motion envelopes, we compared envelope signals obtained during passive self-motion measured at the subject’s head to those measured when the subject is absent (i.e. external stimuli) (Fig 5A). Specifically, we investigated the contributions of filtering by the human body during passive self-motion in order to distinguish them from the potential effects of active self-motion. Our results show that, overall, the power spectra of self-motion envelope signals measured externally were well-fit by a single power law over the entire frequency range across all six motion dimensions (Fig 5B, 5C, 5D, 5E, 5F and 5G). Indeed, the low and high frequency best-fit power law exponents were not significantly different from one another or from the one obtained by fitting a single power law over the entire frequency range (Fig 6). We note that the power spectra of external stimuli are lower than that measured when the subject is present (compare curves in Figs 3 and 5). These differences are likely due to resonance properties of the human body (see e.g. ) whose frequency highly depends on posture (see e.g. ).
A: Schematic showing the MEMS module (gold box) located on the subject’s head and placed on the seat during passive self-motion. B,C,D,E, F, G: Trial-averaged power spectra of signals in the external environment (green) during passive self-motion for inter aural (B), Fore-Aft (C), Vertical (D), LARP (E), RALP (F), and YAW (G). The power spectra were in general well fit by a single power law over the entire frequency range (blue lines).
Subject-averaged best-fit power law exponents for the envelopes of external stimuli during passive self-motion when fitting a power law over the entire frequency range (blue) and when fitting two power laws over the low (gray) and high (black) frequency ranges.
When considering rotational and Fore-Aft translational envelope signals, the power spectra of signals measured at the subject’s head during passive self-motion decayed slowly for low and more sharply for high frequencies (Fig 3C, 3E, 3F and 3G, right panels) whereas those measured when the subject is absent instead decayed uniformly (Fig 5C, 5E, 5F and 5G). These results suggest that filtering by the human body causes significant deviation from scale invariance for rotational envelope signals and Fore-Aft translations. However, when instead considering Inter-Aural and Vertical translations, the power spectra of envelope signals measured at the subject’s head (Fig 3B and 3D, right panels) and when the subject is absent (Fig 5B and 5D) all tended to decay uniformly with increasing frequency. These results suggest that filtering by the human body causes minimal deviation from scale invariance for Inter-Aural and Vertical translational envelope signals.
Thus, our results suggest that translational and rotational envelope signals deviate from scale invariance primarily for different reasons. Specifically, while active self-motion makes the primary contribution for the former, filtering by the body instead makes the primary contribution for the latter. The one notable exception to this rule is Fore-Aft translations for which filtering by the human body rather than active self-motion causes deviation from scale invariance.
Predicting afferent responses to natural envelopes
So far we have focused on characterizing the statistics of natural self-motion envelopes as well as potential mechanisms that cause deviation from scale invariance in their structure. In the following, we instead focus on making predictions as to how peripheral vestibular afferents respond to natural self-motion envelopes. To do so, we used well-established models that reproduce the response dynamics of afferents seen experimentally (Fig 7A, see Methods). Specifically, we first used transfer functions based on experimental findings  to predict the firing rate response to the carrier signal. Importantly, the sensitivity of the model irregular afferent to the carrier was higher than that of its regular counterpart across the relevant frequency range [28, 55] (Fig 7B). Fig 7C shows the predicted responses of the model regular and irregular afferents to a natural stimulus. Notably, the stimulus gave rise to greater changes in firing rate for the model irregular afferent because of its higher sensitivity. As such, the model irregular afferent tends to be driven more into cutoff (i.e. cessation of activity) and saturation than its regular counterpart (Fig 7C). In order to quantify tuning to the envelope, we computed the sensitivity as a function of temporal frequency (see Methods). This is a standard measure that has been used previously to characterize neural responses to envelopes in the electrosensory system [8, 10, 21] and that is equivalent to temporal modulation transfer function measures that have been used extensively to characterize neural responses to envelopes in the auditory system [48, 49] (see  for review). Fig 7D shows the envelope sensitivity as a function of frequency for both the model regular and irregular afferents in response to the envelope. Both were relatively independent of envelope frequency but the envelope sensitivity computed for the model irregular afferent was approximately twice that computed for the model regular afferents. Thus, our simulations predict that irregular afferents will display higher sensitivities to envelopes than their regular counterparts.
A: Schematic showing the vestibular end organs as well as regular and irregular vestibular afferents projecting to the vestibular nuclei. B: Sensitivity to the carrier for the regular (dashed black) and irregular (solid red) model afferents. C: Time series showing a segment of the envelope stimulus (solid black) and the responses of the model regular (dashed black) and irregular (solid red) afferents. D: Gain to the envelope as a function of frequency for the regular (dashed black) and irregular (solid red) model afferents. In both cases the gain is relatively independent of frequency but is about twice higher for the irregular model afferent.
Summary of results
We investigated the envelope statistics of self-motion stimuli experienced by human subjects during everyday activities. We found that these could reach high values (~450deg/s for rotations and ~4G for translations), were characterized by probability distributions with high kurtosis, and displayed power spectra that decreased slowly for lower (< 2 Hz) and more steeply at higher (> 2 Hz) frequencies. These statistics were seen across all six motion dimensions. We found that different mechanisms underlie deviation from scale invariance depending on whether one considers translational or rotational self-motion envelopes. Indeed, our data suggests that active self-motion and filtering by the human body make the primary contribution to deviation from scale invariance for the former and latter, respectively. The one notable exception to this rule is Fore-Aft translations, for which filtering causes deviation from scale invariance. To understand the implications of the present findings for envelope coding by the vestibular system, we used well-established models of the vestibular periphery to simulate afferent responses to natural envelope stimuli. Our simulations predict that irregular afferents are more sensitive to envelopes than their regular counterparts.
Functional roles of envelopes in vestibular pathways
Envelopes can carry behaviorally relevant information. For example, in the visual system, these are crucial for edge detection in visual scenes [56, 57] while, in the auditory system, they carry crucial information required to perceive timbre in music as well as speech perception [14, 22, 23]. In the active electric sense of weakly electric fish, envelopes carry crucial information about both distance and identity of conspecifics [19, 20]. While previous studies carried out in other systems have shown that natural envelope signals display scale invariance , our results suggest that natural envelope self-motion signals instead display deviation from scale invariance due to active self-motion and filtering by the human body. This is interesting, since studies of natural stimuli have typically looked at the stimuli themselves (e.g., natural visual images) without taking into account active movements (e.g., eye saccades when freely viewing an image). Indeed, a recent study has shown that active eye movements cause deviation from scale invariance in natural first-order visual signals . It is thus conceivable that active motion will also cause deviation from scale invariance for second-order (i.e. envelope) sensory signals in other systems.
While the functional role of envelopes has not been fully established in the vestibular system, there is evidence that their detailed structure is processed and retained in vestibular pathways. We speculate that envelope coding is important for central processes that integrate vestibular input over time to adapt to the current amplitude range of self-motion stimuli. Indeed, there is evidence that vestibular reflexive and perceptual responses to a sustained directional stimulus are reduced over time [38, 39], and that vestibular perceptual and balance responses are regulated, over the course of minutes, as a function of the self-motion envelope . Furthermore, psychophysical studies in humans have suggested that a mechanism for inducing motion sickness involves integrating the amplitude of vibrations over time . The regulation of amplitude range, reciprocal connections between the vestibular cerebellum (i.e., cerebellar nodulus and uvula) and vestibular nuclei are known to lengthen the time constant of the semicircular canals. This process, termed velocity storage, shapes the dynamics of both the perception of self-motion and vestibular-driven behaviors. Notably, motion sickness sensitivity is decreased following training that reduces velocity storage [59–65], providing further support for the proposal that motion sickness is triggered by the integration of motion stimuli over time. Moreover, anti-motion sickness drugs enhance adaptation of this mechanism allowing progressive exposure to higher levels of stimulation without symptoms being elicited [66–68]. Interestingly, alterations of velocity storage may also contribute to vertigo susceptibility in vestibular migraine patients [69, 70], suggesting that the envelopes of vestibular signals have additional clinical relevance.
Envelope coding in vestibular pathways: Functional role of neuronal variability
It is well-known that vestibular afferents display strong heterogeneities in their responses to self-motion stimulation that are in part due to differential hair cell morphology and patterns of innervation. These neurons are typically classified as either regular or irregular based on their resting discharge variability . Despite over 40 years of work, the functional role of each afferent class is still not fully understood.
As stated above, the envelope of a signal can only be extracted mathematically by performing a nonlinear transformation. The conventional wisdom is that early vestibular processing is inherently linear [28, 32, 33, 71]. However, the stimuli used in these previous studies consisted of artificial sinusoidal and noise stimuli whose amplitude is actually much lower than that seen in natural self-motion [34, 40]. More recent studies have shown that semi-circular and otolith afferents as well as central vestibular neurons display strong nonlinearities in their responses to naturalistic signals [34, 35]. However, static nonlinearities such as rectification and saturation, which are necessary for a neuron to encode second-order attributes [14, 19], tend to be more reliably elicited for irregular afferents experimentally as these tend to have higher sensitivities to carrier self-motion signals than their regular counterparts . Such nonlinearities are necessary in order for neurons to respond to envelopes [14, 19] and our simulations predict that they will give rise to envelope responses in vestibular afferents that will be transmitted to higher order brain areas. Moreover, our simulations predict that regular and irregular afferents have different functional roles for envelope coding. If correct, this would provide new insight into the longstanding problem of why the primate vestibular system has two afferent classes. Further studies are however needed to verify these predictions and, if true, characterize the tuning properties of individual regular and irregular afferents, as well as those of central vestibular neurons, to envelopes.
Comparison between the statistics and coding of carrier and envelope self-motion signals
Our results show that active movements cause deviations from scale invariance for translational self-motion envelope signals prior to sensory transduction. As such, our results strongly differ from those of a recent study that instead investigated the statistics of carrier self-motion signals . Indeed, this prior study reported that filtering by the body is primarily responsible for deviations from scale invariance in both translational and rotational carrier self-motion signals prior to reaching the vestibular sensors in the head . Thus, the mechanisms that cause deviation from scale invariance in carrier and envelope self-motion signals are different when considering Inter-Aural and Vertical translations and similar when instead considering rotations and Fore-Aft translations.
This has important implications for neural coding as there is growing evidence that sensory systems can efficiently process natural stimuli by ensuring that coding strategies are matched to input statistics [1–5, 11, 72]. While the statistics of natural stimuli in other sensory modalities (e.g. auditory, visual) have been known for quite some time [12, 73], the statistics of natural self-motion stimuli have only been investigated recently in humans  and non-human primates . Importantly, a recent study has shown that irregular semicircular and otolith vestibular afferents can more efficiently encode natural carrier self-motion signals than their regular counterparts, suggesting that the coding strategies used by the primate vestibular system are adapted to natural carrier self-motion statistics . We speculate that the probability distributions of envelope signals presented in the current study, together with the tuning properties of afferents to envelopes, might show that irregular afferents are more adapted to natural envelope statistics than their regular counterparts. Moreover, we predict that, if vestibular coding strategies are matched to natural self-motion statistics, then our results showing that translational envelopes resulting from active and passive self-motion have fundamentally different statistics implies that these should be processed differentially in the brain. Further experimental studies are however needed to test these predictions.
Parallel processing of carrier and envelope signals
The coding of both carrier and envelope components of natural stimuli remains an important problem in systems neuroscience. While the statistics of carrier vestibular signals have been recently reported , the statistics of envelope vestibular signals had not been investigated prior to this study. Our results characterizing the statistics of natural envelope vestibular signals pave the way for future electrophysiological investigations aimed at understanding how these signals are processed in the brain. To that effect, a general strategy used by the brain to encode both components is to devote separate neural circuits in order to encode each. Indeed, such parallel processing is thought to occur in the visual system [56, 57, 74] and has been demonstrated in the electrosensory system . Based on arguments presented above, it is possible that such parallel processing might begin to occur as early as the vestibular periphery since irregular afferents are predicted to respond more strongly to envelope self-motion signals than their regular counterparts. However, how central vestibular neurons integrate input from both afferent classes in order to ensure that both carrier and envelope components are accurately represented is not clear and should be the focus of future studies.
This research was supported by Canadian Institutes of Health Research (K.E.C., M.J.C.), and the Canada research chairs (M.J.C.). M.J.C. and K.E.C. designed research, J.C. and M.J. performed research and analyzed data, M.J.C. and K.E.C. wrote the manuscript. The authors declare no competing financial interests.
- Conceptualization: KEC MJC.
- Data curation: JC.
- Formal analysis: JC MJ.
- Funding acquisition: KEC MJC.
- Investigation: JC MJ.
- Methodology: KEC MJC MJ JC.
- Project administration: KEC MJC.
- Resources: KEC MJC.
- Software: JC MJ.
- Supervision: KEC MJC.
- Validation: JC MJ KEC MJC.
- Visualization: JC.
- Writing – original draft: JC MJ KEC MJC.
- Writing – review & editing: JC MJ KEC MJC.
- 1. Laughlin S. A simple coding procedure enhances a neuron's information capacity. Z Naturforsch C. 1981;36(9–10):910–2. Epub 1981/09/01. pmid:7303823.
- 2. Wark B, Lundstrom BN, Fairhall A. Sensory adaptation. Current Opinion in Neurobiology. 2007;17(4):423–9. pmid:17714934;
- 3. Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annual Review of Neuroscience. 2001;24:1193–216. pmid:11520932
- 4. Dan Y, Atick JJ, Reid RC. Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. J Neurosci. 1996;16(10):3351–62. pmid:8627371.
- 5. Pozzorini C, Naud R, Mensi S, Gerstner W. Temporal whitening by power-law adaptation in neocortical neurons. Nat Neurosci. 2013;16(7):942–8. pmid:23749146.
- 6. Rodriguez FA, Chen C, Read HL, Escabi MA. Neural modulation tuning characteristics scale to efficiently encode natural sound statistics. J Neurosci. 2010;30(47):15969–80. Epub 2010/11/26. pmid:21106835;
- 7. Attias H, Schreiner CE. Coding of naturalistic stimuli by auditory midbrain neurons. In: Jordan M, Kearns M, Solla S, editors. Advances in Neural Information Processing Systems. 10. Cambridge: MIT press; 1998. p. 103–9.
- 8. Huang CG, Zhang ZD, Chacron MJ. Temporal decorrelation by SK channels enables efficient neural coding and perception of natural stimuli. Nature communications. 2016;7:11353. pmid:27088670;
- 9. Zhang ZD, Chacron MJ. Adaptation to second order stimulus features by electrosensory neurons causes ambiguity. Sci Rep. 2016;6:28716. pmid:27349635;
- 10. Huang CG, Chacron MJ. Optimized Parallel Coding of Second-Order Stimulus Features by Heterogeneous Neural Populations. J Neurosci. 2016;36(38):9859–72. pmid:27656024.
- 11. Wang XJ, Liu Y, Sanchez-Vives MV, McCormick DA. Adaptation and temporal decorrelation by single neurons in the primary visual cortex. J Neurophysiol. 2003;89(6):3279–93. pmid:12649312.
- 12. Attias H, Schreiner CE. Low-order temporal statistics of natural sounds. Advances in Neural Information Processing Systems. 1997;9:27–33.
- 13. Lewicki MS. Efficient coding of natural sounds. Nature Neuroscience. 2002;5(4):356–63. Epub 2002/03/16. pmid:11896400.
- 14. Heil P. Coding of temporal onset envelope in the auditory system. Speech Communication. 2003;41:123–34.
- 15. Joris PX, Schreiner CE, Rees A. Neural processing of amplitude-modulated sounds. Physiol Rev. 2004;84(2):541–77. Epub 2004/03/27. pmid:15044682.
- 16. Mante V, Frazor RA, Bonin V, Geisler WS, Carandini M. Independence of luminance and contrast in natural scenes and in the early visual system. Nature Neuroscience. 2005;8(12):1690–7. Epub 2005/11/16. nn1556 [pii] pmid:16286933.
- 17. de Kock CP, Sakmann B. Spiking in primary somatosensory cortex during natural whisking in awake head-restrained rats is cell-type specific. PNAS. 2009;106(38):16446–50. Epub 2009/10/07. 0904143106 [pii] pmid:19805318;
- 18. Theunissen FE, Elie JE. Neural processing of natural sounds. Nat Rev Neurosci. 2014;15(6):355–66. pmid:24840800.
- 19. Stamper SA, Fortune ES, Chacron MJ. Perception and coding of envelopes in weakly electric fishes. J Exp Biol. 2013;216:2393–402. pmid:23761464.
- 20. Metzen MG, Chacron MJ. Weakly electric fish display behavioral responses to envelopes naturally occurring during movement: implications for neural processing. J Exp Biol. 2014;217(Pt 8):1381–91. Epub 2013/12/24. pmid:24363423;
- 21. Metzen MG, Chacron MJ. Neural heterogeneities determine response characteristics to second-, but not first-order stimulus features. J Neurosci. 2015;35(7):3124–38. Epub 2015/02/24. pmid:25698748;
- 22. Shannon RV, Zeng FG, Kamath V, Wygonski J, Ekelid M. Speech recognition with primarily temporal cues. Science. 1995;270(5234):303–4. pmid:7569981.
- 23. Shannon RV, Zeng FG, Wygonski J. Speech recognition with altered spectral distribution of envelope cues. J Acoust Soc Am. 1998;104(4):2467–76. Epub 1999/09/24. pmid:10491708.
- 24. Bertoncini J, Serniclaes W, Lorenzi C. Discrimination of speech sounds based upon temporal envelope versus fine structure cues in 5- to 7-year-old children. J Speech Lang Hear Res. 2009;52(3):682–95. Epub 2008/10/28. pmid:18952853.
- 25. Chang AEB, Vaughan AG, Wilson RI. A Mechanosensory Circuit that Mixes Opponent Channels to Produce Selectivity for Complex Stimulus Features. Neuron. 2016;92:888–901. pmid:27974164
- 26. Cullen KE. The neural encoding of self-motion. Curr Opin Neurobiol. 2011;21(4):587–95. pmid:21689924.
- 27. Cullen KE. The vestibular system: multimodal integration and encoding of self-motion for motor control. Trends Neurosci. 2012;35(3):185–96. pmid:22245372;
- 28. Goldberg JM. Afferent Diversity and the Organisation of central vestibular pathways. Exp Brain Res. 2000;130:277–97. pmid:10706428
- 29. Baird RA, Desmadryl G, Fernandez C, Goldberg JM. The vestibular nerve of the chinchilla. II. Relation between afferent response properties and peripheral innervation patterns in the semicircular canals. J Neurophysiol. 1988;60(1):182–203. Epub 1988/07/01. pmid:3404216.
- 30. Fernandez C, Baird RA, Goldberg JM. The vestibular nerve of the chinchilla. I. Peripheral innervation patterns in the horizontal and superior semicircular canals. J Neurophysiol. 1988;60(1):167–81. Epub 1988/07/01. pmid:3404215.
- 31. Straka H, Vibert N, Vidal PP, Moore LE, Dutia MB. Intrinsic membrane properties of vertebrate vestibular neurons: function, development and plasticity. Progress in neurobiology. 2005;76(6):349–92. Epub 2005/11/03. S0301-0082(05)00116-4 [pii] pmid:16263204.
- 32. Massot C, Chacron MJ, Cullen KE. Information transmission and detection thresholds in the vestibular nuclei: single neurons versus population encoding. J Neurophysiol. 2011;105:1798–814. pmid:21307329
- 33. Bagnall MW, McElvain LE, Faulstich M, du Lac S. Frequency-independent synaptic transmission supports a linear vestibular behavior. Neuron. 2008;60(2):343–52. Epub 2008/10/30. pmid:18957225;
- 34. Schneider AD, Jamali M, Carriot J, Chacron MJ, Cullen KE. The increased sensitivity of irregular peripheral canal and otolith vestibular afferents optimizes their encoding of natural stimuli. J Neurosci. 2015;35(14):5522–36. pmid:25855169;
- 35. Massot C, Schneider AD, Chacron MJ, Cullen KE. The vestibular system implements a linear-nonlinear transformation in order to encode self-motion. PLoS Biol. 2012;10:e1001365. pmid:22911113
- 36. Metzen MG, Jamali M, Carriot J, Avila-Akerberg O, Cullen KE, Chacron MJ. Coding of envelopes by correlated but not single-neuron activity requires neural variability. PNAS. 2015;112(15):4791–6. Epub 2015/04/01. pmid:25825717;
- 37. Fitzpatrick RC, Watson SR. Passive motion reduces vestibular balance and perceptual responses. J Physiol. 2015;593(10):2389–98. pmid:25809702;
- 38. St George RJ, Day BL, Fitzpatrick RC. Adaptation of vestibular signals for self-motion perception. J Physiol. 2011;589(Pt 4):843–53. pmid:20937715;
- 39. Guedry FE, Lauver LS. Vestibular reactions during prolonged constant angular acceleration. Journal of Applied Physiology. 1961;16:215–20.
- 40. Carriot J, Jamali M, Chacron MJ, Cullen KE. Statistics of the vestibular input experienced during natural self-motion: implications for neural processing. J Neurosci. 2014;34(24):8347–57. Epub 2014/06/13. pmid:24920638;
- 41. Dong DW, Atick JJ. Statistics of Natural Time-Varying Images. Network-Computation in Neural Systems. 1995;6(3):345–58. WOS:A1995RR35700003.
- 42. Voss RF, Clarke J. '1/f noise' in music: music from 1/f noise. Journal of the Accoustical Society of America. 1978;63:258–63.
- 43. Voss RF, Clarke J. 1/F noise in music and speech. Nature. 1975;258:317–8.
- 44. Savard M, Krahe R, Chacron MJ. Neural heterogeneities influence envelope and temporal coding at the sensory periphery. Neuroscience. 2011;172:270–84. pmid:21035523
- 45. McGillivray P, Vonderschen K, Fortune ES, Chacron MJ. Parallel coding of first- and second-order stimulus attributes by midbrain electrosensory neurons. J Neurosci. 2012;32(16):5510–24. Epub 2012/04/20. pmid:22514313;
- 46. Sadeghi SG, Minor LB, Cullen KE. Response of vestibular-nerve afferents to active and passive rotations under normal conditions and after unilateral labyrinthectomy. J Neurophysiol. 2007;97(2):1503–14. pmid:17122313.
- 47. Martinez D, Metzen MG, Chacron MJ. Electrosensory processing in Apteronotus albifrons: implications for general and specific neural coding strategies across wave-type weakly electric fish species. J Neurophysiol. 2016;116(6):2909–21. pmid:27683890.
- 48. Eggermont JJ. Temporal modulation transfer functions for single neurons in the auditory midbrain of the leopard frog. Intensity and carrier-frequency dependence. Hear Res. 1990;43(2–3):181–98. pmid:2312413.
- 49. Edelman G, Gall W, Cowan W, Schreiner CE, Langner G. Coding of temporal patterns in the central auditory nervous system. Auditory function Neurobiological bases of hearing. New York: Wiley; 1988. p. 337–61.
- 50. Rosenberg A, Issa NP. Visual Demodulation by the Y Cell Pathway. Neuron. 2011;71:348–61.
- 51. Fotowat H, Harrison RR, Krahe R. Statistics of the Electrosensory Input in the Freely Swimming Weakly Electric Fish Apteronotus leptorhynchus. J Neurosci. 2013;33:13758–72. pmid:23966697
- 52. Kuang X, Poletti M, Victor JD, Rucci M. Temporal encoding of spatial information during active visual fixation. Current biology: CB. 2012;22(6):510–4. pmid:22342751;
- 53. Randall JM, Matthews RT, Stiles MA. Resonant frequencies of standing humans. Ergonomics. 1997;40(9):879–86. pmid:9306739.
- 54. Fairley TE, Griffin MJ. The apparent mass of the seated human body: vertical vibration. J Biomech. 1989;22(2):81–94. pmid:2708398.
- 55. Sadeghi SG, Chacron MJ, Taylor MC, Cullen KE. Neural Variability, Detection Thresholds, and Information Transmission in the Vestibular System. J Neurosci. 2007;27:771–81. pmid:17251416
- 56. Baker CL Jr. Central neural mechanisms for detecting second-order motion. Curr Opin Neurobiol. 1999;9(4):461–6. Epub 1999/08/17. pmid:10448168.
- 57. Baker CL Jr., Mareschal I. Processing of second-order stimuli in the visual cortex. Prog Brain Res. 2001;134:171–91. Epub 2001/11/13. pmid:11702543.
- 58. Lawther A, Griffin MJ. Prediction of the incidence of motion sickness from the magnitude, frequency, and duration of vertical oscillation. J Acoust Soc Am. 1987;82(3):957–66. pmid:3655126.
- 59. Cramer DB, Graybiel A, Oosterveld WJ. Successful transfer of adaptation acquired in a slow rotation room to motion environments in Navy flight training. Acta Otolaryngol. 1978;85(1–2):74–84. pmid:305183.
- 60. Graybiel A, Knepton J. Prevention of motion sickness in flight maneuvers, aided by transfer of adaptation effects acquired in the laboratory: ten consecutive referrals. Aviat Space Environ Med. 1978;49(7):914–9. pmid:666686.
- 61. Graybiel A, Knepton J. Bidirectional overadaptation achieved by executing leftward or rightward head movements during unidirectional rotation. Aviat Space Environ Med. 1978;49(1 Pt 1):1–4. pmid:623559.
- 62. Reason JT, Graybiel A. Changes in subjective estimates of well-being during the onset and remission of motion sickness symptomatology in the slow rotation room. Aerosp Med. 1970;41(2):166–71. pmid:5418845.
- 63. Reason JT, Graybiel A. Progressive adaptation to Coriolis accelerations associated with 1-rpm increments in the velocity of the slow rotation room. Aerosp Med. 1970;41(1):73–9. pmid:5309794.
- 64. Graybiel A, Deane FR, Colehour JK. Prevention of overt motion sickness by incremental exposure to otherwise highly stressful Coriolis accelerations. NAMI-1044. NASA Contract Rep NASA CR. 1968:1–13. pmid:5303710.
- 65. Graybiel A, Thompson AB, Deane FR, Fregly AR, Colehour JK, Ricks EL Jr. Transfer of habituation of motion sickness on change in body position between vertical and horizontal in a rotating environment. Aerosp Med. 1968;39(9):950–62. pmid:5672463.
- 66. Cohen B, Dai M, Yakushin SB, Raphan T. Baclofen, motion sickness susceptibility and the neural basis for velocity storage. Prog Brain Res. 2008;171:543–53. pmid:18718351.
- 67. Lackner JR, Graybiel A. Use of promethazine to hasten adaptation to provocative motion. J Clin Pharmacol. 1994;34(6):644–8. pmid:8083396.
- 68. Levine ME, Chillas JC, Stern RM, Knox GW. The effects of serotonin (5-HT3) receptor antagonists on gastric tachyarrhythmia and the symptoms of motion sickness. Aviat Space Environ Med. 2000;71(11):1111–4. pmid:11086664.
- 69. Lewis RF, Priesol AJ, Nicoucar K, Lim K, Merfeld DM. Dynamic tilt thresholds are reduced in vestibular migraine. J Vestib Res. 2011;21(6):323–30. pmid:22348937;
- 70. Kim CH, Shin JE, Song CI, Yoo MH, Park HJ. Vertical components of head-shaking nystagmus in vestibular neuritis, Meniere's disease and migrainous vertigo. Clin Otolaryngol. 2014;39(5):261–5. pmid:25042770.
- 71. Cullen KE, Roy JE. Signal processing in the vestibular system during active versus passive head movements. J Neurophysiol. 2004;91(5):1919–33. Epub 2004/04/08. 91/5/1919 [pii]. pmid:15069088.
- 72. Lundstrom BN, Higgs MH, Spain WJ, Fairhall AL. Fractional differentiation by neocortical pyramidal neurons. Nature Neuroscience. 2008;11(11):1335–42. Epub 2008/10/22. nn.2212 [pii] pmid:18931665;
- 73. Dong DW, Atick JJ. Statistics of natural time-varying images. Network. 1995;6:345–58.
- 74. Zhou YX, Baker CL Jr. A processing stream in mammalian visual cortex neurons for non-Fourier responses. Science. 1993;261(5117):98–101. Epub 1993/07/02. pmid:8316862.