Overview of state feedback control

The computational model used in this investigation implements a state feedback control (SFC) architecture to simulate a participant’s response to an unexpected pitch perturbation during a sustained vowel production. The theory of state feedback control has been well established as a plausible neural mechanism for non-speech motor tasks [23, 24] and has more recently been explored in the field of speech motor control [14, 25, 26].

This SFC model of voice control we consider is greatly simplified from the full task of voice control. Our model simulates only the control of f_o in ongoing voice output, which we idealize as controlling the rest-length of a spring in a single damped spring-mass system that is somewhat analogous to the cricothyroid muscle’s control of pitch [25, 27]. The controls generated by laryngeal motor cortex are modeled as desired changes in the rest-length of a single muscle determining f_o in our simplified larynx.

The SFC architecture for motor control of this simplified larynx is shown in Fig 2. Motor controls are calculated by comparing the desired laryngeal state to an internal estimate of the current state of the larynx, which is maintained by a process of state prediction and subsequent correction based on auditory and somatosensory feedback signals. The controls are scaled by a tunable controller gain g_c. An efference copy of these commands is used to predict the state of the larynx for the following time step, and consequently the expected auditory and somatosensory feedback. Meanwhile, the actual sensory feedback is generated by the larynx model. This simulated feedback can be altered to simulate the effects of a pitch perturbation experiment. Each sensory feedback signal contains Gaussian noise with zero mean and tunable variance σ and has some tunable delay Δ. The delayed and noisy feedback is compared with the predicted feedback to generate an error signal. The error signals from each feedback modality are used to correct the internal estimate of laryngeal state. The error signals are weighted by a Kalman gain matrix, which is calculated based on the noise in each signal, to determine the appropriate correction to the internal state estimate. Across multiple frames, the auditory output of the larynx model can be compared to a behavioral pitch perturbation response.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overview of state feedback control.

A state feedback model of vocal f_o control where Δ_a is auditory feedback delay, Δ_s is somatosensory feedback delay, σ is overall feedback noise variance, r is feedback noise ratio, and g_c is controller gain. Parameters tuned in this investigation are highlighted in red.

https://doi.org/10.1371/journal.pcbi.1011986.g002

State space representation

The plant of the control system, the larynx, is modeled using a state space representation of a dynamical system (Eqs 1 and 2; [28]). Specifically, the larynx is modeled as a simple spring-mass system in which the length of the spring represents muscle tension on the vocal folds, and is therefore linearly related to vocal pitch. There exist many more complicated and accurate models of the generation of voice by the complex system of muscles controlling the larynx (see [29–31]); however, for the current study, a very simple model was used to represent broadly the dynamics of the plant that must be controlled. (1) (2)

In these equations, x is the state (position and velocity of the mass on the spring) of the larynx dynamical system (elsewhere referred to as “larynx”), t is the current time step, u is the control input, and w is Gaussian process noise. The variable y is a vector containing auditory and somatosensory feedback and v is Gaussian measurement noise. A and B are matrices encoding the spring constant k, damping constant b, and mass of the system m (, ), which would be determined by the anatomy of the individual (stiffness of laryngeal muscles, muscle damping, and mass of vocal folds, respectively), and C encodes the transformation from state to sensory feedback. These three matrices are constants of the larynx plant (Eq 3) and remain fixed throughout the study at the values used by Houde et al. [27]. Including these plant constants k, b, and m as tunable parameters across a wide range of values did not substantially change the results of the investigation, and the values used here are within the 95% Bayesian credible interval of those inferred to optimally fit the Houde et al. [9] data set. (3)

Process noise w is defined such that (4) where σ_q is the variance of noise applied to each element of laryngeal state. σ_q was held constant at 1e-8 throughout the simulations, a value that was found to produce stable simulator output.

Measurement noise v is defined such that (5) where σ_a is the variance of Gaussian noise on the auditory feedback signal and σ_s is the variance of Gaussian noise on the somatosensory signal.

Another state space system defined by A = 0, B = 1, C = 1 is connected in series preceding this main system, serving to integrate over the main system and thus produce a laryngeal position from the motor commands output by the main system. Additionally, the full continuous system is discretized using zero-order hold methods with a 0.004 s sampling time.

Pitch perturbation

The variable y is a two-element vector containing auditory and somatosensory feedback from the larynx. The pitch perturbation is implemented as an addend to the auditory component of the vector as follows: (6)

Controller

The controller command to change the state of the larynx is defined by (7) where g_c is the controller gain, x_target is the desired laryngeal state and x_estimate is the internal estimate of the laryngeal state.

Observer

The observer provides the internal estimate of the state of the larynx through iterative prediction of state and subsequent update of the prediction using sensory feedback. The state is predicted by (8) where A and B matrices are identical to those used to define the larynx (Eq 3) and u[t − 1] is the efference copy of the controller commands from the previous timestep (Eq 7). Predicted state is used to predict the sensory feedback by (9) where C is identical to the state-to-feedback transformation matrix of the laryngeal system (Eq 3). The error between the predicted sensory feedback and the actual sensory feedback is then determined by (10) where Δ_a is auditory feedback delay and Δ_s is somatosensory feedback delay. Sensory feedback error is used to estimate the error in laryngeal state by (11) where K is the steady-state Kalman gain. Kalman gain is the optimal gain matrix calculated by first solving the discrete-time algebraic Ricatti equation (Eq 12) using the Python Control Systems Library (python-control; [32]) for where the equation is satisfied when . (12)

In the above equation, A and B are defined by the larynx dynamical system (Eq 3) and Q and R are defined by noise variances (Eqs 4 and 5). Optimal Kalman gain is then calculated as follows where C is defined by the larynx dynamical system (Eq 3). (13)

Finally, the predicted laryngeal state is updated by the error to find the estimated state using (14)

Tunable parameters

The values of a set of tunable parameters in the SFC model affect the shape of the model output, a simulated time course of vocal f_o in response to a mid-trial pitch perturbation. These parameters include the sensory delay parameters Δ_a and Δ_s, controller gain g_c, and sensory feedback noise variance parameters. In order to separate the effects of absolute feedback noise level from the relative amount of noise between the two sensory feedback modalities, the noise variance parameters σ_a and σ_s were parameterized to σ and r such that σ_a = σ and σ_s = σ/r. Thus σ represents the variance of sensory feedback noise overall in both auditory and somatosensory feedback modalities and r represents the ratio of auditory feedback noise variance to somatosensory feedback noise variance. Overall feedback noise variance σ was explored on a log₁₀ scale in order to more effectively search many orders of magnitude of noise variance. Thus the parameter set tuned in this investigation was θ = {Δ_a, Δ_s, σ, r, g_c}.

Expression of study hypotheses in the context of the SFC model

Houde et al. [9] proposed two possible explanations for the behavioral pitch perturbation response differences between CA and control groups: 1) an increase in the CA group’s reliance on auditory feedback, or 2) an increase in the CA group’s reliance on all types of sensory feedback collectively compared to that of the control group. These hypotheses can be examined using the feedback noise parameters, which are inversely related to reliance on sensory feedback through the calculation of Kalman gain K. K can be broken down into (15) where K_a scales auditory feedback error and K_s scales somatosensory feedback error. To be precise, K_a and K_s are each vectors containing Kalman gains on both position and velocity error, but since we are simply conceptually illustrating the effect of feedback noise on error scaling, we will imagine these as single values. An increase in the variance of the noise distribution of a particular feedback modality (e.g., σ_a) results in a lower Kalman gain on that feedback signal (e.g., K_a). Thus the hypotheses can be conceptualized in terms of the SFC model as follows:

Hypothesis 1: Compared to the control group, the CA group exhibits increased reliance on auditory feedback over somatosensory feedback. Mathematically, this can be expressed as: (16)

Since Kalman gain is inversely related to noise variance, this is equivalent to: (17) or (18)

Hypothesis 2: Compared to the control group, the CA group displays an increased reliance on all types of sensory feedback collectively due to impairment of the feedforward system. Mathematically, this can be expressed as: (19) which is equivalent to: (20) or simply: (21)

Thus Hypothesis 1 can be examined by comparing the feedback noise ratio parameter r between the two groups, while Hypothesis 2 can be examined by comparing the overall feedback noise variance parameter σ between groups. The finding of a smaller value for r in the CA group compared with the control group would support Hypothesis 1, while a smaller value for σ in the CA group would support Hypothesis 2.

Parameter effect size

Bayesian inference was used to fit a five-parameter SFC model of voice f_o to the empirical pitch perturbation responses of CA and control groups as observed in Houde et al. [9]. For each empirical data set, simulation-based inference (SBI; https://github.com/sbi-dev/sbi/; [21, 22] was used to generate a posterior likelihood distribution across values for each parameter. To improve robustness, the posterior distributions from 10 repetitions of the inference procedure were combined [13, 33]. Fig 3 shows the combined posterior distributions generated for each parameter, with the median and 95% Bayesian credible interval marked in each distribution. Since tests of significance lose meaning under the high statistical power of simulated data [34], effect size was used to quantify which parameters were most different between control and CA groups. Glass’s delta effect size was used because it is designed to compare populations with unequal variances. The mean and standard error of effect size calculated from 100 bootstrap samples of size 1000 are annotated on each subplot. Controller gain and feedback noise ratio had the largest effect size, while somatosensory feedback delay, auditory feedback delay and overall feedback noise variance had relatively small effect size. Notably, feedback noise ratio had a much larger effect size than the overall variance of sensory feedback noise, suggesting that the relative amount of noise between feedback modalities contributed much more to group differences than the overall level of feedback noise. All of the distributions are unimodal, which suggests there was a single optimal parameter set rather than several local optima.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Parameter likelihood distributions.

Posterior likelihood distributions (pooled from 10 repetitions of inference) for each parameter, with control group distributions on the left (red) and CA group distributions on the right (blue). The 95% Bayesian credible interval is indicated by horizontal bars and the median value of each distribution is indicated by a black dot. Glass’s delta effect size (mean and standard error from bootstrap samples) is printed at the top of each subplot.

https://doi.org/10.1371/journal.pcbi.1011986.g003

Model fit

The median of each marginal likelihood distribution was chosen as the inferred parameter set for each participant group (see Table 1). The inferred parameter sets were validated by using them in the SFC simulator to check that the results were broadly similar to the empirical data from each group. The close alignment of the tuned model with the empirical data can be seen in Fig 4. The mean of 100 simulator outputs using each group’s inferred parameter set were plotted to account for stochasticity within the SFC simulator. The standard error of these simulator outputs was also plotted, but was too small to be distinguished from the mean. Root mean square error (RMSE) between the simulator output and the behavioral data was used to quantify the quality of model fit for each group with a statistic not explicitly optimized during training. RMSE (mean ± standard error across 100 simulations) was 0.8613 ± 0.0002 cents for the control group (4.06% of the range of the data) and 1.5544 ± 0.0003 cents for the CA group (3.84% of the range of the data).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Model fit.

The simulator output (solid lines) using the inferred parameter set for each group closely aligned with the behavioral data (dotted lines) previously seen in Fig 1. For simulator output and behavioral data, the mean is plotted with a thick line and standard error with a thin line, although standard error on simulator output was too small to visibly distinguish from the mean. Blue lines indicate data associated with the CA group and the corresponding simulator output, while red lines indicate those associated with the control group.

https://doi.org/10.1371/journal.pcbi.1011986.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Inferred values for control and CA groups.

https://doi.org/10.1371/journal.pcbi.1011986.t001

Ablation study

Above, we have shown a successful model fit of the pitch perturbation responses for CA and control groups and quantified the parameter distribution differences between groups; however, the impact of these parameter differences on the simulator output are not immediately clear from previous results. Thus, ablation techniques [35] were used to understand the extent to which differences in inferred parameter values may translate to meaningful changes in model output. In this ablation study, the impact of each parameter on group differences was assessed by fitting the behavioral data set from the CA group with a series of reduced models. For each reduced model, one of the five original parameters was fixed to the control group’s inferred value for that parameter (as listed in Table 1) and a model composed of the four remaining parameters was used to fit the behavioral data. This allowed us to quantify the impact of each parameter on group differences and simulate the pitch perturbation response we might expect to see if speakers with CA did not differ from controls in terms of each parameter.

As seen in Fig 5, the quality of the model fit for the CA group was most greatly impacted by fixing the feedback noise ratio parameter, followed by the controller gain parameter. This provided additional evidence that these parameters were the most impactful in explaining the differences between the pitch perturbation responses of the two groups. Fixing either the somatosensory feedback delay or feedback noise variance parameter did not seem to change the quality of model fit from that of the full five-parameter model, supporting the idea that these parameters have low impact in explaining the differences between CA and control groups. Once again, the impact of the feedback noise ratio parameter was much greater than that of the feedback noise variance parameter, showing that the absolute amount of noise in each feedback signal is less impactful than the relative amount of noise between signals. To validate the method, the reduced models were also used to fit the control group data. As expected, RMSE between the simulated and behavioral control group responses remained similar to that of the full model, showing that changes in the RMSE of the CA group in the reduced model fits are attributable to group differences in parameter values rather than an artifact of reducing the number of parameters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Quality of fit for reduced models.

Quality of fit for each 4-parameter model with one parameter fixed to the inferred value for the control group. The quality of each model’s fit is quantified by RMSE between empirical data and optimized model output (mean RMSE across 100 simulator runs).

https://doi.org/10.1371/journal.pcbi.1011986.g005

For each four-parameter model, the optimized simulator output of control and CA group data are shown in Fig 6, along with the posterior distributions of the remaining parameters. The optimized simulator output for each four-parameter model provides insight regarding the impact of each parameter on the shape of the pitch perturbation response, while the distributions of the remaining parameters indicate which parameters may interact with the ablated parameter.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Reduced models: Simulator output and posterior distributions of remaining parameters.

Inference results for each of five reduced models in which a single parameter is fixed to the inferred value of the control group from the full model fit and the remaining four parameters are used to fit the behavioral pitch perturbation data. Shown for each reduced model are the posterior distributions of the four remaining parameters and the simulator output (mean of 100 simulations) using the inferred reduced parameter set.

https://doi.org/10.1371/journal.pcbi.1011986.g006

The results of the ablation of the feedback noise ratio parameter are shown in Fig 6A. We can see from the posterior distributions that while the somatosensory feedback delay, feedback noise variance, and auditory feedback delay parameter distributions remained mostly similar to those of the full model, the controller gain parameter distribution for the CA group shifted up to a much larger median value. This allowed the simulated response to nearly reach the peak response of the behavioral CA data, but it dipped far below the behavioral CA data during the period from 0.7 to 1.0 s after perturbation onset, resulting in a large RMSE.

Alternately, when the controller gain parameter was ablated, the feedback noise ratio greatly increased in effect size as the median value of the CA group distribution shifted down from 1.04 to 0.62. The median value of the somatosensory delay parameter also increased but the effect size of this parameter increased only slightly due to the large variance of the distribution. These changes allowed the simulator output to nearly reach the peak magnitude of the behavioral data of the CA group, but with the controller gain fixed at the value of the control group, the optimized simulator output was higher and more oscillatory than the behavioral data during the period from 0.7 to 1.0 s after perturbation onset. Thus contributions from both the feedback noise ratio and controller gain parameters are needed to replicate both the large magnitude of the CA group response and the flat, moderate time course of f_o after the end of the perturbation.

When auditory feedback delay was fixed to the inferred value for the control group (Fig 6C), the CA group response became more oscillatory, contributing to a small increase in error from the behavioral data. Fixing either somatosensory feedback delay or overall feedback noise variance, meanwhile, did not appear to change the model fit. Each of these four-parameter models achieved similar RMSE to the full model, and Fig 6D and 6E show similar fit and parameter distributions to the full model (Figs 3 and 4). Thus small changes in the remaining four parameters can account for the group differences in these parameters.

Marginal sensitivity analysis

To ensure that the ablation study was not confounded by the amount of change each parameter effects in the model output, a marginal sensitivity analysis was conducted. Specifically, each parameter was varied one and two standard deviations of the posterior distribution above and below the inferred value while keeping all other parameters fixed at their inferred values for each group. Fig 7 shows that the parameters that were least important in the ablation study, somatosensory feedback delay and feedback noise variance, were actually the most sensitive to standard deviation-sized changes, while the parameters that were most important in the ablation study, feedback noise ratio and controller gain, were the least sensitive. The model parameters were fairly robust within two standard deviations of their inferred values, with differences from baseline <4 cents, except for somatosensory feedback delay. When other parameters were set to the inferred values of the control group, somatosensory delay was sensitive to high values due to instability. Additionally, no RMSE value could be obtained for somatosensory delay two standard deviations less than the inferred value of the CA group, because this was an invalid parameter value less than the frame rate of the model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Marginal sensitivity analysis.

The top row of plots shows the change in model output when the parameter is varied one and two standard deviations of the posterior distribution above and below the inferred value while all other parameters are fixed at their inferred values. The change in model output is quantified as RMSE from the baseline model output where all parameters are set to their inferred values. Also shown are the model outputs that result from varying each parameter around the inferred values of the control group (second row) and the CA group (third row).

https://doi.org/10.1371/journal.pcbi.1011986.g007

The model outputs resulting from varying individual parameters are also shown in Fig 7. Somatosensory feedback delay and feedback noise variance seem to affect the oscillations in the model output, although somatosensory delay seems to have greater effect on the magnitude of the oscillations while feedback noise variance has greater effect on the frequency of the oscillations. Auditory feedback delay translates the curve on the time axis relative to perturbation onset. Controller gain and feedback noise ratio both adjust the magnitude of the response, but with distinct effects on the persistence of the response after 0.7 s. As can be seen from the differences between the second and third rows of Fig 7, the effects of each parameter on model output are dependent on the context of the other parameter values. This highlights the importance of performing parameter fitting using techniques like simulation-based inference which account for changes in multiple parameters simultaneously.

Evidence for overreliance on auditory feedback

The high effect size of the feedback noise ratio suggests that the CA group may show increased reliance on auditory feedback relative to somatosensory feedback. This may appear to conflict with the results of Li et al. [10], which showed in individuals with spinocerebellar ataxia a decreased cortical P2 response in the right superior temporal gyrus (STG), primary auditory cortex (A1), and supramarginal gyrus (SMG) during a pitch perturbation task. However, while this finding may provide evidence against the idea that the absolute value of auditory feedback error gain is larger in the CA group than the control group ((K_a)_CA > (K_a)_control), we argue that it does not rule out the idea of increased sensitivity to auditory feedback as we have defined it in this study, that is, relative to somatosensory feedback (; Hypothesis 1). If both auditory and somatosensory gains are smaller in the CA group compared to controls, it is possible that the resulting ratio between auditory gain and somatosensory gain may still be larger in the CA group compared to the control group (i.e., if the somatosensory gain in the CA group were much smaller compared that of the control group). Defining feedback noise in terms of a ratio between sensory modalities has been previously used by Crevecoeur et al. [23] to model relative sensitivity to visual and proprioceptive feedback in arm reaching. Furthermore, the P2 response may not be a direct measure of gain on auditory feedback error; the functional relevance of P2 response remains controversial and has been suggested to have contributions from multiple sensory modalities including auditory and somatosensory [36]. Although this view is highly speculative, we argue that the measurement of decreased P2 response in the CA group [10] does not completely rule out Hypothesis 1 and that our findings warrant further investigations.

Lack of evidence for overall overreliance on feedback

Given the ties between internal models and the cerebellum [19, 20], it is indeed somewhat surprising that we did not find evidence of overreliance on sensory feedback overall. It would be reasonable to expect that if the internal models, and thus the state prediction process, are disrupted in CA, sensory feedback would be more heavily weighted to compensate for impairment in the state prediction [9]. However, our results indicated similar weighting of sensory feedback overall between the two groups, with perhaps even slightly lower weighting (increased noise) of sensory feedback in the CA group. If future studies find similar results across speech tasks, we speculate that perhaps the nature of the disruption of internal models impairs the integration of sensory feedback, which may have a similar effect to a slightly decreased weighting. However, it is possible that this is an artifact of the nature of the task; for example, perhaps studying an auditory rather than a somatosensory perturbation created a result in which auditory feedback appears to be emphasized rather than overall feedback, or perhaps an unexpected rather than a consistent perturbation created a result in which differences in feedback systems were emphasized. Future work should examine somatosensory perturbation studies and adaptation studies involving speakers with CA to better understand this result.

Novel hypothesis generated by the model: Controller gain parameter

In addition to support for the previously stated hypothesis that individuals with CA show increased reliance on auditory feedback relative to somatosensory feedback, our results lead us to propose an additional hypothesis to be tested in future investigations. The high impact of the controller gain parameter suggests a difference between speakers with CA and typical speakers in the scaling of the motor command to the larynx, which has likely neural substrate in the motor cortex [25]. While Kalman gain can be interpreted as sensitivity to prediction error, or the difference between the actual sensory consequences of an action and those predicted by the internal model, the controller gain parameter can be interpreted as sensitivity to target error, or the difference between the actual production and the goal, or desired production. Although any direct ties between this parameter and the cerebellum are unknown, we speculate that perhaps the increased value of controller gain in the CA group may indicate a learned mechanism in the motor cortex to compensate for deficits related to the disorder. Another possibility is the weakening of an inhibitory effect on controller gain, similar to the mechanism suggested by Behroozmand et al. [37] in their study of pitch perturbation response in individuals with Parkinson’s disease. Further study is needed to investigate this hypothesis in the context of CA.

Efficacy of simulation-based inference for model parameter estimation

Simulation-based inference allowed us to obtain posterior likelihood distributions across each parameter rather than a single set of optimal values. This made it possible to take into account the spread of each distribution in addition to the inferred value and thus determine the effect size of the difference between groups for each parameter. Additionally, in a complex system it is possible to have many locally optimal parameter sets that can achieve high quality fit to the empirical data. Obtaining a unimodal posterior likelihood distribution for each parameter showed that the parameter set selected could approximate a globally optimal solution within the bounds of the prior. Simulation-based inference also offers advantages over other Bayesian methods since it does not require an analytical form of the model (allowing us to analyze a complex model like state feedback control) and is more computationally efficient than other numerical techniques such as Markov Chain Monte Carlo methods [38]. Since an analytical form of the model is not required, simulation-based inference can be performed using any computational model of speech motor control, and future work comparing parameter distributions generated using other models could be valuable.

Limitations

It is important to note that the current work is beneficial for generating hypotheses, rather than drawing conclusions. The larynx was modeled here as a simple spring-mass system. While this implementation may approximate many of the dynamics of laryngeal movement, future studies should investigate how results may vary when a more detailed model of laryngeal muscles (e.g., [29–31]) is implemented as the plant. Future, more detailed models may also explore integrating sensory feedback delays into the state space equations to maintain the optimal properties of Kalman gain.

Additionally, real speakers vary in their vocal tract dimensions, damping, and other properties that are not possible to measure. The model cannot quantitatively reflect all of these properties to represent the vocal tract anatomy of an actual speaker. However, in a follow-up analysis that included the anatomical parameters spring constant k, damping constant b, and mass m as tunable parameters, the effect size between groups was very small for each of these parameters (< 0.1) and the 95% Bayesian credible interval for each parameter contained the value that was used in the original study. Thus the anatomical parameter values used were appropriate to the data set and no differences were indicated between the CA group and the control group in terms of these parameters.

Although the anatomical parameter values used for this investigation were determined to be appropriate for fitting the group-average data sets, leaving these properties unchanged throughout the present set of simulations may not represent the physiological diversity of the individuals included in the behavioral data set. Each model output plotted in this investigation was the mean output of 100 simulator runs, which helped account for stochasticity in the model. However, standard error on model output was so small (< 0.1 cent) that it could not be differentiated from the mean production on the plot, while standard error on the empirical mean was larger, in the range of 0–3 cents. This demonstrates that the model does not account for the variation that is present among individual speakers, and future work may investigate variation among individuals and subgroups.

The present study was also limited to the model changes that could be captured by five model parameters. The same model architecture was used to model both control and CA groups, with all differences captured in the tuning of model parameters. Thus possible differences in the structure of neural systems between the groups were not tested in the current study. Furthermore, many of the model parameters tested here are abstract concepts whose precise neural implementation is not yet fully understood. Each parameter that is represented in the model as a single value may in fact be the result of multiple complex neural processes. Furthermore, only five parameters were considered in the present study. Since training data was generated by testing values for each parameter in combination with all other parameters, the number of parameters tested was in exponential tradeoff with the resolution of values tested and the range of the prior for each one. While the irrelevance of other untested model parameters cannot be proven, the tested parameters can be argued to be sufficient since the model output for each set of inferred values closely matched the behavioral data.

Finally, this investigation focused on fitting behavioral responses to an unpredictable pitch perturbation, which provided insight regarding within-utterance control of the larynx. Future work is needed to fit the model to data from a pitch adaptation paradigm, in which can provide insight regarding control of the larynx before sensory feedback impacts behavior. To our knowledge, this behavior has not yet been studied in individuals with CA, so additional data collection will be needed. Future work is also needed to fit the model to behavioral responses to shifts in formant frequencies. While here control of the larynx was simplified to a one-dimensional model, the control of multiple articulators needed to adjust formant frequencies will require a multi-dimensional model.

Conclusion

This work has shown that the controller gain and feedback noise ratio parameters have high effect size and large contributions to the group differences between the pitch perturbation responses of speakers with CA and typical speakers. These results (a) provide support for the previous hypothesis that individuals with CA show increased sensitivity to auditory feedback prediction error and (b) generate a new hypothesis of increased sensitivity to target error in the CA group. Furthermore, this work demonstrates the value of simulation-based inference methods in analyzing behavioral speech data using tunable parameters of a state feedback control model.

Simulation-based inference overview

The SBI package in Python (https://github.com/sbi-dev/sbi/; [21, 22]) was used to obtain posterior likelihood distributions of five tunable parameters in the SFC model (see Model section for an overview of SFC) for the CA and control group average pitch perturbation responses observed in Houde et al. [9]. As detailed in Fig 8, SBI takes as input a computational model with a finite set of input parameters (“simulator”), a prior distribution for each parameter, and an empirical observation analogous to the output of the simulator. It generates a training set by running the simulator with inputs from the prior distributions of the parameters, and then, using the Sequential Neural Posterior Estimation (SNPE) option for inference, trains a deep neural density estimator to predict the posterior distributions of parameters given the empirical observation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. SBI overview.

A pipeline for performing simulation-based inference [21, 22]. Parameter values are inferred for a particular empirical observation given a mechanistic model (“simulator”) and a prior distribution for each tunable parameter.

https://doi.org/10.1371/journal.pcbi.1011986.g008

Simulator

A Python implementation (https://github.com/jessicagaines/1d-sfc) of a state feedback control model of f_o [25, 27] was used as the simulator (see Model section for more details). The input parameter set included the following parameters: auditory feedback delay Δ_a, somatosensory feedback delay Δ_s, controller gain g_c, feedback noise variance σ, and feedback noise ratio r. The parameters σ and r are parameterizations of auditory feedback noise variance σ_a and somatosensory feedback noise variance σ_s such that σ_a = σ and σ_s = σ/r. This idea of exploring relative levels of feedback noise between different sensory modalities was also used by Crevecoeur et al. [23] in their state feedback control model of arm reaching. The output simulated a time course of voice f_o in response to a -100 cent, 400 ms, mid-utterance perturbation of f_o auditory feedback. Random noise was added to model output during training as Jin et al. [33] found that this increased reconstruction accuracy. Uniform noise distributions of increasing width were tested and since the quality of model fit stopped improving for training noise distribution wider than 7 cents, noise with distribution ∼U(−3.5, 3.5) was added to the simulator output during training.

Prior distribution of parameters

The SBI inference procedure requires the input of a prior distribution for each parameter, which defines the search space of the training data. A uniform prior was used for each parameter. For some parameters, the bounds of the prior could be estimated from measurable quantities. For example, Abbs and Gracco [39] indicate that latencies in response to a somatosensory perturbation are on the order of tens of milliseconds, so a prior of 3 to 80 ms was used for somatosensory delay. Latencies to auditory perturbations, meanwhile, have been reported in the range of 100–200 ms [40], so a prior of 50 to 200 ms was used. Measurable latencies may be greater than delays since they include motor response time, so the lower bounds were set lower than the measured response latencies. The lower bound for somatosensory delay is quite low, in the range of what is typically associated with non-cortical reflex [41], but since no minimum delay value can be definitively measured, we opted not to restrict the prior based on this information. The remaining parameters could not be estimated from measurable quantities, so wide initial priors were selected to fully explore the space. Initial bounds of 0.1 to 10 were selected for the feedback noise ratio and controller gain parameters to include two orders of magnitude, and the range of 1e-10 to 1e-1 was selected for feedback noise variance. The auditory feedback noise variance parameter was converted to a base-10 logarithmic scale to search many orders of magnitude more effectively. For likely parameter sets in this regime, the simulator was found to be unstable for feedback noise variance less than 1e-6.5, and so the lower bound for this parameter was increased to 1e-6.5 to train the likelihood estimator on stable simulator outputs. Finally, the results of the wide prior showed that the tails of the posterior likelihood distributions of feedback noise variance, feedback noise ratio, and controller gain parameters were far from the upper bounds of each prior. The bounds were narrowed slightly to increase the search resolution for each parameter and decrease the computational resources needed. The final bounds selected are shown in Table 2. This choice of prior is validated by the result that the likelihood distributions for each parameter (see Fig 3) lie comfortably within these bounds, except for feedback noise variance, which was restricted for stability, and somatosensory delay, which by definition cannot be less than one frame of simulator operation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Uniform priors with the following bounds were selected for each parameter.

https://doi.org/10.1371/journal.pcbi.1011986.t002

Empirical observation

As described in the Introduction, behavioral group-average pitch perturbation responses from CA and control groups previously published by Houde et al. [9] were used as empirical observations to sample the posterior. Participants in the behavioral study included 16 individuals with cerebellar degeneration of various subtypes (10 male, 6 female, mean age 50 ± 12 years) and 11 age-matched controls (7 male, 4 female, mean age 51 ± 11 years). Many subtypes of CA were represented including spinocerebellar ataxia (SCA) type 2, SCA type 3, SCA type 5, SCA type 6, SCA type 7, SCA type 8. There were also six participants with unknown/idiopathic cerebellar atrophy [9]. Each behavioral data set was downsampled from 413 to 300 frames per 1.2 s trial to match the output of the simulator.

Inference

10⁵ simulations were used to train the neural density estimator [33]. The posterior was then sampled 10⁴ times for each group. To improve robustness, this procedure was repeated 10 times [13, 33, 42] and the samples from each repetition were pooled to obtain the final parameter distributions. A 95% Bayesian credible interval was calculated for each distribution. Glass’s delta was used to calculate the effect size of each parameter between groups. To assess the quality of model fit, the median of each pooled distribution was considered the “inferred value” for each parameter and each inferred parameter set was supplied as input to the simulator. To account for stochasticity within the simulator, 100 simulations were run with each inferred parameter set and the mean of these was plotted. The quality of the fit was assessed quantitatively by calculating the point-wise root mean square error (RMSE) between the model output and the empirical data [13, 42]. This statistic was not used in training the neural network and therefore provided a separate method of quantifying the success of the model fit.

Ablation study

Next, an ablation study was used to further understand the impact of each parameter on model output [35]. One at a time, each parameter was ablated by fixing it to the inferred value (the median value of the pooled posterior distribution) of the control group and repeating the inference procedure to generate posterior distributions for the four remaining parameters. The medians of these distributions were once again used in the simulator to assess the quality of fit for each reduced model using RMSE and compare the result to that of the full model. A greater increase in error for a particular reduced model indicated that the parameter ablated in that model had greater impact on group differences.

Marginal sensitivity analysis

Finally, a marginal sensitivity analysis was used to determine the sensitivity of the model output to changes in each parameter. First, a baseline output was determined by averaging 100 simulations using the inferred parameter set for each group. Then each parameter was varied, one at a time, while all other parameters remained fixed at their inferred values. Using the standard deviation from each parameter’s posterior distribution, parameter values 0, 1, and 2 standard deviations above and below the inferred value were tested (and again averaged across 100 simulations). The resulting model output of each test was plotted and the difference from baseline model output was quantified using RMSE.