^{1}

^{1}

^{2}

The authors have declared that no competing interests exist.

Closed-loop neurotechnologies often need to adaptively learn an encoding model that relates the neural activity to the brain state, and is used for brain state decoding. The speed and accuracy of adaptive learning algorithms are critically affected by the learning rate, which dictates how fast model parameters are updated based on new observations. Despite the importance of the learning rate, currently an analytical approach for its selection is largely lacking and existing signal processing methods vastly tune it empirically or heuristically. Here, we develop a novel analytical calibration algorithm for optimal selection of the learning rate in adaptive Bayesian filters. We formulate the problem through a fundamental trade-off that learning rate introduces between the steady-state error and the convergence time of the estimated model parameters. We derive explicit functions that predict the effect of learning rate on error and convergence time. Using these functions, our calibration algorithm can keep the steady-state parameter error covariance smaller than a desired upper-bound while minimizing the convergence time, or keep the convergence time faster than a desired value while minimizing the error. We derive the algorithm both for discrete-valued spikes modeled as point processes nonlinearly dependent on the brain state, and for continuous-valued neural recordings modeled as Gaussian processes linearly dependent on the brain state. Using extensive closed-loop simulations, we show that the analytical solution of the calibration algorithm accurately predicts the effect of learning rate on parameter error and convergence time. Moreover, the calibration algorithm allows for fast and accurate learning of the encoding model and for fast convergence of decoding to accurate performance. Finally, larger learning rates result in inaccurate encoding models and decoders, and smaller learning rates delay their convergence. The calibration algorithm provides a novel analytical approach to predictably achieve a desired level of error and convergence time in adaptive learning, with application to closed-loop neurotechnologies and other signal processing domains.

Closed-loop neurotechnologies for treatment of neurological disorders often require adaptively learning an encoding model to relate the neural activity to the brain state and decode this state. Fast and accurate adaptive learning is critically affected by the learning rate, a key variable in any adaptive algorithm. However, existing signal processing algorithms select the learning rate empirically or heuristically due to the lack of a principled approach for learning rate calibration. Here, we develop a novel analytical calibration algorithm to optimally select the learning rate. The learning rate introduces a trade-off between the steady-state error and the convergence time of the estimated model parameters. Our calibration algorithm can keep the steady-state parameter error smaller than a desired value while minimizing the convergence time, or keep the convergence time faster than a desired value while minimizing the error. Using extensive closed-loop simulations, we show that the calibration algorithm allows for fast learning of accurate encoding models, and consequently for fast convergence of decoder performance to high values for both discrete-valued spike recordings and continuous-valued recordings such as local field potentials. The calibration algorithm can achieve a predictable level of speed and accuracy in adaptive learning, with significant implications for neurotechnologies.

Recent technological advances have enabled the real-time recording and processing of different invasive neural signal modalities, including the electrocorticogram (ECoG), local field potentials (LFP), and spiking activity [

Closed-loop neural systems need to learn an encoding model that relates the neural signal (e.g., spikes) to the underlying brain state (e.g., motor intent) for each subject. The encoding model is often taken as a parametric function and is used to derive mathematical algorithms, termed decoders, that estimate the subject’s brain state from their neural activity. These closed-loop neural systems run in real time and often require the encoding model parameters to be learned in closed loop, online and adaptively (

Closed-loop neural systems often need to learn an encoding model adaptively and in real time. The encoding model describes the relationship between neural recordings and the brain state. For example, the relevant brain state in motor BMIs is the intended velocity and in DBS systems is the disease state, e.g., in Parkinson’s disease. The neural system uses the learned encoding model to decode the brain state. This decoded brain state is then used, for example, to move a prosthetic in motor BMIs while providing visual feedback to the subject, or to control the stimulation pattern applied to the brain in DBS systems. A critical parameter for any adaptive learning algorithm is the learning rate, which dictates how fast the encoding model parameters are updated as new neural observations are received. An analytical calibration algorithm will enable achieving a predictable level of accuracy and speed in adaptive learning to improve the transient and steady-state operation of neural systems.

A critical design parameter in any adaptive algorithm is the learning rate, which dictates how fast model parameters are updated based on a new observation of neural activity (

To date, however, adaptive algorithms have chosen the learning rate empirically. For example, in batch-based methods, once a new batch estimate is obtained, the parameter estimates from previous batches are either replaced with these new estimates [

Here, we develop a mathematical framework to optimally calibrate the learning rate for Bayesian adaptive learning of neural encoding models. We derive the calibration algorithm both for learning a nonlinear point process model for discrete-valued spiking activity—which we term point process encoding model—, and for learning a linear model with Gaussian noise for continuous-valued neural activities (e.g., LFP or ECoG)—which we term Gaussian encoding model. Our framework derives an explicit analytical function for the effect of learning rate on parameter estimation error and/or convergence time. Minimizing the convergence time and the steady-state error covariance are competing requirements. We thus formulate the calibration problem through the fundamental trade-off that the learning rate introduces between the convergence time and the steady-state error, and derive the optimal calibration algorithm for two alternative objectives: satisfying a user-specified upper-bound on the steady-state parameter error covariance while minimizing the convergence time, and vice versa. For both objectives, we derive analytical solutions for the learning rate. The calibration algorithm can pre-compute the learning rate prior to start of real-time adaptation.

We show that the calibration algorithm can analytically solve for the optimal learning rate for both point process and Gaussian encoding models. We use extensive Monte-Carlo simulations of adaptive Bayesian filters operating on both discrete-valued spikes and continuous-valued neural observations to validate the analytical predictions of the calibration algorithm. With these simulations, we demonstrate that the learning rate selected analytically by the calibration algorithm minimizes the convergence time while satisfying an upper-bound on the steady-state error covariance or vice versa. Thus the algorithm results in fast and accurate learning of the encoding model. In addition to the encoding model, we also examine the influence of the calibration algorithm on decoding by taking a motor BMI system, which uses discrete-valued spikes or continuous-valued neural activity (e.g., ECoG or LFP), as an example. We perform extensive closed-loop BMI simulations [

We derive the calibration algorithm for adaptation of two widely-used neural encoding models—the linear model with Gaussian noise for continuous-valued signals such as LFP and ECoG, and the nonlinear point process model for the spiking activity. In the former case, the calibration algorithm adjusts the learning rate of an adaptive KF, and in the latter case it adjusts the learning rate of an adaptive PPF. We design the calibration algorithm for adaptive PPF and KF, as these filters have been validated in closed-loop non-human primate and human experiments both in our work and in other studies (e.g., [

In both the adaptive PPF and the adaptive KF, the learning rate is dictated by the noise covariance of the decoder’s prior model for the parameters. In what follows, we derive calibration algorithms for two possible objectives: to keep the steady-state parameter error covariance smaller than a user-specified upper-bound while minimizing the convergence time, or to keep the convergence time faster than a user-specified upper-bound while minimizing the steady-state error covariance. We first derive analytical expressions for both the steady-state error covariance and the convergence time as a function of the learning rate by writing the recursive error dynamics and the corresponding recursive error covariance equations for the adaptive PPF and adaptive KF. By taking the limit of these recursions as time goes to infinity, we find the analytical expressions for the steady-state error covariance and the convergence time as a function of the learning rate. We then find the inverse maps of these functions, which provide the optimal learning rate for a desired objective. We also introduce the numerical simulation setup used to evaluate the effect of the calibration algorithm on both encoding models and decoding. The flowchart of the calibration algorithm is in

In this section, we derive the calibration algorithm for continuous signals such as LFP and ECoG. We first present the observation model and the adaptive KF for these signals. We then find the steady-state error covariance and the convergence time as functions of the learning rate. Finally, we derive the inverse functions to select the optimal learning rate.

We denote the continuous observation signal, such as ECoG or LFP, from channel _{t} denoting the encoded brain state. Also, ^{c} = [^{c}, (^{c})′]′ is a column vector containing the encoding model parameters to be learned. In particular, ^{c} is the baseline log-power and ^{c} depends on the application. Finally, ^{c}. As an example, in motor BMIs, we take the brain state _{t} as the intended velocity command whether in moving one’s arm or in moving a BMI. We thus select ^{c}‖ the modulation depth and ^{c} the preferred direction of channel ^{c}. In some cases, it may also be desired to learn ^{c} adaptively. Here, we first focus on adaptive learning of the parameters ^{c} and the derivation of the calibration algorithm. We then present a method to learn ^{c} concurrently with the parameters.

We write a recursive Bayesian decoder to learn the parameters ^{c} recursively in real time. In neurotechnologies, such as BMIs, neural encoding model parameters are either time-invariant or change substantially slower compared with the time-scales of parameter learning (days compared with minutes, respectively; see e.g., [

Assuming that all channels are conditionally independent [_{t} is a white Gaussian noise with covariance matrix _{n}(_{n} is the identity matrix and _{t} is simply used to model our uncertainty at time

Combining _{t} from _{1}, ⋯, _{t}. KF finds the minimum mean-squared error (MMSE) estimate of the parameters, which is given by the mean of the posterior density. Denoting the posterior and prediction means by _{t|t} and _{t|t−1}, and their covariances by _{t|t} and _{t|t−1}, respectively, the KF recursions are given as

Note that _{t|t} specifies the relative weight of the neural observation _{t} compared with the previous parameter estimate in updating the current parameter estimate and thus determines how fast _{t|t} is learned in _{t|t} is a function of

We define _{t|t} in _{t → ∞} _{t|t}] = _{t|t}]) to converge to _{t|t}]‖, while keeping the convergence time below a desired upper-bound. We derive the calibration algorithm for each of these objectives and provide them in Theorems 1 and 2.

Regardless of the objective, to derive the calibration algorithm we first need to write the error dynamics in terms of the learning rate _{t} = _{t|t}. We denote the estimation error covariance at time _{t|t} is asymptotically unbiased by Appendix C in _{t}] as functions of the learning rate

To find the steady-state error covariance _{t}] as a function of _{t}] as a function of the learning rate _{t} during the training session (i.e., the experimental session in which parameters are being learned adaptively) is periodic with period

_{t} _{ave} = _{1}, …, _{n})_{i} ≤ _{i+ 1}).

_{t}]

_{t}]

Since _{ave}, which is not related to _{ave} does not require complete knowledge of

Now that we have an analytical expression for the steady-state error covariance and the convergence rate as functions of the learning rate

We now derive the inverse functions of Eqs _{bd} is the desired upper-bound on the steady-state error covariance. We want the largest learning rate that satisfies this relationship because the convergence time is a decreasing function of the learning rate and hence will benefit from larger rates. The key step in solving this inequality is observing that the 2-norm

For the learning rate optimization to satisfy a given convergence time upper-bound, the goal is to calculate the smallest learning rate _{bd}, where _{bd} is the upper-bound of the convergence time and _{rest} is the relative estimation error (e.g., 5%) at which point we consider the parameters to have reached steady state. We want the smallest learning rate that satisfies the convergence time constraint because the steady-state error decreases with smaller learning rates. The key in solving this inequality is noting that ‖E[_{t}]‖ converges exponentially with the inverse convergence rate defined in Theorem 1. So

We provide the conclusions of the above derivations resulting in the inverse functions for both objectives in the following theorem:

_{1} _{ave} _{bd} on the steady-state error covariance while allowing for the fastest convergence time is given by

_{bd} on the convergence time, defined to be the time-point at which the relative parameter error is E_{rest}, is given by

To summarize, if the objective is to bound the steady-state error covariance, then the user will select the upper-bound _{bd}, calculate _{ave} defined in Theorem 1, and apply _{bd}, what percentage of error at convergence time they are willing to tolerate _{rest}, calculate _{ave}, and use

So far we have assumed that the observation noise variance, _{min} and _{max}) and use the calibration algorithm to compute the learning rate for both _{min} and _{max}. Then for the first calibration objective, we can select the smaller of the two _{min} and _{max}. This smaller _{ave} in Theorem 1 is monotonic with respect to _{1}, …, _{n}). From _{1}. Together, these imply that the learning rate is a monotonic function of

Finally, to adaptively estimate _{t|t−1} and the prediction covariance _{t|t−1}, we use _{t|t} using _{t|t} for _{t|t}.

We now follow the same formulation used for continuous-valued signals, such as LFP or ECoG, to derive the calibration algorithm for the discrete-valued spiking activity. The derivation follows similar steps but, due to the nonlinearity in the observation model, has some differences that we point out. Given the nonlinearities, in this case, the calibration algorithm can be derived for the main first objective, i.e., to keep the steady-state error covariance below a desired upper-bound while minimizing convergence time (

The spiking activity can be modeled as a time-series of 0’s and 1’s, representing the lack or presence of spikes in consecutive time-steps, respectively. This discrete-time binary time-series can be modeled as a point process [_{t} [^{c}(⋅) is the firing rate of neuron ^{c} = [^{c}, (^{c})′]′ are the encoding model parameters to be learned. Note that the normalization constant in

For spikes, a PPF can estimate the parameters using data in a training session in which the encoded state can be either observed or inferred [^{c} using an optimal feedback-control model to infer the intended velocity, resulting in fast and robust parameter convergence [^{c} can be updated separately [_{t} is a white Gaussian noise with covariance matrix _{n}(_{t} is simply used to model our uncertainty at time

Given the observation model in _{t|t} and _{t|t−1}, and their covariances by _{t|t} and _{t|t−1}, respectively, the adaptive PPF—derived using the Laplace Gaussian approximation to the posterior density—is given by the following recursions [_{t|t} determines the relative weight of the neural observation _{t} compared with the previous parameter estimate in updating the current parameter estimate and thus determines how fast _{t|t} is learned in _{t|t} is governed by _{t} is the encoded behavioral/brain state (e.g., rat position in a maze or intended velocity in BMI), which is either observed or inferred. In studying the hippocampal place cell plasticity, for example, rat position can be observed. In motor BMIs, the intended velocity can be inferred using a supervised training session in which subjects perform instructed BMI movements [_{t|t}]→_{t|t}]‖.

Learning rate calibration for spikes can again be posed as an optimization problem. We denote the error vector by _{t} = _{t|t} and the error covariance by _{t|t}, which is PPF’s estimate of the parameters, is asymptotically unbiased (lim_{t → ∞} _{t|t}] = _{t}]→

We derive the calibration algorithm similar to the case of continuous signals. We first find a recursive equation for _{t}}, is periodic with period

_{t} _{1}, …, _{n})_{i} ≤ _{i+ 1})

Compared with the steady-state error covariance _{i} with _{i} and _{i} with _{i} and _{ave} includes the firing rate λ(_{ave} has the same role as ^{−1} in _{ave} for KF, and since _{bd}.

For both discrete and continuous signals, we considered a periodic behavioral state (e.g., intended velocity) in the training data for the derivations to satisfy the mild conditions in Appendix C in _{ave} and _{ave} for the continuous and discrete signals, respectively, which are simply the average values of functions of the state {_{t}}. So the core information needed in the calibration algorithm is not the state periodicity, but its expected value, which we can compute empirically for any state evolution. As detailed in Appendix E in _{t} is simply required to ensure that the mean of the prediction covariance _{t+ 1|t} is well-defined at steady state. If we ignore some mathematical rigorousness and instead assume that _{t+ 1|t} has bounded steady-state moments (which is a relatively mild requirement), then this calibration algorithm can be generalized to the case with non-periodic _{t} directly. That is precisely why, as we show using simulations in the Results section, the calibration algorithm works even in the case of random evolution for the states {_{t}} in the training experiment. Periodicity is simply required to guarantee the _{t+ 1|t} at steady state (instead of assuming this existence) in the derivations, as detailed in Appendix E in

To validate the calibration algorithm, we run extensive closed-loop numerical simulations. We show that the calibration algorithm allows for fast and precise learning of encoding model parameters, and subsequently for a desired transient and steady-state behavior of the decoders (

In motor BMIs, the relevant brain state is the intended movement. The BMI needs to learn an encoding model that relates the neural activity to the subject’s intended movement. We simulate a closed-loop BMI within a center-out-and-back reaching task with 8 targets. In this task, the subject needs to take a cursor on a computer screen to one of 8 peripheral targets, and then return it to the center to initiate another trial [

To simulate the intended movement, we use the OFC model. We assume that movement evolves according to a linear dynamical model [_{t} and _{t} being the position and velocity vectors in the two-dimensional space, respectively. Here _{t} is the control signal that the brain decides on to move the cursor and _{t} is white Gaussian noise with covariance matrix

The OFC model assumes that the brain quantifies the task goal within a cost function and decides on its control commands by minimizing this cost. For the center-out movement task, the cost function can be quantified as [_{v} and _{r} are weights selected to fit the profile of manual movements. For the linear dynamics in _{t} in our simulations [

The subject’s intended velocity _{t} is in turn encoded in neural activity. We first test the performance of the calibration algorithm for continuous ECoG/LFP recordings. We then test this performance for discrete spike recordings.

For the continuous signals, we simulate 30 LFP/ECoG features whose baseline powers and preferred directions in _{0|0}, and the true value,

For spikes, we simulate 30 neurons. Here since the state _{t} is the intended velocity, we can also interpret _{t} is the direction of _{t}, ^{c} is the preferred direction of the neuron (or direction of ^{c} = ‖^{c}‖[cos ^{c}, sin ^{c}]′), and finally ‖^{c}‖ is the modulation depth. For each neuron, we select the baseline firing rate randomly between [4, 10] Hz and the maximum firing rate randomly between [40, 80] Hz. We select each neuron’s preferred direction in

We also examine the effect of the calibration algorithm on kinematic decoding. For continuous signals, we use a KF kinematic decoder as in prior work (e.g., [

We first investigate whether the calibration algorithm can analytically approximate two quantities well: the steady-state error covariance and the convergence time of the encoding model parameters as a function of the learning rate. We do so by running multiple closed-loop BMI simulations with different learning rates. These Monte-Carlo simulations allow us to compute the true value of the two quantities. We then compare these true values with the analytically-computed values from the calibration algorithm. We find that, for both continuous and discrete signals, the calibration algorithm accurately computes its desired quantity (i.e, either the error covariance or the convergence time) for any type of behavioral state trajectory in the training data (i.e., periodic or not). Thus the calibration algorithm can find the optimal learning rate for a desired trade-off between the parameter convergence time and error covariance. We also show how the inverse function can be used to compute the learning rate for a desired trade-off. Moreover, we examine how the calibration algorithm—and consequently the learned encoding model—affects decoding performance. We show that, by finding the optimal learning rate, the calibration algorithm results in fast and accurate decoding. In particular, compared to the optimal rate, larger learning rates could result in inaccurate steady-state decoding performance and smaller learning rates result in slow convergence of the decoding performance.

We first assess the accuracy of the analytically-computed error covariance and convergence time by the calibration algorithm. As described in detail in Numerical Simulation section, we run a closed-loop BMI simulation in which the subject performs a center-out-and-back task to eight targets in counter-clockwise order. We simulate 30 LFP/ECoG features.

We define the convergence time as the time when the estimated parameters reach within 5% of their true values, i.e., ‖_{t|t} − _{0|0} − _{rest} = 0.05; as defined before _{t|t}, _{0|0} are the current parameter estimate, the true parameter value, and the initial parameter estimate, respectively.)

(A) The analytically-computed and the true error covariance and convergence time of the encoding model parameters (baseline, _{x}, and _{y} in

In the above analysis, we considered estimating the encoding model parameters _{t|t} in _{t|t} to show that it converges to the true value regardless of the learning rate _{t|t} converges to the true value with all tested learning rates, which cover a large range (5 × 10^{−7} to 5 × 10^{−3}). Moreover, even when estimating both _{t|t} and the noise variance _{t|t} jointly, the analytically-computed error covariance is still close to the true one (normalized RMSE is 4.5%). Overall, the analytically-computed error covariance is robust to the uncertainty in _{t|t} because _{t|t} converges to the true value at steady state regardless of the learning rate (

Here we show how the inverse functions in Theorem 2 can be used to select the learning rate. In our example, we require the 95% confidence bound of the estimated encoding model parameters (i.e., ±2 standard deviations of error) to be within 10% of their average value. Thus this constraint provides the desired upper-bound on the steady-state error covariance _{bd}. In general, _{bd} can be selected in any manner desired by the user. Once _{bd} is specified, we use _{1} = 5.6 × 10^{−5}. Hence the calibration algorithm dictates that the learning rate should be smaller than _{1} to satisfy the desired error covariance upper-bound.

Let’s now suppose that we want to ensure that the convergence time is within a given range. In our example, we require the estimation error to converge within 7 minutes, where convergence is defined as reaching within 5% of the true value (_{rest} = 0.05). This constraint sets the upper-bound on the convergence time to be _{bd} = 7min = 420 sec. The calibration algorithm using ^{−5}.

Taken together, for the above constraints for error covariance and convergence time, any learning rate 4.75 × 10^{−5} < ^{−5} is admissible. For conciseness and as an illustrative example, we select the learning rate ^{−5}, which satisfies both criteria above. In the next section, we examine the effect of this learning rate on the estimated model parameters over time, i.e., on the adaptation profiles (

(A–C) show sample adaptation profiles of the model parameters _{t|t} for different learning rates

We also examined the evolution of the estimated encoding model parameters _{t|t} in time, which we refer to as the parameter adaptation profiles. Plotting the adaptation profile provides a direct way of investigating the influence of the learning rate on the estimated encoding model. We plot the adaptation profiles for the optimal learning rate in our example above, i.e., ^{−5}. We also show these profiles for a smaller and a larger learning rate (

The adaptation profiles confirm the accuracy of the calibration algorithm as expected from

In the algorithm derivation and for rigorousness to ensure the existence of the mean of the prediction covariance _{t+ 1|t} at steady state (instead of simply assuming this existence; Appendix E in _{t}}, e.g., the trajectory, is periodic in the training data. However, in computing the error covariance and the convergence time, the only aspect of _{t} needed by the calibration algorithm is not periodicity, but an average of a function of _{t} over time, which is _{ave}. Indeed, if we assume _{t+ 1|t} has bounded steady-state moments, then our derivation directly applies to the general non-periodic case (Appendix E in _{t|t} from _{t|t} simultaneously with parameters, the calibration algorithm can approximate the error covariance well (normalized RMSE is 2.6%). Taken together, these results demonstrate that the calibration algorithm can generalize to a wide range of problems since the training state-evolution when adapting the encoding models could have a general form.

Figure convention is the same as

We also validate the calibration algorithm for discrete-valued spiking observations. We run multiple closed-loop BMI simulations with either a periodic or a non-periodic trajectory. The simulation setting is the same as that for continuous signals and given in Numerical Simulation section.

(A) The analytically-computed and the true steady-state error covariance as a function of the learning rate

In the case of spikes, the inverse function can again be used to select the learning rate for a given upper-bound on the steady-state error covariance. For example, we can require the error covariance to be within 7% of the average values for all parameters, which provides the value of _{bd}. Again, _{bd} can be selected as desired by the user. Once _{bd} is specified, we use the inverse function using Theorem 3 and ^{−7}.

We also confirm the accuracy of the calibration algorithm using the parameter adaptation profiles. We plot three realizations of the estimated point process parameters, _{t|t}, under different learning rates ^{−7}, and a smaller and a larger learning rate in

(A)–(C) show sample adaptation profiles of model parameters _{t|t} in a closed-loop BMI simulation under different learning rates

Finally, even though the convergence time cannot be analytically obtained in the case of spike observations, it is still significantly affected by the learning rate ^{−9}), the parameter estimate _{t|t} does not converge to its true value even in 2000 sec. In comparison, this convergence time is only about 200 sec for an intermediate learning rate (^{−7}). Hence to allow for fast convergence, it is critical to select the maximum possible learning rate that satisfies a desired upper-bound constraint on error covariance. This was the basis for the calibration algorithm.

The selection of the optimal learning rate is critical not only for fast and accurate estimation of the encoding model, but also for accurate decoding of the brain state. Here we show that the selection of the appropriate learning rate by the calibration algorithm can improve both the transient and the steady-state operation of decoders. We simulate closed-loop BMI decoding under various learning rates. Since the optimal trajectory for reaching a target in a center-out task should be close to a straight line connecting the center to the target, as the measure of decoding accuracy we use the RMSE between the decoded trajectory and these straight lines [

To study the effect of the learning rate on steady-state BMI decoding, we adaptively estimate the encoding model parameters under different learning rates. We fix the estimated parameters after varying amounts of adaptation time. We then use the obtained fixed models to run the closed-loop BMI simulations without adaptation. We run these simulations both for continuous LFP/ECoG observations decoded with a KF kinematic decoder, and for discrete spike observations decoded with a PPF kinematic decoder (Figs

(A) The evolution of the decoded trajectory as the adaptation time is increased under different learning rates ^{−1}) even at steady state; this means that depending on exactly when we stop the adaptation and fix the decoder, performance widely oscillates due to the large steady-state model parameter error. (B) RMSE of the decoded trajectory under different learning rates for different adaptation times. RMSE is computed for a fixed decoder that was obtained by stopping the adaptation at various times (different colors). RMSE converges faster as the learning rate is increased (^{−5} to 5 × 10^{−3}, for example). However, if the learning rate is selected too large (^{−1}), RMSE oscillates depending on when adaptation is stopped, without converging to a stable number. These results show that appropriately calibrating the learning rate is important not only for encoding model estimation but also for a desired trade-off between convergence time and steady-state RMSE in decoding.

Figure conventions are the same as ^{−3}), i.e., the performance widely oscillates. (B) RMSE of the decoded trajectory under different learning rates for different adaptation times. RMSE is computed for a fixed decoder that was obtained by stopping the adaptation at various times (different colors). RMSE converges faster as the learning rate is increased (^{−7} to 10^{−5}, for example). However, if the learning rate is selected too large (^{−3}), RMSE oscillates without converging to a stable number. These results again demonstrate the importance of calibrating the learning rate for fast convergence and accuracy of decoding.

By comparing the small and medium learning rates, we find that a small learning rate results in a slow rate of convergence for the decoder performance, without improving the steady-state performance (two-sided t-test

It is interesting to note that due to feedback-correction in closed-loop BMI, the decoder can tolerate a larger steady-state parameter error than we would typically allow if our only goal is to track the encoding model parameters. This is evident by noting, for example, that using a learning rate of ^{−3} for continuous signals results in a relatively large steady-state parameter error as shown in

Developing invasive closed-loop neurotechnologies to treat various neurological disorders requires adaptively learning accurate encoding models that relate the recorded activity—whether in the form of spikes, LFP, or ECoG—to the underlying brain state. Fast and accurate adaptive learning of encoding models is critically affected by the choice of the learning rate [

To derive the calibration algorithm, we introduced a formulation based on the fundamental trade-off that the learning rate dictates between the steady-state error and the convergence time of the estimated parameters. Calibrating the learning rate analytically requires deriving two functions that describe how the learning rate affects the convergence time and the steady-state error covariance, respectively. However, currently no explicit functions exist for these two relationships for Bayesian filters, such as the Kalman filter or the point process filter. We showed that the two functions can be analytically derived (Eqs

To allow for rigorous derivations in finding tractable analytical solutions for the learning rate, we performed the derivations for the case in which the behavioral state in the training experiment evolved periodically over time. This is the case in many applications; for example, in motor BMIs, models are often learned during a training session in which subjects perform a periodic center-out-and-back movement. However, we found that the calibration algorithm only depended on an average value of the behavioral state rather than on its periodic characteristics. Indeed, we showed that with a simplifying assumption, the derivation extends to the general non-periodic case (Appendix E in

We derived the calibration algorithm for Bayesian adaptive filters, i.e., KF for continuous-valued activity and PPF for discrete-valued spikes. Here the KF and PPF were used to adaptively learn the neural encoding model parameters, which were assumed to be unknown but essentially fixed within the time-scales of parameter learning. This scenario is largely the case that arises in neurotechnologies for learning encoding models/decoders for two reasons. First, in neurotechnologies, such as BMIs, the parameters of the encoding models are initially unknown because they need to be learned in real time during closed-loop operation (cannot be learned offline and a-priori before actually using the BMI). Second, even though these parameters are unknown, they are largely fixed at least within relevant time-scales of parameter learning (e.g., minutes) in BMIs (and even typically within time-scales of BMI operation in a day, e.g., hours; see for example [

In deriving the calibration algorithm, we assumed that recorded signals (whether continuous or discrete) are conditionally independent over channels and in time, similar to prior work [

The selected learning rate in the calibration algorithm depends on the user-specified upper-bound on the error covariance or convergence time. The values of these upper-bounds could be chosen by the user based on the goal of adaptation. If the adaptation goal is to accurately estimate the encoding model parameters (e.g., to study learning), then the acceptable error upper-bound may be selected to be small. In such a case, the calibration algorithm would select a small learning rate. However, we showed that if the goal of calibration is to enable accurate decoding in a closed-loop BMI, then larger errors in the estimated parameters may be tolerated. This is due to feedback-correction in BMIs, which can compensate for the parameter estimation error (Figs

The calibration algorithm may also serve as a tool to help examine the interaction between model adaptation and neural adaptation. In closed-loop neurotechnologies, neural representations can change over time resulting in neural adaptation, e.g., due to learning over multiple days. For example, in motor BMIs, the brain can change its encoding of movement (e.g., the directional tuning of neurons) to improve neuroprosthetic control [

To validate the calibration algorithm, we used a motor BMI as an example. The calibration algorithm, however, can be applied to other closed-loop neurotechnologies that need to decode various brain states, for example, interest score in closed-loop cortically-coupled computer vision for image search [

Our main contribution is the derivation of a novel analytical calibration algorithm for both nonlinear point process and linear Gaussian encoding models (Eqs

Here our focus was on deriving an analytical calibration algorithm for both nonlinear point process and linear Gaussian encoding models for spikes and continuous neural recordings, respectively. Thus to validate our analytical approach, we used extensive closed-loop Monte-Carlo simulations. These simulations allowed us to examine the generalizability of the calibration algorithm across different neural signal modalities. The closed-loop simulations closely conformed to our prior non-human primate experiments [

Finally, the calibration algorithm has the potential to be generalized to Bayesian filters beyond the KF and PPF, e.g., the unscented Kalman filter [

Simulation of a BMI system in which parameters are estimated at the beginning of each day and fixed for the rest of the day. This is the setup used in the vast majority of BMI systems because encoding model parameters are either largely time-invariant or change much slower compared with the relevant time-scales of parameter adaptive learning (e.g., minutes) in BMIs and even the time-scale of BMI operation in a day (e.g., hours) (see _{ave} is simply an expectation (average) of a function of _{ave} based on the same average quantity to compute the optimal learning rate on both days. The calibration algorithm satisfies the user-specified criteria on parameter estimates on day 1. We then assume that on day 2 parameters have shifted. On day 2, parameters can again be estimated using the same Kalman filter whose learning rate is selected with the calibration algorithm. Similar to day 1, on day 2 the requirements on steady-state error and convergence time are again satisfied.

(TIF)

The derivation of the calibration algorithm with a periodic encoded state _{t} during the training session follows the blue arrows. If we assume that the prediction covariance _{t+1|t} has bounded steady-state moments, then the proof generalizes to the non-periodic _{t} as shown by the red arrows (see Appendix E in _{t+1|t} has bounded steady-state moments, then the proof generalizes to the non-periodic _{t} (_{t+1|t} at steady state can be approximated using _{ave} in Theorem 3 to find the optimal learning rate. Here DRE refers to the discrete Riccati equation.

(TIF)

(PDF)