Fig 1.
Closed-loop neural systems often need to learn an encoding model adaptively and in real time. The encoding model describes the relationship between neural recordings and the brain state. For example, the relevant brain state in motor BMIs is the intended velocity and in DBS systems is the disease state, e.g., in Parkinson’s disease. The neural system uses the learned encoding model to decode the brain state. This decoded brain state is then used, for example, to move a prosthetic in motor BMIs while providing visual feedback to the subject, or to control the stimulation pattern applied to the brain in DBS systems. A critical parameter for any adaptive learning algorithm is the learning rate, which dictates how fast the encoding model parameters are updated as new neural observations are received. An analytical calibration algorithm will enable achieving a predictable level of accuracy and speed in adaptive learning to improve the transient and steady-state operation of neural systems.
Fig 2.
Flowchart of the calibration algorithm.
Fig 3.
The calibration algorithm accurately computes the steady-state error covariance and convergence time as a function of learning rate for continuous signals.
(A) The analytically-computed and the true error covariance and convergence time of the encoding model parameters (baseline, ηx, and ηy in (1)) for different learning rates s, across a wide range of s. The top left panel shows the relation between the three quantities. The other three panels are projections of this plot to three planes, showing each of the three pair-wise relationships. All axes are in log scale. True quantities are computed from BMI simulations with periodic center-out-and-back training datasets. The analytically-computed values are obtained by the calibration algorithm according to Eqs (7) and (8). The analytically-computed and true values match tightly across a wide range of learning rates, showing that the calibration algorithm can accurately compute the learning rate for a desired trade-off between steady-state error and convergence time. (B) Adaptive estimation of the unknown observation noise variance using (11) under different learning rates s. The bottom three panels are zoomed-in versions of the top panels to show the transient behavior of the estimated noise variance, which converges to its true value in all cases.
Fig 4.
Parameter adaptation profiles confirm the accuracy of the calibration algorithm with continuous signals.
(A–C) show sample adaptation profiles of the model parameters ψt|t for different learning rates s in ascending order. For each learning rate, the estimated parameters are within the analytically-computed 95% confidence bounds by the calibration algorithm about 96% of the time, demonstrating the accuracy of the calibration algorithm.
Fig 5.
The calibration algorithm generalizes to training datasets with non-periodic state trajectories.
Figure convention is the same as Fig 3. Here the true quantities are computed in closed-loop BMI simulations with a non-periodic trajectory generated by selecting targets randomly and uniformly. The analytically-computed error covariance and convergence times given by the calibration algorithm closely match their true values across a wide range of the learning rate s, showing that the calibration algorithm extends across training datasets with different state-evolution trajectories.
Fig 6.
The calibration algorithm accurately computes the steady-state error covariance for discrete spiking activity.
(A) The analytically-computed and the true steady-state error covariance as a function of the learning rate r. True values are found from closed-loop BMI simulations with a periodic center-out-and-back trajectory. The calibration algorithm analytically computes the covariance based on (19). The calibration algorithm closely approximates the steady-state error covariance as demonstrated by the closeness of the analytically-computed and true curves across a wide range of r. (B) Figure convention is the same as (A) except that all true values are computed in closed-loop BMI simulations with a non-periodic trajectory generated by selecting one of the eight targets randomly and uniformly in each trial. The calibration algorithm can again closely approximate the steady-state error covariance, demonstrating the generalizability of the approach to training datasets with varying state-evolution trajectories.
Fig 7.
Parameter adaptation profiles confirm the accuracy of the calibration algorithm with discrete spiking activity.
(A)–(C) show sample adaptation profiles of model parameters ϕt|t in a closed-loop BMI simulation under different learning rates r in ascending order. Increasing the learning rate increases the error covariance. Also, about 96% of the time, the parameter estimates at steady state are within the 95% confidence bounds computed by the calibration algorithm; this demonstrates that the calibration algorithm can closely approximate the error covariance and consequently the confidence bounds.
Fig 8.
Learning rate calibration affects both the transient and the steady-state performance of closed-loop BMI decoders with continuous neural activity.
(A) The evolution of the decoded trajectory as the adaptation time is increased under different learning rates s. Note that the decoder is fixed after a given adaptation time is completed (as noted on each row). The fixed decoder is then used to generate the displayed trajectories. Each color corresponds to one learning rate. Decoding performance is unstable when the learning rate is large (s = 5 × 10−1) even at steady state; this means that depending on exactly when we stop the adaptation and fix the decoder, performance widely oscillates due to the large steady-state model parameter error. (B) RMSE of the decoded trajectory under different learning rates for different adaptation times. RMSE is computed for a fixed decoder that was obtained by stopping the adaptation at various times (different colors). RMSE converges faster as the learning rate is increased (s = 5 × 10−5 to 5 × 10−3, for example). However, if the learning rate is selected too large (s = 5 × 10−1), RMSE oscillates depending on when adaptation is stopped, without converging to a stable number. These results show that appropriately calibrating the learning rate is important not only for encoding model estimation but also for a desired trade-off between convergence time and steady-state RMSE in decoding.
Fig 9.
Learning rate calibration affects both the transient and the steady-state performance of closed-loop BMI decoders with discrete spiking activity.
Figure conventions are the same as Fig 8. (A) The evolution of the decoded trajectory across time under different learning rates r. Each color corresponds to one learning rate. As in Fig 8, the decoder is fixed after a given adaptation time is completed (as noted on each row). The fixed decoder is then used to generate the displayed trajectories. The decoding performance is unstable when the learning rate is large (r = 10−3), i.e., the performance widely oscillates. (B) RMSE of the decoded trajectory under different learning rates for different adaptation times. RMSE is computed for a fixed decoder that was obtained by stopping the adaptation at various times (different colors). RMSE converges faster as the learning rate is increased (r = 10−7 to 10−5, for example). However, if the learning rate is selected too large (r = 10−3), RMSE oscillates without converging to a stable number. These results again demonstrate the importance of calibrating the learning rate for fast convergence and accuracy of decoding.