A computational scheme for internal models not requiring precise system parameters

Utilization by humans of a precise and adaptable internal model of the dynamics of the body in generating movements is a well-supported concept. The prevailing opinion is that such an internal model ceaselessly develops through long-term repetition and accumulation in the central nervous system (CNS). However, a long-term learning process would not be absolutely necessary for the formation of internal models. It is possible to estimate the dynamics of the system by using a motor command and its resulting output, instead of constructing a model of the dynamics with precise parameters. In this study, a computational model is proposed that uses a motor command and its corresponding output to estimate the dynamics of the system and it is examined whether the proposed model is capable of describing a series of empirical movements. The proposed model was found to be capable of describing humans’ fast movements which require compensation for system dynamics as well as sensory delays. In addition, the proposed model shows equifinality under inertial perturbations as seen in several experimental studies. This satisfactory reproducibility of the proposed computation raises the possibility that humans make a movement by estimating the system dynamics with a copy of motor command and sensory output on a momentary basis, without the need to identify precise system parameters.


Introduction
Humans are endowed with a remarkable ability to execute limb movements even in the presence of changing loads arising either from interaction with the environment or from variation in properties of the sensorimotor system. Despite substantial delays in feedback loops and the dynamical properties of the sensorimotor system, they are capable of producing fast movements in a smooth trajectory. To explain these motor abilities, the so-called "internal model" was devised in the field of human motor behavior [1][2][3][4][5][6][7][8][9][10]. The internal model is a hypothesized controller residing in various brain regions including the motor cortex and/or cerebellum. Evidence of the existence of the internal model can be found in several phenomena. For instance, deafferented primates are able to reach a target point with their arms even in the absence of sensory feedback. Too, interacting forces between joints during movements need to be compensated to minimize movement error, yet this is nearly impossible with feedback control alone [4].
Though it is not a universal phenomenon, especially in destabilizing force fields [11][12][13][14][15], it is empirically observed that humans are capable of reaching a target position in a range of transient and smooth mechanical perturbations [16][17][18][19][20][21]. The equifinality property could not be explained using model-based motor control formulations with an internal model that suppose that the system adjusts the internal model by integrating the new load condition in response to the perturbation and accurately produces perfect corrective torques to bring the system to the same final position [21]. In model-based motor control formulations, integration of a perturbation into internal models requires practice involving considerable trials (e.g., [11]). However, equifinality is exhibited in movements without the need of adaptation to applied perturbations. Experimental evidence has supported equifinality by humans even within a single trial through online corrections. An intact monkey can return his arm back to a predefined trajectory immediately after his arm is perturbed while moving to a target [22]. Equifinality alludes the possibility that internal models can be formed and updated even on a momentary basis, as well as, on a trial-by-trial basis [11,15,23,24]. This possibility is supported by research activities that have evaluated continuous adaptation of humans in motor behaviors [25][26][27][28].
An input-output relationship exists between the motor command and its resulting limb kinematics; the input acts on the neuromuscular system and environment in contact with the limb, while the output reflects the dynamic response. Although it is difficult to identify the properties of the system and environments with only the input and output available, the quantitative relationship between the input and output can be used to estimate the dynamics of the neuromuscular system for the purpose of formulating subsequent control actions. With an estimate of the limb and environment dynamics made at the previous step, the dynamics of the musculoskeletal system and environment in the current step could be compensated. In other words, the estimate can be used in place of a system model.
In fact, time-delay control (TDC) in the controls field utilizes the mechanism, which is called time-delay estimation (TDE) [29]. TDE uses the previous-step sensor reading and a record of previous-step command to quantitatively estimate the system dynamics and disturbances, which are otherwise difficult to identify precisely. The estimate cancels out the system dynamics and uncertainties through their incorporation into the control at the current step. Consequently, employing the TDE technique leads to accuracy and robustness of control in the presence of a wide class of uncertainties under infinitesimally small sampling intervals. TDE alleviates computation load for the system dynamics and uncertainties in comparison with other robust control schemes [30][31][32][33][34]. It does not require the use of high gain control. TDC provides an insight as to how humans form internal models.
In this study, a computational model of human motor control is proposed that estimates system dynamics in a similar way to TDC. The question as to whether humans estimate the dynamics of the neuromuscular system using the input-output relationship is examined. Regarding the issue of delays in sensory feedback loops, the proposed model handles these delays by taking the architecture of the Smith predictor. In 1993, Miall and colleagues borrowed the Smith predictor from control engineering to describe behaviors of biological systems with delays in feedback loops [35]. The Smith predictor explained how biological systems might overcome feedback delays. The Smith predictor is a model-based controller, and accordingly requires system parameters. The forward and inverse models of the Smith predictor are liberated from the requirement for a model of the dynamics with precise parameters with the aid of the TDC principle.
Through simulation studies, it is investigated whether or not the proposed computation is capable of reproducing a series of fast movements. Feedback control alone cannot be considered for fast movements due to the delays inherent in sensory systems. Anticipatory control using forward models needs to be involved to make an appropriate action before sensory information is available. Also, feedback control alone cannot compensate for intersegmental interaction forces that arise during multi-joint movements [4]. These movements support the existence of internal models. If the proposed computation successfully reproduces fast movements, this would imply the possibility that the human motor control system produces movements in a similar manner to the operation of TDC. That is, in humans it would be possible to estimate system dynamics using the input-output relationship, not requiring a model of limb dynamics with precise parameters. This hypothesis is in line with the perspective that a functionally good enough representation/estimation of the system is sufficient to produce a rapid response to perturbations [36,37].
A further investigation regarding whether the proposed computation captures movements under unexpected change in load is carried out. Perturbations by inertial changes would offer a tool to investigate the issue regarding whether or not internal models involve a precise system model. Attempts to reproduce the unexpectedly perturbed movements with a modelbased internal model controller and the proposed controller provide an insight into the possibility that humans make movements without system parameter identification.

Computed torque method for human movements
In 2002, Jagacinski and Flash introduced a controller to describe human movements [38]. This controller, which is based on the computed-torque control (CTC) for a single-DOF arm model, is re-formulated here for a multi-DOF arm model.
The dynamics of the arm is generally described as where θ 2 R n denotes the joint angles; M(θ) 2 R n×n the inertia matrix; Vðy; _ yÞ 2 R n the Coriolis and centrifugal forces; G r (θ) 2 R n the gravitational forces; Fð _ yÞ 2 R n the friction; U the unknown disturbances; τ 2 R n the control inputs.
Eq (1) is considered as a system model throughout this study. The controller presented in [38] for the arm model consists of system dynamics cancellation, the feedforward component and the feedback component.
yÞ þ G r ðyÞ þ Fð _ yÞ þ U |ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl {zffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl ffl } where θ d 2 R n denotes a desired angle vector, and K v , K p 2 R n×n are diagonal viscosity and stiffness matrices with diagonal elements K v1 , K v2 , � � �, K vn and K p1 , K p2 , � � �, K pn , respectively. Dynamics cancellation is carried out by putting force components of the arm system into the system through the control inputs. The feedforward component is proportional to the desired acceleration € y d , because the controlled system is an acceleration control system [38]. The feedforward control utilizes the inverse dynamics of the system; the torque component is created by the acceleration of the desired trajectory, which is programmed according to an intended movement. In the case that the initial condition of the arm is quiescent and no uncertainty exists, the actual position of the arm converges to the desired one [38]. The third component, feedback, plays a role in diminishing the error between the desired trajectory and actual trajectory measured by the sensory systems.
Injecting the control inputs into the arm produces the following error dynamics: From this error dynanmics, a simplified block diagram of the arm system linearized by the controller can be derived as shown in Fig 1. The closed-loop dynamics of the arm can be expressed as a cascade combination of a controller C and a plant G.
However, the CTC is not tolerant to sensory delays. This indicates that the CTC is not suitable to describe human motor control involving sensory delays.

Smith predictor
The Smith predictor, proposed in [39], is a control architecture for systems with delays, as shown in Fig 2. The outer control loop feeds back the actual state of the system G, but due to the delay of the feedback loop, use of the outer loop alone would not provide satisfactory control performance and lead to instability in the worst case. Thus, the inner loop is added to send the (estimated) current state to the controller C. The current state is estimated using a system modelĜ that is supposed to be simulated using a copy of the control input. The Smith predictor delays the estimated state as long as the actual state is delayed so that the delayed actual state and delayed estimated state cancel one another. If the perfect match between these two delayed states is made, the controller C can show control performance with no influence of delay on the outer feedback loop. Miall and colleagues [35] employed this Smith predictor in human control modelling to describe motor ability by the CNS even in the face of sensory delays. They assumed that the controller C and system modelĜ are devised for the inverse model and forward model in the cerebellum, respectively. Now, let us try to combine the CTC with the Smith predictor. From the lineaized arm system with the CTC, which is shown in Fig 1. The controller C CTC of the CTC can match with the controller C of the Smith predictor and the linearized plant G CTC of the CTC with the plant G of the Smith predictor. Then the forward model can be expressed aŝ Note that the forward model equals to the plant linearized by the CTC. This implies that the forward model provides estimated states using the same plant dynamics controlled under the CTC.

Proposed control
So far, the CTC has been combined with the Smith predictor to liberate the CTC from the sensory delay issue. But the CTC does require a model of the dynamics with precise parameters. Time-delay estimation (TDE) can eliminate this requirement while it reduces computational load.
The equations of motion (1) can be algebraically manipulated to express it with an explicit input-output relationship. Introducing a diagonal constant matrix � b 2 R n�n consisting of  diagonal elements � b 1 ; � b 2 ; � � � ; � b n , and grouping the system dynamics plus the disturbance term U into one term (H), Eq (1) is expressed as follows: where In Eq (5), the input τ is linked to the output € y through the term H, which suggests that the term H can be quantitatively estimated using the input and output. Meanwhile, it is acceptable to assume that the term H is piece-wise continuous if the disturbance term U is continuous. This implies that the value of the term H (t) at an instant t can be approximated by its value at the previous instant t − dt. The shorter the gap dt between two consecutive instants of time leads to the more accurate approximation. An estimate of the value of the term H can be obtained in this way: In practice, the value of the H at the previous instant can be calculated using the input and output at the previous instant, as suggested in Eq (5), as follows: If the value of (t ðtÀ dtÞ À � b € y ðtÀ dtÞ ) is included in the motor command, the system dynamics plus the disturbance U are cancelled out. As in the CTC, the motor command injects feedforward torques ( € y d ) and restoring torques that can be realized by placing a spring (K p ) and damper (K v ) between the actual limb position and the desired limb position of each joint.
Then, the control law is expressed as TDC provides the similar desired error dynamics as the CTC: Accordingly, TDC can be adjustable to the architecture of the Smith predictor with a forward model, which is selected asĜ From the error dynamics (10), it is possible to obtain the estimates of the state vectors y; _ y and € y, defined asŷ; _ y and € y respectively, as follows: where The initial values of estimatesŷ; _ y can be assumed to be the same as those of the actual state vectors y; _ y. With the forward model, the current states are estimated and fed back to the controller. Note that this forward model does not require a model of the dynamics with precise parameters.
As shown in Fig 4, the actual position that is delayed in the feedback loop and its estimate that is intentionally delayed are compared. The difference between them is compared with the desired position. These differences can be expressed as θ r : where t d denotes the time delay of the feedback loop andt d is its estimate. Then, the controller forces the controlled system to follow θ r , reflecting that the controller receives the current state estimated by the forward model. The control law of the proposed model is designed as According to the closed-loop dynamics that the controller pursues, one of the formulations for the forward model needs to be modified to It would be possible to assume that the CNS selects appropriate values of the matrix � b and modulates the muscle viscosity and stiffness K v , K p according to a given task. The matrix � b determines the accuracy of dynamics estimation by TDE. A proof is presented in an Appendix.

Simulation
The proposed control model is validated through a series of simulation studies, examining whether the model is capable of reproducing empirical phenomena obtained from human subject experiments.
The first study investigates whether or not the proposed model is able to reproduce fast movements of short duration during which the sensory delay is too long to allow feedback corrections. It is predicted that computational models with anticipatory control can reproduce fast movements.
The second study evaluates whether the proposed model is capable of dealing with interaction forces that arise from two-joint movements. The proposed model is predicted to compensate for interaction forces successfully.
The last study attempts to address the issue regarding whether or not internal models are formed grounded in a model of the dynamics of the system. Model-based computation presents prediction that varies with changes in the dynamic system. If a load unexpectedly perturbs the system during movement, the system output would deviate from the planned end-point. In contrast, the proposed computational model, which is not model-based, would show that the planned end-point is reached even in the presence of a perturbation.

Arm model implementation
The experimental data considered in this study were produced by horizontal arm movements; the gravitational force was neglected. The arm is modeled with a 2 DOF system involving the shoulder and elbow joints, as depicted in Fig 5 and described as follows: where The parameters of the arm are adopted from [24]; for the upper arm, J 1 = 0.0141 (kgm 2 ), M 1 = 1.93 (kg), l 1 = 0.31 (m), l m1 = 0.165 (m), for the forearm, J 2 = 0.0188 (kgm 2 ), M 2 = 1.52 (kg), l 2 = 0.34 (m), l m2 = 0.19 (m). The lengths l m1 and l m2 denote the distances between the center of mass and the proximal joint.
It is assumed that human subjects plan a minimal-jerk trajectory in joint space and Cartesian space according to the given task [40]. The desired (planned) trajectory is designed as  where q 0 , q f denote the initial and final positions of the hand, T denotes the duration of the movement.
Movements to be reproduced I. Single-joint fast movement. Kistemaker and colleagues investigated single-joint (elbow) fast movements [41]. Participants were asked to direct their hand from the marked intermediate of one block to that of another block as fast as possible, once an auditory cue was presented. The participants practiced until they could move quickly to the target with minimal overshoot. Fig 4A in [41] shows trajectories by 6 participants of a flexion of 145 degrees starting from an initial flexion of 45 degrees for a duration of 0.2 s. Note that these trajectories can be described by the minimum-jerk trajectory.
II. Two-joint fast movement. Koike and colleagues had a participant make five movements to five different positions with a duration between 0.5 s and 0.75 s using the shoulder and elbow joints [42]. It was observed that deviations from the desired paths were more significant for fast movements than those for slow movements since fast movements involve larger interaction forces. Also it was found that deviations from transverse paths were more significant than those from radial paths. The path from point (-0.2 m, 0.5 m) to point (0.25 m, 0.35 m) is selected for this simulation study. A comparison in movements between 0.5 s and 1 s is made to examine whether the proposed control is able to compensate for increased interaction forces during fast movements. The desired paths are designed by the minimum-jerk trajectory as in [4].
III. Movement with unexpected inertial changes. Pinter and colleagues studied kinematic changes during movements under unexpected inertial perturbations [43]. They asked participants to move their elbow joint from a 50 degree to an 85 degree of flexion angle as fast as possible within 0.2 s but the inertial load was changed without informing the participants. The participants familiarized themselves with movements with an inertial load that was then replaced with either a lighter one (25% decrease) or a heavier one (25% increase).
The main findings of the study are that there is no significant error in the end-point position in the presence of changes in inertial loads and that motor commands are customized for a certain inertial load taking several trials.
In the present study, the trial of the low inertial load following continuous trials of the intermediate inertial load is labelled ML and the trial of the high inertial load following continuous trials of the intermediate inertial load is labelled MH. The label MM is made in a similar way.

Computational models for comparison
For a comparative study with a model-based control system, an optimal control model is chosen which supports the internal-model hypothesis [44][45][46][47]. This model is accompanied by a predictor for estimating the current states based on the sensory information delayed on feedback loops. Details on the optimal control used in this study are presented in the Appendix. The other computational model is an equilibrium-point controller that operates using sensory information with no anticipatory component. A modified equilibrium-point controller that was developed by adding velocity feedback loop to accommodate feedback delays is adopted [48,49]. The control law is t ¼ Dð _ y dðtÞ À _ y ðtÀ t dv Þ Þ þ Pðy dðtÞ À y ðtÀ t dp Þ Þ; where D, P are feedback gains and t dv , t dp denote feedback delays on velocity loop and position loop.

Simulation settings
Feedback signals are considered as an integration of all usable sources of sensory information about limb movements including proprioceptive and visual feedback. The sensory delay t d is moderately set as 65 ms [4,24]. Since a perturbation in sensory feedback was not given in the experiment, it is assumed that the CNS estimates the exact value of the sensory delay sot d is set to 65 ms. For the equilibrium-point model, a delay of 65 ms is put into the position feedback loop while a delay of 25 ms is imposed on the velocity feedback loop. For the study involving fast movements, all control gains are tuned so that they, within a reasonable range, give the minimal deviation from the desired trajectory between 0.1 s and 0.3 s (see Fig 4A in [41]). After the 0.3 s, it would be possible that the arm impedance drastically changes to stop the arm movement at the planned position. This range is not taken into account in the simulation studies where impedance is assumed to be constant. In the case of the optimal control, a steady-state linear quadratic (LQ) method based on the solution of the algebraic Riccati equation is used to set the values of the matrix K.
For the study about movements under unexpected inertial load changes, it is the first task to ascertain control gains that enable the arm to closely follow the minimal-jerk trajectory between the angles of 50  Simulations are conducted in Matlab using ODE45. The optimization toolbox in Matlab is used to check out the possibility of a significantly improved fit between the simulated trajectory and empirically observed trajectory. The values of Q and R in Eq (40) are set to [10 0;0 10] and 2, respectively, but as long as values of Q and R are not too low, it is observed that the optimal control can produce the minimum-jerk trajectory, noting that the forward model for this computation effectively deals with the delay of 65 ms on the feedback loop (41).

Single-joint fast movement
For the equilibrium-point control, the P gain and D gain in (25) are tuned to 0 and 0.76, respectively (the velocity feedback loop is faster than the position feedback loop). Note that the control gains are tuned by the optimization toolbox until the best match with the planed trajectory is achieved. However, this computation is unable to reproduce the minimum-jerk movement. This result implies that the non-existence of the forward model leads to the failure of reproducing.
The proposed control reproduces the minimum-jerk movement within the range between the 0.   It is observed that the two models reproduce the empirical results with the same patterns in terms of the peak angular velocity, the time to peak angular velocity relative to the total movement time, and the number of oscillations from the reversal point reversal point at which absolute angular velocity dropped below 5 degrees/s to the point where the amplitude of the However, the two computational models offer different predictions in terms of the distance between the target point and reversal point. While the output of the proposed model converges to the target position regardless of inertial loads presented, the optimal model does not. The two trajectories of conditions ML and MH in the optimal prediction reach the vicinity of the target point, but the end-point errors of the optimal model are notably greater than those of the proposed model, although the optimal model generates greater amplitude commands than the proposed model. Since the proposed model eliminates the system dynamics and injects planned dynamics, it is reasonable to expect the system simulated by the proposed control converges to a desired point asymptotically. Meanwhile, in the optimal control, estimates of the current state are produced with a system model in forward models and are fed back to internal models. If the system model becomes inaccurate due to unexpected loads, then the estimates become inaccurate accordingly, which leads to residual error in the end-point.

Discussion
TDC-type control shows robust performance against system parameter uncertainties. Using the motor command and its outcome at the previous instant, TDC-type control estimates the controlled system dynamics interacting with the environment. With no need of high stiffness and damping gains, TDC-type control achieves accurate and robust tracking tasks. Humans show a remarkable ability to execute limb movements even in the presence of changes in the environment as well as in the properties of the sensorimotor system, even with low stiffness [50]. Even in the case that the arm is perturbed gradually or abruptly during task, corrective torques will be generated to compensate for the perturbation and the arm will be positioned as planned in the end. These common points bring forward a question regarding whether humans control their limbs in a similar way to TDC-type control. This study proposed a

Model of human control
The proposed computation consists of the inverse model and forward model components as in typical computational models that support the existence of internal models in the CNS [2,3,6,35,45,[51][52][53]. From the view of advocators for the role of internal models in motor control, when initiating a reach to a target, a human sets a motor plan including a trajectory to be followed, based on the initial movement conditions. During movement, a forward model of the biological system integrates sensory inflow and a copy of motor outflow to estimate the consequence of the motor commands sent to a limb [1,[54][55][56]. The estimated position of the movement is continuously compared to the target position and the differences between them cause an error signal that modifies the motor command. The proposed model follows this mechanism, except for some points presented below.
The inverse model of the proposed computation calculates motor command that moves the controlled limb to a desired state. The inputs to the inverse model include an efferent copy of motor command and the position, velocity and acceleration values of the desired limb state and (estimated) actual limb state. Unlike typical internal-model-based controllers, an efferent copy of the motor command is sent to inverse models as well as forward models. The copy, with its estimated corresponding acceleration of the limb, is used to cancel out the dynamics of the limb and environment, and desired dynamics injected is formed by a combination of the position, velocity and acceleration of the limb based on muscle viscoelasticity [57]. It can be emphasized that a model of the dynamics with precise parameters is not required in the inverse model. The proposed forward model estimates the current limb state that is otherwise substantially delayed by the transmission along feedback loops. The model takes a copy of motor command and sensory information as inputs. The copy provides the forward model with information about the desired shape of the system dynamics designed in the inverse model. The forward model supposes that the inverse model realizes the desired dynamics during movements. Using the designed dynamics, the current limb state is estimated from the delayed sensory information through a recursive process. Note that a system model is not required in the proposed forward model.
The structure of the Smith predictor was used to enhance stability against feedback delays. In 1993, Miall and colleagues published a seminal study that showed the possibility that the forward model in dealing with feedback delays can be captured by the Smith predictor [35]. An estimate from the forward model is intentionally delayed to match the sensory delay and compared with the sensed state. The error between prediction and sensory feedback is then used to modulate the motor command. The forward model component in the Smith predictor, which requires a system model, is replaced with one that does not require a system model. More recently, Miall presented evidence against their Smith predictor model [58]. The study examined manual tracking tasks with a long visual feedback delay of 300 ms and found that adaptation to the delay led to reduction in tracking error and the mean power of tracking responses. However, the empirical results are against the prediction of an increase in response frequencies by the Smith predictor [58]. In the proposed model, it is assumed that the CNS keeps adjusting the values of the matrix � b to improve performance as it adapts to the task. The matrix � b has an effect on response frequencies, depending on its value. Hence, it would be possible to speculate that the proposed model is able to describe the empirical phenomena of the decrease in response frequencies in contrast to the Smith predictor. However, in this study the author focuses on the compensation of computational models for system dynamics, rather than the compensation for substantial delays. Here we do not look into the issue on whether or not the proposed model is capable of reproducing the tracking tasks with visual feedback delays.

Neurophysiological correlates
How can the proposed computation be realized in a real biological system? The cerebellum receives afferent sensory information about the limb and reafference carrying copies of the motor commands and information required for movements including a desired trajectory from the primary motor, somatosensory and parietal cortex. Efferent copies of descending motor commands could be stored in the CNS and be transmitted by motor neurons [59]. Brodmann area 5 in the parietal cortex would be thought to possess a device that generates desired trajectories composed of position, velocity and acceleration values [4,60,61]. It was demonstrated that excitations of Brodmann area 5 cells were correlated with position, velocity and acceleration. Muscle length and lengthening velocity are measured through muscle spindles and mossy fibers [62]. The measured actual limb state ascends to the cerebral cortex and is inputted into the forward model after comparison. As for the states dealt with by the forward model, mossy fibers as well as area 5 cells are thought to transmit the acceleration component in addition to the position and velocity components [63]. The parietal lobe and cerebellum appear to play a crucial role in estimating the current state [64,65].

Single-joint fast movement
The ability of the proposed control to drive fast movements was evaluated. During fast movements for which duration is comparable to the delays of feedback loops, the roles of internal models in sensorimotor processing are emphasized [2,53,66]. Even during movements in which sensory information has an influence, online corrections depending only on sensory feedback could lead to instability. Feedback control is highly sensitive to delays and should maintain the open-loop gains low at high frequencies where the delays would introduce a phase lag of 180 degrees to prevent instability. This restriction makes feedback control impossible to produce fast movements.
In the simulation of the fast movement, it is observed that the optimal control model generates a trajectory following the minimum-jerk trajectory. The proposed control successfully reproduces the minimum-jerk trajectory. This indicates that the proposed forward model in the architecture of the Smith predictor efficiently estimates the current states of the limb and feeds them back to the inverse model. While the forward model in the optimal control generates the current states by simulating behaviors of the system using a model of the dynamics with precise parameters in the CNS, the proposed forward model calculates the current states from desired dynamics. In contrast, the equilibrium-point control model that does not involve internal models fails to reproduce the minimum-jerk movement, as shown in Fig 6.

Two-joint fast movement
In the simulation of the multi-joint movement, it was investigated whether the proposed control compensates for system dynamics in multi-joint movements involving simultaneous motion at the shoulder and elbow. Interaction forces arise at one joint (e.g., the shoulder), because of motion of limb segments about other joints (e.g., the elbow), which include inertial forces from movements of other joints, centripetal torques and Coriolis torques, disturb achieving planned movements. That is, these interaction forces act as disturbances that need to be compensated. As movements become faster, interaction forces increase more. It was asserted that the cerebellum should possess a priori knowledge of the arm's dynamics to compensate for interaction forces by simulating empirical movements with two kinds of computational models [4]. One of the computational models that is not equipped with an inverse model does not show accurate reproduction of fast movements, whereas the other one with an inverse model does. The proposed control does not show a notable difference in deviation between slow and fast movements, as shown in Fig 8. This suggests that the inverse model of the proposed control efficiently carries out dynamics compensation. Note that identifying interaction forces requires an explicit system model including the inertia and length of each segment of the limb (see Eq (19)). But the proposed control does not require a model of the dynamics with precise parameters. The proposed control builds inverse models using the relationship between the motor command and its responses. Through the compensation of the system dynamics and sensory delay, the proposed control captures the human's voluntary movements, which is in agreement with a study presented in [67].

Movement with unexpected inertial changes
The main characteristic of the proposed computational model is that it estimates and cancels out the system dynamics using a copy of motor outflow and (estimated) sensory inflow. This would have the system reach the target even when the properties of the system unexpectedly change. Model-based computations including the optimal control presented in this study probably show outcomes that depend on system properties. In the case of the optimal control, it could be that the forward model component generates an estimate or the inverse one produces a command based on the system model recognized in the CNS.
To investigate whether internal models in the CNS require a precise system model or not, an experimental study of [43] was revisited that evaluated the effect of changes in the inertial property of the controlled system on movement. They revealed that participants pointed to the target position regardless of whether an inertia was added or subtracted. Their second finding is that motor commands were designed and customized according to the inertia that the participants expected; EMG signals varied with the inertia to be presented. Another similar study showed that equifinality was preserved when the inertia of the arm system was changed unexpectedly [20]. The author would say that these studies not merely suppose that motor commands are tailored to the circumstance before the movements begin, but also imply that the movements might not be programmed using a model of the dynamics with precise parameters. From the view of the internal model hypothesis, equifinality has been refuted [12,13,68,69]. Rather, equifinality is one of the characteristics supported by the equilibrium-point hypothesis that does not require a system model to calculate motor commends. However, the equilibrium-point hypothesis [18,[70][71][72], even with an assumption that it can deal with feedback delays, would be unable to explain the results by a study in [43]. Prediction by an equilibriumpoint model would be similar to that by the optimal or proposed models during the ML condition, as shown in Fig 9. When the intermediate load is replaced with the high load and moved by an equilibrium-point controller; the commend designed for the intermediate load lacks to move the high load on the planned trajectory. But, in the case that the intermediate load is replaced with the low load, the system under equilibrium-point control would follow the trajectory that is displayed for the MM condition.
It would be possible to assume that the values of the matrix � b are updated as the CNS adapts to a task. Indeed, Pinter and colleagues found that it takes 6-10 trial to adapt to a new inertial load [43]. It was also demonstrated that participants familiarized themselves with an inertial load through several trials [73]. After the customization trials, participants produce closely overlapping trajectories (perhaps the minimum-jerk trajectory) under the high and low loads (see Fig 1 in [43]). The proposed model produces the minimum-jerk trajectory if appropriate values of the gain � b are found for the system with the high or low loads. If the value is much smaller or much larger than the one suitable to move the system along the planned trajectory, the produced command is correspondingly smaller or larger than the command that enables the system to stay on the planned trajectory during the movement. The value of the gain � b impacts how accurately the system dynamics is estimated by TDE. TDE error, which is determined by the values of the matrix � b, leads to deviations from the planned trajectory; according to the value of the matrix � b, the movement can either fall short of or go beyond the planned trajectory during tracking. But, in the end, the movement approaches the planned end-point asymptotically. TDE error decreases as the movement begins to lose its momentum, diminishing the error between the planned position and actual position. Fig 10 shows the sensitivity of the parameters � b and K p (K v is arbitrarily set as 0.4K p ) on tracking error between the desired and actual trajectories during a single-joint fast movement. Overall, it turns out that tracking error is low when � b and K p are both set high, whereas tracking performance becomes worse when � b and K p are both set low. The minimum-jerk trajectory is achieved in a satisfactory manner over wide ranges of the values of � b and K p (7×10 −3 � � b and � 2×10 −2 150 � K p � 500). It is agreeable that fast movements are typically accompanied by high stiffness through muscle contractions and high stiffness would lead to Internal model without system parameters accurate tracking. Meanwhile, the analysis shows that the limb fails to accurately track the desired trajectory even with high stiffness when � b is set low (around 5×10 −3 in this case), suggesting that dynamics compensation is poorly achieved. With relatively low muscle stiffness (around 50 in this case), both a low value and a great value of � b lead to an increase in tracking error. These results perhaps originate from undercompensation and overcompensation of dynamics, while the stiffness is too low to suppress these compensation errors. Fig 11 displays the effect of the value of the estimated time delayt d in feedback loops, while the value of the actual delayt d is fixed at 65 ms. It is revealed that the proposed control model tolerates the difference in value between the actual delay and its estimate within a certain range regardless of whether or not the value of the estimate is greater than that the actual Internal model without system parameters delay. However, the system is unstable with a great difference between the two values, which is in accordance with the study presented in [35].

Role of matrix � β
The system (1) can be rewritten as where function variables are omitted for simplicity. Eq (26) can also be re-expressed with a diagonal constant matrix � b, as where H ≜ ðMðyÞ À � bÞ € y þ f: The control input (17) where n ≜ € y r þ K v ð _ y r À _ yÞ þ K p ðy r ÀŷÞ: Originally, the proposed computational model compensates for H (t) with its previous-step value H (t−dt) , but there exists a difference between H (t) and H (t−dt) . The estimation error � is defined as � ðtÞ ≜ � b À 1 ðH ðtÞ À H ðtÀ dtÞ Þ: ð30Þ This estimation error can be written in terms of the new control input ν and the angular acceleration € y through a combination of Eqs (27) and (29)  From the viewpoint of the estimation error �, the terms η, z in Eq (33) are regarded as forcing functions that are bounded in the case of a sufficiently small sampling period. If the roots of ðI À M À 1 ðy ðkÞ Þ � bÞ are all placed inside the unit circle, � is asymptotically bounded [74]. The coefficient ððI À M À 1 ðy ðkÞ Þ � bÞ, in particular, the matrix � b determines how accurately dynamics estimation is achieved; the gains affect the difference between H andĤ, which eventually affect the tracking error [75].

Optimal control
The control law of optimal control can be typically designed as where K denotes a matrix. The state feedback gain matrix K is selected in a way to minimize the following performance index: where e ≜ ½ðq d ÀqÞ T ð _ q d À _ qÞ T � T , and Q denotes an a state-weighting matrix and R denotes an input-weighting matrix. Note that these methods require a model of the dynamics with precise parameters to determine K that minimizes the index I. Estimates of the current states required for motor command are provided by a predictor that acts as the forward model in the CNS, which can be constructed as [47] where A s and B s are matrices, respectively.
The predictor estimates the current states from the sensed states, which are t d -ms delayed, based on a system model (matrices A s and B s ). The matrices A s and B s represents the limb system in state space: In the case of the forearm, the matrices A s and B s can be defined as, neglecting viscoelasticity,