Conceived and designed the experiments: BB. Performed the experiments: BB EC. Analyzed the data: BB. Contributed reagents/materials/analysis tools: BB FN. Wrote the paper: BB EC FN TP.
The authors have declared that no competing interests exist.
An important issue in motor control is understanding the basic principles underlying the accomplishment of natural movements. According to optimal control theory, the problem can be stated in these terms: what cost function do we optimize to coordinate the many more degrees of freedom than necessary to fulfill a specific motor goal? This question has not received a final answer yet, since what is optimized partly depends on the requirements of the task. Many cost functions were proposed in the past, and most of them were found to be in agreement with experimental data. Therefore, the actual principles on which the brain relies to achieve a certain motor behavior are still unclear. Existing results might suggest that movements are not the results of the minimization of single but rather of composite cost functions. In order to better clarify this last point, we consider an innovative experimental paradigm characterized by arm reaching with target redundancy. Within this framework, we make use of an inverse optimal control technique to automatically infer the (combination of) optimality criteria that best fit the experimental data. Results show that the subjects exhibited a consistent behavior during each experimental condition, even though the target point was not prescribed in advance. Inverse and direct optimal control together reveal that the average arm trajectories were best replicated when optimizing the combination of two cost functions, nominally a mix between the absolute work of torques and the integrated squared joint acceleration. Our results thus support the cost combination hypothesis and demonstrate that the recorded movements were closely linked to the combination of two complementary functions related to mechanical energy expenditure and jointlevel smoothness.
To reach an object, the brain has to select among a set of possible arm trajectories that displace the hand from an initial to a final desired position. Because of the intrinsic redundancy characterizing the human arm, the number of admissible joint trajectories toward the goal is generally infinite. However, many studies have demonstrated that the range of actual trajectories can be limited to those that result from the fulfillment of some optimal rules. Various cost functions were shown to be relevant in the literature. A peculiar aspect of most of these costs is that each one of them aims at optimizing one specific feature of the movement. The necessary motor flexibility of everyday life, however, might rely on the combination of such cost functions rather than on a single one. Testing this cost combination hypothesis has never been attempted. To this aim we propose a reaching task involving target redundancy to facilitate the comparisons of different candidate costs and to identify the bestfitting one (possibly composite). Using a numerical inverse optimal control method, we show that most participants produced movements corresponding to a strict combination of two subjective costs linked to the mechanical energy consumption and the jointlevel smoothness.
Numerous experimental studies have demonstrated that biological motion exhibits invariant features, i.e. parameters that do not significantly change with movement size, speed, load and direction
The first one is methodological: in many cases, models based on divergent assumptions and minimizing different costs can yield similar arm trajectories
In order to test this cost combination hypothesis, our approach was twofold. First, we wanted to stress the differences between the predictions of different classical models already existing in literature. To this aim, we designed a pointing task with target redundancy. Precisely, we reduced the external constraints of the task by asking subjects to reach to a vertical bar. Thus no accuracy requirement was present in the vertical axis, which had the interesting advantage of discriminating better between different cost functions than during classical pointtopoint experiments (see
A. Simulated hand paths for pointtopoint movements in the horizontal plane. Targets (T1 to T6) were located approximately as in
The experimental results show that participants adopted a consistent behavior although the final point was not imposed by the experimenter. Inverse optimal control reveals that their average behavior mainly relied on a composite cost function, combining the minimization of mechanical energy expenditure (here the absolute work of torques) with the maximization of joint smoothness (here the integrated squared acceleration). Further analyses demonstrate that this mixofcost model replicated the most important features of arm movements and performed better than any other single cost function on which our method was based. Results provided therefore support the cost combination hypothesis and, in particular for this task, emphasize two complementary and subjective costs.
Twenty naive subjects (16 males, and,
The motor task that we considered is illustrated in
The reachable region from the sitting position is emphasized on the bar. The 5 initial postures under consideration are also shown (P1 to P5). B. Experimental trajectories for a representative subject. Dotted lines depict the initial arm posture of the subject (upper arm and forearm). The average fingertip path is shown in thick black line for each initial posture, from P1 to P5. The 20 trials are depicted in thin gray lines for every initial postures. C. Experimental angular displacements and finger velocity profiles for the most typical subject. First column: joint displacements at the shoulder and elbow joints; Second column: Finger velocity profiles with shaded areas indicating the standard deviation. Time is normalized, not amplitude.
The initial references were positioned using a wooden hollow frame containing 1.5 cmspaced thin vertical fishing wires to which lead weights (small spheres) indicating the requested fingertip initial position were attached. Differently colored pieces of scotchtapes were stuck on the leads to easily identify the references. This colorcode was then used to verbally specify the initial posture that the subject had to select at the beginning of each movement. By imposing the initial finger position, a unique starting posture of the arm was thus defined in the parasagittal plane. The positions of the leads were adjusted before the experiment, based on the subject's upper arm and forearm lengths and the vertical distance shoulderground.
The experimenter then gave the following instruction to the participants: look at the bar in front of you, close the eyes and quickly show the location of the bar by touching it with the fingertip, performing a oneshot movement. No instruction was given to the subjects with respect to where and how to reach the bar. Because of the features of the task itself, participants had to implicitly control the finger position along the anteroposterior and lateral directions whereas full freedom was left along the vertical one. Note that the challenge for the subjects (i.e. the objective reward of the task) was to be precise enough to actually touch the bar, since no online vision was allowed. Since subjects were free to moved in 3D, touching the bar was not so easy because of the presence lateral and anteroposterior errors and the absence of online visual feedback. Nevertheless, it is worth noting that reaching any point on the vertical bar allowed the subject to perform the task successfully. During the protocol, the five initial postures were tested in a random order. For each initial posture, twenty trials were recorded, so that a total of 100 movements per subject were monitored. A few trials were repeated during the experiment (less than 5%), when the subjects clearly missed the bar or did not perform a oneshot movement. Every set of 25 movements, subjects were allowed to rest. Data from a total of 2000 pointing movements were collected for this reachingtoabar task.
Arm and head motion were recorded by means of a motion capture system (Vicon, Oxford, UK). Ten cameras were used to capture the movement of six retro reflective markers (15 mm in diameter), placed at welldefined anatomical locations on the right arm and head (acromial process, humeral lateral condyle, ulnar styloid process, apex of the index finger, external cantus of the eye, and auditory meatus).
All the analyses were performed with custom software written in Matlab (Mathworks, Natick, MA) from the recorded threedimensional position of the six markers (sampling frequency, 100 Hz). Recorded signals were lowpass filtered using a digital fifthorder Butterworth filter at a cutoff frequency of 10 Hz (Matlab
The temporal finger movement onset was defined as the instant at which the linear tangential velocity of the fingertip exceeded 5% of its peak and the end of movement as the point at which the same velocity dropped below the 5% threshold. All time series were normalized to 200 points by using Matlab routines of interpolation (Matlab
For subsequent analyses and comparisons with models, we projected the 3D coordinates of the markers onto a vertical plane. It will be shown thereafter that the movements carried out by the participants almost lay on a parasagittal plane. The motion capture system was calibrated such that the axes
Angular displacements of the arm segments (upper arm and forearm) were then evaluated using the inverse kinematic function, relating the
The shoulder joint was defined as the origin of the frame of reference (see
The extrinsic and intrinsic coordinates are denoted by
Finally, additional taskrelevant parameters were computed. The
Moreover, to assess whether the finger path had a convex or concave curvature, we computed the signed Index of Path Curvature (sIPC). This was defined as the averaged ratio between the maximum path deviation from a segment connecting the initialfinal finger positions and the length of this segment, attributing a positive sign when the finger position was above the straight line (for concavity). Thus, this parameter evaluates the average or global convexity or concavity of a hand path. In addition, joint coupling was calculated as the determination coefficients between the shoulder and elbow angular displacements. In order to compare models predictions and measured data, we computed the
We used quantilequantile plots to visually check that the data were normally distributed (
Previous models of optimal control for arm movements were originally designed by their respective authors on the basis of some particular assumptions and restrictions. In order to compare several different costs proposed in the literature and to apply the inverse optimal control technique described thereafter, we consider a homogeneous framework, compatible with most existing models. The next subsections describe the musculoskeletal model, the inverse and direct optimal control techniques that we employed. Details are deferred to the supplementary documents
It will be shown that the recorded 3D arm movements approximately lied on the parasagittal plane. Thus, a reasonable approximation for modeling is to consider the arm as a twojoint rigid body moving in the vertical plane. A classical application of Lagrangian mechanics allows us to express the arm dynamics using the general form
Furthermore, we modeled the fact that the joint torques
The control variable
It is noteworthy that, for considering several costs within the same framework, we did not model neither agonist and antagonist muscles, nor the complex mechanism of muscle contraction. Nevertheless, additional verifications (through direct optimal control) suggested that the results presented in this study do not critically depend on this choice (see
The goal of inverse optimal control is to automatically infer the cost function from observed trajectories that are assumed to be optimal. Thus, in inverse optimal control problems (inverse OCPs), the optimal solution is known and the objective is to recover the performance criterion which has been optimized. Addressing the motor planning problem in this way is generally more difficult than using the more standard direct optimal control approach, which consists of guessing a plausible cost and comparing its predictions with the experimental data. However, inverse OCP is better suited to provide, with less a priori, the cost or mix of costs that must be optimized to replicate the measured arm trajectories. In this paragraph, we present a numerical method for solving an inverse OCP, which was initially described by
The method relies on the selection of a set of plausible costs. For the optimal control of arm movements, several costs were already proposed in the literature. The models generally fall into four general classes, each of which making different assumptions on the relevant variables for the CNS. First, there are the
Criterion  Cost function ( 
References 
Hand jerk 


Angle jerk 


Angle acceleration 


Torque change 


Torque 


Geodesic 


Energy 


Effort 

Therefore, we selected the following costs for further investigation: hand smoothness (Cartesian jerk), joint smoothness (angular acceleration and angle jerk models), torque change, torque, geodesic, mechanical energy, and neural effort (each of which denoted by
The parameter
Thus, the OCP corresponding to the cost
Let us denote this problem by
Let us now denote by
The outer loop (also referred to as “upper level” by some authors) consists in solving an optimization problem for the unknown
How to define a good “metric” in the space of trajectories is still an open question in motor control
Solving the bilevel problem is not straightforward for several reasons. First, the objective function
To improve the algorithm efficiency, we found useful to appropriately scale the step size along each dimension of the search space. We used a rescaling vector,
As explained above, the inner loop of the bilevel problem requires solving direct OCPs for a given
Apart from the inverse approach, a verification was also conducted by directly analyzing the predictions of each single cost model (defined by
The behavior of a representative subject is illustrated in
P1  P2  P3  P4  P5  
Movement duration (s) 





Mean velocity (m/s) 





Time to Peak velocity 





Vpeak/Vmean 





Curvilinear distance (m) 





Constant error on 





Subjects could reach wherever they desired on the bar (i.e. on the vertical axis). Therefore, it appeared important to verify whether their behavior was consistent or not. An analysis of the consistency index (CI, a parameter similar to a normalized
A.
It has to be noted that among all the tested subjects, only two behaved quite atypically. One of them exhibited a highly variable behavior, exploring the whole bar across trials. The second one started to increase drastically his trialtotrial variability during the second half of the experiment while being invariant in the first half. This kind of behavior can be considered as quite marginal since it appeared for only 2 of our 20 participants, and reflected uncommon motivations/intentions.
The average behavior is illustrated in
Finally, we also conducted an analysis on the movement vector angle (see
A visual inspection of the shape of paths showed that they were generally curved in the
Angular displacements were generally monotonic for all subjects and conditions, except for instance for posture P4 at the elbow joint (see
The finger velocity profiles were always bellshaped, meaning that movements were oneshot without terminal adjustments (that is they showed unique acceleration and deceleration phases, as depicted in
By means of inverse optimal control, we could identify the cost or mix of cost that best accounted for the experimental data.
A. Weighting coefficients, i.e., elements of the vector
Subject  S1  S2  S3  S4  S5  S6  S7  S8  S9  S10 
Mean error (cm)  4.1  3.9  1.8  6.5  3.3  3.5  4.6  3.9  3.4  2.9 
Subject  S11  S12  S13  S14  S15  S16  S17  S18  S19  S20 
Mean error (cm)  3.2  6.1  2.5  4.6  2.5  1.7  2.7  6.2  3.6  1.8 
Similar results were obtained for several subjects, despite the differences in their movement durations and anthropometric parameters. The bestfitting weighting vectors
A. Weighting coefficients, i.e. elements of the vector
Taken together, the above results provide clues on which costs must be considered to capture the basic characteristics of human movements during the reachingtoabar task. The majority of subjects (15/20) clearly adopted a behavior optimizing a wellcharacterized hybrid cost, essentially mixing the absolute work and the angular acceleration (i.e., the other costs are somehow residual). Consequently, for the further investigations using direct optimal control, we included this identified composite cost to compare it with the basis costs. Since the ratio between the weighting coefficients of energy and angle acceleration was roughly 10∶1, the hybrid cost was chosen to be
A preliminary inspection of
A. Typical experimental data in order to facilitate comparisons (already depicted in
To quantify the matching between models and real data an analysis of the finger path was conducted, including all subjects and all initial postures. The difference between simulated and measured paths was first measured through the area
This parameter qualifies as a general error measure. Values were first averaged across initial postures for each participant, and then, the mean and standard deviation were finally reported across participants. It is apparent that the energy and angle jerk/acceleration models performed quite well (with a lower standard deviation for the energy model), while the geodesic and hand jerk models performed moderately. The worst models were the torque change, effort and torque models, given in decreasing order of performance. The best model was the hybrid model, in agreement with the results provided by the inverse optimal control approach.
A specific analysis of taskrelevant parameters was also performed (see
A and B depict the reached point (RP) and movement vector (MV) parameters, which are the relevant parameters for the finger path. An analysis confirms that energy and angle jerk models, as well as the hybrid model, are quite efficient in predicting the terminal point on the bar and the movement direction (upward or downward). C and D depict the signed index of path curvature (sIPC) and joint coupling (
Concerning the shape of the path (sIPC parameter,
The joint coupling analysis (
We also checked that the hybrid model predicted plausible angular displacements and finger velocity profiles.
A. Angular displacements at the shoulder and elbow joints. B. Finger velocity profiles. In both graphs, solid lines correspond to the experimental data, which are recalled from
Finally, the observed movement variability shows that the behavior of subjects was in fact approximately optimal on a trialtotrial basis.
Above all, the modeling analysis showed that the hybrid model, maximizing jointlevel smoothness and minimizing mechanical energy expenditure, accounted well for many spatial and temporal features of the observed behaviors, and much better than single cost models (and any other linear cost combination from the inverse optimal control analysis).
In this study we investigated the cost combination hypothesis for the optimal control of arm movements. To this aim we adopted an inverse optimal control methodology to identify the cost function that best replicates the participants' behavior during a task with target redundancy. Inverse optimal control revealed that the observed hand paths were close to the solutions of an optimal control problem relying on a composite cost function mixing mechanical energy expenditure and joint smoothness. This hybrid cost was found to fit well the experimental data, not only much better than any single other cost under comparison, but also better than any other linear combination of the candidate costs.
Reaching to objects involving target redundancy is a very common task in everyday life. For instance, grasping a small ball can be achieved through many taskequivalent solutions, depending on how one chooses to put his fingers on it. In such a case, like for the bar, target point discriminability is greatly reduced and, therefore, decision confidence in the brain decreases
Certain limitations however remain such as the uniqueness of the solution and the problem of local minima, which are hardly avoidable in the context of complex nonlinear optimal control. Uniqueness of the solution has been addressed recently in static inverse optimization
Inverse optimal control results showed that most subjects (15/20) adopted a behavior which essentially corresponded to a strict mixture of two subjective costs (absolute work of torques and angular acceleration energy). More precisely, mixing these two costs was found to fit better the observed hand paths than other linear combinations of the eight candidate costs we considered. Each subject could use a different weighting of those two costs but on average their contribution to the total movement cost was roughly the same (about 40% of the total movement cost). These findings were quite robust as confirmed by the results when using an alternative metric (
Further evidence for mixing energy and smoothness optimality criteria was provided by the direct optimal control analysis. The bar reaching experiment revealed that several previously proposed costs did not generalize well to the present task. In general, it was relatively easy to discriminate between different models. Clearly, the most discrepant model was the minimum torque model, which assumes that the total amount of (squared) torques needed to drive the movement has to reach a minimum. This model was mainly influenced by the maximum exploitation of gravity to reach the bar. The minimum torque change model, which maximizes smoothness in the dynamic space, also predicted nonbiological paths since even the movement direction was poorly predicted in most cases. Similarly, the minimum effort model, optimizing the amount of neural input to control the movement, was unable to predict some basic features of the recorded arm trajectories. Other simulations showed that neither modeling agonist/antagonist muscles as lowpass filters nor separating the control of static (gravitational) and dynamic forces (speedrelated) could improve drastically the model predictions for this task (large errors on the movement directions were still clear, see
Maximizing smoothness at the level of the hand was also found to be generally irrelevant with respect to the geometry of the paths. The minimum hand jerk model predicted to follow the shortest Euclidean path to reach the bar. It is worth mentioning that this model had been validated originally for horizontal movements performed with a robotic device
It is interesting to note that the geodesic model had been initially validated for unconstrained 3D pointtopoint movements
It is undeniable that a theory of motor planning assuming that the CNS is able to combine different objectives depending on the task would be very powerful for explaining almost every experimental fact and could be unfalsifiable
The fact that, in this study, energy and smoothness were jointly optimized in roughly similar proportions further supports the relevance of combining subjective costs: minimizing only energy may be detrimental to smoothness and viceversa. The complementarity of cost functions has been rarely discussed in the motor control literature, even though it constitutes the main motivation for mixing different goals in the same motor plan. For energy and smoothness, the complementarity is evident. However, other costs turn out to be more correlated, in the sense that minimizing the one can imply a decrease of the other. For example, minimizing the amount of motor command (effort cost) may result in a “not so large” torque change cost. Due to nonlinearities, it is nevertheless difficult to establish general rules. In the same vein, the similarity of joint acceleration and jerk costs is the reason why, in this study, we only conclude about the optimization of a quite generic “joint smoothness” term. In general, objective costs are also optimized for the task achievement per se and are thus complementary to subjective costs. Consider for instance the task of drawing a straight line on a sheet on paper. In this case, optimizing the jerk at hand would be the best solution to produce such a path. Energy and joint smoothness costs could however be integrated in the motor plan to determine the remaining degrees of freedom (i.e. joint angles, muscles activities...). Conversely, when trying to jump at a maximal height, it is likely that the weight given to the energy cost is decreased. Joint smoothness should instead remain still present to avoid injuries and fulfill goal achievement. We propose therefore that planning is a dynamic process weighting flexible objective costs (e.g. pointing accuracy, path tracking, viapoint etc.) with more deeply anchored subjective costs. This combination of cost would crucially yield the necessary flexibility for the sensorimotor system to achieve a variety of tasks, which agrees with other recent results obtained in the stochastic optimal control context
We still remain ignorant about the detailed neural mechanisms underlying such flexible combinations of cost functions. We may suggest however that subjective cost functions are encoded at a low level of the CNS, while objective cost functions are determined at a higher level. Autonomic motor system that control basic involuntary function through the sympathic system dealing with body's resources might regulate the selection and combination of costs. In other words, we speculate that hypothalamus, reticular formation and spinal cord, which ensure the regulation of internal body states contributing to overall physiological balance, would control the optimization process, however remaining under the influence of descending pathways. Such a hierarchical view of motor planning and control is reminiscent of the theory proposed in
General settings of the optimal control problems and details about their solutions.
(PDF)
Materials to verify and support the results described in the main text.
(PDF)
We thank Marco Jacono for technical assistance; Ioannis Delis and Christian Darlot for useful suggestions and comments.