Fig 1.
The computational complexity of human grasp selection.
(a) Possible (b) Impossible (c) Possible but uncomfortable or unstable grasps.
Fig 2.
Setup and stimuli for Experiments 1 and 2.
(a) Experimental setup. Seated participants performed grasping movements with their right hand. Following an auditory signal (coinciding with the shutter window turning transparent) they moved from one of the starting positions to the object and grasped it with a precision grip. They transported and released the object at the goal position and returned to the start position. (b) In Experiment 1 we employed four objects made of wooden cubes. Each object had a unique shape (that here we name L, U, S, V) and was presented at one of two different orientations with respect to the participant. (c) In Experiment 2 the objects had the same shapes as in Experiment 1, but now were made of wood and brass cubes. The brass and wood cubes were organized either in an alternate pattern (middle), so that the CoM of the object would remain approximately the same as for the wooden object, or grouped so that the CoM would be shifted either closer to (right) or away from (left) the participant’s hand starting location.
Fig 3.
(a) Human grasps from Experiment 1. Grasps are represented as thumb (red triangles) and index finger (blue diamonds) contact positions, connected by dotted black lines. (b) Human grasps (blue blobs) for object U, orientation 2, when projected in a unitless 2D representation of the space of potential grasps, cluster around three distinct modes. (c) Distribution of thumb contact points on object L, orientation 2. Different colors represent grasps from different participants. (d) The level (%) of grasp similarity expected for grasps randomly distributed on the object surface (i.e. random combinations of thumb and index finger positioning attemptable on an object) and the observed level of between- and within-participant grasp similarity, averaged across objects and orientations. Error bars are 95% bootstrapped confidence intervals of the mean. ** p<0.01, *** p<0.001.
Fig 4.
(a) Difference in grasp similarity across orientations when grasps were encoded in object-centered (allocentric) vs human-centered (egocentric) coordinates, as a function of magnitude of rotation across the two orientation conditions. (b) Average grasp trajectories viewed in the x-y plane (red curves) from start location towards the objects (always contained within the gray shaded region). The average human grasp (red dot) across conditions is biased toward shorter reaching movements compared to the object centroids (black dot). In both panels data are means, error bars/regions represent 95% bootstrapped confidence intervals. *** p<0.001.
Fig 5.
(a) Attraction towards the object CoM for grasps executed onto light (Experiment 1) and heavy (Experiment 2) objects compared to the average CoM distance of grasps uniformly distributed on the object surfaces (zero reference). (b) Attraction towards the object CoM in Experiment 2 as a function of trial repetition. Red line is the best-fitting regression line through the data (c) Human grasps from Experiment 2 onto object S presented at orientation 2. (d) Attraction towards the object CoM compared to Experiment 1 grasps (zero reference), for Experiment 2 grasps onto heavy objects whose CoM is closer, the same distance as, or farther than the light wooden objects from Experiment 1. In panels a, b, and d, data are means, error bars represent 95% bootstrapped confidence intervals. ** p<0.01, *** p<0.001.
Fig 6.
A framework that unifies distinct aspects of grasp selection.
(a) Mesh model of object in same 3D reference frame as participant poised to execute grasp. (b) Discrete sampling of the reachable surface defines a 2D space containing all potential combinations of index and thumb contact points on the object. (c) Color-coded maps showing penalty values for each potential grasp for each penalty function. (d) Overall penalty function computed as the linear combination of maps in (c). (e) Human grasps projected into 2D penalty-function space neatly align with minimum of combined penalty map.
Fig 7.
Examples of grasps with (a) low penalty and (b) high penalty force closure.
Fig 8.
Examples of grasps with (a) low penalty and (b) high penalty torque.
Fig 9.
Examples of grasps with (a) low penalty and (b) high penalty grasp axis.
Fig 10.
Examples of grasps with (a) low penalty and (b) high penalty aperture.
Fig 11.
Examples of grasps with (a) low penalty and (b) high penalty visibility.
Fig 12.
(a) Grasping patterns reconstructed through the normative framework (right) closely resemble human grasps onto real objects varying in shape, orientation, and material (left). Simulated grasp patterns are generated with no knowledge of our human data (i.e. model not fit to human grasps). (b) Population level grasp similarity, i.e. similarity of human and unfitted model grasps to medoid human grasp across all participants. (c) Individual level grasp similarity, i.e. similarity of human, unfitted, and fitted model grasps to the medoid grasp of each participant. In panels (b, c), dashed line is estimated chance level of grasp similarity due to object geometry, bounded by 95% bootstrapped confidence intervals. (d) Pattern of fitted weights across Experiments 1 and 2. (e) Relative weight of the minimum torque constraint in Experiments 1 and 2. (f) Relative weight of the visibility constraint in Experiments 1 and 2. Data are means; error bars, 95% bootstrapped confidence intervals. ***p<0.001.
Fig 13.
Model predictions for novel objects align with human grasps.
(a) Grasping patterns predicted through the normative framework for novel objects with smooth and curved surface geometry. (b) Human grasps onto 3D printed versions of the objects align with model predictions. (c) Similarity of human and predicted model grasps to medoid human grasp across objects and participants. Dashed line is estimated chance level of grasp similarity, bounded by 95% bootstrapped confidence intervals.
Fig 14.
All panels show model performance (in terms of grasp similarity to human data from Experiment 3) as a function of different perturbations. Grasp similarity for the original model implementation is shown in green. Red and black dashed lines are respectively human and chance levels of grasp similarity, bounded by 95% bootstrapped confidence intervals. (a) Model grasp similarity with input meshes subsampled by varying degrees. (b) Model grasp similarity for model implementations employing increasing aperture thresholds. (c, d) Model grasp similarity for models implemented with deviated natural grasp axis along the transverse (c) and sagittal (s) planes.
Fig 15.
Setup and stimuli for Experiment 3.
(a) Experimental setup. Seated participants performed grasping movements with their right hand. Following an auditory signal, they opened their eyes, and moved from the starting position to the object and grasped it with a precision grip. They transported and released the object at the goal position and returned to the start position. (b) We employed four 3D-printed objects. Two objects had an abstract shape (that here we name ‘swan’ and ‘blob’), the other two objects were printed versions of a croissant and a cat. They were presented to the participant in the orientations displayed in here.