Fig 1.
The EMG+FMG band used for the experiment.
We employed 8 EMG sensors and 8 FMG sensors which were housed in 3D printed housings and placed in an alternating sequence on a Velcro strap. The sensors were positioned such that they could be spaced equidistantly around the participant’s forearm before the band was strapped to the arm.
Fig 2.
The reaching positions, standing zones, and hand gestures.
(a) The reaching positions and standing zones for the experiment [42]. The subject would reach to positions 1-4 while standing at zone A and positions 5-8 while standing at zone B. The participant’s elbow was bent at 90 degrees at position 2, between 90 degrees and fully extended for positions 1, 3, and 4, and fully extended for positions 5-8. (b) The 4 hand gestures used for the experiment along with the 3D printed manipulandum. The manipulandum was made of two parts which allowed it to be top loaded and grasped with a variety of hand gestures. The weights used for the experiment consisted of a no weight condition (the weight of the manipulandum, 53 g), 250g, 500g, 750g, and 1000g.
Fig 3.
Overview of the 3 tests conducted in the experiment.
Test 1 was split into 2 analyses, as shown by the dashed line. The first analysis examined how accurately each of the sensing modalities could classify combinations of limb position, grasped load, and hand gesture. The second analysis compared classifying hand gestures in the neutral and unloaded position to varying limb positions and grasped loads. Test 2 was also split into 2 analyses and investigated different training and testing conditions, the first being a constant grasped load with varying limb positions and the second being a constant limb position with varying grasped loads. Test 3 characterized how each sensing modality was affected by large variations in limb position and grasped load by training the model in a neutral and unloaded condition and testing it in 4 other conditions.
Fig 4.
The average confusion matrix for classifying combinations of hand gesture, grasped load, and limb position for FMG.
The confusion matrix is broken down into 4 sections, one for each hand grasp (key, pinch, power, and tripod). These sections are further broken down into the 5 grasped loads (0, 250, 500, 750, and 1000g) which are subsequently broken down into 8 positions, ordered 1-8. The color bar on the side visually shows the percent of times a label is classified as another. Darker colors indicate lower percent while lighter colors correspond to a higher percent. The maximum and minimum average accuracies across the participants are included along with the standard deviation of the average classification accuracies.
Fig 5.
Average classification accuracies for each sensing modality classifying hand grasp, grasped load, and limb position combinations.
These accuracies were calculated by averaging the classification accuracies for each participant per sensing modality. Using a LME model, statistical differences were found between EMG+FMG and the two sensing modalities separately (p<0.001). Further, EMG and FMG were found to be not statistically different from one another.
Table 1.
The average classification accuracy (%), standard error, and standard deviation for each sensing modality for test 1.
Fig 6.
The average classification accuracy for each sensing modality at the neutral and unloaded position compared to varying grasped loads and limb positions.
Averages were calculated by aggregating and then averaging all participant’s classification accuracies by sensing modality. For the neutral and unloaded position, EMG was found to be statistically different from EMG+FMG. For the varying weight and position condition, all sensing modalities were found to be statistically different from one another. Further, when comparing within a sensing modality, it was found that the neutral and unloaded condition was statistically different than the varying weight and position condition. * p < 0.05, ** p < 0.01, *** p < 0.001.
Table 2.
The average accuracy, standard error, and standard deviation for comparing the classification accuracy of the 4 hand gestures at the neutral and unloaded condition to the varied weight and position condition.
Fig 7.
The spread of all participant’s gesture classification accuracy for EMG
+FMG, EMG and FMG when trained at a single position and tested at the remaining positions under a constant grasped load of 500g. For each modality, the classifier was trained in one of 8 positions, as shown by the different colors in the legend. Once trained, it was tested in the remaining 7 positions. This was done at a constant grasped load of 500g. The overall variation of classification accuracy for each sensing modality is depicted by the black overlay which also shows the median classification accuracy.
Table 3.
The median of the classification accuracies from the constant weight and varied position tests reported with the interquartile range.
Fig 8.
The spread of all participant’s gesture classification accuracy for EMG+FMG, EMG and FMG when trained at a single grasped load and tested at the remaining loads while at a constant position of position 1.
For each modality, the classifier was trained at 1 of 5 loading conditions, as shown by the different colors in the legend. Once trained, the classifier was then tested in the remaining 4 loading conditions. This was done at a constant position of position 1. The overall variation of classification accuracy for each sensing modality is depicted by the black overlay which also shows the median classification accuracy.
Table 4.
The median of the classification accuracies from the constant position and varied grasped loads tests reported with the interquartile range and the standard deviation.
Fig 9.
Average classification accuracies for each modality when training in the neutral and unloaded condition (position 2, 0g) and testing in the 4 most extreme conditions for each sensing modality: Neutral (position 2, 0g), Loaded (position 2, 1000g), Outstretched (position 5, 0g), and Outstretched and Loaded (position 5, 1000g).
For each sensing modality, participant data was averaged to portray average classification accuracies over all participants. Fig 9a depicts how the neutral condition was statistically different from the other 3 conditions for each sensing modality (P < 0.001). Across sensing modalities for each condition, it was found that EMG and EMG+FMG were statistically different at the neutral condition and FMG and EMG were statistically different at the outstretched condition (P < 0.05). Fig 9b–9d compare across conditions for the same sensing modality (from right to left: FMG, EMG, EMG+FMG). It was found for all sensing modalities that the neutral condition was statistically different from all other conditions (P < 0.001). For the other conditions, it was found that loaded – outstretched, loaded – outstretched and loaded, and outstretched – outstretched and loaded were significantly different for FMG and EMG+FMG. For EMG, the loaded condition was significantly different from the others. * p < 0.05, ** p < 0.01, *** p < 0.001.
Table 5.
The average accuracy, standard error, and standard deviation for each of the sensing modalities when trained at the neutral condition and tested in the conditions listed.