Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Upper Extremity Functional Evaluation by Fugl-Meyer Assessment Scoring Using Depth-Sensing Camera in Hemiplegic Stroke Patients

  • Won-Seok Kim ,

    Contributed equally to this work with: Won-Seok Kim, Sungmin Cho

    Affiliation Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, South Korea

  • Sungmin Cho ,

    Contributed equally to this work with: Won-Seok Kim, Sungmin Cho

    Affiliation Department of Mechanical and Aerospace Engineering, Seoul National University, Seoul, South Korea

  • Dongyoub Baek,

    Affiliation Department of Mechanical and Aerospace Engineering, Seoul National University, Seoul, South Korea

  • Hyunwoo Bang,

    Affiliation Department of Mechanical and Aerospace Engineering, Seoul National University, Seoul, South Korea

  • Nam-Jong Paik

    njpaik@snu.ac.kr

    Affiliation Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, South Korea

Upper Extremity Functional Evaluation by Fugl-Meyer Assessment Scoring Using Depth-Sensing Camera in Hemiplegic Stroke Patients

  • Won-Seok Kim, 
  • Sungmin Cho, 
  • Dongyoub Baek, 
  • Hyunwoo Bang, 
  • Nam-Jong Paik
PLOS
x

Abstract

Virtual home-based rehabilitation is an emerging area in stroke rehabilitation. Functional assessment tools are essential to monitor recovery and provide current function-based rehabilitation. We developed the Fugl-Meyer Assessment (FMA) tool using Kinect (Microsoft, USA) and validated it for hemiplegic stroke patients. Forty-one patients with hemiplegic stroke were enrolled. Thirteen of 33 items were selected for upper extremity motor FMA. One occupational therapist assessed the motor FMA while recording upper extremity motion with Kinect. FMA score was calculated using principal component analysis and artificial neural network learning from the saved motion data. The degree of jerky motion was also transformed to jerky scores. Prediction accuracy for each of the 13 items and correlations between real FMA scores and scores using Kinect were analyzed. Prediction accuracies ranged from 65% to 87% in each item and exceeded 70% for 9 items. Correlations were high for the summed score for the 13 items between real FMA scores and scores obtained using Kinect (Pearson’s correlation coefficient = 0.873, P<0.0001) and those between total upper extremity scores (66 in full score) and scores using Kinect (26 in full score) (Pearson’s correlation coefficient = 0.799, P<0.0001). Log transformed jerky scores were significantly higher in the hemiplegic side (1.81 ± 0.76) compared to non-hemiplegic side (1.21 ± 0.43) and showed significant negative correlations with Brunnstrom stage (3 to 6; Spearman correlation coefficient = -0.387, P = 0.046). FMA using Kinect is a valid way to assess upper extremity function and can provide additional results for movement quality in stroke patients. This may be useful in the setting of unsupervised home-based rehabilitation.

Introduction

Stroke is a leading cause of disabilities worldwide[1] and hemiplegia is the most common impairment after stroke, [2] resulting in upper extremity (UE) dysfunction. UE impairment is associated with limitation of activities and worse health-related quality of life.[3] Because recovery of UE impairment is marked in the first 6 to 12 months after stroke onset and can continue slowly up to one year,[4, 5] optimal rehabilitation is recommended for maximal recovery during this period, even after home discharge. However, only about 30% of stroke survivors in the United States receive outpatient stroke rehabilitation,[6] which is lower than the expected percentage considering the clinical practice guideline. This may be associated with barriers including costs, travel and limited use of public transportation due to disabilities.[6] Furthermore, adequate rehabilitation facilities are limited in developing countries and use of outpatient rehabilitation facilities are likely very low.[7]

A home-based virtual rehabilitation system could be a useful alternative for conventional rehabilitation to overcome barriers for outpatient rehabilitation in stroke patients, considering its low cost and greater accessibility.[7] Among many components in home-based rehabilitation systems, objective functional assessment is important to plan current function-based rehabilitation and monitor recovery as well as to motivate patients. Tele-based assessments by therapists using video are possible but may necessitate scheduling an appointment with the therapist and would involve additional cost. Some scores during virtual gaming can be used for assessment, but these are not intuitive, are typically not familiar to therapists.[8, 9]

The Fugl-Meyer Assessment (FMA) is a comprehensive measurement tool for motor function after stroke. FMA is valid, reliable and responsive to change.[10] Among the five FMA domains, the motor domain is most widely used and has the primary value of monitoring motor recovery after stroke. Most items in the UE motor domain are based on patient motion, although reflex or resistance has to be measured in a few items.[11] A depth-sensing camera, such as Kinect (Microsoft, USA), can detect joint movement three-dimensionally[12] and may be used to predict the FMA score. In one previous study, scores calculated from motion data captured by Kinect correlated well with the motor scores in chronic stroke patients.[13] In addition, real-time joint tracking with depth-sensing cameras makes motion analysis possible[1416] and can provide quantitative results for movement quality, which is difficult using the manual assessment tool.[14, 15] Kinect is a relatively cheap sensor that needs no additional hardware, except for a computer, to acquire motion data for FMA.

The primary objectives of this study were to investigate whether Kinect motion data could be used to predict FMA score and whether predicted scores correlated with those measured by an experienced therapist in hemiplegic stroke patients. Secondarily, the usefulness of Kinect movement quality analysis was investigated.

Materials and Methods

Subjects

Patients were recruited from December 2013 to February 2015. Patients were eligible for inclusion if they had unilateral hemiplegia caused by ischemic or hemorrhagic stroke. Patients were excluded if they were younger than 18 years of age; had serious medical complications requiring intensive care, such as pneumonia, urinary tract infection, acute coronary syndrome, inability to provide written informed consent and any other conditions that might interfere with participation. All subjects received detailed information about the study and provided written consent. The individual in Fig 1 provided written informed consent (as outlined in the PLOS consent form) to publish the picture. This research protocol was approved by the Seoul National University Bundang Hospital institutional review board and was conducted in accordance with the regulatory standards of Good Clinical Practice and the Declaration of Helsinki (World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects, 2000).

thumbnail
Fig 1. Motion data recording program.

The recording program includes subjects' abbreviation, recording arm side, assessment item number. When pushing the record button and starting an item of Fugl-Meyer assessment, upper extremity skeleton of a subject can be shown in the monitor.

https://doi.org/10.1371/journal.pone.0158640.g001

Experimental design

Subjects were seated comfortably in a chair to test UE FMA. Among the 33 items for UE evaluation, 13 were selected for Kinect motion data recording: flexor synergy (shoulder retraction, shoulder elevation, shoulder abduction, shoulder external rotation, elbow flexion, forearm supination), extensor synergy (shoulder adduction and internal rotation, elbow extension, forearm pronation), volitional motion mixing dynamic flexor and extensor synergy (hand to lumbar spine, shoulder flexion 0° to 90°) and volitional movement with little or no synergy dependence (shoulder abduction 0° to 90°, shoulder flexion 90° to 180°). One occupational therapist with two-year experience in the FMA test did the evaluations. Subject motion was recorded simultaneously by Kinect for all 13 items. Kinect motion data were saved as a separate file, which is upper-limb joint data including time. The saved data and FMA scores were transferred to an engineering department for analysis.

System for recording motion data with Kinect

The Kinect depth-sensing camera was operated with a frame-rate of 30Hz and was positioned in front of each subject to track the entire arm during FMA motions. Before the motion was recorded, the therapist entered subject information including recording arm side and the recording assessment item number into the recoding program. The therapist activated the recording function using a recording button in the Graphical User Interface (GUI) panel (Fig 1) after instructing the subject concerning the assessment item to capture the subject’s response. Data were stored sequentially with time for the UE joint positions comprising 31 variables including time, and positions of the head, shoulder center, shoulder, elbow, wrist and hand. Data were saved in text file format.

Data extraction and normalization of Kinect motion data

The recorded joint movement data from each FMA assessment were extracted. For the left arm, as an example, left hand, left wrist, left elbow, left shoulder, shoulder center and head joint position data were extracted. To match the coordinates of both arms for machine learning, the right side data was mirrored to the left side based on the sagittal plane of the subject. Then data from both arms could be put into the learning system together. Data recorded at the start and end of each motion were clipped by thresholding of the joint distance between frames. The detailed clipping process is described in the S1 Appendix. To remove the differences of seating locations and to normalize body size, all joint data were transformed by minus of initial shoulder center and by dividing the summation of each body length (i.e. wrist-elbow, elbow-shoulder and shoulder-shoulder center).

FMA scoring based on pattern recognition from Kinect data

To predict a FMA score for each assessment item, an artificial neural network (ANN) among various pattern recognition algorithms was adopted. The prediction target of each item score (0, 1 or 2) was evaluated by one therapist. In machine learning and cognitive science, ANNs are statistical learning models inspired by biological neural networks that have become popular in solving various problems in diverse fields. In particular, it has been adopted to solve motion recognition problems in computer vision.[17, 18]

To properly classify motion patterns, features must be extracted from the captured motion data, which contains the positional information of every upper limb joint. Angles and distances between two joints (for example, hand-shoulder, hand-head and elbow-head) are computed from the original position data. Normalized jerky data based on jerky motion analysis is also used as an additional feature. In particular, bounding area and variance data for each feature are also used because the range of the motion increases as the FMA score increases.

The extracted features from motion captured data and the corresponding FMA scores that were evaluated by one therapist were used to train the ANN model. Predicting a score for each assessment depends on different features. Dimensionality reduction using principal component analysis (PCA) was performed to distinguish major features from all existing features. The original feature dimension was about 100 with slight variation from item to item. Reduction to between four and 10 dimensions was done for highly associated principle components. Therefore, different numbers of principal components were used to achieve the best accuracy for each assessment item. Dimensionality reduction is explained in more detail, in the S2 Appendix.

Thirteen assessments (26 scores in total) among all UE FMA determinations were predicted. An identical ANN structure (i.e. number of neurons, number of hidden layers and activation functions) was applied to predict a score for all assessments. However, a different number of principal components were selected for each assessment after PCA dimensionality reduction. Thus, the dimensions of input data depended on the assessments.

Cross validation of FMA prediction models

Experimental data for each assessment were collected from 41 subjects. As both normal side and hemiplegic side data were collected for each subject, 82 motion data captures in total were used to train the ANN model. However, the collected score data displayed a skewed distribution for some assessments. Thus, it was not reliable for the validation to merely divide the collected data into training and testing data.

Using conventional validation, such as fixed partitioning the data set, the error of the training set is not a useful estimator of model performance and the error of the test data is not reliable in various testing data sets. Therefore, to reduce variability, multiple rounds of cross-validation were performed using different partitions. The validation results are averaged over the rounds and derive a more accurate estimate of model prediction performance. In this manner, 8- to 10-fold cross validations for each FMA item were performed. The k-fold means that the sample is randomly partitioned into k subsamples. One of the subsamples constitutes testing data and others are training data. Our cross-validation average error is shown in the prediction accuracy result (Fig 2). The overall process of this cross validation is described in more detail in the S3 Appendix.

thumbnail
Fig 2. Prediction accuracies(%) of Fugl-Meyer assessment (FMA) scores using Kinect for real FMA scores in each item.

https://doi.org/10.1371/journal.pone.0158640.g002

Assessing the degree of jerky motion using Kinect motion data

The evaluation of the movement impairment in this study is based on the integrated squared jerk.[19] The smoothness of motion is one element for subject assessment. The movement of the joint center was used for the Jerky motion analysis. An integrated jerky motion varies greatly with the duration and length (distance) of the movement.[20, 21] Because the movements of each patient have very different durations and lengths, comparison of each movement is difficult using only jerk data (third derivation of position data). In order to solve this issue, we generated integrated jerky motion data that were dimensionless.[19] Jerk is the third derivation of position data. The integration of the square of the jerk has a unit of duration5/length2. The value includes movement times and distance. Therefore, in order to remove the effects, we made the value dimensionless by multiplying the overall duration with the length. The dimensionless value therefore made the original data comparable. Because the integrated squared jerk varies with duration and size of the displacement, it was normalized by multiplying duration5/length2.[22] The squared root was taken to obtain a quantity proportional to the absolute jerk. The equation for normalized jerk is described in the following equation:

is a movement vector composed of the positions of the joint centers. Jerk(t) is an 18 dimensional vector because subject motion data has 18 variables (six joint x three dimension). Jerk2(t) is two-norm of the jerk vector. Duration is the length of the clipped data. Length is the maximum distance of a position vector (time t) from the initial (time T1), which is the greatest difference of motion distance from the start of motion. A higher jerky score derived from this method indicates more jerky movement (Fig 3). Jerky scores during the motion for flexion synergy in FMA were used for analysis.

thumbnail
Fig 3. Jerky motion analysis.

(A) and (B) are the results from y-direction hand movements from UE numbers 10 and 61 during the motion for the flexion synergy item in FMA. (C) is an example of the results of jerky motion analysis. A smooth curve movement like (A) has a lower jerky score value, whereas a high trembling curve like (B) has a higher jerky motion score.

https://doi.org/10.1371/journal.pone.0158640.g003

Statistical analyses

Continuous variables are presented as mean ± 1 SD for normally distributed data and as median with interquartile range for skewed data. Categorical variables are presented as frequencies (percentages). Prediction accuracies of FMA scores using Kinect for each item (number of subjects showing agreement in scores between FMA using Kinect and real FMA / 41 (total number of subjects)) were calculated and are represented as a percentage. It was regarded as accurate if FMA score using Kinect was exactly the same as real FMA for each item. The scores of the 13 items for FMA using Kinect and real FMA were summed to assess the correlation. Total score of FMA for selected items ranged from 0 to 26. Pearson’s correlation coefficients were calculated to see the correlation between FMA scores using Kinect and real FMA scores in the affected upper extremity for the selected items. In addition, correlation between FMA score using Kinect for selected items and total real FMA score (0–66 points) was investigated using Pearson’s correlation coefficient. Log (jerky scores) between hemiplegic and non-hemiplegic side were compared using paired t-test. Correlations between Brunnstrom arm stage[23] and log (jerky score) were investigated using Spearman’s correlation coefficient in subjects with Brunnstrom arm stage from 3 to 6. Statistical analysis was performed using SPSS version 18.0 (SPSS, Chicago, IL, USA). Data used for analysis are available from S1 File.

Results

Characteristics of the patients

Among 44 patients who agreed to participate, 41 completed the FMA. The other three patients were not removed from the study during the FMA but refused to do the test after enrollment. Demographic and clinical characteristics are summarized in Table 1.

Prediction accuracies for FMA scores using Kinect for each item

Prediction accuracies of FMA scores using Kinect for real FMA in the hemiplegic side were above 70% in nine of the 13 selected items (Fig 2). Four items (forearm supination, hand to lumbar spine, shoulder abduction 0° to 90° and shoulder flexion 90° to 180°) showed prediction accuracies between 60% and 70%.

Correlations between FMA scores using Kinect and real scores

Summed predicted FMA scores using Kinect for the 13 selected items showed high correlation with summed real FMA scores for 13 items in hemiplegic UEs (Pearson correlation coefficient = 0.873, P<0.0001) (Fig 4A). Correlation between summed predicted FMA scores using Kinect for the 13 selected items and summed FMA scores for the 33 items of the hemiplegic UEs were also high (Pearson correlation coefficient = 0.799, P<0.0001) (Fig 4B).

thumbnail
Fig 4. Correlation data.

(A) Correlation between summed Fugl-Meyer assessment (FMA) using Kinect and real FMA scores for 13 selected items in the hemiplegic upper extremity (Pearson correlation coefficient = 0.873, P<0.0001). (B) Correlation between summed FMA using Kinect for the 13 items and real FMA scores for total of 33 items in the hemiplegic upper extremity (Pearson correlation coefficient = 0.799, P<0.0001).

https://doi.org/10.1371/journal.pone.0158640.g004

Degree of jerky motion

Jerky scores during flexion synergy motion (instructed to fully supinate the forearm, flex the elbow, and bring the hand to the ear on the opposite side) of FMA calculated from Kinect motion data were log transformed for normalization. Log (jerky score) in the hemiplegic UE was 1.81 ± 0.76 in the hemiplegic arm, which was significantly higher than that in the non-hemiplegic UE (1.21 ± 0.43) (P<0.0001). Log (jerky score) in the hemiplegic UE showed significant negative correlations with Brunnstrom arm stage (Spearman correlation coefficient = -0.387, P = 0.046) in the patients with Brunnstrom arm stage from 3 to 6 (n = 27) (Fig 5).

thumbnail
Fig 5. Correlation between log (jerky score) and Brunnstrom arm stage (3 to 6) in the hemiplegic upper extremity (n = 27) (Spearman correlation coefficient = -0.387, P = 0.046).

https://doi.org/10.1371/journal.pone.0158640.g005

Discussion

In this study with hemiplegic stroke patients, predicted FMA scores using Kinect were highly correlated with real FMA scores. In addition, jerky scores calculated from Kinect motion data can assess the degree of motion smoothness quantitatively, which can not be provided by conventional FMA with observations.

A few tools in the virtual environment to assess the motor impairment after stroke have been investigated in several prior studies, but the correlations with conventional assessment tools were modest (correlation coefficient: 0.53–0.66).[8, 9, 25] One study evaluated the activities of daily living (ADL) in the virtual environment and compared it with the scores of Wolf Motor Function Test (WMFT).[25] Because WMFT is assessed during actual manipulation of objects, recognition of object's size, weight and texture and sensory feedback is important but the virtual ADL assessment tool using Kinect cannot give this haptic feedback.[25] This limitation may be associated with a modest correlation; an additional device and expense will be required to overcome this limitation. Although two studies using manipulating devices with virtual realities showed modest correlations with conventional assessment tool, additional costs and space are required.[8, 9] In this context, FMA is the best assessment tool which can be predicted using motion tracking with Kinect. Kinect is a relatively inexpensive depth-sensing camera and no additional space and devices are required. The purchase cost continues to decrease and camera performance continues to increase. FMA scoring using Kinect has potential as valid assessment tool for motor function after stroke in the home-based rehabilitation setting.

FMA is valid and is widely used for motor function assessment in stroke patients. But, many items place a time burden on the assessor and patients. In this context, there have been efforts to reduce the FMA item; one study suggested reducing the items for UE evaluation to six.[26] Because of the limitation of Kinect to track various UE motions, only 13 FMA items were presently included. The correlation was high between summed FMA scores using Kinect and real FMA scores for the 13 selected items. Although correlation between summed FMA scores using Kinect and real FMA scores for all 33 UE hemiplegic items was reduced, the correlation coefficient of 0.799 was still high. This indicates that the number of items used in FMA using Kinect could feasibly be decreased, which would decrease the burden on patients and caregivers during assessment.

Coordinated movement is impaired after stroke; motions are not smooth but rather become jerky. Traditional Brunnstrom recovery phase reveals the recovery of coordinated movement and emergence from synergistic movements.[23] Quantitative measurement of movement smoothness using a robotic device has revealed improvement during the recovery after stroke.[27] Jerky scores using Kinect in our study were well negatively correlated with Brunnstrom stage. This quantitative measure can be used for follow-up of changes in movement in a manner that equivalent in quality to robotic devices but less expensive.

The FMA scoring system using only one Kinect in this study does have some limitations. One is the occlusion of the body part during tracking by Kinect. For instance, Kinect can not track the hand when it is moved to the lumbar spine for FMA. One of the solutions for the occlusion problem is using multiple Kinect Sensors, but this may be associated with increased cost. Wearable sensors such as smart watches or wrist bands providing positional information can also be applied to solve the occlusion problem in our system. Another limitation is that the Kinect Software Development Kit is not appropriate for hand tracking of FMA motion, because it is unable to track pronation/supination, radial/ulnar direction and hand grasp below the wrist during full-body tracking. Some other FMA scores that could not be predicted in this research are required to detect the above hand information with accurate hand position tracking. The items in FMA showing low prediction accuracy are forearm supination, shoulder abduction 0° to 90° and shoulder flexion 90° to 180°, which require the information on forearm rotation. If the resolution for the hand during full-body tracking with Kinect is increased in the future, the prediction accuracy for items including information on forearm rotation are expected to improve and the FMA items excluded in this study can be added. While waiting for a more advanced form of Kinect, another solution is fusion with other hand tracking sensor, such as the Leap Motion device (Leap Motion, USA), which allows precise hand tracking using a hand point cloud below the wrist. We plan to use Leap Motion for precise hand tracking, and add more items of UE FMA.

More and varied movement data of each assessment item would increase the precision of the FMA system score using Kinect. Imbalance of real FMA scores decreases the prediction accuracies of each item. Techniques explored to solve the imbalance issues include re-sample techniques,[28] adaptive training algorithms,[29] particle swarm optimization[30] and parameter searching. We used re-sample techniques and cross-validation to minimize errors from dataset imbalance. However, the best way to solve this problem is to gather the more patient data. Although we used the data from 41 stroke patients with various motor impairments, decrease of the imbalance by collecting more data sets and adoption of the above techniques could help increase prediction accuracies. In further work, web-based uploading system of FMA Kinect motion data and real FMA data in various area could help acquire more patient data. Furthermore, use of a cloud computing system with machine learning ability, such as Microsoft Azure ML, Amazon Machine Learning or IBM Watson Analytics, will facilitate develop of a prediction model capable of self-learning whenever new patient data is uploaded, and to predict FMA score using the model in the absence of a specialist. In our study, the recording of FMA using Kinect was conducted in the hospital with the supervision of a therapist. Further study to validate the usefulness of advanced system in the real home-setting is required.

Conclusions

The FMA scoring system using Kinect is valid and provides additional quantitative measures of motion smoothness in stroke patients. This tool has the potential to be a useful and inexpensive tele-assessment tool of post-stroke motor function in the home-based setting. Acquisition of more patient data may increase the accuracy of this tool. Further study to validate this tool in the home-based setting is required.

Author Contributions

Conceived and designed the experiments: WSK HB NJP. Performed the experiments: WSK SC DB. Analyzed the data: WSK SC NJP. Contributed reagents/materials/analysis tools: WSK SC. Wrote the paper: WSK SC DB HB NJP. Supervision of technical issues during the study: HB. Review the manuscript and suggest some corrections: HB NJP.

References

  1. 1. Feigin VL, Forouzanfar MH, Krishnamurthi R, Mensah GA, Connor M, Bennett DA, et al. Global and regional burden of stroke during 1990–2010: findings from the Global Burden of Disease Study 2010. The Lancet. 2014;383(9913):245–55.
  2. 2. Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD, Brown TM, et al. Heart Disease and Stroke Statistics—2011 Update. Circulation. 2011;123:e18–e209.
  3. 3. Nichols-Larsen DS, Clark P, Zeringue A, Greenspan A, Blanton S. Factors influencing stroke survivors’ quality of life during subacute recovery. Stroke. 2005;36:1480–4. pmid:15947263
  4. 4. Kong K-H, Chua KS, Lee J. Recovery of upper limb dexterity in patients more than 1 year after stroke: frequency, clinical correlates and predictors. NeuroRehabilitation. 2011;28:105–11. pmid:21447911
  5. 5. Kong K-H, Lee J. Temporal recovery and predictors of upper limb dexterity in the first year of stroke: A prospective study of patients admitted to a rehabilitation centre. NeuroRehabilitation. 2013;32:345–50. pmid:23535798
  6. 6. Centers for Disease Control and Prevention. Outpatient rehabilitation among stroke survivors—21 States and the District of Columbia, 2005. MMWR Morb Mortal Wkly Rep 2007;56(20):504–7. pmid:17522589
  7. 7. Brainin M, Teuschl Y, Kalra L. Acute treatment and long-term management of stroke in developing countries. Lancet Neurol 2007;6:553–61. pmid:17509490
  8. 8. Kowalczewski J, Ravid E, Prochazka A. Fully-automated test of upper-extremity function. In: Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE 2011: 7332–5.
  9. 9. Lambercy O, Fluet M-C, Lamers I, Kerkhofs L, Feys P, Gassert R. Assessment of upper limb motor function in patients with multiple sclerosis using the Virtual Peg Insertion Test: A pilot study. In: Rehabilitation Robotics (ICORR), 2013 IEEE International Conference:1–6.
  10. 10. Gladstone DJ, Danells CJ, Black SE. The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabil Neural Repair 2002;16:232–40. pmid:12234086
  11. 11. Fugl-Meyer AR, Jaasko L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand J Rehabil Med 1975;7:13–31. pmid:1135616
  12. 12. Zhang Z. Microsoft kinect sensor and its effect. MultiMedia, IEEE 2012;19:4–10.
  13. 13. Olesh EV, Yakovenko S, Gritsenko V. Automated assessment of upper extremity movement impairment due to stroke. PLoS ONE 2014;9:e104487. pmid:25100036
  14. 14. Gabel M, Gilad-Bachrach R, Renshaw E, Schuster A. Full body gait analysis with Kinect. In: Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE 2012: 1964–7.
  15. 15. Kurillo G, Chen A, Bajcsy R, Han JJ. Evaluation of upper extremity reachable workspace using Kinect camera. Technology and Health Care 2012;21:641–56.
  16. 16. Guna J, Jakus G, Pogačnik M, Tomažič S, Sodnik J. An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 2014;14:3702–20. pmid:24566635
  17. 17. Yu H, Sun G-M, Song W-X, Li X. Human motion recognition based on neural network. Proceedings in 2005 International Conferene on Communications, Circuits and Systems 2005.
  18. 18. Barton G, Lisboa P, Lees A, Attfield S. Gait quality assessment using self-organising artificial neural networks. Gait Posture 2007;25:374–9. pmid:16784857
  19. 19. Teulings HL, Contreras-Vidal JL, Stelmach GE, Adler CH. Parkinsonism reduces coordination of fingers, wrist, and arm in fine motor control. Exp Neurol 1997;146:159–70. pmid:9225749
  20. 20. Kitazawa S, Goto T, Urushihara Y. Quantitative evaluation of reaching movements in cats with and without cerebellar lesions using normalized integral of jerk. In: Mano N, Hamada I, DeLong MR, editors. Role of the cerebellum and basal ganglia in voluntary movement. Amsterdam: Elsevier; 1993. pp.11–19.
  21. 21. Tresilian JR, Stelmach GE, Adler CH. Stability of reach-to-grasp movement patterns in Parkinson's disease. Brain. 1997;120:2093–2111. pmid:9397024
  22. 22. Caimmi M, Carda S, Giovanzana C, Maini ES, Sabatini AM, Smania N, et al. Using kinematic analysis to evaluate constraint-induced movement therapy in chronic stroke patients. Neurorehabil Neural Repair 2008;22:31–9. pmid:17595381
  23. 23. Brunnstrom S. Movement Therapy in Hemiplegia: A Neurophysiological Approach. 1st ed. New York: Harper & Row Publishers; 1970.
  24. 24. Brott T, Adams H, Olinger CP, Marler JR, Barsan WG, Biller J, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. 1989;20:864–70. pmid:2749846
  25. 25. Adams RJ, Lichter MD, Krepkovich ET, Ellington A, White M, Diamond PT. Assessing upper extremity motor function in practice of virtual activities of daily living. Neural Systems and Rehabilitation Engineering, IEEE Transactions on 2015;23:287–96. pmid:25265612
  26. 26. Hsieh Y-W, Hsueh I-P, Chou Y-T, Sheu C-F, Hsieh C-L, Kwakkel G. Development and validation of a short form of the Fugl-Meyer motor scale in patients with stroke. Stroke 2007;38:3052–4. pmid:17916763
  27. 27. Rohrer B, Fasoli S, Krebs HI, Hughes R, Volpe B, Frontera WR, et al. Movement smoothness changes during stroke recovery. The Journal of Neuroscience 2002;22:8297–304. pmid:12223584
  28. 28. Lu Y, Guo H, Feldkamp L. Robust neural learning from unbalanced data samples. In: Neural Networks Proceedings, 1998 IEEE World Congress on Computational Intelligence The 1998 IEEE International Joint Conference on; 1998: 1816–21.
  29. 29. Nguyen GH, Bouzerdoum A, Phung SL. A supervised learning approach for imbalanced data sets. In: Pattern Recognition, 2008 ICPR 2008 19th International Conference on; 2008: 1–4.
  30. 30. Adam A, Shapiai I, Ibrahim Z, Khalid M, Chew LC, Jau LW, et al. A Modified Artificial Neural Network Learning Algorithm for Imbalanced Data Set Problem. In: Computational Intelligence, Communication Systems and Networks (CICSyN), 2010 Second International Conference on; 2010: 44–8.