Combined real-time fMRI and real time fNIRS brain computer interface (BCI): Training of volitional wrist extension after stroke, a case series pilot study

Objective Pilot testing of real time functional magnetic resonance imaging (rt-fMRI) and real time functional near infrared spectroscopy (rt-fNIRS) as brain computer interface (BCI) neural feedback systems combined with motor learning for motor recovery in chronic severely impaired stroke survivors. Approach We enrolled a four-case series and administered three sequential rt-fMRI and ten rt-fNIRS neural feedback sessions interleaved with motor learning sessions. Measures were: Arm Motor Assessment Tool, functional domain (AMAT-F; 13 complex functional tasks), Fugl-Meyer arm coordination scale (FM); active wrist extension range of motion (ROM); volume of activation (fMRI); and fNIRS HbO concentration. Performance during neural feedback was assessed, in part, using percent successful brain modulations during rt-fNIRS. Main results Pre-/post-treatment mean clinically significant improvement in AMAT-F (.49 ± 0.22) and FM (10.0 ± 3.3); active wrist ROM improvement ranged from 20° to 50°. Baseline to follow-up change in brain signal was as follows: fMRI volume of activation was reduced in almost all ROIs for three subjects, and for one subject there was an increase or no change; fNIRS HbO was within normal range, except for one subject who increased beyond normal at post-treatment. During rt-fNIRS neural feedback training, there was successful brain signal modulation (42%–78%). Significance Severely impaired stroke survivors successfully engaged in spatially focused BCI systems, rt-fMRI and rt-fNIRS, to clinically significantly improve motor function. At the least, equivalency in motor recovery was demonstrated with prior long-duration motor learning studies (without neural feedback), indicating that no loss of motor improvement resulted from substituting neural feedback sessions for motor learning sessions. Given that the current neural feedback protocol did not prevent the motor improvements observed in other long duration studies, even in the presence of fewer sessions of motor learning in the current work, the results support further study of neural feedback and its potential for recovery of motor function in stroke survivors. In future work, expanding the sophistication of either or both rt-fMRI and rt-fNIRS could hold the potential for further reducing the number of hours of training needed and/or the degree of recovery. ClinicalTrials.gov ID: NCT02856035.

-S1 declined to fully complete the AMAT-F and AMAT-T measures at follow-up, citing disinterest at that point of 3Mo follow-up, having had no intervention for the prior 3 months.   Table 1c. shows clinically significant improvement in the AMAT-F and FM, in response to treatment. By post-treatment, S3 exhibited a gain of 18 points which is well above the MCID for the FM coordination scale. At followup, she had continued improvement in AMAT-F, which suggests consolidation of improved coordination into functional task performance (Daly 2019). At post-treatment, she had solid gains of wrist extension from both test positions, essentially maintained at follow-up. Subject 4. S4 had an improvement in functional task performance of .58 points which is above the MCID of .44 points for the AMAT-F (Table 1d). At post-treatment, S4 exhibited a gain of 13.5 points for the FM coordination scale, which is well above the MCID of 4.25 points. These gains were a reflection of improvement in shoulder, elbow, forearm movements; he made minimal gains in the items assessing wrist. That is, beginning from the neutral wrist start position, he had no volitional wrist extension at baseline (0 degrees), which continued to be the case throughout the study (Table  1d). Still, he did regain some volitional wrist extension movement, but only starting from the fully-flexed wrist position. For that test position, he improved from 0 degrees at baseline to 21 degrees of wrist extension (beginning from the fully flexed position) by post-treatment. By follow-up, he lost some wrist movement control, falling back to 15 degrees of volitional wrist extension, beginning from the fully flexed start position.  (Table 2A below, column C) provide context in which to show that mean ROI size for the stroke survivors is, for the most part, within the range of the healthy controls. Percent activation. Also, in column C in the Tables below, we are providing the range of activation volume within each ROI for the healthy adult group. Across the ROIs, minimum for at least two healthy adults was 0% percent, indicating that  for this very simple, single joint movement, some adult subjects accomplished it with such minimal neural drive that none was detected with our threshold values; other healthy adult subjects engaged neural drive ranging up to 58% across the ROIs, with one exception in the Hand Knob region for which 100% activation was engaged for at least one healthy adult subject. This range of activation values for this simple task is understandable in terms of the variation across persons in their manner of focusing attention, motivation to perform perfectly, and understanding of the simplicity/complexity of the task.

Stroke Survivor Subjects
Tables below contain individual subject fMRI data for the four stroke survivor subjects (Table 2a (Subject 1), 2b (Subject 2), 2c (Subject 3), 2d (Subject 4)), for fMRI Outcome Measures. Associated brain maps were provided in the body of the paper.     1.3. fNIRS OUTCOME MEASURE, INDIVIDUAL SUBJECT DATA fNIRS HbO concentration for pre-, post, f/u data acquisition sessions, individual subject data S1. The HbO remained fairly consistent across study participation (Figure 1a, S1). S2. In comparison to baseline, HbO was higher at post-treatment and returned to baseline by follow-up. (Figure  1b, S2). S3. HbO showed a decreasing trend from pre-treatment to follow-up (Figure 1c, S3). S.4. HbO remained consistent throughout study participation (Figure 1d, S4). Figures 1a, 1b, 1c, 1d. show data from pre-, post, f/u data acquisition sessions (no feedback). Oxyhemoglobin concentration response was measured from the ipsilesional hand-knob motor region channel. Values were derived from the deconvolved response.  Tables 3a, 3b, 3c, 3d below provide individual data for each of the four subjects, respectively, regarding percent volume of activation across the three rt-fMRI neural training sessions.     Figures 2a, 2b, 2c, 2d show the percent of trials for which each subject, respectively, was able to elevate the brain signal above threshold. Figures 2a, 2b, 2c, 2d show the percent of trials for which the subject was able to elevate the brain signal above threshold.   Figures 3a, 3b, 3c, 3d. rt-fNIRS signal, HbO concentration for each of the 10 training sessions.

III. Confluence of Results for Each Case.
The confluence of results for each case may by helpful in future work to improve neural feedback training for motor learning in stroke survivors using rt-fMRI and rt-fNIRS in conjunction with motor learning sessions. S1. Performance during neural feedback training sessions. Across the three real-time neural fMRI training sessions (rt-fMRI), S1 showed volume of activation during wrist extension practice at 100% activation of the lesioned hemisphere Hand Knob, throughout. She enhanced activation in the premotor and Primary Motor-Hand Knob regions, which may reflect an effort at more completely extending the wrist movement. At the same time, she reduced activation in all regions of the non-lesioned hemisphere, which could be indicative of focusing neural control in the lesioned hemisphere, as would normally occur for this simple motor task. During the subsequent rt-fNIRS 10 sessions, her control of the brain signal lagged from sessions 3 -6 (<40% success), but picked up again for sessions 7-10, ranging 45% to 60% success in reaching the signal threshold.
Outcome measures. The combination of rt-fMRI, rt-fNIRS, and the motor learning sessions resulted in a dovetailing of measures showing improvement. For example, at post-treatment, motor function gains were clinically significant for both functional task performance (AMAT-F, 1.0-point gain) and upper limb coordination (FM, 5-point gain). Also, she gained 10 degrees of wrist extension beginning from neutral position, and she gained 41 degrees of motion beginning from the position of fully flexed wrist.
Additionally, in the lesioned hemisphere, fMRI signal decreased from baseline to mid-treatment and was maintained at follow-up at the diminished values for Hand Knob and all regions, this lessening of activation may indicate increasing efficiency of brain activity. For the non-lesioned hemisphere, the abnormally elevated percent activations observed at baseline were reduced by mid-treatment and remained so for Hand Knob, Primary Motor-Hand Knob, and sensory regions. At post-treatment S1 reported that she was tired and "may have dozed at times" during the session, which relaxed state could perhaps contribute to an explanation of the zero activation levels for that session only. Finally, there was an increase in oxyhemoglobin concentration change, as measured by fNIRS signal, at follow-up, to a maximum value of 0.5 µM. fNIRS signal amplitude was within normal range (main paper, Figure 5, Panel C, S1).

S2.
Performance during neural training. S2 was severely impaired at baseline according to the FM score of 19, with no active wrist extension and a 'Trace' muscle contraction (palpable), a flaccid parasis (hypotonia) of forearm, wrist, and hand muscles. Additionally, she could not actively pronate the forearm into a functional position. Across the three real-time neural fMRI training sessions (rt-fMRI), she showed high attention to the wrist extension task at 100% activation of the lesioned hemisphere Hand Knob, throughout. She decreased activation by the third rt-fMRI session in the Primary Motor-Hand Knob and the sensory regions, suggesting a more efficient neural activity. However, in the non-lesioned hemisphere, she either maintained a high activation level or increased activation (in all but the sensory region), suggesting the effort to recruit motor control from that hemisphere. During the subsequent rt-fNIRS 10 sessions, success rate ranged from 42% to over 80%. During the 10 rt-fNIRS sessions, with the exception of session 3, there was no observable change in HbO with the exception that the highest trials per session (topmost blue dots per session, Figure 3b, above) showed an increasing upward trend from session 4 -10; values were within the normal range shown in Figure 5, main paper (compare Panel A, healthy adults).
Outcome measures. For S2, the combination intervention showed a confluence of improved measures across motor and brain signal variables. At post-treatment, there was a 4-point gain in FM, trending toward the 4.25 benchmark for clinical significance. By post-treatment, the functional task improvement (AMAT-F) had not reached clinical significance, but further improvement was realized by follow-up, so that there was a near clinically significant gain in functional task performance from baseline to follow-up. Additionally, she had recovery of active wrist extension as follows: 15 degrees of motion from neutral position; and 50 degrees of motion from fully flexed wrist position. Given her long-standing flaccid paralysis, these gains are notable; in other work, we reported a requirement of long-duration (6 months) treatment for this type of impairment (Daly 2000). Additionally, fMRI activation during wrist extension (and no feedback) showed a 25% to 50% decrease in lesioned hemisphere ROIs; and in the non-lesioned hemisphere there was a lessening of activation to within normal limits for the Hand Knob and the Primary Motor-Hand Knob regions. Finally, there was an increase in fNIRS HbO at post-treatment. This greater elevation at post-treatment could reflect greater effort at that time, given that S2 had observed some improved motor control during the weeks of the study by post-treatment.

S3.
Performance during neural training. S3 was severely impaired at baseline according to her FM score of 20. At baseline, she was able to extend the wrist only 4 degrees starting from the neutral position, and able to extend the wrist 60 degrees starting from the fully flexed wrist position (normal = 150 degrees). Across the three rt-fMRI neural training sessions, she lessened brain activation in all ROIs. In the lesioned hemisphere, she decreased the Hand Knob activation 100% to 71%, and decreased the remaining ROIs to below 9%. In the non-lesioned hemisphere, she decreased brain activation to 0%, within the normal range. These dramatic decreases suggest a pattern of progressively greater refinement, ultimately leaving the greatest activation in the lesioned contralateral Hand Knob, which is a normal pattern of neural control for this simple wrist movement. During the rt-fNIRS 10 training sessions, HbO was within the normal range (compare normal performance, main paper, Figure 5, Panel A versus panel C, S3); S1 began at < 20% successful brain signal control, but subsequent sessions 5, 7, 9, and 10 ranged from 55% to 70%. HbO from fNIRS during rt-fNIRS training was variable across the 10 sessions, with the highest of the four movement-series per session (topmost blue dot per session, Figure 3c above) trending higher in sessions 7, 8, and 9.
Outcome measures. For S3, there was a clinically significant gain in FM by post-treatment; this improvement in joint movement coordination of 18 points in the FM coordination scale produced a gain in the AMAT-F by post-treatment, of a clinically significant gain (.54 points improvement). It appears that for S3, this consolidation of recovering coordination continued to occur even during the 3 months prior to follow-up testing, at which time, the AMAT-F had continued to improve, showing a highly clinically significant gain of .96 points (>double the MCID) compared to baseline. Potentially driving these motor gains, S3 had her highest values of fMRI activation at mid-treatment in all ROIs for both hemispheres (mid-treatment, just after the last neural training session). Then, activation in all ROIs decreased by post-treatment (except for Primary Motor-Hand Knob in both hemispheres). By follow-up, activations in all ROIs ranged from 0 to 16%, except for Hand Knob ROIs (63%, lesioned hemisphere; 20% non-lesioned hemisphere).

S4.
Performance during neural training. With an initial upper limb FM score of 18, S4 was the most disabled at baseline. He was unable to extend the wrist, regardless of the start position. Across the three rt-fMRI training sessions, activation in the lesioned hemisphere Hand Knob did not reach significance (0%). There was a decrease in activation across the three sessions for Primary Motor-Hand Knob in both hemispheres and non-lesioned hemisphere sensory region. All other ROI activations were maintained across the three sessions. During rt-fNIRS training, S4 showed brain signal control ranging from 70% to 89% across the 10 sessions. For fNIRS HbO across the 10 sessions, values varied.
Outcome measures. Though severely impaired in distal limb movement control, by post-treatment, the FM gain was 13.5 points (> 3 times the MCID) and the AMAT-F gain score was 0.58, both of which are clinically significant. The continuing inability throughout the study to extend the wrist from the neutral position is consistent with the fact that gains in the FM and AMAT-F were largely due to improvements in the more proximal limb control. The fMRI activation data for S4 show that from pre-to post-treatment, there was a decrease in activation in all ROI's, with the exception of the lesioned hemisphere Hand Knob (0%) throughout and Primary Motor-Hand Knob (close to zero throughout). The zero fMRI values for S4 at follow-up may reflect the effect of motivation on brain signal activation. That is, throughout his study participation, he was energetic and highly motivated, but at follow-up, he expressed his great disappointment that the study treatment had ended; he stated that during the follow-up period with no treatment, he had not maintained any exercises, he knew he would not perform well at follow-up, and he was resigned to his current level of function. He expressed lack of motivation to expend effort for the fMRI follow-up testing, and his zero activations in all fMRI ROIs may reflect that. Finally, fNIRS HbO remained consistent from baseline through follow-up.
The recovery of volitional wrist extension, but only partially and only in the fully flexed position, is an early sign of recovery of motor function at the wrist. Other studies have reported that the recovery of chronic severe motor control can sometimes occur, but requires many months of treatment (Daly 2000); in keeping with those results, and because of the severity of S4's motor impairment, further recovery would have required